Design guideline for saving big byte stream in c#

I have an application where I am receiving big byte array very fast around per 50 miliseconds.

The byte array contains some information like file name etc. The data (byte array ) may come from several sources.

Each time I receive the data, I have to find the file name and save the data to that file name.

I need some guide lines to how should I design it so that it works efficient.

Following is my code...

public class DataSaver
{
    private Dictionary _dictFileStream;

    public void SaveData(byte[] byteArray)
    {
        string fileName = GetFileNameFromArray(byteArray);
        FileStream fs = GetFileStream(fileName);
        fs.Write(byteArray, 0, byteArray.Length);
    }

    private FileStream GetFileStream(string fileName)
    {
        FileStream fs;
        bool hasStream = _dictFileStream.TryGetValue(fileName, out fs);
        if (!hasStream)
        {
            fs = new FileStream(fileName, FileMode.Append);
            _dictFileStream.Add(fileName, fs);
        }
        return fs;
    }

    public void CloseSaver()
    {
        foreach (var key in _dictFileStream.Keys)
        {
            _dictFileStream[key].Close();
        }
    }
}

How can I improve this code ? I need to create a thread maybe to do the saving.

You need to ask yourself 4 questions to start with:

• How "big" is "big byte array"?

• How fast is "very fast" ? Can we take 20 Hz as a design assumption?

• Should data be appended in-order?

• What is the expected overall size for a given file?

Just for the taste of it, appending 4MB to a file, on my 10 YO machine, takes less than 10 ms. A modern SSD and CPU will outdo this by an order of magnitude.

Anyway, knowing nothing, I would start with the most simple imaginable scheme, which is completely synchronous, and let it fail:

public static class FileSaver
{
    public static bool TrySaveData(byte[] data)
    {
        int dataOffset = 0;
        var file = GetFileName(data, out dataOffset);
        return TryAppend(file, data, dataOffset);
    }

    private static string GetFileName(byte[] data, out int offset)
    {
        string res = ...   // get the file name
        offset = ...       // depends on your format                    
        return res;
    }

    private static bool TryAppend(string file, byte[] data, int offset = 0)
    {
        try
        {
            using (var stream = new FileStream(file, FileMode.Append))
            {
                stream.Write(data, offset, data.Length - offset);
            }
            return true;
        }
        catch (Exception ex) { /* log some error */ }
        return false;
    }
}

Why is this good? Because by the time it fails, you'll be smart enough to answer 1-4, and then we would design something else based on real understanding of the problem.

StackExchange Code Review Q#18266, answer score: 6