HiveBrain v1.2.0
Get Started
← Back to all entries
patterncsharpMinor

Split a list into equal parts that using LINQ in a way that has good performance

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
linqequalintowayhassplitthatusingperformancegood

Problem

So I have a method that looks like this:

public static List>> Split(List> source, int chunksize)
   {
       return source
           .Select((x, i) => new { Index = i, Value = x })
           .GroupBy(x => x.Index / chunksize)
           .Select(x => x.Select(v => v.Value).ToList())
           .ToList();
   }


While this is nice and elegant and readable, unfortunately that groupby is killing performance because of the implicit sort. I need a way to rewrite this to be more performant, while still being reasonably elegant.

Solution

We can make the method more generic:

public static IEnumerable> Split(IEnumerable source, int chunkSize)
{


Then we can store the current chunk in a List. Using the constructor that takes the initial capacity should be good for performance:

var chunk = new List(chunkSize);
    foreach (var item in source)
    {
        chunk.Add(item);
        if (chunk.Count == chunkSize)
        {
            yield return chunk;
            chunk.Clear();
        }
    }


Finally we need to return the final (non-empty) chunk

if (chunk.Count > 0)
    {
        yield return chunk;
    }
}


Depending on your use case, you might want to make a copy of the list and yield return chunk.ToList() instead.

Code Snippets

public static IEnumerable<IEnumerable<T>> Split<T>(IEnumerable<T> source, int chunkSize)
{
var chunk = new List<T>(chunkSize);
    foreach (var item in source)
    {
        chunk.Add(item);
        if (chunk.Count == chunkSize)
        {
            yield return chunk;
            chunk.Clear();
        }
    }
if (chunk.Count > 0)
    {
        yield return chunk;
    }
}

Context

StackExchange Code Review Q#72201, answer score: 6

Revisions (0)

No revisions yet.