C# Entity Framework 6: Out Of Memory when it absolutely should not run out of memory

c# entity-framework-6 out-of-memory

Question

I've simplified things as much as possible. This is reading from a table that has around 3,000,000 rows. I want to create a Dictionary from some concatenated fields of the data.

Here's the code that, in my opinion, should never, ever throw an Out Of Memory Exception:

public int StupidFunction()
{
    var context = GetContext();
    int skip = 0;
    int take = 100000;
    var batch = context.VarsHG19.OrderBy(v => v.Id).Skip(skip).Take(take);
    while (batch.Any())
    {
        batch.ToList();
        skip += take;
        batch = context.VarsHG19.OrderBy(v => v.Id).Skip(skip).Take(take);
    }

    return 1;
}

In my opinion, the batch object should simply be replaced each iteration and the previous memory allocated for the previous batch object should be garbage collected. I would expect that the loop in this function should take a nearly static amount of memory. At the very worst, it should be bounded by the memory needs of one row * 100,000. The Max size of a row from this table is 540 bytes. I removed Navigation Properties from the edmx.

1
1
9/9/2016 3:53:05 PM

Popular Answer

You can turn off tracking using AsNoTracking. Why not use a foreach loop though on a filtered IEnumerable from the DbSet? You can also help by only returning what you need using an anonymous type using Select() – Igor

Thanks for the Answer Igor.

public int StupidFunction()
{
    var context = GetContext();
    int skip = 0;
    int take = 100000;
    var batch = context.VarsHG19.AsNoTracking().OrderBy(v => v.Id).Skip(skip).Take(take);
    while (batch.Any())
    {
        batch.ToList();
        skip += take;
        batch = context.VarsHG19.AsNoTracking().OrderBy(v => v.Id).Skip(skip).Take(take);
    }

    return 1;
}

No Out of Memory Exception.

2
9/9/2016 4:32:19 PM


Related Questions





Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow