Entity Framework bulk inserts - creating a large number of objects / entities is the slow part?

c# entity-framework entity-framework-6

Accepted Answer

The Detect Changes area of your "building all the items" section is the time-consuming component.

Always choose AddRange over Add.

  • Addition: Track modifications for each object.
  • AddRange: Only ever detect changes (after all object are added)

This code should speed up the object creation process:

DataTable ref1 = ConvertCSVtoDataTable(csv, firstRowsToDelete: 15); // Return's a Datatable from a CSV

List<Entity> list = new List<Entity>();

foreach(string file in ListOfFilesToProcess)
{
    DataTable tbl = loadExcelFiles(file);

    foreach(DataRow dr in tbl.Rows)
    {
         Entity newEntity = new Entity();
         Entity.property1 = dr["Property1"].ToString();
         ... // Keep mapping properties to elements in the datatable
         Entity.Child.Add(new ChildEntity() { prop1 = ref1["ChildProp1"].ToString() });

        list.Add(newEntity);
    }
}

// Add all newly created entities to the context
db.Entity.AddRange(list);

// Save the context
db.SaveChanges();

Try using a third-party BulkSaveChanges / Bulk Insert library if, after this patch, you still experience performance issues (this time coming from the database).

An article on these libraries can be found here: Reviews and comparisons of the bulk insert library for entity framework.

  • Extended Entity Framework (Recommanded, support everything)
  • EntityFramework.BulkInsert
  • EntityFramework.Utilities

I am the project's owner, which is Disclaimer.

1
2/22/2018 1:02:25 PM

Popular Answer

ZZZ_tmp


Related Questions





Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow