Entity Framework Bulk Load Too Slow Add Seed

Question

    protected override void Seed(Fitlife.Domain.Concrete.EFDBContext context)
    {
        List<List<string>> foodweights = GetLines(basePath + "FoodWeights.txt");
        int counter = 0;
        foodweights.ForEach(line =>
        {
            FoodWeights newVal = new FoodWeights()
            {
                 FoodCode = int.Parse(line[0]),
                  PortionCode = int.Parse(line[1]),
                 PortionWeight = decimal.Parse(line[2])
            };
            context.FoodWeights.Add(newVal);

            if (++counter == 1000)
            {
                counter = 0;
                context.SaveChanges();                  
            }
        });
}

Above method is used to populate my database. But it takes 50 seconds for 1000 entries i have a file with 470k entries, how can i improve performance i am using entity framework and this method is called when i do PM> update-database with Package manager. i need similar functionality, i am very new to asp.net and entity framework any guidance will be appreciated thanks.

PS: Is it ok to take 50 seconds for 1000 entries or am i doing something wrong.

Answer 1

The Seed method runs every time the application starts, so the way you have coded it will attempt to add the FoodWeights over and over again. EF have provided the AddOrUpdate as a convenient method to prevent that but it is really not appropriate for bulk inserts.

You could use sql directly on the database - and if you are using sql server that sql could be 'BULK INSERT'.

I would put the sql in an Up migration because you probably only want to run the insert once from a known state, and it avoids having to worry about the efficiency of the context and tracking changes etc.

There is example code and more information here: how to seed data using sql files