How to export large amount of data fast via Web API

c# csv entity-framework-6 export

Question

I'm using ASP.NET Web API 2 (.NET 4.6.1) and AngularJS on the client side. My requirement is to export the tables data in either CSV or Excel format. I cannot use bcp or sqlcmd because of security reasons. Also, cannot use DTS package as the parameters will be coming from web UI and I need to pass it to stored procedure to retrieve the result and send back the file via api.

Currently, I'm trying to read all 500K records using entity framework 6 to IEnumerable type. My table has around 500K records with 40+ columns. I was trying to convert it to CSV and then return as plain text and open in browser as blob.

Questions:

  1. Is there any better way to export and return the data within few seconds.
  2. If IEnumberable to CSV is the only way then should i first write it file and return file or return plain text as response from web api.
  3. I have tried few code from the web to convert IEnumerable to CSV (using String.Join and StringBuilder and are not effecient. Appreciate, if anyone can share how large data can be converted to CSV very fast or other way to export data.
1
1
4/16/2016 2:17:57 PM

Popular Answer

I dont' think it will work that fast, as we are looking at (roughly) 200MB dataset that you first pull from DB and then send it to client. You could try using DotNetZip library to zip the CSV before sneding to client...

0
4/16/2016 2:00:31 PM


Related Questions





Related

Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow
Licensed under: CC-BY-SA with attribution
Not affiliated with Stack Overflow