OleDbDataAdapter.Fill(DataTable) not working with big xls file size

Question

I'm trying to get a DataTable, reading a .xls file. If I run the below code with a .xls file with size 25kb, it works fine, but if I load a bigger size file (7,52MB), it doesn't work.

string filenamePath = System.IO.Path.Combine(Server.MapPath("/Uploads"), FileUpload1.FileName);
FileUpload1.SaveAs(filenamePath);

string[] validFileTypes = { "xls", "xlsx"};                
string ext = System.IO.Path.GetExtension(FileUpload1.PostedFile.FileName);
bool isValidFile = false;                
string fileName = FileUpload1.FileName;
for (int i = 0; i < validFileTypes.Length; i++)
{
    if (ext == "." + validFileTypes[i])
    {
        isValidFile = true;
        break;                    
    }
}
if (isValidFile)
    {
        DataTable dt = ConvertXLSTpXLM.convertXLSToDb(filenamePath, "ELEMENT", "ELEMENTS", true, fileName);

    }

This is the convertXLSToDb method

public static DataTable convertXLSToDb(string filePath, string firstElement, string secondElement, bool hasHeaders, string filename)
{
    DataTable dtexcel = new DataTable();
    string HDR = hasHeaders ? "Yes" : "No";
    string strConn;
    if (filePath.Substring(filePath.LastIndexOf('.')).ToLower() == ".xls")
        strConn = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + filePath + ";Extended Properties=\"Excel 12.0;HDR=" + HDR + ";IMEX=0\"";
    else
        strConn = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + filePath + ";Extended Properties=\"Excel 8.0;HDR=" + HDR + ";IMEX=0\"";
        OleDbConnection conn = new OleDbConnection(strConn);
        conn.Open();
        DataTable schemaTable = conn.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, new object[] { null, null, null, "TABLE" });
        DataRow schemaRow = schemaTable.Rows[0];
        string sheet = schemaRow["TABLE_NAME"].ToString();
        if (!sheet.EndsWith("_"))
        {
            string query = "SELECT  * FROM [" + sheet + "]";
            OleDbDataAdapter daexcel = new OleDbDataAdapter(query, conn);
            dtexcel.Locale = CultureInfo.CurrentCulture;
            daexcel.Fill(dtexcel);
        }

        conn.Close();
        return dtexcel;
}

No error occurs, but on the "daexcel.Fill(dtexcel);" the page runs infinitely.

Answer 1

if I run the below code with a .xls file with size 25kb, it works fine, but if I load a bigger size file (7,52MB), it doesn't work.

Putting the fact that you don't Dispose a lot of IDisposable resources to one side, it's possible that there's nothing wrong with the size of the file, rather the problem is the contents of the file. That is to say, you might have a row with "curious" data and if you were to get that curious data into the 25kb file it would not work, either.

Microsoft explicitly state that using ACE in a service environment is a bad idea:

The Access Database Engine 2010 Redistributable is not intended:

...

To be used by a system service or server-side program where the code will run under a system account, or will deal with multiple users identities concurrently, or is highly reentrant and expects stateless behavior. Examples would include a program that is run from task scheduler when no user is logged in, or a program called from server-side web application such as ASP.NET, or a distributed component running under COM+ services.

Similar caveats apply to JET but they're not documented quite as explicitly.

To diagnose where the problem is, get the code out of ASP .NET and put it into an interactive console application and run it.

If the code still exhibits problems you could consider pausing the process with the debugger and trying to inspect where the code has "frozen". Being paused on GetOleDbSchemaTable instead of Fill might be illuminating.

Beyond that, given that neither JET nor ACE are recommended in a server environment you might consider giving up .xsl support and using something like EPPlus to read your .xlsx files.

Answer 2

The File Upload control can by default upload files upto 4MB only. Refer the Note on the page below.

http://msdn.microsoft.com/en-us/library/ms227669(v=vs.90).aspx

You can change the httpRuntime maxRequestLength in the web config as mentioned below.

How to programmatically set (using GET SET property) "httpRuntime maxRequestLength" in ASP.NET with C# as code behind

Hope this helps!