Trouble executing bulk load

Question

I'm getting an error when trying to upload a .csv to my sql server using asp.net uploadfile tool. The application is on the localhost and the SQL Server is NOT located on the same machine. It creates the table inside the database with the proper columns, just doesn't fill it up with the data. Here is the code error: "Cannot bulk load because the file "c:\users\visual studio 2010\Projects\webImport\webImport\UploadedCSVFiles\Employee.csv" could not be opened. Operating system error code 3(The system cannot find the path specified.).

Here is the code behind:

        private void LoadDataToDatabase(string tableName, string fileFullPath, string delimeter)
    {
        string sqlQuery = string.Empty;
        StringBuilder sb = new StringBuilder();
        sb.AppendFormat(string.Format("BULK INSERT {0} ", tableName));
        sb.AppendFormat(string.Format(" FROM '{0}'", fileFullPath));
        sb.AppendFormat(string.Format(" WITH ( FIELDTERMINATOR = '{0}' , ROWTERMINATOR = '\n' )", delimeter));

        sqlQuery = sb.ToString();
        using (SqlConnection sqlConn = new SqlConnection(GetConnectionString()))
        {
            sqlConn.Open();
            SqlCommand sqlCmd = new SqlCommand(sqlQuery, sqlConn);
            sqlCmd.ExecuteNonQuery();
            sqlConn.Close();
        }
    }

    protected void BTNImport_Click1(object sender, EventArgs e)
    {
        if (FileUpload1.HasFile)
        {
            FileInfo fileInfo = new FileInfo(FileUpload1.PostedFile.FileName);
            if (fileInfo.Name.Contains(".csv"))
            {

                string fileName = fileInfo.Name.Replace(".csv", "").ToString();
                string csvFilePath = Server.MapPath("UploadedCSVFiles") + "\\" + fileInfo.Name;

                //Save the CSV file in the Server inside 'MyCSVFolder' 
                FileUpload1.SaveAs(csvFilePath);
                //Fetch the location of CSV file 
                string filePath = Server.MapPath("UploadedCSVFiles") + "\\";
                string strSql = "SELECT * FROM [" + fileInfo.Name + "]";
                string strCSVConnString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + filePath + ";" + "Extended Properties='text;HDR=YES;'";
                // load the data from CSV to DataTable 
                OleDbDataAdapter adapter = new OleDbDataAdapter(strSql, strCSVConnString);
                DataTable dtCSV = new DataTable();
                DataTable dtSchema = new DataTable();
                adapter.FillSchema(dtCSV, SchemaType.Mapped);
                adapter.Fill(dtCSV);

                if (dtCSV.Rows.Count > 0)
                {
                    CreateDatabaseTable(dtCSV, fileName);
                    Label2.Text = string.Format("The table ({0}) has been successfully created to the database.", fileName);

                    string fileFullPath = filePath + fileInfo.Name;
                    LoadDataToDatabase(fileName, fileFullPath, ",");
                    Label1.Text = string.Format("({0}) records has been loaded to the table {1}.", dtCSV.Rows.Count, fileName);
                }
                else
                {
                    Label3.Text = "File is empty.";
                }
            }
            else
            {
                Label3.Text = "Unable to recognize file.";
            }

        }
    }

I'm guessing it has something to do with the Server.MapPath. Is it possible to route the file to be saved and executed on a different server then the application. if that makes any sense.

Answer 1

If you use the TSQL form of bulk load, that code executes on the SQL server, so the file path you specify is relative to the DB server.

In your case, I think you should look at SqlBulkCopy and the "Fast CSV Reader" from code project; they work very well together. Then you are processing the CSV on your web-server, and sending the bulk stream to the SQL server via a very efficient API.

Answer 2

BULK INSERT paths are used from the frame of reference of the server. If I am using the Management Studio or another application on my machine to send query code to a different machine that is running the actual SQL server, the server will get my query and attempt to resolve the path from the server's view of the file system.

What this means is that my computer might have a file on the path C:\folder\foo.txt. If I connect to the computer running the SQL server and try a BULK INSERT DBO.FOO from 'C:\folder\foo.txt', the server will look in it's own C:\ partition to try and locate foo.txt.

I've managed to work around this by either:

• Putting the file to bulk insert on the actual machine running the SQL server
• Using a network share both machines have access to. This can be expensive bandwidth-wise, though.