Follow up: Execute .sql files from python

Question

Over a year ago someone asked this question: Execute .sql files that are used to run in SQL Management Studio in python.

I am writing a script in python that connects to a SQL server and creates and populates a database based on SQL commands in a large (several GBs) .sql file.

It looks like SQLCMD requires a download and install of SQL Server Express. Are there other ways to execute a .sql file from python without requiring everyone who uses my script to download and install SQL Server? Does pyodbc have this capability?

EDIT:

Here's another similar question: execute *.sql file with python MySQLdb

Here, again, the solution is to call a utility from command (in this case, mysql.exe) with the file listed as an argument.

It seems to me that there should be a way to do this using one of Python's DB API libraries, but I haven't found it so I'm looking for an *.exe like SQLCMD or MYSQL that I can use to run the file from command line.

P.S. Please feel free to correct me if I'm not looking at this correctly. Maybe the code below is just as efficient as running from command line:

for line in open('query.sql','r'):
    cursor.execute(line)

Answer 1

I found it's actually faster to read the file in python and execute in batches using pyodbc than it is to use the SQLCMD utility externally (and I don't have to install SQLCMD on every computer I run the scripts on!).

Here is the code I used (because pyodbc doesn't seem to have an executescript() method):

with open(scriptPath, 'r') as inp:
    for line in inp:
        if line == 'GO\n':
            c.execute(sqlQuery)
            sqlQuery = ''
        elif 'PRINT' in line:
            disp = line.split("'")[1]
            print(disp, '\r')
        else:
            sqlQuery = sqlQuery + line
inp.close()

Answer 2

Not sure if this is what you are asking but why not use MS SQL directly from Python? There are libraries like pymssql that will allow you to do that. Or you can use ODBC.

See http://wiki.python.org/moin/SQL%20Server for a list of Python MS SQL drivers

Answer 3

SQLCMD and other management utilities are freely available for download. They are part of what Microsoft calls "Feature pack for SQL Server".

I understand you want to perform large bulk imports. To do this you may want to check out the the BCP utility. Download http://www.microsoft.com/en-us/download/details.aspx?id=16978

Using SQL you may also perform a BULK INSERT. This command can insert data from a file to a SQL db.

Another way is of course to use SQL Server Integration Services for pure ETL jobs.