SQL Server 2012 consumes too much network traffic to fetch all data

Question

Today I did a test using AnyDAC (firedac) to fetch all data available in my remote SQL Server 2012.

My table that I fetch data from has these simple columns:

1. date - (size 3 byte)
2. time - (max 5 byte)
3. int - (4 byte)
4. bit - (1 byte)
5. int - (4 byte)
6. float - (4 byte)
7. float - (4 byte)
8. int - (4 byte)
9. int - (4 byte)

Total row size should be max 33 bytes.

Well, after fetching all rows available in my table (more than 2.14 million rows), I checked the received tcp traffic by FireDAC and observed that it is around 280MB which means each row needs around 130 byte while my expected value something close to 33 bytes.

Another measurement that I did by using a stored procedure defined on server side which have an insert sql to the same table above and I called the stored procedure using Array DML feature of AnyDAC. Array size was 300K and totally I have added 18.8 million records using it. Traffic used for it realized as 2.85 GB. (so 150 bytes per row)

Is there any configuration on FireDAC or SQL Server side to reduce traffic? Obviously, there is something wrong here. Any suggestions?

Thanks.

Info output:

================================
Connection definition parameters
================================
User_Name=*****
Password=*******
SERVER=sql.***.gen
ApplicationName=Bist
Workstation=NB
DATABASE=BIST
MARS=yes
DriverID=MSSQL
================================
FireDAC info
================================
Tool = D18 Architect
FireDAC = 8.0.1 (Build 3279)
Platform = Windows 32 bit
Defines = AnyDAC_Unicode;AnyDAC_DBX;AnyDAC_NOLOCALE_META;
  AnyDAC_MONITOR
================================
Client info
================================
Loading driver MSSQL ...
  Loading odbc32.dll driver manager
  Creating ODBC environment handle
  Searching for ODBC driver ...
    Checking for ODBC driver [SQL SERVER NATIVE CLIENT 11.0] ...
      Found [SQL Server Native Client 11.0]
Driver Manager version = 03.80.7601.0000
================================
Session info
================================
Current catalog = 
Current schema = dbo
Driver name = sqlncli11.dll
Driver version = 11.00.2100
Driver conformance = 3
DBMS name = Microsoft SQL Server
DBMS version = 11.00.2100

Answer 1

Your network traffic to the SQL database server includes MUCH more than just the data you're looking to transmit (as you have observed by seeing much more traffic received than just row data)

Before I start, let me disclaim a few things: I am in no way an expert on network traffic and protocols, but have spent a fair amount of time playing around with them to understand enough. I also do not know much about what protocol the particular DAC software, query combination, etc. would employ to bring you your requested data back. That being said, I cannot give an exact example. but the concepts should still apply.

Communication with SQL server occurs through various protocols, however we will focus on just 1 for the sake of argument: TDS (Tabular Data Stream) protocol. (You can read more about SQL Server Protocols at: You can read more on SQL Server Protocols at: http://msdn.microsoft.com/en-us/library/ff420673(v=sql.105).aspx). TDS uses the TCP protocol, thus a TDS packet is nothing more than a specification wrapped in a TCP packet).

With TCP, communications are basically with a Request / Response type of exchange involving many request / response / acknowledge packets. Again, I am by no means an expert on this, but you can read more at http://en.wikipedia.org/wiki/Transmission_Control_Protocol

The "acknowledge" (or confirmation) aspect of the communication alone accounts for a fair amount of overhead beyond just the row sizes you referenced. You can see an example of this in the image below where communication is occurring between a client application and a SQL server:

Additionally, within the actual packet being sent pack to the application from the DB server, there is a lot of overhead within the TCP packet protocol itself along with overhead within the TDS packet.

Lets focus just on the TDS packet's overhead that adds to your overall data transfer payload. Below are all documented (http://msdn.microsoft.com/en-us/library/dd340794.aspx) of what can be found in a TDS packet:

• ALTMETADATA
• ALTROW
• COLINFO
• COLMETADATA
• DONE
• DONEINPROC
• DONEPROC
• ENVCHANGE
• ERROR
• FEATUREEXTACK
• INFO
• LOGINACK
• NBCROW
• OFFSET
• ORDER
• RETURNSTATUS
• RETURNVALUE
• ROW <- This is where your data is
• SESSIONSTATE
• SSPI
• TABNAME
• TVP ROW

And all the while, really your ACTUAL data (in the case you described above) is located within the ROW portion of the packet.

Below is a screenshot showing an ASCII version of 1 TDS packet. The highlighted portion of the packet frame (on the right) is the actual row data. All the rest is just overhead that makes the entire communication and ancillary things just work.

And this does not even include the overhead (albeit necessary) created by TCP itself.

In summary

You're transferring an extremely large amount of data and that data will always have overhead.

• If the data is going to be summarized in the end, then summarize it on the DB server before shipping it.
• If the data is going to be searched, offload that search to the DB server and just return what you need
• If you need to list the data out (each row), then you're probably showing only a sub-set of results per page - only return those results you need through something like a SELECT TOP 50 .... WHERE ....

There are many other solutions depending upon your actual use of the data, but these are just some ideas.

Hope this helps!