What happens to data when running a query against a federated SQL Server?

Question

SQL Azure give you an option to setup federated SQL servers - e.g. essentially partition your data across multiple servers by some key. However, when you run select * from order, it puts all the data back together for you and the entire process is transparent to your app.

Now, let's say I have a stored procedure that makes liberal use of temp tables. So in a situation like this:

-- query uses data from each partitioned server
select tbl1.Column1, tbl2.Column2
into #tmpTable
from tbl1
join tbl2.id = tbl1.id
where tbl2.Column3 = '234'

-- also uses data from each partitioned server
select #tmpTable.*
from tbl3
join tbl3.fkey = #tmpTable.Column1

In this scenario, does the data flow each time from one server to the main one? Where is the temp table actually stored? All on the main box, or split up between federated ones?

Answer 1

Avkash is correct. A federation is split into federation members, which is a collection of databases that share nothing. When you run a statement like SELECT * FROM Order, assuming the Order table has been federated, the records being returned vary depending on which federation member you use.

The USE FEDERATION command allows you to switch federation member without disconnecting/reconnecting. Your code needs to know the list of federation members and loop through each member to fetch all the records; one federation member at a time. Then on the client code, you aggregate all the recordsets to create a single data set that your application can use. You can craft your code to perform all these requests in parallel.

You can see how to implement this using a library I posted on codeplex. It's called the Enzo Shard Library. You need to download the BETA version, which contains the code for Federations. The code will allow you to fetch all records across federation members using multiple threads for optimum performance.

Answer 2

I am not thorough in SQL Azure Federation yet, however I was able to get the following information for you:

If you are referring to federation in sql azure, federation do not fanout your query to all members today. You basically have a collection of databases and scope of transactions and queries stay within a single member on the server side.

With that, there is no mystery for temp tables. each member is just another sql azure db which comes with its own tempdb.