Best approach for data transfer from remote database to local database

Question

I am in a scenario where I want to work with many databases. Some are in my project and some are external. My application would transfer data from the external database(one which reside in a remote machine. I know the ip and user credentials) to my temp database. I want to create the tables in that database to my database. What is the best method to do this? I would work with ASP.NET 3.5 . Any recommendations like WCF or Web service?

Answer 1

You have to answer yourself some questions:

1. Is the schema of source and target database stable?
2. Can you afford downtime during synchronization?
3. Do you transfer data and structure or only data?
4. How often do you need synchronization? (once a day or always recent data)

The answer depends on this questions but one simple solution ist to use SMO and the "Transfer" task.

      Server srv = default(Server); 
   srv = new Server(); 
   //Reference the AdventureWorks database 
   Database db = default(Database); 
   db = srv.Databases("AdventureWorks"); 
   //Create a new database that is to be destination database. 
   Database dbCopy = default(Database); 
   dbCopy = new Database(srv, "AdventureWorksCopy"); 
   dbCopy.Create(); 
   //Define a Transfer object and set the required options and properties. 
   Transfer xfr = default(Transfer); 
   xfr = new Transfer(db); 
   xfr.CopyAllTables = true; 
   xfr.Options.WithDependencies = true; 
   xfr.Options.ContinueScriptingOnError = true; 
   xfr.DestinationDatabase = "AdventureWorksCopy"; 
   xfr.DestinationServer = srv.Name; 
   xfr.DestinationLoginSecure = true; 
   xfr.CopySchema = true; 
   //Script the transfer. Alternatively perform immediate data transfer 
   // with TransferData method. 
   xfr.ScriptTransfer(); 

Answer 2

I'll elaborate on what Richard & Chris said -

Replication is a set of technologies for copying and distributing data and database objects from one database to another and then synchronizing between databases to maintain consistency. Using replication, you can distribute data to different locations and to remote or mobile users over local and wide area networks, dial-up connections, wireless connections, and the Internet.

Transactional replication is typically used in server-to-server scenarios that require high throughput, including: improving scalability and availability; data warehousing and reporting; integrating data from multiple sites; integrating heterogeneous data; and offloading batch processing. Merge replication is primarily designed for mobile applications or distributed server applications that have possible data conflicts. Common scenarios include: exchanging data with mobile users; consumer point of sale (POS) applications; and integration of data from multiple sites. Snapshot replication is used to provide the initial data set for transactional and merge replication; it can also be used when complete refreshes of data are appropriate. With these three types of replication, SQL Server provides a powerful and flexible system for synchronizing data across your enterprise.

In addition to replication, in SQL Server 2008, you can sychronize databases by using Microsoft Sync Framework and Sync Services for ADO.NET. Sync Services for ADO.NET provides an intuitive and flexible API that you can use to build applications that target offline and collaboration scenarios.

Answer 3

I haven't actually used it, but I think the Microsoft Sync Framework was created with this type of scenario in mind.

Answer 4

Sounds like you need to read about database replication.