Copy all Rows to another Table in Azure Table Storage

Question

What is the best way to copy all the rows from one table to another table?
I tried below code to get all the rows in a table:

    TableServiceContext _dataContext;
    public IEnumerable<T> GetAllEntities()
    {
        IQueryable<T> query = null;
        try
        {
            query = _dataContext.CreateQuery<T>(_tableName);
        }
        catch (Exception ex)
        {

        }
        return query.ToArray();
    }

but it doesnt get rows more than around 900. I have few hundreds of thousands of rows.
Updated Code:

 public class TableRepository<T> : IRepository<T> 
    where T : TableEntity
{
    protected readonly string _tableName;
    protected readonly TableServiceContext _dataContext;
    protected readonly CloudTable _tableReference;

    public TableRepository(string tableName, CloudTableClient tableClient)
    {
        _tableName = tableName;
        _dataContext = tableClient.GetTableServiceContext();
        _tableReference = tableClient.GetTableReference(tableName);
        _dataContext.ResolveType = ResolveEntityType;
        _dataContext.MergeOption = System.Data.Services.Client.MergeOption.NoTracking;
    }

    public IEnumerable<T> GetAllEntities()
    {
        List<T> allEntities = new List<T>();
        try
        {
            Microsoft.WindowsAzure.Storage.Table.TableContinuationToken tableContinuationToken = null;
            do
            {
                var queryResponse = _tableReference.ExecuteQuerySegmented<T>(null, tableContinuationToken, null, null);
                tableContinuationToken = queryResponse.ContinuationToken;
                allEntities.AddRange(queryResponse.Results);
            }
            while (tableContinuationToken != null);

        }
        catch (Exception ex)
        {
            throw new DALException(_tableName,_dataContext.BaseUri.OriginalString, "An error occured while querying data", ex);
        }
        return allEntities;
    }

}

but with error:

Error 121 'T' must be a non-abstract type with a public parameterless constructor in order to use it as parameter 'TElement' in the generic type or method 'Microsoft.WindowsAzure.Storage.Table.CloudTable.ExecuteQuerySegmented

Answer 1

The reason you're getting only 900 results back is because you're running into continuation tokens. By default a single request to table service will return a maximum of 1000 entities. It could be less than 1000 entities (and even 0) but never more than 1000. If there are more entities available, then the table service returns a continuation token which should be used to fetch next set of entities.

So your code should look for the continuation token and should keep on fetching the entities till the time token is returned by table service. Do take a look at the sample code below:

private IEnumerable<T> FetchAllEntities()
{
    List<T> allEntities = new List<T>();
    CloudStorageAccount storageAccount = CloudStorageAccount.DevelopmentStorageAccount;
    CloudTable table = storageAccount.CreateCloudTableClient().GetTableReference("MyTable");
    Microsoft.WindowsAzure.Storage.Table.TableContinuationToken tableContinuationToken = null;
    do
    {
        var queryResponse = table.ExecuteQuerySegmented<T>(null, tableContinuationToken, null, null);
        tableContinuationToken = queryResponse.ContinuationToken;
        allEntities.AddRange(queryResponse.Results);
    }
    while (tableContinuationToken != null);
    return allEntities;
}

Update

For your error, try changing the following

public class TableRepository<T> : IRepository<T> 
    where T : TableEntity

to

public class TableRepository<T> : IRepository<T> 
    where T : TableEntity, new()

notice the addition of new() after TableEntity.