Create new StringBuilder by looping through list, remove last element

Question

I know this might be a bad example, but my question in this case has more to do with the best / most efficient way to deal with a StringBuilder rather than my SQL query itself, so please bear with me...

As an example to show what I'm looking to achieve, suppose I had a large list of tables in a database, all of which had the same structure, and I wanted to create a query to pull data from all of them in one single query.

The way I thought of doing this was:

    Dim MyTables As List(Of String) = {"Table1", "Table2", "Table3" ... "TableN"}.ToList
    Dim SQLQuery As New StringBuilder

    For Each tbl As String In MyTables
        SQLQuery.AppendLine("SELECT [col1], [col2], [col3] FROM " & tbl)
        SQLQuery.AppendLine("UNION")
    Next

Again, ignoring whether this is the best way to do that (although I'd love to know if there is a better way, but I don't want the answers to be skewed in that direction - I'm more interested in knowing how to deal with the StringBuilder efficiently in this case).

Given the way I do this, my StringBuilder is going to have an extra UNION & vbCrLf at the end of it that I need to remove - I know I could do it using something along the lines of:

SQLQuery.Remove(SQLQuery.Length - ("UNION" & vbCrLf).Length, ("UNION" & vbCrLf).Length)

And, of course, I could make that more efficient, but my question basically is what is the best / most efficient way to create such a stringbuilder using a loop where the "concatenator", as such, is a fairly large string that would be appended to the end if you did it my way?

I know this question was written in VB, but I'm equally as comfortable with answers in C#. THANKS!!!

Answer 1

When you need to build a string composed of elements of a collection separated by a separator string of your choice, you do not need to use StringBuilder directly. Moreover, you do not even need a loop: you can use String.Join method that takes a separator and an IEnumerable<T>, and returns a string composed of the collection's elements separated by whatever separator you specify:

// C# answer
var query = string.Join(
    "\nUNION\n"
,   MyTables.Select(tbl => "SELECT [col1], [col2], [col3] FROM "+ tbl)
);

Note: unless you really want your RDBMS to eliminate duplicates, consider using UNION ALL operator in place of UNION. When the number of rows is large, the effects of this change on the performance could be significant.

Answer 2

I'd do something like this instead:

Dim myTables = {"Table1", "Table2", "Table3" ... "TableN"}
Dim queries = myTables.Select(Function(tbl) "SELECT [col1], [col2], [col3] FROM " & tbl)

Return String.Join(vbNewLine & "UNION" & vbNewLine, queries)

And anyway, the sql query will be what takes most of the time. The gain in time switching from concatenation to string.join to StringBuilder will be nothing compared to the time your query takes to execute (think 1ms vs 1000ms).

Answer 3

I'd suggest appending it to a List<string> and joining it in the end, but I don't know how it can affect your performance.

An alternative is to use a flag that shows that you are in the first iteration and append UNION before the query, not after.

bool first = true;

foreach (string tbl in MyTables)
{
    if (first)
        first = false;
    else
        SQLQuery.AppendLine(" UNION ");

    SQLQuery.Append("SELECT [col1], [col2], [col3] FROM ");
    SQLQuery.AppendLine(tbl);
}

Answer 4

Well you could just use a VIEW in your database if it supports it and the tables involved are always the same.

But staying with the StringBuilder approach then I will try to truncate the length with just

 SQLQuery.Lenght -= 7

ot (if you want to be clear)

 SQLQuery.Lenght -= ("UNION"+ Environment.Newline).Length

Then I suggest to create the StringBuilder with an estimated capacity. For example, if the mean length one of your SELECT statement is 300 chars and you have 10 tables you could start creating a StringBuilder with an internal capacity of 4000 chars

Dim SQLQuery = New StringBuilder(4000)

This would avoid a constant rework to enlarge the internal buffer while new data is added.