Organizing sqlite3 C/C++ prepared statements (avoiding mess of global code)

Question

I'm writing an application that needs to use prepared statements to get the performance benefit, but I'm wondering when you have application with dozens if not hundreds of prepared statements, how do you manage it without it turning into a mess of global code? Do you just have to prepare all of the statements in a constructor/function somewhere entirely different than where they're used?

Using sqlite_exec is nice since the query is right there where you're actually using it, but with prepared statements I end up having them in entirely different areas of code, with it vague/confusing about what variables need to be bound in the functions that actually use them.

Specifically right now I just have a Database singleton that has private variables,

sqlite3_stmt *insertPacketCount;
sqlite3_stmt *insertPortContacted;
sqlite3_stmt *insertPacketSize;
sqlite3_stmt *computePacketSizeVariance;
 ...

With the constructor having dozens of calls to sqlite3_prepare_v2,

// Set up all our prepared queries
res = sqlite3_prepare_v2(db,
    "INSERT OR IGNORE INTO packet_counts VALUES(?1, ?2, ?3, 1);",
    -1, &insertPacketCount,  NULL);
expectReturnValueAndFail(SQLITE_OK);
 ...

And the actual functions that use them elsewhere,

void Database::IncrementPacketCount(std::string ip, std::string interface, std::string type, uint64_t increment)
{
    int res;

    res = sqlite3_bind_text(incrementPacketCount, 1, ip.c_str(), -1, SQLITE_STATIC);
    expectReturnValue(SQLITE_OK);
    res = sqlite3_bind_text(incrementPacketCount, 2, interface.c_str(), -1, SQLITE_STATIC);
    expectReturnValue(SQLITE_OK);
    res = sqlite3_bind_text(incrementPacketCount, 3, type.c_str(), -1, SQLITE_STATIC);
    expectReturnValue(SQLITE_OK);
    res = sqlite3_bind_int(incrementPacketCount, 4, increment);
    expectReturnValue(SQLITE_OK);

    res = sqlite3_step(incrementPacketCount);
    expectReturnValue(SQLITE_DONE);
    res = sqlite3_reset(incrementPacketCount);
    expectReturnValue(SQLITE_OK);
}

Inside of the functions that use the prepared queries, I keep having to tediously go back to the prepare statement to figure out what the bind indexes are.

Sorry if this is vague, but are there any libraries or methods for organizing this sort of thing without sacrificing any performance/safety (like just appending strings and calling sqlite_exec)?

Answer 1

You should not move the query string away from the query execution. If you write a helper function that prepares a statement only when it is first needed, you also avoid having to prepare all of them at the beginning:

void Database::PrepareStatementIfNeeded(sqlite3_stmt **stmt,
                                        const std::string& sql)
{
    if (*stmt == NULL) {
        res = sqlite3_prepare_v2(db, sql.c_str(), -1, stmt, NULL);
        ...
    }
}

(But you still have to ensure to finalize all of them.)

Furthermore, if you give the parameters names instead of numbers, you can use sqlite3_bind_parameter_index to get the proper index:

void BindParam(sqlite3_stmt *stmt,
               const char *name,
               const std::string& value) // overload for other types
{
    int index = sqlite3_bind_parameter_index(stmt, name);
    if (index == 0)
        error...;
    int res = sqlite3_bind_text(stmt, index, value.c_str(), -1, SQLITE_TRANSIENT);
    ...
}

void Database::IncrementPacketCount(...)
{
    PrepareStatementIfNeeded(&incrementPacketCount,
                             "INSERT OR IGNORE INTO packet_counts"
                             " VALUES(:ip, :intf, :type, 1)");
    BindParameter(incrementPacketCount, "ip",   ip);
    BindParameter(incrementPacketCount, "intf", interface);
    BindParameter(incrementPacketCount, "type", type);
    ...
}

(And those sqlite3_step/sqlite3_reset calls look as if you are repeating them umpteen times; create a helper function for them, too.)