How can a SQL Server CE database be simultaneously connectable and un-?

Question

I have a SQL Server CE database in my Windows CE app. I can see it (HHS.sdf) in my project in Server Explorer:

...but the Columns folders for the tables won't expand; clicking them gives me:

The path property for HHS.sdf reads:

Data Source=Mobile Device\Program Files\hhs\HHS.sdf

If the path is not valid, why is Server Explorer showing me the database at all? If I create a new connection in Server Explorer (by selecting "Add Connection..." from the Data Connections context menu), I can navigate to the .SDF file on the device (\Program Files\hhs), test the connection in the "Add Connection" dialog:

...and get a "Test connection succeeded" corroboration.

But at runtime, when trying to populate one of the tables:

var dt = new DataTable();
dt.Columns.Add(new DataColumn("Id", typeof(Int32)));
dt.Columns.Add(new DataColumn("DeptNumber", typeof(Int32)));
dt.Columns.Add(new DataColumn("DeptName", typeof(string)));

foreach (HHSUtils.Department dept in depts)
{
    try
    {
        dt.Rows.Add(dept.Id, dept.DeptNumber, dept.DeptName);
        countAdded++;
    }
    catch (Exception ex)
    {
        MessageBox.Show(string.Format("deptId == {0}, deptNumber == {1},  
          deptName == {2} ({3})", dept.Id, dept.DeptNumber, dept.DeptName, ex.Message));
    }
}
// Now move it all into the table
using (var bc = new SqlCeBulkCopy(dataSource))
{
    bc.DestinationTableName = "Departments";
    bc.WriteToServer(dt);
}

...I get, "Source column 'DEPTNUM' does not exist and destination does not allow nulls" on the WriteToServer() line.

Evaluating "dt" in the debugger, I see that the DataSet property has a red "error" icon and says, "could not evaluate expression"; the same goes for the "Site" and "XMLTest" properties; The dataSet (as opposed to DataSet) property is null.

The word "DEPTNUM" appears nowhere in the client source code, nor is it anywhere in the server source code...

The same code for the InventoryItems table executes without an err msg (but the Columns are still exceedingly bashful in Server Explorer).

I am creating the tables this way:

// Got this from http://stackoverflow.com/questions/20254214/how-can-i-programmatically-determine-if-a-table-exists-within-a-sql-server-ce-da/20339969?noredirect=1#20339969
public static bool TableExists(SqlCeConnection connection, string tableName)
{
    using (var command = new SqlCeCommand())
    {
        command.Connection = connection;
        var sql = string.Format(
            "SELECT COUNT(*) FROM information_schema.tables WHERE table_name = '{0}'", tableName);
        command.CommandText = sql;
        var count = Convert.ToInt32(command.ExecuteScalar());
        return (count > 0);
    }
}

public static void ConditionallyCreateTables()
{
    const string sdfPath = @"\Program Files\hhs\HHS.sdf";
    string dataSource = string.Format("Data Source={0}", sdfPath);

    if (!File.Exists(sdfPath))
    {
        using (var engine = new SqlCeEngine(dataSource))
        {
            engine.CreateDatabase();
        }
    }

    using (var connection = new SqlCeConnection(dataSource))
    {
        connection.Open();
        using (var command = new SqlCeCommand())
        {
            command.Connection = connection;

            // It may be that I'll want to replace "if (!TableExists())" with "if (TableExists())" and then either drop the table in that
            // case and recreate it, or delete its contents
            if (!TableExists(connection, "InventoryItems"))
            {
                command.CommandText = "CREATE TABLE InventoryItems (Id nvarchar(50) NOT NULL, PackSize smallint NOT NULL, Description nvarchar(255), DeptDotSubdept numeric, UnitCost numeric, UnitList numeric, UPCCode nvarchar(50), UPCPackSize smallint, CRVId int);";
                command.ExecuteNonQuery();
            }

            if (!TableExists(connection, "Departments"))
            {
                command.CommandText = "CREATE TABLE Departments (Id int NOT NULL, DeptNum int NOT NULL, DepartmentName nvarchar(255))";
                command.ExecuteNonQuery();
            }

            if (!TableExists(connection, "Subdepartments"))
            {
                command.CommandText = "CREATE TABLE Subdepartments (Id int NOT NULL, DeptId int NOT NULL, SubdeptId int NOT NULL, DepartmentName nvarchar(255))";
                command.ExecuteNonQuery();
            }

            if (!TableExists(connection, "Redemptions"))
            {
                command.CommandText = "CREATE TABLE Redemptions (Id int NOT NULL, RedemptionId nvarchar(50), RedemptionItemId nvarchar(50), RedemptionName nvarchar(255), RedemptionAmount numeric, RedemptionDept nvarchar(50), RedemptionSubdept nvarchar(50))";
                command.ExecuteNonQuery();
            }
        }
    }
}

[{ }] [{ }] [{ }] [{ }] [{ }] [{ }] [{ }] [{ }] [{ }] [{ }] [{ }] [{ }]

On a side note, I have come to the conclusion (after 20 years of chipping away at the walls in this salt mine of a profession, and being the hapless victim of countless cave-ins of varying severity[*]) that to be a computer programmer you either need to be insane (so that you are not driven anywhere by the needling tediousness of it all) or absolutely impervious to insanity (for the same reason); if you start off at either end of the spectrum, you'll probably be okay; for those in the middle of that continuum, though (which I hold to be a goodly fraction of the general populace): beware, lest your senses be crushed to a formless pulp or pile-driven to shrieking (if muffled) madness.

[*****] Yea, verily, on each disinterment I was found clutching the pick in my bleeding hands.

UPDATE

I solved the "DEPTNUM" error by making sure the column names in the temp table corresponded exactly to the column names in the SQL Server CE table (which, according to Server Explorer, don't exist anyway):

var dt = new DataTable();
dt.Columns.Add(new DataColumn("Id", typeof(Int32)));
dt.Columns.Add(new DataColumn("DeptNum", typeof(Int32)));
dt.Columns.Add(new DataColumn("DepartmentName", typeof(string)));

However, as hinted at above, the main conundrum (the post's nominal question) remains unsolved.

UPDATE 2

Letting the computer take a nap and "think about it" hasn't helped anything; the same holds true today. The code to add records to the SQL Server CE table works fine; is it just that Server Explorer, although being able to connect to the handheld device's database file fine (test connection succeeds) can't see the file, or pretends that it can't? BTW/AWTTW: I'm going to bountify this ASAP (sometime tomorrow).

UPDATE 3

This test:

if (!File.Exists(sdfPath))
{
    using (var engine = new SqlCeEngine(dataSource))
    {
        engine.CreateDatabase();
    }
}

...fails (the conditional code is not entered), so the handheld/Windows CE app does see the file as existing at runtime.

UPDATE 4

I am able to read from and write to the tables, so the app can definitely see them; I can also look at the tables and their data on the handheld device by navigating to the .SDF file; a SQL Server CE tool is invoked when 2-clicking the .SDF file. Server Explorer in Visual Studio 2008 needs a wake-up call, though.

Answer 1

Well first I'll say that I have never, ever, ever used the "Add Connection" stuff to try to connect directly to a database on the device. May pas dozen-plus years of developing for Windows CE have taught me that just about anything that requires interaction between the PC and the device outside of the debugger itself is fraught with peril. ActiveSync == "There be Monsters Here" on the map.

So am I surprised this doesn't work? Not at all. I'm actually more surprised it's even an option on the dialog. It's possible that maybe the device version of SQL CE and the PC version differ, and it's that difference that's making it so the desktop can't read the remove file properly. Or maybe it's just a buggy connection (again, see "ActiveSync").

So how do you move forward? Easy. Copy the database from the device to the PC, then open up the PC copy with Studio. Seriously. That's exactly how I do it, and it works just fine.

I think I've already recommended this to you before, but it's worth repeating. Avoid DataTables. They are gigantic memory hogs, and under CE you can't afford a memory hog. Basically it duplicates the entire source table into RAM for you. SqlCeDataReader and SqlCeResultSet are your friends.