Importing TSV to MongoDB on meteor servers

Question

I am experimenting with meteor and have deployed my app to meteor's servers. The app is a simple dynamic filtering engine where the data came from a TSV file. On my home machine, I used mongoimport (localhost:3001) to import the TSV into a local db but on the meteor servers, I received a fresh new empty database. I'm wondering how to import a TSV to a database hosted on meteors servers. I'm sure there are security peculiarities dealing with a publicly provided hosting service and guess that's my issue.

I have logged into my developer account on the meteor servers via the terminal on OSX, yielding server information like: production-db-c1.meteor.io:27017/myApp_meteor_com

With that server info, I follow on with ./mongoimport like so: (not a cut and paste so ignore typos)

./mongoimport --host myApp_meteor_com/production-db-c1.meteor.io:27017 --collection myCollection -u User -p Pass --type tsv --headerline --file blah.tsv

The User/Pass info I obtained by first adding myself as a user with 'readWrite' access and then querying the db.system.users collection on the meteor servers for my app. There were three users in the collection, one of which was named, "myApp_meteor_com", one I added myself and another that apparently was the connected terminal client. I tried each user/pass combinations in the ./mongoimport string above.

*cough It's not really called 'myApp' but you get the idea

Here's the long winded echo from the terminal after executing mongoimport

Thu Mar  6 12:38:38.412 kern.sched unavailable
Thu Mar  6 12:38:38.415 starting new replica set monitor for replica set myApp_meteor_com with seed of production-db-c1.meteor.io:27017
Thu Mar  6 12:38:38.467 successfully connected to seed production-db-c1.meteor.io:27017 for replica set myApp_meteor_com
Thu Mar  6 12:38:38.518 warning: node: production-db-c1.meteor.io:27017 isn't a part of set: myApp_meteor_com ismaster: { setName: "production-c", ismaster: true, secondary: false, hosts: [ "production-db-c1.meteor.io:27017", "production-db-c3.meteor.io:27017", "production-db-c2.meteor.io:27017" ], arbiters: [ "production-dbarb-c2.meteor.io:27017", "production-dbarb-c1.meteor.io:27017" ], primary: "production-db-c1.meteor.io:27017", me: "production-db-c1.meteor.io:27017", maxBsonObjectSize: 16777216, maxMessageSizeBytes: 48000000, localTime: new Date(1394131118501), ok: 1.0 }
Thu Mar  6 12:38:40.518 warning: No primary detected for set myApp_meteor_com
Thu Mar  6 12:38:40.519 All nodes for set myApp_meteor_com are down. This has happened for 1 checks in a row. Polling will stop after 29 more failed checks
Thu Mar  6 12:38:40.519 replica set monitor for replica set myApp_meteor_com started, address is myApp_meteor_com/
couldn't connect to [myApp_meteor_com/production-db-c1.meteor.io:27017] connect failed to replica set myApp_meteor_com/production-db-c1.meteor.io:27017

Any help from meteor/mongodb experts is greatly appreciated.

Answer 1

The above helped me out quite a bit, but I ran into authentication issues. So after a bit more research into how the meteor servers issue temporary user/ passwords, I finally was able to import a TSV file into my meteor deployed app's mongoDB.

The "key" for me was to use:

meteor mongo --url myApp

in order to gain a user/pass. It is my understanding that when this command is run, a new user/pass is created each time on the meteor servers and is only good for a very short while (60 secs?). When I ran that, I received this echo at the command prompt

mongodb://client-faddb071:a74c4b2a-15bc-2dcf-dc58-b5369c9ebee3@production-db-c1.meteor.io:27017/myApp_meteor`_com

From that info I was able to extract the username: "client-faddb071" and the password "a74c4b2a-15bc-2dcf-dc58-b5369c9ebee3"

and then in another terminal window (because the user/pass doesn't last long), I was ready to go with the mongo import command:

>  ./mongoimport --host production-db-c1.meteor.io:27017 --username client-faddb071  --password a74c4b2a-15bc-2dcf-dc58-b5369c9ebee3  --db myApp_meteor_com --collection myCollection --drop --type tsv --headerline --file /path/to/file.tsv 

That worked verbatim for me and 3888 records from the TSV were successfully loaded into my meteor hosted mongoDB. Thanks all for the input, it all contributed to my final knowledge and success.

Answer 2

Highlighting these to make it clear:

Thu Mar 6 12:38:38.518 warning: node: production-db-c1.meteor.io:27017 isn't a part of set: myApp_meteor_com ismaster: { setName: "production-c", ismaster: true, (...)

Thu Mar 6 12:38:40.518 warning: No primary detected for set myApp_meteor_com

You are connecting to the primary, so that is okay. But you have the wrong name for the replica set. The first warning tells you the name of the replica set ("production-c") that this host is a member of and also dumps the configuration list of members.

A better connection argument would be:

./mongoimport --host production-c/production-db-c1.meteor.io,production-db-c3.meteor.io, production-db-c2.meteor.io --collection myCollection -u User-p Pass --type tsv --headerline --file blah.tsv

As this contains some seed list info in the event your current primary was switched to a secondary.