Mongoose Populate returning null or undefined

Question

I'm sorry if this is a n00b question, I've been searching Google & Stack for hours now and I've got to ask!

I have two schemas, User and Story, shown below. I am trying to reference the User for a Story using the Ref option to Populate in a Query - I've using mySQL before and so wanted to try to replicate a JOIN statement. Whenever I try to use populate I just get the objectID returned, or null (shown below).

Edited 12 Nov to fix hardcoded IDs & add console data

story-schema.js

var mongoose = require('mongoose'),
Schema = mongoose.Schema,
User = require('./user-schema');

var StorySchema = new Schema({  
title: { type: String, required: true },  
author_id: { type: Schema.Types.ObjectId, ref: 'User' },
summary: { type: String, required: true },
rating: { type: String, required: true }
});

module.exports = mongoose.model('Story', StorySchema, 'stories');

user-schema.js

var mongoose = require('mongoose'),
Schema = mongoose.Schema;

var UserSchema = new Schema({
    username: { type: String, required: true, index: { unique: true } },
    is_admin: {type: Boolean, required: true, default: false }
});

save - id hardcoded for example

app.post('/api/new_story', function(req, res){
var story;
story = new Story({
    title: req.body.title,
    author_id: mongoose.Types.ObjectId(req.params._id), 
            /* ex of req.params._id: 527fc8ff241cdb8c09000003*/
    summary: req.body.summary,
    rating: req.body.rating
});

story.save(function(err) {
    if (!err) {
        return console.log("created");
    } else {
        return console.log(err);
    }
});
return res.send(story);
});

example user when logged in terminal

{
"__v" : 0,
"_id" : ObjectId("527fc8ff241cdb8c09000003"),
"is_admin" : false,
"username" : "ted"
}

example story when logged in terminal

{
"title" : "Test Story",
"author_id" : "527fc8ff241cdb8c09000003",
"summary" : "Test summary",
"rating" : "12",
"_id" : ObjectId("52827692496c16070b000002"),
"__v" : 0
}

queries

//other mongoose/app includes above
User = require('./config/schema/user-model'),
Story = require('./config/schema/story-model');

// data.author_id = 527fc8ff241cdb8c09000003 
// data.author_id.username = undefined
app.get('/api/query/:id', function (req, res){
return Story.findOne({_id:req.params.id})
.populate( { path: 'User' } )
.exec(function (err, data) {
            console.log(data.author_id);
            console.log(data.author_id.username);
          if (err) {
            return res.json({error:err})
          }
        })
});

// data.author_id = null 
app.get('/api/query2/:id', function (req, res){
return Story.findOne({_id:req.params.id}) //_id hardcoded for example
.populate( 'author_id' )
.exec(function (err, data) {
            console.log(data.author_id);
          if (err) {
            return res.json({error:err})
          }
        })
});

In the first query I get the author_id I already saved back, which kind of makes sense as that's what I saved - but I access the username.

In the second query I can't even access the author_id I've already saved.

Edit: I can run a normal GET query fine without the 'populate'

What I'd like to happen

Is to be able to access the author information from the story - this is more like a proof of concept. Eventually I'd like to reference the Story _id in the the User model as there can be many Stories to a User, but only one User per Story but thought I'd start here first.

Answer 1

Since your example includes raw Object IDs not even quoted as valid javascript strings, it's hard to understand what is a mistake in your question vs a mistake in your code, but mostly your second query looks like it should work. I suspect you are mixing up your ObjectId values. You are using the same ObjectId for your story and your user and that won't work:

   author_id: mongoose.Types.ObjectId(528149b38da5b85003000002),

That's the same ID you use to lookup the story. So I think the fix is:

1. Make sure you have a valid user record in the users collection
2. Make sure your test story record's story.author_id matches that
3. Make sure you are looking up the story by the story Id and not the author_id.

If you still can't get this working, code the entire thing in a single .js file with both schemas, both models, and some code to create the records then do the queries, and post that.

Answer 2

I was stuck with this problem for a week. After trying everything and making sure that I had no circular references, I decided to reset my DB.

After removing all collections, and storing them again, all my relations managed by mongoose's populate() worked.

It is so frustrating because this issue is not documented and I lost 20-30 hours trying to make a stupid collection reference work.

Answer 3

Is it possible that you forget to include a "ref" field in your Schema? I just encountered this issue and after adding a ref field to my Schema it works immediately without any need to reset the database. Hope this information helps.

I found an example documented here: https://mongoosejs.com/docs/3.0.x/docs/populate.html. Although it is a documentation for version 3.0.x, I found it helpful even for version 5.5.5 which is now I am using.

Answer 4

I stuck in this problem for 5-6 hours and found a very simple solution for that. In this case you can see here :

    {
   "title" : "Test Story",
   "author_id" : "527fc8ff241cdb8c09000003",
   "summary" : "Test summary",
   "rating" : "12",
   "_id" : ObjectId("52827692496c16070b000002"),
   "__v" : 0
   }

in author_id value '527fc8ff241cdb8c09000003' is basically a "string" not ObjectId. This happens when you restore the database from json or csv and during this process ObjectId converts into the string. In My case I have dumped data using Python and try to retrieve the data using express. So to solve this problem simply I updated the data using express and now "Populate" is working try updating the data :

 const { ObjectId }  = require('bson');
    app.get('/update-data', (req,res)=>{
         Story.find()
        .exec((err,posts)=>{
            for(let value of posts)
            {
             User.findOne({ _id : value.author_id })
             exec((err,userData)=>{
                      Story.update({ _id : value._id }, {  author_id : ObjectId(userData._id) })
              .exec((error, resData)=>{
                   console.log(resData);
                 });
               })
            
            }
            res.send('Data Updated, try populating data again');
        });
    });
    

and that's it, after reestablishing the relation with user try populating the field again.

Update 1 :
I forgot to mention required package for 'ObjectId()' must include this

const { ObjectId }  = require('bson');

Update 2 : After digging more deeper I found that updating is not the solution. I found that you need to include the all model you want to populate. for example : there is a countryModel which is connected to user. then in your code country model and user model both should be included like this way:

var userModel = require('../<path to model>/userModel');
var countryModel = require('../<path to model>/countryModel');

however you are not using countryModel but this need to be initiate. And for country Model code should be : countryModel.js

const mongoose = require('mongoose');

const countrySchema = mongoose.Schema({
country_name : String,
flag_image : String,
country_code : String
});

//user model 
var countryModel = mongoose.model('countries', countrySchema);
module.exports = countryModel;

Answer 5

Firstly this happens when you add populate method after creating some collections.

You can make new request and create new collections for the connected models(all). Then populate method should work.

Previously created models will still send null but newly created collections should return populated data.

Answer 6

If your data is very small then Try clearing your data from tables related to populate and main one

Answer 7

Sometimes this error occurs when you try to populate something that doesn't exist anymore in the database. I encountered this error because of this reason.

Answer 8

I stumbled upon this and the problem was referencing a deleted document. reset your DB if you can or double check the referenced id

Answer 9

I managed to solve this problem when declared the ref as model class reference instead of string with the name of the model.

// before
let Schema = mongoose.Schema({
    name: String,
    author: { type: Schema.Types.ObjectId, ref: 'Author' }
});

//after
let Schema = mongoose.Schema({
    name: String,
    author: { type: Schema.Types.ObjectId, ref: Author }
});