Lazy loading dojo tree powered by JsonRestStore and jsonp

Question

I am trying to implement a cross domain, lazy loading tree using the Dojo toolkit. So far I have the top level nodes displaying correctly, but on clicking the expando I get a "deferred has already been resolved" error and I'm not sure why. I can see that the fetch method seems to be working by looking at the firebug network tab. I think my problem is within my _processResults method, perhaps something to do with the defining of the _loadObject within it...

I feel like I should know Dojo better given all the time I've spent trying to understand it. But alas, its quite a beast... I've seen some mention of JSONP and lazyloading not working in one of the sitepen blogs (http://www.sitepen.com/blog/2008/06/25/web-service-data-store/ ), but it doesn't mention why it shouldn't be possible other than JSONP being asyncronus. If Dojo is going to just stuff incoming json data into the store, I don't understand why that should matter.

Perhaps it has to do with my formating of the data - another example on sitepen (http://www.sitepen.com/blog/2010/01/27/efficient-lazy-loading-of-a-tree/) using the jsonreststore doesnt load an item until you expand a node, whereas my format loads the item but doesnt load the children nodes until you expand it...

Without further ado, here is ta codez...

<script type="text/javascript">
dojoConfig = {
parseOnLoad: true,
isDebug: true,
usePlainJson: true  
};
</script>
<script type="text/javascript" src="scripts/dojo_16/dojo/dojo.js"></script>

<script type="text/javascript">
    dojo.require("dojo.parser");
    dojo.require("dojo.io.script");
    dojo.require("dojox.rpc.Service");
    dojo.require("dojox.data.ServiceStore");                    
    dojo.require("dijit.tree.ForestStoreModel");                    
    dojo.require("dijit.Tree");         

    dojo.addOnLoad(function(){

        var mySmd = {
                "SMDVersion": "2.0",
                "id": "http://urlbehindfirewall/testtree/", 
                "description": "This is the service to get to the finder app backend data",

                transport: "JSONP",
                envelope: "URL",
                additionalParameters: true,
                target: "http://urlbehindfirewall/testtree/",               

                services: {
                    "getChildrenNodes": {
                    "target": "getChildrenNodes.php",
                        parameters: [
                            { name: "nodeId", type: "integer"}                             
                        ]
                    }
                }

        };

        var myService = new dojox.rpc.Service(mySmd);

        var myStoreParams = {               
            service : myService.getChildrenNodes,
            idAttribute : "Node_Id",
            labelAttribute : "Display_Name",
            _processResults: function(results, def){                                        
                var _thisStore = this;
                for(i in results){
                    results[i]._loadObject = function(callback){
                        _thisStore.fetch({
                        query: { nodeId: this.Node_Id },
                        onItem: callback
                        });
                    delete this._loadObject;
                    };
                }                   
                return {totalCount: results.length, items: results};                                                
            }
        };


        var myStore = new dojox.data.ServiceStore(myStoreParams);

        //create forestTreeModel
        var treeModelParams = {
            store: myStore,
            deferItemLoadingUntilExpand: true,
            childrenAttrs: ["Children_Nodes"],              
            rootId : 1,
            query: 1
            };

        var treeModel = new dijit.tree.ForestStoreModel(treeModelParams);

        var myTree = new dijit.Tree({
            model: treeModel,
            id: "myTree",
            showRoot: false 
            });

        dojo.byId("treeContainer").appendChild(myTree.domNode);
        myTree.startup();                   
    });

</script>

And here is an example of the json data structure: unfortunately the service is behind a network firewall currently... i will work on putting up a public version to demonstrate in a bit. Meanwhile, this is the response for searching on the root node, node 1:

    [
   {
      "Node_Id":"2",
      "Display_Name":"LeftNode:2",
      "Secondary_Names":"Parent:1",
      "Obi_Id":"10002",
      "Children_Nodes":[

      ],
      "Has_Children":true,
      "Child_Node_Ids":[
         "5",
         "6",
         "7",
         "8"
      ]
   },
   {
      "Node_Id":"3",
      "Display_Name":"Middle Node:3",
      "Secondary_Names":"Parent:1",
      "Obi_Id":"10003",
      "Children_Nodes":[

      ],
      "Has_Children":true,
      "Child_Node_Ids":[
         "9",
         "10"
      ]
   },
   {
      "Node_Id":"4",
      "Display_Name":"Right Node:4",
      "Secondary_Names":"Parent:1",
      "Obi_Id":"10004",
      "Children_Nodes":[

      ],
      "Has_Children":true,
      "Child_Node_Ids":[
         "11",
         "12"
      ]
   }
]

Expanding any of the above nodes would then get the children of that node - so 2 would get the node array of 5,6,7,8 . (It may not be necessary to have the Child_Node_Ids and Children_Nodes for the current implementation, but shouldn't be breaking anything no?)

So I'm sure eyes are glazing by now, to restate the issue - what is creating this "deferred has already been resolved" error? Is lazy loading in a tree possible with JSONP? Would a different json structure solve my lazy loading problems? Is it possible to reformat my data within dojo so that it works? (I thought that was the point to the _processResults method... ) Are there any publicly accessible tree dataservices out there to practice on?

Thanks everyone!

Answer 1

After a lot of experimentation and frustration these are my findings:

Yes, lazy-loading of a tree is possible using JSONP. Yes, putting my data into a different format helped in the resolving of the lazy loading issue. I found a number of stumbling blocks along the way, which I will mention later.

Here is the code for a working implementation. First the service:

pretendService.php

$callback = $_GET["callback"];
$nodeName = $_GET["node"];
$fileName = "pretendServiceJson/".$nodeName.".js";

print($callback . "(" . file_get_contents($fileName) . ")" );

?>

pretendServiceJson/0.js - this is the initial data load note it is an array!

[
{
   "Node_Id":"0",
   "Display_Name":"",
   "Children":[
      {
         "Node_Id":"1",
         "Display_Name":"node 1",
         "Obi_Id":"02",
         "Secondary_Names":"Blah blah secondary name node 1"
      },
      {
         "Node_Id":"2",
         "Display_Name":"node 2",
         "Obi_Id":"o2",
         "Secondary_Names":"Blah blah secondary name node 2"
      },
      {
         "$ref":"3",
         "Display_Name":"node 3",
         "Obi_Id":"o3",
         "Secondary_Names":"Blah blah secondary name node 3",
         "Children":true
      },
      {
         "Node_Id":"4",
         "Display_Name":"node 4",
         "Obi_Id":"o4",
         "Secondary_Names":"Blah blah secondary name node 4"
      },
      {
         "Node_Id":"5",
         "Display_Name":"node 5",
         "Obi_Id":"o5",
         "Secondary_Names":"Blah blah secondary name node 5"
      }
   ]
}
]

pretendServiceJson/3.js - this will be the first lazy loaded item - note it is an object!!!

{
    "Node_Id": "3",
    "Display_Name": "node 3",
    "Obi_Id": "o3",
    "Secondary_Names": "Blah blah secondary name node 3",
    "Children": [
        {
            "$ref": "6",
            "Display_Name": "node 6",
            "Obi_Id": "o6",
            "Secondary_Names": "Blah blah secondary name 06",
            "Children":true
        },
        {
            "Node_Id": "7",
            "Display_Name": "node 7",
            "Obi_Id": "o7",
            "Secondary_Names": "Blah blah secondary name 07"
        }
    ]
}

There is another json file 6.js but I think you get the point. Finally the magic...

            dojo.require("dojo.parser");
        dojo.require("dojo.io.script");
        dojo.require("dojox.rpc.Service");
        dojo.require("dojox.data.JsonRestStore");                   
        dojo.require("dijit.tree.ForestStoreModel");                    
        dojo.require("dijit.Tree");         

        dojo.addOnLoad(function(){

            var mySmd = {
                    "SMDVersion": "2.0",
                    "id": "http://localhost/pretendService.php", 
                    "description": "This is the service to get to the finder app backend data",

                    transport: "JSONP",
                    envelope: "URL",
                    additionalParameters: true,
                    target: "http://localhost/",                

                    services: {
                        "getNode": {
                        "target": "pretendService.php",
                            parameters: [
                                { name: "node", type: "string"}                            
                            ]
                        }
                    }
            };

            var myService = new dojox.rpc.Service(mySmd);                       

            var myStore = new dojox.data.JsonRestStore({                
                service : myService.getNode,
                idAttribute : "Node_Id",
                labelAttribute : "Display_Name"             
            });     

            //create forestTreeModel
            var treeModelParams = {
                store: myStore,
                deferItemLoadingUntilExpand: true,
                childrenAttrs: ["Children"],                
                //rootId : "0",
                query: "0"
                };

            var treeModel = new dijit.tree.ForestStoreModel(treeModelParams);

            var myTree = new dijit.Tree({
                model: treeModel,
                id: "myTree",
                showRoot: false,
                persist: false
                });

            dojo.byId("treeContainer").appendChild(myTree.domNode);
            myTree.startup();

        });

    </script>
</head>

<body class="tundra">

<div id="treeContainer"></div>

</body>
</html>

The biggest lesson here is that there are two seperate methods (as best as I am able follow) that put the data into the store, and then subsequently into the tree. The first dataload will come from a fetch, and it will expect an array of items. The lazy-loaded items that follow (assuming you have setup the service correctly and you are getting responses back) will go through loadItem and that method will expect an object.

If you have the initial load data coming in as an object you will not see any tree on your page, despite seeing the response in firebug. No errors though.

If you have the lazy-loaded data coming in as an array you get a "TypeError: args.item is undefined" - it seems loadItem gets called 2x and the second time instead of your result object its an empty object.

If you have rootId defined when you create the store it will not display the tree and give you a "Node cannot be inserted at specified Hierarchy" error.

All of the above errors were found using JsonRestStore. The documentation says JsonRestStore inherits the read functionality of ServiceStore, but if I switch out the two stores I get the error "Node cannot be inserted at specified hierarchy".

One of my biggest frustrations with Dojo is simply the lack of documentation specifying how actual data gets parsed successfully into items for the data stores. There is a lot of talk about the flexibility and modularity of the data stores, but how to get my data in there and have it work is still a mystery, and this solution came to me as i grasped at straws. I'd love to have a sitepen article about how data becomes datastore items someday...? :)

I hope this helps someone else out there. Happy coding.