SmartGWT RestDataSource JSON response text does not appear to be in standard response format

Question

I have a web-based application with GWT 2.5.1, SmartGWT 4.0, Spring 3.2.3.Release, and Hibernate 4.1. The front-end uses a SmartGWT RestDataSource that passes data to RESTful web-service, a Spring MVC Controller, that passes data to the front end with Java.

The Controller is unit tested and works great, I use a GET to pass back the data in JSON, the controller calls the back-end, gets my data and I return a UserEntity back to the RestDataSource in JSON format.

The error is: [ERROR] [TestAdmin] - 22:01:22.432:XRP8:WARN:RestDataSource:restLoginDS:RestDataSouce transformResponse(): JSON response text does not appear to be in standard response format.

I have done a lot of Googling, and looking on this site, and I can find people with similar issues, but no good solutions.

Here is the RestDataSource:

public class LoginDataSource extends RestDataSource
{
private static LoginDataSource instance = null;

public static LoginDataSource getInstance()
{
    if (instance == null)
    {
        instance = new LoginDataSource("restLoginDS");
    }
    return instance;
}

private LoginDataSource(String id)
{
    setID(id);
    setClientOnly(false);

    // set up FETCH to use GET requests
    OperationBinding fetch = new OperationBinding();
    fetch.setOperationType(DSOperationType.FETCH);
    fetch.setDataProtocol(DSProtocol.GETPARAMS);
    DSRequest fetchProps = new DSRequest();
    fetchProps.setHttpMethod("GET");
    fetch.setRequestProperties(fetchProps);

    // set up ADD to use POST requests
    OperationBinding add = new OperationBinding();
    add.setOperationType(DSOperationType.ADD);
    add.setDataProtocol(DSProtocol.POSTMESSAGE);
    // ===========================================
    DSRequest addProps = new DSRequest();
    addProps.setHttpMethod("POST");
    // addProps.setContentType("application/json");
    add.setRequestProperties(addProps);

    // set up UPDATE to use PUT
    OperationBinding update = new OperationBinding();
    update.setOperationType(DSOperationType.UPDATE);
    update.setDataProtocol(DSProtocol.POSTMESSAGE);
    // ===========================================
    DSRequest updateProps = new DSRequest();
    updateProps.setHttpMethod("PUT");
    // updateProps.setContentType("application/json");
    update.setRequestProperties(updateProps);

    // set up REMOVE to use DELETE
    OperationBinding remove = new OperationBinding();
    remove.setOperationType(DSOperationType.REMOVE);
    DSRequest removeProps = new DSRequest();
    removeProps.setHttpMethod("DELETE");
    remove.setRequestProperties(removeProps);

    // apply all the operational bindings
    setOperationBindings(fetch, add, update, remove);

    init();
}

private DataSourceIntegerField userIdField; // "userId":"1",

private DataSourceTextField usernameField; // "username":"myusername",
private DataSourceTextField passwordField; // "password":"mypassword",

private DataSourceBooleanField userActiveField; // "active":true,

private DataSourceTextField fullnameField; // "fullname":"Thomas Holmes",
private DataSourceDateField birthdateField; // "birthdate":"1960-10-30",
private DataSourceTextField emailField; // "email":"myemail@test.net",
private DataSourceTextField cellPhoneField; // "cellPhone":"111-222-1234"

private DataSourceIntegerField updatedByField; // "updatedBy":1,
private DataSourceDateField updatedDateField; // "updatedDate":"2013-01-01",
private DataSourceIntegerField createdByField; // "createdBy":1,
private DataSourceDateField createdDateField; // "createdDate":"2013-01-01",

private DataSourceTextField securityQuestion1Field; // "securityQuestion1":"peanuts",
private DataSourceTextField securityAnswer1Field; // "securityAnswer1":"linus"

protected void init()
{
    System.out.println("init: START");

    setDataFormat(DSDataFormat.JSON);
    setJsonRecordXPath("/");

    // set the values for the datasource
    userIdField = new DataSourceIntegerField(Constants.USER_ID, Constants.TITLE_USER_ID);
    userIdField.setPrimaryKey(true);
    userIdField.setCanEdit(false);

    usernameField = new DataSourceTextField(Constants.USER_USERNAME, Constants.TITLE_USER_USERNAME);
    usernameField.setCanEdit(false);

    passwordField = new DataSourceTextField(Constants.USER_PASSWORD, Constants.TITLE_USER_PASSWORD);
    passwordField.setCanEdit(false);

    userActiveField = new DataSourceBooleanField(Constants.USER_ACTIVE, Constants.TITLE_USER_ACTIVE);

    fullnameField = new DataSourceTextField(Constants.USER_FULLNAME, Constants.TITLE_USER_FULLNAME);

    birthdateField = new DataSourceDateField(Constants.USER_BIRTHDATE, Constants.TITLE_USER_BIRTHDATE);

    emailField = new DataSourceTextField(Constants.USER_EMAIL, Constants.TITLE_USER_EMAIL);

    cellPhoneField = new DataSourceTextField(Constants.USER_CELL_PHONE, Constants.TITLE_USER_CELL_PHONE);

    securityQuestion1Field =
        new DataSourceTextField(Constants.USER_SECURITY_QUESTION_1, Constants.TITLE_USER_SECURITY_QUESTION_1);
    securityAnswer1Field =
        new DataSourceTextField(Constants.USER_SECURITY_ANSWER_1, Constants.TITLE_USER_SECURITY_ANSWER_1);

    updatedByField = new DataSourceIntegerField(Constants.USER_UPDATED_BY, Constants.TITLE_USER_UPDATED_BY);
    updatedDateField = new DataSourceDateField(Constants.USER_UPDATED_DATE, Constants.TITLE_USER_UPDATED_DATE);
    createdByField = new DataSourceIntegerField(Constants.USER_CREATED_BY, Constants.TITLE_USER_CREATED_BY);
    createdDateField = new DataSourceDateField(Constants.USER_CREATED_DATE, Constants.TITLE_USER_CREATED_DATE);

    System.out.println("init: FINISH");
    setFields(userIdField, usernameField, passwordField, userActiveField, emailField, cellPhoneField,
        fullnameField, birthdateField, securityQuestion1Field, securityAnswer1Field, updatedByField,
        updatedDateField, createdByField, createdDateField);

    // setFetchDataURL(getServiceRoot() + "/userId/{id}");
    // setFetchDataURL(getServiceRoot() + "/contactId/{id}");
    setAddDataURL(getServiceRoot() + "/create");
    setUpdateDataURL(getServiceRoot() + "/update");
    setRemoveDataURL(getServiceRoot() + "/remove/{id}");
}

protected String getServiceRoot()
{
    return "rest/login/";
}

protected String getPrimaryKeyProperty()
{
    return "userId";
}

/*
 * Implementers can override this method to create a different override.
 */
@SuppressWarnings("rawtypes")
protected void postProcessTransform(DSRequest request)
{
    System.out.println("LoginDataSource: postProcessTransform: START");

    StringBuilder url = new StringBuilder(getServiceRoot());
    System.out.println("LoginDataSource: postProcessTransform: url=" + url);

    Map dataMap = request.getAttributeAsMap("data");
    System.out.println("LoginDataSource: postProcessTransform: dataMap=" + dataMap.toString());
    if (request.getOperationType() == DSOperationType.FETCH && dataMap.size() > 0)
    {
        if (dataMap.get(Constants.USER_USERNAME) != null && dataMap.get(Constants.USER_PASSWORD) != null)
        {
            url.append("user/" + dataMap.get(Constants.USER_USERNAME));
            url.append("/pwd/" + dataMap.get(Constants.USER_PASSWORD));
        }
        else if (dataMap.get(Constants.USER_USERNAME) != null && dataMap.get(Constants.USER_PASSWORD) == null)
        {
            url.append("user/" + dataMap.get(Constants.USER_USERNAME));
            url.append("/pwd/" + dataMap.get(Constants.USER_PASSWORD));
        }
        else if (dataMap.get(Constants.USER_EMAIL) != null)
        {
            url.append("email/" + dataMap.get(Constants.USER_EMAIL));
        }
    }
    System.out.println("LoginDataSource: postProcessTransform: url=" + url.toString());
    request.setActionURL(URL.encode(url.toString()));
}

@Override
protected Object transformRequest(DSRequest dsRequest)
{
    // now post process the request for our own means
    postProcessTransform(dsRequest);

    System.out.println("LoginDataSource: transformRequest: START");
    dsRequest.setContentType("application/json");
    JavaScriptObject jso = dsRequest.getData();

    String jsoText = JSON.encode(jso);
    System.out.println("LoginDataSource: transformRequest: START: jsoText=" + jsoText);

    // this code is used only when there is a password change, otherwise this will be skipped
    String userPassword = JSOHelper.getAttribute(jso, Constants.USER_NEW_PASSWORD);
    if (userPassword != null)
    {
        // This creates the new JSON attribute:
        // ... , "position":{"id":x}
        JSOHelper.setAttribute(jso, "password", userPassword);

        // remove the JSON Attribute: ... , "userPassword":"newPassword"
        JSOHelper.deleteAttribute(jso, Constants.USER_NEW_PASSWORD);
    }

    System.out.println("LoginDataSource: transformRequest: FINISH: url=" + dsRequest.getActionURL());

    String s1 = JSON.encode(jso);
    System.out.println("LoginDataSource: transformRequest: FINISH: s1=" + s1);
    return s1;
}

protected void transformResponse(DSResponse response, DSRequest request, Object jsonData)
{
    System.out.println("LoginDataSource: transformResponse: START");
    JavaScriptObject jsObj = (JavaScriptObject) jsonData;
    String jsoText1 = JSON.encode(jsObj);
    System.out.println("LoginDataSource: transformResponse: jsoText=" + jsoText1);

    System.out.println("LoginDataSource: transformResponse: jsonData=" + jsonData.getClass());

    for (String attr : response.getAttributes())
    {
        System.out.println("LoginDataSource: transformResponse: attr=" + attr + " value="
            + response.getAttribute(attr));
    }

    super.transformResponse(response, request, jsonData);
}
}

The error comes up on the line: super.transformResponse(response, request, jsonData);

I know the data coming back from the Controller is JSON data as follows:

{
"userId":1, 
"username":"my_username", 
"password":"my_password", 
"active":true, 
"fullname":"Thomas Holmes", 
"birthdate":"1960-10-13", 
"email":"test@test.net", 
"cellPhone":"111-222-1234", 
"updatedBy":1, 
"updatedDate":"2013-01-01", 
"createdBy":1, 
"createdDate":"2013-01-01", 
"securityQuestion1":"peanuts", 
"securityAnswer1":"linus"
}

I have tested that these names match the datasource fields in the json data. We should be able to see that by checking the datasource fields above. I have also unit tested with JUnit and Jackson Mapper 2.0 that the JSON string data can be used to create a UserDTO object and a UserEntity object.

I am very much aware of the SmartClient documentation about the data that comes back from a controller and how it must match the required format. This is a SmartGWT RestDataSource and I looked at the response which looks ok.

In the transformResponse code:

for (String attr : response.getAttributes())
{
System.out.println("transformResponse: attr=" + attr + " value="
+ response.getAttribute(attr));
}

Which yields:

transformResponse: jsonData=class  com.google.gwt.core.client.JavaScriptObject$
transformResponse: attr=data value=[object Object]
transformResponse: attr=startRow value=0
transformResponse: attr=status value=0
transformResponse: attr=endRow value=1
transformResponse: attr=totalRows value=1
transformResponse: attr=httpResponseCode value=200
transformResponse: attr=transactionNum value=0
transformResponse: attr=clientContext value=null
transformResponse: attr=httpHeaders value=[object Object]
transformResponse: attr=context value=[object Object]

It looks like the "Object jsonData" is a JavaScriptObject. Ultimately, the JSON that comes back, in a JavascriptObject, I'd like to convert to a UserDTO object.

So, if I can remove this error and solve this objective that would be great.

Thanks!

UPATE

I've been testing this out, and I finally have a small test app which I think shows that Isomorphic broke the RestDataSource in SmartGWT. I say that because my application was working before-hand, and now it doesn't work.

I confirmed hat all this is 100% valid JSON data. Running through various tests on the Net.

test1.json:   {"userId":1}
test2.json:   {"userId":"1"}
test3.json:   [{"userId":1}]
test4.json:   [{"userId":"1"}]

Then I isolated to a very small app to test. This is something similar to what is on the SmartGWT Showcase.

public class TestApp implements EntryPoint
{
    private DataSourceIntegerField userIdField;

    public void onModuleLoad()
    {
    RestDataSource dataSource = new RestDataSource();
    dataSource.setDataFormat(DSDataFormat.JSON);
    dataSource.setDataURL("data/single_user.json");

    // set the values for the datasource
    userIdField = new DataSourceIntegerField("userId", "User Id");
    userIdField.setPrimaryKey(true);
    userIdField.setCanEdit(false);

    dataSource.setFields(userIdField);

    ListGrid grid = new ListGrid();
    grid.setDataSource(dataSource);
    grid.setWidth100();
    grid.setHeight(150);
    grid.setAutoFetchData(true);
    grid.draw();
    }
}

In every case, I get the same error message:

[ERROR] [SoccerAdmin] - 15:20:55.945:XRP6:WARN:RestDataSource:isc_RestDataSource_0:RestDataSouce transformResponse(): JSON response text does not appear to be in standard response format.

However, if I change from a RestDataSource to a DataSource, then I don't have any issues with TransformResponse.

I guess maybe I don't know what TransformResponse is supposed to do with RestDataSource, but I did read the SmartClient Docs for what it's worth.

If I find a great workaround, then I will post an answer.

Answer 1

Ultimately I changed my LoginDataSource from extending RestDataSource to extending DataSource and all my issues went away.

Per the documentation, my response data was complete and accurate. Also, my actual data was a valid JSON object and verified that against several sites.

Not sure what he bug with SmartGWT RestDataSource is, but I feel that it is a bug ... unless they can explain to me why it isn't.

Hope this helps someone else!