Gwt performance issue : huge data and celltable

Question

I have a server-side service that sends me huge DTOs. I need to put them in a CellTable. It's like 10-200 lines, and i need to see everything at the same time.

I have a server-side log that traces the last "man-made" line of code of my service (just before the return).

Best case scenario, there's a 2 min gap between this log and the table fully loaded. Most of the time, something breaks along the process, and it leaves me hanging.

What are my options ?

Thanks

Edit : I tried the idea of Andrei, differentiating the lenght of the data download (rpc callback) and the lenght of the celltable loading.

I wanted to be sure so here's what i did in my callback :

@Override
public void onSuccess(ResponseBean result) {
  Window.alert("success " + new Date().toString());

  resultPanel.updateResults(result);

  Window.alert("celltable " + new Date().toString());
}

The first alert occurs after ~10sec. But the second one is never called.

This is the method behind updateResults :

public void updateResults(ResponseBean response) {
  dataProvider.getList().clear();
  dataProvider.getList().addAll(response.beans());
  myCellTable.setPageSize(response.beans.size());
}

EDIT 2 : I tried a number of things. The problem occurs when i try to fill the celltable with my data. In Chrome, the chrome version of the BSOD appears. In Firefox, it breaks silently, but the remaining js instructions are not executed. If i try to read the console in firebug, my firefox freezes using ~2.5GB ram. If i use the gwt-runner, it freezes also.

In my example, there are only 4 rows (4 objects with multiple dependancies) !

How can i find WHAT breaks and WHERE ?

Thanks for helping :)

FINALLY : Ok, so i'm apparently a moron. Somewhere in the getValues() there was a piece of "business logic" that should not have been there, and that caused OutOfMemory errors.

The takeways though :

• Trace the last call server-side and the first line of the onSuccess client-side to see if the problem is RPC related
• If your celltable breaks but there are only a few lines, you did something you won't be proud of
• Scheduler.get().scheduleDeffered allows you to render the page, then do your business logic. So that's cool.

Thanks guys

Answer 1

Normally I would bet my money on the RPC serialization being the bottleneck rather than the CellTable rendering.
However you are saying that the first alert appears after 10 sec which would speak against that.

Rendering 200 DTO's should be fast enough with a CellTable.
Also sending 200 DTO's over the wire shouldn't be no problem.

I have used CellTable and sent around 5000 DTO's over the wire without any performance issue (though I only display 50 at a time and use paging).

A couple of thoughts:

• Dev Mode is much slower than production mode (especially regarding de-serialization). So test performance also in Production mode.
• To find out what is exactly causing your problem, use Chrome Dev Tools. You can profile your code and see what is causing delays.
• Can you create 200 Dummy DTO's manually in your code and pass it to your updateResults function and see how the performance compares.
• How many columns do you display? If you display many columns, reduce it to only one and see how it affects performance.
• Does any getter which is used in a CellTable column have expensive business logic?

Answer 2

You should use a pager and AsyncDataProvider for that much data. This is a template on how we do this. It may not be complete for your needs but it will give you a starting point. The DatastoreObjectProxy is extended from EntityProxy and uses request factory to fetch the items. You would add your table to this Data provider and the pager will handle fetching chunks of your data.

public abstract class DaoBaseDataProvider<T extends DatastoreObjectProxy> extends AsyncDataProvider<T> implements SearchProvider<T> {

    public static final ProvidesKey<DatastoreObjectProxy> KeyProvider = new ProvidesKey<DatastoreObjectProxy>() {

        @Override
        public Long getKey(DatastoreObjectProxy item) {
            return item.getId();
        }
    };

    public interface HasBaseDataProvider<T extends DatastoreObjectProxy> {
        public void setDataProvider(DaoBaseDataProvider<T> dataProvider);

        public void setSelectionModel(SelectionModel<? super T> selectionModel);
    }

    public DaoBaseDataProvider() {
        super(new ProvidesKey<T>() {
            public Long getKey(T item) {
                // Always do a null check.
                return (item == null) ? null : item.getId();
            }
        });
    }

    @Override
    public void search(String searchText) {

        getSearchQuery(searchText).fire(new Receiver<List<T>>() {

            @Override
            public void onSuccess(List<T> values) {
                updateRowData(0, values);
                updateRowCount(values.size(), true);
            }
        });
    }

    public void showAll() {
        updateRowCount(10, false);
    }

    @Override
    protected void onRangeChanged(HasData<T> display) {
        final Range range = display.getVisibleRange();

        Request<List<T>> request = getRangeQuery(range);
        if (getWithProperties() != null) {
            request.with(getWithProperties());
        }

        request.fire(new Receiver<List<T>>() {

            @Override
            public void onSuccess(List<T> values) {
                updateRowData(range.getStart(), values);
            }
        });
    }

    public abstract String[] getWithProperties();

    public abstract DaoRequest<T> getRequestProvider();

    public void remove(final Set<Long> idsToDelete) {
        getDeleteQuery(idsToDelete).fire(new Receiver<Integer>() {

            @Override
            public void onSuccess(Integer response) {
                for (HasData<T> display : getDataDisplays()) {
                    // Force the display to relaod it's view.
                    display.setVisibleRangeAndClearData(display.getVisibleRange(), true);
                }
            }
        });
    }

    /**
     * This can be overridden so that you can change the query.
     * 
     * @param range
     * @return
     */
    protected Request<List<T>> getRangeQuery(Range range) {
        Request<List<T>> request = getRequestProvider().findRange(range.getStart(), range.getLength());
        if (getWithProperties() != null) {
            request.with(getWithProperties());
        }
        return request;
    }

    protected Request<List<T>> getSearchQuery(String searchText) {
        Request<List<T>> request = getRequestProvider().search(searchText).with(getWithProperties());
        if (getWithProperties() != null) {
            request.with(getWithProperties());
        }
        return request;

    }

    protected Request<Integer> getDeleteQuery(Set<Long> idsToDelete) {
        return getRequestProvider().deleteList(idsToDelete);
    }
}

This is the client side class used to fetch the objects. You would extend this with your own RequestContext.

public interface DaoRequestFactory extends RequestFactory {

    @Service(value = DaoBase.class, locator = DaoServiceLocator.class)
    public interface DaoRequest<T extends DatastoreObjectProxy> extends RequestContext {

        Request<T> find(Long id);

        Request<List<T>> findRange(int start, int end);

        Request<Integer> deleteList(Set<Long> idsToDelete);

        Request<List<T>> search(String searchText);

        Request<Long> save(T obj);

        Request<Void> delete(Long objId);
    }
}