Using/Searching AsyncDataProvider with Objectify / Google App Engine

Question

I currently have an application which uses the activities/places and an AsyncDataProvider.

Right now, everytime the activity loads up - it uses the request factory to retrieve the data (currently not a lot but will get very large coming up here soon) and passes it to the View to update the DataGrid. Before it is updated it is filtered based on a search box.

Right now - I have implemented updating the DataGrid as follows: (this code isn't the prettiest)

private void updateData() {
  final AsyncDataProvider<EquipmentTypeProxy> provider = new AsyncDataProvider<EquipmentTypeProxy>() {

    @Override
    protected void onRangeChanged(HasData<EquipmentTypeProxy> display) {
      int start = display.getVisibleRange().getStart();
      int end = start + display.getVisibleRange().getLength();
      final List<EquipmentTypeProxy> subList = getSubList(start, end);
      end = (end >= subList.size()) ? subList.size() : end;
      if (subList.size() < DATAGRID_PAGE_SIZE) {
        updateRowCount(subList.size(), true);
      } else {
        updateRowCount(data.size(), true);
      }
      updateRowData(start, subList);
    }

    private List<EquipmentTypeProxy> getSubList(int start, int end) {
      final List<EquipmentTypeProxy> filteredEquipment;
      if (searchString == null || searchString.equals("")) {
        if (data.isEmpty() == false && data.size() > (end - start)) {
          filteredEquipment = data.subList(start, end);
        } else {
          filteredEquipment = data;
        }
      } else {
        filteredEquipment = new ArrayList<EquipmentTypeProxy>();
        for (final EquipmentTypeProxy equipmentType : data) {
          if (equipmentType.getName().contains(searchString)) {
            filteredEquipment.add(equipmentType);
          }
        }
      }
      return filteredEquipment;
    }
  };
  provider.addDataDisplay(dataGrid);
} 

Ultimately - what I would like to do is only load up the necessary data at first (the default page size in this application is 25).

Unfortunately, to my current understanding, with Google App Engine there is no order to any of the Id's (one entry has an ID of 3 the next has an entry of 4203).

What I'm wondering, what is the best way to go about retrieving a subset of data from Google App Engine when using Objectify?

I was looking into using Offset and limit but another stack overflow post (http://stackoverflow.com/questions/9726232/achieve-good-paging-using-objectify) basically said this is inefficient.

The best information I've found is the following link (http://stackoverflow.com/questions/7027202/objectify-paging-with-cursors). The answer here says to use Cursors but also says this is inefficient. I'm also using Request Factory so I will have to store the Cursor in my user Session (if that is incorrect please let me know).

Currently since there isn't likely to be a lot of data (maybe 200 rows total for the next few months) I am just pulling back the entire set to the client as a temporary hack - I know this is the worst way to do it but would like to get input to the best way to do it before wasting my time implementing another hack solution. I am worried currently as it seems every single post i've read on doing this makes it seem like there's not really a solid way to do this.

What i am also thinking about doing - currently my searching / page loading is lightning fast because all the data is already on the client side. I use a KeyUpEvent handler in the search box to filter the data - i don't think there is any way to keep this speed by making a call to the server - is there any accepted solution to this problem?

Thank you very much

Answer 1

Go with Cursors. They are as efficient as it gets - cursor stores the point where last query ended and continues from there. The answer you linked actually does not discuss efficiency of cursors vs offset. (there is a comment that is wrong)

You can use limit with Cursors - it does not affect efficiency.

Also, Cursors can be serialized via cursor.toWebSafeString() and sent to client via RPC. This way you do not need to save them in session. Actually you can also use them as fragment identifier (aka history token in GWT parlance) - this way a certain "page" of your result set can be bookmarked.

(Offset is "inefficient" because it actually loads, and charges you, for all entities upto offset+limit, bit it only returns limit entities)

OTOH, if you already know the query parameters when the page is loaded, then just do the query at page generation time, instead invoking it via RPC. Also, if you have a small set of data (<1000) you could just preload all entity IDs s part of page html.