Google App Engine Database Index

Question

I need to store a undirected graph in a Google App Engine database. For optimization purposes, I am thinking to use database indexes. Using Google App Engine, is there any way to define the columns of a database table to create its index?

I will need some optimization, since my app uses this stored undirected graph on a content-based filtering for item recommendation. Also, the recommender algorithm updates the weights of some graph's edges.

If it is not possible to use database indexes, please suggest another method to reduce query time for the graph table. I believe my algorithm does more data retrieval operations from graph table than write operations.

PS: I am using Python.

Answer 1

Perhaps this will help: http://code.google.com/intl/sv-SE/appengine/docs/python/datastore/queriesandindexes.html#Defining_Indexes_With_Configuration

Answer 2

are you actually seeing prohibitively slow queries? i'm guessing not. i suspect this is somewhat premature optimization. the app engine datastore doesn't do any sorting, filtering, joins, or other meaningful operations in memory, so query times are generally fairly constant. in particular, query latency does not depend on the number of entities of your datastore, or even the number of entities that match your query. it only depends on the number of results you ask for.

on a related note, adding indexes to your datastore will not speed up existing queries. if a query needs a custom index, it won't degrade and run slower without it. the query simply won't run at all until you add the index.

for the specific query you mention, select * from edges where vertex1 == x and vertex2 == y, the datastore can run it without a custom index at all. see this section of the docs for more details.

in short, just run the queries you need, and don't think too much about indices or try to optimize as if you were a DBA. it's not a relational database. :P