Implement “following” feature efficiently on GAE

Question

I'm implementing an iPhone app with the "following" feature on Java/GAE. This feature is similar to Instagram's in that a user can follow other users' events. Events can be new photos posted, liking photos, following other users, etc.

Let say we have two entities: Follow and Event, both of which have a reference to the user id.

So I'm thinking about using OR queries where we can list events of target users. But GAE does not support OR queries, and issuing separate queries would be also inefficient as the number of queries can be quite large (e.g. 100+ follows is a norm).

What is a common practice for doing this?

Answer 1

You have to stop thinking in the relational model ;-) We do something similar on our app. Keep a List of UIDs that are "subscribed" to the user who is being followed in the followed user's entity. When the user does something that requires notification create an event entity for each user who is following the user (yes, a popular user could create a whole bunch of entries, this could be offloaded to a Task Queue).

When a user checks for notifications you can fetch them by his UID. Since this is indexed it will return very quickly. When you fetch the events, delete them or mark them read.

Remember, joins and other filtered queries are very slow, but fetching massive numbers of indexed records is not. If something is going to create a large number of records, offload it to a Task Queue.

Answer 2

I'd use a ListProperty for both subscriptions and subscribers. Every time a subscribed user posts, you use a task to add entries for every subscriber to see. Like Rick Mangi pointed out, the relational model won't help you with the datastore. If you are using the new NDB API, you can use a repeated property instead. Since you are creating entities for every observer, the asynchronous API may be handy.

Answer 3

We are implementing the model Etsy has described on their public slide share.

Here is the schema diagram.

Highly recommend reading their presentation in its entirety.

Etsy Activity Feed Arch