Django Activity Feed (Feedly Integration?)

Question

I have built a simple Django photo app. Users can upload photos, follow other users and like photos. To handle relationships amongst the users (following & unfollowing) I use a package called django-relationships by coleifer. It's a great package and very simple to use.

Everything works as it should. I currently have a working activity feed.

I filter the feed into two sections: Following (all the activities from the users that I follow) & You (all the activities that happened to me). I've posted two pictures below from my iOS app that uses my Django photo app as it's back-end:

What I would like to do is add aggregation to the Following Feed. As you can see, user alexperri has liked 5 shots. I would like to aggregate all these items into one line. I don't need to add aggregation for the "You" feed since I would like to see each individual action happening to me. However for the Following feed, it makes sense to add aggregation. There are several applications that do aggregation very well. Fashionlista, Pinterest & Instagram do this well. Here is an example from Instagram to show what I am trying to achieve:

In the example above, you can see the following feed and that lovetoronto liked 5 photos. I started to play around with the Instagram following feed to see how it works. The Instagram following feed shows a maximum of 35 activity entries and each entry can have a maximum of 5 activities of that action type. "lovetoronto liked 5 photos" is one activity entry and it shows the latest 5 pictures that he liked. Since lovetoronto performed the latest action, he is at the top.

I would like to achieve the same setup.

Here is my current model setup:

models.py

from django.db import models
from django.contrib.auth.models import User

class Photographer(models.Model):
    user = models.OneToOneField(User, primary_key=True
    likes = models.ManyToManyField('Photo', through = 'Likes', 
                                   related_name = 'likedby', blank = True)

class Photo(models.Model):
    photographer = models.ForeignKey(Photographer, related_name = 'shot_owner')
    created = models.DateTimeField(auto_now_add=True)
    url = models.CharField(max_length=128)

class Likes(models.Model):
    liked_at = models.DateTimeField(auto_now_add=True, blank=True, null=True)
    photographer = models.ForeignKey(Photographer, related_name = 'liked_by')
    photo = models.ForeignKey(Photo, null=True)

class Activity(models.Model):
    actor = models.ForeignKey(Photographer, related_name = 'actor')
    receiver = models.ForeignKey(Photographer, related_name = 'receiver')
    action = models.CharField(max_length=12)
    post = models.ForeignKey(Photo, null=True, blank=True)
    time = models.DateTimeField(auto_now_add=True)

Every time a 'Like' object is created, I create an Activity object as well, the actor being the person who did the action, the receiver being the person who the action was done to, the action (in this case a string, 'liked'), post (the photo) and the time being the creation of the activity object.

I use django-tastypie to get and create 'Like' & 'Activity' objects.

api.py

from tastypie.resources import ModelResource, ALL, ALL_WITH_RELATIONS
from tastypie.authentication import BasicAuthentication
from tastypie.authorization import DjangoAuthorization, Authorization
from photoapp.photodb.models import *
from tastypie.serializers import Serializer
from relationships.utils import positive_filter
from relationships.models import Relationship
from relationships.models import RelationshipStatus

class LikeResource(ModelResource):
    user = fields.ForeignKey(BasicUserResource, 'user', full=True)
    class Meta:
        queryset = Photographer.objects.all()
        allowed_methods = ['put']
        resource_name = 'like'
        fields = ['user']
        default_format = 'application/json'
        authorization = Authorization()
        authentication = BasicAuthentication()
        serializer = Serializer(formats=['json'])
        always_return_data = True
        include_resource_uri = False

        def hydrate(self, bundle):
            shot = Photo.objects.all().get(id = bundle.data['photo id'])
            user = Photographer.objects.all().get(user = bundle.request.user)
            if(bundle.obj.likes.filter(id = bundle.data['photo id']).exists()):
                Likes.objects.all().filter(photographer=user).filter(photo=shot).delete()

                Activity.objects.filter(actor__user = bundle.request.user,
                    post = shot, action = 'liked').delete()

            else:
                like = Likes(photographer = user, photo=shot)
                like.save()
                user_doing_the_liking = User.objects.get(
                    username=bundle.request.user.username)
                user = Photographer.objects.all().get(user = bundle.request.user)
                user_getting_liked = shot.photographer.user
                photographer_getting_liked = shot.photographer
                newActivity = Activity()
                newActivity.actor = user
                newActivity.receiver = photographer_getting_liked
                newActivity.action = 'liked'
                newActivity.post = shot
                newActivity.save()

    return bundle 

class FollowingFeed(ModelResource):
    actor = fields.ForeignKey(BasicPhotographerResource, 'actor', full=True)
    receiver = fields.ForeignKey(BasicPhotographerResource, 'receiver', full=True)
    post = fields.ForeignKey(BasicPostResource, attribute = 'post', full=True, null=True)
    class Meta:
        queryset = Activity.objects.all()
        allowed_methods = ['get']
        resource_name = 'following-feed'
        fields = ['actor', 'receiver', 'action', 'post', 'id', 'time']
        default_format = "application/json"
        authorization = Authorization()
        authentication = BasicAuthentication()
        serializer = Serializer(formats=['json'])
        always_return_data = True
        include_resource_uri = False

    def get_object_list(self, request):
        return super(FollowingFeed, self).get_object_list(request)\
            .filter(actor__user__in = request.user.relationships.following())\
            .exclude(receiver__user = request.user)\
            .exclude(actor__user = request.user).order_by('-time') 

How can I modify the FollowingFeed resource in such a way that it will aggregate the activity objects? I came across the Feedly project. How can I use it with my current setup?

Answer 1

I don't think you want to do any database level aggregation, because you presumably want to show the individual details as well as the counts e.g. "X like 5 photos" and show the 5 photos. Aggregation, by definition, will exclude the individual data.

Instead, you should do the grouping and sorting in Python code (or Javascript since I think you're using an HTTP API, but I would prefer a server side API that had things already organised).

itertools.groupby might help. I think you need to group by (user and action), and then sort by the timestamp of the first item in each group, so that you might see "Joe liked 5 photos", "Anne posted 2 photos", "Joe posted a photo", "Claire liked 3 photos" etc.

Answer 2

In your feed resource, you are already overriding get_object_list, I would suggest to change the logic to perform a raw query for aggregation logic.

def get_object_list(self, request):
        query = "Your aggregation query logic here"
        feed_model = self._meta.object_class
        return feed_model.objects.raw(query)

This should do what is required. However, you need to think of your query logic. Let me know if you face any other problem.

Thanks!

Answer 3

I think the best way to do this would be to modify the Activity table to store grouped activities. When a new action happens, check for existing ones of the same type and either edit the record to make it 'grouped', or add a new record. You could add ManyToMany relationships to all potential table that contain related records, or just store the data in a json field that contains enough information to render the activity in the feed without making queries to other table objects.

If it's too resource intensive, you can queue the adding/editing of new activities. You're probably better off keeping the activity table just a straight feed that doesn't require any processing when rendered. It's not the easiest solution to implement, but I think that in the long run it makes sense.

Answer 4

Set the aggregation time, for example do you want to aggregate all “likes” within a 10 minute period or for the past 24 hours?

Then you can filter your object by this time frame.

Then you can apply grouping using the .values(‘model__field’) method. Django generates sql that contains ‘GROUP BY'

And then finally add an aggregation limit so that when the number of likes is over this limit you display the aggregated view rather than the single activity view.

Example below (pseudo, not actual):

if (activity.object.filter(time__gt=yourlowertime, time__lt=youruppertime).values(‘activity__action').count() > aggregation_limit) :
     # show aggregated view
else:
     # show normal view