Two different users with different accounts follow same person on twitter and DB stores same feeds twice. How to avoid?

Question

I have a web application that pulls twitter feeds for user accounts. When two people follow the same person, the tweets are storing twice in the DB. I want to store feeds only show this to two users. What is the solution? The web app has PHP and MySQL

Answer 1

A solution would be to have two tables: One which stores tweets, and a join table which links a user and the tweets they have on their feed.

To break down the situation, you want one user to have access to many tweets, and one tweet to be seen by many users. This is known as a many-to-many relationship in relational terms, and a typical solution is to use a join table to connect the two entities (in this case, Users and Tweets)

The tweets table can have an index to store the tweet's official ID, and then you can check against it to test if it already exists in the database.

Very primitive example:

Table user
user_id

Table user_tweet_stream
user_id
tweet_id

Table tweets
tweet_id
twitter_id
tweet_created
tweet

This way, one user can be linked to having multiple tweets on their stream, and a tweet can be linked to be seen by multiple users.

However, there is an argument against this. Essentially, if, for the most part, your users are not going to have overlapping tweets, you're still going to be adding one row in two tables per one tweet received.

Essentially, it's great if overlapping is going to occur a lot, and it generally makes sense if you want to preserve your database integrity. However, if database size is a consideration, then you may want to look into another solution.