Theo's Typesafe Cult•2y ago

post system design question

I am aiming to build a scalable infrastructure (for fun) for a user posting system where timelines exist, such as a feed system. Feeds would include posts of a users followings. Anyways, the current approach is this. 1. UserA follows UserB 2. Save the follows relationship in the DB 3. Create a redis list <userBiD-followers> -> [userAid, ...] 4. UserB creates a post 5. Save that post into the DB 6. Push the <postId> into their Redis list <userId-posts> -> [postId1, postId2, postId3, ...] Now since UserA has a feed redis list <userAid-feed> -> [postId1, postId2, ...] 7. I essentially iterate over UserB followings, and push UserBs new post into all of their timelines. (fan-out) Is this the right approach? Anything better I can do? Another question of mine is, is it much faster to retrieve a redis list of postIds, and pass them into Prisma to get the actual posts than doing queries with just Prisma? Example below.

const redisUserPosts = redis.lrange(`${userId}-posts`, 0, -1) = ['1', '2', ...]

posts.findMany({
 where: {
  id: {
    in: redisUserPosts
  }
 }
})

const redisUserPosts = redis.lrange(`${userId}-posts`, 0, -1) = ['1', '2', ...]

posts.findMany({
 where: {
  id: {
    in: redisUserPosts
  }
 }
})

The alternative would be to find by user, do potential other joins, etc, rather than knowing the exact postId which belongs to a user.

2 Replies

Scot•2y ago

Have a think for how this would perform If one user had 100million followers

l•2y ago

There would need to be a limit for this implementation in terms of # of followers. If its a celebrity you'd have to do it differently

Gaming

Programming

post system design question