Hacker News Re-Imagined

Ask HN: How do I efficiently track record count for big (distributed) DBs?

I've run in to this interesting question multiple times mostly during systems design interviews, but I'm never quite sure what's the "best" way to address this since I've not encountered it in any of my real world projects, unfortunately. This is usually in the context of high demand items such as limited time promotional vouchers, concert tickets, limited stock items, etc. that customers will for sure be rushing to buy, and will sell out in a relatively short period of time.

My naive solution to this is to basically count the number of reservations made (basically, do something like an SQL count of all the rows) and subtract it from some constant limit, which can be safely copied across multiple instances of the reservation/ticketing service. However, for millions of rows, this may be potentially slow and having to do this for thousands of requests per second could be fairly inefficient. Adding some caching in front of a (probably sharded/distributed) database might help, but due to the numbers being updated very quickly, I'm not sure the cache will help that much.

One alternative I can think of is maybe setup a lightweight(?) store (something like Redis perhaps, though I don't have first hand experience with it) to keep track of the count. This will introduce problems related to eventual consistency, but I think it could work. This can be durable store, or we can make it semi-durable and just do a count query if we ever need to restore the service. I always ask if it's possible for us to handle overbooking because there's a chance this could occur. Strong consistency guarantees will prevent this, but at the cost of slower performance.

Another alternative I can think of is adding a sequence number to the reservation records, and query the max instead. I'm not sure if this performs better than count or they're pretty similar in performance.

Any insights or pointers on this?

  • 1 points
  • 12 days ago

  • @extra_rice
  • Created a post

Ask HN: How do I efficiently track record count for big (distributed) DBs?


About Us

site design / logo © 2022 Box Piper