Python - Best database for exponentially growing database? [closed]

Question

Closed. This question is off-topic. It is not currently accepting answers.

---

Want to improve this question? Update the question so it's on-topic for Database Administrators Stack Exchange.

Closed 1 year ago.

Improve this question

So I am making a Discord (social media) Bot in Python using discord.py-rewrite, and I need to save informations of lots of servers and users (such as how much money they have, what items thay have…). This can grow exponentially. My question was, what is the best database to use then (sqlite3, PostGreSQL/asyncpg, MySQL…)? I currently use sqlite3, but I heard it wasn't good enough for exponentially growing databases.

Answer 1

Based on your comment of only storing a few thousand rows, you should be fine with almost any database product. Most relational databases only start to have trouble when they're many orders of magnitude bigger, like TB's of data.

Keep in mind some free offerings, like SQL Server Express Edition do have size limits, but as you have outlined, you won't hit those. SQL Server Express Edition has a 10GB max data size, LINK: https://www.microsoft.com/en-us/sql-server/sql-server-2019-comparison#CP_StickyNav_1.

Found the limits on your chosen platform, https://www.sqlite.org/limits.html

Maximum Database Size

• Every database consists of one or more "pages". Within a single database, every page is the same size, but different database can have page sizes that are powers of two between 512 and 65536, inclusive. The maximum size of a database file is 2147483646 pages. At the maximum page size of 65536 bytes, this translates into a maximum database size of approximately 1.4e+14 bytes (140 terabytes, or 128 tebibytes, or 140,000 gigabytes or 128,000 gibibytes).
• This particular upper bound is untested since the developers do not have access to hardware capable of reaching this limit. However, tests do verify that SQLite behaves correctly and sanely when a database reaches the maximum file size of the underlying filesystem (which is usually much less than the maximum theoretical database size) and when a database is unable to grow due to disk space exhaustion.

You should be just fine in terms of size. The stuff about dynamic data typing is certainly interesting, and something you should be aware of.

Answer 2

You've got to define what best means for you.

The best database product is Oracle or SQL Server, depending on who you ask. These have licensing costs, and you're paying for features that are pretty awesome - if you use them.

If you can't name any features of Oracle or SQL Server that you think are worth paying for, then I'd recommend going with an unlicensed product, such as Postgres.