top | item 30348041

(no title)

mboerwink | 4 years ago

I actually recently delivered a project which uses almost this exact technique. Python lambdas that operate on SQLite files have the benefit of being much simpler and cheaper than most other scalable database solutions (like Aurora) for very light loads.

That said, accessing SQLite databases is surprisingly disk IO heavy. I haven't gone too deeply into measuring the effect, but it seems the core issue is that traditional RDBS wire protocols are better than SQLite's disk accesses wrapped over NFS (or whatever connection the lambda/EFS join is). Stuff especially starts to break down when you need any sort of concurrent access. The small overhead for locking/unlocking files can quickly become awful when multiplied by the EFS latency, so you really do need extremely fine sharding.

discuss

order

No comments yet.