SQLite Release 3.8.11.1

[harperlee]

SQLite should be the default way someone manages information on a binary data file, as opposed to just inventing your format from scratch.

I remember on uni when we started defining our own formats, to later on learn about what goes inside a DBMS, and thinking, "Why on earth would I ever do the first thing again?" (not that there's no space for it, but it's not so big).

Also, when I've found a couple of times that a file that I thought had a proprietary binary format was a SQLite file, it was bliss. So much easier to navigate, inspect and work with that a huge XML file or a plain text one.

[batbomb]

I think this is somewhat incomplete information, and I say this as a SQLite lover. It depends on your type of data, but both HDF5 and SQLite are more than adequate formats for data, and you should be using one of them.

I think HDF5 is actually a better format for long term data management, but I think SQLite is a easier to use in everyday life. There's also Parquet and Avro for these as well.

High-Energy physics also has the "code is the documentation" clusterfuck known as ROOT files.

[Ygor]

Great part from the docs:

"SQLite does not compete with client/server databases. SQLite competes with fopen()."

[dr_zoidberg]

It is indeed one of their use cases: https://www.sqlite.org/whentouse.html

[protomyth]

I agree with that for a lot of things, but I would trade SQLite in a heartbeat for a Newton Soup[1] implementation.

1) https://en.wikipedia.org/wiki/Soup_(Apple)

[batou]

Pretty much everything I've built that is standalone uses exactly this model. It's a wonderful bit of tech!

[shocks]

Relevant change log entry:

> Miscellaneous micro-optimizations result in 22.3% more work for the same number of CPU cycles relative to the previous release. SQLite now runs twice as fast as version 3.8.0 and three times as fast as version 3.3.9. (Measured using cachegrind on the speedtest1.c workload on Ubuntu 14.04 x64 with gcc 4.8.2 and -Os. Your performance may vary.)

[eblume]

SQLite is amazing. It is a tech that no programmer should be unfamiliar with. It takes at most a day or two of playing with it to learn all of its various quirks and tricks, is easy to bake in to any project regardless of language, and performs like mad.

[daheza]

Any particular resources you recommend to learn it?

[gtrubetskoy]

When I was tinkering with http://thredis.org I spent a lot of time looking into SQLite internals, and it is by far the most polished, comment-rich code base that I have ever seen, it reads like a fun book. Especially if you're interested in low-lever relational db stuff - how SQL statements are processed, data is stored, B-Trees, etc. It's better than any book out there (and such books tend to be very expensive).

[coleifer]

I wrote a post a while back singing the praises of SQLite. Some of you may be interested:

http://charlesleifer.com/blog/sqlite-small-fast-reliable-cho...

[joshbaptiste]

Great, write up convinced me to use Sqlite instead of Mysql for a simple in-house webpage.

[applecore]

I'd like to see the same speed measurements done for mobile. A 64-bit Linux desktop isn't representative of the #1 use case for SQLite as a mobile database replacement for Core Data.

[DrJokepu]

I've never worked with Core Data, but I assume it always uses the SQLite version that ships with iOS (as opposed to a version you ship with your app), so that means that you wouldn't see these improvements on iOS for at least a year or so anyway, or am I wrong about this?

[toyg]

sqlite existed before iOS and I doubt that platform is its "#1 use case". Sqlite powers tons of apps on many different platforms.

[developernotes]

>I'd like to see the same speed measurements done for mobile.

While not exactly what you are looking for I can understand the sentiment. SQLCipher provides a project to time the performance of queries when run against standard SQLite vs. SQLCipher on iOS.

https://github.com/sqlcipher/SQLCipherSpeed

[thomaslutz]

Why should it not be similar faster on mobile?

[PythonicAlpha]

SQLite is one of the rare examples of very elaborate implemented software that is very good maintained, too.

One of the best pieces of free software around.

Where else do you find software, that was very good implemented from the beginning and is so carefully enhanced and optimized? (And not only for new "features") -- I remember no official available example.

[arandomdude]

And SQLite is also one of the most tested pieces of software in the world: https://www.sqlite.org/testing.html

[wxs]

We use SQLite extensively in Minuum and I couldn't be happier. It's in all our critical real-time code paths that bring up candidates while you're typing. Because we're on SQLite our predictions keep getting faster for free!

Faster predictions mean we can handle more types of errors (more missed characters, more complex typos) in the same time budget. Thanks SQLite, you guys are awesome.

[billK]

The change in title hides the main point (~2x speed increase) and makes it appear as a simple bugfix release.

[jmnicolas]

You would think with such a performances boost they would have put it first in the change-log, but no it's hidden at the end.

One thing for sure, they never worked for AMD or Nvidia marketing department ;-)

[NelsonMinar]

One of the great things about SQLite is its incredibly comprehensive automated test suite. That allows aggressive optimization without fear you're breaking something.

[shredwheat]

I still remember when Firefox switched to using SQLite for most of its internal data. It felt like it took 3 or 4 releases to work out the concurrency and blocking issues. I remember back in the day it had lots to do with fsync.

Has most of that been worked out between SQLite and newer Linux kernels, or is that a pit waiting for other applications that go the same route?

[coleifer]

I wrote up a quick blog post describing how to build the new FTS5 search extension, which was included in the 3.8.11.1 release:

http://charlesleifer.com/blog/building-the-sqlite-fts5-searc...

[fiveoak]

A little off topic, but is there any benchmarking software to compare embedded databases? For instance SQLite vs H2.

[coleifer]

Anybody heard anything about SQLite4?