Redis Geo

[antirez]

Hello, I'm currently integrating, refactoring and fixing the original code, in the context of this branch: https://github.com/antirez/redis/commits/geo

So far the code was simplified and polished, fixed (it did not returned the right set of elements in all the cases), GEORADIUS speed was improved.

On the other side I removed all the JSON things and the Pub/Sub (IMHO is interesting to subscribe to areas to see objects entering/exiting them, not much to position updates since it's the same of doing PUBLISH+GEOADD from the client side).

Added: GEOHASH to return a Geohash standard string from of a given element, mainly for interoperability. I'm also adding GEODIST (to calculate distances) and GEOPOS (to retrieve latitude/longitude of objects without the need to query for radius).

The work will be available in the context of Redis 3.2, unless I get mad and merge back into 3.0, which is unlikely but not impossible.

EDIT: before of three days ago I was not aware of Geohashing at all, so if you have five mins to check the code, I'll be glad :-) Thanks. I'm pretty confident that it's ok now after I added a fuzzy test that replicates the feature (by brute forcing distance computation) in Tcl that checks against the Redis output, but still...

EDIT2: wanted to add that I'm very glad Matt efforts are not going to be wasted and we are finally merging this.

[emehrkay]

Redis is damn cool. As an effort to better understand it and after reading through the Twitter in Redis example, I decided to see how much of a graph database I could create using it and Python (https://github.com/emehrkay/rgp)

Dynamic Redis seems like a decent project and would make hacking core functionality fun and simple.

[itamarhaber]

Loading plugins into Redis, a-la Dynamic Redis approach or otherwise, has long been discussed in the Redis community. Presently, antirez prefers not to destablize Redis (and thus damage its solid stability reputation) by allowing external code be run in it.

So, despite the obvious perks that we'll enjoy by side-loading our own junk, antirez's "Quality or DEATH" approach on this matter leaves no open questions about the possible inclusion of this functionality. There is, however, another way to get code running in Redis... it is called a GitHub Pull Request :) If you've thunk up something really amazing/useful and coded it, drop by the redis-dev mailing list to discuss it, maybe do an RCP and submit your code. No one promises that your PR will be accepted, but even if it won't make its way into the core, anyone will be able to use it nonetheless.

Another point worth mentioning in this context is that in order to even contemplate such a mechanism, Redis' core will need heavy refactoring of the internal API. This is by all accounts a huge effort that, if taken by the developer, is likely to postpone a lot of other important work.

[kenshiro_o]

It's really cool to see more projects tackling location problems. The API is very simple which is another great plus.

Had Redis Geo been around at the time we replatformed at Hailo we would have given it some serious consideration. But instead we have developed our own solution and open sourced a library in Go called "geoindex" to store and retrieve geo information: https://github.com/hailocab/go-geoindex .

It's part of our core functions within our microservices stack. You can read more about it on our blog too: https://sudo.hailoapp.com/services/2015/02/18/geoindex/

[nchelluri]

This is unrelated, but I've been hearing great things about the readability and teaching ability of the Redis code for people interested in learning good C practices.

I knew/know C but have been doing high level programming for years. Would anyone suggest if I wanted to visit this source tree, I check out an earlier version (like 1.x) or just jump straight to latest stable or master?

[joeskyyy]

The latest stable branch is an amazing show of readability. It's very simple to follow, and very easy to hack at if you want to try and manipulate structures to how you're wanting them.

10/10 would read through again.

[christo16]

I'm also curious in this if anyone has an answer.

[hoprocker]

This is awesome. I've been using Redis -- naively, with hashes -- as a lat/lon cache for some path analysis on my website. Didn't even know this was out there, and can't wait to switch over!

[itamarhaber]

Indeed - and as we speak antirez is busily refactoring, merging, testing and extending it in Redis... happy-meter is off the scales here.

[slowernet]

Follow along at home: https://github.com/antirez/redis/commits/geo

[kyleblarson]

Would love to know how this performs compared with analogous PostGIS operations on a similar sized set of geos.

[jandrewrogers]

It is a little bit apples and oranges. This is a simple point-in-plane index, PostGIS is a full-blown geospatial data modeling capability. PostGIS is going to be much slower but it is also doing much more as part of the process so that it is useful for sophisticated spatial applications. PostGIS is not fast relative to comparable systems but this particular comparison is not fair because PostGIS is expected to do so much more.

That said, I have also implemented large-scale point-in-plane indexing systems, though not in a while. Per CPU core, you can encode and index around 20M geo-points per second if the implementation is good.

[jerrysievert]

i can only ask that this extension requires longitude and latitude in the correct order: x,y.

[antirez]

Somebody opened an issue about this too. While in the GIS world apparently this is the standard, outside the GIS world everybody talks in terms of latitude,longitude (y,x). So I'm not sure what to do. I suspect the majority of the users were not exposed to GIS systems, and that the GIS exposed users will check the parameters better, so I'm inclining in taking y,x in favor of the average person that like me thinks latitude,longitude is normal.

[Wouter33]

Dynamic Redis (which is a Redis Fork that lets you add modules to it) is a great find! Especially with the Geo module added.

I needed a limit parameter for the Georadius method because i wanted a result set sorted by distance in a large radius, but not have thousands of results send from the redis server to my webservers. Anyone interested in such a parameter can use my little fork for the Geo module:

https://github.com/wouter33/krmt

[antirez]

Good idea, I'll add LIMIT. Makes sense to say I want N nearest elements with max distance of Y.

[urlgrey]

I've got a Docker image for Dynamic Redis that includes all of the standard modules, including geo.so: https://registry.hub.docker.com/u/urlgrey/dynamic-redis/

Github repo: https://github.com/urlgrey/docker-dynamic-redis

[krisdol]

We've been using this since day 1 at our startup (well, at this point it's more like early-stage free-time group project). Very happy to see support from antirez

[ppymou]

I never knew that redis can be extended for geo. This is really cool.

I researched geodb a while back and I only found PostGIS but I recently found out that elastic search also supports geo features.

Shameless plug - This is the simple API I built on top of PostGIS https://github.com/moo-mou/GeoGo

[kposehn]

Very interesting. We could have used this ourselves, but ended up writing a way (circa 2010) for Postgres to be queried by MGRS (Military Grid Reference System: https://en.wikipedia.org/wiki/Military_grid_reference_system)

[cg25]

I wrote an introductory article to the new Geo API. Comments are welcome :) http://cristian.regolo.cc/2015/07/07/introducing-the-geo-api...

[arturventura]

I have a question that I haven't seen address in the article. Does the code always assume the "sphere" is Earth or can you provide geometric information for other geometries?

[antirez]

It assumes the Earth is a sphere, also the code is not designed to work well towards the poles, at latitudes less or more than -/+ 85.

[har777]

Hell yeah ! I've been waiting for this to be integrated into Redis.

[sidarape]

Some can ELI5 redis for me? And also this extension?

[doppel]

Redis is an in-memory "data structure server", in that besides simple key-value (think memcached) storage you can also have sets, lists, sorted sets, hashes and more. See redis.io.

This extension adds geo-specific commands and storage to the list of operations supported.