Getting Started on Geospatial Analysis with Python, GeoJSON and GeoPandas

[anc84]

Using geojson.io to visualise the GeoJSON seems like the wrong tool for the job; just use folium, it works perfectly in Jupyter Notebooks: https://github.com/python-visualization/folium

[maurits]

Thank you, just what I was looking for.

[Waterluvian]

I would also be sure to be aware of GDAL and OGR. Both are quite useful when you're doing anything GIS/Remote Sensing.

http://www.gdal.org/

[dagw]

If you're working with python it's also worth looking at rasterio and fiona which provide some nice wrappers around a subset of gdal and ogr.

[pb060]

Why isn't GRASS GIS more widely used in geospatial analysis (or it's just my impression that is not)? I tried several tools but usually GRASS proves to be the fastest and more flexible one.

[dagw]

Because GRASS doesn't play nice others. Calling GRASS from other apps isn't as easy as one might like and GRASS has a very GRASS-specific way it want to do things. Which is a shame because most of GRASS's algorithms are really best in class.

[aw3c2]

A GUI from the 90s, documentation often indecipherable if you are not a subject expert, awkward UX.

I like using it via QGIS but that's as near I can make myself go near it. It saddens me because it seems very powerful and featureful.

[Boothroid]

The trend in GIS these days is away from writing code and towards configurable apps, only dropping into code where necessary. Why not just use QGIS instead? Far simpler.

[sarcher]

Perhaps our experiences are different, I've noticed the opposite. There will always be a strong role for GIS applications, but I rarely see geospatial problem solving these days that doesn't require some amount of code.

I still do 50%+ of my work in QGIS and find the embedded python interpreter to be essential. There are very few projects where I don't open it up, or otherwise have organized/cleaned the data beforehand (often with python, I'm a one trick pony).

In 2017 so far only one project has not required some coding, and that was a print map for a small transit agency. All the data could be easily hand-digitized.

[dagw]

For the case presented using, QGIS would be a far more obvious solution. However another trend in GIS is much larger scale analysis. Someone will do some analysis i QGIS by hand for a small area, then someone will go "wait a minute. We have the necessary data for the whole city/county/country. Why not run the same analysis over all that data?" Then knowing how to do reproduce the whole analysis in code becomes vital.

[danso]

I've tried teaching QGIS. As a long-time developer/OSS user, I greatly appreciate it as a community-driven piece of software. But I also recognize that such a paradigm comes with tradeoffs, including interfaces that feel welded on ad-hoc. I can get through it because I have decent experience with GIS concepts (through coding), but I understand why others can greatly struggle. Even setting it up (on OSX) can be a non-novice challenge.

I've moved to teaching CartoDB, because it has many of the features and it is based off of PostgreSQL and PostGIS. I already teach SQL so PostgreSQL is straightforward and doing GIS with SQL is a natural evolution. Carto is commercial and has its own opaqueness, so I might go back to QGIS.

But for folks who already know Python, I think being able to do GIS with code is hugely advantageous, without being overly cumbersome. R with ggplot2, is actually quite easy and graceful.

[Bedon292]

As others have said as well, I have seen the opposite of this. The real advancements have all been in coding up custom solutions to work with larger data in new ways. Things like ArcGIS / QGIS have just been used for visualization after the fact.

[danso]

Great tutorial, and thank goodness it's in 3.x. About a year ago it was difficult to find many 3.x examples of geospatial analysis. It felt like there was always some dependency that was not quite 3.x compatible, so it made sense that a lot of folks would just stick to 2.x just to get started.

[rkda]

You might also be interested in GeoNotebook, a Jupyter Notebook extension for geospatial analysis

https://github.com/OpenGeoscience/geonotebook

[jasongrout]

Another possibility for interactively manipulating maps is the ipyleaflet Jupyter widget library: https://github.com/ellisonbg/ipyleaflet. See examples of notebooks at https://github.com/ellisonbg/ipyleaflet/tree/master/examples

(disclosure: I have helped with this library)

[Bedon292]

If you are interested in GIS, make sure to check out rasterio and Fiona to go with Shapely. All great tools for GIS in python.

[rkda]

Rasterio ftw! Way easier to use than gdal's Python bindings.

Fun fact: Geopandas uses Fiona and shapely under the hood.

[Iv]

A friend just made a presentation on these exact tools at the Pycon JP 2017 a few days ago, I half expected to see Halfdan's face there: https://www.youtube.com/watch?v=Yd5oEIBFQ_E

[wiredfool]

Where is a good place to start to take a set of lat/long data (e.g., bunch of ride/walk traces) and plot a neat looking map? It seems to be a little harder than hello world, but not worth a full blown GIS stack.

[sbrother]

<self promotion, but relevant> I just launched a product similar to this but a bit more powerful and aimed at a less technical audience (most of my customers are Excel users). There is a python client that does geocoding, census data lookup, and driving distance/time if you are looking for something more than the open source options provide. If you're interested, email me or check out https://cairngeographics.com/

[rkda]

For JavaScript folks, there's Turfjs from Mapbox.

http://turfjs.org/

[tke248]

Would it be possible with something like this to locate all the pools in a give area and find the street addresses?

[dagw]

Sure, but getting the necessary data would be far from trivial.

How would you locate pools? Using areal photographs is an option, but you'd either have to do it manually which would be very time consuming or using image recognition which would be very error prone. Depending on where you live people might have to apply for a building permit to put in a pool on their property in which case the local municipality should have a database over which houses have pools, but there is no guarantee that it is up do date and getting a hold of that database is far from easy.

Connecting points to street addresses is also a bit hit and miss. Things like Google's geocoding API works OK for buildings, but is tends to be quite hit and miss for points outside of buildings. Generally it will give you the address of the closest building rather than the address the plot of land actually belongs to. So if you want to be correct you have to get a map with actual property lines and who owns what property.

So the ease of doing something like that is entirely dependent on what data you have access and how accurate you have to be. Basically the hard part of any GIS project is always data gathering/cleaing/pre-processing and never the actual analysis.

[anc84]

The linked post is just a very basic tutorial. Your idea is quite complicated geospatial analysis depending on the source data and desired correctness.