[Bencarneiro]

OK holy jeez I think I got the server back up. We now have 8X the ram and I'm not sure how, but we broke postgres....

[et-al]

Not to sure how your backend is set up, but it looks like you're generating GeoJSON on the fly and the JSON serialization for this amount of data can be slow.

In typical HN fashion, I would suggest looking into using Tippecanoe to generate some vector tiles of the data and host that on S3. Then the DB can fall back to performing simple lookups via the accident ID. Filtering by time will need to move to the frontend, but that should be fine (if not, look into clustering the data at further zoom levels).

https://github.com/mapbox/tippecanoe

[Bencarneiro]

This is very helpful

[carderne]

You’ve had some other recommendations already but I’d suggest also looking into FlatGeobuf [0] for this use case. Have a look at the MapLibre example with a 12GB example [1]. You don’t need a server at all (unlike MBTiles) and will be able to load far more points at once than your current solution. Can easily generate it with QGIS/GDAL/PostGIS. Not sure what your plans are for the project but would be happy to donate some time to get something like that working.

[0] https://github.com/flatgeobuf/flatgeobuf

[1] https://flatgeobuf.org/examples/maplibre/large.html

[sabman]

Apologies for the shameless plug but do checkout https://geobase.app We built it with such use cases in mind and solves the scaling issues with geo data driven apps. We are in private beta but happy to give you early access if you hit us up.

We were also recently featured on the motherduck blog https://motherduck.com/blog/pushing-geo-boundaries-with-moth...

[munch117]

I was trying out different time periods (comparing 2003-2013 to 2013-2023, to see a rough trend), and I noticed that every click to change the year by one seemed to generate a refresh. Perhaps it would help to lower the load if you change the date selector widget to only refresh after the date selector is closed.

[ppbjj]

godspeed, soldier

[winrid]

Remember to increase the buffer pool size to use that ram!

[NwtnsMthd]

> I mapped almost every USA traffic death in the 21st century

Is your server on that list?

j/k, I’m sorry, I’ll see myself out XD

[mastersummoner]

I'm not a fan of cheap puns, but that was excellent.

[kazinator]

Postgres didn't look both ways before stepping off the curb.