top | item 46292515

(no title)

phildini | 2 months ago

This is super important work, and is kind of why I built https://civic.band and https://civic.observer, which are generalized tools for monitoring civic govts. (You can search for anything, not just ALPR)

discuss

order

tayari-|2 months ago

This is incredible, great work and will definitely be using and sharing this!

Where in the repos can we find the plugin/scraper for given municipalities to help contribute when they seem to be broken? As looks like the last meetings and agendas scraped for Cook County are from March/April of this year

phildini|2 months ago

Hello! The crawlers are not currently public, but I'm happy to take volunteers behind the curtain.

I also fixed Cook County

mdnahas|2 months ago

Also missing: Austin, TX

kpw94|2 months ago

Very cool! And important for sure, thank you.

Few questions:

- is the stack to index those open source?

- is there some standardized APIs each municipality provides, or do you go through the tedious task of building a per-municipality crawling tool?

- how often do you refresh the data? Checked a city, it has meeting minutes until 6/17, but the official website has more recent minutes (up to 12/2 at least)

phildini|2 months ago

Thanks for asking!

- The framework for crawling is open-source. https://github.com/civicband

- There is absolutely not a standardized API for nearly any of this. I build generalized crawlers when I can, and then build custom crawlers when I need.

- Can you let me know which city? The crawlers run for every municipality at least once every day, so that's probably a bug