This is super important work, and is kind of why I built https://civic.band and https://civic.observer, which are generalized tools for monitoring civic govts. (You can search for anything, not just ALPR)
This is incredible, great work and will definitely be using and sharing this!
Where in the repos can we find the plugin/scraper for given municipalities to help contribute when they seem to be broken? As looks like the last meetings and agendas scraped for Cook County are from March/April of this year
- is there some standardized APIs each municipality provides, or do you go through the tedious task of building a per-municipality crawling tool?
- how often do you refresh the data? Checked a city, it has meeting minutes until 6/17, but the official website has more recent minutes (up to 12/2 at least)
- There is absolutely not a standardized API for nearly any of this. I build generalized crawlers when I can, and then build custom crawlers when I need.
- Can you let me know which city? The crawlers run for every municipality at least once every day, so that's probably a bug
ZeWaka|2 months ago
And 404 from https://civic.band/why.html
phildini|2 months ago
tayari-|2 months ago
Where in the repos can we find the plugin/scraper for given municipalities to help contribute when they seem to be broken? As looks like the last meetings and agendas scraped for Cook County are from March/April of this year
phildini|2 months ago
I also fixed Cook County
mdnahas|2 months ago
kpw94|2 months ago
Few questions:
- is the stack to index those open source?
- is there some standardized APIs each municipality provides, or do you go through the tedious task of building a per-municipality crawling tool?
- how often do you refresh the data? Checked a city, it has meeting minutes until 6/17, but the official website has more recent minutes (up to 12/2 at least)
phildini|2 months ago
- The framework for crawling is open-source. https://github.com/civicband
- There is absolutely not a standardized API for nearly any of this. I build generalized crawlers when I can, and then build custom crawlers when I need.
- Can you let me know which city? The crawlers run for every municipality at least once every day, so that's probably a bug
tonymet|2 months ago
https://civic.observer/auth/login
I am interested in monitoring local legislation in Clark County, WA