[giu]

Thanks a lot for your reply!

Scraping websites can be quite the messy business, since some websites change their document structure more often than others.

Nonetheless, it's still a very instructive activity and you can build quite the pipeline around it (scraping multiple websites, joining datasets, efficiently storing the data, etc.).

[iagovar]

Yeah, when data piled up I had to think about how to store it, RAM, and a bunch of other things that I didn't have to consider with sample data. Specifically RAM and how to transform data without so much need of it was a concern for some time.

[rohan_shah]

I am also currently learning to scrape forums. And I am a philosophy student. Could you point to some resources that helped you learn it better?