pablohoffman | 10 years ago | on: Web scraping at scale – a conversation with Pablo Hoffman from ScrapingHub
Hey folks, I'm the one interviewed, happy to answer any follow up question here.
pablohoffman's comments
pablohoffman | 10 years ago | on: Ask HN: Examples of companies with cofounders starting together at distance
Not surprised, I'm pretty low profile :)
pablohoffman | 10 years ago | on: Show HN: CloudScrape – Cloud-based web scraping platform
Portia from Scrapinghub is 100% open source: http://scrapinghub.com/portia/
pablohoffman | 11 years ago | on: Show HN: Scrape.it – Change-Resilient Web Scraper
We released a similar open source tool for visual scraping, earlier this year, called Portia: https://github.com/scrapinghub/portia
It's been getting quite a bit of traction and we're currently working on the integration with Scrapinghub platform (disclaimer: I work there) for those who prefer a hosted version.
pablohoffman | 12 years ago | on: Portia, an open-source visual web scraper
Not yet.
pablohoffman | 12 years ago | on: Portia, an open-source visual web scraper
Yes, you just need to select a different field type ("text", instead of "html").
pablohoffman | 13 years ago | on: Why MongoDB is a bad choice for storing our scraped data
We are using CDH4.2 and have had a very positive experience so far.
Cloudera has in fact been an inspiration for us to follow, you guys have really struck the right balance between open source and commercial support. We follow the same philosophy with Scrapy (an open source web crawling framework), as you do with Hadoop and its ecosystem.
pablohoffman | 13 years ago | on: Why MongoDB is a bad choice for storing our scraped data
We are not plainly complaining about MongoDB, nor saying it's useless. We are just explaining why it's a poor choice for a specific use case: storing scraped data.
FWIW, we still use Mongo in other internal applications, it's just not the right choice for our crawl data storage backend.
pablohoffman | 13 years ago | on: Why MongoDB is a bad choice for storing our scraped data
We went with HBase. Cassandra would have been suitable too, but we already use Hadoop for data processing so it was a natural choice within the infrastructure ecosystem. We will write a followup about that.
pablohoffman | 14 years ago
The title of this post is misleading, it makes you think GitHub is actually broken, when the article is about complaining how their fork feature works.
pablohoffman | 14 years ago | on: Free 5 Billion Page Web Index Now Available from Common Crawl Foundation
I initially submitted this post, but then deleted it and resubmitted to the original post on Common Crawl blog: http://news.ycombinator.com/item?id=3208853
I now regret since this one got much more attention. I was under the impression that linking to the original post was more welcomed here HN, but it seems this is not always the case.
pablohoffman | 14 years ago | on: One-time Secret: Share passwords etc with URIs that work only once
I found strange that no one mentioned Privnote (https://privnote.com), a much older/popular service that does this but more securely since it generates URLs with keys in the fragment to encrypt the messages on client side (so that the plain message never hits the server).
pablohoffman | 14 years ago | on: Google Please Hire Me
Did it work?
pablohoffman | 14 years ago | on: Linus Torvalds dumps Gnome3 for XFCE (G+ discussion)
I've been using tiled window managers for 7 years now (currently on awesome, which I switched 3 years ago) and the interface has always remained pretty consistent. I wonder if Linus has tried tiled WMs?
pablohoffman | 15 years ago | on: Start registeri.ng, .NG domains go on sale
After checking a few registrars, I found they only sell third-level domains (.com.ng, .net.ng), not second-level domains (like "amazi.ng") which is probably what most people here would be interested in.
Example: http://www.webdomains.com.ng/myaccount/whois.php?step=1
It's a shame they're missing the opportunity to sell such a nice top-level domain - or perhaps it's done on purpose?
pablohoffman | 15 years ago | on: Ask HN: What would it take to scrape & index delicious?
Someone already wrote a Delicious scraper in Scrapy (Python): http://news.ycombinator.com/item?id=2015680
pablohoffman | 15 years ago | on: Mark Zuckerberg Named TIME’s 2010 Person of the Year
Feeling sad would imply that you hold some respect to the judges that decided who won, but I think Time magazine just lost the respect of many readers. I'm kinda sad for them, not for Assange.
pablohoffman | 15 years ago | on: Is OpenTable Worth it?
That reminded me of a recent Paul Graham tweet:
You've found market price when buyers complain but still pay.
pablohoffman | 15 years ago | on: Facebook acquires file-sharing service Drop.io
Primary source thread: http://news.ycombinator.com/item?id=1849274
pablohoffman | 15 years ago | on: Why Mongrel2 Doesn't Use INI Files (Zed Shaw)
Is it me, or Zed is starting to get annoyed at the Python community now?. I wonder when he'll write "Python is a Ghetto".
page 1