It would be one thing if Craigslist was doing their best to satisfy their customers and they wanted to shut down others who were scraping their site and leeching off their business. However, Craigslist puts absolutely no effort into making their product usable. The continued existence of Craigslist is just a testimony to the enormous strength of network effect lock in.
Disagree. Craig Newmark has said that he's scared to make drastic changes to the UX specifically because he doesn't know what magic recipe keeps people coming back. Do I like the CL layout? Nope. Does that mean other people should steal the content and profit from it? Probably not.
Agreed. Housing, where location is easily the most important factor, is where Craigslist is least usable as a product. Padmapper's success is a testament to Craigslist's users wanting something better. This is a big middle finger to all those users.
couldn't agree more.
i've tried to write my own scraper in the past to make the search more usable (at least show photos from each post next to the link so that i don't have to click through to every single link to see what pieces are for sale) and i was quickly blocked. keep in mind this was not a public site viewed by 100's of people. it was me making one search worth of individual link requests a couple times. blocked. its craigslist way or nothing.
They created a product that was against their TOS. It's pretty simple really.
If they are doing a great job and are so much better than CL then no doubt everyone will come back and use their site now they have the realtors on board.
I dislike people hating on another company for having a crappy product, just make a better one and compete against them, if they really are that bad then whatever network lock-in they have wont last.
You can't argue they are crap, then try and use them to monetise your own product.
Craigslist, Plenty Of Fish plus some others are examples of sites that took a really minimalist approach to design but managed to pull in a massive market share through fulfilling a customer need. Revenue from advertising paid the bills.
We wrote about ugly design before, how in some cases it actually increases conversion rates. It's odd but sometimes ugly is beautiful...
I am totally upset. I wanted to use padmapper to search for an apartment for my next move. I had been looking with it for months and wow it was super helpful.
Please, make a client-side solution so they can't shut you off! A browser based plugin, for example that allows us to plot the listings on our end.
I wrote this email to craigslist:
Hi Craig and Jim,
I am very upset that you cut off padmapper. Using padmapper cuts out a lot of the spam that ends up in the apartment listings due to the fact that it doesn't have a lat,long pair most of the time. It also makes it much easier to search for apartments based on your preference. With padmapper, I could see if the apartment I'm looking at is in a safe area, how far it is from a potential employer, and a myriad of other great things.
Finding an apartment with craigslist, in spite of the fact that you have the data, is much harder than searching on padmapper.
I really wish there was a way you could work with them!
You guys are a solid blue chip, no doubt. But if you take this oligarchical perspective to your business, someone is going to beat you eventually.
Hi everyone, Eric from PadMapper here, thanks for all the comments!
To everyone saying that this is lifting their content, I disagree - it makes a summary of the content and then points back to the original. If I wanted to lift their stuff, I would have made separate pages that laid things out better than the original. Literally none of the text except the title and location string make it into the summary.
It's not a copyright issue, their main legal beef is that it's against their TOU for anything but general search engines to index their content - no vertical ones allowed. Who knows what their actual motivation is.
I'm a longtime user of Craigslist, but I'm really disappointed in your decision regarding PadMapper. It would be great if you could weigh in on the debate happening on Hacker News or elsewhere.
I hope this doesn't get down-voted for the confusing title (I know it's the title of the Blog post itself; but out of context it's a bit misleading).
When I moved from Atlanta to Boston, PadMapper was invaluable in finding a place to live. Given that 1) we didn't know the area, and 2) most rentals in Boston and Cambridge are rented by individuals rather than part of larger complexes, it was great to be able to have the listings from Craigslist on a map.
Ditto when I moved from Boston to San Jose. My wife and I were only interested in a house for rent and the vast majority of those could be found on Craigslist. Again, we didn't know the area so having a map was beyond valuable.
I sincerely hope Craigslist does a 180 on this one; PadMapper is a great tool that deserves access to that data.
>I hope this doesn't get down-voted for the confusing title
Specially when the site is down. Pretty annoying to find this kind of reddit like title here.
From comments my guess is that it's some blog post from someone who is pissed off at CL for forbidding them to use some web app they built (PadMapper?) that scrap content from the CL site or something like that.
First it was Carsabi and now Padmapper. While I'm not surprised, as the Craigslist ToS explicitly prohibits crawling their site, it is incredibly frustrating that the website with all the data is unwilling to innovate and unwilling to help others innovate.
Craigslist will continue to be a mediocre, "good enough" solution. Since everyone associates online classifieds with craigslist, none of the other classifieds sites seem to have a chance. It's not even as if services like carsabi or padmapper are competing in any way with them.
Wow this is really surprising, especially after reading Craig Newmark's answer on Quora just the other day to the question "Why hasn't anyone built any products on top of Craigslist data?". His response:
"Actually, we take issue with only services which consume a lot of bandwidth, it's that simple."
I've used padmapper twice to get an apartment so far, and It's great! I've loved the fact that it's constantly improving and has exactly the features I need: Walkscore, bookmarking, map, proper filters. It's strictly better. To the author: I would have paid for this.
It sucks that this waas mainly an interface to craigslist, and now they're gone. I hope this gets enough critical mass that eventually so that the unusable but unexplicably popular horror that is craigslist apartments just dies an ugly death. Maybe you can even sell it to them, who knows?
All I know is while CL's response is perfectly rational, I can't help but be pissed off that I'm locked into a shitty product that is only surviving because of head start, critical mass, network effects, etc.
Edit:
An alternate approach (that would admittedly not help Eric) is to open source padmapper and let people do local installs for themselves. Would be pretty hard to ban that. Although I'm not sure on the legality of scraping.
Just a clarification, padmapper will continue to exist with listings from a lot of other places. It will just no longer have Craigslist listings, unless you help send an email to Craig and Jim.
Actually apartment finding in general sucks I can't think of a single goto place to look for apartments other then Craigslist and that just proves point #1!
I guess a lot depends on where you live - Craigslist in the Bay area is like magic for finding apartments, or a room to rent.
The Bay Area has a very advanced craigslist option, where you can sort not only by geographic area (Peninsula), you can even zoom down to a particular city. Add filtering by price and find a place to live in the Bay area has always (Since 2003) been painless for me - particularly as I don't drive, and so getting a place that is _precisely_ in the right place is important. Housingmaps provides a free map interface for those who find such a thing useful.
I don't actually know how anyone would _improve_ craigslist (for me) - I am not sure what the disruption would be - as the free service provides me 100% of what I would want from such a system.
This is about the 3000th time I've read that phrase, or some variation of it, in the past ten years. If that were true then it would have happened already, no? Markets can fail in the short term, but in space as crowded with upstarts as the web, I just don't buy it over such a long period of time.
The thing I like most about Craigslist is that some guy made it just to help other people out. The thing I dislike most about all the people trying to "disrupt" Craigslist is that they're clearly in it for the money. I feel like this non-trivial selling point is lost on those who swear that CL is broken and needs replacing.
I've always recommended padmapper to friends moving to NYC, it has never let me down. We'll see how it goes from here, hopefully people will just start submitting directly on their site.
I rarely hear anyone mention Craigslist in Canada. We have Kijiji [1] but its UI isn't significantly different than that of Craigslist. The home page is essentially just another long list of categories, etc. Maybe there's something about that design that works exceptionally well for this type of site.
I expected padmapper to be shut down a long time ago. Mainly because they are an incredible threat to the websites they are scraping all of their data from.
That's probably why padmapper has no way of making money at the moment.
They were hoping they could slink by, stealing other's content, until they were eventually big enough for people to warrant visiting them to list their apartments for rent. And then sell ads.
Don't get me wrong, I love padmapper, but scraping content is scraping content. And it's generally not allowed.
PadMapper is like the super helpful feature that craigslist did not figure out how to build correctly, or never bothered to build. I'm disappointed that Craigslist chose to respond by making a great social utility disappear from the market and anger the users. What about charging a license fee? What about acquiring PadMapper? What exactly does Craigslist gain by pulling its index off from PadMapper other than damaging its own public image?
Craigslist is an evil monopolist that literally wastes millions of hours of its users time every year through the enforcement of shitty UI (if the amount of money that I save using padmapper is typical). They get a good reputation for some bizarre reason because their CEO is an anti-capitalist, but they are still evil for holding back innovation.
A human lifetime is about 700,000 hours. Craigslist is the moral equivalent of a serial killer.
This is the whole reason I started foreverlist.com. Besides better/more photos, geolocation, comments, etc..., I always wanted to open up the api and let anyone develop on top of it. Alas, competing with craigslist is folly. I regret cloning the design of CL, but the logic was that it would be simple for people to use.
I absolutely LOVE pad mapper and hate craigslist even more now. Many good sites have been killed off by craigslist.
So should they ban Google? Because I use google to search for things on craigslist with much more success than craigslist's system, in the same way I used Padmapper to find a room/house (3 times in the past year... I move a lot)
Carsabi (YC W12) was also affected by this recently. Especially annoying midway through my car search. It is really disappointing that Craiglist won't license their data.
Kudos to CL for standing up for user privacy. When I use CL it's with consideration of their TOS which bans these practices.
When padmapper copies the data they violate my privacy, my copyright, and the agreement between myself and craigslist as outlined in the TOS. What padmapper is doing is a violation of many data privacy laws like PIPEDA, EUDPD, and various copyright acts.
If padmapper wants the data why don't they just obtain consent from the CL posters, instead of copying it without consent.
Would the padmapper team be ok with me deciding to stick their logo where ever I deem it necessary? I bet they'd probably sue me for copyright and trademark violation.
While I appreciate the sentiment, it is very unlikely that a landlord would object to an apartment listing being aggregated so as to expose it to more users.
If padmapper wants the data why don't they just obtain consent from the CL posters, instead of copying it without consent.
How do you propose padmapper do that without violating the Craigslist TOS in some other way like bulk-emailing everyone who lists an apartment?
I'm not saying their conduct wasn't a bit of a grey area, against the TOS and maybe a copyright violation, but I don't think many Craigslist users would object personally to their posts being reposted in this manner.
Here's the email I sent to Craigslist. I tried to use the following template: intro (what are we talking about), how padmapper is good for craigslist, why I care about padmapper, course of action.
Feel free to reuse it.
----
Hi Jim and Craig,
I learned today that Craigslist has sent a Cease and Desist to padmapper.com. Padmapper provides a service to visualize on a map rental listings from sources like Craigslist.
Real estate is all about location, and let's be frank, Craigslist's location-based search sucks. Padmapper helps the searcher easily see where real estate is located. It turns a grueling search into a breeze. It adds great value to Craigslist's listings without competing with Craigslist's business.
Padmapper, along with Craigslist, were essential in helping me find my current home in the San Francisco Bay Area. Without Padmapper, I wouldn't have used Craigslist in my search.
I'm sure you can find a mutually beneficial deal with Padmapper, instead of shutting them out entirely.
Just out of curiosity, why would they not want you to use their classifieds? Unless I am missing something, they don't collect revenue for listings. I thought Craigslist was all about spreading good will. If Pad Mapper provides a more efficient way of spreading that good will, then why not?
As a user of PadMapper, I am very disappointed to see this happen. Craigslist is almost unusable if you don't know the area you are looking to move to.
I'm surprised how many people here are defending Craigslist. They've got a history of both providing a terrible experience for apartment seekers and shutting down any attempts to use their data to provide a better experience. That would be okay in a competitive market, but they've got a strong monopoly in a market with a high barrier to entry. They're playing the textbook role of an evil monopolist.
[+] [-] jamiequint|13 years ago|reply
It would be one thing if Craigslist was doing their best to satisfy their customers and they wanted to shut down others who were scraping their site and leeching off their business. However, Craigslist puts absolutely no effort into making their product usable. The continued existence of Craigslist is just a testimony to the enormous strength of network effect lock in.
[+] [-] binaryorganic|13 years ago|reply
[+] [-] nottombrown|13 years ago|reply
[+] [-] Stefan_H|13 years ago|reply
[+] [-] SonicSoul|13 years ago|reply
[+] [-] djt|13 years ago|reply
If they are doing a great job and are so much better than CL then no doubt everyone will come back and use their site now they have the realtors on board.
I dislike people hating on another company for having a crappy product, just make a better one and compete against them, if they really are that bad then whatever network lock-in they have wont last. You can't argue they are crap, then try and use them to monetise your own product.
[+] [-] base698|13 years ago|reply
[+] [-] unknown|13 years ago|reply
[deleted]
[+] [-] sparknlaunch|13 years ago|reply
We wrote about ugly design before, how in some cases it actually increases conversion rates. It's odd but sometimes ugly is beautiful...
http://sparknlaunch.wordpress.com/2012/02/02/keep-it-simple-...
[+] [-] J3L2404|13 years ago|reply
[+] [-] readme|13 years ago|reply
Please, make a client-side solution so they can't shut you off! A browser based plugin, for example that allows us to plot the listings on our end.
I wrote this email to craigslist:
Hi Craig and Jim,
I am very upset that you cut off padmapper. Using padmapper cuts out a lot of the spam that ends up in the apartment listings due to the fact that it doesn't have a lat,long pair most of the time. It also makes it much easier to search for apartments based on your preference. With padmapper, I could see if the apartment I'm looking at is in a safe area, how far it is from a potential employer, and a myriad of other great things.
Finding an apartment with craigslist, in spite of the fact that you have the data, is much harder than searching on padmapper.
I really wish there was a way you could work with them!
You guys are a solid blue chip, no doubt. But if you take this oligarchical perspective to your business, someone is going to beat you eventually.
Thank you for your continued service
[+] [-] ericd|13 years ago|reply
To everyone saying that this is lifting their content, I disagree - it makes a summary of the content and then points back to the original. If I wanted to lift their stuff, I would have made separate pages that laid things out better than the original. Literally none of the text except the title and location string make it into the summary.
It's not a copyright issue, their main legal beef is that it's against their TOU for anything but general search engines to index their content - no vertical ones allowed. Who knows what their actual motivation is.
[+] [-] kylelibra|13 years ago|reply
[email protected] – Jim Buckmaster, CEO
[email protected] – Craig Newmark, Founder
@jimbuckmaster on twitter - http://twitter.com/jimbuckmaster
@craignewmark on twitter - http://twitter.com/craignewmark
Simple email to copy and paste:
Jim and Craig,
I'm a longtime user of Craigslist, but I'm really disappointed in your decision regarding PadMapper. It would be great if you could weigh in on the debate happening on Hacker News or elsewhere.
Thanks,
[+] [-] bcrescimanno|13 years ago|reply
When I moved from Atlanta to Boston, PadMapper was invaluable in finding a place to live. Given that 1) we didn't know the area, and 2) most rentals in Boston and Cambridge are rented by individuals rather than part of larger complexes, it was great to be able to have the listings from Craigslist on a map.
Ditto when I moved from Boston to San Jose. My wife and I were only interested in a house for rent and the vast majority of those could be found on Craigslist. Again, we didn't know the area so having a map was beyond valuable.
I sincerely hope Craigslist does a 180 on this one; PadMapper is a great tool that deserves access to that data.
[+] [-] ArcticCelt|13 years ago|reply
Specially when the site is down. Pretty annoying to find this kind of reddit like title here.
From comments my guess is that it's some blog post from someone who is pissed off at CL for forbidding them to use some web app they built (PadMapper?) that scrap content from the CL site or something like that.
[+] [-] sliverstorm|13 years ago|reply
Why do they deserve access?
[+] [-] akoumjian|13 years ago|reply
Craigslist will continue to be a mediocre, "good enough" solution. Since everyone associates online classifieds with craigslist, none of the other classifieds sites seem to have a chance. It's not even as if services like carsabi or padmapper are competing in any way with them.
[+] [-] robmaceachern|13 years ago|reply
"Actually, we take issue with only services which consume a lot of bandwidth, it's that simple."
http://www.quora.com/Why-hasnt-anyone-built-any-products-on-...
[+] [-] jaredsohn|13 years ago|reply
[+] [-] makmanalp|13 years ago|reply
It sucks that this waas mainly an interface to craigslist, and now they're gone. I hope this gets enough critical mass that eventually so that the unusable but unexplicably popular horror that is craigslist apartments just dies an ugly death. Maybe you can even sell it to them, who knows?
All I know is while CL's response is perfectly rational, I can't help but be pissed off that I'm locked into a shitty product that is only surviving because of head start, critical mass, network effects, etc.
Edit: An alternate approach (that would admittedly not help Eric) is to open source padmapper and let people do local installs for themselves. Would be pretty hard to ban that. Although I'm not sure on the legality of scraping.
[+] [-] iamwil|13 years ago|reply
[+] [-] X-Istence|13 years ago|reply
[+] [-] kenrikm|13 years ago|reply
Actually apartment finding in general sucks I can't think of a single goto place to look for apartments other then Craigslist and that just proves point #1!
[+] [-] ghshephard|13 years ago|reply
The Bay Area has a very advanced craigslist option, where you can sort not only by geographic area (Peninsula), you can even zoom down to a particular city. Add filtering by price and find a place to live in the Bay area has always (Since 2003) been painless for me - particularly as I don't drive, and so getting a place that is _precisely_ in the right place is important. Housingmaps provides a free map interface for those who find such a thing useful.
I don't actually know how anyone would _improve_ craigslist (for me) - I am not sure what the disruption would be - as the free service provides me 100% of what I would want from such a system.
[+] [-] hyperbovine|13 years ago|reply
This is about the 3000th time I've read that phrase, or some variation of it, in the past ten years. If that were true then it would have happened already, no? Markets can fail in the short term, but in space as crowded with upstarts as the web, I just don't buy it over such a long period of time.
The thing I like most about Craigslist is that some guy made it just to help other people out. The thing I dislike most about all the people trying to "disrupt" Craigslist is that they're clearly in it for the money. I feel like this non-trivial selling point is lost on those who swear that CL is broken and needs replacing.
[+] [-] reustle|13 years ago|reply
[+] [-] AgentConundrum|13 years ago|reply
I rarely hear anyone mention Craigslist in Canada. We have Kijiji [1] but its UI isn't significantly different than that of Craigslist. The home page is essentially just another long list of categories, etc. Maybe there's something about that design that works exceptionally well for this type of site.
[1] http://www.kijiji.ca
[+] [-] localhost3000|13 years ago|reply
[+] [-] GigabyteCoin|13 years ago|reply
That's probably why padmapper has no way of making money at the moment.
They were hoping they could slink by, stealing other's content, until they were eventually big enough for people to warrant visiting them to list their apartments for rent. And then sell ads.
Don't get me wrong, I love padmapper, but scraping content is scraping content. And it's generally not allowed.
[+] [-] AmericanOP|13 years ago|reply
I'm basically a Luddite and yours is the one service I always recommend.
[+] [-] lindablus|13 years ago|reply
[+] [-] jacoblyles|13 years ago|reply
A human lifetime is about 700,000 hours. Craigslist is the moral equivalent of a serial killer.
[+] [-] Mc_Big_G|13 years ago|reply
I absolutely LOVE pad mapper and hate craigslist even more now. Many good sites have been killed off by craigslist.
[edit spelling]
[+] [-] juiceandjuice|13 years ago|reply
This is stupid.
[+] [-] makeramen|13 years ago|reply
[+] [-] fleitz|13 years ago|reply
When padmapper copies the data they violate my privacy, my copyright, and the agreement between myself and craigslist as outlined in the TOS. What padmapper is doing is a violation of many data privacy laws like PIPEDA, EUDPD, and various copyright acts.
If padmapper wants the data why don't they just obtain consent from the CL posters, instead of copying it without consent.
Would the padmapper team be ok with me deciding to stick their logo where ever I deem it necessary? I bet they'd probably sue me for copyright and trademark violation.
[+] [-] ericd|13 years ago|reply
[+] [-] robrenaud|13 years ago|reply
It's putting apartment listings on a map. People listing apartments tend to want it to be widely known.
[+] [-] ArmstrongRSBC|13 years ago|reply
wat??
[+] [-] refurb|13 years ago|reply
[+] [-] Zak|13 years ago|reply
While I appreciate the sentiment, it is very unlikely that a landlord would object to an apartment listing being aggregated so as to expose it to more users.
If padmapper wants the data why don't they just obtain consent from the CL posters, instead of copying it without consent.
How do you propose padmapper do that without violating the Craigslist TOS in some other way like bulk-emailing everyone who lists an apartment?
I'm not saying their conduct wasn't a bit of a grey area, against the TOS and maybe a copyright violation, but I don't think many Craigslist users would object personally to their posts being reposted in this manner.
[+] [-] guelo|13 years ago|reply
[+] [-] unknown|13 years ago|reply
[deleted]
[+] [-] AceJohnny2|13 years ago|reply
Feel free to reuse it.
----
Hi Jim and Craig,
I learned today that Craigslist has sent a Cease and Desist to padmapper.com. Padmapper provides a service to visualize on a map rental listings from sources like Craigslist.
Real estate is all about location, and let's be frank, Craigslist's location-based search sucks. Padmapper helps the searcher easily see where real estate is located. It turns a grueling search into a breeze. It adds great value to Craigslist's listings without competing with Craigslist's business.
Padmapper, along with Craigslist, were essential in helping me find my current home in the San Francisco Bay Area. Without Padmapper, I wouldn't have used Craigslist in my search.
I'm sure you can find a mutually beneficial deal with Padmapper, instead of shutting them out entirely.
[+] [-] ndubya|13 years ago|reply
As a user of PadMapper, I am very disappointed to see this happen. Craigslist is almost unusable if you don't know the area you are looking to move to.
[+] [-] dap|13 years ago|reply