As a team, we’re very proud of the product we built, now used by over 125k developers, data scientists and businesses" ... from FAQ: "service will shut down Feb 29, 2016"
In fact, they are so proud, they're giving all but ~2 weeks notice to said 125K developers, data scientists, and businesses to migrate off the platform? That is an abysmally small amount of time and if I was a paying customer, i'd be furious.
Sort of plays right into the whole argument being made over on another thread (https://news.ycombinator.com/item?id=11105198) that companies shouldn't rely too heavily on proprietary tools to solve their data challenges.
Palantir is the poster child for black box software company that keeps trying to sell you more expensive licenses for their mystery algorithm. I know they've had some success in the government sector but I've seen them make a real mess of things in the private sector. At my last company their system just became a giant paracite where data went in and stuff came out the other end but it was entirely unclear what happened in between. The more we looked into the mystery middle ground the uglier it got.
We at CloudScrape.com are happy to welcome you onboard our platform - you get 20 hours of scraping runtime for free and the ability to navigate much more complex websites, opening up tons more opportunities then you've had previously. And we're here to stay :)
Kind regards,
Henrik Hofmeister, CTO, Co-Founder @ CloudScrape.com
ParseHub is much better at getting websites from shitty, complicated sites. Kimono was great for something simple, but I found ParseHub to be a more powerful and flexible web scraper.
The worst part about Palantir is their ability to masquerade as this hip startup, as though working for them is as innocent as working for Imgur or Twitter or whatever. They recruit the hell out of MIT students, and it sucks seeing my friends interview for this incredibly shitty company that has openly said it won't go public since that would make running their business "very difficult" (hm, because you're doing reprehensible work perhaps?).
I wish people picked their jobs to match their ethics instead of the other way around. It makes me sad when I hear about my friends going to intern for this place.
I know some engineers who work or worked there. Aside from the normal engineers (tools, backend, etc.), they are essentially high-priced tech consultants sent to companies and orgs that have no internal software talent at all, to help them analyze their (atrociously stored/managed) data. None of them work on anything shady. They essentially build web apps and backend apps to analyze and visualize data. Nothing vodoo military about it, although the military is a customer.
The pay actually isn't actually particularly high for SV standards. I think Facebook outpays them, for example - and Facebook is less selective than Palantir. But their reputation is that they're full of top engineers and you'll learn a ton there, and that's why the people I know work/worked there.
The downsides I've heard about are no work/life balance and fratty culture. Those are real downside, not the FUD about ethics.
> it won't go public since that would make running their business "very difficult" (hm, because you're doing reprehensible work perhaps?)
Well, more generally, anyone who has a nonstandard set of ethical beliefs, and is basing their company around that set of beliefs, can't really go public without their belief-set being thrown under the bus of their shareholders' belief-set.
For example, I've often considered starting a company in the game industry that actually hires experienced adults and treats them well, rather than hiring fresh college students and burning them out. I imagine, though, that if my hypothetical company became publicly-traded, I'd be forced to relinquish this policy in the name of short-sighted market competitiveness.
Interning for Palantir is a pretty big resume boost for college students, and they have one of the highest paying internships in the valley. There's some serious benefits to going to Palantir.
On one hand, Palantir have (at best) questionable connections and I wouldn't be a rush to work for them. On the other, with the level of pay/benefits they provide, I'm not surprised they can find people without similar qualms (or people who overlook or aren't aware of them).
I won't share specifics (as they aren't mine to share), but I have a friend working for Palantir overseas. Once essential benefits (e.g. accommodation) are factored in, his salary is north of six figures (GBP). For someone only a few years past university, I imagine that sort of offer is difficult to refuse.
Some business owners just don't want shareholders inserting their opinion where it isn't wanted. If they don't need the funding, there's no reason to go public.
My wife is a student and looking for internship she will work for a company that helps people write about how they ran over than grandma if she gets a legal. Morality and ethics have a price always.
https://webrobots.io - we give free Chrome extension to run scraper robots on your own computer. For paid customers we host robust cloud of scraping instances with all advanced features one can expect. To be fair - you need to understand javascript to write robots on our platform, we are not point-and-click doodling tool. Upside of writing robots in code - nearly unlimited possibilities with dynamic content sites, in-flight data cleaning, etc.
damn. I was gonna say this means it's time to actually launch my version of this I built 3 of 4 years ago, but it looks like apifier is almost the exact same thing.....
ParseHub is especially good at dealing with dynamic websites and multidimensional data (arbitrary relationships instead of just rows and columns).
Our team would be happy to help you migrate from Kimono. We're bootstrapped and profitable, so we don't have pressure to sell when an offer comes along. We also pledged to release the back-end code under a liberal open source license if we were ever in a position where we could no longer provide service.
As someone who used Kimono a few times before the closure...this announcement does not surprise me at all.
The target demographic was developers who were too lazy to write simple scrapers for a given predefined use case. However, scraping most modern websites is not simple, and requires a lot of code to ensure the correct HTML elements are scraped and the data is transformed correctly...which defeats the purpose of using Kimono.
Kimono joining Palantir was likely an acquihire, especially since the core service was permissively free.
"too lazy" seems like an unfair characterization. If a service lets you do what you want faster, why not use it? I'm not familiar with Kimono, but if you can just rip it out and replace it with your own code later, why not?
"access to the world’s most important data problems"
I'm not sure how to put this, but I've used Palantir's products in a couple different warzones. I can't imagine they're not pointing those tools at everyday Americans. In fact I'm certain of it (just read FedBizOpps postings from the DHS and DoD). I'm sure most of the people working there have no idea, but I'm also sure there are some truly bad people working for that company. Man, we really need to regulate this a lot better.
Powered by Palantir, basically it is real Big Brother brain network where all the information like lic. plate readers, video cam feeds, cell phone data, Internet data and everything else is "fused" together. One of the immediate resulting products of it is generation of Suspicious Activity Reports. Hoover (and Beria) would die of envy.
Disappointing to see just 2 weeks notice for their existing users.
* shameless plug * For longer than most such tools, we've been offering what our customers call the "easiest scraper of 'em all": https://feedity.com Try it out and see if it works for your needs.
> Will Palantir (or anyone else) have access to my personal data?
No. All of your user data including names, email addresses, passwords and APIs will not be shared with anyone and will be securely purged from our servers on March 31, 2016.
Uh...but what about the scraped data? I don't think I've used Kimono since it was first announced on HN as a side project...but I'm assuming that you got to save scraped data semi-privately with Kimono? Is that considered "user data"? Kind of a vague term, and one I usually associate with data of the personal information variety. I'm sure someone at Palantir can find some interesting meta-insights from analyzing the data collected by the kind of people (data scientists, etc) who would use Kimono heavily.
Palantir is one of those organizations that makes me throw up a little in the back of my throat whenever I hear about them. They are sort of like the Nickelback of companies.
You can host scrapers you need to migrate on morph.io ( https://morph.io ) a free platform for hosting and running scrapers, storing your data and providing access to it in a range of formats (JSON, CSV, SQLite). It’s free, open source, supports a bunch of languages and is run by an Australian non-profit, where I happen to work ;-).
Disappointing. Was my goto service as it was a joy to use and always seemed to work. Wish they would follow the Parse lead and release something open source. Desktop doesn't really cut it for API use...
I'm always disappointed when I read of Palantir acquihires.
Technically talented employees receive a pittance and are sold into the Palantir machine, a morally unscrupulous high-tech consulting shop that takes in talented engineers and churns out one-off software black boxes designed to lock in customers and extract a long-term tax.
I had a hard time using this product. I was trying to use another developer's APIs but it didn't seem possible. The one positive thing about the announcement, from the FAQ:
Will Palantir (or anyone else) have access to my personal data?
No. All of your user data including names, email addresses, passwords and APIs will not be shared with anyone and will be securely purged from our servers on March 31, 2016.
Their "desktop tool" is complete rubbish. I have to manually click on each source to crawl it and then manually click a few buttons to save the response.
Two months is a short time but I had sort of expected this given how bad their service of performing last few days. That is why I had started moving to other services.
ParseHub is much better at getting websites from shitty, complicated sites. Kimono was great for something simple, but ParseHub is a more powerful and flexible web scraper. https://www.parsehub.com/
[+] [-] drglitch|10 years ago|reply
As a team, we’re very proud of the product we built, now used by over 125k developers, data scientists and businesses" ... from FAQ: "service will shut down Feb 29, 2016"
In fact, they are so proud, they're giving all but ~2 weeks notice to said 125K developers, data scientists, and businesses to migrate off the platform? That is an abysmally small amount of time and if I was a paying customer, i'd be furious.
On the other hand, congrats to Kimono team!
[+] [-] code4tee|10 years ago|reply
Palantir is the poster child for black box software company that keeps trying to sell you more expensive licenses for their mystery algorithm. I know they've had some success in the government sector but I've seen them make a real mess of things in the private sector. At my last company their system just became a giant paracite where data went in and stuff came out the other end but it was entirely unclear what happened in between. The more we looked into the mystery middle ground the uglier it got.
[+] [-] henrik1409|10 years ago|reply
We at CloudScrape.com are happy to welcome you onboard our platform - you get 20 hours of scraping runtime for free and the ability to navigate much more complex websites, opening up tons more opportunities then you've had previously. And we're here to stay :)
Kind regards, Henrik Hofmeister, CTO, Co-Founder @ CloudScrape.com
http://cloudscrape.com
[+] [-] kftaylor|10 years ago|reply
[+] [-] frabcus|10 years ago|reply
My guess, they weren't commercially successful, and aquihire was the best way out.
[+] [-] asdfologist|10 years ago|reply
Why? Money.
[+] [-] vrathee|10 years ago|reply
http://www.datascraping.co
[+] [-] VeilEm|10 years ago|reply
The tone of the announcement is in bad taste given the bad news for their customers.
[+] [-] spesad|10 years ago|reply
[+] [-] rangeva|10 years ago|reply
[+] [-] misterrobot|10 years ago|reply
I wish people picked their jobs to match their ethics instead of the other way around. It makes me sad when I hear about my friends going to intern for this place.
[+] [-] argonaut|10 years ago|reply
The pay actually isn't actually particularly high for SV standards. I think Facebook outpays them, for example - and Facebook is less selective than Palantir. But their reputation is that they're full of top engineers and you'll learn a ton there, and that's why the people I know work/worked there.
The downsides I've heard about are no work/life balance and fratty culture. Those are real downside, not the FUD about ethics.
[+] [-] derefr|10 years ago|reply
Well, more generally, anyone who has a nonstandard set of ethical beliefs, and is basing their company around that set of beliefs, can't really go public without their belief-set being thrown under the bus of their shareholders' belief-set.
For example, I've often considered starting a company in the game industry that actually hires experienced adults and treats them well, rather than hiring fresh college students and burning them out. I imagine, though, that if my hypothetical company became publicly-traded, I'd be forced to relinquish this policy in the name of short-sighted market competitiveness.
[+] [-] whorleater|10 years ago|reply
[+] [-] jonnybgood|10 years ago|reply
[+] [-] stordoff|10 years ago|reply
I won't share specifics (as they aren't mine to share), but I have a friend working for Palantir overseas. Once essential benefits (e.g. accommodation) are factored in, his salary is north of six figures (GBP). For someone only a few years past university, I imagine that sort of offer is difficult to refuse.
[+] [-] b34r|10 years ago|reply
[+] [-] pbreit|10 years ago|reply
[+] [-] tn13|10 years ago|reply
[+] [-] kumarski|10 years ago|reply
http://grabby.io
http://fullcontact.com
http://emailhunter.co
http://clearbit.com
http://toofr.com
http://import.io
http://kimonolabs.com
http://apifier.com (favs)
http://elink.club
http://www.eliteproxyswitcher.com/ - ;)
[+] [-] ddebernardy|10 years ago|reply
https://github.com/scrapinghub/portia/ (our open source equivalent of Kimonolabs)
http://scrapy.org (allows to scrape more complex sites; also OSS)
http://scrapinghub.com (for a cloud-based platform and a smart proxy rotator)
(Disclaimer: working there.)
[+] [-] webrobots|10 years ago|reply
[+] [-] outdooricon|10 years ago|reply
[+] [-] henrik1409|10 years ago|reply
[+] [-] dwynings|10 years ago|reply
[+] [-] jdc0589|10 years ago|reply
[+] [-] ultimatejman|10 years ago|reply
[+] [-] tsergiu|10 years ago|reply
For folks looking for an alternative, check out http://www.parsehub.com
ParseHub is especially good at dealing with dynamic websites and multidimensional data (arbitrary relationships instead of just rows and columns).
Our team would be happy to help you migrate from Kimono. We're bootstrapped and profitable, so we don't have pressure to sell when an offer comes along. We also pledged to release the back-end code under a liberal open source license if we were ever in a position where we could no longer provide service.
[+] [-] minimaxir|10 years ago|reply
The target demographic was developers who were too lazy to write simple scrapers for a given predefined use case. However, scraping most modern websites is not simple, and requires a lot of code to ensure the correct HTML elements are scraped and the data is transformed correctly...which defeats the purpose of using Kimono.
Kimono joining Palantir was likely an acquihire, especially since the core service was permissively free.
[+] [-] michaelmior|10 years ago|reply
[+] [-] mrdrozdov|10 years ago|reply
[+] [-] w1ntermute|10 years ago|reply
[+] [-] marvel_boy|10 years ago|reply
[+] [-] fapjacks|10 years ago|reply
I'm not sure how to put this, but I've used Palantir's products in a couple different warzones. I can't imagine they're not pointing those tools at everyday Americans. In fact I'm certain of it (just read FedBizOpps postings from the DHS and DoD). I'm sure most of the people working there have no idea, but I'm also sure there are some truly bad people working for that company. Man, we really need to regulate this a lot better.
[+] [-] trhway|10 years ago|reply
http://www.dhs.gov/state-and-major-urban-area-fusion-centers
Powered by Palantir, basically it is real Big Brother brain network where all the information like lic. plate readers, video cam feeds, cell phone data, Internet data and everything else is "fused" together. One of the immediate resulting products of it is generation of Suspicious Activity Reports. Hoover (and Beria) would die of envy.
[+] [-] jsprogrammer|10 years ago|reply
Could someone give an example?
Why is access to these most important problems limited to Palantir, or otherwise not accessible to most people (as implied by this shutdown notice)?
[+] [-] nreece|10 years ago|reply
* shameless plug * For longer than most such tools, we've been offering what our customers call the "easiest scraper of 'em all": https://feedity.com Try it out and see if it works for your needs.
[+] [-] danso|10 years ago|reply
https://www.kimonolabs.com/faq#faq-303
> Will Palantir (or anyone else) have access to my personal data?
No. All of your user data including names, email addresses, passwords and APIs will not be shared with anyone and will be securely purged from our servers on March 31, 2016.
Uh...but what about the scraped data? I don't think I've used Kimono since it was first announced on HN as a side project...but I'm assuming that you got to save scraped data semi-privately with Kimono? Is that considered "user data"? Kind of a vague term, and one I usually associate with data of the personal information variety. I'm sure someone at Palantir can find some interesting meta-insights from analyzing the data collected by the kind of people (data scientists, etc) who would use Kimono heavily.
[+] [-] p4wnc6|10 years ago|reply
[+] [-] orng|10 years ago|reply
[+] [-] equivalentideas|10 years ago|reply
Here's a tutorial all about running a scraper on morph.io https://www.openaustraliafoundation.org.au/2015/10/13/ruby-w...
[+] [-] sergiotapia|10 years ago|reply
[+] [-] svarrall|10 years ago|reply
[+] [-] austinhutch|10 years ago|reply
[+] [-] jpatokal|10 years ago|reply
[+] [-] foresight|10 years ago|reply
Technically talented employees receive a pittance and are sold into the Palantir machine, a morally unscrupulous high-tech consulting shop that takes in talented engineers and churns out one-off software black boxes designed to lock in customers and extract a long-term tax.
[+] [-] solveforall|10 years ago|reply
Will Palantir (or anyone else) have access to my personal data? No. All of your user data including names, email addresses, passwords and APIs will not be shared with anyone and will be securely purged from our servers on March 31, 2016.
https://www.kimonolabs.com/faq
[+] [-] tn13|10 years ago|reply
Thanks for nothing guys!
[+] [-] tn13|10 years ago|reply
[+] [-] MeGer|10 years ago|reply