WingNews logo WingNews GitHub [2]
top | item 19619987

Xg2xg: Lookup table of similar tech and services for ex-Googlers

369 points| ra7 | 7 years ago |github.com | reply

126 comments

order
[+] jarym|7 years ago|reply
I had never thought about this before but from this repo I can see all these smart engineers at the big tech firms live within their own walled garden (or prison?).

If it weren’t for such a vibrant open source world then they’d really be stuck if they tried to apply their knowledge outside of their companies.

Thanks for sharing the repo!

[+] WestCoastJustin|7 years ago|reply
It is almost the other way around. Many of these open-source projects, have roots with engineers leaving these big companies and wanting to use the same tools, so they create them. Or, by these big companies releasing papers and then folks creating open-source versions. Here's some examples of the latter: Dapper (w/ OpenTracing) [1], Spanner (w/ CockroachDB) [2], Borg (w/ Mesos) [3], Dremel (w/ Drill) [4]. I'm sure there are tons more examples but these are just off the top of my head.

[1] https://ai.google/research/pubs/pub36356.pdf

[2] https://ai.google/research/pubs/pub39966.pdf

[3] https://ai.google/research/pubs/pub43438.pdf

[4] https://ai.google/research/pubs/pub36632.pdf

[+] asdfasgasdgasdg|7 years ago|reply
A lot more than just ex-FAANG folks would be stuck without the OSS technologies listed in the right column. I daresay, considering much of this software was authored by Xooglers or inspired by Google papers, that it's actually everyone else who would be stuck. The folks who have seen how these problems can be solved at scale would just reimplement the solutions, as indeed they have done.
[+] int_19h|7 years ago|reply
With Microsoft, this has changed substantially in the past decade, and there are many teams running entirely on public online services (GitHub, Azure DevOps, Travis etc) and with end-to-end open source tooling and dependencies. So it's much less of a technological bubble than it used to be. Probably less so than Google at this point. Can't say about Amazon.
[+] akhilcacharya|7 years ago|reply
They don’t, and if you’ve got the G-factor, pun intended, to get an offer at Google you can get an offer anywhere. Which, honestly, makes this repository all the more confusing to me.

I’m sure there are some people that have lived only within this ecosystem, but that’s the possibly the height of privilege.

[+] smueller1234|7 years ago|reply
This is pretty misleading: a lot of the presented open source equivalents are nothing like the original. For example, comparing HDFS to Colossus is... optimistic? Having used / worked on/with both, the similarity is roughly that they both store bytes. This would be a lot less misleading if there were annotations and if the "open source/real world" didn't mix together other large companies' products with open source. For example protobuf is open source, but this makes it look like it isn't.
[+] puzzle|7 years ago|reply
Yeah, HDFS is a toy in comparison. It's barely comparable to the original GFS, which was retired... Almost a decade ago?
[+] alexhutcheson|7 years ago|reply
If you needed something Colossus-like in the open source world, what would you use? Is there anything available that's better than HDFS for the "general-purpose, big, distributed file system" use-case?
[+] sqs|7 years ago|reply
Sourcegraph CEO here. Cool to see we are listed as a CodeSearch alternative. We assembled some links to Google internal docs and studies about how they use their Google internal CodeSearch tool. See the links at the top of https://docs.sourcegraph.com/user/search if interested.
[+] mherdeg|7 years ago|reply
I wonder how many of the we-left-Facebook-and-miss-it sorts of services don't have an obvious Google corollary (I'm thinking of Scuba/Honeycomb)
[+] mdeeks|7 years ago|reply
What does Google (or others) use for an intranet or team documentation? We're using Quip and it is just a black hole where documents go to die, never be seen again no matter how badly you want to find it.
[+] bitskits|7 years ago|reply
Within Google there are several tools, but the most common (and IMO best) is markdown. (Docs is a close second.)

Pros: Get's checked in like code, gets reviewed, and leaves a nice paper trial of edits.

Cons: Doesn't invite comments/discussions as well as Docs does.

[+] alexhutcheson|7 years ago|reply
Amazon had a giant internal wiki that they used for team information, playbooks, design docs, etc. Formatting was kind of ugly, but the barrier to making an edit was super low, and having everything in one searchable place was super useful. From experience, I think any system that requires more than a few minutes to make an edit ends up not being updated and going stale pretty quickly.

Source: https://www.xwiki.com/en/references/amazon

[+] bauerd|7 years ago|reply
Can't tell for Google, but Confluence is widely adopted (still I love to hate on it)
[+] verst|7 years ago|reply
It used to be Google Sites (then part of Google Apps for Business which is now GSuite), Docs and Slides which were indexed by an internal Google search engine called MOMA (which I believe was powered by the Google Search Appliance, but my recollection on this last point is faint)
[+] dsl|7 years ago|reply
I really wish they would add memegen to G Suite.
[+] alpb|7 years ago|reply
Dory is probably more realistic.
[+] myroon5|7 years ago|reply
I'd love to see this for other large tech companies like Amazon and Microsoft
[+] paxys|7 years ago|reply
An equivalent of this for Microsoft is pretty much impossible because there isn't one single way of doing any of these things at the company. Different departments/orgs/teams use different tools, some self-built and some off-the-shelf (including most of the stuff on this list).
[+] nickbarnwell|7 years ago|reply
I can't speak to Amazon, but at Microsoft the equivalents to the Google offering are going to have the same OSS competitors. Having worked on services there, the only thing I really miss is Kusto; it's now available as an Azure service, but I'm at a company all in on AWS :(
[+] tyingq|7 years ago|reply
Great list, and interesting for non-Googlers too. I'd replace, though:

Google -> Real World

free food -> :(

With:

free food -> free time

[+] QuercusMax|7 years ago|reply
Not sure why you think free food is traded for free time. Free food helps me use my time at work more effectively. I usually have breakfast and lunch with teammates, and it's a mix of socialization and work-related conversations. Half an hour in the free cafe > 45-90 minutes going someplace to buy food.
[+] paxys|7 years ago|reply
I'm willing to bet that the average Google employee works as much or even less than the rest of the industry.
[+] dekhn|7 years ago|reply
the free food option at google and now my current startup is a huge time saver. It doesn't come at the expense of free time.
[+] djtriptych|7 years ago|reply
Came to find critique. Left disappointed :(
[+] sqs|7 years ago|reply
We (Sourcegraph) are interested in building an alternative to Critique at some point in the future, or at least enhancing existing code review tools to offer many of the favorite features of Critique.

What are your favorite/must-have Critique features?

(Also would love to jump on a video chat if you or anyone else is interested, and we could live-stream/share it on YouTube for others interested. Email me at [email protected] if interested.)

[+] jaytaylor|7 years ago|reply
Is there something comparable to the web-based IDE called Cider?

Now that Amazon took cloud9 (c9.io) out of the market, it seems like there is a sizeable void.

I want web-based Spacemacs or Evil! Or even just intelli-j.

[+] tsechin|7 years ago|reply
I've tried Eclipse Che (listed in the article), and it's pretty good. I'm looking forward to Che switching its editor to Eclipse Theia, which promises to be "the VS Code experience in the browser".
[+] HuggableSquare|7 years ago|reply
I recently discovered Coder's open source version of their web-based IDE (https://github.com/codercom/code-server). It's pretty much VS Code in the browser, which is almost exactly what I was looking for. You just download the binary, point it to a folder and it works exactly the way you would expect (including the terminal running under the account the server runs as so you get to keep your shell of choice and all that fun stuff).
[+] fhrow4484|7 years ago|reply
Where AWS/GitHub is mentioned, there should be the equivalent Azure stuff, especially for bug tracking, CI/CD, repo subfolder "owners", code review, there's a ton of stuff equivalent or better to buganizer, OWNERS file, etc .

In services there should be vitess since it's both a YouTube and an open source project.

It seem to have a new transactional messaging feature with message sending and Ack: https://vitess.io/docs/advanced/messaging/, which seem to fill a slightly different niche than rabbitmq/pubsub/etc.

[+] romka2|7 years ago|reply
There is no listed equivalent of RecordIO. What do people use for high-reliability journals?

When I needed something like RecordIO to store market data, I couldn't find anything. So I implemented https://github.com/romkatv/ChunkIO. I later learned of https://github.com/google/riegeli (work in progress), which could've saved me a lot of time if only I found it earlier. I think my ChunkIO is a better though.

[+] shereadsthenews|7 years ago|reply
That appears to be exactly RecordIO. It even has transposition. Is there a reason that doesn't meet your requirements?

Edit: it even includes an open source implementation of Cord, which they've renamed to Chain for some reason.

[+] antimora|7 years ago|reply
Anything for Amazon tooling?

For example, I see Pipelines could be replaced by CodeDeploy. I hear nix is similar to the brazil build system. Any other similar mapping by others?

[+] simula67|7 years ago|reply
Is there an alternative to BigTable which written in C or Go ?
[+] q3k|7 years ago|reply
ScyllaDB, which is basically a C++ Cassandra reimplementation, which in turn borrows a lot of design ideas from BT.
[+] puzzle|7 years ago|reply
If C++ is an option, Baidu has a clone called Tera.
[+] malkia|7 years ago|reply
maybe piper|citc -> VFSForGit
[+] noncoml|7 years ago|reply
Hate to be that guy, but why is this even in the front page, let alone #1?
[+] dstick|7 years ago|reply
Well I guess I’m “that guy” who finds it highly interesting as an outsider with a very high interest in search. Never knew what tools Google’s using (never bothered to look it up either) and this list is a feast of “Aha!”’s for me :)

Hopefully that answers your question ;-) Same as with a discussion about patents the other day. It’s fascinating to learn / read / understand how big businesses work as someone who’s never worked at one nor aspires to work at one, but does want to build one ;-)

[+] tomcam|7 years ago|reply
Just winging it here, but maybe because these are some of the most influential companies on the face of the Earth, they employ tens of thousands of HN users, they are the envy of maybe half of all employees on the planet, their programming systems effect everyone reading HN, and many people here are genuinely interested in what happens at ginormous corporations that employ other hackers.