top | item 9142819

From a Ruby monolith to microservices in Go

116 points| nexneo | 11 years ago |sourcegraph.com | reply

89 comments

order
[+] matthewmacleod|11 years ago|reply
Rewriting a monolithic platform as a set of microservices will almost always result in improvements from every perspective. You're replacing legacy code with new, cruft-free code; you're de-coupling parts of the system; you're building a more scalable design.

For example, if you're in the situation where you've gone from 12GB to 5MB of memory consumption, then you have a design issue and not a language one. That's not Ruby's fault.

We are also in the process of converting several monoliths into a microservice-driven system, but we are doing so with Ruby. It remains great language to use for building web apps, and I suspect some of the push back against it is caused by a dislike of the cliché giant Rails monolith that we've all come across, with hundreds of gem dependencies and so on. Rubby and Rails have a habit of encouraging such bad designs primarily through ease of use, but it's hard to see that as a problem with Ruby.

Go is a good tool for doing the same, and single-binary deployment is pretty magic. I'm sure it'll be supported for ages, so if you want to re-tool to use that then go for it! But Ruby's a viable option too, and it's important not to place blame for bad app design in the wrong place.

[+] neikos|11 years ago|reply
> 12GB to 5MB of memory consumption, then you have a design issue and not a language one. That's not Ruby's fault.

I thought the same thing, I find that figure also very hard to believe. Perhaps I just lack experience, but if you get 12G of memory usage that drops to 5M after rewriting it (whether another language or not) seems to me as if they did something pretty wrong in their previous iteration.

[+] apunic|11 years ago|reply
I feel uncomfortable when reading your post. Passive aggressive, attacking the OP's architectural decisions, assuming that he didn't have any clue building a proper Ruby system. I am not a fan of Go, not at all, but I know that Ruby has significant issues which the OP also outlined quite well and you seem to just ignore them.

But Paul Graham knows exactly why you are sticking to the past (just replace 'Blub' by 'Ruby'):

As long as our hypothetical Blub programmer is looking down the power continuum, he knows he's looking down. Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.

When we switch to the point of view of a programmer using any of the languages higher up the power continuum, however, we find that he in turn looks down upon Blub. How can you get anything done in Blub? It doesn't even have y.

By induction, the only programmers in a position to see all the differences in power between the various languages are those who understand the most powerful one. (This is probably what Eric Raymond meant about Lisp making you a better programmer.) You can't trust the opinions of the others, because of the Blub paradox: they're satisfied with whatever language they happen to use, because it dictates the way they think about programs.

Source: http://www.paulgraham.com/avg.html

[+] pjmlp|11 years ago|reply
What is so magic about a compiler feature generally available in all compilers that target native code?
[+] ukigumo|11 years ago|reply
>Rewriting a monolithic platform as a set of microservices will almost always result in improvements from every perspective

That is one dangerous assumption you made there mr. Performance (in terms of response times, IO waits, etc) can be severely affected in distributed designs.

[+] bonn1|11 years ago|reply
Many folks in this thread are expressing that it's the OP's fault and not Ruby's.

The OP made his post as objective as possible, politely written and nobody is loosing his face, neither Ruby as a language nor the Ruby community. But people are still resentful and fire back.

I believe it's not a discussion about Ruby anymore, its strengths, its weaknesses and wether Ruby still fits in 2015—no it's the fear of people that their core competence might be less worth in future. That all the time and energy they invested all the years into one language might be not a good investment anymore. The fear that better tech will replace their tech and that they have to start at zero again.

I know I am going to be heavily downvoted for this post. Before you downvote just think a second again why you'll downvote me.

[+] matthewmacleod|11 years ago|reply
I don't think that's a reasonable view.

The point is that articles about rebuilding entire systems in different languages conflate two things — an architecture change, and a language changes.

This often results in the sort of appraisal you see here – 'Go is better for us than Ruby, because we completely rebuilt a system and it's now better'. That's got some value, but it's frustrating to see it used as something of an absolute metric regarding the suitability of a language when it's conflated by the far more important architectural changes.

So I think your view that there are a bunch of developers who are afraid of the future is a little shallow; rather I suspect there are a bunch of developers who saw exactly the same sort of thing happen when Ruby was becoming popular, and realise that extracting helpful data when there's an ongoing fad ('we're jumping from X to Go') is rather difficult.

[+] dkarapetyan|11 years ago|reply
Others already mentioned several places you made invalid assumptions. Any good engineer worth their salt is language agnostic. I agree there are some that are threatened simply because their favorite language is no longer hip. I personally couldn't care less what the next language du jour is, be it Go or otherwise. What I do care about is that when posts like these are written they confound too many things and squander away an opportunity to let the larger programmer community double check all the numbers and verify that indeed the language/runtime was the culprit instead of something else. As it stands this is just a good story to justify the sunk cost of a re-write and nothing more.
[+] bobofettfett|11 years ago|reply
Ruby is the new Java everyone is bashing. Go is the new Ruby everyone is hyping.

If you replace every mention of Ruby with Java, and every mention of Go with Ruby in the comments on HN, then you've invented a time machine to go back 10 years.

And if you do this with C, welcome to 1995.

[+] sheepmullet|11 years ago|reply
We are a fad driven industry.

They could have rewrote it in java in a similar amount of time, and with the same results. And they could hire from the large pool of expert java devs. Of course that's not "cool".

[+] dkarapetyan|11 years ago|reply
So many wrongs and confounding variables that I don't even know where to begin but I'll try anyway.

1) Ruby is a complex language: Ruby is quite simple. Everything is an object, every method call is a message send that follows the exact same protocol of dispatch. Also, completely irrelevant to the main thrust of the article.

2) Memory consumption and speed: Can't really refute anything since by their own admission they were running a monolithic system.

3) Concurrency: Plenty of options. I like celluloid personally and if you switch to JRuby then you can do all sorts of other stuff as well with automatically managed thread pools.

4) Backcompat: Ruby is not an enterprise ecosystem. Some people call that speed and agility. In their case they really should have gone with Java since they are basically serving an enterprise customer. This is not a knock on Java. I think the language and its ecosystem is by design made for stuff like this.

5) PDF generation: I'm guessing the Ruby code was calling some C library to do the rendering and they swapped out all of it for Go. I don't know what library Go is using to render PDFs if it is the same library then there is something interesting happening here. If it is a different library then they are comparing apples and oranges and there's nothing to see here.

6) Alert service: Completely bound by I/O so doesn't matter what language you use. Bad design for keeping many processes around because you only need to spin them up during spikey times which can be easily handled with better engineering. Also eventmachine and celluloid for the worker internals would have been another option so again they didn't do their research.

7) Gradebook service: No idea again. Thin on details so nothing to refute really.

8) Deployment: Sure. Why not. Single binary is easier or JRuby and a single jar which is just as easy.

In conclusion, they re-wrote everything and I'm sure removed a lot of cruft in the process and called it a victory for Go. In a year or so they're going to run into similar issues and Go is not going to save them because from what I read all I saw was bad engineers blaming their tools.

[+] patio11|11 years ago|reply
This comment is more aggressive than comments should be on HN.

I would exercise a bit of humility when commenting about other people's architectural decisions: they have bled all over that codebase and infrastructure, but I have not. They see it in their dreams, and I do not. They may from time to time decide to tell the community about their adventures. That is an excellent opportunity for me to learn from another engineering team's perspective and experiences, rather than an opportunity for me to demonstrate my intellectual superiority over them. They do not, as a function of having written about their experiences, owe me responses to my off-the-cuff analysis of their engineering tradeoffs.

[+] unknown|11 years ago|reply

[deleted]

[+] Argorak|11 years ago|reply
Every time I read something like that, I get the feeling that "Monolith to Microservices" is the pattern and the language doesn't matter.

At the beginning, a Monolith has huge advantages: easy to deploy, all in one place, smaller operational overhead, easier to manage with unclear requirements.

Later, it's easier to find out what you can split out and you can learn how to route and manage things piece by piece.

[+] nexneo|11 years ago|reply
You exactly captured my sentiment, when I started doing this I don't realized all benefits. Because of Go's simplicity in deploying and maintaining, many small apps doesn't add much overhead. Now you can scale individual component.

Only risk is, you break things into many component then you should so balance is required.

[+] sgt|11 years ago|reply
After refactoring the alert feature into a Go microservice, they’ve seen the following improvements

The same number of workers (that required 12 GB in Ruby) now requires just 5 MB of memory.

Not that I don't approve of pro-Go articles, but doesn't this maybe indicate that something was wrong or inefficient in the Ruby implementation?

[+] Argorak|11 years ago|reply
We did some measurements with Padrino applications in the wild and found that all of them could be improved a lot in relatively simple ways. E.g. by not loading multiple JSON parsers. Padrino being a rather slim framework on top of Sinatra, that often meant getting below 80mb memory usage (on a full stack!).

Our takeway was that the (Rails) monolith model tends to put a lot of things into one process space. Controlling library loads can be a chore.

[+] est|11 years ago|reply
YMMV, my production system, each unicorn worker consumes about 80M RES, so for 12G that's roughly about 150 workers, not that many.
[+] grey-area|11 years ago|reply
I've seen a factor of 10 or so reduction in memory use for similar implementations in Go and Ruby, so it's not entirely down to the implementation.
[+] tonyarkles|11 years ago|reply
Over the last few years, I've tried implementing a few things like this in Go, but the problem I keep running into is the tooling. It'd be awesome to know which modules they used for things like URL routing and application structure. There seems to be a lot of options with no clear winners.

Rails is opinionated, yes, but I'm generally pretty satisfied with their opinions, and it's nice to have a full Batteries-included package.

[+] nexneo|11 years ago|reply
Initially I used https://revel.github.io because I wanted to use many inbuilt things and hot reload.

Later I started use just standard library with http://www.gorillatoolkit.org and used fswatch for hot reload. Now there are lot of alternative in routers but no clear winner.(and thats good thing) I will suggest go with standard library and simple router but be ready to revert back to framework like revel if that doesn't workout.

[+] sfeng|11 years ago|reply
mux is probably the most popular routing library: http://www.gorillatoolkit.org/pkg/mux

As far as application structure, I'm not really sure what you mean. Go projects are divided into packages which can be any number of files. At some point you'll have a main package which has a main() function which gets called to start your server. It's similar to C, for example.

That said, the variety can also be a plus. Some people chose to use something like Martini that does dependency injection. Others think that's not a great idea.

[+] saintfiends|11 years ago|reply
From the looks of it I assume they are using HTTP/1.x as the communication protocol. I've always wondered why most use HTTP for micro-services instead of JSON-RPC (or any other encoding) over a TCP/IP socket. What are the benefits?
[+] patio11|11 years ago|reply
It's easy to understand for developers and "plays well with everything."

Don't underestimate the importance of toolchain support. Want to spin up a Ruby microservice which speaks HTTP? Sinatra and you're done. Go? Whatever the Go HTTP library is and you're done. Need to interact with it from the command line? Curl and you're done. How about from an automated testing script written in Ruby? Net:HTTP/HTTParty and you're done. Thinking about how to deploy it vis-a-vis firewall/etc? Port 80 and you're done. Need coarse-grained access logging? Built into Nginx/etc already; you're done. Uptime monitoring? Expose a /monitor endpoint; provide URL to existing monitoring solution; you're done. Deployment orchestration? Use Capistrano/shell scripts/whatever you already use for the app proper and you're done. Encrypted transport? HTTPS and you're done. Auth/auth? A few options that you're very well-acquainted with and are known to be consumable on both ends of the service trivially.

Edit to note: I'm assuming, in the above, that one is already sold on the benefits of a microservice architecture for one's particular app/team and is deciding on transport layer for the architecture. FWIW, I run ~4 distinct applications, and most of them are comparatively large monolithic Rails apps. My company's bookkeeping runs in the same memory space as bingo card PDF generation.

Things that would tilt me more towards microservices include a very rapid deployment pace, large engineering organizations which multiple distinct teams which each want to be able to control deploys/architecture decisions, particular hotspots in the application which just don't play well with one's main technology stack (as apparently happened in the app featured in this article), etc.

[+] Animats|11 years ago|reply
Their "microservices" are kind of macro. Rendering a PDF file and sending an email are reasonably large operations, so the overhead of local HTTP isn't that bad.

For a finer-grained application, such as the way Facebook generates pages from about 20 servers feeding data in to be assembled, that wouldn't scale. Facebook has Thrift and Google has protocol buffers. Both work, but result in a somewhat complex build procedure with a pre-compiler for interface specs.

(Serializing and de-serializing data into and out of databases and networks is either interpreted and slow, or compiled through clunky tools. Technology in this important area still sucks. There's a long history of clunky solutions to this problem, from DCOM, CORBA, and ASN.1, to SOAP to REST/JSON.

Part of the problem is political. If you expose an API defined in any of those except REST/JSON, there's a formal, checkable definition of the API in machine readable format. If the API doesn't match the spec, the server is broken. With a REST/JSON API, you get to blame the caller for doing it wrong. This is convenient for API developers.)

[+] cmelbye|11 years ago|reply
Simplicity.
[+] pjmlp|11 years ago|reply
So a monolith application gets ported to microservices from an interpreter based version of Ruby to a compiled version of Go and the author is surprised about the gains.

Why do people keep getting amazed about the performance they get when using native code in their applications instead of a naive interpreter?

Not bashing Go, the title could be "From a Ruby monolith to microservices in <pick your language with native compiler available>".

Hey, this could probably even be possible in Ruby, if something like RubyMotion was generally available.

[+] nexneo|11 years ago|reply
Exactly right any language could have worked, and we didn't replaced Ruby, just few part of our big application is rewritten.

I did in Go not because of language is better, ecosystem and culture is better but not language part. I will suggest you to do small real life project in Go.

[+] talwai|11 years ago|reply
Would someone care the elaborate on the following?: "They use nginx to route requests to either the microservices or the main Ruby app... The Go microservices treat the Ruby application as an API client or upstream proxy, which allows authentication to be simple."

What I read this to mean is that the Ruby app accesses the microservices over HTTP with some kind of token-based authentication. This would make sense if there some form of shared session store which each microservice could validate the token against. What I'm confused about is how nginx fits into this? Is it routing requests directly from the client to microservices? Or is it a proxy layer in between the Ruby app server and the microservices which allows the Ruby app to forward auth-token information? In the former scenario, would the Ruby app server be pinging the same nginx instance that forwards it requests in the first place?

[+] nexneo|11 years ago|reply
I agree that line bit confusing, Ruby app doesn't connect with Go service directly, every requests goes through balancer and then nginx router. Nginx routes then using simple location directive. Authentication is not shared, its either token based or Go service act as proxy and send request to Ruby app and modify response received from Ruby app before sending to client(i.e covering html to pdf)
[+] bdcravens|11 years ago|reply
I'm trying to decompose a Ruby monolith into microservices, but is it necessary to change languages? For my use case, I'm less concerned with languages than I am with libraries (AWS library, Sidekiq, Devise, deployment toolchains, etc).
[+] grey-area|11 years ago|reply
Of course it's not necessary to change language, but one nice thing about microservices is they don't have to be the same language.

You might see significant advantages moving to ruby microservices if you have high traffic and a complex monolithic app which would benefit from being split up. They are not a panacea of course as they introduce significant complexity, whatever language they are implemented in - it's a matter of trade-offs and once your monolith reaches a certain size it might be worth splitting it up.

Re libraries, most of the ones you mention have analogues in go or are pretty simple to replace, save AWS, which is apparently coming soon:

AWS - https://aws.amazon.com/blogs/aws/coming-soon-aws-sdk-for-go/

Sidekiq - go myFunc()

Devise - bcrypt.CompareHashAndPassword + the mailer functions

Deployment - Rsync or Ansible - this is simpler in go as all you require are your executable + templates

[+] bobofettfett|11 years ago|reply
As they say: "Your new boy/girlfriend is better because you are better."
[+] jackkinsella|11 years ago|reply
Going from 12GB to 5MB of memory consumption could probably have been achieved through moving the alerting components to a dedicated Ruby (non-Rails) / Linux script configured to read their queue.

Time taken: 1-2 days.

Their high memory consumption was caused by large numbers of unneeded dependencies (e.g. Rails) included in their alerting components. By refactoring so drastically into a separate Go service, they may have squandered tens of thousands of dollars of company/investor money in retooling, retraining and learning a new environment. Good for the programmers, toxic for the company.

[+] ChikkaChiChi|11 years ago|reply
Ruby enthusiasts seem to get apoplectic at any mention of a posting where someone chooses another language over Ruby. I'm sure Ruby is just great at what it does, but aren't we always talking about the right tool for the job?

I don't think anyone disagrees that refactoring and evolving the code would have resulted in gains even if they stayed in Ruby. I'm sure most of us would even agree that if you wrote something, then immediately scrapped it and started over, you might already be able to improve on your original based on some new knowledge you've gained from the experience.

When a company does this, they not only learn new things about the language they are converting to, but they are gaining new and valuable insight into the language they already have used. Posts like this are valuable to all of us because they are sharing their experiences; whether you agree with their choices or not.

[+] bobofettfett|11 years ago|reply
If you move from Ruby to Go and this works better for you, then choosing Ruby in the first place might - if it was not about a prototype or getting VC money or only found Ruby developers - be a failure on your part.
[+] pmontra|11 years ago|reply
Ruby got them customers within the bounds of their time and money budget so it did its job. Nothing in the post suggests they have regrets about it but it would be interesting to hear from them.
[+] bobofettfett|11 years ago|reply
Companies that invest in rewriting everything from Ruby into Go instead of working on the business model seem to have a lot of money and a secure future and no competition.