I played around with a go server to do some simple scaling numbers - looking at possibly using go to implement a large-number-of-idle-connections notification server.
I found the (good) result that I could spawn a new goroutine for each incoming connection with minimal (~4k) overhead. This is pretty much what you'd expect since a goro just needs a page for it's stack if it's doing no real work. I had something like 4 VMs each making ~30k conns (from one process) to the central go server with something like 120k conns.
I found one worrying oddity however. Resource usage would spike up on the server when I shut down my client connections (e.g. ctrl-C of a client proc with ~30k conns).
Reasoning about things a bit, I think this is due to the go runtime allocating an OS thread for each goro as it goes through the socket close() blocking call. I think it has to do this to maintain concurrency. So I end up with hundreds of OS threads (each only lives long enough to close(), but I'm doing a lot at the same time).
Can anyone comment:
- is this guess as to the problem likely to be correct?
- is this "thundering herd" a problem in practice?
- are there ways to avoid this? (Other than not using a goro-per-connection, which I think it the only idiomatic way to do it?)
My situation was artificial, but I could well imagine a case that losing, say a reverse proxy, could cause a large number of connections to suddenly want to close() and it would be a shame if that overwhelmed the server.
An interesting point raised there is that if they instead used a limited thread pool for all goroutines to share when making OS calls you could produce deadlocks.
> I think this is due to the go runtime allocating an OS thread for each goro as it goes through the socket close() blocking call. I think it has to do this to maintain concurrency
I highly doubt that it is creating a thread per goro on client disconnect. If you have a minimalish example of this, the golang mailing list would be very interested in working with you to identify what went wrong and create a patch if it is an issue with the Go implementation.
While JavaScript drags the scars of its hasty standardization around with it, Go was designed very thoughtfully from the beginning, and as a result I find that it’s a pleasure to write.
This is very true. Go is a pleasure to write. In fact, it's such a pleasure then when you hit something that wasn't really well designed it's horrid.
> The biggest promise that Node makes is the ability to handle many many concurrent requests. How it does so relies entirely on a community contract: every action must be non-blocking. All code must use callbacks for any I/O handling, and one stinker can make the whole thing fall apart. Try as I might, I just can’t view a best practice as a feature.
Nonblocking I/O isn't just a "best practice" in the sense that consistent indentation is a "best practice," it's a core tenet of the Node ecosystem. Sure, you could write a Haskell library by putting everything in mutable-state monad blocks, and porting over your procedural code line-for-line. It's allowed by the language, just like blocking is allowed by Node. But the whole point of Haskell is to optimize the function-composition use case.
The Node community has the benefit of designing all its libraries from scratch with this tenet in mind, so in practice you never/rarely need to look for "stinkers" unless they're documented to be blocking. And unless they're using badly-written blocking native code, you can just grep for `Sync` to see any blocking calls.
Node: Everyone knows JavaScript, there's a massive community, there are tons of libraries, and you get very good performance
Go: No one knows this language, there's a small-but-growing community, there are enough libraries to get a lot done, and you get even better performance
Java: They are paying me (money!) to write in this language
The mainstream always lags behind significantly in every aspect of life. If everyone's using it, then you have little competitive advantage using the technology.
Go would be one of the first things I'd reach for if there's any chance server-side concurrency would be involved. The language is minimalistic and unsurprising to the extreme. A joy to program in and use.
More like: Anything web HAS to be Javascript (cause the calendar says 2013 but apparently it's 1970), no choice so oh well, we'll try Javascript on the server cause God knows using the same language everywhere is a good thing :/
Go: The language is in its infancy, growing at a slow pace for now, bears some promises that are yet to be confirmed.
Java: For some reason people still hate the language even though it's the closest to being the most versatile language around (in every single aspect that makes a good language it ranks well against the others)
It sure seems Scala's in this "python paradox" land now. My guess is that you need some startups make it big using Go to evangelize it. Google using it is interesting, but I'm not sure it makes it "cool".
Though, I'm not sure Java was ever a language you could use as a skillset filter. Hm.
I've been using Go a lot lately. It's difficult to overstate just how much simpler it makes writing highly-concurrent server-type programs. Entire classes of bugs, issues, and puzzles just vanish.
I've been using Go a lot recently as well, and it's rapidly become my go-to language (no pun intended) for a lot of problems, even when concurrency is not involved.
The biggest thing Go gives me is that it's really easy to manage code bases that grow organically - refactoring a project that grows from 50 LOC to 5000 LOC is almost painless in Go - no other language that I've seen has dealt with this aspect of code development so well.
> There’s no arguing about whether to use semicolons, or putting your commas at the front of the line — the language knows what it wants. It’s a built-in hipster suppression mechanism.
Clojure is another nice alternative for fast servers, and using a concurrent, immutable and functional language is a huge win. http-kit is a good example of such server: http://http-kit.org/
As noted above - with ridiculously large values for ab this one crashes (although I didn't compile with O-parameters). I think this (and the other haskell solution) ran out of resources.
Both the go and nodejs versions completed without problems.
I was a little disappointed -- I was actually hoping I'd see comparable performance -- even if it is a silly test.
I think it is interesting that simple, idomatic code in go and nodejs didn't crash -- not sure what assumptions might be "wrong" in the underlying haskell code (I'm guessing if anything should be "fixed" it is in the web server libraries used).
I have never understood the focus on speed as a selling point for Node. It may well be very fast, but it seems to me that the primary selling points would be the ability to share code between client and server and that you can start coding server side without learning a new language if all you know is JavaScript.
First off, while some sharing between client and server happens, that tends to be an edge case in my experience. The roles of client and server, and APIs available to each, are rather different. I.e. the environment of the browser and node.js server aren't homogenous.
Second, "start coding in XXX without learning a new language" is a terrible selling point. I've seen this thinking appeal to misguided PHB-types and witnessed the result: immense organizational damage. In my experience, this isn't a necessary or sufficient selling point to good developers. Learning a new language just isn't that hard, and a big part of a shift like this is actually in learning the new environment's paradigms, APIs, and best practices.
To make the latter point more strongly: if you're having doubts about your ability to pick up a new language, definitely take some time to learn a few new languages. Do a tutorial, play with a few small projects, enough to get the flavor of the language. Your hackery will benefit immensely from this, even when you return to your primary language.
Compared to Python and Ruby, node.js is quite fast by the simple virtue of having a JIT (in the most common implementation anyway; of course there are JITs for Python and Ruby but they aren't the mainline implementation).
Benchmarking on macbooks is often an exercise in testing the mediocre default configuration of the network stack, not your language. My macbook pro gets 4k rps with apache, node, go, and nginx. YMMV and all that, but I'm always wary.
I did this test a while ago on my laptop running Linux Mint. With higher concurrency levels (IIRC, about 1k simultaneous connections), I saw just the opposite, with cracks starting to show and Go performing worse than Node, although the V8 engine did consume more memory.
check the memory usage. When I did the comparison, the node server's memory climbed steadily up to about 50MB, while the Go server's memory stabilized at about 17MB.
fyi, in the replies of the original post someone got totally different results. with v0.6 it was significantly faster than Go and with 0.8 was on the same level
I run node/express for most of my web servers and each takes up about 10-15mb RAM. They're very basic no fluff. Anyone know what comparable mem footprint in Go?
I find this post paired with this thread confusing. Yes, Go is tempting and I'd like to try it since a lot of people get quickly into flow with Go, the "package manager is so great" and "everything is just a breath of fresh air".
But what I don't like: the negativity against Node and omitting some facts. In the replies of the orignal post a guy tested two (!) times Node and once it was significantly faster (v0.6) and once it had same speed (v8.0). So, why has mjijackson such different results in this thread at the top?? And maybe we should test it on real servers and not on a MBA. Moreover, we have here some micro benchmark which possibly doesn't reflect reality well. Don't get me wrong, I appreciate any benchmarking between languages but then please do it right and make no propaganda out of it. Further, Go's package manager seems to be nice but it does NOT have version control. How do you want to use this in a serious production environment. Maybe version control will come (but then tell how without loosing its flexibility) or not but this is something serious and definitely not an alternative to any server environment except for some mini services.
EDIT: downvoting is silly, propaganda and won't help the Go community in getting more credibility, better do some further benchmarks; otherwise this post/thread is full of distinct misinformation and should be closed
Go's package manager does have version control. It looks for specially named branches (different ones depending on the version of go you have). The upstream authors can provide a different version of the software for different releases of Go.
If you want to lock down the versions of all the software you're deploying in your organization, that's easy to do too. Just "git clone" all of the libraries you use to some internal server (and/or github repos), and change the URLs to point to that server. You control when everything gets updated.
Golang builds static binaries anyway. So if you test a binary and it works, you just copy it to all the servers you care about and you're done. If you're in a small and informal shop, maybe you don't need to mirror every repository. Due to the nature of git, if the upstream repo ever gets deleted, you can fall back on a local copy of it anyway.
This is all very much in contrast to languages like Java where keeping around the proper version of every jar and carefully deploying that version (and only that version!) on each server is big deal (and despite OSGI, still very much an unsolved problem.)
One advantage of node that wasn't mentioned is the ability to share server side and client side code. Avoiding discrepancies in the same form validation written in two different languages can often be more important than performance gains in server applications.
I am working on a project that has a few thousand lines of Go and compilation takes a second or so. Maybe less; it's fast enough that I don't really think about the fact that it's compiling unless I've done something that it complains about (which is awesome).
[+] [-] jbert|13 years ago|reply
I found the (good) result that I could spawn a new goroutine for each incoming connection with minimal (~4k) overhead. This is pretty much what you'd expect since a goro just needs a page for it's stack if it's doing no real work. I had something like 4 VMs each making ~30k conns (from one process) to the central go server with something like 120k conns.
I found one worrying oddity however. Resource usage would spike up on the server when I shut down my client connections (e.g. ctrl-C of a client proc with ~30k conns).
Reasoning about things a bit, I think this is due to the go runtime allocating an OS thread for each goro as it goes through the socket close() blocking call. I think it has to do this to maintain concurrency. So I end up with hundreds of OS threads (each only lives long enough to close(), but I'm doing a lot at the same time).
Can anyone comment:
- is this guess as to the problem likely to be correct?
- is this "thundering herd" a problem in practice?
- are there ways to avoid this? (Other than not using a goro-per-connection, which I think it the only idiomatic way to do it?)
My situation was artificial, but I could well imagine a case that losing, say a reverse proxy, could cause a large number of connections to suddenly want to close() and it would be a shame if that overwhelmed the server.
[+] [-] evmar|13 years ago|reply
https://code.google.com/p/go/issues/detail?id=4056
An interesting point raised there is that if they instead used a limited thread pool for all goroutines to share when making OS calls you could produce deadlocks.
[+] [-] aaronblohowiak|13 years ago|reply
I highly doubt that it is creating a thread per goro on client disconnect. If you have a minimalish example of this, the golang mailing list would be very interested in working with you to identify what went wrong and create a patch if it is an issue with the Go implementation.
[+] [-] jgrahamc|13 years ago|reply
This is very true. Go is a pleasure to write. In fact, it's such a pleasure then when you hit something that wasn't really well designed it's horrid.
[+] [-] btown|13 years ago|reply
Nonblocking I/O isn't just a "best practice" in the sense that consistent indentation is a "best practice," it's a core tenet of the Node ecosystem. Sure, you could write a Haskell library by putting everything in mutable-state monad blocks, and porting over your procedural code line-for-line. It's allowed by the language, just like blocking is allowed by Node. But the whole point of Haskell is to optimize the function-composition use case.
The Node community has the benefit of designing all its libraries from scratch with this tenet in mind, so in practice you never/rarely need to look for "stinkers" unless they're documented to be blocking. And unless they're using badly-written blocking native code, you can just grep for `Sync` to see any blocking calls.
[+] [-] jacobmarble|13 years ago|reply
Go: No one knows this language, there's a small-but-growing community, there are enough libraries to get a lot done, and you get even better performance
Java: They are paying me (money!) to write in this language
[+] [-] mseepgood|13 years ago|reply
[+] [-] mattgreenrocks|13 years ago|reply
Go would be one of the first things I'd reach for if there's any chance server-side concurrency would be involved. The language is minimalistic and unsurprising to the extreme. A joy to program in and use.
[+] [-] VeejayRampay|13 years ago|reply
Go: The language is in its infancy, growing at a slow pace for now, bears some promises that are yet to be confirmed.
Java: For some reason people still hate the language even though it's the closest to being the most versatile language around (in every single aspect that makes a good language it ranks well against the others)
[+] [-] tristan_juricek|13 years ago|reply
http://www.paulgraham.com/pypar.html
It sure seems Scala's in this "python paradox" land now. My guess is that you need some startups make it big using Go to evangelize it. Google using it is interesting, but I'm not sure it makes it "cool".
Though, I'm not sure Java was ever a language you could use as a skillset filter. Hm.
[+] [-] unknown|13 years ago|reply
[deleted]
[+] [-] burke|13 years ago|reply
[+] [-] chimeracoder|13 years ago|reply
The biggest thing Go gives me is that it's really easy to manage code bases that grow organically - refactoring a project that grows from 50 LOC to 5000 LOC is almost painless in Go - no other language that I've seen has dealt with this aspect of code development so well.
[+] [-] tanel|13 years ago|reply
[+] [-] stcredzero|13 years ago|reply
Major point for saving man hours right there.
[+] [-] SeanDav|13 years ago|reply
Personally I hope that Go does just as well, if not a lot better. I am a bit of a fan of both.
[+] [-] islon|13 years ago|reply
[+] [-] jamwt|13 years ago|reply
https://gist.github.com/jamwt/5017172
Haskell was ghc 7.6.1 with ghc --make -O2
Go is go1.0.2 with "go build".
[+] [-] e12e|13 years ago|reply
Both the go and nodejs versions completed without problems.
I was a little disappointed -- I was actually hoping I'd see comparable performance -- even if it is a silly test.
I think it is interesting that simple, idomatic code in go and nodejs didn't crash -- not sure what assumptions might be "wrong" in the underlying haskell code (I'm guessing if anything should be "fixed" it is in the web server libraries used).
[+] [-] gnuvince|13 years ago|reply
[+] [-] hrwl|13 years ago|reply
[+] [-] saidajigumi|13 years ago|reply
Second, "start coding in XXX without learning a new language" is a terrible selling point. I've seen this thinking appeal to misguided PHB-types and witnessed the result: immense organizational damage. In my experience, this isn't a necessary or sufficient selling point to good developers. Learning a new language just isn't that hard, and a big part of a shift like this is actually in learning the new environment's paradigms, APIs, and best practices.
To make the latter point more strongly: if you're having doubts about your ability to pick up a new language, definitely take some time to learn a few new languages. Do a tutorial, play with a few small projects, enough to get the flavor of the language. Your hackery will benefit immensely from this, even when you return to your primary language.
[+] [-] pcwalton|13 years ago|reply
[+] [-] ebiester|13 years ago|reply
[+] [-] mjijackson|13 years ago|reply
First, go:
Secondly, node.js: Not only does go serve the traffic more quickly, but it also has a much lower standard deviation between slow and long requests. Impressive.[+] [-] fizx|13 years ago|reply
[+] [-] oinksoft|13 years ago|reply
[+] [-] kintamanimatt|13 years ago|reply
[+] [-] zemo|13 years ago|reply
[+] [-] davidw|13 years ago|reply
[+] [-] yaix|13 years ago|reply
[+] [-] tferris|13 years ago|reply
[+] [-] weix|13 years ago|reply
[+] [-] dpweb|13 years ago|reply
[+] [-] codygman|13 years ago|reply
[+] [-] tferris|13 years ago|reply
But what I don't like: the negativity against Node and omitting some facts. In the replies of the orignal post a guy tested two (!) times Node and once it was significantly faster (v0.6) and once it had same speed (v8.0). So, why has mjijackson such different results in this thread at the top?? And maybe we should test it on real servers and not on a MBA. Moreover, we have here some micro benchmark which possibly doesn't reflect reality well. Don't get me wrong, I appreciate any benchmarking between languages but then please do it right and make no propaganda out of it. Further, Go's package manager seems to be nice but it does NOT have version control. How do you want to use this in a serious production environment. Maybe version control will come (but then tell how without loosing its flexibility) or not but this is something serious and definitely not an alternative to any server environment except for some mini services.
EDIT: downvoting is silly, propaganda and won't help the Go community in getting more credibility, better do some further benchmarks; otherwise this post/thread is full of distinct misinformation and should be closed
[+] [-] cmccabe|13 years ago|reply
If you want to lock down the versions of all the software you're deploying in your organization, that's easy to do too. Just "git clone" all of the libraries you use to some internal server (and/or github repos), and change the URLs to point to that server. You control when everything gets updated.
Golang builds static binaries anyway. So if you test a binary and it works, you just copy it to all the servers you care about and you're done. If you're in a small and informal shop, maybe you don't need to mirror every repository. Due to the nature of git, if the upstream repo ever gets deleted, you can fall back on a local copy of it anyway.
This is all very much in contrast to languages like Java where keeping around the proper version of every jar and carefully deploying that version (and only that version!) on each server is big deal (and despite OSGI, still very much an unsolved problem.)
[+] [-] unknown|13 years ago|reply
[deleted]
[+] [-] WhaleFood|13 years ago|reply
[+] [-] dgudkov|13 years ago|reply
Does anybody from HNers use Go with websockets? What package do you use?
[+] [-] stesch|13 years ago|reply
[+] [-] tferris|13 years ago|reply
[+] [-] grey-area|13 years ago|reply
[+] [-] icey|13 years ago|reply
[+] [-] babuskov|13 years ago|reply
[+] [-] logn|13 years ago|reply