"... as a Python programmer, I was the member of an elite cabal of superhuman ultranerds, smarter than those childish Rails/JavaScript/PHP/whatever developers that couldn’t write a bubble sort or comprehend even basic algorithmic complexity, but more in touch with reality than the grey-bearded wizards of Lisp/Haskell/whatever that sat in their caves/towers/whatever solving contrived, nonexistent problems for people that don’t exist, or those insane Erlang programmers who are content writing sumerian cuneiform all day long."
Concurrency support is possible in Python, without gevent-style monkey patching (or callback madness). Have a look at concurrent.futures and http://www.dabeaz.com/coroutines/index.html. It really needs a lot more work before it's part of the language's DNA, though. Also, pypy needs much wider adoption as quickly as possible, to address the speed problems (and its STM branch holds huge potential).
For me, Go's major shortcoming is its community's lack of focus on readability as compared to Python.
"For me, Go's major shortcoming is its community's lack of focus on readability as compared to Python."
Can you be a bit more concrete here? Because whether you like the code styles enforced by gofmt and the compiler itself or not, Go is the most consistently readable language I've ever used (once you adapt yourself to the language). IMO, this seems like an especially odd problem for someone to have with Go.
that was deliberate. There has been a glut of "introduction to Go" blog posts, and the documentation for introductory Go from the Go Team is quite good, so I didn't feel the need to author another introductory post. Most of the stories about why people use Go are bigger, established companies, so I wanted to show how it was perceived by a different audience. I don't remember where I saw it, but someone talked about using Go in contrast with Java, and the overwhelming reaction was "that's cool but I'm a Python programmer, I'm curious what Python programmers think", so I was just trying to give it that angle.
>Have a look at concurrent.futures
but this is part of the problem: it's a library. Concurrency deserves language-level support. Adding concurrency at the library level is like adding an object system at the library level.
I wonder if parent has ever coded with Go? You have to maybe learn a new syntax (partially), but once you have its incredibly readable. for instance, encapsulation is handled by giving semantic significance to the case (upper or lower) of the first character in a variable name. So "myFunc" is local and not visible whereas "MyFunc" is publicly accessible. This and many other features make Go extremely readable for me at least, although I admit, I did have to read the docs and program some of my own projects before I really got accustomed to the syntax.
If someone created a debugging environment for Go based on a VM, which also let one recompile source from within the debugger then continue execution, then it would be, for all intents and purposes, as productive and immediate as the old Smalltalk environments. You'd have the same small-grained cycles of inspecting state, modifying code, rewinding the stack to the place of your choosing, then getting immediate feedback.
Source code changes could be saved as log-structured patch files, which could then be thrown away or applied to the source tree as desired. One could also steal some ideas from the Smalltalk Change Log tool by adding similar editing, search, and filtering commands.
With tools like this, one could recompile for "interpreted debug mode," have complete visibility and control of runtime state to debug a problem, then take the resulting patch file and apply it to the source tree. It would be a best of both worlds scenario -- all the enhanced debugging of an interpreted runtime with the type safety and speed of compiled code.
Interactive coding is a very powerful way of working, it's one of the reasons I'm so productive in Mathematica.
At a Go talk he did, I raised the idea with Russ Cox of having a "repl" package that would allow one to instrument a running program with a live REPL to do debugging and development on it.
The reflect package is powerful enough to make some of that relatively straightforward, but one major problem is that Go can't construct new types at runtime. Another challenge would be dynamic linking of new code -- because the Go toolchain doesn't support dynamic linking there would seem to be no hope of say entering anonymous functions on the REPL.
Unless one builds a full Go interpreter -- but who wants to be in the business of maintaining a fully compliant Go interpreter that can interoperate with the Go runtime?
Still, calling existing functions and banging on variables would be pretty useful.
It would help a bit if the article included at least roughly equivalent Go code next to the Python code. The Go code is wordier, but maybe it takes less time to write because it doesn't require as many decisions (libraries etc.) as with Python.
package main;
import (
"fmt"
"net"
)
func main() {
hosts := []string { "www.google.com", "www.example.com", "www.python.org" }
c := make(chan string)
for _, h := range(hosts) {
go get_ip(h, c)
}
for i := 0; i < 3; i++ {
fmt.Println(<-c)
}
}
func get_ip(host string, c chan string) {
addrs, err := net.LookupHost(host)
if err != nil {
fmt.Println("Host not found:", host)
c <- host + ": <error>"
return
}
c <- host + ": " + addrs[0]
}
This is a very good example of the type of thing I'm talking about. Nobody prompted you to do so, but you gracefully handled the case of a DNS failure in your code, because the control path was obvious throughout. Go makes this type of error handling a topic very early on in the literature. I find this type of clarity when structuring concurrent code to be very helpful in minimizing subtle concurrency bugs.
Another gonatic? I'm immediately reminded of the days when everyone who liked D proclaimed it would overtake C++ and rule the world with terrible, contrived examples: "let me show you why my language is better than yours by completely misunderstanding how to solve a problem, then implementing that broken/overengineered solution in your language, then compare it to something my language's API can do for me, just so we can see how much simpler that bad solution is in my language!"
The insight to async I/O is that this server, for this I/O bound task, will perform as well as your GOMAXPROCS example:
But it's a lot simpler. You don't need to bolt on parallelism when you don't actually need parallelism. There are valid reasons to use Go, and there are valid complaints against Node. I don't see any of either here.
I'd like to see more detail about the real problem he was facing in python. It sounds like he mostly wants a non-blocking background job "I don’t want to set up another daemon, I just want to send some email in the background!" Why not just use Queue (thread-safe, waitable, built-in) and a background thread(pool)?
There's an awful global lock on the actual execution of python code, but unless the problem is performance or contention worrying about it is premature.
Not only that, but there's a multiprocess Queue class as well that is a drop in replacement for the threaded one. So if you are having problems with GIL contention (unlikely, but possible), you can still just as easily use multiple processes.
You should update the node.js docs if you consider the original JS snippet ugly, because that's where I pulled the original code sample from verbatim.
In any case, it's still not an apples to apples comparison. The node.js example forks separate processes that share nothing, while in the Go example it all runs in one process so the HTTP handlers can communicate with each other via channels or other shared data structures.
ah, so, I don't think I wrote that section particularly well. I hold Erlang and Haskell in very high regard; I was really trying to mock my own superiority complex, which I think is fairly common in the Python community. I guess what I was really trying to get at is that Go is verymuch an industry language. Haskell I would argue against being an industry language not because it can't be used in industry; that's not at all what I mean. I don't mean to compare them on their technical merits; what I mean to say is that Go has a broad appeal, in that it is welcoming to beginners, it has large corporate support, and that it is also technically capable. Of the programmers that I know, the Haskell programmers are typically capable of solving the hardest problems, but they also tend to be the most academic. The Erlang programmers typically build the most stable software, but I think Erlang is intimidating to less-experienced programmers (the same is true of Haskell, actually).
That could have been written better. Thanks for the feedback.
For small tasks in the background of a web request, you can just, you know, use a worker thread. This author seemed like he didn't even try regular threads, and went directly from one hyped meme to another. It's often the case that the GIL isn't much of an issue.
I'd like to start porting a few of my little scripts to Go (that do pretty poor messy parallelism in Python), and I was wondering what a good resource/book type thing would be for people learning Go. Like, the equivalent of learn you some haskell or whatnot. Also some advice on "wtf library do I use for this".
Is there some sort of Go package manager? How does all this shit work?
Yes, the Go tool comes with package-management capabilities. "go get labix.org/v2/mgo" will download the very excellent MongoDB driver mgo, for example.
"As a Django developer, there wasn’t a straightforward and obvious way to just do things in the background on a page request. People suggested I try Celery, but I didn’t like that option at all. A distributed task queue? What? I just want to do something in the background without making the user wait; I don’t need some super comprehensive ultimate computing machine. The whole notion that I would need to set up and configure one of these supported brokers made my spidey sense tingle"
It's not really that hard. I just latch on to a broker that I'm already using elsewhere in my stack (Redis). Celery makes it super simple to run a command in the background.
I just want to do something in the background without making the user wait
What's more, the strategy of just spawning a thread to do with async processing doesn't scale. Once you hit the limit of requests that a machine can process, you'll need a distributed work queue anyway. Or do Go coroutines run through some sort of managed execution queue?
Celery is one of the easiest things to set up. It really doesn't get much simpler. Your configuration is living in your django project as python source code, and `celeryd` is no-fuss.
well, the complaint isn't entirely that it's hard, per se, but that there are many different ways to do things concurrently in Python, and that it presents too many choices to be made for the novice programmer. In Go, you just say `go myfunction()` and you're done. There's a lot of value in that.
I keep hearing about go. I am new-ish to programming. Only really getting started on my first project, which depends on pyparsing, which depends on other things. Is go something a novice should be attacking real-world problems with?
Probably not. Go is still in its infancy and not especially well-supported. If pyparsing is solving your problems well, stick with it. Python is great.
Of course if speed is important, you should ditch CPython anyway.
A language is just one tool in your toolset. I love using Go very much, but I would recommend learning on Python. There's much more literature out there, more people you know will know it, and if you are looking for work, there are effectively zero entry-level programming jobs in Go.
The reason you keep hearing about Go is that it just very recently hit its 1.0 release, so it it a comparatively new language, and it's just a topic of conversation. Talking about Python isn't really newsworthy in the same way, because for most people, it's just a fact of everyday life. Go, however, is this new thing that's a little mysterious, that most people haven't tried yet.
Have you ever heard the phrase "you can never really know yourself until you know others"? Well, that's true of programming languages, too; learning new languages can help you to write better code in a language you were already familiar with. Learning new languages is great practice for any programmer.
But to start, I would just stick with Python and write some things that make you happy. That's the most important part; to figure out how to use code to make yourself happy. Which language you use is really just an implementation detail.
I think we can cut the author some slack. He admits in the header to being a relative newbie. (Learned python less than a year ago, recently took Hacker School)
In Go, the coroutines are in the same thread. There is a single thread here too(just like node.js). Coroutines are just multiplexed to the one main thread. Multi-Core Processing is handled by coroutine internals too i.e. there may or may not be more than one threads and even if there are more than one threads, they too will be multiplexed with the one main thread.
Goroutines running Go code are multiplexed onto a variable number of system threads (GOMAXPROCS, in the gc implementation), Goroutines running C code, for example via cgo, or when they call a blocking system call, have their own thread.
I haven't been using Django for a while, but I know there was no way built into the framework to do it. The complaint isn't really that it's too hard, it's that there's too many decisions to make, because it's not supported at the language level.
Sure you can do that (using threads/multiprocessing/greenlets). But usually you don't want to because it's hard to guarantee that the background task will complete without interruption.
What's wrong with good old threads. It's not that hard. And threading in Java will blow away any pseudo concurrency setup in python. Been there done that. Love python but just couldn't get the performance we needed.
[+] [-] juddlyon|13 years ago|reply
This made me laugh, thank you.
[+] [-] ak217|13 years ago|reply
Concurrency support is possible in Python, without gevent-style monkey patching (or callback madness). Have a look at concurrent.futures and http://www.dabeaz.com/coroutines/index.html. It really needs a lot more work before it's part of the language's DNA, though. Also, pypy needs much wider adoption as quickly as possible, to address the speed problems (and its STM branch holds huge potential).
For me, Go's major shortcoming is its community's lack of focus on readability as compared to Python.
[+] [-] georgemcbay|13 years ago|reply
Can you be a bit more concrete here? Because whether you like the code styles enforced by gofmt and the compiler itself or not, Go is the most consistently readable language I've ever used (once you adapt yourself to the language). IMO, this seems like an especially odd problem for someone to have with Go.
[+] [-] zemo|13 years ago|reply
that was deliberate. There has been a glut of "introduction to Go" blog posts, and the documentation for introductory Go from the Go Team is quite good, so I didn't feel the need to author another introductory post. Most of the stories about why people use Go are bigger, established companies, so I wanted to show how it was perceived by a different audience. I don't remember where I saw it, but someone talked about using Go in contrast with Java, and the overwhelming reaction was "that's cool but I'm a Python programmer, I'm curious what Python programmers think", so I was just trying to give it that angle.
>Have a look at concurrent.futures
but this is part of the problem: it's a library. Concurrency deserves language-level support. Adding concurrency at the library level is like adding an object system at the library level.
[+] [-] jfaucett|13 years ago|reply
[+] [-] peter_l_downs|13 years ago|reply
[+] [-] stcredzero|13 years ago|reply
Source code changes could be saved as log-structured patch files, which could then be thrown away or applied to the source tree as desired. One could also steal some ideas from the Smalltalk Change Log tool by adding similar editing, search, and filtering commands.
With tools like this, one could recompile for "interpreted debug mode," have complete visibility and control of runtime state to debug a problem, then take the resulting patch file and apply it to the source tree. It would be a best of both worlds scenario -- all the enhanced debugging of an interpreted runtime with the type safety and speed of compiled code.
[+] [-] taliesinb|13 years ago|reply
At a Go talk he did, I raised the idea with Russ Cox of having a "repl" package that would allow one to instrument a running program with a live REPL to do debugging and development on it.
The reflect package is powerful enough to make some of that relatively straightforward, but one major problem is that Go can't construct new types at runtime. Another challenge would be dynamic linking of new code -- because the Go toolchain doesn't support dynamic linking there would seem to be no hope of say entering anonymous functions on the REPL.
Unless one builds a full Go interpreter -- but who wants to be in the business of maintaining a fully compliant Go interpreter that can interoperate with the Go runtime?
Still, calling existing functions and banging on variables would be pretty useful.
[+] [-] it|13 years ago|reply
[+] [-] zemo|13 years ago|reply
[+] [-] just2n|13 years ago|reply
The insight to async I/O is that this server, for this I/O bound task, will perform as well as your GOMAXPROCS example:
But it's a lot simpler. You don't need to bolt on parallelism when you don't actually need parallelism. There are valid reasons to use Go, and there are valid complaints against Node. I don't see any of either here.[+] [-] SeanDav|13 years ago|reply
This was a great pity because I think it probably is one of the best languages ever made and didn't get the take up it absolutely deserved.
[+] [-] st3fan|13 years ago|reply
Like for example show me how to suspend the HTTP request and wake it up later when you receive a message back from some long running process.
Or simpler: show me how to start a long running async task from a request handler.
[+] [-] aaronblohowiak|13 years ago|reply
Will it?
Luvit, which is implemented on Node's libuv outperforms Node in the trivial case because of higher lua performance (and lua->c interop performance).
[+] [-] mseepgood|13 years ago|reply
No, it doesn't scale on multiple CPUs if you don't use the 'cluster' package.
[+] [-] slurgfest|13 years ago|reply
[+] [-] bcoates|13 years ago|reply
There's an awful global lock on the actual execution of python code, but unless the problem is performance or contention worrying about it is premature.
[+] [-] dignan|13 years ago|reply
[+] [-] ricardobeat|13 years ago|reply
[+] [-] enneff|13 years ago|reply
In any case, it's still not an apples to apples comparison. The node.js example forks separate processes that share nothing, while in the Go example it all runs in one process so the HTTP handlers can communicate with each other via channels or other shared data structures.
[+] [-] geekstrada|13 years ago|reply
[+] [-] MatthewPhillips|13 years ago|reply
[+] [-] mietek|13 years ago|reply
[+] [-] zemo|13 years ago|reply
That could have been written better. Thanks for the feedback.
[+] [-] zzzeek|13 years ago|reply
[+] [-] antihero|13 years ago|reply
Is there some sort of Go package manager? How does all this shit work?
[+] [-] zemo|13 years ago|reply
Yes, the Go tool comes with package-management capabilities. "go get labix.org/v2/mgo" will download the very excellent MongoDB driver mgo, for example.
[+] [-] srj55|13 years ago|reply
It's not really that hard. I just latch on to a broker that I'm already using elsewhere in my stack (Redis). Celery makes it super simple to run a command in the background.
[+] [-] mbreese|13 years ago|reply
What's more, the strategy of just spawning a thread to do with async processing doesn't scale. Once you hit the limit of requests that a machine can process, you'll need a distributed work queue anyway. Or do Go coroutines run through some sort of managed execution queue?
[+] [-] oinksoft|13 years ago|reply
[+] [-] zemo|13 years ago|reply
[+] [-] niels_olson|13 years ago|reply
[+] [-] TillE|13 years ago|reply
Of course if speed is important, you should ditch CPython anyway.
[+] [-] zemo|13 years ago|reply
A language is just one tool in your toolset. I love using Go very much, but I would recommend learning on Python. There's much more literature out there, more people you know will know it, and if you are looking for work, there are effectively zero entry-level programming jobs in Go.
The reason you keep hearing about Go is that it just very recently hit its 1.0 release, so it it a comparatively new language, and it's just a topic of conversation. Talking about Python isn't really newsworthy in the same way, because for most people, it's just a fact of everyday life. Go, however, is this new thing that's a little mysterious, that most people haven't tried yet.
Have you ever heard the phrase "you can never really know yourself until you know others"? Well, that's true of programming languages, too; learning new languages can help you to write better code in a language you were already familiar with. Learning new languages is great practice for any programmer.
But to start, I would just stick with Python and write some things that make you happy. That's the most important part; to figure out how to use code to make yourself happy. Which language you use is really just an implementation detail.
[+] [-] jimbokun|13 years ago|reply
[+] [-] effinjames|13 years ago|reply
[+] [-] zallarak|13 years ago|reply
[+] [-] zemo|13 years ago|reply
[+] [-] mathattack|13 years ago|reply
[+] [-] knodi|13 years ago|reply
[+] [-] realrocker|13 years ago|reply
[+] [-] 4ad|13 years ago|reply
[+] [-] aaronblohowiak|13 years ago|reply
.. until you set runtime.GOMAXPROCS(runtime.NumCPU())
[+] [-] electic|13 years ago|reply
[+] [-] tocomment|13 years ago|reply
[+] [-] zemo|13 years ago|reply
[+] [-] asksol|13 years ago|reply
[+] [-] ishbits|13 years ago|reply
[+] [-] dgregd|13 years ago|reply
Is it too early to switch from Python/Ruby to Go? Maybe it would be better to wait for Dart.
I known that Mozilla and MS won't support Dart however they also don't support Go.