> The fundamental basis of any Unix application is the thread or process. (From the Linux OS perspective, threads and processes are mostly identical; the major difference is the degree to which they share memory.)
It's better to be specific in performance discussions, rather than use 'thread' and 'process' interchangeably.
As well as the article mentioned about memory sharing, threads (which are called Lightweight Processes, or LWPs, in Linux 'ps') are granular.
ps -eLf
NWLP in the command above is 'number of lightweight processes', ie number of threads.
Processes are not granular: they're one or many threads. IIRC it can be beneficial to assign threads of the same process to the same physical core or same die for cache affinity. There's all kind of performance stuff where 'threads' and 'processes' do not mean the same thing. Being specific is rad.
In Linux, they are both just entries in the process table. They are created by the `clone` syscall, and the "normal" way of creating them share different amounts of resources by _default_.
You're right to say that treating them differently can be beneficial in some situations, but it really depends.
> You can reload configuration multiple times per second (and many NGINX users do exactly that)
I thought this was an interesting remark. Can anyone clue me in to what these "many users" might be doing, that requires them to reload configuration so frequently?
Service Discovery. Dynamically generating routing rules so microservices can have pretty URLs. For example with us when you deploy a new version of a microservice it starts on a random port, registers with Consul on startup and then dynamically regenerates and reloads Nginx.
I know of at least one VPS company that build its load balancer SaaS offering on NGINX and it automatically reloads whenever a node is added/removed.
So I would assume it was that function that would cause such rapid reloads. [e.g. If you had a 50 node pool, you might go +/-5 over a span of a second and change the config 5 times in 1s]
"NGINX’s binary upgrade process achieves the holy grail of high-availability; you can upgrade the software on the fly, without any dropped connections, downtime or interruption in service."
Is this really true? I remember seeing an article[1] recently on using an iptables hack to prevent dropping connections when reloading haproxy. Does nginx actually provide zero-downtime configuration reloads?
I didn't want to be a negative nancy in the comments for that haproxy article... but that is a ridiculous ugly hack. It's really a lot easier to achieve real and robust zero-downtime upgrades for a simple unix process.
Remember, fork()ed and exec()ed processes inherit file descriptors (except those marked CLOEXEC), including the listen() fd. Pending connections will queue in the kernel until userspace calls accept() on the listening fd.
So one simple model is to stop calling accept(), cleanly/quickly finish up current connections, set an environment variable to tell the future instance that the listening fd X is already open, and exec your own binary again.
A more complicated one is to fork, have the (identical) child just finish the current connections, while the parent execs itself similar to above. (The client connection fds should be marked CLOEXEC in this case.)
With a more complicated service with more moving parts, libraries, threads, getting the above to work out is more complicated. But that's basically how you want to do it.
The holy grail of high-availability isn't upgrading stateless software. It's upgrading stateful ones.
Like upgrading when a data structure changes between versions. The HTTP protocol nginx serves is stateless and by comparison far simpler. Same goes for Erlang. It offers nothing more than simple function replacement, and that's not enough to handle data structure changes either.
The irc client irssi has something like this in /upgrade where it spawns a new binary but passing along active socket connections and their associated irc room states, I believe
If it's the holy grail, then erlang has had it for a couple decades... and whether it actually works in nginx I don't know, but it does work in erlang.
The way it's handled is that the process where the previous socket was connected remains in place until it terminates but all new sockets connect to the new code.
One thread per CPU, and non-blocking I/O, that's sounds like the usual way to approach the problem. I'm surprised it uses state machines to handle the non-blocking I/O, because modern software engineering provides much more pleasant approaches such as using coroutines.
Coroutines have the distinct disadvantage of needing a stack, much like threads. So-called 'stackless' coroutines aren't really so different to computed gotos in a state machine
I did as they said at the bottom and gave them my e-mail and other personal details so I could download the eBook that they were giving free preview copies of - "Building Microservices". Unfortunately, they sent link to PDF only so it's not usable to me. Just a heads up to others so you save yourself the time of discovering that. (I'll just wait for when the book is finished and then I'll buy it so I get ePub. I like O'Reilly and have bought many books there before.)
Interesting what kind of performance one can get out of GHC nowadays. Article says the authors of Warp had to implement a new parallel IO manager for GHC to get there, but that was merged into GHC 7.8.
Interesting overview. I wish they had some data comparison which could explain the significance and efficiency of this approach vs other/old approaches.
They forgot to mention pool-allocated buffers, zero-copy strings, and very clean, layered codebase - every syscall was counted.
The original nginx is a rare example of what is the best in software engineering - deep understanding of principles and almost Asperger's attention to details (which is obviously good). Its success is justified.
[+] [-] rdtsc|10 years ago|reply
This should round robin accept in the kernel, and not wake up all the epoll listeners.
https://lwn.net/Articles/632590/
[+] [-] 15155|10 years ago|reply
[+] [-] nailer|10 years ago|reply
> The fundamental basis of any Unix application is the thread or process. (From the Linux OS perspective, threads and processes are mostly identical; the major difference is the degree to which they share memory.)
It's better to be specific in performance discussions, rather than use 'thread' and 'process' interchangeably.
As well as the article mentioned about memory sharing, threads (which are called Lightweight Processes, or LWPs, in Linux 'ps') are granular.
NWLP in the command above is 'number of lightweight processes', ie number of threads.Processes are not granular: they're one or many threads. IIRC it can be beneficial to assign threads of the same process to the same physical core or same die for cache affinity. There's all kind of performance stuff where 'threads' and 'processes' do not mean the same thing. Being specific is rad.
[+] [-] alexchamberlain|10 years ago|reply
You're right to say that treating them differently can be beneficial in some situations, but it really depends.
[+] [-] afarrell|10 years ago|reply
[+] [-] cshimmin|10 years ago|reply
I thought this was an interesting remark. Can anyone clue me in to what these "many users" might be doing, that requires them to reload configuration so frequently?
[+] [-] threeseed|10 years ago|reply
[+] [-] fweespeech|10 years ago|reply
So I would assume it was that function that would cause such rapid reloads. [e.g. If you had a 50 node pool, you might go +/-5 over a span of a second and change the config 5 times in 1s]
[+] [-] earless1|10 years ago|reply
[+] [-] Igglyboo|10 years ago|reply
[+] [-] nodesocket|10 years ago|reply
Is this really true? I remember seeing an article[1] recently on using an iptables hack to prevent dropping connections when reloading haproxy. Does nginx actually provide zero-downtime configuration reloads?
[1] https://medium.com/@Drew_Stokes/actual-zero-downtime-with-ha...
[+] [-] ploxiln|10 years ago|reply
Remember, fork()ed and exec()ed processes inherit file descriptors (except those marked CLOEXEC), including the listen() fd. Pending connections will queue in the kernel until userspace calls accept() on the listening fd.
So one simple model is to stop calling accept(), cleanly/quickly finish up current connections, set an environment variable to tell the future instance that the listening fd X is already open, and exec your own binary again.
A more complicated one is to fork, have the (identical) child just finish the current connections, while the parent execs itself similar to above. (The client connection fds should be marked CLOEXEC in this case.)
With a more complicated service with more moving parts, libraries, threads, getting the above to work out is more complicated. But that's basically how you want to do it.
[+] [-] Denzel|10 years ago|reply
[+] [-] gull|10 years ago|reply
Like upgrading when a data structure changes between versions. The HTTP protocol nginx serves is stateless and by comparison far simpler. Same goes for Erlang. It offers nothing more than simple function replacement, and that's not enough to handle data structure changes either.
[+] [-] gregham|10 years ago|reply
[+] [-] 0x0|10 years ago|reply
[+] [-] MCRed|10 years ago|reply
The way it's handled is that the process where the previous socket was connected remains in place until it terminates but all new sockets connect to the new code.
[+] [-] amelius|10 years ago|reply
[+] [-] nly|10 years ago|reply
[+] [-] blt|10 years ago|reply
[+] [-] McElroy|10 years ago|reply
[+] [-] andorov|10 years ago|reply
http://www.amazon.com/gp/sendtokindle/email
[+] [-] justincormack|10 years ago|reply
[+] [-] simi_|10 years ago|reply
The whole book is worth a read, although I found some sections painfully boring (perhaps my limited attention span is to blame).
[+] [-] the_why_of_y|10 years ago|reply
http://www.aosabook.org/en/posa/warp.html
Interesting what kind of performance one can get out of GHC nowadays. Article says the authors of Warp had to implement a new parallel IO manager for GHC to get there, but that was merged into GHC 7.8.
[+] [-] saurabhtandon|10 years ago|reply
[+] [-] dschiptsov|10 years ago|reply
The original nginx is a rare example of what is the best in software engineering - deep understanding of principles and almost Asperger's attention to details (which is obviously good). Its success is justified.