Okay here's a peeve of mine: just because something is in C ("no overhead!") doesn't mean it's faster than something in another language in all situations.
As an example, this server uses a thread pool architecture. This architecture will perform poorly with slow clients (common on the public Internet), servers which have to interact with slow disks or external services, and is useless for long-polling. It's only useful for CPU-bound applications when you can assume fast clients and short requests.
In fact, I could make this server grind to a halt by opening one connection per worker, issuing partial requests to each, then letting the connection hang. So to be used in production, this server will have to sit behind something like nginx, which can insulate your application from pathologically slow clients.
Threaded servers are not necessarily slower than asynchronous architectures, though they can be. In fact, threaded models are coming back into fashion these days, and can (sometimes) be faster than asynchronous models when each thread is affine to a particular physical CPU. This is due to cache effects and the substitution of fast stack allocation for slow heap allocation.
Bazillions of threads using too much stack space have been a problem in the past, but these days servers have plenty of memory, and to some extent size can also be mitigated by tail call optimizations.
I don't think your particular DoS attack will work with this server, which implements an idle timer to recover stack space. However, they could go one step further and only allow worker threads to use n% of server memory, and killing old, slow connections when the thread is needed and the memory limit has been reached.
That's a terrible example, both because it has nothing to do with speed (your complaint is about scalability, not speed) and because it is wrong. Having a pool of workers to handle requests while using event driven interfaces is normal now, and gives you the best of both worlds. Your theoretical attack works the same as it would on nginx: you'd need to run it out of file handles.
I choose HTTP/1.1 pipelining. Uncompressed headers are useful. Ordered records are returned (unlike SPDY), where "HTTP/1.1 200 OK" is the record separator. Been using this for a decade. Can't see the benefit of SPDY.
Anyway pipelining is only useful where numerous resources are coming from the same host. But the way the www has evolved, so much (unneeded) crap gets served from ad servers and CDN's. Pipelining isn't going to speed that up.
HTTP/1.1 pipelining was never broken. It was usually just turned off (e.g. in Firefox), while most web servers have their max keep alive set around 100. In plain English, what does that mean? It means "Dear User, You have permission to download 100 files at a time from http://stupidwebsite.com. That is you can make one request for 100 files, instead of 100 separate requests, each for a single file." And what do Firefox and other braindead web browsers do? They make a separate request for each file. But heay, never mind all those numerous connections to ad servers to retrieve marketing garbage (i.e. not the content you are after), lets concentrate on compressing HTTP headers instead. Brilliant.
It's trivial to use pipelining: 1. Feed your HTTP requests through netcat or some equivalent to retrieve the files and save them to a concatenated file, 2. split the concatenated file into separate files if desired, 3. view in your favorite browser.
Would it be an interesting idea to create a framework for making webapps as nginx modules? Sure, it's a pain in the ass, but nginx is evented as opposed to thread pooled, and tried and tested. Because nginx takes up hardly any memory, you could run a single nginx instance and proxy to other nginx instances that are compiled purely to run the app.
Though the scope for creating critical vulnerabilities is huge.
It uses a wide range of 3rd-party nginx modules, and Lua as a scripting language, to form a nice little framework. Extraordinarily fast by any standards.
Ceased reading after mem.c - pool-based memory allocation which cache-locality awareness and data partitioning by access patterns is the must for a modern server. Just to malloc every buffer you need is a naive strategy, which, probably, will result in memory fragmentation and cache misses.
Surprisingly accurate and clean C. This guy is good one for hiring (assuming he wishes to be hired).
It's quite a gift for embedded systems. It's very convenient to have a self-contained solution for a web interface, rather than drag an interpreter and a web server on an already crowded rootfs. Plus, you can get the benefit of static analysis and the like, which is extra useful on such systems.
The more "close to the metal" you can get and more awkward and painful it is to program in, the more cred you get. Actual ramifications of slow connections, long polling etc... be damned.
Come to think of it, how come we don't have a web server in Var'aq?
Why the ISC license? The site just says "Kore is licensed under the ISC license allowing it to be used in both free and commercial products." but don't Apache 2, BSD, and MIT all satisfy this as well.
[+] [-] a-priori|12 years ago|reply
As an example, this server uses a thread pool architecture. This architecture will perform poorly with slow clients (common on the public Internet), servers which have to interact with slow disks or external services, and is useless for long-polling. It's only useful for CPU-bound applications when you can assume fast clients and short requests.
In fact, I could make this server grind to a halt by opening one connection per worker, issuing partial requests to each, then letting the connection hang. So to be used in production, this server will have to sit behind something like nginx, which can insulate your application from pathologically slow clients.
[+] [-] mpweiher|12 years ago|reply
Is the README wrong? It says:
"Event driven architecture and worker processes for throughput"
To me this indicates using events for the external interface (slow clients), and threads for taking advantage of multiple cores.
[+] [-] codex|12 years ago|reply
Bazillions of threads using too much stack space have been a problem in the past, but these days servers have plenty of memory, and to some extent size can also be mitigated by tail call optimizations.
I don't think your particular DoS attack will work with this server, which implements an idle timer to recover stack space. However, they could go one step further and only allow worker threads to use n% of server memory, and killing old, slow connections when the thread is needed and the memory limit has been reached.
[+] [-] papsosouid|12 years ago|reply
[+] [-] richo|12 years ago|reply
Choose one, SPDY mandates gzip, and gzip in SSL/TLS is vulnerable to leaking repeated plaintext, eg cookies in nearly every implementation.
http://en.wikipedia.org/wiki/CRIME_(security_exploit)
[+] [-] osth|12 years ago|reply
Anyway pipelining is only useful where numerous resources are coming from the same host. But the way the www has evolved, so much (unneeded) crap gets served from ad servers and CDN's. Pipelining isn't going to speed that up.
HTTP/1.1 pipelining was never broken. It was usually just turned off (e.g. in Firefox), while most web servers have their max keep alive set around 100. In plain English, what does that mean? It means "Dear User, You have permission to download 100 files at a time from http://stupidwebsite.com. That is you can make one request for 100 files, instead of 100 separate requests, each for a single file." And what do Firefox and other braindead web browsers do? They make a separate request for each file. But heay, never mind all those numerous connections to ad servers to retrieve marketing garbage (i.e. not the content you are after), lets concentrate on compressing HTTP headers instead. Brilliant.
It's trivial to use pipelining: 1. Feed your HTTP requests through netcat or some equivalent to retrieve the files and save them to a concatenated file, 2. split the concatenated file into separate files if desired, 3. view in your favorite browser.
No ad server BS.
Now that's "SPEEDY".
[+] [-] dcsommer|12 years ago|reply
[+] [-] alkou|12 years ago|reply
[+] [-] antihero|12 years ago|reply
Though the scope for creating critical vulnerabilities is huge.
[+] [-] eric_bullington|12 years ago|reply
http://openresty.org/
It uses a wide range of 3rd-party nginx modules, and Lua as a scripting language, to form a nice little framework. Extraordinarily fast by any standards.
[+] [-] FuzzyDunlop|12 years ago|reply
[+] [-] dschiptsov|12 years ago|reply
Surprisingly accurate and clean C. This guy is good one for hiring (assuming he wishes to be hired).
[+] [-] betterunix|12 years ago|reply
[+] [-] weland|12 years ago|reply
[+] [-] m0nastic|12 years ago|reply
People like to use what they like.
[+] [-] eksith|12 years ago|reply
http://www.reocities.com/connorbd/varaq/index.html
The more "close to the metal" you can get and more awkward and painful it is to program in, the more cred you get. Actual ramifications of slow connections, long polling etc... be damned.
Come to think of it, how come we don't have a web server in Var'aq?
[+] [-] kaoD|12 years ago|reply
[+] [-] bobol|12 years ago|reply
[+] [-] alepper|12 years ago|reply
[+] [-] anuragramdasan|12 years ago|reply
[+] [-] ndesaulniers|12 years ago|reply
Edit: This is web development in C: https://github.com/jorisvink/kore_website/blob/master/src/si...
[+] [-] aray|12 years ago|reply
IANAL, but here's a quick summary of ISC vs MIT: http://www.tldrlegal.com/compare?a=MIT+License&b=ISC+License
[+] [-] protomyth|12 years ago|reply
[+] [-] drdaeman|12 years ago|reply
Even WTFPLv2 is not good.
[+] [-] nvartolomei|12 years ago|reply
Libuv is used in nodejs.
[+] [-] running_foo|12 years ago|reply
As for the server, I wonder how performance compares to say, OpenResty loading Lua compiled to bytecode.
[+] [-] ams6110|12 years ago|reply
http://www.aolserver.com/docs/devel/c/
[+] [-] X4|12 years ago|reply
[+] [-] bobol|12 years ago|reply
Brilliant idea, I especially love the SSL only. The web needs to move to pure SSL.
[+] [-] davis_m|12 years ago|reply