Proxygen, Facebook's C++ HTTP Framework

[dcsommer]

Hey there, I work on Proxygen at Facebook. I'm happy to answer any questions you have about the project.

[lost_name]

This question is a bit naive, but outside of Facebook, can you think of what kind of application this is well suited for?

[sitkack]

What are the security implications of running native code on a public facing service? How many RCEs has Proxygen had? Has it been audited for security? What is the testing procedure?

[leeber]

I currently use nginx for all my projects, which for me mostly consist of (1) serving static content and (2) proxying to gunicorn. My setups are usually not overly complicated, basic redirects for stuff like www/non-www or ssl/non-ssl, installing ssl certs can be a pain sometimes, and for the most part I use a lot of default settings and have never needed to go in-depth with tweaking settings.

Does somebody like me have a reason to check out Proxygen?

[djb_hackernews]

Question for you and the general C++ community. I am a Java developer mostly but it looks like I'll be transitioning to a few C++ projects in the near future. Any good resources you can recommend for learning modern C++? Particularly anything you've used to get developers on your team similarly up to speed.

I've cruised through the Proxygen code base and there are definitely some head scratchers.

[matt_d]

Hi! Looks interesting!

I have a question -- in particular, the blog post mentions that the framework "includes both server and client code", any my question is about the second part :-)

I'm wondering, how does it compare to the other C++ HTTP client solutions: Is it closer to higher-level libraries like cpp-netlib, Casablanca, or POCO -- or more on a lower-level / comparable to Asio (or Boost.Asio)?

In your view, what are the main relative advantages/disadvantages (namely in the scenario mentioned in the blog post, i.e., integration into existing applications)?

[invisible]

Hey! So Proxygen was originally a reverse-proxy load balancer. Is that still how Facebook utilizes it now? If not, what is its current role? Are there any plans for integrating this with Hiphop/PHP in any way?

[plainOldText]

I'm curious to know why there is a file named PATENTS in the repo (alongside the LICENSE file). What kind of legal protections would this "additional grant" give you?

[egonschiele]

I would love to see a comparison between Proxygen and another server (ideally nginx or golang server). The numbers are impressive, but the client is on the same box, it is a big box, and it is a simple and short response. So I'm not sure if I should be wowed or not. From reading the post, Proxygen was written as a server that would be well-integrated into Facebook tools. I don't really use Facebook tools, so I'm not sure if Proxygen would be right for me?

[syncopate]

Hey, I am a bit late, but I wonder how you generally handle unicode in C++, since the language itself does not have much support for it. That's what always makes me weary of writing any kind of server in C++.

[mikebo]

How tested is the websocket support? Does facebook use this for TLS termination?

[thrownaway2424]

Could you comment on the verisimilitude of the benchmark parameters? I would expect in Facebook's production environment the number of active sockets would be hundreds or thousands of times greater than 400.

[brian_herman]

Are there any plans for spdy and http2?

[b3tta]

I really like your idea of the 4-part abstraction. Still… Your library really is not "that" easy to use.

I'm currently working on my own library using libuv, http-parser, nghttp2 and wslay, which is very similiar in it's use to node.js. As you might guess a echo server is therefore only about 15 lines of code, but about as performant as your framework. The downsite is that it's not as flexible due to the missing "4-part abtraction" (really… an excellent idea).

That's why your release somehow saddens me: When I'm going to release my framework to the public, it might be pretty good for cross platform apps etc. compared to others, but it will never ever be as popular as yours. Heck… I don't even have 10 twitter followers.

[Igglyboo]

Care to show us what you've got instead of just saying how much better it is than Proxygen?

Also Facebook is a group of people so saying "your" doesn't really sound right.

[rphillips]

is it open source?

[beliu]

dcsommer (author of proxygen) gave a great talk about this at the last Sourcegraph open-source meetup. Here's the video: https://www.youtube.com/watch?v=-yxQIRl6Qic

[dcsommer]

I'm flattered, but I'm just an author. Proxygen is the work of about a dozen people over 4 years at Facebook.

[thomasreggi]

"You will need at least 2 GiB of memory to compile proxygen and its dependencies." What?

[port8080]

fbthrift needs 2GiB of memory to compile (https://github.com/facebook/fbthrift) and proxygen has a dependency on fbthrift.

[btian]

That is not unreasonable for modern software. You don't need 2GB of RAM to use it.

[Igglyboo]

Completely reasonable for modern software of this size/complexity.

[maccard]

i wouldnt like to try compiling my work projects on anything less than 16GB ram, never mind 2GB

[rdrock]

We used to do nginx + gunicorn for our rest services, it was not responding well beyond a point (for a given ec2 instance). We replaced that with nginx + lua (openresty module), we saw almost 10x increase in response times. Would it make sense for us to invest in something like this and hope to see a significant performance gain? Lowering response times is not a big deal but being able to get those same response times on a lower priced instance would definitely help. We have no real C++ skills in the team but we could learn or hire.

[cthulhuology]

what are the units on the table? the top looks like number of workers, but large numbers are unitless.

[dcsommer]

It's requests per second (averaged over a 60 second test run).

[bsaul]

Looks to me as facebook's answer to golang ?

Building simple, standalone http services with good performances seems to me what those two projects (proxygen and golang) are really about.

Now the question is, how much faster using C++ is, and how much safer and faster writing golang is...

[Jare]

I think Facebook is a lot into D, sounds a more appropriate, for lack of a better word, "replacement" for golang.

[pjmlp]

Go is a nice language, but so is C++14, specially if you like the expressiveness it provides.

Plus, there are still lots of scenarios where Go tooling still doesn't provide support for. They will eventually, but not if you are starting a project today.

[derengel]

I'm guessing that at facebook scale a GC would be expensive.

[pcunite]

Excellent, I've been toying around with my own and looking at LibUV. I think the time is right for something like this. I want to maintain state on everything that connects to me.

[72deluxe]

Looks interesting. Does anyone have a comparison of all these plethora of C++ HTTP frameworks, such as Civetweb etc.?

[dimman]

Facebook oughta hire some better scripters, the deps.sh is of terribly low quality. I didn't get more than a couple of lines until i stumble upon this (which tells me the author has no clue :):

'start_dir=`pwd`; trap "cd $start_dir" EXIT;'...

No need to say that the script can be dangerous, in case directory change fails for instance, there's no checks but sudo make uninstall is run anyway in another dir than the intended one.

[dcsommer]

Bash isn't my expertise and I put this together pretty quickly. Please send pull requests! Forgive my ignorance, but what's the danger of the cd'ing in the EXIT trap? Also, I did set -e, so there's no problem of running "sudo make uninstall" from the wrong directory, afaict.

[amelius]

Skimming through the article, it seems to me that this server spawns a thread per connection, is that correct?

[dcsommer]

We use a very different model actually. Since spawning OS threads is expensive, we opted for the popular nonblocking-IO approach. Each worker thread (usually 1 per core on the CPU) is given connections in a round robin fashion from the listening socket. The worker thread runs an event loop processing events on the accepted socket.