[zng00]

This is great, thanks for sharing. I'm curious about the downstream proxy process (i.e. ::1234) and how you scale it and balance load across multiple instances of the process. You can't really use iptables to load balance your processes as either the DNAT or REDIRECT mechanism will modify the destination address, right?

Ex. # TPROXY directs all traffic to :1234, and these rules load balance to 4 different processes

iptables -t nat -I OUTPUT -p tcp -o lo --dport 1234 -m state --state NEW -m statistic --mode nth --every 4 --packet 0 -j DNAT --to-destination 127.0.0.1:8080

iptables -t nat -I OUTPUT -p tcp -o lo --dport 1234 -m state --state NEW -m statistic --mode nth --every 4 --packet 1 -j DNAT --to-destination 127.0.0.1:8081

iptables -t nat -I OUTPUT -p tcp -o lo --dport 1234 -m state --state NEW -m statistic --mode nth --every 4 --packet 2 -j DNAT --to-destination 127.0.0.1:8082

iptables -t nat -I OUTPUT -p tcp -o lo --dport 1234 -m state --state NEW -m statistic --mode nth --every 4 --packet 3 -j DNAT --to-destination 127.0.0.1:8083

[majke]

We have a single Accept queue for all the ports. For TCP it doesn't create any problems - the new connection rate is rarely significant.

For the accept-queue load balancing see these blog posts:

https://blog.cloudflare.com/the-sad-state-of-linux-socket-ba...

https://blog.cloudflare.com/syn-packet-handling-in-the-wild/

[zng00]

Wow, these are some great resources. Thanks for sharing! I have a call with one of your colleagues in 5 minutes ;)