I install a reverse proxy on every remote machine (raspberry pi or odroid) I manage. These machines are usually tucked behind someone's router, and it is hard to tell every router owner to forward the ssh port. When I install a reverse proxy, the machines can proactively tunnel right into a beacon machine whenever they are online and I can ssh into them from the beacon machine directly. I usually set up frpc as a service on the remote machine. It is also possible to use autossh with a monitoring port for this, but I noticed that frpc can bypass firewall restrictions much easier. This scheme also lets me use tools like Cockpit to view the health and status of my remotes.
An esoteric alternative is to host a Tor hidden service.
Tor hidden services don't require you open ports, and you don't need a static IP address.
Your residential IP is protected by the onion router, and the .onion domain name is free!
This is actually a cool idea and an example of thinking outside the box.
You run a Tor proxy (bind to 127.0.0.1:9050) on a server on the Internet, and then run a Tor hidden service inside your LAN.
Then, you acquire an .onion domain for the hidden service (running inside LAN), and configure the Tor proxy (running on a server on the Internet) to access the hidden service via the .onion domain (for example, use Nginx to reverse proxy the hidden service).
This should work. However, I imagine it could be quite a bit slower compare to "normal access", and requires extra steps to secure the transport connection between the hidden service and the proxy as you might not want to have everybody on Tor to be able to access that hidden service.
This is what I've always done when I was a poor student and didn't have a public IP.
The latency can be painful but it's way better than nothing.
I will say you should probably be careful about running a webserver on TOR. Almost all my machines have a basic web server just for copying static files to windows hosts I don't want to give my password to. People would find these on TOR and then find my resume, I got plenty of emails asking to set up "hidden sites on the dark web for sharing pictures."
I mean, you’re technically correct, but isn’t this kind of like saying that the .local TLD is free of charge? It’s “free” because it’s not a part of the “normal” internet.
There are a surprising number of tools that all do essentially the same thing, and it's often hard to tell how they're different from the other 30 options.
Making a really good solution to this problem has been the focus of my free time for the past month since I wrote this comment[0].
The project I'm working on:
* 100% open source and MIT licensed
* Strive for simplicity throughout. Very few dependencies. No config files, and only a single CLI parameter required to run the server.
* Clients can be remote controlled by the server, and each client can open an arbitrary number of tunnels. ie you don't have to start a separate ngrok process for each tunnel. Just set up the client on each machine you want to tunnel out of and forget about it. Everything is controlled
through a simple web UI (or REST api).
* Automate as much as possible. Tunnel creation and Let's Encrypt are already working. Eventually you'll be able to add an API token from your DNS provider and that'll be automated as well.
It's not quite ready (launch planned for 31 Oct 2020), but you can check it out here: https://boringproxy.io/
I have a similar setup but instead of SSH Tunnels i use wireguard.
In my case some services are not routed to the public internet and only accessible from within my privat wireguard network. E.g my smart home hub. I really like the flexibility to decide whats available for who.
I did the same. It works really well for hole-punching through my stupid Comcast router! And I anticipate it will continue to work really well as I add more devices on my parent's, sibings', etc networks. It's cool to not have to think at all about where my devices are, since they're just all directly accessible through their dedicated IP on the wireguard VPN.
I also have my phone, laptop, etc all on the wireguard VPN, so I can ssh to anything from anywhere. I also have my laptop open to password-based ssh only from my wireguard VPN, so I can ssh (via termux) into my laptop, which has been extremely convenient in addition to being just plain cool. I can't speak highly enough of wireguard!
Up until about 2 years ago, I used to use the same method as you've listed! Actually, I still use this method, but do it a bit differently. I have a ZeroTier network setup to connect a VM and RPI for internal communication, then I use nginx with the stream module [1] in order to do SNI proxying down to the RPI from the VM over the internal ZT network.
Why use the stream module vs the normal proxy module you might ask? You can forward TLS connections down to an edge device without needing to have the TLS certs on the cloud server. This is especially useful so I can easily share the VM with other people without having to worry about if they can intercept my network traffic, while still being able to host other things on the machine. This also has the added benefit of allowing me to forward full wildcard subdomains down to the edge device without having to create a new config entry for it each time.
Now, I use (shameless self plug) sish [2] to set this up. It uses SSH, so I get the added benefit of all of the above, but I don't really worry about using an internal network for ingress communication. It's also very easy to share the service with friends/coworkers so they can get the added benefit of it too. I also make use of the loadbalancing feature for test pi clusters when doing distributed testing which is a breeze now.
There are a bunch of these popping up now which I'm happy to see (serveo, remotemoe, even ngrok has a SSH gateway!) so there's a ton of competition here. I definitely recommend checking at least one of them out since they're pretty easy to setup and can be used in a variety of ways!
I wish we were a little farther along so I'd have more to say, but we're hoping to solve this with https://hoppy.network/.
It's nothing you couldn't do yourself, but we host it for you so you don't have to. We host WireGuard servers which assign a stable public IPv4 and IPv6 on your device's WireGuard tunnel interface.
As far as your device is concerned, it's like its tunnel interface is a publicly addressable connection. Because it is! Just with one extra encrypted hop at the end.
Nothing to install besides WireGuard itself. WireGuard's biggest papercut (Dynamic DNS) is solved because we provide you a connection to a server with a static IP.
No promotional emails. No spam. Just a simple service for a specific, niche need.
Hope to post about its launch here soon. ([email protected] for questions!)
I just set up a raspberry pi with ZeroTier. So far it seems like a really easy way to make the pi accessible whatever network it is in, and took minimal configuration to have it forward requests to any of the other devices on the internal network.
The only drawback is I don't think it would be possible to connect to a peer from something like heroku where you don't have full control over the server..
I don't get it, I may be missing some context thought: why the author didn't go for running said webapps directly on the VPS in the first place? Why bother running it on the Pi and set up the tunnel (which is a nice hack in itself, I get that)?
If you are on AT&T u-verse you can just assign a part of your IPv6 subnet (which appears to be static) to your RPi and disable the firewall to that host. Then you can accept incoming connections on ports 80, 22 and even 25 (for 25 I had to send a chat message in my AT&t account and it was opened the same day). Then you can have a full IPv6 web server at no additional cost to the price of your u-verse home internet.
I think a better option would be to create a Wireguard tunnel between Raspberry Pi and the remote server instead of a SSH Tunnel. Then there is no need to add or change ports and restart the tunnel for every service.
while i think this is true (any other vpn software would work too though) i want to point out that you actually can bring up a tun interface using ssh with the "-w local_tun[:remote_tun]" flag somewhat easily if you want to. It is also possible to make forwarding work in either direction using an integrated socks proxy using "-R" or "-D" flags ...
Having used SSH for this purpose for some time, I decided to build an automatic service for easing reverse HTTP, HTTPS, and SSH ProxyJump through SSH tunnels, its available on Github as well, but if you'd like, try it out:
ssh -R80:localhost:80 remote.moe
it should output an URL where your local port 80 is now available
I mean, in 90+% of cases, you just put in your Caddyfile
*host* {
reverse_proxy 0.0.0.0:*port*
}
and you get a fast reverse proxy that just works, with letsencrypt enabled by default to boot, nothing to worry about.
However, if you find yourself against a service that requires a slightly more advanced configuration, good luck making sense of their opaque configuration, especially since they now went v2 so everything old is out of the window, so if you search for something, chances are it's no good anymore.
Caddy underwent a complete rewrite that also invalidated most of the existing config. Personally I don't see any benefits in using caddy or another non mainstream httpd as it adds more risk to my project with usually very little benefits. Nothing against using it for hobby projects but in production I've migrated back to nginx.
One can also use the Diode P2P network to get the same without a server: https://support.diode.io/article/ss32engxlq works for ssh remote access, websites and stuff similar to ngrok just without the strings attached.
Side conversation: I'm interested in being pretty good at this sort of thing: setting up servers, making them connect, understand the protocols enough to put them to practice. I believe this is in the realm of IT. What are some good resources to read to get to my goal? Thanks!
I'd recommend doing almost exactly what OP describes. Start on your local network first. Get a raspberry pi, and figure out how to set up a webserver on it you can access from other computers on your network.
> I set GatewayPorts clientspecified in my /etc/ssh/sshd_config on the server. This is needed so that the client (RaspberryPi) can specify which ports to enabled reverse tunneling on.
This isn't quite what GatewayPorts does. By default, the client can open a tunnel to any ports the user has access to, but those ports can only be connected to from localhost on the server. So even if you don't have a firewall, external IPs can't connect to your tunnel (but your reverse proxy can, if it's running on the same machine). You can use GatewayPorts to allow binding from other IPs. This is useful if you want to tunnel arbitrary TCP traffic, not just HTTP through your reverse proxy.
What next? Well I setup some handy shell functions:
1) copy the latest screenshot into webroot/screenshots/${randstr}.png and copy the url to clipboard. this replaces dropbox's cool screenshot sharing feature. love sharing screenshots with coworkers this way, the files stays with you, you can redact easily.
2) serve static files the same way, if needed too. the magic's all in the shell scripts.
3) explore new open source webapps running on local docker.
Further?
Build a sort of like a devtools backend server, and a mobile app paired to it to do all sorts of cool stuff: run crons, get notifications for events, monitor servers / ssl expiry etc.
Limit's your imagination. But, security is also very important.
I have a similar setup as a fallback method to make sure I can always access my local server at home on any network (the primary methods are using zerotier and opening a port on my router), but I used autossh managed by supervisord to make sure the ssh tunnel stay up. Without autossh, the ssh tunnel sometimes randomly closed. I tried configuring ServerAliveInterval to some low value but the issue still happen sometimes, and when it happened sometimes the port it binds on the vps can't be reused anymore (port in use error). Autossh fixed all of those issues.
I did something similar with Wireguard but wasn't able to get it to do what I need. I'm trying to give my VPS access to other computers on my home network. The Pi and my VPS can communicate with each other just fine. For some reason the pi won't forward any packets from the VPS, they just seem to drop.
I've scoured all config that I could think of, iptables, network adaptors, Wireguard, but can't find the cause. Anyone encounter this or have a place to point me for help?
Have you tried this setting? This worked for me and I use my Pi as a media server and my VPN as a way to get all my machines to connect to it (a hub, I guess).
If you're running Docker, I believe it sets this by default, but try setting this on your VPS. I don't know enough about devops to know if there is an inherit danger to this, so take my advice with a grain of salt, but this worked for me.
[+] [-] nurettin|5 years ago|reply
https://github.com/fatedier/frp
[+] [-] viraptor|5 years ago|reply
[+] [-] tylerfontaine|5 years ago|reply
https://tailscale.com/
[+] [-] xrendan|5 years ago|reply
[+] [-] kzrdude|5 years ago|reply
[+] [-] wil421|5 years ago|reply
Do you have a static IP or a domain that your beacon uses?
[+] [-] belzebalex|5 years ago|reply
[+] [-] nirui|5 years ago|reply
You run a Tor proxy (bind to 127.0.0.1:9050) on a server on the Internet, and then run a Tor hidden service inside your LAN.
Then, you acquire an .onion domain for the hidden service (running inside LAN), and configure the Tor proxy (running on a server on the Internet) to access the hidden service via the .onion domain (for example, use Nginx to reverse proxy the hidden service).
This should work. However, I imagine it could be quite a bit slower compare to "normal access", and requires extra steps to secure the transport connection between the hidden service and the proxy as you might not want to have everybody on Tor to be able to access that hidden service.
[+] [-] swiley|5 years ago|reply
The latency can be painful but it's way better than nothing.
I will say you should probably be careful about running a webserver on TOR. Almost all my machines have a basic web server just for copying static files to windows hosts I don't want to give my password to. People would find these on TOR and then find my resume, I got plenty of emails asking to set up "hidden sites on the dark web for sharing pictures."
[+] [-] hnarn|5 years ago|reply
I mean, you’re technically correct, but isn’t this kind of like saying that the .local TLD is free of charge? It’s “free” because it’s not a part of the “normal” internet.
[+] [-] eeZah7Ux|5 years ago|reply
The daemon takes very little memory and CPU.
[+] [-] anderspitman|5 years ago|reply
https://github.com/anderspitman/awesome-tunneling.
There are a surprising number of tools that all do essentially the same thing, and it's often hard to tell how they're different from the other 30 options.
Making a really good solution to this problem has been the focus of my free time for the past month since I wrote this comment[0].
The project I'm working on:
* 100% open source and MIT licensed
* Strive for simplicity throughout. Very few dependencies. No config files, and only a single CLI parameter required to run the server.
* Clients can be remote controlled by the server, and each client can open an arbitrary number of tunnels. ie you don't have to start a separate ngrok process for each tunnel. Just set up the client on each machine you want to tunnel out of and forget about it. Everything is controlled through a simple web UI (or REST api).
* Automate as much as possible. Tunnel creation and Let's Encrypt are already working. Eventually you'll be able to add an API token from your DNS provider and that'll be automated as well.
It's not quite ready (launch planned for 31 Oct 2020), but you can check it out here: https://boringproxy.io/
[0]: https://news.ycombinator.com/item?id=24475946
[+] [-] lhoff|5 years ago|reply
In my case some services are not routed to the public internet and only accessible from within my privat wireguard network. E.g my smart home hub. I really like the flexibility to decide whats available for who.
[+] [-] twicetwice|5 years ago|reply
I also have my phone, laptop, etc all on the wireguard VPN, so I can ssh to anything from anywhere. I also have my laptop open to password-based ssh only from my wireguard VPN, so I can ssh (via termux) into my laptop, which has been extremely convenient in addition to being just plain cool. I can't speak highly enough of wireguard!
[+] [-] allthingsequal|5 years ago|reply
[+] [-] antoniomika|5 years ago|reply
Why use the stream module vs the normal proxy module you might ask? You can forward TLS connections down to an edge device without needing to have the TLS certs on the cloud server. This is especially useful so I can easily share the VM with other people without having to worry about if they can intercept my network traffic, while still being able to host other things on the machine. This also has the added benefit of allowing me to forward full wildcard subdomains down to the edge device without having to create a new config entry for it each time.
Now, I use (shameless self plug) sish [2] to set this up. It uses SSH, so I get the added benefit of all of the above, but I don't really worry about using an internal network for ingress communication. It's also very easy to share the service with friends/coworkers so they can get the added benefit of it too. I also make use of the loadbalancing feature for test pi clusters when doing distributed testing which is a breeze now.
There are a bunch of these popping up now which I'm happy to see (serveo, remotemoe, even ngrok has a SSH gateway!) so there's a ton of competition here. I definitely recommend checking at least one of them out since they're pretty easy to setup and can be used in a variety of ways!
[1] http://nginx.org/en/docs/stream/ngx_stream_core_module.html [2] https://github.com/antoniomika/sish
[+] [-] eximius|5 years ago|reply
It's nothing you couldn't do yourself, but we host it for you so you don't have to. We host WireGuard servers which assign a stable public IPv4 and IPv6 on your device's WireGuard tunnel interface.
As far as your device is concerned, it's like its tunnel interface is a publicly addressable connection. Because it is! Just with one extra encrypted hop at the end.
Nothing to install besides WireGuard itself. WireGuard's biggest papercut (Dynamic DNS) is solved because we provide you a connection to a server with a static IP.
No promotional emails. No spam. Just a simple service for a specific, niche need.
Hope to post about its launch here soon. ([email protected] for questions!)
[+] [-] johtso|5 years ago|reply
The only drawback is I don't think it would be possible to connect to a peer from something like heroku where you don't have full control over the server..
[+] [-] bpizzi|5 years ago|reply
[+] [-] cft|5 years ago|reply
[+] [-] voltagex_|5 years ago|reply
[+] [-] iamd3vil|5 years ago|reply
[+] [-] mercora|5 years ago|reply
[+] [-] thespoonbends|5 years ago|reply
Solutions can always be improved, but it's not always worth doing that.
[+] [-] upgradess|5 years ago|reply
ssh -R80:localhost:80 remote.moe
it should output an URL where your local port 80 is now available
https://github.com/fasmide/remotemoe
[+] [-] jeromenerf|5 years ago|reply
I wonder how a somehow simpler config syntax (Caddyfile) compares to a simpler maintenance system (apt).
[+] [-] birracerveza|5 years ago|reply
I mean, in 90+% of cases, you just put in your Caddyfile
and you get a fast reverse proxy that just works, with letsencrypt enabled by default to boot, nothing to worry about.However, if you find yourself against a service that requires a slightly more advanced configuration, good luck making sense of their opaque configuration, especially since they now went v2 so everything old is out of the window, so if you search for something, chances are it's no good anymore.
[+] [-] rmoriz|5 years ago|reply
[+] [-] anderspitman|5 years ago|reply
[+] [-] deccanchargers|5 years ago|reply
H2O web server seems to be faster than nginx , litespeed and supports http1/http2/http3 . It's config file is yaml file.
For HTTPS, i recommend https://acme.sh . It's very very easy and autorenews certificates using cron job.
[+] [-] dominicl|5 years ago|reply
[+] [-] anderspitman|5 years ago|reply
[+] [-] culopatin|5 years ago|reply
[+] [-] anderspitman|5 years ago|reply
[+] [-] anderspitman|5 years ago|reply
This isn't quite what GatewayPorts does. By default, the client can open a tunnel to any ports the user has access to, but those ports can only be connected to from localhost on the server. So even if you don't have a firewall, external IPs can't connect to your tunnel (but your reverse proxy can, if it's running on the same machine). You can use GatewayPorts to allow binding from other IPs. This is useful if you want to tunnel arbitrary TCP traffic, not just HTTP through your reverse proxy.
[+] [-] gprasanth|5 years ago|reply
What next? Well I setup some handy shell functions:
1) copy the latest screenshot into webroot/screenshots/${randstr}.png and copy the url to clipboard. this replaces dropbox's cool screenshot sharing feature. love sharing screenshots with coworkers this way, the files stays with you, you can redact easily.
2) serve static files the same way, if needed too. the magic's all in the shell scripts.
3) explore new open source webapps running on local docker.
Further?
Build a sort of like a devtools backend server, and a mobile app paired to it to do all sorts of cool stuff: run crons, get notifications for events, monitor servers / ssl expiry etc.
Limit's your imagination. But, security is also very important.
[+] [-] anderspitman|5 years ago|reply
[+] [-] neurostimulant|5 years ago|reply
[+] [-] driverdan|5 years ago|reply
I've scoured all config that I could think of, iptables, network adaptors, Wireguard, but can't find the cause. Anyone encounter this or have a place to point me for help?
[+] [-] granra|5 years ago|reply
Also is your home network's cidr in allowed ips on the vps' wireguard interface?
[1]: https://docs.mysirena.xyz/centos-7/prefilight-configuration/...
[+] [-] jjice|5 years ago|reply
https://linuxconfig.org/how-to-turn-on-off-ip-forwarding-in-...
If you're running Docker, I believe it sets this by default, but try setting this on your VPS. I don't know enough about devops to know if there is an inherit danger to this, so take my advice with a grain of salt, but this worked for me.
[+] [-] johtso|5 years ago|reply
The key was enabling IP forwarding, and setting the right iptables rules.