and that's not even getting into the assorted awesome HTTP request smuggling and SSRF vulns when 3 or more http implementations rub up against one another
I've usually used nginx for this purpose, proxying requests to a 3rd party server but tacking on authorization headers, for example, keeping them secret from the client. This strategy's always works great for me, but I guess it's limited to fairly simple pass-through use cases.
This seems cool if there are more complex needs or configuration that can't be done with simple proxying and nginx rules.
Nginx is quite extendable, there are tons of nginx plugins to help you add more customizations.
There is OpenResty, a version of nginx [0]. It allows you to script all sorts of stuff with Lua inside nginx itself.
Tools like lockbox are not necessary, nginx, caddy, etc or heck even a normal 70 line python3 fastapi based script works just fine and should be more extendable than lockbox.
I like the idea. It reminds me of the incredible issue with US banks where the most common way to setup 3rd party integrations with it is to give your username and password. Like, WTF?
It’s funny because as a passionate self-hoster, I consider as an advantage what you consider as a drawback.
and, to add insult to injury, it's my recollection that one must give those credentials to Plaid, not even entering them into your bank's website like a sane oauth2 dance. I always ensure I go to my bank's website and change my password afterward if I'm forced to use that MITM junk
I'm so jealous of PDS2 in the EU <https://www.berlin-group.org/> (although I guess it's easy for me to be jealous since I don't know if their banks actually implement the standards, but it's still a bunch better than the situation here)
This is super neat! I like the idea of better auditing of _what_ data gets sent over these integrations, not just what integrations are currently set up.
We do something similar with our tool (generative code thingy), where users can make API calls to various services (without credentials), and then we add the necessary auth tokens/headers for their code to actually run.
I'd love to give folks better control over their own data/credentials, Lockbox could be an interesting workaround without us having to build anything. Thanks and nice work!
Smokescreen is a HTTP CONNECT proxy. It proxies most traffic from Stripe to the external world (e.g., webhooks).
Smokescreen restricts which URLs it connects to:
It uses a pre-configured hostname ACL to only allow requests addressed to certain allow-listed hostnames, to ensure that no malicious code is attempting to make requests to unexpected services.
It also resolves each domain name that is requested, and ensures that it is a publicly routable IP address and not an internal IP address. This prevents a class of attacks where, for instance, our own webhooks infrastructure is used to scan Stripe’s internal network. Smokescreen can also be further configured to allow or deny specific IP addresses or ranges.
Maybe the most amazing and ultimate end-game here would be an API trust layer that can separate the most private data from 3p APIs.
- Location anonymization to fetch any resources like restaurants.
- Sensitive personal data encryption to verify credit info securely.
My guess is not all of the above could be easy/possible, its a leaky abstraction to think of APIs that way. But the biggest bang for the buck could be for privacy in the most mundane of usecases.
I like the core idea, but I fail to see why this is a good solution. IIUC the workflow is:
1. Run a Lockbox server
2. Define the API request in automation tool as a webhook.
3. Proxy it via lockbox to add auth.
All this really does it allow you to revoke auth at any time. As they can freely use the key. I see there are plans to restrict the requests allowed but doing that on the fully composed request it going to be extremely difficult to do reliably.
If you are already hosting some API why not do something like this instead.
1. Run a Lockbox server.
2. Send a webhook from the automation tool that just contains the relevant variables.
3. Lockbox composes the variabless into a request.
Basically if you want any sort of security and access control that is reliable you just want to send the parameters of the operation to Lockbox, then have the actual request generated on your server. Rather then doing the request generation in the automation tool then trying to validate that it was generated correctly.
Do IFTTT/Zapier even allow switching the Host for such requests? For companies like Zapier, it makes sense to request complete access to Google Sheets, but it gets very problematic for least-privileges, and this is a reasonable solution. I've even considered building something similar in the past, but getting vendors to switch hosts is not always easy.
That's nice. I was wondering, is it possible to tell Zapier or other tools to go through a proxy or use a different API URL for the services you configure there? E.g. tell them to use the Stripe API but call it at https://foo.mydomain.com instead of api.stripe.com?
This is a cool API but I wonder if something like Zapier would support redirecting their slack integration via your onsite proxy.
> You can restrict access to external APIs in a more fine grained manner
I've done this very successfully for many APIs before. I've found weird things among providers where using AWS (or GCP or Azure) gives you crazy fine-grained access controls (which are great after you spend two days figuring out how to use them well), but, some of the low cost competitors have core services that work just as well but their APIs are entirely binary (either you have full access, or none) but by adding a thin API layer above that you can enable all sorts of useful things. Provider doesn't support R/O API keys for terraform plans? Just run a proxy which enables that. Provider won't allow you to give someone an API key which can only reboot VMs, but, not delete them? Just write a proxy which does.
(Zapier co-founder) Perhaps the least known feature of Zapier is that you can use the dev platform (https://zapier.com/developer) to make your own private apps for any API (including internal/private services) and then sign requests with auth credentials. The dev platform even has a full WYSIWYG app builder! Then, anyone in your account can use the private app to make Zaps.
Relevant to OP, all platform auth creds are secured and not exposed after initial setup.
[+] [-] mdaniel|2 years ago|reply
and that's not even getting into the assorted awesome HTTP request smuggling and SSRF vulns when 3 or more http implementations rub up against one another
[+] [-] cypherpunks01|2 years ago|reply
This seems cool if there are more complex needs or configuration that can't be done with simple proxying and nginx rules.
[+] [-] teitoklien|2 years ago|reply
There is OpenResty, a version of nginx [0]. It allows you to script all sorts of stuff with Lua inside nginx itself.
Tools like lockbox are not necessary, nginx, caddy, etc or heck even a normal 70 line python3 fastapi based script works just fine and should be more extendable than lockbox.
[0](https://openresty.org/en/)
[+] [-] ahyesthd|2 years ago|reply
[deleted]
[+] [-] ibizaman|2 years ago|reply
It’s funny because as a passionate self-hoster, I consider as an advantage what you consider as a drawback.
[+] [-] mdaniel|2 years ago|reply
I'm so jealous of PDS2 in the EU <https://www.berlin-group.org/> (although I guess it's easy for me to be jealous since I don't know if their banks actually implement the standards, but it's still a bunch better than the situation here)
[+] [-] teeray|2 years ago|reply
[+] [-] jumploops|2 years ago|reply
We do something similar with our tool (generative code thingy), where users can make API calls to various services (without credentials), and then we add the necessary auth tokens/headers for their code to actually run.
I'd love to give folks better control over their own data/credentials, Lockbox could be an interesting workaround without us having to build anything. Thanks and nice work!
[+] [-] jjak82|2 years ago|reply
[+] [-] throwaway888abc|2 years ago|reply
https://github.com/stripe/smokescreen
Smokescreen is a HTTP CONNECT proxy. It proxies most traffic from Stripe to the external world (e.g., webhooks).
Smokescreen restricts which URLs it connects to:
It uses a pre-configured hostname ACL to only allow requests addressed to certain allow-listed hostnames, to ensure that no malicious code is attempting to make requests to unexpected services. It also resolves each domain name that is requested, and ensures that it is a publicly routable IP address and not an internal IP address. This prevents a class of attacks where, for instance, our own webhooks infrastructure is used to scan Stripe’s internal network. Smokescreen can also be further configured to allow or deny specific IP addresses or ranges.
[+] [-] itissid|2 years ago|reply
- Location anonymization to fetch any resources like restaurants.
- Sensitive personal data encryption to verify credit info securely.
My guess is not all of the above could be easy/possible, its a leaky abstraction to think of APIs that way. But the biggest bang for the buck could be for privacy in the most mundane of usecases.
[+] [-] kevincox|2 years ago|reply
1. Run a Lockbox server
2. Define the API request in automation tool as a webhook.
3. Proxy it via lockbox to add auth.
All this really does it allow you to revoke auth at any time. As they can freely use the key. I see there are plans to restrict the requests allowed but doing that on the fully composed request it going to be extremely difficult to do reliably.
If you are already hosting some API why not do something like this instead.
1. Run a Lockbox server.
2. Send a webhook from the automation tool that just contains the relevant variables.
3. Lockbox composes the variabless into a request.
Basically if you want any sort of security and access control that is reliable you just want to send the parameters of the operation to Lockbox, then have the actual request generated on your server. Rather then doing the request generation in the automation tool then trying to validate that it was generated correctly.
[+] [-] captn3m0|2 years ago|reply
[+] [-] ThePhysicist|2 years ago|reply
[+] [-] AdamJacobMuller|2 years ago|reply
> You can restrict access to external APIs in a more fine grained manner
I've done this very successfully for many APIs before. I've found weird things among providers where using AWS (or GCP or Azure) gives you crazy fine-grained access controls (which are great after you spend two days figuring out how to use them well), but, some of the low cost competitors have core services that work just as well but their APIs are entirely binary (either you have full access, or none) but by adding a thin API layer above that you can enable all sorts of useful things. Provider doesn't support R/O API keys for terraform plans? Just run a proxy which enables that. Provider won't allow you to give someone an API key which can only reboot VMs, but, not delete them? Just write a proxy which does.
Very powerful model if you adopt it well.
[+] [-] mikeknoop|2 years ago|reply
Relevant to OP, all platform auth creds are secured and not exposed after initial setup.
Separately, we just launched custom actions which allows anyone to extend existing public apps on Zapier with new endpoints (https://help.zapier.com/hc/en-us/articles/16277139110157-Cre...)
[+] [-] baconhigh|2 years ago|reply
Noho ora mai
[+] [-] tjpnz|2 years ago|reply
[+] [-] LysPJ|2 years ago|reply
I really liked the layout:
I wish more projects would follow this format :)[+] [-] unknown|2 years ago|reply
[deleted]