Launch HN: Slauth (YC S22) – auto-generate secure IAM policies for AWS and GCP

[DanielSlauth]

Hi HN, We're Daniel and Bruno and working on https://slauth.io/. Slauth.io is a CLI to auto-generate secure IAM policies for AWS, and GCP (Azure in the next few days!). We enable development teams to speed up creating secure policies and reduce over-permissive policies being deployed to the cloud.

Check out the video or give our open-source CLI a try with one of the sample repo's on https://github.com/slauth-io/slauth-cli https://www.loom.com/share/bd02211659eb4c7f9b335e34094b57cb?...

We got into the cloud access market by coincidence and were amazed by the amount of money spent on IAM. Current tooling such as http://Ermetic.com and http://wiz.io/ visualize IAM misconfigurations post deployment but don't actually change engineering behavior, leaving organizations in a constant loop of engineers deploying over-permissive policies ⇒ security engineers/CISO's getting alerts ⇒ Jira tickets created begging developers to remediate ⇒ New over-permissive policies being deployed again.

We interviewed hundreds of developers and DevOps engineers and discovered two key pain points:

1. *IAM is a Hassle:* Developers despise dealing with IAM intricacies. 2. *Speed vs. Security:* IAM was slowing them down in deploying quality code swiftly.

So the objective is automate policy creation so that developers don't have to deal with it, and harden IAM security pre-deployment.

We employ Large Language Models (currently OpenAI GPT-4) to scan code in any language. Through a series of prompts, we identify service calls and the actions required. The resource name receives a placeholder unless its embedded in the code. We aim in the future to create a static code analyzer in order to not send any source code to LLM's but for now using LLM's is the fastest way to market and somewhat accepted by the industry through the use of Github Copilot etc.

You can use the CLI in the terminal or actually integrate it in your CI/CD and have it become a part of your development team workflow.

Three main questions we receive

1. *Security Concerns:* How can I trust [Slauth.io](http://slauth.io/) to access my source code? 2. *Policy Accuracy:* How can I trust [Slauth.io](http://slauth.io/) creates the right policies? 3. *Differentiation:* How are you different from IAMLive, IAMBic AccessAnalyzer or Policy Sentry?

To address the first concern, we don't access your code directly. Instead, we offer a CLI that integrates into your CI/CD pipeline, allowing local code scanning. http://slauth.io/ uses your OpenAI key to convert the code into a secure policy, with the option to output results to *`stdout`* or a file for artifact upload and download. That does mean OpenAI has access to the source code located in the path you set to be scanned as we need to know which SDK calls are performed to generate the policies.

We have extensively tested it on AWS , Typescript and GPT 4 with very good results (>95% accuracy). We do know these accuracies drop when using GPT 3.5 so if possible, use GPT 4 as we are improving the prompts. GCP and Azure have been tested less but the results when using GPT 4 seem equally high. We also have experienced some hallucinations but they have not effected the outcome of a secure policy but merely the structure of how the policy is generated. That is not to say that it is 100% reliable hence we aim to provide toolings to double check policies through policy simulators and other means.

Compared to competitors, we focus mainly on generating secure policies pre-deployment and automating as much as possible. We were inspired by IAMLive but it wasn't as scalable to use across development teams. Policy Sentry is great for templates but with http://Slauth.io you actually get a granular least privilege policy. Lastly, access analyzer is used to harden security policies which have already been deployed which is similar to other cloud scanning tools and creates a strange reactive process to security. The new access-analyzer feature checks policy diffs in your CDK but again doesn't actually generate the policy pre-deployment.

We recognise some engineers are very capable of creating secure policies but similar to using Checkov and TFscan in the context of IaC deployment, we believe using Slauth.io will become a necessity in your CI/CD when deploying service permissions to make sure no IAM misconfiguration appear in the cloud.

Would love to get your feedback and feel free to interact on our Github repo and join our Slack community.

[lijok]

I'd like to challenge you on what seems to be the main claim behind why Slauth is a necessary product: "the amount of money that is being spent on tooling to scan for IAM misconfigurations in the cloud".

1. The tooling you're quoting specifically, wiz.io and ermetic.com do an incredible amount more than just "scan for IAM misconfigurations". In fact, I understand that to be one of their most insignificant features. Yet it sounds, from the numbers being quoted (I saw the "millions" figure being thrown around), that you are equating a company purchasing wiz.io as them purchasing "tooling to scan for IAM misconfigurations" exclusively. How much does the IAM scanning tooling actually cost, and what is the material cost of delayed remediation of over-permissioned entities?

2. Were a company to introduce Slauth into their stack, are you under the impression that they would then not need to scan their IAM for misconfigurations and would therefore be able to save "millions"? Would it not be fair to say that the presence of Slauth would not remove the need for IAM scanning tools, since IAM deployments could happen out of bounds, which is not something that Slauth removes from a companies threat model?

[DanielSlauth]

>I'd like to challenge you on what seems to be the main claim behind why Slauth is a necessary product: "the amount of money that is being spent on tooling to scan for IAM misconfigurations in the cloud.

The quote you use got me to further research the market and speak to users of those toolings. From speaking to the users it was evident that the amount of misconfigurations being deployed wasn't being reduced.

I imagine users of cloud scanning tools would also use a pro-active tool like Slauth or any other shift-left tool that would aim at preventing as opposed to reacting.

[verdverm]

Repost: https://news.ycombinator.com/item?id=34038663 (11 months ago)

> We employ Large Language Models (currently OpenAI GPT-4)

For IAM, this seems like a disaster waiting to happen. Combining hallucination problems with security settings is not a path I would consider

[DanielSlauth]

Do you think humans are doing a better job? Research shows that 95% of the permissions granted to users aren't used which creates huge problems and is a reason for spending millions in security tools. Why not use Slauth and other checks such as policy simulators to get tightened policies pre-deployed

[debarshri]

It feels like taking one security problem and creating another problem.

[jgalt212]

I dunno. LLM generated config + formal verification could work.

[coredog64]

If it was me, I’d still run QC tools on the generated policy just like I would for manually authored policies. Specific to AWS, the IAM Access Analyzer will confirm that you’re using correct grammar. Further, there are techniques like SCP and permission boundaries to downscope what would normally be all actions/resources.

[justrealist]

The space of "real" options in IAM is small enough that hallucination is not a real problem.

Anecdotally I've used copilot to help write a lot of IAM polities in Terraform and the accuracy is basically 100% already.

[jedberg]

Why are you using (very expensive) GPT, or any LLM for that matter, when this was already a solved problem using rulesets? Netflix for example has open source that does this already: https://github.com/Netflix/consoleme

Instead of analyzing your code, you just run your code with no permissions and it automatically detects permission failures and thens open those permissions, with a UI showing you what it did so you can remove any permissions you don't want.

That actually seems much more secure than trying to divine the rules from reading the code.

What value is the LLM adding here?

[codegeek]

Not to knock on the OP but in general, if you are doing a startup in 2023, you cannot do it without AI otherwise no one will take your seriously. I am not joking. AI is the new Gold Rush that blockchain used to be. Personally, I do think that AI is awesome and has lot of great use cases but unfortunately, most VCs/Investors are looking for that keyword if you wanna get funded so I feel a lot of startups are forcing AI into their stuff.

[nextworddev]

So it can call itself an AI startup?

[Eridrus]

IAM is horrific, but I feel like it's not really the application-specific stuff that is annoying for me, it's the stuff that AWS wants configured for AWS features to work and the fact that the error messages when you get it wrong are useless at pinpointing your mistake, when you do not know that access is mediated by IAM.

Just as an example, I setup a containerized app on Fargate with a custom role, and the need to configure the ability for ecs to assume the role, read from ecr, write to cloudwatch, create channels for debugging was super annoying.

Comparatively, having a policy for it to read from an s3 bucket with my data was trivial.

[teaearlgraycold]

I just set up an S3 bucket - probably the most common use case for IAM policies. My policy file was invalid in a way that AWS never warned me about and looked good to my untrained eyes. After a few hours of debugging GPT-4 was able to explain I needed to break up my rules into bucket-level and key-level sections. Afterwards the 403 errors went away.

Just sharing my story. IAM sucks and GPT-4 is a good backup for configuring it.

[lasermike026]

You get used to it.

[biccboii]

If I were going to use ChatGPT to generate my IAM policies, why do I need a middleman to do that?

[sbarre]

It sounds like the value they bring is the custom prompts they've written?

And probably some quality-of-life wrappers around all that process?

[Eumenes]

The problem with IAM from my experience is it's never truly owned by a single entity. If you have an IT team, its sometimes them. Sometimes its devops, sometimes security. However as a startup grows, the owners change. Policy is rarely developed from the ground up and more patchwork to accomodate teams or timelines

[nutbear]

Yes. Good points. Agreed with patchwork as sometimes IAM can take a backseat to different priorities such as application development or feature development.

There's a couple different models for IAM ownership. At some places, the application teams own IAM along with the application. Sometimes, it's owned by central teams (such as security).

And agreed, with companies growing and changing, ownership changes as well.

Those factors can all complicated IAM development and policy maintenance as it becomes more difficult to find the right fit for IAM to application. For that, it would require someone who knows exactly what the application needs access to and the IAM actions taken as well as how to configure IAM.

[joshuanapoli]

I was excited to read your project description. It would be really great to automatically align the security policy for each component with the intent of the component's author. Tightening an overly permissive policy is an awful job. I think that it often has to be done through a long trial-and-error process; remove all the permissions, and add back permissions one by one in response to observed program failures. So it's great to see another way to avoid that tedious chore.

A challenge with Slauth will be to organize the generated policies in a way that makes them legible. I would like the IAM policy to help clarify its intent. Allowing each in-use API endpoint is technically required to let the service work. It might be technically following the principal of least privilege. But the endpoint-by-endpoint rules do a poor job of summarizing the purpose of the policy or how it relates to the service. One way that we do this is by having resource providers declare managed policies that allow use of the resource. So the "encabulator" provider also defines a "mutate encabulator" managed policy. Then services that need to invoke "mutate encabulator" can reference the managed policy. They don't need to compute the ARN of every endpoint. The dependent service doesn't end up with an inline policy that has detailed dependencies on the implementation details of each target resource.

[justchad]

I imagine there will be times where the LLM hallucinates or gives too many permissions. I'm sure it is still better than the try and see approach that most humans take when it comes to IAM setup. I'm just wondering if this product becomes irrelevant as Amazon Q rolls out and probably has some of this functionality baked in.

Regardless, anything to make IAM provisioning easier is worth a go as long as you verify the results using a simulator or something.

[DanielSlauth]

Thanks and let's see what Q will look like. I'm hoping the project will further evolve in integrating it in your CI/CD with PR's when commit requires IAM changes.

[bradleyy]

I love the idea of this.

The big disconnect here is I can't share code with OpenAI for various reasons.

Would you consider using something like AWS Bedrock+Anthropic Claude, where we have better (more predictable/risk profile) control over data-sharing, etc?

[SOLAR_FIELDS]

I’ve thought for a long while that using OpenAI on things that touch internal infra/data components is closing off a huge potential market. I’ve had a few people on the cutting edge of AI tell me that the risk profile is acceptable for a lot of companies, but I’m extremely skeptical that’s the case based on my own experience building cloud and data infra SaaS. Corps want self hosted solutions for their most critical components. I might be proven wrong, but the people who are telling me I’m wrong often have a vested interest in that not being true (building some SaaS product where the only way hockey stick growth the VC’s want can be achieved is by putting the eggs in the cloud basket)

[sbarre]

Yeah we're in the same boat, I was somewhat excited about this at first read and then I got to "LLMs" and "OpenAI" and I just stopped reading. :-(

But I'm sure others are ok with it, so that's great.

[callalex]

Can someone explain to me what is so difficult about writing security policies? Are people really deploying services in production without understanding the upstream and downstream dependencies of the service?

Also at cloud-scale 95% accuracy is completely unacceptable.

[nutbear]

IAM Policies in AWS are inherently difficult - there's a lot of nuance to the policies such as evaluation logic (allow/deny decisions), resource scoping, conditionals, and more. It's often more straightforward to start with a broad IAM policy and then leave it without reducing privilege as to not adversely impact the application. Proper IAM also takes dev cycles, and may not be top priority to get a policy correct. I think it's rare to find a 100% properly scoped IAM policy for an application.

Datadog recently did a State of Cloud Security and one of their findings in https://www.datadoghq.com/state-of-cloud-security/ is that a substantial portion of cloud workloads are excessively privileged (with more data points there).

[vrosas]

> Are people really deploying services in production without understanding...

Oh you sweet summer child. But in reality I've seen the pattern over and over, especially in GCP. 1. Create service account 2. Give it Owner permission for the whole project 3. Download key and check it into source control 4. deploy.

[zebomon]

I know you can't take responsibility for the 5% failure rate GPT-4 produces, but maybe things change when you have your simulator running. At that point, what kind of SLA do you plan on offering with the service?

[verdverm]

More important than SLAs is who takes on the liability for these mistakes? With the changing laws and regulations for breaches, do I want to rely on an LLM that isn't going to own that liability?

[DanielSlauth]

I believe the minute you connect a Dev or staging environment to Slauth.io and we can run simulations and show divs we can offer pretty strong SLA's..

[lijok]

How are you dealing with invalid policies generated by GPT? For example, in your loom video and the gif on the website, the resource for the s3:PutObject permission is incorrect: it should be "arn:aws:s3:::my_bucket_2/*" not "arn:aws:s3:::my_bucket_2".

Does this support resource policies? If so, how are you ensuring serious security vulnerabilities such as confused deputy are not introduced by this tool?

[slalmeidabbm]

That's a very good example of the type of hallucinations that can happen, we still need to develop a way to double check that the generated policies are indeed valid and hopefully find a way to simulate them.

As is stands, Slauth doesn't support resource-based policies.

[nutbear]

Good catch on the bucket vs object level permissions with S3 and s3:PutObject.

I'd also be curious for future plans with resource policies as that's another layer of complexity to manage - where the resource policy would manage access to potentially many applications -> 1 resource. Vs 1 application -> many resources which I think is the use case Slauth is solving for initially.

Confused Deputy would be interesting, could be done via Condition Keys such as SourceArn and SourceAccount, but gets complex for cross-account use cases.

[debussyman]

I like this approach, I've always thought that IaC could be generated by scanning application code. Although I share the skepticism that IAM is the best place to start.

I'm curious though how well an LLM performs for newly released AWS services? This is where I've experienced the most arcane IAM definitions personally, but I wonder if GPT 4 is trained well enough on newer sources.

[Boxxed]

Interesting use of GPT; it's cool that it works as well as it does but I'd be nervous about the various insidious ways it can fail.

On another note, are there tools that will scan your AWS/GCP logs and emit configuration to limit permissions to what you're actually using? I could even see GPT doing better here too, or at least it would be easier to test.

[slalmeidabbm]

We're currently focusing on a full shift-left approach to policy creation. Using AWS/GCP logs to create policies would work very well but it would need a few things to happen:

1. The service needs to be deployed 2. To produce an actual result, the calls that make use of the sdk need to be triggered

This is something that would be better included as an addition to monitor policy usage and adjust.

[janalsncm]

My question is why I would need another service to do this when I’m already paying for gpt4 and I can already ask it detailed questions to generate an IAM config. Essentially you have built a GPT4 which does one thing, but I can easy get GPT4 that does many things.

[kkapelon]

A bit off topic but possible name clash/confusion with https://www.sleuth.io/

[srameshc]

Unrelated question: How can one learn more about configuring and securing IAM for Google cloud platform ?

[bsamuels]

At my previous job we used GCP and went through so much effort/tooling to try to fix IAM. We definitely would have given this tool a spin. Ignore the HN flashmob.

Another use case you might run in to as you talk with more clients is figuring out what developer IAM roles need to be. This was the far bigger problem for us as we had a ToS that restricted employees from viewing/accessing user data.

[nextworddev]

I guess this is their 2nd pivot after LLMs took off