(no title)
EigenLord | 10 months ago
>makes it easier to accidentally expose sensitive data.
So does the "forward" button on emails. Maybe be more careful about how your system handles sensitive data. How about:
>MCP allows for more powerful prompt injections.
This just touches on wider topic of only working with trusted service providers that developers should abide by generally. As for:
>MCP has no concept or controls for costs.
Rate limit and monitor your own usage. You should anyway. It's not the road's job to make you follow the speed limit.
Finally, many of the other issues seem to be more about coming to terms with delegating to AI agents generally. In any case it's the developer's responsibility to manage all these problems within the boundaries they control. No API should have that many responsibilities.
TeMPOraL|10 months ago
Well, yes. A knife cuts things, it's literally its only job. It will cut whatever you swing it at, including people and things you didn't intend to - that's the nature of a general-purpose cutting tool, as opposed to e.g. safety razor or plastic scissors for small children, which are much safer, but can only cut few very specific things.
Now, I get it, young developers don't know that knives and remote access to code execution on a local system are both sharp tools and need to be kept out of reach of small children. But it's one thing to remind people that the tool needs to be handled with care; it's another to blame it on the tool design.
Prompt injection is a consequence of the nature of LLMs, you can't eliminate it without degrading capabilities of the model. No, "in-band signaling" isn't the problem - "control vs. data" separation is not a thing in nature, it's designed into systems, and what makes LLMs useful and general is that they don't have it. Much like people, by the way. Remote MCPs as a Service are a bad idea, but that's not the fault of the protocol - it's the problem of giving power to third parties you don't trust. And so on.
There is technical and process security to be added, but that's mostly around MCP, not in it.
Joker_vD|10 months ago
And admonishments of "don't use it when people are around, but if you do, it's those people's fault when they get cut: they should've be more careful and probably wore some protective foot-gear" while technically accurate, miss the bigger problem. That is, that somebody decided to strap a sharp knife to a roomba and then let it whiz around in the space full of people.
Mind you, we have actual woodcutting table saws with built-in safety measures: they instantly stop when they detect contact with human skin. So you absolutely can have safe knives. They just cost more, and I understand that most people value (other) people's health and lives quite cheaply indeed, and so don't bother buying/designing/or even considering such frivolities.
skybrian|10 months ago
As usual, the question is what counts as a reasonable safety improvement, and to do that we would need to go into the details.
I’m wondering what you think of the CaMeL proposal?
https://simonwillison.net/2025/Apr/11/camel/#atom-everything
noodletheworld|10 months ago
> As mentioned in my multi-agent systems post, LLM-reliability often negatively correlates with the amount of instructional context it’s provided. This is in stark contrast to most users, who (maybe deceived by AI hype marketing) believe that the answer to most of their problems will be solved by providing more data and integrations. I expect that as the servers get bigger (i.e. more tools) and users integrate more of them, an assistants performance will degrade all while increasing the cost of every single request. Applications may force the user to pick some subset of the total set of integrated tools to get around this.
I will rephrase it in stronger terms.
MCP does not scale.
It cannot scale beyond a certain threshold.
It is Impossible to add an unlimited number of tools to your agents context without negatively impacting the capability of your agent.
This is a fundamental limitation with the entire concept of MCP and needs addressing far more than auth problems, imo.
You will see posts like “MCP used to be good but now…” as people experience the effects of having many MCP servers enabled.
They interfere with each other.
This is fundamentally and utterly different from installing a package in any normal package system, where not interfering is a fundamental property of package management in general.
Thats the problem with MCP.
As an idea it is different to what people trivially expect from it.
weird-eye-issue|10 months ago
Also, in my experience, there is a huge bump in performance and real-world usage abilities as the context grows. So I definitely don't agree about a negative correlation there, however, in some use cases and with the wrong contexts it certainly can be true.
TeMPOraL|10 months ago
You kept adding more tools and now the tool-master "agent" is overwhelmed by the amount of choice? Simple! Add more "agents" to organize the tools into categories; you can do that up front and stuff the categorization into a database and now it's a rag. Er, RAG module to select tools.
There are so many ways to do it. Using cheaper models for selection to reduce costs, dynamic classification, prioritizing tools already successfully applied in previous chat rounds (and more "agents" to evaluate if a tool application was successful)...
Point being: just keep adding extra layers of indirection, and you'll be fine.
empath75|10 months ago
kiitos|10 months ago
Huh?
MCP servers aren't just for agents, they're for any/all _clients_ that can speak MCP. And capabilities provided by a given MCP server are on-demand, they only incur a cost to the client, and only impact the user context, if/when they're invoked.
Spivak|10 months ago
The real level of trust is on the order OAuth flows where the data provider has a gun sighted on every integration. Unless something about this protocol and it's implementations change I expect every MCP server to start doing side-channel verification like getting an email "hey your LLM is asking to do thing, click the link to approve." Where in this future it severely inhibits the usefulness of agents in the same vein as Apple's "click the notification to run this automation."
zoogeny|10 months ago
A lot of these issues seem trivial when we consider having a dozen agents running on tens of thousands of tokens of context. You can envision UIs that take these security concerns into account. I think a lot of the UI solutions will break down if we have hundreds of agents each injecting 10k+ tokens into a 1m+ context. The problems we are solving for today won't hold as LLMs continue to increase in size and complexity.
ZiiS|10 months ago
A better metaphor is the car, not the road. It is legally required to accurately tell you your speed and require deliberate control to increase it.
Even if you stick to a road; whoever made the road is required to research and clearly post speed limits.
jacobr1|10 months ago
TeMPOraL|10 months ago
Don't blame roads for not being rail, when you came in a car because you need the flexibility that the train can't give you.
fsndz|10 months ago
Eisenstein|10 months ago
sshh12|10 months ago
peterlada|10 months ago
Like a bad urban planner building a 6 lane city road with the 25mph limit and standing there wondering why everyone is doing 65mph in that particular stretch. Maybe sending out the police with speed traps and imposing a bunch of fines to "fix" the issue, or put some rouge on that pig, why not.
Someone|10 months ago
In some sense, urban planners do design roads to make you follow the speed limit. https://en.wikipedia.org/wiki/Traffic_calming:
“Traffic calming uses physical design and other measures to improve safety for motorists, car drivers, pedestrians and cyclists. It has become a tool to combat speeding and other unsafe behaviours of drivers”
reliabilityguy|10 months ago
Good road design makes it impossible to speed.
unknown|10 months ago
[deleted]