(no title)
ericb | 6 months ago
* LLM's are lousy at bugs
* Apps are a bit like making a baby. Fun in the moment, but a lifetime support commitment
* Supporting software isn't fun, even with an LLM. Burnout is common in open source.
* At the end of the day, it is still a lot of work, even guiding an LLM
* Anything hosted is a chore. Uptime, monitoring, patching, backing up, upgrading, security, legal, compliance, vulnerabilities
I think we'll see github littered with buggy, unsupported, vibe coded one-offs for every conceivable purpose. Now, though, you literally have no idea what you're looking at or if it is decent.
Claude made four different message passing implementations in my vibe coded app. I realized this once it was trying to modify the wrong one during a fix. In other words, claude was falling over trying to support what it made, and only a dev could bail it out. I am perfectly capable of coding this myself, but you have two choices at the moment--invest the labor, or get crap. But, then we come to "maybe I should just pay for this instead of burning my time and tokens."
mccoyb|6 months ago
One technique which appears to combat this is to do “red team / blue team Claude”
Red team Claude is hypercritical, and tries to find weaknesses in the code. Blue team Claude is your partner, who you collaborate with to setup PRs.
While this has definitely been helpful for me finding “issues” that blue team Claude will lie to you about — hallucinations are still a bit of an issue. I mostly put red team Claude into ultrathink + task mode to improve the veracity of its critiques.