[eden_hazard]

I feel like most engineers on hackernews/reddit have been underestimating AI tech. People always say it sucks and it gets starts hallucinating nonexistent libraries and what not but we are still in the early phase of this technology. And that how it could never replace them as AI will make current engineers more efficient.

I've been scared of AI since seeing chatgbt a couple of years ago. I feel like it's only a matter of time until a dev can feed an AI machine it's entire code base and business requirements. And then a separate AI could carry out the manual/integration testing tasks. AI could potentially cut down the number of devs required to maintain a webapp or ios app after it's built.

I feel triggered by this post especially because I've made it a career writing automation code haha.

[xyst]

HN also underestimated cryptocurrency craze. Yet years later, it is busted.

Seeing the same patterns with AI. Every startup is now incorporating “AI” or “deep learning” or “OpenAI” into their decks/motto/pitch.

Have yet to see anything worth using beyond the initial hype. Using AI to me is like learning another programming language. Same shit. Different interface.

[65]

A lot of the AI programming things you can do would be easier to either

1. type out yourself 2. copy and paste from StackOverflow 3. find a library that does the thing you want

It's not like CoPilot is any better. It's like when Microsoft and Google tried to force text completion on emails, it just gets in the way and makes me lose my train of thought.

AI is really great for very specific tasks that would be difficult to incorporate into a traditional algorithm. I really like Photoshop's background removal tool for example. But general purpose AI to me is blown out of proportion in terms of hype. Not everything needs iPhone levels of scaling. 3D printing, VR, Web 3, Cryptocurrencies, NFTs, Metaverse, AI. The list goes on. These things have niche use cases. AI is great for a lot of niche use cases (video upscaling, for example). But for general purpose software interaction? Maybe not.

[dotty-]

Is cryptocurrency busted? Sure, it didn't replace the global monetary system, but it seems to be a proven technology now. And sure, it had its crashes, but BTC is up 112% over 1Y, and lots of cryptos have legitimate payment purposes.

[seba_dos1]

Just like cryptocurrencies, transformer models have their valid use-cases, but are being so massively overhyped and oversold that it prompts natural resistance and spoils the whole thing. The result is that whenever someone talks about "AI" (or "crypto"), it's just safest to assume that they have no idea what they're talking about and want to sell it to you because more people getting hyped is how they get their investment back.

[koliber]

Our grandfathers underestimated the steel manufacturing craze and a century later, it is busted.

[jameshart]

It's actually worse than that.

Why do we need web apps or iOS apps? They're just task specific computer interfaces.

It's possible that this kind of AI tech eliminates the need for task specific computer interfaces at all.

You don't need to tell the LLM to code up a TODO list app so you can sell it in the App Store.

The user doesn't even need to tell an LLM to make them a TODO list app.

Given an LLM that can persist and restore context, the user can just use the LLM as a personal assistant that keeps track of their TODO list.

Whatever the software is that we're working alongside our AI colleagues to build in ten years' time, I don't think it's going to be automated tests for apps and websites.

[amluto]

I’m somewhat skeptical, at least for LLMs per se. They are language models. They have, in many respects, superhuman abilities, but I wouldn’t really want to trust things that require accuracy to an unaided human or even an unaided superhuman.

I would expect better results from LLMs using programming languages, perhaps ones tailored to LLMs, to prepare tasks on behalf of their users.

(Also, LLMs doing anything direct are an incredibly inefficient use of computing resources. There are quite a few orders of magnitude of difference between the FLOPs needed to do basic calculations and the FLOPs needed to run inference on a large model that may be able to do those calculations if well trained.)

[karmasimida]

Task interface is for human though, when LLM goes wrong, you can intervene. But I agree, it will be less critical moving forward.

[wolverine876]

This stage is called denial. There's also anger, bargaining, depression, and finally, in the end, acceptence. I'm saying in a funny way, but I suspect it's true.

Who can say how AI will develop, but beware of happily-ever-after stories. It could be a nightmare for all of civilization, for just engineers, or just not develop much futher.

[jhrmnn]

The problem with this is that it could be a universal argument against any skepticism. So unless skeptics are always wrong, it doesn‘t really work

[hooverd]

Nah, it will bifurcate the industry between people who have a mental model of how things work and people who only know how to press the code dispenser button. Offshore teams will be able to sling 10x times the slop 10x times faster.

[mtrovo]

And this is the worse it will ever be at doing this task, it's impressive in a scary way.

It's extra funny when you think that resilience was the first thing Sam Altman answered when asked what kids should be learning today .https://youtube.com/shorts/OK0YhF3NMpQ

[tomatohs]

We're building an AI Agent that can perform manual testing on feature branches [1]. I can tell you, it works, and it's going to get better, and it's going to happen fast. It's not hard at all for an AI to read text on the screen and click it.

What's amazing is the social impact this has - often people don't believe it's real. It feels like when I had to explain to my parents that in my online multiplayer game, that the other characters were other kids at home on their own computers.

I think it's a matter of denial. Yes, software is made for humans and we will always need to validate that humans can use that software. But should a human really be required to manually test every PR in 10k person teams?

Again, as a founder of an AI Agent for E2E testing, we work with this every day. If I was a QA professional right now, I would watch the space closely in the next 6 months. The other option is to specialize in the emotional human part like in gaming. You can't test for "fun."

1. https://testdriver.ai. Demo: https://www.youtube.com/watch?v=HZQxgQ1jt4g

[qlk1123]

> You can't test for "fun."

Sounds intuitive, but there are gaming researches working on that regard. Two related terms (learnt from IEEE Conference of Games) that come to mind:

1. Game refinement theory. The inventors of this theory see games as if they were evolving species, so this is to describe how game became more interesting, more challenging, more "refined". Personally I don't buy that theory because the series of papers had only a limited number of examples and it is questionable how related statistics were generated (especially the repeatedly occured baselines Go and Mahjong), but nonetheless there is theory on that.

2. Deep Player Behaviory Modeling (DPBM): This is the more interesting one. Game developers want their game to be automatically testable, but the agents are often not ready or not true enough. Says AlphaZero for Go or AlphaStar for StarCraft II, they are impressive ones but super-human, so the agnet's behavior give us little insight on how the quality of the game is and how to further improve the game. With DPBM, the signature of real human play can be captured and reproduced by agents, and thus auto-play testing is possible. Balance, fairness, engagement, etc. can then be used as the indirect keys to reassemble "fun."

[dvngnt_]

Your site says 'No more writing automated tests or waiting on manual testing.'

but this solution only appears to do e2e testing ignoring api and unit testing. Additionally, automated test are mostly used for regression testing not exploratory testing of new features where most bugs will be found.

[gerdesj]

Well, you say we underestimate but are you sure you are not the "underestimater"?

Amongst other things, one of the tasks I tried to put ChatGPT to was writing scripts in a not so popular dialect of a very popular language: PowerCLI (PowerShell). Gemini is even worse!

The issue is of course the relative lack of PowerCLI vs the huge body of PowerShell generic stuff. Hallucinations include invented function parameters and much worse. It doesn't help that PowerCLI and MS's Hyper-V effort (whatever that is) both have a Get-VM function etc.

These things are "only" next token/word guessers. They are not magic and they are certainly not intelligent. I do get great results in other domains and with a bit of creativity but you have to be really careful.

No need to feel triggered. Use these tools as best works for you and crack on but do be careful to be an engineer and critically examine the output from the tool.

[cjbprime]

> These things are "only" next token/word guessers.

This is precisely the kind of vacuous "this is technology, I know technology, this is simple" hubristic underestimation that's being called out.

There is no upper bound to the intelligence of a "next token/word guesser". You can end up incorporating an entire world model to your predictions to improve their accuracy, and arguably this has already happened, to a currently-unreliable and basic level. It is possible that no technological advances are required to reach better than human intelligence from this point -- only more compute, bigger models and datasets, and (therefore) better next-token predictions.

[saurik]

If you can automate my job--and I maintain we not only aren't there yet even but still won't be even if you make it so I never have to write another line of "syntax" ever again (as that isn't now and never was the job of a programmer)--you clearly can automate the jobs of most of the value chain above me, all the way up to the limited partners at the VC firms being paid management fees to pick portfolio companies... the issue here honestly isn't "programmers will be replaced and y'all are in denial", it is "humanity is going to have a crisis of purpose when capitalism can't figure out what to do with an entire civilization of meatbags that suddenly become cost in-effective for most any purpose" and I don't know what you expect anyone to do about that other than like, stock weapons or whatever (as there simply aren't enough jobs doing skilled physical labor to absorb enough people that betting on civilization feels like it makes sense) :/.

[Swizec]

> what to do with an entire civilization of meatbags that suddenly become cost in-effective

Services! All sorts of things are going to become cost-effective that currently aren't.

Want a personal trainer? Motivational coach? Someone to sit next to you slapping your phone out of your hands every time you open social media? Personal runner doing errands? You can afford that now!

We're going to have an ever-shrinking pool of highly critical un-automatable people with an army of support folk keeping them running at peak productivity at all times.

You already see this trend in people who think of themselves as a business. They hire everyone from personal assistants to nannies. All in the name of "Well I make $200/h and there's this chore that costs only $50/h to delegate ..."

[skydhash]

People keep forgetting that the value is not in the code itself, but in the solution that it realizes. Right now, you need developers to design and implement the code, and because they're humans, you need, people to manage them, people to manage everyone, people to manage salaries and expenses,... If you automate the job that is closest to the product, you remove the whole stack up to the founder. And history has proven that a founder is not really essential when the business is already running.

[twobitshifter]

I would extend that from engineer to knowledge professionals in general. Our specialized tools, strange syntaxes, and invented worlds of code may shield us somewhat, but when you just want expertise and opinion typed out? That’s firmly in the LLM ballpark. LLMs are starting to nail memorization with perfect recall, beating humans. My own memories are always fuzzy and I need emails and notes and books to help me along. An LLM won’t need all that and even if it did it can access all of that much quicker than I.

[Jerrrry]

I'm a "KM" "SME" and I can tell you that there is no doubt that every productivity gain in the last year has been $bigcorp employees feeding pseudo-anonymized business requirements and contexts into ChatGPT.

OpenAI + any half-assed data broker could easily infer the company, as I am sure they have already done.

All hail Microsoft, I am glad I chose the right AI megacorp overload early.

[littlestymaar]

I feel literally the opposite: the reason why I'm disappointed by the current state of AI is because I really want it to work so I can focus on the actually useful part of my job instead of doing boring stuff.

What's the useful part of my job? It's bringing values to my customers: they want to do something, and they need help. But today, unfortunately I must always tell them to reduce their ambition because they wouldn't be able to afford it or to be stuck waiting for the project to deliver even if they could fund it.

I've seen a enormous shift in developers ability to actually ship valuable products to customers when opensource became mainstream, thanks to github and things like npm: no more custom half-backed libraries for doing all the features necessary for the product to work, we could just re-use an existing library. More than half of our job disappeared these days, yet nobody regrets the days when you had to write your own code for absolutely every features, and the number of programmers exploded since then[1].

I wish AI assistant could be as impactful as github and npm, and I'm pretty sure they will eventually, and that day will be a great day for developers, not a bad one.

We're not going to lose our job, because from the perspective of the dude who hold the money, our job is to be the weirdo that talks to machines to deliver what the his ego want his company to be. Hence, the more you are able to deliver, the happier the money holder, and the more money you make. Our job will be threatened when the guys with money will be willing to actually make the effort of talking with the machine, but I don't see this day coming anytime soon. The ambition of man is unlimited, but his will to make efforts is in scarce supply.

The only realistic risk with AI, is big corporations grabbing all the benefits of the added productivity, this is a serious risk, and a very good reason not to be using OpenAI so we don't trigger a self-reinforcing feedback loop that give them a monopoly position where we all end up losing because we depend on them.

[1]: this has caused lots of sustainability issues for the library authors, but not for the developers using the libraries.

[cauliflower2718]

The machine has been getting easier to talk to over time. The machine also doesn't complain or get tired or sick.

[g-b-r]

> from the perspective of the dude who hold the money, our job is to be the weirdo that talks to machines to deliver what the his ego want his company to be. Hence, the more you are able to deliver, the happier the money holder, and the more money you make. Our job will be threatened when the guys with money will be willing to actually make the effort of talking with the machine, but I don't see this day coming anytime soon.

There's plenty of managers who prefer hacking an excel and vba than relying on developers; those will probably be more than glad of using some IA tool, to do something more advanced; of course mostly just for internal stuff, in the short term.

Anyhow many managers have a distaste for developers; they consider them overpaid slackers who'll lose time on anything, rather than producing money for the business.

They'll be more than glad to turn the job to the cheapest unqualified guy around, if with an LLM he can make something that looks passable

[QuantumGood]

I'm frankly amazed its been tried/tested as much as it has. Sure, it's amazing compared to where it used to be, very facile with language, but it's really very bad a lot of the time at present, especially compared to an employee who's supposed to know something about what is being worked on. Skillful users can get benefit from it in narrow situations, but it really needs to improve a lot.

[dvngnt_]

i don't see how the posted video is easier than just using the playback and recording tools that have been around for years

[pton_xd]

What's the new "just learn to code, bro" platitude we can throw around?

[FirmwareBurner]

"Just learn to do plumbing bro."

Seriously, the demand for skilled handymen in the cities is insane as people can't do shit anymore. As per the South Park episode, you have to treat them well if you want them to pick up the phone or return your calls.