The author doesn’t understand the reason for the bimodal distribution of attorney compensation. If you look at a distribution of salaries overall, they are not smooth. There will be small peaks at round numbers, $60k, $70k, 80k, etc., rather than a smooth curve from 60->80. People like round numbers.
The top peak in the law is due to human bias just the same.
>Just like how a partner at Cravath likely sketches an outline of how they want to approach a particular case and swarms of largely replaceable lawyers fill in the details…
Except those “replaceable” attorneys at Cravath are the top peak of the distribution. The distribution is caused by social decisions of what firms to compete with. Top salary information is well-publicized and firms who want to be in the category of “BigLaw” basically match each other.
The lower peak of the distribution is more local attorneys. Your local estate law attorney, government workers like district attorneys/public defenders, etc. who are not competing with Cravath for talent or clients.
I would assume this is already what exists in programming with “BigTech” salaries being equivalently publicized and matched across firms concerned with being perceived to be a “FAANG+”. If it doesn’t, I would guess the reason is that not enough positions actually exist in “BigTech” to make a significant blip on a population-wide salary graph. BigLaw attorneys exist in almost every medium-large city in the US, but you can’t say the same about BigTech programmers.
Law is extreme because you have people who went to a relative handful of law schools, clerked for federal court justices including SCOTUS, and work at white shoe big city law firms (like Cravath)--and everyone else. Yes, senior partners make more than junior partners make more than associates at those firms but it's also sort of up or out (and the out associates tend to move on to other things).
And then there are the, as you say, local attorneys doing routine real estate transactions or whatever. Which tend to be pretty middling compensation generally.
The story isn't quite that neat of course. Big company in-house counsels tend to be reasonably well compensated professional jobs and there are successful owners/partners of local law practices but there's a pretty good bimodal narrative in general.
You can imagine why there might be a modality for people who can land jobs at specific tech companies that pay outsized salaries because they're so profitable and can afford to be very selective based on (they hope) relevant criteria but it's certainly based on less clear metrics if nothing else.
Agreed. I was entering the law field around 2001.
1) During 1990's, silicon valley startups started poaching talented lawyers from west coast law firms.
2) To combat this, some top west coast law firms raised their salaries to retain talent.
3) At the same time, junior lawyers in NYC and CA starting sharing salary information on Internet message boards.
4) Almost over night, top NYC law firms raised their salaries to match the CA salaries.
5) So the bimodal distribution was formed from the "top" law firms all match each other's salaries.
There is research [1] that suggests using GPS navigation could impact your brain function negatively. I wonder how this code generation “assistant” will impact the coding skills of its users in long term.
As for the LLMs, stochastic parroting (hallucination), which is inherent to its design, makes it untrustworthy for precision work. Maybe some other architecture will prove more useful. LLMs can generate BS authoritatively which is easy to detect for toy examples and forgivable by the AI hype train but can be very costly or hard to fix in a complex system with a lot of moving parts.
I’m actually optimistic about the future of AI. At the same time I believe we should know the capabilities and understand the limitations of our tools in order to use them more effectively. Unfortunately people are misunderstanding tools like ChatGPT, etc. for we are human and vulnerable to anthropomorphism.
I wonder how this code generation “assistant” will impact the coding skills of its users in long term.
My brain is already negatively impacted having to juggle 3 separate package managers, know the syntax and inputs for 17+ different dotfiles in a repo and then actively context switch between at least 3-4 languages on a daily basis. I counted it up recently and I've built products in at least 20 languages over the past two+ decades. At some point you really just can't effectively compartmentalize it all and context switch perfectly. I try by using different fonts+colors per lang etc but it still is hard. AI has been a net win for syntax recall just by giving it gentle nudges towards intent.
I agree but there is a certain "god of the gaps" process that's happening here.
To me what's been achieved so far is encoding text into natural (and programming) languages. These models can't reason and e.g., can't pass a Turing test. For example, if you play with GPT-3 or ChatGPT enough you can definitely get it to fail on a task a human would succeed at.
When models so far have achieved "reasoning" behavior, usually it's in conjunction with a classical algorithm -- for example AlphaZero used a model to evaluate chess positions but then ran a tree search algorithm to decide what move to play.
I think if LLMs can be combined effectively with some kind of computation process like in AlphaZero or some kind of "chains of thought" [1], then we might see them do things we'd truly call "reasoning".
Some "gaps" have been (1) "(encoding into) images is hard" (2) "(encoding into) natural language is hard" and now (3) "'reasoning' is hard" and afterwards (4) "subjective experience is hard".
Whether to be optimistic or pessimistic about AI depends on the final "gap" -- whether there is an irreplicable/noncomputable/etc. property of human consciousness other than qualia. That is, if "understanding" is hard (e.g., via some voodoo like [2]). If not humans will become obsolete, so I hope the answer is yes.
> There is research [1] that suggests using GPS navigation could impact your brain function negatively.
Wrong, that research only shows that when you use a GPS, you don't learn routes as quickly. That is only equal to "impacts brain function negatively" in a narrow sense, which you should specify rather than use such a broad and scary-sounding phrase!
> There is research [1] that suggests using GPS navigation could impact your brain function negatively. I wonder how this code generation “assistant” will impact the coding skills of its users in long term.
I wonder how code completion and syntax highlighting have altered the brains of programmers negatively as well.
Most programmers wish to be stupid. They willingly embrace philosophies predicated on their inability to be conscious and learn.
Goddamn do I hate this industry. I work hard to sow dissent in my blue-chip, VC startup though. Guerrilla tactics. And I get to do it raking in a quarter mil a year in my underwear.
I just tried ChatGPT today after weeks of hype on Twitter, and I was surprised to NOT be impressed?
First I tried it on a bunch of programming topics -- it definitely gets in the right ballpark, but then the details are frequently wrong.
I asked which languages have the F# |> operator, and it told me that OCaml, Elixir, Haskell and Scala do. 2 of them are correct, and 2 appeared wrong, so I googled it.
After Googling, then I asked it specifically if Haskell does, and it says no. I ask it if Scala does, and it apologizes for its earlier mistake.
It gave me a good although slightly shallow answer about how PyPy works, but then fell down hard for a prolog type checker, and a few other programming queries. It doesn't understand the question.
---
It is generally very mealy-mouthed and vague. It often doesn't answer the question directly, but just tells you what it happens to know in the ballpark.
I like another framing I saw on Twitter -- it's "mansplaining as a service". If you want someone to confidently explain things you didn't really ask about, and be frequently wrong, that's kinda what it does!!
---
After programming, I asked it about Black Sabbath, UFC, travel and biking, health, and got similarly vague/slightly incorrect answers.
It confabulates A LOT.
The best I can say is that Google is filled with spam now, so chatting with ChatGPT was a relatively fun and focused experience, after I took into account the fact that it is not "honest" and just trying to "please me" with closely related words.
I honestly don't get the hype, and I feel like there must have been some coordinated marketing aimed at me ...
If this chat is representative of what it can do for programming, then I think this article saying that "programmer salaries will be affected by LLM" is very weird.
It is assuming something that simply has NOT happened!
---
I mean even Geoff Hinton (who I respect) said several years ago that "we should stop training radiologists", and that plainly did not happen. Likewise with Elon and the fleet of self-driving taxis.
I'd honestly be surprised if someone wants to argue that a tech company staffed with LLMs + a few smart people is NOT like those predictions. Of course the blog post doesn't give a time frame, so it can't be falsified. But it appears to be nonsense.
(FWIW it doesn't matter to me personally, as I left the industry several years ago. Also FWIW I did work a little bit on deep learning, when the hype was in its upswing.)
I agree that a lot of it is anthropomorphism -- projecting some kind of intelligence, knowledge, or intention on what is really a big, expensive bag of data organized to respond to inputs.
Maybe there are other models that are better, but I was really, honestly not impressed with ChatGPT. Maybe people were hyping it too much for the last several weeks, and inflated my expectations.
I mean "generate BS authoritatively" seems fairly similar to "fake it till you make it".. Some people writing code now generate BS authoritatively, some don't.
Comment here seem to largely miss the point of the article, which isn't saying that nobody will employ software engineers. It is saying that instead of the current model of:
1 very highly compensated staff+ engineer
4 highly compensated senior engineers
4 pretty well-compensated mid-level engineers
2 decently compensated junior engineers
The future might look like:
1 yet-more-highly-compensated staff+ engineer who specializes in architecture
10 decently compensated junior engineers who specialize in error-checking LLMs and detail-intensive technical work
In the same way that paralegals aren't unskilled people, and aren't paid terribly -- but they aren't doing the same job as the partner-level lawyer who's crafting the legal strategy, and they don't have a simple clean progression to eventually getting his job.
"Note that querying LLMs and making the outputted code conform to specifications is probably a lot easier than writing the code yourself"
This overestimates the time developers spend typing text and underestimates the time spent thinking about problems, verification and locating bugs/code, etc.
Just because the output of a model is text doesn't mean it is a replacement. You would need extraordinary "single shot" functionality where the LLM always outputs correct code and knows how to deal with any potential specification. The moment it fails then you wouldn't want "paralegal" developers, you would want normal, maybe inexperienced, developers.
Single shot functionality is very impressive because humans can't do it, they build things incrementally, but once your program gets large, it can only be written incrementally, which at least implies that the AI needs to keep track of unbounded context.
Much like the adage that you need to be smarter to debug code than to write code, you need to be smarter to "touch up" AI generated code than to write it yourself.
The problem with AI generated code is that it contains mistakes and because of the nature of how it works, those mistakes may be entirely unlike the kind of mistakes a human would make or may be completely undetectable unless you know what the code should be doing.
The best use case for AI code generation is auto-complete. Depending on the complexity of the code, that can be sufficient to do 90% of the work of writing code, but it's very different from "solving problems" which tends to be the tricky part. It also requires the AI user to already know the code they want the AI to generate before it does it.
An interesting application of AI outside of simple code generation however is what GitHub is currently experimenting with as "code brushes", i.e. "touching up" existing code, trying to fix bugs or apply refactorings. This fits in the gap of "single shot functionality" but operates on existing code rather than requiring a blank slate.
But at the end of the day I think AI will fail to kill programming just like other attempts like WYSIWYG UI builders and COBOL have and for the same reason: competence as a programmer is not about the ability to write code but the ability to understand programs and systems, both at scale and in depth. A beginner will not write bad code simply because they lack experience in using the language, they will write bad code because they lack understanding of what the code does and how it interacts with its environment (and how that environment interacts with it in turn, e.g. UX).
Only yesterday, I worked on a ticket for few hours, inspecting several Mongo collections in QE and prod. And then had to make 4 to 5 lines of changes across 3/4 python scripts. And then manually run them to patch the mongo collections. Which was the root cause of the bug. I don't see GPT doing this kind of stuff anytime soon.
And yes, very less percentage of the time, do we right chunks of fresh code, in which this could help. But that again to an extent. It can generate functions. But the glue code is still complicated. So it can definitely be a better Stack Overflow replacement, perhaps.
The smaller the problem the easier it is to use AI. Which makes the window where it's useful very small.
Its easier to understand and test code that you wrote than code you just read. If you ask it to write a 100k line program you're going to have to trace through everything and read it countless times over to fix any issues.
If you ask it to write a small 10 line snippet that you need to test you might as well do it yourself.
I think this makes assumptions about the demand for software engineers that seem unproven. Law salaries are bimodal because the demand for lawyers is limited, so there's only so many big law firms that can exist. In the near future, at least, it seems like demand for software engineers far outstrips supply and I think this will stay true in the long run because the market for software engineers is just different from the market for lawyers, in that software engineers can work on a multitude of different industries (faang covers personal devices, social media, entertainment, e-commerce, search, and more).
I think the real concern is for when the rate of new startups decreases, because I view startups as basically the tech industry's way for searching for ways to increase the overall size of the pie, and startups slowing down means demand for software engineers will start to stagnate.
> it seems like demand for software engineers far outstrips supply
Definitely not the case when layoff right now is in full force. Also this doesn't necessarily indicate high salary anyway. There is a huge demand for fruit picker, but due to the nature of economy and margin of business, the salary will stay deflated thus the shortage persists. In a high interests, no free money environment, will software firms still able to demonstrate ability to generate return for investors? That would the ultimate key for whether the high salary is going to sustain or disappear.
Regardless, I think this article has a point. We might be as well observing the end of the golden era for software engineers. LLM is a long-term salary oppressor, but in short term, simple dynamics of economy would test the market sufficiently.
AI/ML will solve some great problems, many that are critical. Similar to how Wix solves your basic webpage for the average person. Does Wix solve anything for your typical enterprise or SAAS app? Even if it did, would these institutions not manage to add a new layer to the layer cake?
It's kind of been one of my main critiques of companies like Palantir. You really can't just dump a company's data into the big ML tumbler and expect useful/sellable products to just pop out. Take a random use case like, I don't know, Pepsi. Maybe Pepsi can get some insights about their supply chain and customer preferences. Alright? Okay. Did the world end because that happened? Not really.
Shifting gears back to programmers - AI will be a super smart programmer that will write a bunch of your code. I worked with some super smart programmers, and they can manage to write a lot of super smart code that eventually becomes a clusterfuck too.
We'll just go from unwrapping human shit code, to unwrapping AI shit code (or even better, unwrapping a mix of human shit and AI shit). The job prospects here are even greater than you imagine.
Long as you maintain the belief in the fundamental truth, that anything in existence is capable of writing shit code, then the world will never end for programmers.
Sometimes though we shouldn't have the AI write the code first.
So many times, I look over things people copy and pasted piece by piece from blogs and stackoverflow, when if they thought about it for 5 minutes, they'd realize a simpler implementation would have worked and been easier to write in the end. AI, if chatgpt is any indication, is moving toward overly flowerly and long prose while telling lies, that is stuff that looks "impressive" at a glance like the large ornate class hiearchies of java and c++ in the 90s looked but are things that could be done in a shell script.
I am worried then that like stackoverflow, AI will make a generation of developers who are extremely lazy and not critically thinking, just like stackoverflow did, and leave more of a mess for us to pick up after. Of course, in the midst of this, management won't know the difference. Just like chatgpt sounding impressive as it tells falsehoods, AI-code will look impressive to fools.
I think it will be more like airline pilots with the addition of autopilot. Maybe you could write whole services, and networks of services with AI, but you need engineers to be the guiding hand. Engineers will always be better at using AI to build systems than non-developers.
Also, it’s worth pointing out that AI is using existing data. If a new iOS API comes out, the AI will need code bases with it already implemented to be able to train with it.
So in my mind, it will hurt the junior developers doing grunt work more than general devs.
I think no code/low code products are more of a risk to hurting devs than AI.
There is a lot of doom and gloom about programming and AI.
However, let me talk about something to be excited for.
I am really excited about LLMs taking the drudgery out of programming. There are many times what I have a project in mind that I want to do, but there is a lot of boilerplate (writing configurations, or writing tests, etc) that I have to do and slows me down or for personal projects even keeps me from doing them. However, if I have an AI assistant that I can assign all the drudgery to, and I can focus on the interesting work, I feel that my creativity can really be unleashed.
I am most excited to see the results of this in games. Right now, to produce a AAA game is a lot of work and requires massive teams (if only for the art assets). However, I can see a future where once again a single programmer can write very compelling games (and even have the AI generate the art assets for the game based on the programmer's vision).
Chatgpt has been pretty transformative to my programming, probably surpassing the only other thing coming close of really leveraging powerful ides like IntelliJ. It can generate all the scaffolding I need by just inputting a couple of sketches of Plantuml or even just markdown lists. I can build tools, unit tests, test data, fuzz harnesses, admin uis, dashboards, all within a few hours. I can transform a couple of words into very legible documentation. If you seed it with just a couple of lines of your own code and setup, it’s able to usually build stuff that matches the style of the codebase. It’s broken and wrong 80% of the time, but in a way that I can fix within a few minutes.
This leaves me with 4h of high quality thinking and programming time instead of 1. It means I can try out 3 different approaches with full tooling instead of 1. It means I can build a whole terraform for aws and gcp and bare iron in 30 minutes.
The fact that I can build solid tools and a full slurry of unit tests and end up with 4x time in my day is just blowing my mind. The quality of my software has tremendously gone up despite using something that mostly outputs broken code.
I'm going to be the optimist here and instead of AI killing engineering jobs it's going to turn most engineers into much better engineers and what tech company doesn't want a whole bunch of better engineers? Building applications right now is time intensive, it takes years to build products, months to build features, and maybe we change it to be weeks and days instead, respectively. Maybe it's all just upside.
I think people are more impressed than they should be because of a misconception about what programmers really do.
For most big software projects the bottleneck is not writing code and test boilerplate that runs correctly locally. Which of course LLMs can do reasonably well. Most time is spent on design, on integrations with other internal services, scaling concerns, and reasoning about other things that these models can't really reason about yet.
I'm obvious impressed that these models can generate simple client/server websites and generate code to solve competitive programming problems. But I don't think they've "solved" software engineering; they've only solved "what you boss thinks software engineering is".
>Building applications right now is time intensive, it takes years to build products, months to build features, and maybe we change it to be weeks and days instead, respectively.
You want to increase churn? Are you insane?
I feel like if you poll computer / phone users (ie., the vast majority of the population) one of the most annoying things users hate about modern devices are mandatory updates, like the infamous meme video where someone screams at their window machine updating. Now, you want to make it easier for middle managers to push new unneeded updates so they can pad their resumes, how absurd.
Speed of feature is less compelling than quality. There is a saying about EHRs, that they propagate error at the speed of light. Likewise speeding up feature accrual seems as if all the worst ideas will be implemented first as the best ones need time to think through and acculturate.
On the other hand, if the models can be Atkinson-like and produce -2000 lines of code [0], that is a day to which I would look forward.
What I want is an AI that can do static analysis of code and report defects that traditional static analysis couldn't hope to detect.
I suspect that detecting errors that way will be a part of training non-terrible code generators. But with the added benefit of being useful to human programmers as well.
The other day I pasted chatgpt some code that I wanted it to add functionality to and it pointed out a bug in that code first (without me asking it to do that)
Except you can't do a web search on proprietary code within your business, and hosting AI on-premises to model your codebase would likely be a huge waste of money and slower than just telling a real life person what you want.
There has been zero-progress towards AI comprehension, so you can only get out of it what you are already familiar with. The human must always do the mental heavy lifting to tease out a good result, or indeed even know if it's a good result and not just a hallucinated mess.
This post is way to bullish on ChatGPT for coding, and underestimates how big the gap is between ChatGPT and a mid level FANG eng. ChatGPT is very impressive in what it can do, but if you deviate problem statements even a little bit from what it's seen before, it completely fails.
Just an example try asking it to write Battleship in Python. It does very well. But then, try changing the rules a bit, and adding some requirements. It spits out code that looks correct if you squint, but has terrible style, and often doesn't even work. Check out the test prompt [1].
Maybe GPT-4 is 1000x better at coding, but barring that, I don't see ChatGPT being useful as a replacement for human coders.
[1] Prompt: Write a version of Battleship with modified rules for sinking a ship. In this version, a ship is sunk when the "head" and "tail" of the ship (its beginning and end parts) are hit. Hits to the center do not matter. Use object oriented programming, with classes for the Board and for Ships. There should be support for multiple ships and two board instances. Write it in Python.
Yeah I'd never heard LLM, and I know what ChatGPT is and read hacker news pretty much every day.
Just as an example for future writers – one line at the beginning of the piece explaining the acronym and using the ChatGPT example would go a long way.
If I did search & replace on this article, replacing “programmer” with “manager”, would the conclusions and reasoning be much different (regardless of whether they’re spot on or not)?
A funny thing I noticed when googling whether melamine was microwave-safe: I got a lot of clickbaity articles from sketchy sites. Some said melamine is microwave safe, some said it is not, and some seemed to contradict each other.
An Asimov story called "Liar!" concerns a robot which has the ability to read minds. When asked to tell what's going on in another person's mind (does so-and-so have a crush on me? is my boss planning to give me a promotion?) the robot, following its First Law directive not to allow humans to come to harm (including emotional harm), tells the querent what they would like to hear based on the robot's reading of the querent's mind, instead of the truth because the truth can hurt. When the robot is shown that its actions hurt humans anyway, it promptly shuts down and refuses to function.
These language models seem like that robot. The articles on melamine suggest to me that the AIs which generated them are producing output that their statistical models suggest the querent wants to hear, rather than output with correct answers. In light of this, it seems that using these models for assistance in coding would be like having an obsequious but shitty coworker, whose output you must carefully review and vet because even money it's not a solution to your problem, just a guess at what you want to see based on a vast statistical soup built up of thousands of programmers' efforts on millions of tasks. Which means any effort you saved up now will be paid for with extra effort later, as every programmer knows it's much easier and more fun to wrote new code than to review and debug someone else's code, especially if it's shitty, especially^2 if it's subtly shitty. It really is like Teitelman's DWIM evolved, with correspondingly magnified lossage in its failure modes!
> we are perhaps converging to a future where a FAANG L7 can just sketch out architectural details and the programmer equivalent of paralegals will simply query the latest LLM and clean up the output.
This is just proposing waterfall + outsourcing all over again. The author doesn't seem to understand that "sketching out architectural details" is the thing that developers do.
This is very doom and gloom compared to the actual landscape in law. Yeah entry level salaries suck in law, but plenty of decent lawyers who are nothing special end up billing 200 an hour 6 years into their career and taking their 40% of that. While outside of big law is not paying what it used to it is still an upper middle class profession if you stick it out.
Societies demand for software is basically infinite. Right now only people who can pay for developers get to have good custom software. As LLMs increase the output of a programmer more people will be able to comission custom software. More software will get made and programmer salaries will remain high.
[+] [-] elicksaur|3 years ago|reply
The top peak in the law is due to human bias just the same.
>Just like how a partner at Cravath likely sketches an outline of how they want to approach a particular case and swarms of largely replaceable lawyers fill in the details…
Except those “replaceable” attorneys at Cravath are the top peak of the distribution. The distribution is caused by social decisions of what firms to compete with. Top salary information is well-publicized and firms who want to be in the category of “BigLaw” basically match each other.
The lower peak of the distribution is more local attorneys. Your local estate law attorney, government workers like district attorneys/public defenders, etc. who are not competing with Cravath for talent or clients.
I would assume this is already what exists in programming with “BigTech” salaries being equivalently publicized and matched across firms concerned with being perceived to be a “FAANG+”. If it doesn’t, I would guess the reason is that not enough positions actually exist in “BigTech” to make a significant blip on a population-wide salary graph. BigLaw attorneys exist in almost every medium-large city in the US, but you can’t say the same about BigTech programmers.
[+] [-] ghaff|3 years ago|reply
And then there are the, as you say, local attorneys doing routine real estate transactions or whatever. Which tend to be pretty middling compensation generally.
The story isn't quite that neat of course. Big company in-house counsels tend to be reasonably well compensated professional jobs and there are successful owners/partners of local law practices but there's a pretty good bimodal narrative in general.
You can imagine why there might be a modality for people who can land jobs at specific tech companies that pay outsized salaries because they're so profitable and can afford to be very selective based on (they hope) relevant criteria but it's certainly based on less clear metrics if nothing else.
[+] [-] jimmydddd|3 years ago|reply
[+] [-] geraneum|3 years ago|reply
As for the LLMs, stochastic parroting (hallucination), which is inherent to its design, makes it untrustworthy for precision work. Maybe some other architecture will prove more useful. LLMs can generate BS authoritatively which is easy to detect for toy examples and forgivable by the AI hype train but can be very costly or hard to fix in a complex system with a lot of moving parts.
I’m actually optimistic about the future of AI. At the same time I believe we should know the capabilities and understand the limitations of our tools in order to use them more effectively. Unfortunately people are misunderstanding tools like ChatGPT, etc. for we are human and vulnerable to anthropomorphism.
1. https://www.nature.com/articles/ncomms14652
[+] [-] kirse|3 years ago|reply
My brain is already negatively impacted having to juggle 3 separate package managers, know the syntax and inputs for 17+ different dotfiles in a repo and then actively context switch between at least 3-4 languages on a daily basis. I counted it up recently and I've built products in at least 20 languages over the past two+ decades. At some point you really just can't effectively compartmentalize it all and context switch perfectly. I try by using different fonts+colors per lang etc but it still is hard. AI has been a net win for syntax recall just by giving it gentle nudges towards intent.
[+] [-] testtesttesst|3 years ago|reply
I agree but there is a certain "god of the gaps" process that's happening here.
To me what's been achieved so far is encoding text into natural (and programming) languages. These models can't reason and e.g., can't pass a Turing test. For example, if you play with GPT-3 or ChatGPT enough you can definitely get it to fail on a task a human would succeed at.
When models so far have achieved "reasoning" behavior, usually it's in conjunction with a classical algorithm -- for example AlphaZero used a model to evaluate chess positions but then ran a tree search algorithm to decide what move to play.
I think if LLMs can be combined effectively with some kind of computation process like in AlphaZero or some kind of "chains of thought" [1], then we might see them do things we'd truly call "reasoning".
Some "gaps" have been (1) "(encoding into) images is hard" (2) "(encoding into) natural language is hard" and now (3) "'reasoning' is hard" and afterwards (4) "subjective experience is hard".
Whether to be optimistic or pessimistic about AI depends on the final "gap" -- whether there is an irreplicable/noncomputable/etc. property of human consciousness other than qualia. That is, if "understanding" is hard (e.g., via some voodoo like [2]). If not humans will become obsolete, so I hope the answer is yes.
[1]: https://ai.googleblog.com/2022/05/language-models-perform-re...
[2]: https://en.m.wikipedia.org/wiki/Orchestrated_objective_reduc...
[+] [-] fexecve|3 years ago|reply
Wrong, that research only shows that when you use a GPS, you don't learn routes as quickly. That is only equal to "impacts brain function negatively" in a narrow sense, which you should specify rather than use such a broad and scary-sounding phrase!
[+] [-] kqr|3 years ago|reply
To be fair, this could be said of a surprising number of the developers I've interviewed too, and I checked -- they were hired by someone.
[+] [-] pdntspa|3 years ago|reply
Stop anthropomorphizing computers, they hate that!
[+] [-] johnfn|3 years ago|reply
I wonder how code completion and syntax highlighting have altered the brains of programmers negatively as well.
[+] [-] sbst32|3 years ago|reply
[+] [-] whateveracct|3 years ago|reply
Goddamn do I hate this industry. I work hard to sow dissent in my blue-chip, VC startup though. Guerrilla tactics. And I get to do it raking in a quarter mil a year in my underwear.
[+] [-] chubot|3 years ago|reply
First I tried it on a bunch of programming topics -- it definitely gets in the right ballpark, but then the details are frequently wrong.
I asked which languages have the F# |> operator, and it told me that OCaml, Elixir, Haskell and Scala do. 2 of them are correct, and 2 appeared wrong, so I googled it.
After Googling, then I asked it specifically if Haskell does, and it says no. I ask it if Scala does, and it apologizes for its earlier mistake.
It gave me a good although slightly shallow answer about how PyPy works, but then fell down hard for a prolog type checker, and a few other programming queries. It doesn't understand the question.
---
It is generally very mealy-mouthed and vague. It often doesn't answer the question directly, but just tells you what it happens to know in the ballpark.
I like another framing I saw on Twitter -- it's "mansplaining as a service". If you want someone to confidently explain things you didn't really ask about, and be frequently wrong, that's kinda what it does!!
---
After programming, I asked it about Black Sabbath, UFC, travel and biking, health, and got similarly vague/slightly incorrect answers.
It confabulates A LOT.
The best I can say is that Google is filled with spam now, so chatting with ChatGPT was a relatively fun and focused experience, after I took into account the fact that it is not "honest" and just trying to "please me" with closely related words.
I honestly don't get the hype, and I feel like there must have been some coordinated marketing aimed at me ...
If this chat is representative of what it can do for programming, then I think this article saying that "programmer salaries will be affected by LLM" is very weird.
It is assuming something that simply has NOT happened!
---
I mean even Geoff Hinton (who I respect) said several years ago that "we should stop training radiologists", and that plainly did not happen. Likewise with Elon and the fleet of self-driving taxis.
I'd honestly be surprised if someone wants to argue that a tech company staffed with LLMs + a few smart people is NOT like those predictions. Of course the blog post doesn't give a time frame, so it can't be falsified. But it appears to be nonsense.
(FWIW it doesn't matter to me personally, as I left the industry several years ago. Also FWIW I did work a little bit on deep learning, when the hype was in its upswing.)
I agree that a lot of it is anthropomorphism -- projecting some kind of intelligence, knowledge, or intention on what is really a big, expensive bag of data organized to respond to inputs.
Maybe there are other models that are better, but I was really, honestly not impressed with ChatGPT. Maybe people were hyping it too much for the last several weeks, and inflated my expectations.
[+] [-] Kostchei|3 years ago|reply
[+] [-] aetherson|3 years ago|reply
1 very highly compensated staff+ engineer
4 highly compensated senior engineers
4 pretty well-compensated mid-level engineers
2 decently compensated junior engineers
The future might look like:
1 yet-more-highly-compensated staff+ engineer who specializes in architecture
10 decently compensated junior engineers who specialize in error-checking LLMs and detail-intensive technical work
In the same way that paralegals aren't unskilled people, and aren't paid terribly -- but they aren't doing the same job as the partner-level lawyer who's crafting the legal strategy, and they don't have a simple clean progression to eventually getting his job.
[+] [-] imtringued|3 years ago|reply
This overestimates the time developers spend typing text and underestimates the time spent thinking about problems, verification and locating bugs/code, etc.
Just because the output of a model is text doesn't mean it is a replacement. You would need extraordinary "single shot" functionality where the LLM always outputs correct code and knows how to deal with any potential specification. The moment it fails then you wouldn't want "paralegal" developers, you would want normal, maybe inexperienced, developers.
Single shot functionality is very impressive because humans can't do it, they build things incrementally, but once your program gets large, it can only be written incrementally, which at least implies that the AI needs to keep track of unbounded context.
[+] [-] hnbad|3 years ago|reply
The problem with AI generated code is that it contains mistakes and because of the nature of how it works, those mistakes may be entirely unlike the kind of mistakes a human would make or may be completely undetectable unless you know what the code should be doing.
The best use case for AI code generation is auto-complete. Depending on the complexity of the code, that can be sufficient to do 90% of the work of writing code, but it's very different from "solving problems" which tends to be the tricky part. It also requires the AI user to already know the code they want the AI to generate before it does it.
An interesting application of AI outside of simple code generation however is what GitHub is currently experimenting with as "code brushes", i.e. "touching up" existing code, trying to fix bugs or apply refactorings. This fits in the gap of "single shot functionality" but operates on existing code rather than requiring a blank slate.
But at the end of the day I think AI will fail to kill programming just like other attempts like WYSIWYG UI builders and COBOL have and for the same reason: competence as a programmer is not about the ability to write code but the ability to understand programs and systems, both at scale and in depth. A beginner will not write bad code simply because they lack experience in using the language, they will write bad code because they lack understanding of what the code does and how it interacts with its environment (and how that environment interacts with it in turn, e.g. UX).
[+] [-] aws_ls|3 years ago|reply
And yes, very less percentage of the time, do we right chunks of fresh code, in which this could help. But that again to an extent. It can generate functions. But the glue code is still complicated. So it can definitely be a better Stack Overflow replacement, perhaps.
[+] [-] nonethewiser|3 years ago|reply
Its easier to understand and test code that you wrote than code you just read. If you ask it to write a 100k line program you're going to have to trace through everything and read it countless times over to fix any issues.
If you ask it to write a small 10 line snippet that you need to test you might as well do it yourself.
[+] [-] uwuemu|3 years ago|reply
[deleted]
[+] [-] btheshoe|3 years ago|reply
I think the real concern is for when the rate of new startups decreases, because I view startups as basically the tech industry's way for searching for ways to increase the overall size of the pie, and startups slowing down means demand for software engineers will start to stagnate.
[+] [-] karmasimida|3 years ago|reply
Definitely not the case when layoff right now is in full force. Also this doesn't necessarily indicate high salary anyway. There is a huge demand for fruit picker, but due to the nature of economy and margin of business, the salary will stay deflated thus the shortage persists. In a high interests, no free money environment, will software firms still able to demonstrate ability to generate return for investors? That would the ultimate key for whether the high salary is going to sustain or disappear.
Regardless, I think this article has a point. We might be as well observing the end of the golden era for software engineers. LLM is a long-term salary oppressor, but in short term, simple dynamics of economy would test the market sufficiently.
[+] [-] angarg12|3 years ago|reply
[1] https://blog.pragmaticengineer.com/software-engineering-sala...
[+] [-] sanderjd|3 years ago|reply
[+] [-] raincom|3 years ago|reply
[+] [-] fdsfawfds|3 years ago|reply
[deleted]
[+] [-] jesuscript|3 years ago|reply
It's kind of been one of my main critiques of companies like Palantir. You really can't just dump a company's data into the big ML tumbler and expect useful/sellable products to just pop out. Take a random use case like, I don't know, Pepsi. Maybe Pepsi can get some insights about their supply chain and customer preferences. Alright? Okay. Did the world end because that happened? Not really.
Shifting gears back to programmers - AI will be a super smart programmer that will write a bunch of your code. I worked with some super smart programmers, and they can manage to write a lot of super smart code that eventually becomes a clusterfuck too.
We'll just go from unwrapping human shit code, to unwrapping AI shit code (or even better, unwrapping a mix of human shit and AI shit). The job prospects here are even greater than you imagine.
Long as you maintain the belief in the fundamental truth, that anything in existence is capable of writing shit code, then the world will never end for programmers.
[+] [-] noobermin|3 years ago|reply
So many times, I look over things people copy and pasted piece by piece from blogs and stackoverflow, when if they thought about it for 5 minutes, they'd realize a simpler implementation would have worked and been easier to write in the end. AI, if chatgpt is any indication, is moving toward overly flowerly and long prose while telling lies, that is stuff that looks "impressive" at a glance like the large ornate class hiearchies of java and c++ in the 90s looked but are things that could be done in a shell script.
I am worried then that like stackoverflow, AI will make a generation of developers who are extremely lazy and not critically thinking, just like stackoverflow did, and leave more of a mess for us to pick up after. Of course, in the midst of this, management won't know the difference. Just like chatgpt sounding impressive as it tells falsehoods, AI-code will look impressive to fools.
[+] [-] devmunchies|3 years ago|reply
Also, it’s worth pointing out that AI is using existing data. If a new iOS API comes out, the AI will need code bases with it already implemented to be able to train with it.
So in my mind, it will hurt the junior developers doing grunt work more than general devs.
I think no code/low code products are more of a risk to hurting devs than AI.
[+] [-] RcouF1uZ4gsC|3 years ago|reply
However, let me talk about something to be excited for.
I am really excited about LLMs taking the drudgery out of programming. There are many times what I have a project in mind that I want to do, but there is a lot of boilerplate (writing configurations, or writing tests, etc) that I have to do and slows me down or for personal projects even keeps me from doing them. However, if I have an AI assistant that I can assign all the drudgery to, and I can focus on the interesting work, I feel that my creativity can really be unleashed.
I am most excited to see the results of this in games. Right now, to produce a AAA game is a lot of work and requires massive teams (if only for the art assets). However, I can see a future where once again a single programmer can write very compelling games (and even have the AI generate the art assets for the game based on the programmer's vision).
I personally am looking forward to the future.
[+] [-] larve|3 years ago|reply
This leaves me with 4h of high quality thinking and programming time instead of 1. It means I can try out 3 different approaches with full tooling instead of 1. It means I can build a whole terraform for aws and gcp and bare iron in 30 minutes.
The fact that I can build solid tools and a full slurry of unit tests and end up with 4x time in my day is just blowing my mind. The quality of my software has tremendously gone up despite using something that mostly outputs broken code.
[+] [-] rosywoozlechan|3 years ago|reply
[+] [-] testtesttesst|3 years ago|reply
For most big software projects the bottleneck is not writing code and test boilerplate that runs correctly locally. Which of course LLMs can do reasonably well. Most time is spent on design, on integrations with other internal services, scaling concerns, and reasoning about other things that these models can't really reason about yet.
I'm obvious impressed that these models can generate simple client/server websites and generate code to solve competitive programming problems. But I don't think they've "solved" software engineering; they've only solved "what you boss thinks software engineering is".
[+] [-] noobermin|3 years ago|reply
You want to increase churn? Are you insane?
I feel like if you poll computer / phone users (ie., the vast majority of the population) one of the most annoying things users hate about modern devices are mandatory updates, like the infamous meme video where someone screams at their window machine updating. Now, you want to make it easier for middle managers to push new unneeded updates so they can pad their resumes, how absurd.
[+] [-] adolph|3 years ago|reply
On the other hand, if the models can be Atkinson-like and produce -2000 lines of code [0], that is a day to which I would look forward.
0. https://www.folklore.org/StoryView.py?story=Negative_2000_Li...
[+] [-] scj|3 years ago|reply
I suspect that detecting errors that way will be a part of training non-terrible code generators. But with the added benefit of being useful to human programmers as well.
[+] [-] dyno12345|3 years ago|reply
[+] [-] sublinear|3 years ago|reply
There has been zero-progress towards AI comprehension, so you can only get out of it what you are already familiar with. The human must always do the mental heavy lifting to tease out a good result, or indeed even know if it's a good result and not just a hallucinated mess.
[+] [-] marcell|3 years ago|reply
Just an example try asking it to write Battleship in Python. It does very well. But then, try changing the rules a bit, and adding some requirements. It spits out code that looks correct if you squint, but has terrible style, and often doesn't even work. Check out the test prompt [1].
Maybe GPT-4 is 1000x better at coding, but barring that, I don't see ChatGPT being useful as a replacement for human coders.
[1] Prompt: Write a version of Battleship with modified rules for sinking a ship. In this version, a ship is sunk when the "head" and "tail" of the ship (its beginning and end parts) are hit. Hits to the center do not matter. Use object oriented programming, with classes for the Board and for Ships. There should be support for multiple ships and two board instances. Write it in Python.
[+] [-] alar44|3 years ago|reply
Have it give you a list of functions needed to write battleship.
Then have it write the classes.
Then set the rules.
Then have it write a GUI etc.
You'll have a much more successful encounter that way.
[+] [-] herpderperator|3 years ago|reply
[+] [-] dang|3 years ago|reply
[+] [-] suzzer99|3 years ago|reply
Just as an example for future writers – one line at the beginning of the piece explaining the acronym and using the ChatGPT example would go a long way.
[+] [-] larsiusprime|3 years ago|reply
[+] [-] grensley|3 years ago|reply
[+] [-] travisgriggs|3 years ago|reply
[+] [-] bitwize|3 years ago|reply
An Asimov story called "Liar!" concerns a robot which has the ability to read minds. When asked to tell what's going on in another person's mind (does so-and-so have a crush on me? is my boss planning to give me a promotion?) the robot, following its First Law directive not to allow humans to come to harm (including emotional harm), tells the querent what they would like to hear based on the robot's reading of the querent's mind, instead of the truth because the truth can hurt. When the robot is shown that its actions hurt humans anyway, it promptly shuts down and refuses to function.
These language models seem like that robot. The articles on melamine suggest to me that the AIs which generated them are producing output that their statistical models suggest the querent wants to hear, rather than output with correct answers. In light of this, it seems that using these models for assistance in coding would be like having an obsequious but shitty coworker, whose output you must carefully review and vet because even money it's not a solution to your problem, just a guess at what you want to see based on a vast statistical soup built up of thousands of programmers' efforts on millions of tasks. Which means any effort you saved up now will be paid for with extra effort later, as every programmer knows it's much easier and more fun to wrote new code than to review and debug someone else's code, especially if it's shitty, especially^2 if it's subtly shitty. It really is like Teitelman's DWIM evolved, with correspondingly magnified lossage in its failure modes!
[+] [-] bartimus|3 years ago|reply
This is just proposing waterfall + outsourcing all over again. The author doesn't seem to understand that "sketching out architectural details" is the thing that developers do.
[+] [-] jagtstronaut|3 years ago|reply
[+] [-] sjducb|3 years ago|reply