AI can write code like humans, bugs and all

[ctoth]

Lots of people grumping about Copilot, has anyone actually used it? For a personal project writing a transpiler from an old language (LambdaMOO) to JS, I've actually found it quite helpful. I can write code like:

convertList(node: MooASTNode): List {

And then it just spits out:

    const entries: ASTNode[] = node.children.map((child) => {
      return this.convertNode(child);
    });
    return new List(entries, this.sourceLocation(node));
  }

This isn't exactly difficult to write or reason about, but it did save me quite a bit of time to just dash out the code that converts the parse tree to an intermediate tree in an hour or so and then just quickly look over and make a couple corrections.

It also helped making the ASTNodes, for instance:

export class If extends ASTNode {

resulted in:

  constructor(
    public condition: Compare,
    public then: ASTNode,
    public elseDo?: ASTNode,
    public override loc: SourceLocation | null = null
  ) {
    super();
    condition.parent = this;
    then.parent = this;
    if (elseDo) {
      elseDo.parent = this;
    }
  }

  @logCall
  toEstree() {
    return builders.ifStatement(
      this.condition.toEstree(),
      this.then.toEstree(),
      this.elseDo?.toEstree()
    );
  }

}

Again, this code is not gonna win any prizes, but it sure did save me a good chunk of time. Why so much hate for the tool?

[Jensson]

It is usually more work to verify that code is correct than to write correct code in the first place.

Edit: At least for me it isn't obvious at all that those snippets are correct.

[klyrs]

> Why so much hate for the tool?

Whose code did it copy and how is that code licensed?

[minimaxir]

I stopped using GitHub Copilot. Not because of the accuracy of its predictions (or potential lack thereof), but because the cognitive overhead / distraction of getting inline suggestions made using Copilot not strictly a net positive in productivity.

And this is before GitHub will start charging for it.

[fallat]

And you instantly realize the code which is being returned is almost always directly from a human being who wrote it anyway.

[gvb]

AI writes code like a bad intern: pasting in "likely" blocks of code with no understanding of the intent. The difference is the AI pastes in "likely" blocks from github examples where bad interns paste from SO.

[Skyy93]

Garbage in - garbage out. If training data are error-prone the AI learns the same errors. I think the most devs do not have to fear unemployment.

[jameshiew]

I haven't used Github Copilot but from what I understand it generates a completion from the text already in your file. Since OpenAI Codex can "understand" natural language instructions, I've found putting code into a prompt with extra context in natural language on what needs to be achieved can give decent results. e.g to generate a docstring, you might use a Markdown-ish prompt like

  # Writing a good docstring
  
  This is an example of writing a really good docstring that follows a best practice for the given language. Attention is paid to detailing things like 
  * parameter and return types (if applicable)
  * any errors that might be raised or returned, depending on the language
  
  I received the following code:
  
  ```{{{language}}}
  {{{snippet}}}
  ```
  
  The code with a really good docstring added is below:
  
  ```{{{language}}}
  

If you wanted a docstring in a particular format, you could add some specific examples as context to your prompt to get even better results.

[akomtu]

Another snake oil. Coding needs abstract thinking that only smartest of humans can do, while the AI crowd still can't replicate ant-level intelligence.

What would be an immense multiplier of software eng productivity is an intelligent auto-fixer tool: a compiler gives you an error, you know how to fix it, but it's a tedious work that wastes most of your time. Think of fixing build deps, rewriting method signatures to match the parent class, properly adding a library to your project. You'd write "include ssl.h; encrypt(message)" and the tool would add all the plumbing around it, in line with project guidelines.

[spicybright]

Why does everyone assume developers won't read and edit the code generated to be more correct?

Analyzing, debugging, and fixing code is what we do all day.

It reminds me of people strongly against autocomplete for variable/function names.

[tyingq]

Presumably, skilled developers won't be the majority working in roles where copy pasting from the AI tool is a common pattern. I like PHP, for example, but a fair amount of the criticism of existing PHP codebases is because the ecosystem attracts lower-skilled folks.

What you're mentioning will happen, some developers will use the tool the right way. I just don't think that will be the prevailing pattern.

[jwommack]

We’ve been able to write buggy automated software for a very long time. That’s not interesting. It’s also not that interesting that stealing/replicating code replicates bugs.

When the AI/ML programs can actually create lines of code without references and do it with quality that’s when there’s a real story that isn’t playing tricks on tech authors for stock gains.

Tricks like this are why everyone always thinks big innovation is 5 years away when no one is really working on the things that’s make it 5 years away.

[johnmorris100]

It is clear that technical progress is inevitable. I can show you an example of a site where almost all articles are written using AI (GPT-2, GPT-3). https://www.vproexpert.com/what-can-ai-do-today/ Is it badly written?

[codeenlightener]

AI can also read code like a human: https://denigma.app

[Jensson]

First code snippet I pasted in it got a conditional reversed so doesn't seem to be very reliable.

[el_oni]

Well yeah it was trained on humans

[Iv]

Just like the first Go playing programs. Just you wait until it is trained by "self-play" against a compiler.

[endisneigh]

hot take: rather than AI, software development would be better served by just going into LTS maintenance mode. For each domain (web development, android, etc, css) pick a tool and everyone agrees that will be "the" tool for a decade or two.

this will reduce the amount of "possibilities" w.r.t. code by orders of magnitude and therefore make it easier to both read, write, and development tools to automate code.

of course this will necessarily reduce innovation, but at the expense of higher quality code, easier to maintain code and skills that are more transferrable.

[lakis]

> everyone agrees that will be "the" tool for a decade or two Sorry, I found this so funny. Immediately before that, everybody agrees that we will develop in the one and only true language. All other languages will be banished ;-) ;-)

[falcor84]

A nice idea, but it's not a pareto equilibrium though; particularly not for developers.

[pama]

Is there anyone here with practical first hand experience that can comment about their impressions so far?

[jameshiew]

I've been using OpenAI Codex directly (so not Github Copilot) for some coding tasks and it's been useful enough that I am using it daily at this point, though mostly as like a sort of personal Stack Overflow, especially when working with unfamiliar APIs. Prompting it with a function signature/docstring of what you want to achieve can give a helpful and often functional example that uses that API, even if the code is not perfect.

It's also decent at doing tasks that would be easy for a beginner programmer but perhaps tedious. Like you can give it a function for generating random colors and a task like "expand the variable names in this function to be better", and it will change variable names like `r`, `g` and `b` to `red`, `green` and `blue`. Hardly amazing, and the latency of the API means it's not actually that useful in practice (yet), but with the right prompt it can do some impressive things, and I expect it to get much better in the near future. Pulling magic strings out of functions into constants and adding typehints to Python functions are other simple tasks I've found it OK for.

[fridif]

With enough monkeys and typewriters, we don't need AI to go where we need to go.

[hypertele-Xii]

Some of those monkeys will inevitably write AI though.

[d13]

AI is the monkeys

[jupp0r]

This is the future of Software Engineering: fixing AI's bugs.

[josefx]

The future of AI will not be skynet, but priests of the adeptus mechanicus praying to the machine spirit and performing the rites in the hope that it will perform correctly.

[tacotacotaco]

I expect programmers' jobs will only be to write tests. If a bug is found another test is written and the AI will generate new code until the new test passes.

[vsareto]

At least the AI won’t be tempted to be clever with its code unless we tell it to

[TheDudeMan]

Zero surprises here.