I let LLMs write an Elixir NIF in C; it mostly worked

[flax]

"it mostly worked" is just a more nuanced way of saying "it didn't work". Apparently the author did eventually get something working, but it is false to say that the LLMs produced a working project.

[hombre_fatal]

Well, yeah. It’s a more nuanced way of saying that because “it didn’t work” isn’t very useful nor descriptive.

What if it wrote all of the boilerplate and just let you focus on the important bit that deserves your scrutiny?

You could say I failed every single project I ever built because it took many iterations to get to the final deliverable stage with lots of errors along the way. So more nuance would be needed.

But when it comes to LLMs suddenly we get all gleeful about how negatively we can frame the experience. Even among HN tech scholars.

[jgalt212]

I dunno. Depending on the writer and their particularly axe to grind the definition can vary widely. I would like it to mean, "any fixes I needed to make were minimal and not time intensive."

[overbring_labs]

What is your definition of "a working project"? It does what it says on the tin (actually it probably does more, because splint throws some warnings...)

[faangguyindia]

I'd say 98% people who say they are programmer in market can't produce good as good as LLM can.

[vrighter]

the mostly-er it works, the harder it is to find out which tiny part does not work.

[brokencode]

Ok. But what are you even reacting to? Who is saying that it produced a working product?

As you said, the very title of the article acknowledged that it didn’t produce a working product.

This is just outrage for the sake of outrage.

[drumnerd]

I would never ever let an LLM anywhere near C code. If you need help from LLM to write a NIF that performs basic C calls to the OS, you probably can’t check if it’s safe. I mean, it needs at least to pass valgrind.

[simonw]

You can use something like Claude Code or Codex CLI and tell it to run valgrind as part of iterating on the code.

[overbring_labs]

I mean, you aren't wrong. I'm looking into converting it into Rust with Rustler right now.

[true_religion]

Security is a spectrum. If you totally control the input going into a program, it can be safe even if you didn't test it for memory leaks. The only errors that occur will be truly erroneous, not malicious and for many solutions that's fine.

At the very least, it's fine for personal projects which is something I'm getting into more and more: remembering that computers were meant to create convenience, so writing small programs to make life easier.

[simonw]

For anyone wondering, the article clarifies that "A NIF is a function that is implemented in C instead of Erlang".

I had a bunch of fun getting ChatGPT Code Interpreter to write (and compile and test) C extensions for SQLite last year: https://simonwillison.net/2024/Mar/23/building-c-extensions-...

[victorbjorklund]

Not only C. Can be done in any compiled language (C, Rust, Zig, etc). Not sure if can be done with GC language.

[cultofmetatron]

built my startup in elixir. love it but nifs are one of the few ways you can crash the VM. I don't trust myseld to write a nif in production. no way I'd do it with AI in c. Thank god theres projects like rustler which can catch panics before it crashes the main VM.

[worthless-trash]

You could always use ports, or the external port drivers.

[lawik]

I've done this. The NIF worked as in that it ran and was a correct enough NIF. It did not work in terms of solving what I needed it to do. Iteration was a bit painful because it was tangled with a nasty library that needed to be cross-compiled. So when I made a change it seg faulted and I bailed.

I essentially ran out of patience and tried another approach. It involved an LLM running C code so I could check the library output compared to my implementation to make sure it was byte-for-byte.

The C will never ship. I don't have practice writing C so I am very inefficient at it. I read it okay. LLMs are pretty decent help for this type of scrap code.

[_ea1k]

I once wrote a little generalized yaml templating processor in Python by using an LLM for assistance. It was working pretty well and passing a lot of the tests that I was throwing at it!

Then I noticed that some of the tests that failed were failing in really odd ways. Upon closer inspection, the generated processor had made lots of crazy assumptions about what it should be doing based upon specific values in yaml keys that were obviously unrelated to instructions.

Yeah, I agree with the author. This stuff can be incredibly useful, but it definitely isn't anything like an AGI in its current form.

[bcardarella]

I tried to do this a few weeks ago, I tried to build a NIF around an existing C lib. I was using Claude Opus and burned over $300 (I didn't have Pro) on tokens with no usable results.

[cpursley]

Get Pro, 4 is quite good at Elixir now but you have to stay on it. 3.5 was not, so I imagine next version of Claude will be able to handle the more esoteric things like NIFs, etc.

[weatherlight]

Why C instead of Rust or Zig? Rustler and Zigler exist. I feel like a Vibecoded NIF in C is the absolute last thing I would want to expose the BEAM to.

[overbring_labs]

Given the amount of issues the code had when I ran splint on the C file, I agree. The question was for me whether I can get something working to get over the "speed bump" of lacking such a function for the API client I'm writing.

I'm now re-vibe-coding it into Rust with the same process, but also using Grok 4 to get better results. It now builds and passes the tests on Elixir 1.14 to 1.18 on macOS and Ubuntu, but I'm still trying to get Grok 3 and 4 to fix the Windows-specific parts of the Rust code.

[qualeed]

Why does every post that mentions something other than Rust or Zig get a comment saying "Why not Rust or Zig"?

[leansensei]

Why not C? It made no difference, we're talking about a few function calls.

[ch4s3]

The author was trying to learn about https://github.com/elixir-sqlite/exqlite which uses C.

[overbring_labs]

Well, it's in Rust now, thanks to your and someone else's comments! https://github.com/waseigo/disk_space

[SweetSoftPillow]

It's interesting why the author used weaker models (like Grok 3 when 4 is available, and Gemini 2.5 Flash when Pro is), since the difference in coding quality between these models is significant, and results could be much better.

[overbring_labs]

It's just that Grok 3 is faster than 4, so I've set it by default. But point taken, I'll try out the newer ones now that I'm converting it to Rust.

[overbring_labs]

Holy moly, you weren't kidding. Grok 4 is so much better. Thanks!

[wordofx]

This was built copy pasting results from chats? Not using an ide or cli like Claude Code or Amp? Why such a manual process. This isn’t 2023…

[overbring_labs]

Because what difference would it make, given the bad quality of code?

Also, is Claude Code free to use?

The manual process has the upside that you get to see how the sausage is (badly) made. Otherwise, just YOLO it and put your trust in GenAI completely.

Furthermore, if there is the interim step of pushing to GitHub to trigger the build & test workflow and see if it works on something other than Linux, is the choice of Vibe-Coding IDE really the limiting factor in the entire process?

[juped]

So all this arose because you didn't read the docs and note that get_disk_info/1 immediately fetches the data when called? The every-30-minutes-by-default checks are for generating "disk usage is high" event conditions.

[leansensei]

Thanks, that was not clear to me from skimming the docs.

However, this NIF also returns more fields than the disksup function.