My fast zero-allocation webserver using OxCaml

[hutao]

Over the past few years, OCaml has seen an abundance of new language features and standard library additions. OCaml 4.08 added let-binding operators (syntactic sugar for continuation-passing style and monadic programming), OCaml 5 implemented a multicore runtime and effect handlers, OCaml 5.3 finally added dynamic arrays to the standard library, and OCaml 5.4 added labeled tuples.

With unboxed types, I believe OCaml would achieve similar granularity over memory allocations as C#: garbage-collected, but supporting "structs" which are allocated on the stack (or inline the same heap allocation when part of a reference type). I think there is an unexplored space for "soft" systems programming languages that retain a garbage collector by default, while also allowing the programmer to tightly control memory allocations in performance-critical code.

If OCaml hits this sweet spot in abstraction, what domains would adopt it? Could OCaml potentially compete with C#, or Swift?

[pjmlp]

OCaml already has that spot for quite some time, the problem is more cultural than technical.

Back in my day we would consider writing compiler toolchains systems programming as well.

"Unix system programming in OCaml", originally published in 1991, latest update 2014.

https://ocaml.github.io/ocamlunix/

[int3trap]

> In the steady state, a webserver would have almost no garbage collector activity

I recently wrote my own zero allocation HTTP server and while the above statement is possible to achieve, at some point you need to make a decision on how you handle pipelined requests that aren't resolved synchronously. Depending on your appetite for memory consumption per connection, this often leads to allocations in the general case, though custom memory pools can alleviate some of the burden.

I didn't see anything in the article about that case specifically, which would of been interesting to hear given it's one of the challenges I've faced.

[avsm]

Good point; I've decided to simply not support HTTP/1.1 pipelines, and to have a connection pooling layer for HTTP/2 instead that takes care of this.

In OxCaml, it has support for the effect system that we added in OCaml 5.0 onwards, which allows for a fiber to suspend itself and be restarted via a one-shot continuation. So it's possible to have a pipelined connection stash away a continuation for a response calculation and be woken up later on when it's ready.

All continuations have to be either discarded explicitly or resumed exactly once; this can lead to memory leaks in OCaml 5, but OxCaml has an emerging lifetime system that guarantees this is safe: https://oxcaml.org/documentation/parallelism/01-intro/ or https://gavinleroy.com/oxcaml-tutorial-icfp25/ for a taste of that. Beware though; it's cutting edge stuff and the interfaces are still emerging, but it's great fun if you don't mind some pretty hardcore ML typing ;-) When it all settles down it should be very ergonomic to use, but right now you do get some interesting type errors.

[boltzmann-brain]

it's a massive crime that decades into FP, we still don't have a type system that can infer or constrain the amount of copies and allocations a piece of code has. software would be massively better if it did - unnecessary copies and space leaks are some of the most performance-regressing bugs out there and there simply isn't a natural way of unearthing those.

[avsm]

We do now though, with OxCaml! The local stack allocation mode puts in quite a strong constraint on the shape of the allocations that are possible.

On my TODO list next is to hook up the various O(x)Caml memory profiling tools: we have statmemprof which does statistical sampling, and then the runtime events buffer, and (hopefully) stack activity in OxCaml's case from the compiler.

This provides a pretty good automation loop for a performance optimising coding agent: it can choose between heap vs local, or copy vs reference, or fixed layout (for SIMD) vs fragmentation (for multicore NUMA) depending on the tasks at hand.

Some references:

- Statmemprof in OCaml : https://tarides.com/blog/2025-03-06-feature-parity-series-st...

- "The saga of multicore OCaml" by Ron Minsky about how Jane Street viewed performance optimisation from the launch of OCaml 5.0 to where they are today with OxCaml https://www.youtube.com/watch?v=XGGSPpk1IB0

[zozbot234]

> infer or constrain the amount of copies and allocations a piece of code has

That's exactly what substructural logic/type systems allows you to do. Affine and linear types are one example of substructural type systems, but you can also go further in limiting moves, exchanges/swaps etc. which helps model scenarios where allocation and deallocation must be made explicit.

[AlotOfReading]

Allocations and copies are one of the things substructural typing formalizes. It's how E.g. Rust essentially eliminates implicit copies.

[aseipp]

There are ongoing projects like Granule[1] that are exploring more precise resource usage to be captured in types, in this case by way of graded modalities. There is of course still a tension in exposing too much of the implementation details via intensional types. But it's definitely an ongoing avenue of research.

[1] http://granule-project.github.io/granule.html

[cryptonector]

https://ghc.gitlab.haskell.org/ghc/doc/users_guide/exts/line...

[3836293648]

There is discussion about this in the Rust world, though no attempts at implementation (and yet further from stabilisation)

[zokier]

Wouldn't such analysis in the general case run afoul of Rices theorem?

[smartmic]

From the article:

> I am also deeply sick and tired of maintaining large Python scripts recently, and crave the modularity and type safety of OCaml.

I can totally relate. Switching from Python to a purely functional language can feel like a rebirth.

[IshKebab]

OCaml isn't pure.

[voidUpdate]

While python isn't type safe, you can use Pylance or similar in combination with type hinting to get your editor to yell at you if you do something bad type-wise. I've had it turned on for a while in a large web project and it's been very helpful, and almost feels type-safe again

[ttoinou]

Does it look like functional programming anymore ?

[cess11]

Looks pretty ML:ish to me, even in a segment like this:

   let parse_int64 (local_ buf) (sp : span) : int64# =
     let mutable acc : int64# = #0L in
     let mutable i = 0 in
     let mutable valid = true in
     while valid && i < I16.to_int sp.#len do
       let c = Bytes.get buf (I16.to_int sp.#off + i) in
       match c with
       | '0' .. '9' ->
         acc <- I64.add (I64.mul acc #10L) (I64.of_int (Char.code c - 48));
         i <- i + 1
       | _ -> valid <- false
     done;
     acc

[boltzmann-brain]

Yes - high-performance Haskell code looks similar. There isn't much to be said there - it's a little less clean-looking because FP optimizes for the most useful scenario and trying to do highly advanced stuff like that will be more verbose. This is in contrast to OOP where everything is verbose, and sometimes high-perf stuff that falls into the shape of globals + mutation + goto looks very succinct.

[seanhunter]

Looks like 100% idiomatic normal OCaml to me.

[le-mark]

I think there are more succinct snippets in here and some this more verbose exposition is for pedagogical purposes. I am not a fan of ocaml because tacking on the object syntax made SML more verbose (ugly imo). Looks like 0xcaml continued trend.

[pjmlp]

Depends on what one means as FP.

When I learnt FP, the choice was between Lisp, Scheme, Miranda, Caml Light and Standard ML, depending on the assignment.

Nowadays some folks consider FP === Haskell.