Smoltcp: A small TCP/IP stack in Rust

[littlestymaar]

This is a really good example of the kind of safety benefit you get when using Rust: this is a fairly low-level piece of software (not a kernel, but I'd say it's lower-level than at least 80% of what's usually written) and yet, only 76 lines of codes are located inside unsafe blocks for the whole code-base. To be safe from memory vulnerabilities, all you need is audit theses 76 lines. It can still be a tough job, and bugs may slip past the review, but it's a huge improvement over having to audit thousands of lines.

[makomk]

That's not quite right. To be safe from memory vulnerabilities, you need to understand how those 76 lines interact with the entire rest of the codebase. How hard this is depends exactly on what interfaces they expose and to where. If I temember correctly this isn't just a theoretical issue, there was at least one memory safety issue in the Rust standard library that could only be spotted by considering the unsafe code, the safe code in that library, and how other safe code could make use of it at the same time.

[nsajko]

> safety benefit

Has "safety" become a buzzword? What we really strive for is correctness. Hypothetically, Rust could still be detrimental to software correctness compared to a more usual language, if some of its traits as a language (eg., complexity) encourage bugs. Sometimes it seems like Rust afficionados think that buffer overflows etc. are the main kind of bugs.

And if you need absolute correctness (when human life depends on the software) you would probably use Ada and Spark or something else that enables you to actually prove correctness.

[MuffinFlavored]

Is it not possible to write a 100% safe TCP/IP stack in Rust? Why is there a need for 76 lines of unsafe code?

[Zapsofar]

I'm not convinced that "memory vulnerabilities" are more than a small part of correctness. Nothing says that you followed the details of a complex spec properly, for example. Or that memory use is bounded. Or that allocated memory isn't spread over disparate cache lines, with different results each run. Or that the architecture isn't open to DDoS opportunities.

It's "safe" at the lowest level. No information after that.

[nickpsecurity]

Alternatively, you can use a separate tool(s) to verify those 76 lines are correct or at least safely use the unsafe calls.

[k__]

Is the amount of code in unsafe blocks really a good metric for safety?

[haberman]

I have a distant memory of Alan Cox saying something back in the day along the lines of "there are so many things left undocumented in the TCP/IP RFCs that if you just implement it straight from the spec your stack won't work at all."

But I can't find any reference to this now. Does anyone else remember this?

Is this true? Can you interoperate with the Internet by just implementing the spec?

[jsnell]

No, it's not true. You can implement something that can load a webpage from 99.9% of the world's web servers from a couple of RFCs in a day. But this ease of implementation means that for basically any issue where the specs give any degree of freedom, somebody will have deviated from the norm. (I've implemented TCP from scratch three times, one of those implementations basically ran the traffic of entire countries.)

The real problem is that every step toward 100% gets progressively harder to find and debug. See e.g. [0] for a middle box that would mangle connections if it received two identical SYN packets, or [1] for a way in which almost all servers anyone currently runs are accidentally resilient to a certain kind of connection-killing packet corruption, but S3 isn't.

[0] https://www.snellman.net/blog/archive/2014-11-11-tcp-is-hard... [1] https://www.snellman.net/blog/archive/2017-07-20-s3-mystery/

[nabla9]

TCP/IP is relatively stable ecology where stacks in routers, applications etc. have evolved to survive with each other and intentionally malicious actors by developing coping mechanisms. Introducing a clean implementation of TCP/IP stack is like introducing a new species. It has no immune system against others and others don't know its 'signature behavior' either.

You can probably get interoperability at least part of the time. You may have deteriorated throughput and more broken connections with some stack and not with others. But you have introduced a new species. If your brand new stack has a tiny bug or new kind of misconfiguration and you start spreading it fast, the hell can break loose and you may ruin the day of many people before they find you and cut you off.

[whitequark_]

I designed smoltcp (and wrote most of the code currently in it). The original TCP/IP RFC (RFC 793) contains several ambiguous requirements, and as a result they do not specify a well-defined system. There are also some outright incorrect statements. There are a few follow up RFCs (e.g. RFC 1122) that clarify these issues, and there are more RFCs (e.g. RFC 7414) that describe the TCP features that you should avoid using.

By using this collection of TCP/IP RFCs that grew over the years, it is indeed possible to implement a stack from first principles and have it interoperate with other existing stacks without much trouble. (At least so long as you don't put the same bugs in your test suite as you do in your stack... which you will.)

However, being able to transmit some bytes reliably, and having a high-performance stack that works well in real world conditions are different. You might be able to do the former from RFCs, but the latter absolutely requires a nontrivial amount of tribal knowledge that you have to collect crumb by crumb, and often quite painfully, too.

Smoltcp is somewhere halfway between. It's pretty reliable, but I am sure there is much to be improved in its operation in adverse conditions, with obscure peers, and so on.

[maltalex]

> Is this true? Can you interoperate with the Internet by just implementing the spec?

I've never implemented the entire stack. Few people have. But from the protocols I've implemented, that's only a slight exaggeration.

As soon as a popular implementation has a bug or decides not to behave according to spec, everyone has to adapt. You can typically use the spec to get 95% of the way to full interoperability.

[toast0]

I don't think it's quite true of TCP/IP that you wouldn't work at all, although you do have to be careful, because there are a lot of RFCs, and it's not always clear which ones are important.

Also, one of the RFCs has wrong functions for calculating or adjusting checksums. I think there's also some convention on tcp option ordering that may be important but not well documented.

Either way, I would keep a couple other implementations close -- if not to peek at their code ocassionaly, at least to inspect their output.

[adwn]

> Its design anti-goals include complicated compile-time computations, such as macro or type tricks, even at cost of performance degradation.

Why? That sounds more like an ideological decision than a pragmatic, engineering-driven one. Especially for a TCP/IP stack, where performance is typically a major concern, be it in a desktop, server, or embedded environment.

[whitequark_]

At the time when I started working on smoltcp, there were a few Rust libraries for working with TCP/IP on the wire level, and they heavily used metaprogramming. Unfortunately, the task of implementing a generic binary protocol serializer/deserializer that can handle TCP/IP is not small, and as far as I could tell, it overtook implementing anything beyond that.

So I made the decision to do the simplest possible thing: write the packet serializers/deserializers entirely by hand. It took very little time and adding any new features was easy and predictable. I believe it was the right decision as it allowed me to focus on the hard parts of a TCP/IP stack.

[toast0]

Most of the performance of TCP stacks has more to do with the order that comparisons are made for incoming packets (sometimes called fast path -- check for and handle normal packets first), locking of data structures, and congestion strategies (including retransmit behavior, SACK, etc). Macros or typing is less likely to make that faster.

For a new stack, though, correctness and readability are more important than performance.

[q3k]

It can be an engineering decision: write readable, easy to understand and reason about code.

[teddyh]

Somewhat relatedly, there exists a Python-like language for when you only have a few kb of memory, and can’t even run MicroPython. It’s called “Snek”:

https://keithp.com/snek/

[blinkingled]

You might not care for low level TCP/IP details but this project will make for a great learning experience for anyone wanting to dig deeper into Rust and network programming.

It works with tun/tap interfaces and there's a tcpdump clone in the example using raw sockets that works on any regular Linux network interfaces.

Pretty cool!

[Klasiaster]

Here are memory-safe network services on Linux with smoltcp and an optional switch for running multiple userspace network stacks:

https://github.com/ANLAB-KAIST/usnet_sockets → Socket library for Rust using smoltcp as userspace network stack (provides types compatible with the standard library, tokio is unpublished WIP)

https://github.com/ANLAB-KAIST/usnetd → Memory-safe L4 Switch for Userspace Network Stacks (firewalls the kernel network stack, see Ideas section for alternatives, e.g., transparently piping the kernel network packets through smoltcp)

[pencillr]

An educational question: If this is a tcp/ip implementation where is it run? On a router? Or on a network interface controller? Both?

[q3k]

> It is used in embedded systems such as the ARTIQ core device and ionpak. [1]

and

> ARTIQ (Advanced Real-Time Infrastructure for Quantum physics) is a leading-edge control system for quantum information experiments. [2]

[1] - https://m-labs.hk/smoltcp.html

[2] - https://m-labs.hk/artiq/index.html

[progval]

The org developping smoltcp makes embedded systems, so the point of smoltcp is probably to use it there.

smoltcp is also used by Redox, an OS written from scratch in Rust.

[chaosite]

"smoltcp is a standalone, event-driven TCP/IP stack that is designed for bare-metal, real-time systems."

My understanding based on that description is that it is meant for applications that run directly on the hardware, without an OS in the middle. I'm thinking embedded applications.

So I'm thinking that this is meant for IoT-style appliances and the like. Maybe I'm wrong :)

[abhishekjha]

I think it would be run on a network interface. Isn't this or an equivalent implementation that comes packaged with every OS so that you can connect to a network?

I may be wrong here and others are more than welcome to correct me.

EDIT: Added "or an equivalent"

[nn3]

>Congestion control is not implemented.

Wit that, don't bother with it so far. That's a toy. It would be dangerous on any real network. If you doubt it please read up on "congestion collapse"

[garmaine]

Not sure why you are being down-voted. Congestion control is pretty central to how TCP/IP works. Deploying without it will basically DoS your network.

[kahlonel]

Any idea what would it take to write this without using _any_ C library function?

[whitequark_]

It doesn't use any C library functions by itself. (In fact in all of its applications at M-Labs, there is no C code running on the same chip.)

The only uses of libc are in the TUN/TAP driver, which is necessary if you want to bind it to a virtual OS interface.

[gpm]

I haven't read the code, but probably very little, since libc has rust implementations that you could plug in in it's place.

[gazarullz]

Can anyone point out any resources to acomplish this in the JVM space?

[IAmLiterallyAB]

A zero heap allocation _anything_ in the JVM? Probably impossible

[gazarullz]

Whoever downvoted my question can explain why? or it was just for fun?

[fluffything]

Lots of warnings on `cargo build`, undefined behavior on a couple of places (e.g. crating &mut uninitialized), raw calls to libc as opposed to using a safe wrapper over it like nix, ...

[eeZah7Ux]

What a poor choice for naming. Please don't use stupid internet memes for naming, it makes all of us look silly in the eyes of non-techies.

[WAHa_06x36]

Are you expecting non-techies to be paying attention to an embedded TCP/IP stack, and also worrying that they think you are having too much fun?

[bananocurrency]

get over it https://z.cash/technology/jubjub

[OskarS]

Naming your product after dumb internet memes is a great way to make me not want to use them.

Don’t get me wrong, from first look it seems like a perfectly fine product, but the name is atrocious.

[ignaloidas]

This is dumb. Many products have dumb names, we just got accustomed to them, i.e. git. If you are hating only on this specific meme - I present you SmolV - the leading(and only?) SpirV compressor. If you aren't going to use good things because of naming, then you deserve to only use bad things.

[hajhatten]

Comments like these make me not want to read them.

Don’t get me wrong, from first look it seems like a perfectly fine comment, but the attitude is atrocious.

[zaptheimpaler]

It's not a product, and if you don't want to use it - don't :)

[noselasd]

For the uninitiated, which meme is this ?