A pure NumPy implementation of Mamba

[thoronton]

Why is it so difficult to write a short description what the project does? With too many open source projects people, who are not familiar with it, have to play detective to figure out what it actually is doing. "Wait a package manager based on numphy? That doesn't make any sense. Oh they mention LLM? So it must have something to do with AI"

[Tomte]

The author did not post it to HN to confuse you. He did not post it here, at all.

Why are you entitled to have every single GitHub repo explained, tailored to your individual knowledge?

Many other people understood exactly what this is.

Maybe the submitter could add a comment on HN with an explanation, but the author owes you nothing.

[edflsafoiewq]

The doc comment at the top of the .py file is sufficiently descriptive

    """Simple, minimal implementation of Mamba in one file of Numpy adapted from (1) and inspired from (2).

    Suggest reading the following before/while reading the code:
        [1] Mamba: Linear-Time Sequence Modeling with Selective State Spaces (Albert Gu and Tri Dao)
            https://arxiv.org/abs/2312.00752
        [2] The Annotated S4 (Sasha Rush and Sidd Karamcheti)
            https://srush.github.io/annotated-s4

[grandma_tea]

That's a fair criticism of many open source projects, however this one does link to the Mamba paper at the bottom of the (short) readme.

[arthurcolle]

realistically, it's like a classification problem

at this moment, in this time, if you see Mamba, either you know or you don't

[mint2]

Okay so is Mamba also an llm? There’s too much name overloading!

I’m familiar with mamba, the conda like thing in python, but a numpy implementation of that makes no sense.

[nerdponx]

I'm not normally one to gripe about name conflicts, but I knew this was going to get confusing. You could install an implementation of the Mamba LLM using the Mamba package manager!

[exe34]

maybe we should get rid of names entirely, and call it all software. "software: a super fast and fun retro-encabulator written in rust!"

[Hugsun]

Is there a benefit to implementing it in numpy over pytorch or tf?

[blagie]

Yes.

A numpy program will work tomorrow.

ALL of the machine learning frameworks have incredible churn. I have code from two years ago which I can't make work reliably anymore -- not for lack of trying -- due to all the breaking changes and dependency issues. There are systems where each model runs in its own docker, with its own set of pinned library versions (many with security issues now). It's a complete and utter trainwreck. Don't even get me started on CUDA versions (or Intel/AMD compatibility, or older / deprecated GPUs).

For comparison, virtually all of my non-machine-learning Python code from the year 2010 all still works in 2024.

There are good reasons for this. Those breaking changes aren't just for fun; they're representative of the very rapid rate of progress in a rapidly-changing field. In contrast, Python or numpy are mature systems. Still, it makes many machine learning models insanely expensive to maintain in production environments.

If you're a machine learning researcher, it's fine, but if you have a system like an ecommerce web site or a compiler or whatever, where you'd like to be able to plug in a task-specific ML model, your downpayment is a weekend of hacking to make it work, but your ongoing rent of maintenance costs might be a few weeks each year for each model you use. I have a million places I'd love to plug in a little bit of ML. However, I'm very judicious with it, not because it's hard to do, but because it's expensive to maintain.

A pure Python + numpy implementation would mean that you can avoid all of that.

[ptspts]

Contrary to what the title says, this is note a pure Pyhon + numpy implementation: in fact it also imports einops, transformers and torch.

For me pure X means: to use this, all you have to install is X.

[qwertox]

I was struggling with the fairness of your comment because the libraries are not used as a replacement to NumPy, but to ease dealing with the data. This made me check and it turns out that:

"Yes, the comment you mentioned is fair and reflects a common perspective in the programming and data science communities regarding the usage of "pure" implementations. When someone refers to a "pure X implementation," the typical expectation is that the implementation will rely solely on the functionalities of library X, without introducing dependencies from other libraries or frameworks."

TIL.

[rsfern]

I don’t see a PyTorch import, and the transformers import is just for the tokenizer which I don’t really consider a nontrivial part of mamba

So it’s just numpy and einops, which is pretty cool. I guess you could probably rewrite all the einops stuff in pure numpy if you want to trade readable code for eliminating the einops dependency

Edit: found the torch import, but it’s just for a single torch.load to deserialize some data

[nobodywillobsrv]

Yes exactly. It's classic mis-selling.

[rowanG077]

Totally unclear what this is. I scrolled through the readme and it didn't even mention once what it does.

[sva_]

It's an LLM architecture competing with transformers: https://arxiv.org/abs/2312.00752

Proponents of it usually highlight it's inference performance, in particular linear scaling with the input tokens.

[wodenokoto]

It totally mentions what it does. It takes the sentence "I have a dream that" and extends it to: "I have a dream that I will be able to see the sunrise in the morning."

It's an LLM.

[piqufoh]

Completely - I assumed it was an implementation of https://github.com/mamba-org/mamba

I also assumed that "a pure NumPy implementation" meant that it was built purely with numpy, which it isn't smh