WingNews logo WingNews GitHub [2]
top | item 38971966

Show HN: Marimo – an open-source reactive notebook for Python

448 points| akshayka | 2 years ago |github.com

Hi HN! We’re excited to share marimo, an open-source reactive notebook for Python [1]. marimo aims to solve well-known problems with traditional notebooks [2]: marimo notebooks are reproducible (no hidden state), git-friendly (stored as Python files), executable as Python scripts, and deployable as web apps.

GitHub repo: https://github.com/marimo-team/marimo

In marimo, a notebook’s code, outputs, and program state are always consistent. Run a cell and marimo reacts by automatically running the cells that reference its declared variables. Delete a cell and marimo scrubs its variables from program memory, eliminating hidden state. Our reactive runtime is based on static analysis, so it’s performant. If you’re worried about accidentally triggering expensive computations, you can disable specific cells from auto-running.

marimo comes with UI elements like sliders, a dataframe transformer, and interactive plots that are automatically synchronized with Python [3]. Interact with an element and the cells that use it are automatically re-run with its latest value. Reactivity makes these UI elements more useful and ergonomic than Jupyter’s ipywidgets.

Every marimo notebook can be run as a script from the command line, with cells executed in a topologically sorted order, or served as an interactive web app, using the marimo CLI.

We’re a team of just two developers. We chose to develop marimo because we believe that the Python community deserves a better programming environment to do research and communicate it; experiment with code and share it; and learn computational science and teach it. We’ve seen lots of research start in Jupyter notebooks (much of my own has), only to fail to reproduce; lots of promising prototypes built that were never made real; and lots of tutorials written that failed to engage students.

marimo has been developed with the close input of scientists and engineers, and with inspiration from many tools, including Pluto.jl and streamlit. We open-sourced it recently because we feel it’s ready for broader use. Please try it out (pip install marimo && marimo tutorial intro). We’d appreciate your feedback!

[1] https://github.com/marimo-team/marimo

[2] https://docs.marimo.io/faq.html#faq-problems

[3] https://docs.marimo.io/api/inputs/index.html

106 comments

order
[+] simonw|2 years ago|reply
This is amazing. I'm a big user of both Jupyter notebooks and Observable notebooks (https://observablehq.com/) and the thing I miss most from Observable when I'm using Jupyter is the lack of cell reactivity.

You've solved that incredibly well!

I also really like that the Marimo file format is just Python. Here's an example saved file from playing around with the intro: https://gist.github.com/simonw/e6e6e4b45d1bed9fc1482412743b8...

Nice that it's Apache 2 licensed too.

Wow, I just found the GitHub Copilot feature too!

[+] mscolnick|2 years ago|reply
Myles here (other core contributor) -

We are thrilled to see you have such a strong positive reaction. It means a lot coming from you - I initially learned web development using Django and landed my first contracting gig with Django.

I drifted away from writing Python and towards Typescript - but marimo has brought me back to writing Python.

[+] LoulouMonkey|2 years ago|reply
Hi Simon, slightly unrelated question.

I'm a big fan of your work, and as I've learnt a lot from reading your blog posts over the years, I'd be curious to know a bit more about typical use cases for wanting to work with Observable notebooks.

The only reason why I'm using A JavaScript notebook tool (Starboard.gg) is to be able to access cool visualisation packages like Anychart or Highcharts.

Given the hype around Observable notebooks, I feel that I'm missing something.

What makes you decide to start something in an Observable notebook rather than in Jupyter?

Thanks!

[+] bluish29|2 years ago|reply
That's one interesting project. As someone who relies heavily on collaboration with people using Jupyter Notebook. The most annoying points about reproducing their work are the environment and the hidden state of Jupyter Notebooks.

This does to address directly the second problem. It does however by sacrificing flexibility. I might need to change a cell just to test a new thing (without affecting the other cells) but thats a trade off if you focus on reproducibility.

I know that requirements.txt is the standard solution to the other problem. But generating and using it is annoying. The command pio freeze will list all the packages in bloated way (there is better ways) but I always hoped to find a notebook system that will integrate this information natively and have a way to embed that into a notebook in a form that I can share with other people. Unfortunately I can't see support for something in any of the available solutions (at least up to my knowledge).

[+] akshayka|2 years ago|reply
Yes, the second half of reproducibility is for sure packages. A solution for reproducible environments is on our roadmap (https://marimo-team.notion.site/The-marimo-roadmap-e5460b9f2...), but we haven't quite figured it out yet.

It's a bit challenging because Python has so many different solutions for package management. If you have any ideas we'd love to hear them.

[+] Micoloth|2 years ago|reply
Wow.. Really great work, finally someone is doing it!

Since I've thought about this for a long time (I've actually even made a very simplified version last year [1]), I want to contribute a few thoughts:

- cool that you have a Vscode extension, but I was a little disappointed that it opens a full browser view instead of using the existing, good Notebook interface of Vscode. (I get you want to show the whole Frontend- But I'd love to be able to run the Reactive Kernel within the full Vscode ecosystem.. Included Github Copilot is cool, but that's not all)

- As other comments said, if you want to go for reproducibility, the part about Package Management is very important. And it's also mostly solved, with Poetry etc...

- If you want to go for easy deployment of the NB code to Production, another very cool feature would be to extract (as a script) all the code needed to produce a given cell of output! This should be very easy since you already have the DAG.. It actually even existed at some point in VSCode Python extension, then they removed it

Again, great job

[1] https://github.com/micoloth/vscode-reactive-jupyter

[+] smacke|2 years ago|reply
You're probably referring to nbgather (https://github.com/microsoft/gather), which shipped with VSCode for a while.

nbgather used static slicing to get all the code necessary to reconstruct some cell. I actually worked with Andrew Head (original nbgather author) and Shreya Shankar to implement something similar in ipyflow (but with dynamic slicing and a not-as-nice interface): https://github.com/ipyflow/ipyflow?tab=readme-ov-file#state-...

I have no doubt something like this will make its way into marimo's roadmap at some point :)

[+] paddy_m|2 years ago|reply
Very exciting! I took a quick look and I have a couple of questions.

1. Can you describe your interactive widget story? I see that you integrated altair, and there is some custom written react code around it [0] [1]. I'd be interested in porting my table widget to your platform at some point.

2. How much, if any does this depend on the jupyter ecosystem?

3. How does this interact with the jupyter ecosystem?

[0] https://github.com/marimo-team/marimo/blob/b52faf3caf9aa73f4... [1] https://github.com/marimo-team/marimo/blob/b52faf3caf9aa73f4...

[+] akshayka|2 years ago|reply
1. We don't have a public plugin API yet, but we will in the future. Our (internal) plugins are represented as custom elements: Python writes the HTML (e.g., `<marimo-vega ...>` and the frontend instantiates it. In the meantime, maybe we can help you port your table widget and make it a marimo plugin. You can reach us in our Discord (https://discord.gg/JE7nhX6mD8) or at Github.

2. marimo was built from scratch, it doesn't depend on Jupyter or IPython at all.

3. marimo doesn't interact with the Jupyter ecosystem. We have brainstormed the possibility of a compatibility layer that allows Jupyter widgets to be used as marimo plugins, but right now that's just an idea.

[+] ingenieroariel|2 years ago|reply
The list of dependencies seems very short, apart from tornado it does not seem like the other ones pull in a lot of other deps.

Congrats, this looks very useful and awesome.

  dependencies = [
    # cli
    "click>=8.0,<9",
    # python 3.8 compatibility
    "importlib_resources>=5.10.2; python_version < \"3.9\"",
    # code completion
    "jedi>=0.18.0",
    # compile markdown to html
    "markdown>=3.4,<4",
    # add features to markdown
    "pymdown-extensions>=9.0,<11",
    # syntax highlighting of code in markdown
    "pygments>=2.13,<3",
    # for reading, writing configs
    "tomlkit>= 0.12.0",
    # web server
    "tornado>=6.1,<7",
    # python <=3.9 compatibility
    "typing_extensions>=4.4.0; python_version < \"3.10\"",
    # for cell formatting; if user version is not compatible, no-op
    "black",
  ]
[+] wisty|2 years ago|reply
Cool. On a side note, I think the old Jupytext extension is hugely underrated. It lets Jupyter run a .py file (with markdown notes as comment in the file, displayed as notes in the web page).

Both of these solve the most important part of this problems in iPython - horrible git interaction, horrible programming practice to discouraging writing library files, though Jupyter fixes most of the weird non-deterministic behaviour by forcing you to rerun the script every time you load it (rather than reactive techniques). State is OK for power users but it's known to be a massive pain for people who are just learning programming, and an issue in large projects or with interaction.

With this new project having reactive updates I think it's definitely going to be great for beginners, or in gnarly projects.

I wonder if it runs on pyodide (a cPython compiled to run in the browser, with matplotlib and scipy bundled).

[+] smacke|2 years ago|reply
I'm a big fan of Marimo (and of Akshay and Myles in particular); it's great to finally see a viable competitor to Jupyter as it can only mean good things for the ecosystem of scientific tooling as a whole.
[+] nnx|2 years ago|reply
Very interesting project, a breeze of fresh air and welcome competition to Jupyter.

I guess it's still very early but the onboarding for Mario VSCode is not great at the moment, no idea how to actually start writing a Marimo notebook (no "Create: New Marimo notebook" option like Jupyter's).

Then I then tried clone the cookbook repo, and get "module not found" errors that are even less friendly than when it happens on Jupyter: have to figure out which cell the error actually comes from to even know which module is missing.

[+] SushiHippie|2 years ago|reply
Looks cool!

Have you looked into WASM? Something like a jupyterlite [0] alternative for marimo?

And are there plans to integrate linting and formatting with ruff? [1]

[0] https://jupyterlite.readthedocs.io/en/stable/

[1] https://github.com/astral-sh/ruff (ruff format is almost 100% compatible with black formatting)

[+] yowlingcat|2 years ago|reply
This is very cool. I think I need to play around with this a bit more to wrap my head around the reactivity element, but the basic shift of ipynb to standard Python would be such a huge workflow improvement for my team. We use jupyter notebooks when prototyping and trying to code review unwieldy python-in-JSON is miserable. Great to see an alternative that's worked its way around that.
[+] rossjudson|2 years ago|reply
Arrggghh. Now I have to learn Python, which I've been actively resisting and making jokes about for years.
[+] hedgehog|2 years ago|reply
This looks quite nice and it might compose well with a cache library like the one posted on HN recently (XetCache, https://news.ycombinator.com/item?id=38696631).
[+] noahlt|2 years ago|reply
Yeah, having worked on alternative notebooks before, one of the big implicit features of Jupyter notebooks is that long-running cells (downloading data, training models) don't get spuriously re-run.

Having an excellent cache might reduce spurious re-running of cells, but I wonder if it would be sufficient.

[+] oakhaven|2 years ago|reply
At first I thought that in effect, this project only removes a couple of Ctrl+Enter keystrokes in Jupyter-notebook workflow. But after trying out the intro I think it looks good, I really like the simple convert to a webapp.

I wonder if the state/data in the generated app are stored server-side of sent to browser.

I went through the slider example in the intro and I noticed that when I change the icon, the slider position goes back to 1. I tried to fix it so that the slider-selected value is preserved over icon changes, but didn't manage, it doesn't seem straightforward.

[+] Onawa|2 years ago|reply
Aren't many of the issues with Jupyter being mentioned in this thread solved by Quarto? I have been advocating for it's use more at work, and NIH has even started offering classes on it through the NIH library.
[+] TtjM|2 years ago|reply
Exactly my thoughts too, especially regarding reproducibility issues Quarto has been great in the past for projects at my workplace.

I have yet to try Marimo but synchronised code cells are what seems to set it apart. Quarto + jupyter-cache [1] was the closest I have managed to get to that experience but that approach has its constraints.

[1]: https://github.com/executablebooks/jupyter-cache

[+] bsdz|2 years ago|reply
This is a great idea. I'd been planning to create something similar where cells are topologically ordered based on their dependency structure; although I was thinking perhaps to integrate with Jupyter more, eg use their existing kernel web sockets infrastructure. In my mind, one would be able to zoom out and see a graph view where hovering over a node would show its corresponding cell with content / output. Each node might be coloured according to execution status. That said, I'm not a UI expert and I never got around to it. So thanks for your efforts, I'll definitely give it a spin.
[+] akshayka|2 years ago|reply
That sounds really cool! marimo has a dependency graph viewer built-in, but we could definitely improve it. Coloring nodes by execution status, and annotating cells with their variable defs/refs, would be great quality-of-life improvements.
[+] robsh|2 years ago|reply
It would be amazing if it could be deployed with pyodide/wasm as an alternative to a Python web server. Truly a standalone interactive notebook, hosted with plain html.
[+] bitsrat|2 years ago|reply
I read in a comment that Marimo is an alternate to Jupyter. Does it not depend on Jupyter Server or ipykernel ? Is it a replacement for Jupyter lab ?

I am thinking of Jupyter as all the components in this diagram - https://docs.jupyter.org/en/latest/projects/architecture/con...

Sorry did not get to look into the codebase yet

[+] mscolnick|2 years ago|reply
Correct, it does not depend on Jupyter. It’s built from the ground up with different principles in mind