The updated syntax looks brilliant — can't wait to try it out by upgrading some of my older notebooks! I was worried about the future of Observable since the Canvas announcement — it felt like notebooks were being slowly phased out, but I'm glad to be proven wrong.
Any timeline on bringing the new format to the Web editor? For me, the biggest draw of Observable is being able to instantly start exploring and visualizing some data without ANY setup (creating some local files, installing dependencies, launching a dev server etc.) — just click "New notebook" and off you go, frictionless. I hope this remains a priority.
Also worth mentioning the data viz community aspect, which I used to enjoy a ton — it's a bummer that things like being able to follow certain users and see their new notebooks in a feed were phased out, and that trending notebooks are now hidden under the Help menu instead of being featured on the user's homepage. This might be one of the reasons I stopped coming back to the platform as often, although I realize building a community is very difficult and hard to monetize — just thought I'd share.
Thank you. I don’t want to commit to a public timeline on the web editor yet, but rest assured it is a top priority, and we also value the convenience of a web editor, instant collaboration, and community!
I'm not sure I understand what the offering is here. I read the website, and watched the video, but it doesn't really make sense.
Is Notebook 2.0 just a HTML subset that starts with e.g. <notebook> instead of <html>?
Then what's the notebook kit npm package?
Then there's an editor?
I've come away from the page a little irritated, because starts with hyperbole like "the future of notebooks is here", and then the "hello world" in the new notebook 2.0 format looks like HTML, and a video of what could have just been a generic AI chat window.
Maybe I'm too hungry/tired to make sense of it right now. Will be interesting to read what others think.
They probably could have explained better what hasn’t changed. Observable Notebooks work like a spreadsheet or makefile. The cells aren’t executed top to bottom (like a Jupyter Notebook). Instead they’re executed in dependency order.
Looks like this is still true with the new system:
> The Build API analyzes cell source code to find unbound references, such that the appropriate variable graph can be initialized using the Observable Runtime. For non-JavaScript cells (such as Markdown, HTML, and SQL), the Build API also transpiles the source into a tagged template literal expression.
This is unlike other document formats. They made the cells look like standard JavaScript now (versus the JavaScript variant they had before) but the file format is necessarily different, because the execution model is different. There is a build step to ensure that the cells will be executed in the right order on page load.
I'd love to love them but I don't: I can't seem to justify moving away from Python to use this. What advantages does JS offer for this use case? I' ve never felt that I couldn't do any visualization with Python (but I don't do nice newspaper figures).
Im in the same boat. In theory JavaScript holds more potential to make finely crafted visuals. But you're right, the ecosystem is so mature I still find other ecosystems lacking.
But if anything would change my opinion this has the right set of values
Exporting as client-side webpages, maybe? I liked using it to prototype some interactive D3.js charts that I would later move to my website. Also some people just are more comfortable with JS than Python
It depends where you're coming from, I suppose. The web is pretty popular. Web developers are more familiar with HTML, CSS, and JavaScript, and with various visualization libraries in the JavaScript ecosystem. I quite like Observable Plot [1].
I'm not familiar enough with Python or Jupyter to know how you would build similar visualizations with them. What would you use?
I used the older online notebooks for a bit and really loved the immediacy but I’m always hurting for TypeScript syntax support. The is this a thing w 2.0? Something I’ll need to add later with Vite and only available locally?
It was a bunmer to not be able to copy paste code from my projects into observable. With local notebooks I see more potential for homegrown workflow but for me the benefit was always the platform and how starting a new thing was as simple as going to a webpage and clicking plus.
Anyways excited to play with this once it ends up in the online editor
We’re interested in adding a TypeScript cell mode, probably first with type stripping rather than validating types. I was hoping for a pure JavaScript implementation; maybe we’ll try https://github.com/bloomberg/ts-blank-space? I’ve also been looking at https://github.com/val-town/codemirror-ts for the editor, but it’s a lot of machinery.
The video demonstrating the AI powered desktop app perfectly demonstrates my discomfort with AI. When asked to fix the mapping between county codes, it generated a lookup table itself.
HAVE FUN TESTING THAT. And when you don't, enjoy random countries being mixed up but it mostly working.
Yeah. I can believe that this will improve in the future, and this is a technology preview and all. But my takeaway from the video is that it is still too vibey for me to trust it. If I was coaching a junior data analyst at my company, and they wrote this code, I would happily give some feedback so that they can improve. With AI, I'd probably just do the analysis myself.
In addition to the hardcoded lookup table, here are some other notes on the generated code:
1. Silently assuming that the first page of results contains all of the data feels a bit fragile. If we're not going to bother with paging otherwise, I'd at least assert that the first page contains everything.
2. Unlike the code comment claims, 2022 is not the latest year that has data available, 2023 is. The reason this is worrisome is not that the difference is massive, but because of methodological implications. It looks like the year 2022 came from "remembering" details from some other code that was analyzing this same data set instead of just looking at current version of the data set we're supposed to be analyzing.
3. The code for removing the world aggregate doesn't actually work (although it doesn't matter for the map). The place where it says d.country.id !== "WLD" should be either d.country.id !== "1W" or d.countryiso3code !== "WLD" instead. Also, if it would actually be important to filter this then presumably it would also be important to filter out a bunch of other aggregates as well.
4. The text says "darker colors indicating higher life expectancy", which is pretty much the opposite of how I would describe this color scheme.
5. The analysis given is: "Notice how certain regions tend to cluster together in terms of life expectancy, reflecting similar economic, healthcare, and social conditions". This is such a bland take. It feels like something I could have written before seeing the data. I would try to encourage the analyst to look for something more concrete and more interesting.
6. The actual thing that immediately pops out from the map is that the Central African Republic is a massive outlier with a life expectancy of 18.8 years, whereas every other country is over 50. This doesn't seem plausible. I would do something about that just so that it doesn't mess up the colors of the entire map.
I wish Bostock et al got paid. This is literally the pain of doing open source for some. You basically watch as other people get rich off of your life’s work.
That rambling aside, I like the idea of having a desktop app. Not sure where this notebook fits in with all their other products. They have the Framework which is actually pretty useful.
The Observable Desktop macOS app is a 11.8MB download which decompresses to a 23.9MB application. In a world filled with 200+MB Electron apps this is a very refreshing change!
I've found Bostock Observable to be a fascinating and inspiring platform since the outset in my limited experiments with it. I wish they'd give it a name that isn't a well-known design pattern that it implements, because I end up having to call it "you know, Bostock's thing" in conversation.
However, I definitely don't want to have to type "<script id="2" type="module" pinned>" in order to calculate 1 + 2, and I don't want to have to scan past that in order to find the actual calculations. So I guess I'll have to keep being inspired by Bostock Observable Notebooks 1.0, which I can keep writing in GitHub-flavored Markdown, because it's free software and already checked into Git.
They have or at least used to make their notebook format public, and I always thought if I ever see someone else adopt it then maybe the project will be worth looking at again. I think there could be better ways to capitalize on d3 but this one seems self limiting.
I work on a quantitative user research team, and we've found notebooks to be really useful for crunching and visualizing the data from experiments we've run.
I work at Google and my UXR colleagues are more comfortable with Python than JS, so we use Google Colab, but I'd use Observable if those weren't true.
Out of curiosity, why `<notebook>` as an element and not a standard web-component `<observable-notebook>` or something like that if the goal is to go all-in on "normal" web development?
They made the syntax look familiar so that standard editors (and LLM’s) will work, but this is still a custom file format that requires a build step to convert into an HTML page. See my other comment.
The build tool is open source and unlike a Jupyter notebook, it’s easy to edit without a custom editor. So you don’t need their editor, but it will still be nicer to edit with their editor.
self-plug that you might be interested in: I built a thin wrapper around the Observable Runtime that allows you to more easily use it in a pure HTML context: https://maxbo.me/celine/
I think the goal is explicitly not to go all-in on normal web development, to maintain some opportunity to monetize.
D3.js started as/still is just a (collection of) JS libraries. Then Observable came along, and while it is a nice tool for tinkering with D3, it was not at all obvious how to then move your finished Observable D3 viz to a fully self-hosted, regular website (without paying a subscription to use their custom runtime or servers or whatever).
Now I guess they realize they strayed way too far from "the web" and are back-tracking.
It's pretty clear the future is not really here yet when the LLM can't even debug a simple table join issue on its own. The code to produce the table and chart should reasonably be like 2 lines of code with proper tool design. It looks like the LLM accomplishes it in 100 LOC and still can't manage to do it without significant human intervention.
And this is the take they published to sell the thing!
This looks great. I love the idea behind notebooks and for a long time it was my favourite environment to program in. But slowly I stopped using them because it never quite felt like the code was entirely mine, and alternatives became easier due to llms. This looks like exactly the remedy I was hoping for. I'm excited to start using them again.
Observable's pricing is bit out of bounds for what they offer. It feels like a lot of these data startups are just looking for large companies that will unsuspectingly spend a ton of money on something they don't need.
This is really, really incredible stuff. I can't wait to replace a bunch of terrible BI dashboards.
I've seen and used so many amazing in-house dashboarding and reporting tools, and this suite blows them all out of the water. Can't believe this is all open source too.
I think this is a great way to slice up the product/feature set from a monetization perspective: hosting w/ collaboration is paid, but the tools are all free to use.
I'd love to see support for inline TeX in the markdown. Back when I used observable on the web, I hacked up a `md2` string template to replace /\$(.*?)\$/g with the katex equivalent before calling `md`, but I don't know if there is a workaround in the desktop notebook application.
[+] [-] mourner|7 months ago|reply
Any timeline on bringing the new format to the Web editor? For me, the biggest draw of Observable is being able to instantly start exploring and visualizing some data without ANY setup (creating some local files, installing dependencies, launching a dev server etc.) — just click "New notebook" and off you go, frictionless. I hope this remains a priority.
Also worth mentioning the data viz community aspect, which I used to enjoy a ton — it's a bummer that things like being able to follow certain users and see their new notebooks in a feed were phased out, and that trending notebooks are now hidden under the Help menu instead of being featured on the user's homepage. This might be one of the reasons I stopped coming back to the platform as often, although I realize building a community is very difficult and hard to monetize — just thought I'd share.
[+] [-] mbostock|7 months ago|reply
[+] [-] 1dom|7 months ago|reply
Is Notebook 2.0 just a HTML subset that starts with e.g. <notebook> instead of <html>?
Then what's the notebook kit npm package?
Then there's an editor?
I've come away from the page a little irritated, because starts with hyperbole like "the future of notebooks is here", and then the "hello world" in the new notebook 2.0 format looks like HTML, and a video of what could have just been a generic AI chat window.
Maybe I'm too hungry/tired to make sense of it right now. Will be interesting to read what others think.
[+] [-] skybrian|7 months ago|reply
Looks like this is still true with the new system:
> The Build API analyzes cell source code to find unbound references, such that the appropriate variable graph can be initialized using the Observable Runtime. For non-JavaScript cells (such as Markdown, HTML, and SQL), the Build API also transpiles the source into a tagged template literal expression.
This is unlike other document formats. They made the cells look like standard JavaScript now (versus the JavaScript variant they had before) but the file format is necessarily different, because the execution model is different. There is a build step to ensure that the cells will be executed in the right order on page load.
[+] [-] chaps|7 months ago|reply
It's a neat tool that solves a lot of specific sharing/visualization problems (think of newsroom data analysis).
I like observable but I hate how much it feels like the service has been taken over by product managers looking for that next bump.
[+] [-] dr__mario|7 months ago|reply
[+] [-] MantisShrimp90|7 months ago|reply
But if anything would change my opinion this has the right set of values
[+] [-] wonger_|7 months ago|reply
[+] [-] jeffbee|7 months ago|reply
[+] [-] skybrian|7 months ago|reply
I'm not familiar enough with Python or Jupyter to know how you would build similar visualizations with them. What would you use?
[1] https://observablehq.com/@observablehq/plot-gallery
[+] [-] Hasnep|7 months ago|reply
[+] [-] nsonha|7 months ago|reply
Yes but, probably what they are shooting for are mini apps that you can just copy pasted into a front-end codebase and not some weird-ass python dsl
[+] [-] unknown|7 months ago|reply
[deleted]
[+] [-] jitl|7 months ago|reply
It was a bunmer to not be able to copy paste code from my projects into observable. With local notebooks I see more potential for homegrown workflow but for me the benefit was always the platform and how starting a new thing was as simple as going to a webpage and clicking plus.
Anyways excited to play with this once it ends up in the online editor
[+] [-] mbostock|7 months ago|reply
[+] [-] nixpulvis|7 months ago|reply
HAVE FUN TESTING THAT. And when you don't, enjoy random countries being mixed up but it mostly working.
[+] [-] MrManatee|7 months ago|reply
In addition to the hardcoded lookup table, here are some other notes on the generated code:
1. Silently assuming that the first page of results contains all of the data feels a bit fragile. If we're not going to bother with paging otherwise, I'd at least assert that the first page contains everything.
2. Unlike the code comment claims, 2022 is not the latest year that has data available, 2023 is. The reason this is worrisome is not that the difference is massive, but because of methodological implications. It looks like the year 2022 came from "remembering" details from some other code that was analyzing this same data set instead of just looking at current version of the data set we're supposed to be analyzing.
3. The code for removing the world aggregate doesn't actually work (although it doesn't matter for the map). The place where it says d.country.id !== "WLD" should be either d.country.id !== "1W" or d.countryiso3code !== "WLD" instead. Also, if it would actually be important to filter this then presumably it would also be important to filter out a bunch of other aggregates as well.
4. The text says "darker colors indicating higher life expectancy", which is pretty much the opposite of how I would describe this color scheme.
5. The analysis given is: "Notice how certain regions tend to cluster together in terms of life expectancy, reflecting similar economic, healthcare, and social conditions". This is such a bland take. It feels like something I could have written before seeing the data. I would try to encourage the analyst to look for something more concrete and more interesting.
6. The actual thing that immediately pops out from the map is that the Central African Republic is a massive outlier with a life expectancy of 18.8 years, whereas every other country is over 50. This doesn't seem plausible. I would do something about that just so that it doesn't mess up the colors of the entire map.
[+] [-] lvl155|7 months ago|reply
That rambling aside, I like the idea of having a desktop app. Not sure where this notebook fits in with all their other products. They have the Framework which is actually pretty useful.
[+] [-] gaws|7 months ago|reply
They do get paid. Observable sells team and enterprise licenses: https://observablehq.com/pricing
[+] [-] jeffbee|7 months ago|reply
[+] [-] chaps|7 months ago|reply
[+] [-] simonw|7 months ago|reply
[+] [-] kragen|7 months ago|reply
However, I definitely don't want to have to type "<script id="2" type="module" pinned>" in order to calculate 1 + 2, and I don't want to have to scan past that in order to find the actual calculations. So I guess I'll have to keep being inspired by Bostock Observable Notebooks 1.0, which I can keep writing in GitHub-flavored Markdown, because it's free software and already checked into Git.
[+] [-] th0ma5|7 months ago|reply
[+] [-] stared|7 months ago|reply
[+] [-] mritchie712|7 months ago|reply
[+] [-] bsimpson|7 months ago|reply
I work at Google and my UXR colleagues are more comfortable with Python than JS, so we use Google Colab, but I'd use Observable if those weren't true.
[+] [-] svieira|7 months ago|reply
[+] [-] skybrian|7 months ago|reply
The build tool is open source and unlike a Jupyter notebook, it’s easy to edit without a custom editor. So you don’t need their editor, but it will still be nicer to edit with their editor.
[+] [-] mbo|7 months ago|reply
[+] [-] pphysch|7 months ago|reply
D3.js started as/still is just a (collection of) JS libraries. Then Observable came along, and while it is a nice tool for tinkering with D3, it was not at all obvious how to then move your finished Observable D3 viz to a fully self-hosted, regular website (without paying a subscription to use their custom runtime or servers or whatever).
Now I guess they realize they strayed way too far from "the web" and are back-tracking.
[+] [-] drewda|7 months ago|reply
FWIW, the HTML custom elements spec does require a hyphen: https://html.spec.whatwg.org/multipage/custom-elements.html#...
[+] [-] ModernMech|7 months ago|reply
And this is the take they published to sell the thing!
[+] [-] unknown|7 months ago|reply
[deleted]
[+] [-] jwilber|7 months ago|reply
[+] [-] RobinL|7 months ago|reply
[+] [-] ujkhsjkdhf234|7 months ago|reply
[+] [-] gaws|7 months ago|reply
[+] [-] xyzzy_plugh|7 months ago|reply
I've seen and used so many amazing in-house dashboarding and reporting tools, and this suite blows them all out of the water. Can't believe this is all open source too.
I think this is a great way to slice up the product/feature set from a monetization perspective: hosting w/ collaboration is paid, but the tools are all free to use.
[+] [-] dunham|7 months ago|reply
[+] [-] amdivia|7 months ago|reply
For the time being I hacked something using Observable Framework [1], but still, it isn't the full set
1: https://observablehq.com/framework/
Edit: nvm just saw the notebook kit
[+] [-] gaws|7 months ago|reply
No? How would they make money?
[+] [-] blef|7 months ago|reply