Inre your first link, things can get a bit awkward if someone uses javascript features that aren’t supported in your browser. The error message is not especially non-expert-reader-friendly: https://i.imgur.com/Hw9NDPA.png
Any plans to add some kind of transpilation or ...?
Just in case people were wondering, the site seems to have been overwhelmed for the past 10-15 minutes. But there's a YouTube demo of the tech https://news.ycombinator.com/item?id=16275040
I guess an oversimplified description would be that this is like a Jupyter Notebook specifically for JavaScript. Libraries like D3 are pre-loaded and immediately accessible. Am definitely interested in hearing the details about what it is built with and medium to long-term plans for the service.
Note that the Jupyter Notebook service generally requires you to be installing and running Python etc. on your own computer. Jumping into an Observable notebook is as easy as opening your browser and signing in via GIthub
It’s easy to use D3 within Observable, but it’s not pre-loaded or specifically built-in. Any browser library published to npm can be loaded via require (for AMD) or dynamic import (for ES modules). I’ve published a bunch of notebooks with D3 but you’ll find other ones with THREE.js, Vega-Lite, Leaflet and other libraries.
Do you want it to be so easy that you jump into the notebook immediately upon logging in, the hard part is that javascript will run in your browser - but python or R need a kernel running somewhere?
Please, Observable, put something on the page that says what language this is!
I figured this was like Colaboratory (https://colab.research.google.com) or Azure Notebooks (https://notebooks.azure.com/) or some other hosted-jupyter variant. There was no information about how it should work when I went to the scratchpad, and I could barely tell it was Javascript when picking one of the sample notebooks.
This looks fantastic. The model appears very clean and different than Jupyter in many ways:
- cells can have names (it seems each cell can only export one name that is visible to the rest of the notebook)
- dependent cells auto-update when a source cell changes
- cells can be generators that auto update up to 60 times/sec
- cell values can be tied to UI elements easily
So instead of Jupyter's model where all cells execute in the same namespace, here you have named cells explicitly connected to each other in a graph and executed in topological order (not linear order).
This is really cool and would be a great way to introduce programming concepts in an educational setting. Instant-feedback, inline documentation in a narrative style, and running on a browser really lowers the barrier for entry.
I would have loved something like this when I was learning programming in middle-school decades ago.
Since it comes from the guys behind D3 it's not surprising that much of the visuals come from D3. This is really evocative, but it may limit some of the audience for this tool. I would encourage the developers to take some cues from Apple's swift playgrounds and add visuals and sliders for even some of the simpler language constructs like loops and add some "simple" wrapper functions for drawing elementary shapes. This would definitely be a shift from the intended purpose as I see it right now, but it would have the side-benefit of increasing debugability as well.
Great work - look forward to having the occasion to using it.
I would have loved something like this while learning to program as well. I mean ... the Ruby REPL was fun, but this sort of thing is just so much more tactile and visual.
We haven't had time to produce as many demo notebooks as we might have liked, but these notebooks really aren't D3-specific in the slightest. They're a reactive flavor of JavaScript, and anything that produces a value or can be rendered to the DOM will work.
Remember — a "slider" here isn't anything special. It's just a chunk of DOM that produces a number as a value. It can be replaced with anything else you can imagine that produces a number as a value. There's no API there ;)
The app itself is well done, but the editing experience is quite lacking...It's 2018 and people don't want to write html/css/js as if writing txt files. https://codesandbox.io can be a good inspiration to learn from. I believe its popularity is a result of using Monaco[0] and supporting Language Servers[1].
I would advise looking into editing/language support early. Auto completion (for DOM API, D3 API, etc) and error checking can be very helpful to the user.
Still, congratulations to the launch (the app looks beautiful and UX is pretty solid)! When I was reading @mbostock's medium post[2] a few months ago I knew something like this will popup.
Anyway, back to building my own thing which focuses on the editing side of playgrounds, which I think is what most online ones (Codepen / JSFiddle) lack.
This works by executing Javascript code written by the user. Since these notebooks are shareable, and there is also a log-in feature, do any security features need to be put in place to prevent, say, someone linking me to a notebook that sends my login credentials for observablehq.com to a malicious host? That is, is the javascript inside the notebook sandboxed in any way?
I don't fully understand how modern web security works. But "executing arbitrary javascript written by user A on website B viewed by user C" seems worrying.
Yep, Observable runs user JavaScript code, but that code has no access to your login credentials or anything from the 'application': user code runs in a sandboxed iframe on a separate host - host that doesn't have any cookies or user information. The application communicates with that iframe host through serialized postMessage information only. That's the gist of the 'user code' portion - the rest of the application has additional layers of best-practice security like very strict content security policies and so on that defend from other kinds of attacks.
The homepage copy didn't quite capture me... until I realized it was the new thing by D3's Mike Bostock! Maybe worth name-dropping that somewhere on that page? Though I don't know how many people would respond to that other than myself (hopefully lots of people).
Feedback: I instantly understood that it was something similar to Jupyter, and my very first mental question was "I hope it's in JS", and this took a surprisingly long few seconds and two page navigations to establish -- I expected the choice of language to be on the splash page.
Anyway, very well done, and it's quite clear to me that JS is going to trounce Python for nearly everything long-term. The JS ecosystem is just an unstoppable freight train.
And a fancier version that spins, reuses the canvas element instead of re-rendering it from scratch, projects the quakes as circles on the surface of the sphere, and offers a bunch of other options to configure (like if you'd like to chart the past day, week or month of quakes): https://beta.observablehq.com/@jashkenas/quakespotter-0-1
This couldn't have come at a more convenient time for me. After years of telling myself I'd learn D3 I have finally started to as well as use it at work. I was using an extension in Jupyter but it was behaving unpredictably, and not being a frontend engineer by trade I found the process of sublime + browser foreign and cumbersome. As a big fan of jupyter and zeppelin I am stoked to see notebooks entering the js data viz domain.
Curious as to know how developers view the notebook paradigm? I feel like there is a sense of pride in working from the command line or vi, so wonder how these UI heavy approaches are received by the js/python/scala community.
I learned D3 a year back and am reasonably fluent at it now and I'm not sure I would ever pick this platform to learn D3. The easiest way to learn is simply to have a blank HTML file with your D3 code in it. The reason you want to do this is because you want to take advantage of powerful debugging tools that exist within modern browsers (Chrome dev tools, or equivalent from Firefox). ObservableHQ will simply add a layer in between that will make everything more annoying to debug and slow down experimentation.
I think it is really cool, and I have mountains of respect for Mike for his contributions to the field of data visualization/javascript, but my feeling is that this is more suited for sharing "notebooks" that are interactive and also to potentially allow for collaborative notebook building/demos.
In essence, this is a giant evolution over his previous work on bl.ocks.org (https://bl.ocks.org/mbostock)
I know many developers who've integrated Jupyter Notebooks in their workflow too. For Python-specific things, it's a great way to replicate bugs, try out a few initial solution paths, and share them with collaborators who can then interact with.
Regarding D3, I haven't tried that out yet, although I've used the `folium` Python package, which generates leaflet.js code you can put on a website. Otherwise, maybe check out [Jupyter Dashboards](https://github.com/jupyter/dashboards), whose interactive widgets seem to be in the same vein as OP.
To me, a netbook like this is similar to a REPL and works well for visualization tasks. You can figure it out in the netbook/REPL and then move it to "real code" once you've got it figured out.
To me the killer app for this is to empower data scientists to create useful, ad-hoc visualization apps for their results. Right now the html / presentation story for Jupyter notebooks is awful.
Mike, Jeremy, and Tom, this looks awesome. I'm looking forward to put it through the paces.
>Right now the html / presentation story for Jupyter notebooks is awful.
How so?
Ive found it pretty straightforward to convert matplotlib plots to gif/mp4 for presentation (note: this can be done automatically within the notebook), and plotly is pretty good for truly interactive stuff.
Seeing a lot of posts of the "Observable vs Jupyter" variety. I really don't think this has to be a this versus that kind of discussion. There is plenty of room for both.
I like the idea, I've even written something similar myself, but for me a few things were not clear.
How would I load in data that is stored on my harddrive (not web addressable) without having to run my own server or go through the file selector popup box each time. User JS can't just read the harddisk (a requirement for web security). If I have to run my own server then a major selling point of this is gone.
I work with large tabled data and very often use vectorised functions using numpy or pandas, working on entire columns with a fantastically simple interface. A = B * x, for arrays B, scalars x returning a new array A. Is there something as beautiful available, working around the lack of operator overloading in JS? I've only seen string abuse like p('A') = p('B*x'). Auh.
Good question! Mike wrote a notebook about getting data into Observable here: https://beta.observablehq.com/@mbostock/introduction-to-data - the gist is that right now any host that supports CORS does the trick, and the combination of GitHub Gist & RawGit is pretty good for the task. It'd be nice to drag & drop data right in the app, though, so... stay tuned.
And, yep - something like pandas for JavaScript would also be nice... also stay tuned.
So I finally got around to taking a good look at this last night after work. I adapted a single page d3 app that I'd written many years ago to model sous vide cooking.
A few comments:
- Probably needs some more documentation. Maybe a quick reference for the API that's available and a list of keystrokes. I will admit though, I bailed halfway through the tutorials so I could start playing with it.
- I'd like to be able to drag and drop cells (or otherwise rearrange them). I created my page, but then wanted to better organize it. Making new cells and cut/pasting content was annoying.
- The Opt-Return thing is killing me. It executes the cell in Jupyter and splits the cell into two in Observable. I can't think of a reason I'd want to split a cell in the middle. I believe splitting in the middle will always result in two cells with syntax errors. Perhaps just always create and focus a new, empty cell?
- It'd be nice if there was an easy way to delete a cell. Currently, I'm emptying it and then joining with the previous via option-del, which concatenates cells. I believe joining two cells with content will always result in a syntax error, so I'm not sure the utility of that.
- The samples were useful, especially for learning how d3 fits into this. I ended up breaking up a long function (to build the d3 graph) into multiple cells, taking cues from the samples. (I also learned that d3 has changed a little since v3.)
It would also be nice to be able to rename variables (i.e. refactor), but that's not a critical feature.
Glanced over this yesterday, came back to it today when I saw it was by jashkenas, who has done some real innovative work over the years. @jashkenas -- is this your primary project now? Are you building a business around this or is it more for fun/open source?
A few differences. 1. You don’t have to install anything, so running (or forking) someone else’s notebook is as simple as clicking a link. 2. It’s reactive, which means simpler code and better feedback, particularly when building user interfaces or analyzing realtime data. 3. It runs in the browser so you have direct access to powerful graphics (and GPU computation): Canvas, SVG, WebGL. 4. You can quickly import code for reuse from other notebooks, making a notebook a kind of lightweight library as well as a human-readable document.
basically if you have a bunch of code blocks that depend on each others' results - you don't have to remember which ones to update if you changed one, it will update automatically.
especially useful with data visualization/exploration, but also helps a lot if you have a complex sequence where it's easy to get lost and forget to recompute something. there's a youtube link in the comments here which explains a lot.
[+] [-] jashkenas|8 years ago|reply
Nick Strayer shows how t-SNE is similar (equivalent?) to a force-directed graph layout: https://beta.observablehq.com/@nstrayer/physics-based-t-sne
Jim Bumgardner published a tutorial that starts with drawing circles with canvas, then fibonacci spirals, then colorful sunflower seeds: https://beta.observablehq.com/@jbum/circles-spirals-and-sunf...
Kamil Slowikowski drew a lovely Barnsley Fern: https://beta.observablehq.com/@slowkow/barnsley-fern
Alan Palazzolo did a Mandelbrot fractal with randomized coloring: https://beta.observablehq.com/@zzolo/mandlebrot-set
Justin Palmer uses PROJ4JS to transform and scale a local map: https://beta.observablehq.com/@caged/local-map-projection-wi...
[+] [-] jacobolus|8 years ago|reply
Any plans to add some kind of transpilation or ...?
[+] [-] jdironman|8 years ago|reply
[+] [-] danso|8 years ago|reply
I guess an oversimplified description would be that this is like a Jupyter Notebook specifically for JavaScript. Libraries like D3 are pre-loaded and immediately accessible. Am definitely interested in hearing the details about what it is built with and medium to long-term plans for the service.
Note that the Jupyter Notebook service generally requires you to be installing and running Python etc. on your own computer. Jumping into an Observable notebook is as easy as opening your browser and signing in via GIthub
[+] [-] mbostock|8 years ago|reply
It’s easy to use D3 within Observable, but it’s not pre-loaded or specifically built-in. Any browser library published to npm can be loaded via require (for AMD) or dynamic import (for ES modules). I’ve published a bunch of notebooks with D3 but you’ll find other ones with THREE.js, Vega-Lite, Leaflet and other libraries.
[+] [-] goatlover|8 years ago|reply
[+] [-] eoinmurray92|8 years ago|reply
https://aws.amazon.com/sagemaker/
https://cloud.google.com/datalab/
https://gryd.us/
https://kyso.io (disclaimer - I'm a founder)
Do you want it to be so easy that you jump into the notebook immediately upon logging in, the hard part is that javascript will run in your browser - but python or R need a kernel running somewhere?
[+] [-] aw3c2|8 years ago|reply
[+] [-] make3|8 years ago|reply
[+] [-] avip|8 years ago|reply
[+] [-] anentropic|8 years ago|reply
[+] [-] web007|8 years ago|reply
I figured this was like Colaboratory (https://colab.research.google.com) or Azure Notebooks (https://notebooks.azure.com/) or some other hosted-jupyter variant. There was no information about how it should work when I went to the scratchpad, and I could barely tell it was Javascript when picking one of the sample notebooks.
[+] [-] shalabhc|8 years ago|reply
- cells can have names (it seems each cell can only export one name that is visible to the rest of the notebook)
- dependent cells auto-update when a source cell changes
- cells can be generators that auto update up to 60 times/sec
- cell values can be tied to UI elements easily
So instead of Jupyter's model where all cells execute in the same namespace, here you have named cells explicitly connected to each other in a graph and executed in topological order (not linear order).
The introduction covers these pretty well: https://beta.observablehq.com/@mbostock/five-minute-introduc...
[+] [-] ryanianian|8 years ago|reply
I would have loved something like this when I was learning programming in middle-school decades ago.
Since it comes from the guys behind D3 it's not surprising that much of the visuals come from D3. This is really evocative, but it may limit some of the audience for this tool. I would encourage the developers to take some cues from Apple's swift playgrounds and add visuals and sliders for even some of the simpler language constructs like loops and add some "simple" wrapper functions for drawing elementary shapes. This would definitely be a shift from the intended purpose as I see it right now, but it would have the side-benefit of increasing debugability as well.
Great work - look forward to having the occasion to using it.
[+] [-] jashkenas|8 years ago|reply
We haven't had time to produce as many demo notebooks as we might have liked, but these notebooks really aren't D3-specific in the slightest. They're a reactive flavor of JavaScript, and anything that produces a value or can be rendered to the DOM will work.
For example, here are a whole mess of Plotly charts: https://beta.observablehq.com/@jashkenas/plotly-js
Here are some (wild) WebGL, Three.js and regl sketches: https://beta.observablehq.com/collection/webgl
And here are some simple native inputs you can use to control your sketches: https://beta.observablehq.com/@jashkenas/inputs
Remember — a "slider" here isn't anything special. It's just a chunk of DOM that produces a number as a value. It can be replaced with anything else you can imagine that produces a number as a value. There's no API there ;)
[+] [-] octref|8 years ago|reply
I would advise looking into editing/language support early. Auto completion (for DOM API, D3 API, etc) and error checking can be very helpful to the user.
Still, congratulations to the launch (the app looks beautiful and UX is pretty solid)! When I was reading @mbostock's medium post[2] a few months ago I knew something like this will popup.
Anyway, back to building my own thing which focuses on the editing side of playgrounds, which I think is what most online ones (Codepen / JSFiddle) lack.
[0]: https://microsoft.github.io/monaco-editor
[1]: https://microsoft.github.io/language-server-protocol
[2]: https://medium.com/@mbostock/a-better-way-to-code-2b1d2876a3...
[+] [-] rntz|8 years ago|reply
I don't fully understand how modern web security works. But "executing arbitrary javascript written by user A on website B viewed by user C" seems worrying.
[+] [-] tmcw|8 years ago|reply
Yep, Observable runs user JavaScript code, but that code has no access to your login credentials or anything from the 'application': user code runs in a sandboxed iframe on a separate host - host that doesn't have any cookies or user information. The application communicates with that iframe host through serialized postMessage information only. That's the gist of the 'user code' portion - the rest of the application has additional layers of best-practice security like very strict content security policies and so on that defend from other kinds of attacks.
[+] [-] jashkenas|8 years ago|reply
If you inspect them, you'll see that those frames are hosted on observableusercontent.com instead of observablehq.com.
It's a similar security model to how raw files on GitHub are served from githubusercontent.com.
[+] [-] simonw|8 years ago|reply
[+] [-] trevyn|8 years ago|reply
Feedback: I instantly understood that it was something similar to Jupyter, and my very first mental question was "I hope it's in JS", and this took a surprisingly long few seconds and two page navigations to establish -- I expected the choice of language to be on the splash page.
Anyway, very well done, and it's quite clear to me that JS is going to trounce Python for nearly everything long-term. The JS ecosystem is just an unstoppable freight train.
[+] [-] rrherr|8 years ago|reply
"Observable: An Earthquake Globe in Ten Minutes" https://www.youtube.com/watch?v=uEmDwflQ3xE
[+] [-] jashkenas|8 years ago|reply
The finished notebook from the screencast (which you can fork): https://beta.observablehq.com/@jashkenas/earthquakes
And a fancier version that spins, reuses the canvas element instead of re-rendering it from scratch, projects the quakes as circles on the surface of the sphere, and offers a bunch of other options to configure (like if you'd like to chart the past day, week or month of quakes): https://beta.observablehq.com/@jashkenas/quakespotter-0-1
[+] [-] cpsempek|8 years ago|reply
Curious as to know how developers view the notebook paradigm? I feel like there is a sense of pride in working from the command line or vi, so wonder how these UI heavy approaches are received by the js/python/scala community.
[+] [-] _coveredInBees|8 years ago|reply
I think it is really cool, and I have mountains of respect for Mike for his contributions to the field of data visualization/javascript, but my feeling is that this is more suited for sharing "notebooks" that are interactive and also to potentially allow for collaborative notebook building/demos. In essence, this is a giant evolution over his previous work on bl.ocks.org (https://bl.ocks.org/mbostock)
[+] [-] leblancfg|8 years ago|reply
Regarding D3, I haven't tried that out yet, although I've used the `folium` Python package, which generates leaflet.js code you can put on a website. Otherwise, maybe check out [Jupyter Dashboards](https://github.com/jupyter/dashboards), whose interactive widgets seem to be in the same vein as OP.
[+] [-] spot|8 years ago|reply
[+] [-] softawre|8 years ago|reply
[+] [-] krebby|8 years ago|reply
Mike, Jeremy, and Tom, this looks awesome. I'm looking forward to put it through the paces.
[+] [-] joshuamorton|8 years ago|reply
How so?
Ive found it pretty straightforward to convert matplotlib plots to gif/mp4 for presentation (note: this can be done automatically within the notebook), and plotly is pretty good for truly interactive stuff.
[+] [-] krazydad|8 years ago|reply
[+] [-] bayonetz|8 years ago|reply
[+] [-] MJSplot_author|8 years ago|reply
How would I load in data that is stored on my harddrive (not web addressable) without having to run my own server or go through the file selector popup box each time. User JS can't just read the harddisk (a requirement for web security). If I have to run my own server then a major selling point of this is gone.
I work with large tabled data and very often use vectorised functions using numpy or pandas, working on entire columns with a fantastically simple interface. A = B * x, for arrays B, scalars x returning a new array A. Is there something as beautiful available, working around the lack of operator overloading in JS? I've only seen string abuse like p('A') = p('B*x'). Auh.
[+] [-] tmcw|8 years ago|reply
And, yep - something like pandas for JavaScript would also be nice... also stay tuned.
[+] [-] d--b|8 years ago|reply
[+] [-] dunham|8 years ago|reply
A few comments:
- Probably needs some more documentation. Maybe a quick reference for the API that's available and a list of keystrokes. I will admit though, I bailed halfway through the tutorials so I could start playing with it.
- I'd like to be able to drag and drop cells (or otherwise rearrange them). I created my page, but then wanted to better organize it. Making new cells and cut/pasting content was annoying.
- The Opt-Return thing is killing me. It executes the cell in Jupyter and splits the cell into two in Observable. I can't think of a reason I'd want to split a cell in the middle. I believe splitting in the middle will always result in two cells with syntax errors. Perhaps just always create and focus a new, empty cell?
- It'd be nice if there was an easy way to delete a cell. Currently, I'm emptying it and then joining with the previous via option-del, which concatenates cells. I believe joining two cells with content will always result in a syntax error, so I'm not sure the utility of that.
- The samples were useful, especially for learning how d3 fits into this. I ended up breaking up a long function (to build the d3 graph) into multiple cells, taking cues from the samples. (I also learned that d3 has changed a little since v3.)
It would also be nice to be able to rename variables (i.e. refactor), but that's not a critical feature.
[+] [-] peregrine|8 years ago|reply
[+] [-] chatmasta|8 years ago|reply
Would be curious to hear plans to monetize it.
[+] [-] orbifold|8 years ago|reply
[+] [-] polalavik|8 years ago|reply
[+] [-] mbostock|8 years ago|reply
[+] [-] wizzard0|8 years ago|reply
especially useful with data visualization/exploration, but also helps a lot if you have a complex sequence where it's easy to get lost and forget to recompute something. there's a youtube link in the comments here which explains a lot.