[abefetterman]

This is actually a really exciting development to me. (Note, what is exciting is the "optometrist algorithm" from the paper [1] not necessarily googles involvement as pitched in the guardian). Typically a day of shots would need to be programmed out in advance, typically scanning over one dimension (out of hundreds) at a time. It would then take at least a week to analyze the results and create an updated research plan. The result is poor utilization of each experiment in optimizing performance. The 50% reduction in losses is a big deal for Tri Alpha.

I can see this being coupled with simulations as well to understand sources of systematic errors, create better simulations which can then be used as a stronger source of truth for "offline" (computation-only) experiments.

The biggest challenge of course becomes interpreting the results. So you got better performance, what parameters really made a difference and why? But that is at least a more tractable problem than "how do we make this better in the first place?"

[1] http://www.nature.com/articles/s41598-017-06645-7

[marco_salvatori]

Though this work may seem exciting, there is an existing, respected body of work available on how to mathematically structure a search over a large parameter space and how to mathematically interpret experimental responses. That body of work is a subset of applied statistics called design of experiments. It helps scientists avoid the common failures that result from doing exactly what was done here, random space exploration and non rigourous evaluation of results.

For this to be exciting I would expect some indication as to how this method extends and enhances the existing science of experimental methods and the trade offs involved with using their method. I dont see that.

[adrianratnapala]

It would not surprise me if high-tech companies are inventing new, useful things in this field.

In my career as first a scientist and then an engineer, I've found very few practical users of highly technical experimental design theory, and all of them were in industry. These algorithms move about intelligently along all dimensions of some search space, whereas in the lab we prefered to turn just one knob at a time.

One reason is that the algorithms are optimally seraching for "known unknowns" -- that is they assume they roughly understand the problem. The lab is a world of unkown unknowns where the more plodding, understandable protocols tend to be safer.

But in industry, some problems are of the known-unknowns type. And experiment runs can burn up seriously expensive hardware time. So it makes sense for fusion researchers and cloud-computing giants a like to invent new practical ways to optimise searches.

Besides, optimising searches is what Googlers are for.

[abefetterman]

Reading their actual paper further, it seems I read a bit too much into the original article. However, as their paper mentions:

> The parameter space of C-2U has over one thousand dimensions. Quantities of interest are almost certainly not convex functions of this space. Furthermore, machine performance is strongly affected by uncontrolled time-dependent factors such as vacuum impurities and electrode wear.

I'm not aware of DOE procedures that are robust to these types of issues, and would certainly appreciate any literature you have on the subject.

Regardless of theoretical literature, this procedure has enabled a dramatic shift in how these scientists think about their experiment. Furthermore it has enabled them to achieve results much faster than before (if you have been following Tri Alpha, it has been a real slog). Both of these are exciting to me even if they don't break new ground in the design of experiments.

[Libbum]

Interesting. Are you able to provide some links to decent resources on this topic?

[jlarocco]

As a complete outsider, I don't understand what's special about the "optometrist algorithm." As described in the Nature article it's just hill climbing using humans as the evaluation function.

Isn't it basically the same thing they were already doing but more granular?

[abefetterman]

Basically nobody was using automated gradient descent / etc because of the proclivity of these algorithms to get stuck on a boundary. The problem is the boundaries are not well defined. One example might be a catastrophic instability. If it gets triggered it has the potential to damage the machine. But the exact parameters in which the instability occurs are not well known. So with this algorithm you mix the best of both worlds: the human can guide away from the areas where we think instabilities are, the machine can do it's optimization thing. It's pretty simple overall but enables a big shift in how experiments are run.

Edit to add: these instabilities often look just like better performance on a shot-to-shot basis, which makes the algos especially tricky. Using a human we could say "this parameter change is just feeding the instability" vs "oh this is interesting go here"

[amelius]

Perhaps a stupid question, but why can't the whole experiment be run as a simulation?

[zaph0d_]

Even if this would be the dream of a lot of theoretical physicists to replace experiments with simulations, this must not happen! Ever! Even if every complex system in the world could be simulated in reasonable time it would still require experiments to verify or falsify the simulation results. A simulation is essentially just a calculation from a model someone came up with to describe a system. In order to check how good the model is one has to check it against experimental data. Just expanding the models without experimental verification will not necessarily result in a good theoretical description. It would be like writing software without testing the components and expecting it to work correctly when you're done. There was recently an article on HN where economists were described as the astrologers of our time [1] since they do not verify their mathematical models to an extent where they can predict economical systems. This is another example where more experimental data should be considered in order to falsify certain theories.

Those are the reasons why string-theorist will not (and should not) get any Nobel price in the next decades. Since its predictions are hard to measure on those small scales there's no way of telling if the model is any good until it is compared against suitable experimental data.

[1] https://aeon.co/essays/how-economists-rode-maths-to-become-o...

[paulsutter]

Because fusion simulations are really hard. This simulation[1] took 15 million hours of CPU time to model a cubic cm of plasma. The results were used to update 5 scalar parameters in a model.

[1] http://news.mit.edu/2016/heat-loss-fusion-reactors-0121

[kleer001]

The numbers are too big, and nature is hiding stuff from us.

So we can't simulate it because we don't know enough to simulate it. And even if we did know there's not enough computing power to do so.

[noobermin]

The system is fundamentally 6^N dimensional with N~10^23.