dustintran's comments

Hello. I'm the person that was linked to in that GitHub issue!

I sympathize with the post's frustration. The TF tutorials on the official website are well-written. But they mostly cover basic features, and as a recent Reddit thread described (https://old.reddit.com/r/MachineLearning/comments/e4pxqp/d_i...), the support ecosystem is lacking as StackOverflow and blog posts are out-of-date due to all the software churning. I'm not a TF engineer, but as someone with experience designing libraries on top of TF, even I find myself sifting through Stack Overflow/blog post code to find the new best practices..

Regarding Bayesian layers, it's actually a NeurIPS paper this year (https://papers.nips.cc/paper/9607-bayesian-layers-a-module-f...). I worked on an early prototype in TensorFlow Probability but ended up abandoning the design as I found it inflexible in practice. The solution is the NeurIPS paper, and it's experimental: there are no promises of stability (in fact, we even moved the code from Tensor2Tensor to another repository (https://github.com/google/edward2/), of which has yet to have an official package release!).

Software for uncertainty models is more on the research fringe, and this should be made clearer in official TensorFlow solutions building on these designs.

The author doesn't really apply Bayesian inference. He applies Bayes' rule which is a mathematical property.

Note that this should not validate or invalidate his claims, but here are videos of his teaching: http://math1afall2015.blogspot.com

You've voiced my own feelings exactly.

The paper described in the article: http://arxiv.org/pdf/1506.05439v1.pdf

Hi, stan dev here. I think viewing Stan as a better BUGS is helpful but limiting.

The syntax is similar, but the class of models Stan fits is far more general. The class of algorithms we have available also goes beyond MCMC, e.g., variational inference, optimization, and interfaces to Stan exist on all primary programming languages. It's more helpful to think of Stan as its own probabilistic programming language, and arguably the biggest entity with the largest user base.

Most all probabilistic programming languages in fact treat every model as equivalent to an HMM. So certainly inference on them can be done.

The conundrum during all research talks.

To clarify, it has been studied in Zoubin Ghahramani's group [1] (and also more recently in Ryan Adam's group [2]), and it's most widely known through Radford Neal [3] who's won a lot of competitions using the Bayesian approach to NNs.

[1] http://mlg.eng.cam.ac.uk

[2] http://hips.seas.harvard.edu

[3] http://www.cs.utoronto.ca/~radford/res-neural.html

This is not completely accurate. Deep learning at the moment simply means the use of "deep" architectures in neural networks. Graphical models, standard Bayesian hierarchical models, and the likes all form hierarchies of features as well and are commonly practiced.

I strongly disagree with not using linear models, at least to build some theory and intuition before continuing with more sophisticated algorithms. What I find to be more egregiously misused when doing machine learning in practice is that everyone too often flocks to the state of the art with little understanding why. There's no reason for example to spend weeks (or months) tuning a incredibly deep neural network if the current predictive ability is enough and there are higher priority matters to work on.

Moreover, there's just too much of an emphasis on prediction. Design and analysis of experiments, handling missing data and the context of the data sets, and quantifying one's uncertainty about parameters in a principled manner for robust estimators are very underappreciated skills in the community. Using p values arbitrarily and "95% confidence intervals" based on an unchecked normal approximation is incredibly more harmful than not doing anything at all. There's just so much more to machine learning than supervised learning.

I'm curious to see when spam bots will eventually use sophisticated language models in order to generate text, aimed to combat a search engine's own machine learning algorithms which detect spam. Barring the actual scenario where Google's resources are larger and Google's employees more intelligent than spammers, it would be interesting to see spam bots generate more and more intelligent content in a war of the AI.

The problem is the more than likely chance that only doing one resample (in bootstrap terminology) may not lead to any clear rejection of statistical significance. That is, it is quite likely that in practice of this "A/A/B" testing, the second control group may look pretty much like the first, and so you can't clearly argue that the control deviates far away or not from the "Great Idea".

What do you then? Well, the natural answer is to try the resample again. Do this n times, get an average for the variability, and that is precisely bootstrap.

This is not a good analogy. Money is the obvious reason one stays at the "lower hill". Thus it is not a question of whether the person can stoop down from the "lower hill" and get to the "higher" one. It's a question of which hill is really higher (by that person's standards).

Yup, it seems k was fixed since the first time these scripts were made for NIPS 2012 (?). Some of the more well-established advances since LDA would also likely help, like HDP.

I imagine more preventative solutions to develop earlier which are more effective, e.g., "treat" citizens who are predicted highly likely to be cause a dangerous outcome in the future.

Or in the simple context of computer vision and the camera, raise warnings as individuals on screen act increasingly more suspicious. I imagine future technology will at least have intelligent security cameras that signal alarms based on the current video feed, not just when it's suddenly shut off for example.

You can split screens (and virtually never use the mouse) in Vim. What particular feature are you looking for here?

I'm not sure how useful this would be. Do people willfully drive from work to a coffee shop in order to get work done, and not simply walk to their favorite coffee shop within walking distance? It seems like distance is more the contributing feature than anything else.

Shameless hijacking and also just not as good..