I was going to interview at LandingAI. I was asked before the interview to install a spyware browser extension to monitor my traffic to detect if I was cheating during the interview. I respectfully declined and didn't have that interview.
Wow if you can “cheat” during an interview - meaning either that they’re asking trivial, google-able stuff or that they’re so bad at interviewing that they can’t tell if you actually know your stuff - then their hiring process is pretty bad.
It's literally our job to not just assume the possible solution that rolls off the top of our heads might not be the most up to date / best practice and to research it
I'd just note that if pushed by circumstances (if one was willing to be interviewed in spite of their ways), the interview environment could be (would be) on a throwaway virtual machine...
Possibility which, by the way, makes the interviewer's cautionary move generally useless.
My understanding is that they are trying to automate the data preparation steps that seasoned ML practitioners are doing anyway today.
The fact that he tries this in manufacturing makes the case stronger. In most manufacturing companies you do not have access to top ML talent.
You have Greg who knows python and recently visualized some production metrics.
If we could empower Greg with automated ML libraries that guide him in the data preparation steps in combination with precooked networks like autogluon, then manufacturing could become a huge beneficiary of the ML revolution.
Greg probably also knows SAS and AMPL, and has a good knowledge of ops research, which is within stone-tossing distance of whatever ML is pretending to be this week.
Visual inspection in manufacturing is a very solved problem, especially in the AI field. The big bucks are in pattern matching anyway... it's a dumb company.
That is the problem with generalization and cop outs like these. It's no good to people in the field doing actual work where the devil is in the detail.
Big data is fairly important to a lot of things, for example I was listening to Tesla's use of Deep net models where they mentioned that there were literally so many variations of Stop Signs that they needed to learn what was really in the "tail" of the distribution of Stop Sign types to construct reliable AI
I’ve been wondering about the limits of data-centric approach – there seems to be this implicit notion that more data equals better performing ML or AI. I think it would be interesting to imagine a point of diminishing return on additional data if we consider that our ability to perceive is probably largely based on two parts - sensory input and knowledge. Note that I’m making an explicit distinction here on the difference between data and knowledge.
For instance, an English speaker and a non-English speaker may listen to someone speaking English and while the auditory signals received by both are the same, the meaning of the speech will only be perceived by the English speaker. When we’re learning a new language, it’s this ‘knowledge’ aspect that we’re enhancing in our brain, however that is encoded.
This knowledge part is what allows us to see what’s not there but should be (e.g. the curious incident of the dog in the night) and when the data is inconsistent (e.g. all the nuclear close calls). I’m really not sure how this ‘knowledge’ part will be approached by the AI community but feel like we’re already close to having squeezed out as much as we can from just the data side of things.
Somewhat related, we have a saying in Korean – ‘you see as much as you know’.
It does in general, but what is elaborated and how? Structuring patterns is not the same as "knowledge" (there are missing subsystems), and that fed data is not fed efficiently, with ideal efficiency - compare with the realm in which "told one notion you acquire it" (this while CS is one of the disciplines focusing on optimization, so it would be a crucial point).
I have a feeling that too much knowledge might slow learning process as it's harder to spot/test observe steepest gradient. At least that's how it feels intuitively from human PoC. From computation that would be just little more computation but I guess would mean slower convergence also. Taking math as more extreme example it's hard to understand something complex unless you understand basic algebra.
Anyone knows if this might be true mathematically speaking? Does order of data matters?
Can't you consider that knowledge is a function of previous data? In your example, the 2 individuals actually didn't receive the same amount of data because the English speakers received data previously that allowed him to build some kind of "knowledge" that allows him to solve specific related tasks (understanding a spoken sentence). This would be the equivalent of transfer learning where "knowledge" is a model trained on previous, more general, data.
data quality is important. every ai project i've worked on has started with visualizing the data and thinking about it.
it's easy to get complacent and focus on building big datasets. in practice, looking at the data often reveals issues sometimes in data quality and sometimes scope of what's in there (if you're missing key examples, it's simply not going to work).
Glad to see the term ML being used more often than AI in the comments as it looks like most "AI" models are trained for image classification.
Having said that, the idea of "doing more with less" sounds interesting and I wonder what it means exactly. Does it mean taking a dataset of 50 images and to create 1000s of synthetic images from it?
Yeah I was very interested about that point in particular. I think synthetic data is one of the ideas, but I got the sense that he also means helping to identify what makes a data set good, even if small. It looks like Andrew Ng is developing a platform for automatically detecting whether a dataset is suitable and, if not, what are the steps to improve it. A sort of automated ML consultant, allowing you to sell capabilities much cheaper than if you needed to consult an actual expert.
Pretty interesting. Mr. Ng claims that for some applications having a small set of quality data can be as good as using huge set of noisy data.
I wonder if, assuming the data is of highest quality, with minimal noise, having more data will matter for training or not. And if it matters, on what degree?
In general you want to add more variants of data but not so much that the network doesn't get trained by them. Typical practice is to find images whose inclusion causes high variation in final accuracy (under k-fold validation, aka removing/adding the image causes a big difference) and prefer more of those.
Now, why not simply add everything? Well in general it takes too long to train.
For industrial application, there are already mature systems based on CV. For majority of those applications, there is no need for deep learning or multilayer CNN. Shocked to see Andrew Ng talking like a marketing guy.
What's the role of these tools? Can't a developer just write the code to get those data points?
At a first glance it seems like the hassle of integrating such a product into an existing ML codebase/pipeline is larger than solving the problem by hand.
> simply saying "we should make this better thing" isn't enough.
Besides the references to his company which has customers and a product that already works on these principles the literature currently shows that this is very much possible if you dig into the correct niches. Besides the SOTA in few-shot and meta-learning it is possible to smartly choose the correct few samples for the network that yield the same results.
It has also been my primary focus for the past 5 years and the core of the company I founded.
[+] [-] notsag-hn|4 years ago|reply
[+] [-] kevsim|4 years ago|reply
[+] [-] cloogshicer|4 years ago|reply
https://news.ycombinator.com/item?id=15251769
[+] [-] weego|4 years ago|reply
[+] [-] mdp2021|4 years ago|reply
Possibility which, by the way, makes the interviewer's cautionary move generally useless.
[+] [-] tromp|4 years ago|reply
[+] [-] whatever1|4 years ago|reply
The fact that he tries this in manufacturing makes the case stronger. In most manufacturing companies you do not have access to top ML talent.
You have Greg who knows python and recently visualized some production metrics.
If we could empower Greg with automated ML libraries that guide him in the data preparation steps in combination with precooked networks like autogluon, then manufacturing could become a huge beneficiary of the ML revolution.
[+] [-] overkalix|4 years ago|reply
[+] [-] GabeIsko|4 years ago|reply
[+] [-] itissid|4 years ago|reply
Big data is fairly important to a lot of things, for example I was listening to Tesla's use of Deep net models where they mentioned that there were literally so many variations of Stop Signs that they needed to learn what was really in the "tail" of the distribution of Stop Sign types to construct reliable AI
[+] [-] vasco|4 years ago|reply
[+] [-] IIAOPSW|4 years ago|reply
[+] [-] bennyg|4 years ago|reply
[+] [-] a_square_peg|4 years ago|reply
For instance, an English speaker and a non-English speaker may listen to someone speaking English and while the auditory signals received by both are the same, the meaning of the speech will only be perceived by the English speaker. When we’re learning a new language, it’s this ‘knowledge’ aspect that we’re enhancing in our brain, however that is encoded.
This knowledge part is what allows us to see what’s not there but should be (e.g. the curious incident of the dog in the night) and when the data is inconsistent (e.g. all the nuclear close calls). I’m really not sure how this ‘knowledge’ part will be approached by the AI community but feel like we’re already close to having squeezed out as much as we can from just the data side of things.
Somewhat related, we have a saying in Korean – ‘you see as much as you know’.
[+] [-] mdp2021|4 years ago|reply
It does in general, but what is elaborated and how? Structuring patterns is not the same as "knowledge" (there are missing subsystems), and that fed data is not fed efficiently, with ideal efficiency - compare with the realm in which "told one notion you acquire it" (this while CS is one of the disciplines focusing on optimization, so it would be a crucial point).
[+] [-] machiaweliczny|4 years ago|reply
Anyone knows if this might be true mathematically speaking? Does order of data matters?
[+] [-] Longwelwind|4 years ago|reply
[+] [-] aj7|4 years ago|reply
Did this make any of you a little queasy?
[+] [-] mdp2021|4 years ago|reply
[+] [-] a-dub|4 years ago|reply
it's easy to get complacent and focus on building big datasets. in practice, looking at the data often reveals issues sometimes in data quality and sometimes scope of what's in there (if you're missing key examples, it's simply not going to work).
most ml is actually data engineering.
[+] [-] atbpaca|4 years ago|reply
[+] [-] spupe|4 years ago|reply
[+] [-] DeathArrow|4 years ago|reply
I wonder if, assuming the data is of highest quality, with minimal noise, having more data will matter for training or not. And if it matters, on what degree?
[+] [-] frozenport|4 years ago|reply
In general you want to add more variants of data but not so much that the network doesn't get trained by them. Typical practice is to find images whose inclusion causes high variation in final accuracy (under k-fold validation, aka removing/adding the image causes a big difference) and prefer more of those.
Now, why not simply add everything? Well in general it takes too long to train.
[+] [-] kavalg|4 years ago|reply
[+] [-] xiphias2|4 years ago|reply
Of course few shot learning is important for models, but for example for Pathways it was already part of the evaluation.
[+] [-] unknown|4 years ago|reply
[deleted]
[+] [-] kappi|4 years ago|reply
[+] [-] leobg|4 years ago|reply
[+] [-] jstx1|4 years ago|reply
At a first glance it seems like the hassle of integrating such a product into an existing ML codebase/pipeline is larger than solving the problem by hand.
[+] [-] TOMDM|4 years ago|reply
I also want cars that run on salt water.
I'm not saying that small data ai is equally impossible, but simply saying "we should make this better thing" isn't enough.
[+] [-] Datenstrom|4 years ago|reply
Besides the references to his company which has customers and a product that already works on these principles the literature currently shows that this is very much possible if you dig into the correct niches. Besides the SOTA in few-shot and meta-learning it is possible to smartly choose the correct few samples for the network that yield the same results.
It has also been my primary focus for the past 5 years and the core of the company I founded.
[+] [-] sanxiyn|4 years ago|reply
Already in 2018 SenseTime reported that for face recognition, clean dataset surpasses accuracy of 4x larger raw dataset.
https://arxiv.org/abs/1807.11649
[+] [-] mdp2021|4 years ago|reply
Only, the article seemed to show a very conservative Ng about the algorithms, a focus on data management - so it's still ML.
[+] [-] technocratius|4 years ago|reply
[+] [-] atulsnj|4 years ago|reply
[+] [-] tacosbane|4 years ago|reply
[+] [-] girvo|4 years ago|reply