I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth!
I think this same thing is happening with ML. It was a hiring bonanza. Every big corp wanted to get an ML/AI strategy in place. They were forcing ML in to places it didn't (and may never) belong. This "recession" is mostly COVID related I think - but companies will discover that ML is (for the vast majority) a shiny object with no discernible ROI. Like Big Data, I think we'll see a few companies execute well and actually get some value, while most will just jump to the next shiny thing in a year or two.
"Like Big Data, I think we'll see a few companies execute well and actually get some value, while most will just jump to the next shiny thing in a year or two."
Here's another aspect - in many places nobody listens to the actual people doing the work. In my last job I was hired to lead a Data Science team and to help the company get value of Stats/ML/AI/DL/Buzzword. And I (and my team) were promptly overridden on every decision of what projects an expectations were realistic and what were not. I left, as did everybody else that reported to me, and we were replaced by people who would make really good BS slides that showed what upper management wanted to see. A year after that the whole initiative was cancelled.
Back in 2000 I was in a similar position with a small company jumping on the internet as their next business model. Lots of nonsense and one horrible web based business later, the company failed.
It's the same story over and over again. Some winners, lot of losers, many by self-inflicted wounds.
> they paid more to get those insights than they were worth!
This understates how awful ML is at many of these companies.
I've seen quite a few companies that rushed to hire teams of people with a PhD in anything that barely made it through a DS/ML boot camp.
To prove that they're super smart ML researchers without fail these hires rush to deploy a 3+ layer MLP to solve a problem that need at most a simple regression. They have no understanding of how this model works, and have zero engineering sense so they don't care if it's a nightmare of complexity to maintain. Then to make sure their work is 'valuable' management tries to get as many teams as possible to make use of the questionable outputs of these models.
The end is a nightmare of tightly coupled models that nobody can debug, trouble shoot or understand. And because the people building them don't really understand how they work the results are always very noisy. So you end up with this mess of expensive to build and run models talking noise to each other.
When I saw this I realized data science was doomed in the next recession, since the only solution to this mess is to just remove it all.
There is some really valuable DS work out there, but it requires real understanding of either modeling or statistics. That work will probably stick around, but these giant farms of boot camp grads churning out keras models will disappear soon.
I also witnesses this first hand at a Biotech company I worked at... we were using many variants of machine learning algorithms to develop predictive models of cell culture and separation processes. Problem is... the models have so many parameters in order to get a useful fit that the same model can also fit a carrot or an elephant.
We found that dynamic parameter estimation on ODE/DAE/PDE system models, while harder to develop, actually worked much better and gave us real insight into the processes.
So now my advice is others is "if you can start with some first principles equation or system of equations... start there and use optimization/regression to fit the model to the data."
AND: "if you don't think such equations exist for your problem... read/research more, because some useful equations probably do exist."
This is usually pretty straightforward for engineering and science applications... equations exist or can be derived for the system under study.
In my very limited exposure to other areas of machine learning application... I have found quite a bit of mathematical science related to marketing, human behavior, etc.
Ironically, I worked on a product that had a classic use case for machine learning during this time period and still had great difficulty getting results.
It was difficult to attract top ML talent no matter how much we offered. Everyone wanted to work for one of the big, recognizable names in the industry for the resume name recognition and a chance to pivot their way into a top role at a leading company later.
Meanwhile, we were flooded with applicants who exaggerated their ML knowledge and experience to an extreme,
hoping to land high paying ML jobs through hiring managers who couldn’t understand what they were looking for. It was easy to spot most of these candidates after going through some ML courses online and creating a very basic interview problem, but I could see many of these candidates successfully getting ML jobs at companies that didn’t know any better. Maybe they were going to fake it until they made it, or maybe they were counting on ML job performance being notoriously difficult to quantify on big data sets.
Dealing with 3rd party vendors and consulting shops wasn’t much better. A lot of the bigger shops were too busy with never ending lucrative contracts to take on new work. A lot of the smaller shops were too new to be able to show us much of a track record. Their proposals often boiled down to just implementing some famous open source solution on our product and letting us handle the training. Thanks, but we can do that ourselves.
I get the impression that it is (or was) more lucrative to start your own ML company and hope for an acquisition than to do the work for other companies. We tried to engage with several small ML vendors in our space and more than half of them came back with suggestions that we simply acquire them for large sums of money. Meanwhile, one of the vendors we engaged with was acquired by someone else and, of course, their support dried up completely.
Ultimately we found a solution from a vendor that had prepared a nice solution for our exact problem.the contracts were drawn up in a way that wouldn’t be too disastrous if (when?) they were acquired.
I have to wonder if an industry-wide slowdown to the ML frenzy is exactly what we need to give people and companies time to focus on solving real problems instead of just chasing easy money.
This is sadly so consistent with what I'm seeing at a big corporation. We are working so hard to make a centralized ML platform, get our data up to par, etc. but so many ML projects either have no chance of succeeding or have so little business value that they're not worth pursuing. Everyone on the development team for the project I'm working on is silently in agreement that our model would be better off being replaced by a well-managed rules engine, but every time we bring up these concerns, they're effectively disregarded.
There are obviously places in my company where ML is making an enormous impact, it's just not something that's fit for every single place where decisions need to be made. Sometimes doing some analysis to inform blunt rules works just as well - without the overhead of ML model management.
I don't agree, most of the low hanging fruit in ML engineering hasn't been picked yet. ML is like electricity 100 years ago, it will only expand and eat the world. And the research is not slowing down, on the contrary, it advances by leaps and bounds.
The problem is that we don't have enough ML engineers and many who go by this title are not really capable of doing the job. We're just coming into decent tools and hardware, and many applications are still limited by hardware which itself is being reinvented every 2 years.
Take just one single subfield - CV - it has applications in manufacturing, health, education, commerce, photography, agriculture, robotics, assisting blind persons, ... basically everywhere. It empowers new projects and amplifies automation.
With the advent of pre-trained neural nets every new task can be 10x or 100x easier. We don't need as many labels anymore, it works much better now.
I've seen similar patterns with clients and companies I've worked at as well. My experience was less that ML wasn't useful, it's just that no organization I worked with could really break down the silos in order for it to work. Especially in ML, the entire process from data collection to the final product and feedback loop needs to be integrated. This is _really_ difficult for most companies.
Many data scientists I knew were either sitting on their hands waiting for data or working on problems that the downstream teams had no intention of implementing (even if they were improvements). I still really believe that ML (be it fancy deep learning or just evidence driven rules-based models) will effectively be table stakes for most industries in the upcoming decade. However, it'll take more leadership than just hiring a bunch of smart folks out of a PhD program.
Curious if there is a correlation with companies that failed to capitalize with the ones who relied on consultants versus really reshaping their own people.
I worked for a financial services co that saw massive gains from big data/ML/AWS. Given, we were already using statistical models for everything, we just now could build more powerful features, more complex models, and move many things to more-real time, with more frequent retrains/deploys bc of cloud.
I do agree that companies who don't already recognize the value of their data and maybe rely on a consultant to tell them what to do might not be in the position to really capitalize on it and would just be throwing money after the shiny object. It really does take a huge overhaul sometimes. We retooled all of our job families from analysts/statisticians to data engineers and scientists and hired a ton of new people
Yeah the big data comparison is apt, and a few years ago was The Block-Chain that got middle managers frothing like Pavlov's dog.
It is clear that for most of the companies who are investing in deep learning are tangible results are always around the corner, and maybe 1 in 100 will build something worthwhile. But here is the carrot driving them all on, it's like the lottery: you have to be in to win. The stick is the fear that their competitors will do so.
This field is more art than science, give talented people incentive to play and don't expect too much for the next decade.
The problem I see is that in most non tech businesses they are not at the stage where they need ML, they are simply struggling with the basics: being able to seamlessly query or have consolidated up to date metrics and dashboards of the data scattered in all their databases. Of course the Big Data/AI “we’ll transform your data into insights” appealed to them, but that’s not what they need (also see the comments on the Palantir thread the other day).
That happened/is happening at my job. There's been a push to implement features that utilise AI/ML.
Not because it would be a good use case (although there are some for our product), or because it would be of any practical benefit, but because it makes for good marketing copy. Never mind the fact that nobody on the team has any experience with machine learning (I actually failed the paper at university).
Could it be also because for most companies after large investment in DS/ML/DL, they couldn't create a promising solution because they don't have as much access to the data/hardware/talent as Google/Amazon/MS does? And at the end of the day using just an API from the former gives better ROI?
(or) In simple terms, is profitable commercial Deep Learning just for oligarchies?
The company I work for is a large health care company. I started in robotic automation about a year ago. The company said its next three huge initiatives would be:
1) AI
2) Machine Learning
3) Robotic Process Automation
They felt RPA would help them stay more competitive since there are tons of smaller health care companies who are moving faster and innovating faster because they're not buying up companies and having to integrate all their technology at a sloth's pace. They thought RPA would be a way to mitigate these issues.
18 months later and the one manager, director and VP in my org has all but said they don't care about RPA, all their money is going into ML and AI. Even though in all the presentations I've seen them put on, its all blue skies and BS about "IF we had this, we COULD do this." Nothing concrete at all about how the plan to use ML to increase profit margins or reduce overhead.
Right now, our team is basically an afterthought in the company and I'm already starting to interview elsewhere with the knowledge at some point, they're going to kill my team and cut everybody loose.
Without ML, our business today is literally impossible (from a financial perspective).
I work in 2D animation and we were able to design our current pipeline around adopting ML at specific steps to remove massive amounts of manual labor.
I know this doesn't disprove your anecdote, I just wanted to point out that real businesses are using ML effectively to deliver real value that's not possible without it.
This has happened since the dawn of the computer age, and probably before.
Any technology too complex for the managers who purchase for it to understand fully can be sold and oversold by marketing people as "the next big thing".
Managers may or may not see through that, but if their superiors want them to pursue it or if they need to pursue something in order to show they're doing something of value, then they're happy to follow where the marketers lead.
Java everywhere, set top TV boxes, IOT devices, transitioning mainframes to minis, you name it... the marketers have made a mint selling it, usually for little benefit to the companies that bought into it.
ML is a shiny object with often no discernible ROI but occasionally very large ROI, and companies are understandably nervous about missing out. Spending a small amount to hedge their bets isn't necessarily irrational.
That's because it didn't get a chance to mature and to show how it could be powerful. People kept trying to force hadoop into it and call themselves "big data experts"
We've gotten a bit more clarity in this world with streaming technologies. However, there hasn't been a good and clear voice to say "hey .. this is how it fits in with your web app and this is what you expect of it". (I'm thinking about developing a talk on this.. how it fits in [hint.. your microservice app shouldn't do any heavy lifting of processing data])
People have been trying to used algorithms of various sorts to increase sales (actionable insights) forever. The buzzwords change, but the results are always the same. No permutation of CPU instructions will turn a product people don't want to pay for into a product people want to pay for.
The easiest way to solve many problems is through lexers, regular expressions and plain-old pattern matching. But that doesn't sell, so, they call it AI anyways.
When I hear ML, deep learning, etc, I consider it a red flag for exactly the reasons you state.
It's kind of batty actually, people looking for ideas to make money just been taking old ideas and attaching ML to the side of it as if that automatically made it better. And then not educating their customers on the limitations of ML both generally and with respect to their data size.
I personally think the companies that make and sell the software that the police used to make incorrect arrests should be legally liable. Yes, the police shouldn't have blindly trusted the software, but I guaran-fucking-tee you part of why they did is the marketing from the company themselves.
According to data from Revealera.com, if you normalize the data, the % of job openings that mention 'deep learning' has actually remained stable YoY: https://i.imgur.com/sDoKwD0.png
* Revealera.com crawls job openings from over 10,000 company websites and analyzes them for technology trends for hedge funds.
Yeah I had a suspicion that the trend shown in the chart in that thread regarding the decline of DL job posts largely resembles the trend of total job posts.
I think the fact the original tweet was not normalized in this incredibly obvious way is at least one valid reason companies could use less deep learning folk
Btw. I don't like twitter's new feature that prevents everyone from responding to a tweet that was used by @fchollet. It no longer feels like twitter if you can't engage.
Disingenuous framing of data or a laughably fundamental misreading of it? This is akin to trying to gain insight from a bunch of data on a map that simply has a strong correlation with population density.
Something I've learned: when non-engineers ask for an AI or ML implementation, they almost certainly don't understand the difference between that and an "algorithmic" solution.
If you solve "trending products" by building a SQL statement that e.g. selects items with the largest increase of purchases this month in comparison to the same month a year ago, that's still "AI" to them.
In the future, I expect ML to also fall out of the "AI" umbrella - it gets used primarily for "smart code we don't know• how to write", so once that understanding comes, it gets a more-specific name and is no longer "AI".
•"know" being intentionally vague here, as obviously we can write both query planners and ML engines, but the latter isn't nearly as commonplace yet to completely fall out of the umbrella.
Exactly! I run a data science department at a corporation. I've had exactly one production level project that was sufficiently complicated to require deep learning. I am currently working on the second. I always start with the simplest approach. That's a message that I push hard for every recent grad and intern who comes here.
Engineers tend to overestimate how difficult machine learning is. That is exactly how a good data scientist would solve this problem. If (and only if) this initial solution is not sufficient then you can iterate on it (maybe we should also take into account monthly trends, maybe one category of products is overrepresented, ...).
Most data-related problems, or extraction of knowledge from data, simply doesn't benefit from Deep Learning.
In my experience, what many organizations lack is simple but high-quality "Business Analytics": Reporting & dashboards are developed that look good but jam too much information together. It is often the wrong information:
Something is requested, and the developer develops exactly what was asked. The problem is that it wasn't what was needed because the person making the request couldn't articulate the question in the same terms the developer would understand. The request will say "Give me X & Y" when the real question is "I want to understand the impact of Y on X". The person gets X & Y, looks at it every day in their dashboard, and never sees much that is useful. The initial request should always be the start of a conversation, but that often doesn't happen. A common result are people in departments spending tons of time in Excel sorting, counting, making pivot tables, etc., when all of that could be automated.
This is part of the reason why companies often go looking for some new "silver bullet" to solve their data problems. They don't have the basics down, and don't understand the data problems well enough to seek out a solution.
I think we’re starting to see peak managerialism. The latest wave in stats has shown more than anything that a significant shortfall in basic statistics knowledge makes it almost impossible to make good decisions with vast amounts of data.
Without the skillsets to work with and then understand that data, they are forced into this long process of asking for data to be put into reporting and dashboards and then once they finally get them, either fixating on the limited metrics it provides while being oblivious to other context not in front of them, or to instead forced to start another long iteration to adjust that reporting and dashboards.
We’ve gone almost 30 years believing management was the sole skill required to manage teams and companies, but dealing with the new era of data is starting to show the limits
I observe the state of the art on most Nlp tasks since many years:
In 2018,2019 there was huge progress made each year on most tasks.
2020,except for a few tasks have mostly stagnated...
NLP accuracy is generally not production ready but the pace of progress was quick enough to have huge hopes.
The root cause of the evil is:
Nobody has build upon the state of the art pre trained language: XLnet while there are hundreds of declinaisons of BERTs. Just because of Google being behind it, if XLnet was owned by Google 2020 would have been different.
I also believe that pre trained language have reached a plateau and we need new original ideas such as bringing variational autoencoder to Nlp and using metaoptimizers such as Ranger.
The most pathetic one is that:
Many major Nlp tasks have old SOTA in BERT just because nobody cared of using (not improving) XLnet on them which is absolute shame, I mean on many major tasks we could trivially win many percents of accuracy but nobody qualified bothered to do it,where goes the money then? To many NIH papers I guess.
There's also not enough synergies, there are many interesting ideas that just needs to be combined and I think there's not enough funding for that, it's not exciting enough...
I pray for 2021 to be a better year for AI, otherwise it will show evidence for a new AI progress winter
I do not agree with this. I work heavily with NLP models for production in the Legal domain (where my baseline is where a 8GB 1080 must predict more than 1000 words/sec). This year was when our team glued enough pieces of Deep Learning to outperform our previous statistic/old ML pipeline that was been optimized for years.
Little things compound such as optimizers ( Ranger/Adahessian), better RNN ( IndRNN, Linear Transformers, Hopfield networks ) and techniques (cache everywhere, Torch script,gradient accumulation training)
Just to clarify one of your points regarding Google's involvement: XLnet, and the underlying TransformerXL technology, did have Google researchers involved:
My understanding is that a CMU student interned at Google and developed most of the pieces of TransformerXL, which formed the basis of XLNet. The student and the Google researcher further collaborated with CMU researchers to finalize the work.
(For the record, I think the remainder of your points do not match my understanding of NLP, which I do research in, but I just really wanted to clarify the XLNet story a bit).
It'd be ironic if your comment was generated by GPT-3. But forget GPT-3. In 10 years, looking back at AI history, the year 2020 will probably be viewed as the point separating pre GPT-4 and post GPT-4 epochs. GPT-4 is the model I expect to make things interesting again, not just in NLP, but in AI.
While this sounds plausible and has a lot of "prior" credibility coming from someone as central to deep learning as François Chollet, I'd love to see corroborating signal in actual job-posting data, from LinkedIn, Indeed, GlassDoor, etc. Backing up this kind of claim with data is especially important given the fact that the pandemic is disrupting all job sectors to varying degrees.
As you can imagine, searching Google for "linkedin job posting data" doesn't work so great. The closest supporting data I could find is this July report on the blog of a recruiting firm named Burtch Works [1]. They searched LinkedIn daily for data scientist job postings (so not specifically deep learning) and observed that the number of postings crashed between late March and early May to 40% of their March value, and have held steady up to mid-June, where the report data period ends.
There's also this Glassdoor Economic Research report [2], which seems to draw heavily from US Bureau of Labor Statistics data available in interactive charts [3]. The most relevant bit in there is that the "information" sector (which includes their definitions of "tech" and "media") has not yet started an upward recovery in job postings, as of July.
I feel like it was also a classic case of running before we could crawl. Jumping from A to Z before we could go from 0 to 1.
I work at an Residential IoT company, there are quite a few really valid use cases for Big Data and even ML. (Think about predictive failure).
We hired more than one expensive data scientist in the past few years, and had big strategies more than once. But at the end of the day it's still "hard" to ask a question such as "if I give you a MAC Address give me the runtime for the last 6 months".
We're trying to shoot for the moon, when all I've ever asked is I want an API to show me indoor temp for particular device over a long period.
This is absolutely right. And when you think about it, the reason behind has been staring us in the face: people who want to do machine learning approach everything as a machine learning problem. It's really common to see people handwave away the "easy stuff" because they want to get credit for doing the "hard stuff."
It's not just the data scientists fault. I once heard our chief data scientist point out that they don't want to hand off a linear regression as a machine learning model -- as if a delivered solution to a problem has a minimal complexity. She absolutely had a point.
Clients are paying for a Ph.D. to solve problems in a Ph.D way. If we delivered the client a simple, yet effective solution, there's the risk of blow-back from the client for being too rudimentary. I'm certain this extends attitude extends to in-house data scientists as well. Nobody wants to be the data "scientist" who delivers the work of a data "analyst." Even when the best solution is a simple SQL query.
Our company kind of sidesteps this problem by having a tiered approach, where companies are paying for engineering, analysis, visualization, and data science work for all projects. So if a client is at the simple analysis level, we deliver at that level, with the understanding that this is the foundational work for more advanced features. It turns out to be a winning strategy, because while every client wants to land on the moon, most of them figure out that they are perfectly happy to with a Cessna once they have one.
Everyone wants to fire up Tensorflow, Keras and PyTorch these days. Fewer people want to work in Airflow and SSIS, spend days tuning ETL, etc. This is the domain of data engineering, which bridges software engineering and data science with a dash of devops. I’ve been working in this field for a couple of years and it’s clear to me that data engineering is a necessary foundation and impact multiplier for data science.
My impression too. I earn my money turning your mess into a data "landscape" - I saw people wanting to jump on the ML wagon, who did not even heard of version control for code before. Not a winter, no, but a long bumpy road ahead.
Meh, only for people who bought into the hype without real use cases. Which I agree may be numerous.
In my company though, we've been applying DL with great success for a few years now, and there are at least five years of work remaining. And that's not spending any time doing research or anything fancy: just picking the low-hanging fruit.
I think many companies have real problems, but find that DL ends up being a poor solution in practice for various reasons.
You need not only real use cases, but use cases that happens to well with DL’s trade offs and limitations. I think many companies hired with very unrealistic expectations here.
I managing some teams right now that do a mix of high-end ML stuff with more prosaic solutions. The ML team is smart, and pretty fast with what they do, but they tend to (as many comments here have mentioned) focus on delivering only PhD level work. This translates into taking simple problems and trying to deorbit the ISS through a wormhole on it rather than just getting something in place that answers the problem.
In conjunction with this, it turns out 99% of the problems the customer is facing, despite their belief to the contrary, aren't solved best with ML, but with good old fashioned engineering.
In cases where the problem can be approached either way, the ML approach typically takes much longer, is much harder to accomplish, has more engineering challenges to get it into production, and the early ramp-up stages around data collecting, cleaning and labeling are often almost impossible to surmount.
All that being said, there are some things that are only really solvable with some ML techniques, and that's where the discipline shines.
One final challenge is that a lot of data scientists and ML people seem to think that if it's not being solved using a standard ML or DL algorithm then it isn't ML, even if it has all of the characteristics of being one. The gatekeeping in the field is horrendous and I suspect it comes from people who don't have strong CS backgrounds wrapping themselves too tightly against their hard-earned knowledge rather than having an expansive view of what can solve these problems.
Get your math and your domain knowledge straight and you can do a lot with little.
Lots of programmers want to be ml engineers because the prestige is higher because you normally take in PhDs.
The big problem is hype, people are throwing AI at everything as...garbage marketing. It’s at the point where if you say you use AI in your software title, I know you suck, because you aren’t focusing on solving a problem you are focusing on being cool which will never end well.
There's a lot of what I call "model fetishism" in machine learning.
Instead of focusing our energies on the infrastructure and quality of data around machine learning, there's eagerness to take bad data to very high-end models. I've seen it again and again at different companies, usually always with disastrous consequences.
A lot of these companies would do better to invest in engineering and domain expertise around the problem than worry about the type of model they're using to solve the problem (which usually comes later, once the other supporting maturity pieces are in place)
This is why my interview question focuses around applying linear regression to a complex domain. It weeds out an enormous number of candidates.
There are 5 ML models that we maintain where I work, and none of then are more complicated than linear regression or random forests. Convincing me to use something more complex would take an enormous amount of evidence. Domain knowledge is king.
Yes! I feel this quite a lot, I've just finished my degree. I remember reading quite a few papers for my thesis where there is little discussion of the actual data that is used, what might be graspable from the data with basic DS techniques such as PCA, clustering and such. Instead, it goes right to the model and default evaluation methods, just a table of numbers.
We did have courses explaining the "around" of the whole process though, but that's not as hyped.
This is an anecdote with no data. And the entire global economy is in a recession, so the fact deep learning might have fewer job postings isn't particular notable.
I'll note that in my personal anecdote, the megacorps remain interested in and hiring in ML as much as ever.
This agrees with what I see, but megacorps and in general many large organizations are often slow to move both in and out. They can take years to stop building up experience in areas that changed from being a new promising technology to mature fields to oversold fads. They also have a lot of money help weather many overpriced hires. So I am not sure that megacorps hiring is a very strong counter-argument. Just my 2c.
However, megacorps do not seem to suffer much for such continuous lagging in hiring. I do not know why this is so: is it that they still hire smart engineers who can easily change groups and fields or do they work on their core technology to help build the next peak (after the debris are washed away in a fad crash there is often a technology renaissance).
Missing in the original chart/data: have ML/DL job postings decrease more or less than other comparable job categories (programming, business analyst, etc.)
I'm completely unsurprised by this. Regularly, at lunch, I'll ask my coworkers if they know of any DL applications that are making O($billions), and no one knows any outside of FAANG.
FAANG is making an insane amount of money due to DL. Outside of them though, I don't know who's making money here. When I was interviewing for jobs, there were a ton of startups that were trying to do things with DL that would have been better done with a few if statements and a random forest, and that had a total market size in the millions.
I think that, eventually, there'll be a market for this stuff, but I'm not convinced that it's anywhere near being widespread.
I was also a consultant before my current role. The vast majority of non-tech firms don't have their data in well organized + cleaned databases. Just moving from a mess of Excel sheets to Python scripts + SQL databases would have made a HUGE difference to the vast majority of clients I worked with, but even that was too big of a transformation.
Basically, everyone with the sophistication to take advantage of DL/ML already has the in-house expertise to do it. There's almost no one in the intersection of "Could make $$$ doing DL" && "Has the technical infrastructure to integrate DL".
99% of the time you don't need a deep recurrent neural network with an attention based transformer. Most times, you just need a bare-bones logistic regression with some carefully cleansed data and thoughtful, domain-aware feature engineering.
Yes, you're not going to achieve state-of-the-art performance with logistic regression. But for most problems the difference between SOTA and even simple models is not nearly as large as you might think. And two, even if you're cargo-culting SOTA techniques, it's probably not going to work unless you're at an org with an 8-digit R&D budget.
I know very little about the DL/ML space, but as a full-stack engineer it feels like most companies have tried to replicate what FAANG companies do (heavy investment in data/ml) when the cost/benefit simply isn't there.
Small companies need to frame the problem as:
1) Do we have a problem where the solution is discrete and already solved by an existing ML/DL model/architecture?
2) Can we have one of our existing engineers (or a short-term contractor) do transfer learning to slightly tweak that model to our specific problem/data?
Once that "problem" actually turns into multiple "machine learning problems" or "oh, we just need todo this one novel thing", they will probably need to bail because it'll be too hard/expensive and the most likely outcome will be no meaningful progress.
Said in another way: can we expect an engineer to get a fastai model up and running very quickly for our problem? If so, great - if not, then bail.
ie: the solution for most companies will be having 1 part-time "citizen data scientist" [1] on your engineering team.
The way I see it, only those companies that had already been using a data oriented approach to business can really reap the benefits of ML. From a company's point of view, ML/AI should be a natural evolution of an existing tool set to better solve problems they have been trying to solve in the past using deterministic methods and then statistical methods, etc. Any other project that is diving right into ML is likely to fail because
1. There's no clear problem statement. They have never formulated one and now trying to bolt ML on to their decision making.
2. They don't have well catalogued data for engineers/scientists to work with because they never tried to do rigorous analysis of data before ML became a thing.
3. Managers have no idea how to deal with data driven insights. What if the results are completely unintuitive to them? Are they going to change their processes abruptly? What if the results are aligned with what they have been always doing? Is it worth paying for something that they have been doing intuitively for decades?
I'm not a data scientist. But the biggest complaint I hear from my colleagues is that they lack data to train models.
Isn't it the same pattern every 10 years or so for "AI" related tech ? Some people hype tech X as being a game changer - tech X is way less amazing than advertised - investors bail out - tech X dies - rinse and repeat.
This is more akin to the Internet bubble than the previous AI winter. The technology is valuable for business, but the hype is huge and companies aren't ready for it yet.
Companies trying to add machine learning to everything they do like if that's going to solve all their problems or unlock new revenue streams.
80 or 90% of what companies are doing with machine learning results in systems with a high computing cost that are clearly unprofitable if seen as revenue impacting units. Many similar things can be achieved with low-level heuristics that result in way smaller computing costs.
But nobody wants to do that anymore. There's nothing "sexy" or "cool" about breaking down your problems and trying to create rule-based systems that addresses the problem. Semantic software is not cool anymore, and what became cool is this super expensive blackbox that requires more computer power than regular software. Companies have developed this bias for ML solutions because they seem to have this unlimited potential for solving problems, so it seems like a good long term investment. Everyone wants to take that bus.
Don't get me wrong. I love ML, but people use it for the stupidest things.
That may be true in the research arena (where Mr Chollet works), but I don't think that's the case in terms of where deep learning is actually applied in industry, nor will it be the case for years to come IMO.
It's just that much that needed to be invented has been invented and now it's time to apply it everywhere it can be applied, which is a great many place.
I've been using voice commands on my android phone, in situations where I can't use my hands. Most often all I want to do is.
1. Start and stop a podcast.
2. Play music
3. Ask for the time
The phone understands me, but then android breaks the flow, so I have to use my hands.
1. It will ask me to unlock the phone first? I have gloves and a mask on. It won't recognize my face, and my gloves don't register touches. Why do I have to unclock the phone to play music in the first place.
2. It gets confused on which app to play the music/podcast on. Wants to open youtube app, or spotify, and so on ...
3. Not consistent. I can say the same thing, and sometimes it will do one things, and another next time.
4. If I'm playing a video, and I want to show it full screen. I have to maximize and touch the screen. Why can't it play full screen be default.
I have similar with my Google Home; it can play Netflix but can't work out BBC iPlayer most of the time. And many times if I ask it to play music, it'll give an error saying it can't play on YouTube Music because I don't have a subscription, even though my default music player is Spotify in my account.
I'm currently a masters student and I'm rather glad I opted not to take a specialized degree such as Machine Learning, taking on computer science instead. All this discussion about DS, ML, AI (and even CS) becoming over-saturated has made me rather wary and I worry that I'm choosing the wrong 'tracks' to study (currently doing ML and Cybersecurity as I genuinely am interested in those fields). I won't be graduating until next year but I'm forcing myself to be optimistic that the tech job market will be in a better place by then.
It's not exactly a great year for extrapolating trends about what people are doing with their time. I wonder how much of this is 2020-specific and not just due to the natural cycle of AI winters.
Finally! Big companies need to realize they must understand what what they are doing with technology to get any value of out it.
They've long resisted that, of course, but I'm pretty sure half the popular of deep learning was it leveled the playing field, making engineers as ignorant of the inner-workings of their creations as the middle managers.
May the middle-manager-fication of work, and acceptance of ignorance that goes with, fail.
-----
Then again, I do prefer it when many of those old moronic companies flounder, so maybe this is a bad thing that they're wising up.
I would have expected a comparison to job postings in general: how do deep learning job postings compare to job postings for any kind of technical position?
Meanwhile, the academic job market, certainly in my area, ie linguistics/computational linguistics, has collapsed, too. A colleague did a similar and equally nice analysis here: https://twitter.com/ruipchaves/status/1279075251025043457
Data science and ML In big companies are pulling resources away from the real value add activities like proper data integrity, blending sources, improving speed performance. Yes Business Intelligence is not cool anymore. Yes I also call my team “data analytics”. But let’s not forget the simple fact that “data driven” means we give people insights when and where they need them. Insights could be coming from an sql group by, ML, AI, watching the flying of birds, but they are still simply a data point for some human to make a decision. That means we need to produce the insight, being able to communicate it to people, have the the credibility for said people to actually listen to what we are saying. Focusing on how we put that data point together is irrelevant, focusing on hiring PHDs to do ML is most likely going to end in a failure because PHDs are not predictive of great analytical skills, experience and things like sql are much better predictors.
On the plus side, ML systems have become commoditized to the point that any reasonably skilled software engineer can do the integration. From there, it really comes down to understanding the product domain inside and out.
I have seen so many more projects derailed by a lack of domain knowledge than I have seen for lack of technical understanding in algorithms.
Every company of course is very different, but I have seen that companies understood that fro Deep Learning you need a Pytorch or TF expert or maybe some other framework and most of these experts already work in Google/Facebook or any other advanced companies (NVIDIA, Microsoft, Cruise, etc), hiring is very difficult and cost is high. Then you can start using regular SQL and/or AutoML to get some insights. For a large number of companies that's enough. When there is so much complexity, such as DL modeling there's little transparency and management want to understand things. After COViD time will tell, but my take is that only a few companies need DL.
In general does anyone know if its a good time to look for a new dev job? I was really going to move this year, but it seems sensible to wait. Just sucks to see friends with RSUs going up in value so quickly.
No harm in having a recruiter or two feed you opportunities on a regular basis to interview at (just be up front with them that you're holding out for a solid fit for your criteria). Better to have a job while interviewing than be under pressure to accept the first half decent thing that comes along.
For most companies ML is just part of the long term strategy, with covid priorities have shifted from long term R&D to short term survival, so I don't see anything out of the ordinary here
A lot of thee c folks aren't tech folks or even math folks. They want to try to use deep learning to do prediction or get some insight when something as simple as regression would have worked.
what's particularly surprised me is how effective gradient boosting is in practise. I've seen so many cases of real world applications where just using catboost or whatever worked ~95% as well or even just as well as some super complicated deep learning approach and it saves you ten times the cost
Deep Learning has been so commoditized and compartmentize over the past 5 years, now I think average SDE with some basic understanding of it can do a reasonable job in application.
I don't think anyone should freak out when they see a tweet like this: deep learning is just one particularly trendy part of ML, which is just one piece of data science, which is just one job title in the "working with data" career space. I think that most people with backgrounds or interests in DL are very well equipped to participate in the (ever more important) data science world.
No question ML is powerful and can do great things. Also no question a lot of companies where just throwing money at stuff for fear of being seen as behind in this space. When the going gets tough such vanity efforts are the first things to go.
Teams adding measurable value for their companies should be fine but others might not be.
In my industry (research), we still have a strong line of business. Some commercial clients have killed their contracts with us to save money during the COVID era, but government contracts are still going strong. In areas where there's a clear use case I think there is still work to go around.
Good riddance. The majority of it is snakeoil, relabeling and "smoke and mirrors". A lot of smart or lucky people made a lot of money, a lot of dumb people with power over money lost... probably insignificant amounts of it.
ML/DL is at the exploratory phase for most companies. I have no surprise when seeing this post. Nevertheless, this also open new opportunities in other domains and new kind of business based on data. I have no doubt.
Is there a LinkedIn tool that allows you to make similar trend plots as shown in the Twitter thread, or has the author been archiving the data over time?
He labels anyone who criticizes him as a troll. Unfortunately he is a public figure in the ML space and does have his share of trolls, but doesn't take too well to even well thought out replies.
This needs to be normalized to “job posting collapse in the past 6 months” unless you expect DL jobs to grow while everything shrinks? I’m somewhat surprised by the analysis from someone’s who’s “data driven.” I mean, he even says so as much in the twitter thread:
“To be clear, I think this is an economic recession indicator, not the start of a new AI winter.”
So, looks like he discovered an economic recession.
eric_b|5 years ago
I think this same thing is happening with ML. It was a hiring bonanza. Every big corp wanted to get an ML/AI strategy in place. They were forcing ML in to places it didn't (and may never) belong. This "recession" is mostly COVID related I think - but companies will discover that ML is (for the vast majority) a shiny object with no discernible ROI. Like Big Data, I think we'll see a few companies execute well and actually get some value, while most will just jump to the next shiny thing in a year or two.
apohn|5 years ago
Here's another aspect - in many places nobody listens to the actual people doing the work. In my last job I was hired to lead a Data Science team and to help the company get value of Stats/ML/AI/DL/Buzzword. And I (and my team) were promptly overridden on every decision of what projects an expectations were realistic and what were not. I left, as did everybody else that reported to me, and we were replaced by people who would make really good BS slides that showed what upper management wanted to see. A year after that the whole initiative was cancelled.
Back in 2000 I was in a similar position with a small company jumping on the internet as their next business model. Lots of nonsense and one horrible web based business later, the company failed.
It's the same story over and over again. Some winners, lot of losers, many by self-inflicted wounds.
baron_harkonnen|5 years ago
This understates how awful ML is at many of these companies.
I've seen quite a few companies that rushed to hire teams of people with a PhD in anything that barely made it through a DS/ML boot camp.
To prove that they're super smart ML researchers without fail these hires rush to deploy a 3+ layer MLP to solve a problem that need at most a simple regression. They have no understanding of how this model works, and have zero engineering sense so they don't care if it's a nightmare of complexity to maintain. Then to make sure their work is 'valuable' management tries to get as many teams as possible to make use of the questionable outputs of these models.
The end is a nightmare of tightly coupled models that nobody can debug, trouble shoot or understand. And because the people building them don't really understand how they work the results are always very noisy. So you end up with this mess of expensive to build and run models talking noise to each other.
When I saw this I realized data science was doomed in the next recession, since the only solution to this mess is to just remove it all.
There is some really valuable DS work out there, but it requires real understanding of either modeling or statistics. That work will probably stick around, but these giant farms of boot camp grads churning out keras models will disappear soon.
cashsterling|5 years ago
So now my advice is others is "if you can start with some first principles equation or system of equations... start there and use optimization/regression to fit the model to the data."
AND: "if you don't think such equations exist for your problem... read/research more, because some useful equations probably do exist."
This is usually pretty straightforward for engineering and science applications... equations exist or can be derived for the system under study.
In my very limited exposure to other areas of machine learning application... I have found quite a bit of mathematical science related to marketing, human behavior, etc.
PragmaticPulp|5 years ago
It was difficult to attract top ML talent no matter how much we offered. Everyone wanted to work for one of the big, recognizable names in the industry for the resume name recognition and a chance to pivot their way into a top role at a leading company later.
Meanwhile, we were flooded with applicants who exaggerated their ML knowledge and experience to an extreme, hoping to land high paying ML jobs through hiring managers who couldn’t understand what they were looking for. It was easy to spot most of these candidates after going through some ML courses online and creating a very basic interview problem, but I could see many of these candidates successfully getting ML jobs at companies that didn’t know any better. Maybe they were going to fake it until they made it, or maybe they were counting on ML job performance being notoriously difficult to quantify on big data sets.
Dealing with 3rd party vendors and consulting shops wasn’t much better. A lot of the bigger shops were too busy with never ending lucrative contracts to take on new work. A lot of the smaller shops were too new to be able to show us much of a track record. Their proposals often boiled down to just implementing some famous open source solution on our product and letting us handle the training. Thanks, but we can do that ourselves.
I get the impression that it is (or was) more lucrative to start your own ML company and hope for an acquisition than to do the work for other companies. We tried to engage with several small ML vendors in our space and more than half of them came back with suggestions that we simply acquire them for large sums of money. Meanwhile, one of the vendors we engaged with was acquired by someone else and, of course, their support dried up completely.
Ultimately we found a solution from a vendor that had prepared a nice solution for our exact problem.the contracts were drawn up in a way that wouldn’t be too disastrous if (when?) they were acquired.
I have to wonder if an industry-wide slowdown to the ML frenzy is exactly what we need to give people and companies time to focus on solving real problems instead of just chasing easy money.
plants|5 years ago
There are obviously places in my company where ML is making an enormous impact, it's just not something that's fit for every single place where decisions need to be made. Sometimes doing some analysis to inform blunt rules works just as well - without the overhead of ML model management.
visarga|5 years ago
The problem is that we don't have enough ML engineers and many who go by this title are not really capable of doing the job. We're just coming into decent tools and hardware, and many applications are still limited by hardware which itself is being reinvented every 2 years.
Take just one single subfield - CV - it has applications in manufacturing, health, education, commerce, photography, agriculture, robotics, assisting blind persons, ... basically everywhere. It empowers new projects and amplifies automation.
With the advent of pre-trained neural nets every new task can be 10x or 100x easier. We don't need as many labels anymore, it works much better now.
twelfthnight|5 years ago
Many data scientists I knew were either sitting on their hands waiting for data or working on problems that the downstream teams had no intention of implementing (even if they were improvements). I still really believe that ML (be it fancy deep learning or just evidence driven rules-based models) will effectively be table stakes for most industries in the upcoming decade. However, it'll take more leadership than just hiring a bunch of smart folks out of a PhD program.
blaird|5 years ago
I worked for a financial services co that saw massive gains from big data/ML/AWS. Given, we were already using statistical models for everything, we just now could build more powerful features, more complex models, and move many things to more-real time, with more frequent retrains/deploys bc of cloud.
I do agree that companies who don't already recognize the value of their data and maybe rely on a consultant to tell them what to do might not be in the position to really capitalize on it and would just be throwing money after the shiny object. It really does take a huge overhaul sometimes. We retooled all of our job families from analysts/statisticians to data engineers and scientists and hired a ton of new people
OscarTheGrinch|5 years ago
It is clear that for most of the companies who are investing in deep learning are tangible results are always around the corner, and maybe 1 in 100 will build something worthwhile. But here is the carrot driving them all on, it's like the lottery: you have to be in to win. The stick is the fear that their competitors will do so.
This field is more art than science, give talented people incentive to play and don't expect too much for the next decade.
tarsinge|5 years ago
ellisv|5 years ago
> They were forcing ML in to places it didn't (and may never) belong.
I find that I spend a lot of time as a senior MLE telling someone why they don’t need ML
unknown|5 years ago
[deleted]
toomanybeersies|5 years ago
Not because it would be a good use case (although there are some for our product), or because it would be of any practical benefit, but because it makes for good marketing copy. Never mind the fact that nobody on the team has any experience with machine learning (I actually failed the paper at university).
Abishek_Muthian|5 years ago
(or) In simple terms, is profitable commercial Deep Learning just for oligarchies?
at-fates-hands|5 years ago
1) AI 2) Machine Learning 3) Robotic Process Automation
They felt RPA would help them stay more competitive since there are tons of smaller health care companies who are moving faster and innovating faster because they're not buying up companies and having to integrate all their technology at a sloth's pace. They thought RPA would be a way to mitigate these issues.
18 months later and the one manager, director and VP in my org has all but said they don't care about RPA, all their money is going into ML and AI. Even though in all the presentations I've seen them put on, its all blue skies and BS about "IF we had this, we COULD do this." Nothing concrete at all about how the plan to use ML to increase profit margins or reduce overhead.
Right now, our team is basically an afterthought in the company and I'm already starting to interview elsewhere with the knowledge at some point, they're going to kill my team and cut everybody loose.
erichocean|5 years ago
I work in 2D animation and we were able to design our current pipeline around adopting ML at specific steps to remove massive amounts of manual labor.
I know this doesn't disprove your anecdote, I just wanted to point out that real businesses are using ML effectively to deliver real value that's not possible without it.
Accujack|5 years ago
Any technology too complex for the managers who purchase for it to understand fully can be sold and oversold by marketing people as "the next big thing".
Managers may or may not see through that, but if their superiors want them to pursue it or if they need to pursue something in order to show they're doing something of value, then they're happy to follow where the marketers lead.
Java everywhere, set top TV boxes, IOT devices, transitioning mainframes to minis, you name it... the marketers have made a mint selling it, usually for little benefit to the companies that bought into it.
insomniacity|5 years ago
However someone is trying to make robotic process automation the Next Big Thing - which I think is hysterically funny.
Lxr|5 years ago
monksy|5 years ago
That's because it didn't get a chance to mature and to show how it could be powerful. People kept trying to force hadoop into it and call themselves "big data experts"
We've gotten a bit more clarity in this world with streaming technologies. However, there hasn't been a good and clear voice to say "hey .. this is how it fits in with your web app and this is what you expect of it". (I'm thinking about developing a talk on this.. how it fits in [hint.. your microservice app shouldn't do any heavy lifting of processing data])
jacobsenscott|5 years ago
cnst|5 years ago
The easiest way to solve many problems is through lexers, regular expressions and plain-old pattern matching. But that doesn't sell, so, they call it AI anyways.
bishalb|5 years ago
RandoHolmes|5 years ago
It's kind of batty actually, people looking for ideas to make money just been taking old ideas and attaching ML to the side of it as if that automatically made it better. And then not educating their customers on the limitations of ML both generally and with respect to their data size.
I personally think the companies that make and sell the software that the police used to make incorrect arrests should be legally liable. Yes, the police shouldn't have blindly trusted the software, but I guaran-fucking-tee you part of why they did is the marketing from the company themselves.
AznHisoka|5 years ago
* Revealera.com crawls job openings from over 10,000 company websites and analyzes them for technology trends for hedge funds.
dsiegel2275|5 years ago
james412|5 years ago
Tepix|5 years ago
Btw. I don't like twitter's new feature that prevents everyone from responding to a tweet that was used by @fchollet. It no longer feels like twitter if you can't engage.
datameta|5 years ago
cnst|5 years ago
simonw|5 years ago
If you solve "trending products" by building a SQL statement that e.g. selects items with the largest increase of purchases this month in comparison to the same month a year ago, that's still "AI" to them.
Knowing this can save you a lot of wasted time.
jon_richards|5 years ago
Izkata|5 years ago
In the future, I expect ML to also fall out of the "AI" umbrella - it gets used primarily for "smart code we don't know• how to write", so once that understanding comes, it gets a more-specific name and is no longer "AI".
•"know" being intentionally vague here, as obviously we can write both query planners and ML engines, but the latter isn't nearly as commonplace yet to completely fall out of the umbrella.
Breza|5 years ago
ma2rten|5 years ago
unknown|5 years ago
[deleted]
ellis-bell|5 years ago
ineedasername|5 years ago
In my experience, what many organizations lack is simple but high-quality "Business Analytics": Reporting & dashboards are developed that look good but jam too much information together. It is often the wrong information:
Something is requested, and the developer develops exactly what was asked. The problem is that it wasn't what was needed because the person making the request couldn't articulate the question in the same terms the developer would understand. The request will say "Give me X & Y" when the real question is "I want to understand the impact of Y on X". The person gets X & Y, looks at it every day in their dashboard, and never sees much that is useful. The initial request should always be the start of a conversation, but that often doesn't happen. A common result are people in departments spending tons of time in Excel sorting, counting, making pivot tables, etc., when all of that could be automated.
This is part of the reason why companies often go looking for some new "silver bullet" to solve their data problems. They don't have the basics down, and don't understand the data problems well enough to seek out a solution.
momokoko|5 years ago
Without the skillsets to work with and then understand that data, they are forced into this long process of asking for data to be put into reporting and dashboards and then once they finally get them, either fixating on the limited metrics it provides while being oblivious to other context not in front of them, or to instead forced to start another long iteration to adjust that reporting and dashboards.
We’ve gone almost 30 years believing management was the sole skill required to manage teams and companies, but dealing with the new era of data is starting to show the limits
The_rationalist|5 years ago
The most pathetic one is that: Many major Nlp tasks have old SOTA in BERT just because nobody cared of using (not improving) XLnet on them which is absolute shame, I mean on many major tasks we could trivially win many percents of accuracy but nobody qualified bothered to do it,where goes the money then? To many NIH papers I guess.
There's also not enough synergies, there are many interesting ideas that just needs to be combined and I think there's not enough funding for that, it's not exciting enough...
I pray for 2021 to be a better year for AI, otherwise it will show evidence for a new AI progress winter
bratao|5 years ago
Little things compound such as optimizers ( Ranger/Adahessian), better RNN ( IndRNN, Linear Transformers, Hopfield networks ) and techniques (cache everywhere, Torch script,gradient accumulation training)
liviosoares|5 years ago
* https://ai.googleblog.com/2019/01/transformer-xl-unleashing-...
* https://arxiv.org/pdf/1901.02860.pdf
* https://arxiv.org/pdf/1906.08237.pdf
My understanding is that a CMU student interned at Google and developed most of the pieces of TransformerXL, which formed the basis of XLNet. The student and the Google researcher further collaborated with CMU researchers to finalize the work.
(For the record, I think the remainder of your points do not match my understanding of NLP, which I do research in, but I just really wanted to clarify the XLNet story a bit).
lacker|5 years ago
dpflan|5 years ago
p1esk|5 years ago
EForEndeavour|5 years ago
As you can imagine, searching Google for "linkedin job posting data" doesn't work so great. The closest supporting data I could find is this July report on the blog of a recruiting firm named Burtch Works [1]. They searched LinkedIn daily for data scientist job postings (so not specifically deep learning) and observed that the number of postings crashed between late March and early May to 40% of their March value, and have held steady up to mid-June, where the report data period ends.
There's also this Glassdoor Economic Research report [2], which seems to draw heavily from US Bureau of Labor Statistics data available in interactive charts [3]. The most relevant bit in there is that the "information" sector (which includes their definitions of "tech" and "media") has not yet started an upward recovery in job postings, as of July.
[1] https://www.burtchworks.com/2020/06/16/linkedin-data-scienti...
[2] https://www.glassdoor.com/research/july-2020-bls-jobs-report...
[3] https://www.bls.gov/charts/employment-situation/employment-l...
deepGem|5 years ago
supergeek133|5 years ago
I work at an Residential IoT company, there are quite a few really valid use cases for Big Data and even ML. (Think about predictive failure).
We hired more than one expensive data scientist in the past few years, and had big strategies more than once. But at the end of the day it's still "hard" to ask a question such as "if I give you a MAC Address give me the runtime for the last 6 months".
We're trying to shoot for the moon, when all I've ever asked is I want an API to show me indoor temp for particular device over a long period.
mywittyname|5 years ago
It's not just the data scientists fault. I once heard our chief data scientist point out that they don't want to hand off a linear regression as a machine learning model -- as if a delivered solution to a problem has a minimal complexity. She absolutely had a point.
Clients are paying for a Ph.D. to solve problems in a Ph.D way. If we delivered the client a simple, yet effective solution, there's the risk of blow-back from the client for being too rudimentary. I'm certain this extends attitude extends to in-house data scientists as well. Nobody wants to be the data "scientist" who delivers the work of a data "analyst." Even when the best solution is a simple SQL query.
Our company kind of sidesteps this problem by having a tiered approach, where companies are paying for engineering, analysis, visualization, and data science work for all projects. So if a client is at the simple analysis level, we deliver at that level, with the understanding that this is the foundational work for more advanced features. It turns out to be a winning strategy, because while every client wants to land on the moon, most of them figure out that they are perfectly happy to with a Cessna once they have one.
pbourke|5 years ago
throwaway7281|5 years ago
joelthelion|5 years ago
In my company though, we've been applying DL with great success for a few years now, and there are at least five years of work remaining. And that's not spending any time doing research or anything fancy: just picking the low-hanging fruit.
freyr|5 years ago
You need not only real use cases, but use cases that happens to well with DL’s trade offs and limitations. I think many companies hired with very unrealistic expectations here.
abrichr|5 years ago
bane|5 years ago
In conjunction with this, it turns out 99% of the problems the customer is facing, despite their belief to the contrary, aren't solved best with ML, but with good old fashioned engineering.
In cases where the problem can be approached either way, the ML approach typically takes much longer, is much harder to accomplish, has more engineering challenges to get it into production, and the early ramp-up stages around data collecting, cleaning and labeling are often almost impossible to surmount.
All that being said, there are some things that are only really solvable with some ML techniques, and that's where the discipline shines.
One final challenge is that a lot of data scientists and ML people seem to think that if it's not being solved using a standard ML or DL algorithm then it isn't ML, even if it has all of the characteristics of being one. The gatekeeping in the field is horrendous and I suspect it comes from people who don't have strong CS backgrounds wrapping themselves too tightly against their hard-earned knowledge rather than having an expansive view of what can solve these problems.
danielscrubs|5 years ago
softwaredoug|5 years ago
Instead of focusing our energies on the infrastructure and quality of data around machine learning, there's eagerness to take bad data to very high-end models. I've seen it again and again at different companies, usually always with disastrous consequences.
A lot of these companies would do better to invest in engineering and domain expertise around the problem than worry about the type of model they're using to solve the problem (which usually comes later, once the other supporting maturity pieces are in place)
actusual|5 years ago
There are 5 ML models that we maintain where I work, and none of then are more complicated than linear regression or random forests. Convincing me to use something more complex would take an enormous amount of evidence. Domain knowledge is king.
fhennig|5 years ago
We did have courses explaining the "around" of the whole process though, but that's not as hyped.
arcanus|5 years ago
I'll note that in my personal anecdote, the megacorps remain interested in and hiring in ML as much as ever.
arvindch|5 years ago
ptero|5 years ago
However, megacorps do not seem to suffer much for such continuous lagging in hiring. I do not know why this is so: is it that they still hire smart engineers who can easily change groups and fields or do they work on their core technology to help build the next peak (after the debris are washed away in a fad crash there is often a technology renaissance).
occamrazor|5 years ago
mritchie712|5 years ago
unknown|5 years ago
[deleted]
fnbr|5 years ago
I'm completely unsurprised by this. Regularly, at lunch, I'll ask my coworkers if they know of any DL applications that are making O($billions), and no one knows any outside of FAANG.
FAANG is making an insane amount of money due to DL. Outside of them though, I don't know who's making money here. When I was interviewing for jobs, there were a ton of startups that were trying to do things with DL that would have been better done with a few if statements and a random forest, and that had a total market size in the millions.
I think that, eventually, there'll be a market for this stuff, but I'm not convinced that it's anywhere near being widespread.
I was also a consultant before my current role. The vast majority of non-tech firms don't have their data in well organized + cleaned databases. Just moving from a mess of Excel sheets to Python scripts + SQL databases would have made a HUGE difference to the vast majority of clients I worked with, but even that was too big of a transformation.
Basically, everyone with the sophistication to take advantage of DL/ML already has the in-house expertise to do it. There's almost no one in the intersection of "Could make $$$ doing DL" && "Has the technical infrastructure to integrate DL".
dcolkitt|5 years ago
Yes, you're not going to achieve state-of-the-art performance with logistic regression. But for most problems the difference between SOTA and even simple models is not nearly as large as you might think. And two, even if you're cargo-culting SOTA techniques, it's probably not going to work unless you're at an org with an 8-digit R&D budget.
tomhallett|5 years ago
Small companies need to frame the problem as:
1) Do we have a problem where the solution is discrete and already solved by an existing ML/DL model/architecture?
2) Can we have one of our existing engineers (or a short-term contractor) do transfer learning to slightly tweak that model to our specific problem/data?
Once that "problem" actually turns into multiple "machine learning problems" or "oh, we just need todo this one novel thing", they will probably need to bail because it'll be too hard/expensive and the most likely outcome will be no meaningful progress.
Said in another way: can we expect an engineer to get a fastai model up and running very quickly for our problem? If so, great - if not, then bail.
ie: the solution for most companies will be having 1 part-time "citizen data scientist" [1] on your engineering team.
[1]: https://www.datarobot.com/wiki/citizen-data-scientist/
kovac|5 years ago
1. There's no clear problem statement. They have never formulated one and now trying to bolt ML on to their decision making.
2. They don't have well catalogued data for engineers/scientists to work with because they never tried to do rigorous analysis of data before ML became a thing.
3. Managers have no idea how to deal with data driven insights. What if the results are completely unintuitive to them? Are they going to change their processes abruptly? What if the results are aligned with what they have been always doing? Is it worth paying for something that they have been doing intuitively for decades?
I'm not a data scientist. But the biggest complaint I hear from my colleagues is that they lack data to train models.
scollet|5 years ago
lm28469|5 years ago
https://en.wikipedia.org/wiki/AI_winter
rjtavares|5 years ago
nutanc|5 years ago
Very few businesses I know actually have a deep learning problem. But they want a deep learning solution. Lest they get left out of the hype train.
rjtavares|5 years ago
calebkaiser|5 years ago
Have they? Specifically, have they "collapsed" relative to the average decline in job listings mid-pandemic?
whoisjuan|5 years ago
80 or 90% of what companies are doing with machine learning results in systems with a high computing cost that are clearly unprofitable if seen as revenue impacting units. Many similar things can be achieved with low-level heuristics that result in way smaller computing costs.
But nobody wants to do that anymore. There's nothing "sexy" or "cool" about breaking down your problems and trying to create rule-based systems that addresses the problem. Semantic software is not cool anymore, and what became cool is this super expensive blackbox that requires more computer power than regular software. Companies have developed this bias for ML solutions because they seem to have this unlimited potential for solving problems, so it seems like a good long term investment. Everyone wants to take that bus.
Don't get me wrong. I love ML, but people use it for the stupidest things.
ur-whale|5 years ago
It's just that much that needed to be invented has been invented and now it's time to apply it everywhere it can be applied, which is a great many place.
insomniacity|5 years ago
cochne|5 years ago
jungletime|5 years ago
1. Start and stop a podcast.
2. Play music
3. Ask for the time
The phone understands me, but then android breaks the flow, so I have to use my hands.
1. It will ask me to unlock the phone first? I have gloves and a mask on. It won't recognize my face, and my gloves don't register touches. Why do I have to unclock the phone to play music in the first place.
2. It gets confused on which app to play the music/podcast on. Wants to open youtube app, or spotify, and so on ...
3. Not consistent. I can say the same thing, and sometimes it will do one things, and another next time.
4. If I'm playing a video, and I want to show it full screen. I have to maximize and touch the screen. Why can't it play full screen be default.
physicsguy|5 years ago
adverbly|5 years ago
mijail|5 years ago
proverbialbunny|5 years ago
atsushin|5 years ago
Kednicma|5 years ago
hprotagonist|5 years ago
unknown|5 years ago
[deleted]
Ericson2314|5 years ago
They've long resisted that, of course, but I'm pretty sure half the popular of deep learning was it leveled the playing field, making engineers as ignorant of the inner-workings of their creations as the middle managers.
May the middle-manager-fication of work, and acceptance of ignorance that goes with, fail.
-----
Then again, I do prefer it when many of those old moronic companies flounder, so maybe this is a bad thing that they're wising up.
SomeoneFromCA|5 years ago
poorman|5 years ago
not2b|5 years ago
realradicalwash|5 years ago
It's tough atm.
kfk|5 years ago
andrewprock|5 years ago
I have seen so many more projects derailed by a lack of domain knowledge than I have seen for lack of technical understanding in algorithms.
dboreham|5 years ago
sunopener|5 years ago
[deleted]
spicyramen|5 years ago
x87678r|5 years ago
flavor8|5 years ago
gdsdfe|5 years ago
samfisher83|5 years ago
Barrin92|5 years ago
tanilama|5 years ago
camoverride|5 years ago
alpineidyll3|5 years ago
code4tee|5 years ago
Teams adding measurable value for their companies should be fine but others might not be.
astrea|5 years ago
darepublic|5 years ago
mac01021|5 years ago
ponker|5 years ago
Traubenfuchs|5 years ago
emmap21|5 years ago
unknown|5 years ago
[deleted]
ISL|5 years ago
unknown|5 years ago
[deleted]
rch|5 years ago
make3|5 years ago
itg|5 years ago
hankchinaski|5 years ago
dgellow|5 years ago
MattGaiser|5 years ago
phre4k|5 years ago
magwa101|5 years ago
SrslyJosh|5 years ago
pts_|5 years ago
arthurcolle|5 years ago
booleanbetrayal|5 years ago
recursivedoubts|5 years ago
rahimiali|5 years ago
eanzenberg|5 years ago
“To be clear, I think this is an economic recession indicator, not the start of a new AI winter.”
So, looks like he discovered an economic recession.
AznHisoka|5 years ago
m0zg|5 years ago
bitxbit|5 years ago