top | item 36242778

(no title)

h_mirin | 2 years ago

What might be seen as exponential progress could actually be the accumulation of a few tremendous leaps, I believe. For instance, within the architecture realm, we've witnessed leaps such as:

Stacked neural networks (deep learning) / CNN / Transformer

From the application perspective, there are also areas that have seen significant progress:

AlphaGo (An application of reinforcement learning) / AlphaFold (An application for biological sequences) / ChatGPT (An application of Large Language Models / Reinforcement Learning from Human Feedback)

You may pick different ideas. However, what I want to say is that these great advancements happened approximately once every two or three years. It may take some time before we witness another major leap.

In truth, the majority of research and development does not contribute to exponential progress. It's sad. However, this does not render such effort meaningless. Although it may seem as though there's too much competition within a narrow field, this may not necessarily be detrimental to humanity.

discuss

order

rfergie|2 years ago

> What might be seen as exponential progress could actually be the accumulation of a few tremendous leaps

This is what exponential progress is as long as the leaps continue happening

h_mirin|2 years ago

While I agree with your point, I think it ultimately depends on the timescale we're looking at. In other words, if we're looking at a timescale of several years, it's conceivable that we've been experiencing such leaps several times over the previous rate (or more?) for about the past decade. However, if we're looking at a matter of a few weeks, it may seem like we're stagnating. The point is that we cannot predict if these leaps will continue to occur, and when they will happen.