I will say that less is more and "The Bitter Lesson" applies here. Chasing biologically-inspired rabbits, such as STDP and LIF (see paper above/wikipedia), does seem to be a waste of time, especially when we have this thing entirely outside of biology that can arbitrarily replicate, serialize, mutate and simulate billions of instances of the same parent candidate in minutes-hours.
Leaky charge carriers and inability to persist learned weights between candidate instantiations are limitations, not features to emulate. Imagine if you could be reincarnated with all of the exact knowledge you have today. Then clone that 1000 times and apply subtle mutations to it. Then, put these clones in an FFA arena and see who wins based upon a very well crafted fitness function. Then, do all of that over and over thousands of times per hour.
>Chasing biologically-inspired rabbits, such as STDP and LIF (see paper above/wikipedia), does seem to be a waste of time
Unless physiological compatibility with a biological brain is considered an interesting endpoint? If you think about Neuralink for example, wouldn’t it be interesting if our brains could directly engage the model? Not just a translation layer but perceive it directly and natively though some kind of synaptic modem that converts analog exchanges of neurotransmitters to the synthetic network in the digital domain.
> Then clone that 1000 times and apply subtle mutations to it. Then, put these clones in an FFA arena and see who wins based upon a very well crafted fitness function. Then, do all of that over and over thousands of times per hour.
I spent many years on an ALife+ANN simulation that did this. For each new generation, I kept the top X% unchanged, the next Y% were changed a little, next Z% changed a lot, etc.
It was pretty fun and I wish I had the time+money to continue on a larger scale.
bob1029|1 year ago
I will say that less is more and "The Bitter Lesson" applies here. Chasing biologically-inspired rabbits, such as STDP and LIF (see paper above/wikipedia), does seem to be a waste of time, especially when we have this thing entirely outside of biology that can arbitrarily replicate, serialize, mutate and simulate billions of instances of the same parent candidate in minutes-hours.
Leaky charge carriers and inability to persist learned weights between candidate instantiations are limitations, not features to emulate. Imagine if you could be reincarnated with all of the exact knowledge you have today. Then clone that 1000 times and apply subtle mutations to it. Then, put these clones in an FFA arena and see who wins based upon a very well crafted fitness function. Then, do all of that over and over thousands of times per hour.
jcims|1 year ago
Unless physiological compatibility with a biological brain is considered an interesting endpoint? If you think about Neuralink for example, wouldn’t it be interesting if our brains could directly engage the model? Not just a translation layer but perceive it directly and natively though some kind of synaptic modem that converts analog exchanges of neurotransmitters to the synthetic network in the digital domain.
RaftPeople|1 year ago
I spent many years on an ALife+ANN simulation that did this. For each new generation, I kept the top X% unchanged, the next Y% were changed a little, next Z% changed a lot, etc.
It was pretty fun and I wish I had the time+money to continue on a larger scale.