top | item 29646146 (no title) davefol | 4 years ago Seems odd to claim 25x reduction in size when the algo involves looking into a database of a trillion chunks of text. discuss order hn newest visarga|4 years ago The "algo" here refers to the neural net itself. The text index is considered an easy problem as you can do lookups in logarithmic time. ehsankia|4 years ago The word "Algo" here is definitely awkward. The point is though that what matters most here is the number of parameters, as those correlate quite closely with training and inference time. Storage space is pretty trivial, but TPU cycles are less so. davefol|4 years ago Thanks for this. Rereading + your comment and I think I have a better understanding of why this is progress.
visarga|4 years ago The "algo" here refers to the neural net itself. The text index is considered an easy problem as you can do lookups in logarithmic time.
ehsankia|4 years ago The word "Algo" here is definitely awkward. The point is though that what matters most here is the number of parameters, as those correlate quite closely with training and inference time. Storage space is pretty trivial, but TPU cycles are less so. davefol|4 years ago Thanks for this. Rereading + your comment and I think I have a better understanding of why this is progress.
davefol|4 years ago Thanks for this. Rereading + your comment and I think I have a better understanding of why this is progress.
visarga|4 years ago
ehsankia|4 years ago
davefol|4 years ago