savant_penguin's comments

savant_penguin | 1 year ago | on: LLMs can't do probability

1. Incorrect. The output of the decoder LLM is the probability distribution of the next token given the input text. Temperature=0 means that the output distribution is not pushed to be closer to a uniform distribution. The randomness comes from the sampling of the next token according to the output distribution to generate text. If you want determinism you always get the argmax of the distribution.

savant_penguin | 2 years ago | on: Majority of gig economy workers are earning below minimum wage: research

"Some jobs will go away, that's fine."

To whom is that fine? To the people going from $2/hour to zero it certainly isn't.

I honestly cannot grasp how some people can _forcibly_ remove other people's _options_ then pat themselves on the back as some sort of armchair savior.

Yes, for some people a bad option could still be their best option. What sort of moral superpower is that that enables you to forbid a contract between two consenting adults A and B such that A wants to work for X and B that's willing to pay X, but you as a C that just won't let it happen and is willing to use force to stop it from happening

savant_penguin | 3 years ago | on: Speedup from switch to +=

In this case I believe python is faster by a few months.

Jokes aside this is pytorch so this is compiled to C++ or cuda, the problem likely comes from the different functions that are called for += vs +

savant_penguin | 3 years ago | on: Speech by Chair Powell on monetary policy and price stability

If the disruption of supply chains is the actual culprit for inflation, shouldn't we observe gradual deflation as they recover?

If on the other hand it's printing money (as I'd expect from the trillions of free money in recent months) it should only get worse. What I observe is more consistent with money printing than disruption of supply chains

savant_penguin | 3 years ago | on: “Science must respect the dignity and rights of all humans”

"In some cases, however, potential harms to the populations studied may outweigh the benefit of publication."

I'm sure our demigods will gladly enlighten us with their wisdom to distinguish between misthoughts and correct thoughts.

So that when journalists claim to be following the science there will be no dissenting voice in research to disagree.

Those who defend opposing ideas will have no leg to stand on and will rightfully be labeled science deniers

savant_penguin | 3 years ago | on: How to pay your rent with your open source project (2020)

Just to comment in the "little bit of free code" there are projects like python and numpy that would take ages to do properly and depending on the company would never ship (or function as reliably as it does right now)

Sometimes they're buying decades worth of debugging/testing

savant_penguin | 3 years ago | on: Why do tree-based models still outperform deep learning on tabular data?

A important point is that it's an absolute pain in the ass to preprocess tabular data for neural networks.

Categorical > one hot encoding > deal with new categories in test time (sklearn does this, but it's really slow and clunky)

Numerical > either figure it out the data distribution for each column and normalize by that or normalize everything by z score. Found an outlier?? Oops, every feature collapsed to 0

Can you that for 10 features? Sure, now try it again with 500, it's not fun

Ok, now that you've done all that you can begin training and possibly get some reasonable result.

Compare that with tree models: data>model>results

savant_penguin | 3 years ago | on: The Uber leak exposes the global war on workers

An Uber driver in his way to being exploited:

Wake up, decide it's a good day to be exploited.

Kiss your wife goodbye as you enter your car "bye honey! On my way to being exploited by Uber!"

Drive away and open your exploitation app. Accept a ride, go to the person: "who's the person that's going to participate in my exploitation together with Uber today? Oh, there he is!". Drive him to the place he needs to be, drive away.

"Ok, do I want to be exploited some more today?"

Decide to be exploited on a few more rides and return home.

Now being serious, if you're an Uber driver and let's say that in the short term that's your best option, why would it be better for him to have this option striped from him? Why do you think anyone else should make that call except him?

page 1