nwbrown's comments

Large language models can have billions, or even trillions, of parameters. But how big do they need to be to achieve acceptable performance?

Last week I posted a retrospective on swing state polls. However, I excluded Nevada and Arizona as both states still had a significant number of votes to count. Since then, both states have gotten to 99%, so I figured I could include them. I also decided I could probably go back a bit further and looked at all polls going back to September 23rd, when the first early voting started.

The result? We finally got a poll outside of the margin of error!

This might sound bad, but I would argue it’s at least a step in the right direction. Margin of errors are supposed to represent 95% confidence intervals. With 101 polls analyzed, we would expect around 5 of them to be outside of the margin of error, but only 1 was. So that indicates there probably is some herding going on. And they still show a bias against Trump, which may be due in part to the above mentioned herding.

The one poll that outside the margin of error was a BullFinch Group Michigan poll that had Harris up by 8 percentage points and a margin of error of 4 percentage points. Remember margins of error refer to the error of a particular response; the margin of error of the gap between candidates would be twice that. So this wasn’t that far off. And the pollster specifically called out the Michigan poll as looking like an outlier. They also point out that they specifically try to avoid herding, so I would argue this is a point in favor of this particular poll.

So does this mean the pollsters did a great job?

No.

The mean margin of error in these polls was about 3.7 percentage points. The median margin of error was 3.5. That means that for most of these polls, one candidate would need to be up by 7 percentage points in order to say they had a clear lead. Arizona was the biggest blowout of these states, and Trump only won it by 5.5 percentage points. There were only two polls with a margin of error small enough to detect such a lead, and neither was in Arizona. They were two Fox News polls in Pennsylvania and Michigan, and both those races were decided by 2 points or less.

So yes, the pollsters were accurate in that they were within the margin of error. But each and every one of these was nowhere near precise enough to be useful. I praised the BullFinch poll for releasing an outlier poll. And I will stand by that praise. Except, it’s still using a 4 point margin of error. The last time the race in Michigan decided by more than 8 points in one direction was 2012, when Michigan was not considered a swing state. So I’m not sure what the intended purpose of such a poll was.

And it wasn’t even the worst poll. There were several with margin of errors approaching 5 points. Meaning a candidate would need nearly a double digit lead for the poll to be outside the margin of error.

And again, these were all swing states. If a candidate has a double digit lead in a state, that state is probably not a swing state.

Look, I know polling is hard and expensive. Especially these days when people generally don’t pick up the phone if they don’t recognize the number. But really, if your margin of error is this big, I’m not sure how the poll is supposed to be useful. Normally you could argue that you could aggregate the polls and get a more accurate result. But that assumes the polls are more or less independent. And it’s pretty clear they are not. I suspect the culprit is the models pollsters use to do their weighing, but there are also other possibilities.

So while I will defend pollsters against charges that they were inaccurate, that’s only looking at part of the picture. They were accurate only because they had margins of error so high it was almost impossible for them to be inaccurate.

How smart is ChatGPT, really?

Warning, this post is going to be about one of the most boring topics possible, font choice.

The Washington Post reports the State Department has ordered a shift in font choice from Times New Roman to Calibri. The justification is that a serif font like Times New Roman (serifs are the little lines that attach to the ends of some letters in some fonts) is less accessible for readers with vision or reading problems than a font like Calibri that lacks serifs (known as sans serif).

That's discussed in the post.

With recent management changes it seems a lot of employees have left the bird app. Turns out if you give people a choice between working long hours with little to no vision or three months severance pay, most people will take the pay. Employers often overestimate how much their employees need their jobs.

So does this mean Twitter will fall apart? Can a company survive 90% of their employees leaving?

The short answer is that because there is a huge variety in quality of high schools in the US, that would result in a lack of diversity.

That's the case in some liberal arts colleges, but is not typical, or at least wasn't in the early 00's when I went to school. Then most students did have to declare a major pretty early, usually when you applied for competitive majors like CS was back then. There was a "University Studies" major which you could be in for the first year if you were undeclared, but it was pretty important to declare a real major pretty quickly.

If by lawyer you mean partner and by doctor you mean a practicing surgeon, both of those are pretty far into their careers. It's not very fair to compare them with an entry level engineering job.

Google results are personalized so your first page is likely different from other people's. To be honest I would probably have to have been there to know for certain whether he was making a joke referencing rape or just making a more innocent comment.

That being said, tech recruiters as a rule are trash and bugging people about a job after they have refused is a pretty jackass thing to do anyway. So I'm fine not giving him the benefit of the doubt.

I am in full support of women's clothing getting oversized pockets.

Especially if it means I can wear cargo pants without getting weird looks from women.

IBM's problems predate Rometty becoming CEO.

Yes, but his performance was more aided by his 7 years of experience than his two weeks of cramming.

I'm guessing it's the 7 years of experience that impressed Google, not the two weeks of cramming.

More likely they aren't testing your knowledge of the algorithm but testing your ability to write code.

Again, you aren't going to get familiar with the patterns in computer science in one week. That requires years of practice. Most likely you will just hit a Dunning-Kruger peak and try to shoehorn in a pattern where it doesn't belong.

And yes, js is used in other contexts as well, but it is primarily used in front end developer. Which is why I used the word 'likely'.

You are unlikely to get asked any of these things during the interview, and if you are it will be obvious that you just crammed for them (much like if just brush your teeth the day before your dental appointment and then arrive at the dentist with bleeding gums).

If JavaScript is your language then you are likely looking at front end development, in which case it will be much more useful to know about frameworks and testing strategies than CS theory.

You aren't going to gain the skills gained by a four year education in a week. And if you try it will likely be obvious. Interviewers (at least the good ones) aren't looking for someone who knows a particular algorithm by memory, they are looking for someone who is familiar with the concepts. And familiarity is easy to distinguish from rote memorization.

The good news though is that unless you lied on your resume, they already know that and are still interested in you. So you likely have other characteristics they are interested in.

I would concentrate instead on learning about the company you are interested in. That's something you can do in the course of a week.

Science didn't replace philosophy. Science is philosophy. That's why scientists used to be called natural philosophers. Yes, in the modern era we have gotten much more specialized in the modern era and that arguably is a loss. But it is kinda necessary given the depth that many fields have gotten.

Legitimately good employees are in demand, therefore there are potential employers willing to pay top dollar for them that you have to compete with. You may think there are aspects of your company worth more than salary, but so do most employers. You want your employees to think of themselves as equals and all worth the same amount, but you refuse to apply the same logic to your competitors? I have nothing against having transparent salaries (in fact I think, implemented correctly, it is a grand idea). But if it makes you unable to hire top talent because you are unwilling to justify a discrepancy of pay to those less skilled, it's going to be more trouble than it is worth.