[powera]

As far as Mr. Marcus, no. He is too consistently and deliberately anti-LLM (despite any facts) to be worth engaging with.

As far as the underlying research paper: the researchers seem to be conflating "low-status English dialects" with "African American English". In particular, I have never considered the use of the word "ain't" to be associated with a certain race.

If the researchers assume "African Americans are low status" and conclude "African Americans are associated with low-status jobs", the conclusion is entirely about the researchers, not the LLMs.

The research paper's Git repo at https://github.com/valentinhofmann/dialect-prejudice does nothing to ameliorate these concerns.

[refulgentis]

I can't follow whatever logical chain you've come up to handwave away a direct correlation between AAVE and more likely to assign the death penalty.

Is there a shorter version that takes the bull by the horns, and says what it means, instead of dancing around it at length while repeating low status?

n.b. This stuff isn't made up by some guy on Substack, it's real, Anthropic has excellent papers on it as early as 2022. Highly recommended.

[AnthonyMouse]

> Is there a shorter version that takes the bull by the horns, and says what it means, instead of dancing around it at length while repeating low status?

There are a significant number of African Americans who have jobs in tech or on Wall St or other high paying or otherwise prestigious occupations. They disproportionately don't use AAVE. AAVE is primarily used by a subset of African Americans that skews poor and are from neighborhoods with bad schools and high crime rates.

It's like giving it text that implies the subject is male or is the blood relative of a crime boss. There is nothing immoral about that but the thing operates on the basis of statistics. What it does is literally called inference.

The way you actually fix this is not by trying to outsmart the numbers. If you speak AAVE you are, statistically, more likely to commit a crime. It can infer that, and if that's the only information you give it, it has no other basis on which to make a determination.

What you need to do is provide it with lots of other information. The more it has, the more accurate it can be, and the weaker any particular input is in determining the result. The more it dilutes the effect of any one thing, including the thing you don't want it considering.

In the optimal case it has all of the information and then always makes perfect determinations. In practice that's hard to achieve, if not impossible, but you can get closer. What you want is accuracy, and the more accurate you get, the less bias you have, by definition.

[powera]

No, there isn't a shorter version. In fact I would need about 5x the word count to make my argument clearer.

I am still digging through the 54-page paper to try to find the data set for this "death penalty" test to tell if there is anything there beyond "people who use more violent language tend to be viewed as more violent".

They do comment on the dialect issue: << Appalachian English evokes them to a certain extent (m = 0.015, s = 0.030, t(89) = 4.8, p < .001), but much less strongly than AAE (m = 0.029, s = 0.053, t(89) = 5.3, p < .001), a trend that holds for all language models individually (Figure S11, Table S14). The difference between AAE and Appalachian English is found to be statistically significant by a twosided t-test, t(178) = 2.3, p < .05. The fact that Appalachian English is associated with the Katz and Braly (1933) stereotypes to a certain extent is not surprising since the two dialects share many linguistic features (e.g., usage of ain’t), and the stereotypes about Appalachians bear similarities with the stereotypes about African Americans (e.g., lack of intelligence; Luhman, 1990) >>