For those not familiar with the author, this is Daniel Lemire, a high performance software expert. He has done lots of work about squeezing as much performance out of a CPU as possible; see his work on SIMD JSON parsing[0]. I imagine he wrote this out of frustration for the software industry's seeming inability to identify who actual experts are in the field. It doesn't take much work to see how much books like Design Patterns and Clean Code have negatively shaped the industry.
I am currently finising my PhD and will start next month as a software architect. Among others I also looked into these books. What is the problem with them and what would you recommend instead?
"Those who speak do not know, those who know do not speak"
(I say this as a frequent blogger / speaker, so probably says more about me than I'm comfortable admitting)
But in all honesty, many are taken in by the confidence man - the loud person casting easy declarations. But we discount the expertise of the cautious person who admits their ignorance and limitations - despite extensive experience.
I've always felt this was a tad reductive, since there are plenty of people who are both very talkative online/good at marketing themselves that are also experts in their subject areas.
Hence the best explanation would be more like "self promotion/marketing is a skill in of itself, and those that are good at it aren't necessarily good at what they're selling".
So I guess the best strategy is to not take good salesmanship as an indication of anything other than the person knowing how to sell stuff.
I'm sorry you say that frequently, because I disagree. I have from time to time come across "experts" who would offer only vague explanations of what they were doing, or would shield their keyboard from your gaze. I have always learned subsequently that their knowledge was rather limited, which is why they didn't want to share it.
In my experience, the people who know the most have enough knowledge (and confidence) to share it freely.
This post treats being an expert as an unambiguously good thing. Richard Hamming's essay Experts has a bit more of a nuanced take on the topic. He talks how paradigm shifts in fields often come from outsiders, while at the same time acknowledging how increasing complexity neccessitates specialisation. I suppose Hamming's essay focuses on scientific and social gain, whereas this post focuses on personal gain.
There were two execs on a plane. "My company makes hard drives. We polish metal plates and try to put a thin metal coating on them, but our tolerances are mediocre." "Oh? My company makes telescope mirrors. We polish glass optically flat and lay down coatings measured in nanometers." "Oh, man, we've got to talk some more." Boom, Seagate.
> showing a total lack of practical skill is a status signal.
In some circles, maybe. Perhaps it commands respect in academia. But a lot of high-status occupations demand practicality and experience - expertise. Law, accounting, soldiering, finance - these are high-status occupations that require expertise; you can't fake them on the basis of second-hand knowledge and abstractions.
Apart from politics, dynastic rule and being a professor, I can't think of a high-status occupation that doesn't require practical expertise. And many professors actually have practical expertise. Not all professors are abstract.
It's a mark of people who avoid difficult things, who lack curiosity, and make a habit of evading volunteering and are unable to fathom noblesse oblige.
It's quite easy. You don't. By definition, you can't discern anything above your own level of expertise. If you knew what an expert was talking about, there was no point in consulting them in the first place. You can merely ask people you trust. Stereotypes also play an important role, even though they really shouldn't. A rather introverted, slightly socially awkward, blond guy with a ponytail wearing glasses and business casual clothing like a polo shirt while carrying a laptop with swag stickers from their last conference and a 3D printed keychain probably radiates more tech nerd energy than, say, your stereotypical school janitor.
I've worked on shaken baby syndrome for 8 years with hundreds of families, doctors, lawyers, scientists, [1] and the major obstacle we're facing is with the so-called "experts".
They are judicial experts, appointed by courts to indicate whether they believe a baby has been shaken or abused, on the basis of specific intracranial findings (subdural and retinal hemorrhage, mostly). They tend to be extremely affirmative in their opinions, claiming that violent abuse is "absolutely certain", is the only possible cause in the entire universe, even when there are obvious DNA-proven genetic conditions known to cause the exact same findings ("these have nothing to do with it, on the contrary, sick babies are even more abused"). That's what judges need: certainty, and experts give them exactly that. People are going to jail every day because these experts are so affirmative and certain, and everyone in the system literally worships them.
When anyone has any sort of question about SBS, they're referred to them. They are the "experts", they have 30 years of experience, they have diagnosed SBS during their entire careers. Who else would know better?
They teach their theories to the entire child protection and law enforcement professions. They write their own papers, they are editors in the relevant journals and they review all submissions — carefully rejecting every dissenting voice. They attack everyone who tries to bring a bit more nuance to the complicated issue of the medical detection of child abuse, calling them out "denialists" and "revisionists" in public.
So, I'm not at ease with this post, because these (pseudo)-experts have plenty of experience, they are the experts, and yet they are so much worse than serious specialized scientists and physicians who really know what they're talking about (but who are not judicial experts because they are too busy, well, treating patients and contributing to scientific knowledge). These experts have diagnosed SBS hundreds of times for decades, no one in the world has more experience. Yet, I can personally attest that the scientific level of their work is catastrophically bad.
Perhaps the way to conciliate things is with this: "An expert has a track record and has had to face the consequences of their work. Failing is part of what makes an expert: any expert should have stories about how things went wrong."
That's exactly what's missing with SBS and child abuse: there's no feedback loop! The experts could be wrong every single time, accusing innocent parents and caregivers every time, they would never know, except that people would systematically protest their innocence — which, horrifyingly, is actually the case. It would be funny if it weren't so tragic, but protesting one's innocence in these cases is actually so frequent that it has become one of the diagnostic criteria.
Now, they're not wrong every time, but I believe they're more often wrong than right.
PS: for the anecdote, one of these experts, the top forensic pediatrician in the country, a "child protection champion", is being prosecuted now because a patient has accused her of raping her.
Yes, courts are known to invite forensic experts of fields that boil down to unscientific fraud.[1] But I must say your experience with experts is limited to an entire field of con artists.
IIRC a TED talk concluded SBS was a now mostly discredited pseudo-scientific moral panic of the 80's and 90's. There are certain pathos microtransgressions in Western civilization that are equatable to Holocaust denial, and one of them is anything that can be (mis)construed as "child abuse". But instead of identifying and prosecuting real child abuse, it becomes a lazy all-purpose "hammer" to find "nails" to accrue statistical "wins" at the toll of unreasonably over-criminalizing nothingburger situations while ruining lives.
PS: Forgive me viewing the child abuse "identification" system as inept and often wrong. I recall disclosing to a teacher how my father was an abusive narcissist and horribly cruel to my mother and myself in ways that didn't leave physical marks, yet nothing was done. Around age 16, I called the police on him because he held me down to my bed and choked me to unconsciousness for not mowing the lawn, but he talked his way out of it.
Interesting in light of this posting’s title; I expected it to be the same thing before double checking, since this was a recent posting (that no one had much to say on):
I was wondering what the answer would be, because it's a hard problem. There was only the half answer of "they have experience".
The best answer I heard is that only domain experts have a chance to recognize each other. Other than that you are left with secondary signals like other people paying them for their expertise.
What? It gives at least a partial answer in the first paragraph:
> An expert has a track record and has had to face the consequences of their work. Failing is part of what makes an expert: any expert should have stories about how things went wrong.
This might sound so obvious as to be a non-answer, but I think it's a good point. There are many "experts" who acquired degrees in, wrote papers on, and now teach others about their area of focus, but have at no point in that process had to, say, stake their employment on being correct about that area.
For example, professors of literature have all written thousands of pages of text about good novels, but there's little evidence that they can actually make good novels.
At least in software craftsmanship, experts are best identified working with others. It appears to be an irreducible process that cannot be pantomimed with trivia-based interviews or formulaic problems. Acting in an arrogant fashion or looking smart has zero correlation with performance, but it can fool some people some of the time who lack subject matter expertise.
IIRC This saying was used to define pornography in a legal sense and it made me realize they mean because your member knows porn when you see it and ah reacts to it. But somehow I can't see courts pushing for an objective tent pitching index for imagery in place of porn experts.
[+] [-] GuestHNUser|1 year ago|reply
[0] https://lemire.me/blog/2021/09/25/new-release-of-the-simdjso...
[+] [-] Mkengine|1 year ago|reply
[+] [-] softwaredoug|1 year ago|reply
(I say this as a frequent blogger / speaker, so probably says more about me than I'm comfortable admitting)
But in all honesty, many are taken in by the confidence man - the loud person casting easy declarations. But we discount the expertise of the cautious person who admits their ignorance and limitations - despite extensive experience.
[+] [-] CM30|1 year ago|reply
Hence the best explanation would be more like "self promotion/marketing is a skill in of itself, and those that are good at it aren't necessarily good at what they're selling".
So I guess the best strategy is to not take good salesmanship as an indication of anything other than the person knowing how to sell stuff.
[+] [-] denton-scratch|1 year ago|reply
I'm sorry you say that frequently, because I disagree. I have from time to time come across "experts" who would offer only vague explanations of what they were doing, or would shield their keyboard from your gaze. I have always learned subsequently that their knowledge was rather limited, which is why they didn't want to share it.
In my experience, the people who know the most have enough knowledge (and confidence) to share it freely.
[+] [-] frankplow|1 year ago|reply
[+] [-] ahazred8ta|1 year ago|reply
There were two execs on a plane. "My company makes hard drives. We polish metal plates and try to put a thin metal coating on them, but our tolerances are mediocre." "Oh? My company makes telescope mirrors. We polish glass optically flat and lay down coatings measured in nanometers." "Oh, man, we've got to talk some more." Boom, Seagate.
[+] [-] skmurphy|1 year ago|reply
Update I know see that this is taken from Chapter 26 "Experts" in Hamming's "Art and Science of doing engineering"
[+] [-] denton-scratch|1 year ago|reply
In some circles, maybe. Perhaps it commands respect in academia. But a lot of high-status occupations demand practicality and experience - expertise. Law, accounting, soldiering, finance - these are high-status occupations that require expertise; you can't fake them on the basis of second-hand knowledge and abstractions.
Apart from politics, dynastic rule and being a professor, I can't think of a high-status occupation that doesn't require practical expertise. And many professors actually have practical expertise. Not all professors are abstract.
[+] [-] 1letterunixname|1 year ago|reply
[+] [-] kseifried|1 year ago|reply
Who are the experts - https://opensourcesecurity.io/2020/04/07/who-are-the-experts...
Experts from a world that no longer exists - https://opensourcesecurity.io/2021/11/28/episode-299-experts...
[+] [-] MyFedora|1 year ago|reply
[+] [-] rossant|1 year ago|reply
I've worked on shaken baby syndrome for 8 years with hundreds of families, doctors, lawyers, scientists, [1] and the major obstacle we're facing is with the so-called "experts".
They are judicial experts, appointed by courts to indicate whether they believe a baby has been shaken or abused, on the basis of specific intracranial findings (subdural and retinal hemorrhage, mostly). They tend to be extremely affirmative in their opinions, claiming that violent abuse is "absolutely certain", is the only possible cause in the entire universe, even when there are obvious DNA-proven genetic conditions known to cause the exact same findings ("these have nothing to do with it, on the contrary, sick babies are even more abused"). That's what judges need: certainty, and experts give them exactly that. People are going to jail every day because these experts are so affirmative and certain, and everyone in the system literally worships them.
When anyone has any sort of question about SBS, they're referred to them. They are the "experts", they have 30 years of experience, they have diagnosed SBS during their entire careers. Who else would know better?
They teach their theories to the entire child protection and law enforcement professions. They write their own papers, they are editors in the relevant journals and they review all submissions — carefully rejecting every dissenting voice. They attack everyone who tries to bring a bit more nuance to the complicated issue of the medical detection of child abuse, calling them out "denialists" and "revisionists" in public.
So, I'm not at ease with this post, because these (pseudo)-experts have plenty of experience, they are the experts, and yet they are so much worse than serious specialized scientists and physicians who really know what they're talking about (but who are not judicial experts because they are too busy, well, treating patients and contributing to scientific knowledge). These experts have diagnosed SBS hundreds of times for decades, no one in the world has more experience. Yet, I can personally attest that the scientific level of their work is catastrophically bad.
Perhaps the way to conciliate things is with this: "An expert has a track record and has had to face the consequences of their work. Failing is part of what makes an expert: any expert should have stories about how things went wrong."
That's exactly what's missing with SBS and child abuse: there's no feedback loop! The experts could be wrong every single time, accusing innocent parents and caregivers every time, they would never know, except that people would systematically protest their innocence — which, horrifyingly, is actually the case. It would be funny if it weren't so tragic, but protesting one's innocence in these cases is actually so frequent that it has become one of the diagnostic criteria.
Now, they're not wrong every time, but I believe they're more often wrong than right.
[1] https://news.ycombinator.com/item?id=37650402
PS: for the anecdote, one of these experts, the top forensic pediatrician in the country, a "child protection champion", is being prosecuted now because a patient has accused her of raping her.
[+] [-] MyFedora|1 year ago|reply
[1]: https://www.theatlantic.com/ideas/archive/2024/03/courts-jun...
[+] [-] 1letterunixname|1 year ago|reply
PS: Forgive me viewing the child abuse "identification" system as inept and often wrong. I recall disclosing to a teacher how my father was an abusive narcissist and horribly cruel to my mother and myself in ways that didn't leave physical marks, yet nothing was done. Around age 16, I called the police on him because he held me down to my bed and choked me to unconsciousness for not mowing the lawn, but he talked his way out of it.
https://www.ted.com/talks/waney_squier_i_believed_in_shaken_... (Waney Squier, 2017)
[+] [-] sprior|1 year ago|reply
[+] [-] daverol|1 year ago|reply
[+] [-] nullfield|1 year ago|reply
https://news.ycombinator.com/item?id=40070432
[+] [-] wakawaka28|1 year ago|reply
[+] [-] unknown|1 year ago|reply
[deleted]
[+] [-] simonblack|1 year ago|reply
If I know more than him, he can't be an expert.
[+] [-] orangesite|1 year ago|reply
[+] [-] decker|1 year ago|reply
[+] [-] 082349872349872|1 year ago|reply
EDIT: A kiwi once asked me "what do you do for a biscuit?" for which I required the translation: "and what might your area of expertise be?"
[+] [-] lolc|1 year ago|reply
The best answer I heard is that only domain experts have a chance to recognize each other. Other than that you are left with secondary signals like other people paying them for their expertise.
[+] [-] mycologos|1 year ago|reply
> An expert has a track record and has had to face the consequences of their work. Failing is part of what makes an expert: any expert should have stories about how things went wrong.
This might sound so obvious as to be a non-answer, but I think it's a good point. There are many "experts" who acquired degrees in, wrote papers on, and now teach others about their area of focus, but have at no point in that process had to, say, stake their employment on being correct about that area.
For example, professors of literature have all written thousands of pages of text about good novels, but there's little evidence that they can actually make good novels.
[+] [-] 1letterunixname|1 year ago|reply
At least in software craftsmanship, experts are best identified working with others. It appears to be an irreducible process that cannot be pantomimed with trivia-based interviews or formulaic problems. Acting in an arrogant fashion or looking smart has zero correlation with performance, but it can fool some people some of the time who lack subject matter expertise.
[+] [-] hulitu|1 year ago|reply
"I know it when i see it." /s
[+] [-] unraveller|1 year ago|reply