Show HN: Duplicate Word Finder

[d4damager]

I always wondered whether they use something like this for BBC Radio 4’s “Just a minute”.

For those not familiar, it’s a radio panel game where the panelists must speak on a subject for up to a minute without hesitation, deviation or repetition.

I imagine they would need a screen like this to highlight repeated words for the host (perhaps in combination with some speech recognition, or stenography).

[pastage]

The competitors are free to protest and try to gaslight you into believing you made a mistake. Part of the fun is beingable to give a good explanation as to why the protest is wrong. The game is a friendly social one, at least the one in Swedish is done like that.

https://en.m.wikipedia.org/wiki/P%C3%A5_minuten

[stevekemp]

Speaking on a subject probably makes it more complex, but reading your description I had a sudden memory of Timmy Mallett's word-association game, from the UK back in the day.

The aim there is to reply spontaneously to random words, without hesitation and repetition. If you fail you get hit on the head by a foam-mallet.

[philipwhiuk]

That would ruin the entire thing.

[bobnamob]

Duplicate word removal seems like a naive, coarse grained method to improve writing style. Smells like the high school trope of throwing a thesaurus at your essay and hoping for the best

[llanowarelves]

Yep, it's actually what we were taught in high school.

Still, sometimes recognizing duplicate words points out some bad/imprecise wording where you're repeating yourself and may not like that.

I find myself using the "edit" feature on comments online a decent amount to catch/add things I only found when reading it back in complete form.

[oneeyedpigeon]

Uncritical duplicate removal, yes. But I edit a lot of work and duplicated words are one of the things that I see time and gain, really dragging writing down. IMO, removing duplicates is one of the cheapest wins, a very easy way of making improvements so long as you do it at all reasonably.

[tgv]

It doesn't even find inflections, such as plural/singular. But as a writing tool, it can only lead to contrived constructions that seem to be the realm of sports journalists: "The multiple medal winner" ... "Last year's 500 meter champion" ... "The Londoner" ... "The former law student" ... when referring to one and the same person during a race.

I do like that it lists them on the side, and allows you to remove them.

[finnhvman]

Someone commented something similar on reddit a while ago, and it is a very good argument to be honest. Still, you might find some situations where the tool might be useful and it's already deployed, so people can use it if they want

[pastage]

ChatGPT: "Are any of these sentences repetition of each other"

Most educational literature from the US are rife with sentences that might not be verbatim repetition but most certainly say the same with a different word soup.

[clarkdale]

Feature request to find duplicates within words contains underscores or hyphens.

I want to use this for reading documentation. I often find myself struggling with acronyms like JWT_AUTH and JWT_OKTA so this would help me visually identify and mentally map the use of common terms.

[rwoerz]

There is no no use for that.

[rlv-dan]

I'm sure you've thought about every single possible situation to conclude your insightful comment.

Otherwise...

There is no use for your comment.

[londons_explore]

Write "the the the the the the"

Result:. "No duplicate words found"

[suddenclarity]

Seems reasonable that those types of words are eliminated for the purpose of the tool. Either by having a letter limit or by whitelisting them. Of course there will be duplicates of "the" and "and".

[londons_explore]

Perhaps a better model might be to look at word frequency analysis, and highlight words that are used substantially more frequently than in typical English text.

[dkarbayev]

Tried “Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo” and it highlighted every word. Tried “James while John had had had had had had had had had had had a better effect on the teacher” and it had no issues with it.

[finnhvman]

you can use the first slider on the toolbar to set the minimum word length, default is 4 (or the third button on the toolbar if you're on phone or smaller screens)

[coffeeri]

You might have to adjust the minimal word length to 3.

[00117]

When I press the cmd key on my mac on Chrome, all the text disappears.

[finnhvman]

the initial text on the site is the instructions. I can agree it's somewhat a questionable design, but I wanted to make sure that the instructions are there for newcomers and easy to clear for the regular users. So any action on the editor will clear the instructions, but the text you type is safe.

[finnhvman]

Find duplicate words and eliminate them from your writings

[pseingatl]

Added text, page hangs.

[finnhvman]

was it a long text?