> Also, what makes me think I can effectively imagine the average person’s vocabulary? People from other countries, ethnic groups, genders, socioeconomic backgrounds, or with different language backgrounds, might have quite different opinions on many borderline words. For example, I was quite conscious of the idea in the last few years that men and women tend to know different words. For example, practically, should I include TAFFETA or not?
Did you consider getting some other people to do this manual curation process? You could then use roughly the intersection of all lists.
I did consider this but the effort of doing a full dictionary pass is a lot to ask for marginal improvement and I don't know anyone quite as obsessed as me who would do it. Paying someone would be possible, but finding the right combination of "willing to do it for pay" and "I trust their judgement" is hard.
In practice, the way I approach this is by reacting to complaints from players who either don't know words I included or were disappointed I didn't include a particular word.
I also developed a word search game and was interested in a methodology for determining what words are "reasonably findable".
The best I could come up without going through the dictionary word-by-word like you did was to use data from actual games played, to see what words were actually being found by the players.
In this game the goal is not completion but to get a high score that you can compare against others (and yourself from previous days). So far I haven't done much with the list of word frequencies except an experimental feature where you can see stats about your game using "commonly found words only", e.g. what percentage of commonly found words did you find.
Also this was a somewhat similar effort to yours, to create the Common English Lexicon word list that can be used on the word game site woogles.io:
https://github.com/Fj00/CEL
> For instance, there is never a chance for the player to enter a word like POESY and feel good about knowing such an obscure word. I feel that given how critical it is for most players to be able to reach 100% completion of a board, this is the right tradeoff to make.
Perhaps words in this category should be on a banlist where you never generate puzzles where it would be a valid guess.
Cyphase|1 year ago
Did you consider getting some other people to do this manual curation process? You could then use roughly the intersection of all lists.
jsharpe|1 year ago
In practice, the way I approach this is by reacting to complaints from players who either don't know words I included or were disappointed I didn't include a particular word.
wdumaresq|1 year ago
The best I could come up without going through the dictionary word-by-word like you did was to use data from actual games played, to see what words were actually being found by the players.
Here is the word frequency from about 900 days of daily challenges (about 15 players per day), ranging from TEE found 4636 times to LAWYERED found once: https://docs.google.com/spreadsheets/d/1OY32E-JyHh2C4A_1ADy4...
In this game the goal is not completion but to get a high score that you can compare against others (and yourself from previous days). So far I haven't done much with the list of word frequencies except an experimental feature where you can see stats about your game using "commonly found words only", e.g. what percentage of commonly found words did you find.
You can try the game here: http://crosswordislandhopper.com/playGob And the daily challenge: http://crosswordislandhopper.com/dailyChallenge
wdumaresq|1 year ago
furyofantares|1 year ago
Perhaps words in this category should be on a banlist where you never generate puzzles where it would be a valid guess.