[grph123dot]

Yesterday I asked chatgpt to compute the probability that two words of five letters both began with the same three letters and it could not solve it. It gave me about 10 different results but all were wrong, it seems the number five mislead the algorithm. Another question was computing the variance of the sum of 60 dices but omitting the results where the sum was greater than 200, the reasoning was incorrect in all cases, finally I suggested creating a simulation to estimate the variance in R, but it made an error about the var command in R suggesting that it has an option for the degree of freedom.

[thatwasunusual]

> Yesterday I asked chatgpt to compute the probability that two words of five letters both began with the same three letters and it could not solve it.

IMO, this is irrelevant, because "normal people" would never ask this question. Because it has never been asked, there's also not a ChatGPT answer to it, because its answers are based on already existing information.

This thinking that ChatGPT is AI is just wrong. It's not AI, nor GI, so stop comparing it with that.

As a human, I can destroy your problem/question with one claim: you claim that the words must be built on the English alphabet. Let's throw some Æ, Ø and Å in there, shall we?

Can YOU solve that problem?

[pugworthy]

Yes, kind of playing the "gotcha" question game.

I really see it kind of like having a good junior programmer or intern helping me out who has an incredibly broad amount of knowledge. Sometimes they need some coaching or gentle correction, sometimes you say, "I'll take it from here", but inevitably there are those times where you think, "Great job! I'm really impressed!"

[qwertygnu]

Are you serious? That's your gotcha? It's just doing some math over a dictionary, it doesn't matter the language or characters. So yes, a human can solve a novel problem.

[esalman]

I've seen stuff like that too. I asked ChatGPT a tricky geometry question and it got a ratio reversed and ended up producing wrong answer.

I have also tried to use ChatGPT like Google search at the beginning by asking short, curt questions, and got underwhelming results.

The trick is not to try to trick ChatGPT but pretend that you are an idiot and it is a very sophisticated colleague who just needs step by step details. Then you can also generate some of the seemingly amazing results people are getting. For example yesterday I used it to scrape physician information from my insurance website, followed by their Google ratings and so on.

[jengland]

I prompted GPT-4 with this and I think it solved it. The first time it assumed the words were generated with random letters and solved it like a math textbook problem. Then I prompted it with this:

Ok, I see you calculated the probably using randomly generated "words" from the letters of the English alphabet. I am interested in the actual probably of two real words in English that are 5 letters wrong share the first three characters.

I am a Python developer, so I will understand it if you give me a Python script.

I gave me this which looks right to me:

  import nltk
  from collections import defaultdict
  
  nltk.download('words')
  from nltk.corpus import words
  
  # Get the English words
  english_words = words.words()
  
  # Filter the words to get only five-letter words
  five_letter_words = [word for word in english_words if len(word) == 5]
  
  # Create a dictionary to store the count of words with the same first three letters
  words_dict = defaultdict(int)
  
  # Count the words with the same first three letters
  for word in five_letter_words:
      key = word[:3]
      words_dict[key] += 1
  
  # Calculate the number of pairs with the same first three letters
  same_first_three_letter_pairs = sum((count * (count - 1)) // 2 for count in words_dict.values())
  
  # Calculate the total number of pairs from the five-letter words list
  total_pairs = (len(five_letter_words) * (len(five_letter_words) - 1)) // 2
  
  # Calculate the probability
  probability = same_first_three_letter_pairs / total_pairs
  
  print(f"Probability: {probability:.4f} or {probability * 100:.2f}%")