[rndphs]

>When a person "has the concept `7`" they can reason thus: "6 eggs would be fewer", "7 is an odd number", "7 is a whole quantity", "with 7 less of 10, i'd have 3" etc.

I just input this into GPT-3. Its responses are in italics, this is first try no rewriting or retrying anything:

This is a test of understanding of the concept of "the number 7".

Question: Is 7 odd? Answer: Yes, 7 is an odd number.

Question: Is 6 eggs fewer than 7 eggs? Answer: Yes, 6 eggs is fewer than 7 eggs.

Question: With 7 less of 10, what would I have? Answer: I would have 3.

Question: Is 7 a whole quantity? Answer: Yes, 7 is a whole quantity.

This is mostly a joke because I think I understand where you are coming from (and that you are hypothesising that gpt3's responses are an elaborate trick of sorts). But I don't believe AI has to take the same route as human intelligence, and I don't think we really understand what a concept is or how it behaves from a signal/data perspective, but I think that may be inconsequential for creating general AI.

Also people are can be really stupid sometimes and also have failures, and the concepts that people hold can be incorrect or flawed etc. So it may be useful also to compare human failures with AI failures, rather than just AI failures with human successes.