[soyiuz]

I have several major problems with this explanation (often endemic in the discussion more generally):

1. Shannon's original paper relates "the capacity [of a channel] to transmit information" as well as the potential of a system (e.g. the English language) to generate information. In other words, your pipe needs to be able to accommodate the potential volume coming from the source ("the entropy of the source determines the channel capacity"). The amount of information in a message or the amount of surprise, as the author has it, should not be confused with the amount of potential information.

2. Instead of "system" and "channel" the author uses the word "variable" which I find misleading. Channel (like a telegraph cable) and system (the English language) are specifically relevant to Shannon's discussion.

3. The discussion of "surprise," as the author has it, is misleading. Shannon is writing his paper in conversation with Hartley and Nyquist---all three specifically attempting to bracket out the subjective psychological factors such as surprise, in order to describe the capacity for information transmission in terms of quantitative measures, based on "physical considerations alone" (Hartley). Surprise reintroduces a subjective, relative, psychological understanding of information the original authors wanted to avoid.

[memetomancer]

With all due respect, you seem to be saying that this 'introductory text' is insufficiently sophisticated, and proceed to dump a high density rebuttal in stilted, academic style.

You may be extra smart in the sense of understanding the topic in a deep way, but it sure does seem foolish and/or myopic to posture this way over a basic introduction that cites the original paper in the first sentence.

[dswilkerson]

(1) A "Random Variable" is a term of art in probability theory and has an entropy.

(2) Your distinction between "capacity" and "potential" is pointless: you have to budget for the expected information you have to transmit.

(3) Your silly games with words would apply to any technical discussion as they all use metaphors. You sound like someone who has never done any technical work at all.

[trombonechamp]

"Surprise", or rather "specific surprise", is actually a technical term in information theory, e.g., see: https://iopscience.iop.org/article/10.1088/0954-898X/10/4/30...

This doesn't seem to be how the author is using it in this arxiv paper though.

[superfist]

1. You are correct, there are two different perspectives: 1. designer of channel (what most of Shannon work is about) and it deals with what you called 'potential information' and 2. message receiver/sender perspective.

2. True

3. Very true, using psychological terms like 'surprise' or in general psychological terms in strict theories is very often misleading but tempting becasue any reader can always contribute something (like own interpretation) to such theories so later he is personly/emotionaly bounded to it more.