How are data types related to information theory?

I was just reading from wikipedia the following about information:


From the stance of information theory, information is taken as a
sequence of symbols from an alphabet, say an input alphabet χ, and an
output alphabet ϒ. Information processing consists of an input-output
function that maps any input sequence from χ into an output sequence
from ϒ. The mapping may be probabilistic or determinate. It may have
memory or be memoryless.

I'm just guessing here, but is the input alphabet the two states of a bit (0 and 1) and the output alphabet governed by the datatypes of the executing program when those bits are accessed?

Interpretation: The Link Between Data Types and Information Theory

I believe the missing link between these two topics, information theory and data types, is interpretation. Interpretation is essentially the "input-output function" for which output (meaning) is derived from input (data). Whether the interpretation is being performed by a computer, at compile-time or run-time, or a human is relative to the task at hand.

At any rate, data types are language-defined/user-defined interpretations of an input alphabet to an output alphabet. Also, language-defined data types usually have a direct relation to the size of storage locations on a computer.

Probabilistic vs. Deterministic "Mapping"

When determining the output based on a given input, is the output definite (deterministic) or is the output uncertain/ambiguous (probabilistic)?

Memory vs. Memoryless "Mapping"

When determining the output based on a given input, does the determining process require memory?

StackExchange Computer Science Q#10631, answer score: 5