patternMinor
Conditions to apply Source Coding Theorem
Viewed 0 times
sourceapplytheoremconditionscoding
Problem
I was wondering what are the conditions to apply source coding theorem (SCT).
I was asking because in my lecture, SCT is only discussed in uniform coding section.
In Wikipedia SCT, it's said $f$ is unique decodable code, then we have SCT. In my lecture, it gives me an impression that SCT is generally applied to all lossless coding.
- Is it applied only to uniform-length coding, what about variable-length coding, does it also satisfy SCT?
I was asking because in my lecture, SCT is only discussed in uniform coding section.
- Is the result of SCT for uniquely decodable coding or is it for lossless coding?
In Wikipedia SCT, it's said $f$ is unique decodable code, then we have SCT. In my lecture, it gives me an impression that SCT is generally applied to all lossless coding.
Solution
-
It is an average in bits/symbol and does apply to all coding schemes.
-
You're correct, the number of bits/symbol needed is equal or larger than the entropy rate (asymptotically, though) of the source for any lossless code.
-
If you allow for lossy coding, then a generalization called rate-distortion function is needed to lower bound the rate required.
It is an average in bits/symbol and does apply to all coding schemes.
-
You're correct, the number of bits/symbol needed is equal or larger than the entropy rate (asymptotically, though) of the source for any lossless code.
-
If you allow for lossy coding, then a generalization called rate-distortion function is needed to lower bound the rate required.
Context
StackExchange Computer Science Q#83222, answer score: 2
Revisions (0)
No revisions yet.