2

Why is Shannon Entropy of individual sections always between 0-8. Also why we need to create a 256 freq array while calculating the Shannon Entropy?

2
2

The channel capacity of a single byte samples has a maximum of 8 bits.

Another way of thinking about it: If a single byte has the same value for every sample, you need 0 additional bits of information to describe the values.

If a single byte takes on 256 different values, [0 to 255], then for every sample you will need 8 additional bits of information to uniquely describe the values.

For example, if you were measuring the Shannon Entropy of a collection of Short values (2 bytes / 16 bits) it would range from 0 (constant) to 16 (completely random).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.