Search Search Any Topic from Any Website Search
Let 𝐻(𝑋) denote the entropy of a discrete random variable 𝑋 taking 𝐾 possible distinct real values. Which of the following statements is/are necessarily true?
Q. Let 𝐻(𝑋) denote the entropy of a discrete random variable 𝑋 taking 𝐾 possible distinct real values. Which of the following statements is/are necessarily true? (A) 𝐻(𝑋)≤logH𝐾 bits (B) 𝐻(𝑋)≤𝐻(2𝑋) (C) 𝐻(𝑋)≤𝐻(𝑋^2) (D) 𝐻(𝑋)≤𝐻(2^X) Answer: Option 1,2, and 4 Solution: Option A: (Correct) if K symbols having equal probability the maximum entropy = log2(K); other wise 𝐻(𝑋)≤logH𝐾 bits Option B: (Correct) -plog2p ≤ -2plog2p Option C: (false) Think about X^2 = 4 then. X = 1,-1. Whenever different outcomes merge into one outcome, entropy decreases. Option D: (Correct) If every value of 2^X produces a unique output, the probabilities are just relabeled. The uncertainty is unchanged, so the entropy stays the same. If exactly one value of XXX maps to yyy, the probability stays the same. If two or more values of XXX map to yyy, you add their probabilities. GATE EC Previo...