AEP with a Twist!

Question

We know by AEP that if random variables $X_1,X_2,...$ are i.i.d. drawn from $P_X$ then the probability of the vectors in the weak typical set $$A_{\epsilon}^n = \{\vec x \in \mathcal{X}^n: |\frac{-1}{n}\log P(\vec x) - H(P_X)| \leq \epsilon \}$$ tends to 1.
I am working on a slightly different problem. Random variables $X_1,X_2,...$ are independently drawn from $P^1_X,P^2_X,..,P^n_X$ respectively. Let use denote $\tilde P_X = \frac{1}{n} \sum_{i=1}^n P^i_X$ (ie, the average of the distributions). Is it true that the sum of the vectors contained in the set $B_{\epsilon}^n$ would have a very high probability? $$B_{\epsilon}^n = \{\vec x \in \mathcal{X}^n: |\frac{-1}{n}\log P(\vec x) - H(\tilde P_X)| \leq \epsilon \}$$

$$ = \{\vec x \in \mathcal{X}^n: |\frac{-1}{n}\sum_{i=1}^n\log P^i_X(x_i) - H(\tilde P_X)| \leq \epsilon \}$$ I have no idea how to prove/disprove this. It seems like I can not use Weak law of large numbers.

Answer 1

Suppose that $X_n = n$ with probability $1$. Then $H(\tilde{P}_X) = \log n$, but the only vector with non-zero probability has $\frac{1}{n} \log \frac{1}{P(\vec{x})} = 0$.