Expected value of a the estimate of Bernoulli distribution

I know that $E[x] = p$ for some Bernoulli random variable X with parameter $p$ where $P(X = 1) = p$ and $P(X = 0) = 1 − p$. Similarly $E[X^2] = p$ and $Var[X] = p − p^2 = p(1 − p)$.

However, I'm having trouble wrapping my head around the following. Let's say $X_i ∼ Bernoulli(p)$. I was told that we can determine the value of $p$ by using some number $t$ of i.i.d. samples$X_1, ... , X_t ∼ Bernoulli(p)$. What is the expected value of the estimate?

1. My intuition isn't quite clear here. My initial attempt it is, let us say we have a coin that lands on heads with $p$ and tails with $1-p$. Then, we can flip it some $t$ times, and if we do something like $\frac{\text{# of heads}}{\text{total # of slips}}$ or equivalently $X = \frac{1}{t}\sum^{t}_{i=1}X_i$, then we can estimate $p$. And as $t$ gets larger, the stronger our estimate will be. Is this the right intuition here?

Secondly, if my intuition is correct, then what exactly is the expected value and variance of $X$ in terms of $p$ and $t$. I realize this might be an obvious question, but I'm having trouble differentiating it from just $E[x] = p$.

Am I just suppose to be doing $E[X] = E[X_1] + E[X_2] ...$? How do I put it in terms of $p$ and $t$.

1 Answer

You apply the Linearity of Expectation.

Since the mean estimator is indeed $\bar X=\tfrac {\sum_{i=1}^tX_t}t$ then the expectation is:-

$$\begin{align}\mathsf E(\bar X) &= \mathsf E\left(\tfrac 1 t\sum_{i=1}^t X_i\right)\\[1ex]&=\tfrac 1t\sum_{i=1}^t\mathsf E(X_i)\end{align}$$

The rest is that these $t$ samples are identically distributed, and for all $t$ samples their expectation is $p$.

So, yeah, $\mathsf E(\bar X)=p$

Similarly for the variance of the mean estimator, we also use independence (and so the variance of the sum equals the sum of variance):- $$\begin{align}\mathsf{Var}(\bar X)&=\mathsf {Var}(\tfrac 1t\sum_{i=1}^t X_i)\\[1ex]&=\sum_{i=1}^t\mathsf{Var}(\tfrac 1{t}X_i)\\[1ex]&~~\vdots\end{align}$$

Recall we have for any scalar $a$ and random variable $Z$, that: $\mathsf {Var}(aZ)=a^2\mathsf{Var}(Z)$