Yule-Walker equation

For the general case AR(p), we can multiply the equation by X_t-k and take the expectation value:

where $\gamma(k)$ is the autocovariance function at lag k.

Since $\gamma(-k) = \gamma(k)$ from (7), we have  

$$\gamma(-k) = \alpha_1 \gamma(-k+1) + \cdots + \alpha_p \gamma(-k+p), \quad \forall k.$$ (9)

This is the Yule-Walker equation.

Replace $\gamma(k)$ by the estimates  

$$\hat{\gamma} (k) = \frac{1}{N} \sum_{t=1}^{N-k} (x_t - \bar{x})(x_{t+k} - \bar{x})$$ (10)

where $\bar{x} = \frac{1}{N} \sum x_t$.Substitute (10) into the (9), we have

and therefore the estimates of AR coefficients $\hat{\alpha_1}, \ldots, \hat{\alpha_p}$can be solved.

Multiply (8) by X_t,

$$X_t^2 = \alpha_1 X_{t-1} X_t + \cdots + \alpha_p X_{t-p} X_t + Z_t X_t$$

and then take the expectation value, we have

$$\sigma_X^2 = \gamma(0) = \alpha_1 \gamma(1) + \cdots + \alpha_p \gamma(p) + \sigma_Z^2$$

and therefore $\hat{\sigma_Z^2}$ can be obtained by replacing $\alpha_k$ and $\gamma(k)$ with the estimates of $\hat{\alpha}_k$ and $\hat{\gamma} (k)$.

Maximum Entropy Method (MEM, or Burg algorithm) is an alternative way to estimate AR coefficients.