One of the earliest techniques to obtain reliable statistical estimators is the jackknife technique. It requires less computational power than more recent techniques.
Suppose we have a sample and an estimator . The jackknife focuses on the samples that leaves out one observation at a time:
for , called jackknife samples. The ith jackknife sample consists of the data set with the ith observation removed. Let be the ith jackknife replication of .
The jackknife estimate of s.e. defined by
The jackknife only works well for linear statistics (e.g., mean). It fails to give accurate estimation for non-smooth (e.g., median) and nonlinear (e.g., correlation coefficient) cases. Thus improvements to this technique were developed.