Sine-Fitting Residual Root Mean Square, Mean, and Variance in the Presence of Phase Noise or Jitter

Fitting a sinusoidal model to a set of data points is a common practice in engineering, where one wants to estimate some quantities of interest by carrying out a sequence of measurements on a physical phenomenon. Analytical expressions are derived for the statistics of the root mean square value of the residuals from the least-squares sine-fitting procedure, when the data points are affected by phase noise or sampling jitter. The two analytical expressions derived, for the mean and for the variance, are numerically validated using a Monte Carlo-type procedure with simulated data for varying amounts of noise present, a varying number of data points, and varying signal amplitude. It will be shown that there is an excellent agreement between the numerical values obtained and those given by the analytical expressions proposed. These can be of use to engineers who need to compute confidence intervals for their estimations or who need to choose the number of signal data points that should be acquired in a given application.