How is standard deviation affected by outliers

What you notice in Figure 3. In this instance, the IQR is the preferred measure of spread because the sample has an outlier. Breadcrumb Home 3 3. Example 3. Measures of Spread or Variation Recall the five-number summary from Example 3. A time-series outlier need not be extreme with respect to the total range of the data variation but it is extreme relative to the variation locally.

For this outlier detection method, the mean and standard deviation of the residuals are calculated and compared. If a value is a certain number of standard deviations away from the mean, that data point is identified as an outlier. The specified number of standard deviations is called the threshold. The default value is 3. This method can fail to detect outliers because the outliers increase the standard deviation.

The more extreme the outlier, the more the standard deviation is affected. We see that the median represents the typical income of people in this sample better than the mean.

The small number of people with higher incomes does not impact the median or the other quartile marks, so the first and third quartile marks give a range of incomes that more accurately represent typical incomes in the sample.

Notice also that this range is always within the overall range of the data, so we will never have the problem that we encountered earlier with the standard deviation.

In a skewed distribution, the upper half and the lower half of the data have a different amount of spread, so no single number such as the standard deviation could describe the spread very well. We get a better understanding of how the values are distributed if we use the quartiles and the two extreme values in the five-number summary. Both of these examples also highlight another important principle: Always plot the data.

We need to use a graph to determine the shape of the distribution. By looking at the shape, we can determine which measures of center and spread best describe the data. Privacy Policy. A single extreme value can have a big impact on the standard deviation.

Standard deviation might be difficult to interpret in terms of how large it has to be when considering the data to be widely dispersed. The magnitude of the mean value of the dataset affects the interpretation of its standard deviation. This is why, in most situations, it is helpful to assess the size of the standard deviation relative to its mean. The reason why standard deviation is so popular as a measure of dispersion is its relation with the normal distribution which describes many natural phenomena and whose mathematical properties are interesting in the case of large data sets.

When a variable follows a normal distribution, the histogram is bell-shaped and symmetric, and the best measures of central tendency and dispersion are the mean and the standard deviation.

Confidence intervals are often based on the standard normal distribution.