INNER CODE UNIT · Python
outlier_detect_IQR
ashishpatel26/Amazing-Feature-Engineering · feature_cleaning/outlier.py:21
def outlier_detect_IQR(data,col,threshold=3):
'''
outlier detection by Interquartile Ranges Rule, also known as Tukey's test.
calculate the IQR ( 75th quantile - 25th quantile)
and the 25th 75th quantile.
Any value beyond:
upper bound = 75th quantile + （IQR * threshold）
lower bound = 25th quantile - （IQR * threshold）
are regarded as outliers. Default threshold is 3.
'''
IQR = data[col].quantile(0.75) - data[col].quantile(0.25)
Lower_fence = data[col].quantile(0.25) - (IQR * threshold)
Upper_fence = data[col].quantile(0.75) + (IQR * threshold)
para = (Upper_fence, Lower_fence)
tmp = pd.concat([data[col]>Upper_fence,data[col]<Lower_fence],axis=1)
outlier_index = tmp.any(axis=1)
print('Num of outlier detected:',outlier_index.value_counts()[1])