Large-scale outlier detection for low-cost PM sensors

Evaluating the air quality of classrooms is important as children spend a large amount of time at school. Massey University (NZ) led the development of a low-cost and affordable Indoor Air Quality (IAQ) platform called SKOMOBO that was deployed on a large scale across the classrooms of primary schools in New Zealand. When the data from SKOMBO units were collected, it was important to detect any unexpected high air pollution events. To address this concern, we propose a study of outlier detection for PM10 dataset from SKOMOBO units using MSD-Kmeans. MSD-Kmeans combines the statistical method of Mean and Standard Deviation (MSD) with the machine learning clustering algorithm K-means where the former eliminates as many noisy data to minimize the inference on clustering while the latter is able to achieve better local optimal clustering. We compare the performance of MSD-Kmeans with other similar outlier detection algorithms. Our experimental results illustrate that MSD-Kmeans outperforms the majority of performance indicators (e.g., TPR, FPR, Accuracy, F-measures) compared to other similar methods. We conclude that it is feasible to use MSD-Kmeans as an effective outlier detection tool on large scale datasets.