ACM Transactions on Software Engineering and Methodology (TOSEM) 
2023

Impact Factor: 4.4

Tiny, Always-on and Fragile: Bias Propagation through Design Choices in On-device Machine Learning Workflows

Abstract:

Billions of distributed, heterogeneous and resource constrained IoT 
devices deploy on-device machine learning (ML) for private, fast 
and offline inference on personal data. On-device ML is highly 
context dependent, and sensitive to user, usage, hardware and 
environment attributes. This sensitivity and the propensity towards 
bias in ML makes it important to study bias in on-device settings. 
Our study is one of the first investigations of bias in this 
emerging domain, and lays important foundations for building fairer 
on-device ML. We apply a software engineering lens, investigating 
the propagation of bias through design choices in on-device ML 
workflows. We first identify reliability bias as a source of 
unfairness and propose a measure to quantify it. We then conduct 
empirical experiments for a keyword spotting task to show how 
complex and interacting technical design choices amplify and 
propagate reliability bias. Our results validate that design 
choices made during model training, like the sample rate and input 
feature type, and choices made to optimize models, like light-weight 
architectures, the pruning learning rate and pruning sparsity, can 
result in disparate predictive performance across male and female 
groups. Based on our findings we suggest low effort strategies for 
engineers to mitigate bias in on-device ML.


Pre-camera PDF 

ACM Library Open Access

@article{Hutiri:TOSEM23,
author = {Hutiri, Wiebke (Toussaint) and Ding, Aaron Yi and Kawsar, Fahim and Mathur, Akhil},
title = "Tiny, Always-on and Fragile: Bias Propagation through Design Choices in On-device Machine Learning Workflows",
journal = "ACM Transactions on Software Engineering and Methodology",
year = "2023",
  volume={32},
  number={6},
  issn = {1049-331X},
  articleno = {155},
url = {https://doi.org/10.1145/3591867},
doi = {10.1145/3591867}
}

Wiebke Toussaint Hutiri, Aaron Yi Ding, Fahim Kawsar, Akhil Mathur. "Tiny, Always-on and Fragile: Bias Propagation through Design Choices in On-device Machine Learning Workflows", in ACM Transactions on Software Engineering and Methodology, Volume 32, Issue 6, 2023.