IEEE ICDCS 2024 (acceptance rate 21%)

The SPATIAL Architecture: Design and Development Experiences from 
Gauging and Monitoring the AI Inference Capabilities of Modern Applications 

Abstract:

Despite its enormous economical and societal impact, lack of human-perceived 
control and safety is re-defining the design and development of emerging 
AI-based technologies. New regulatory requirements mandate increased human 
control and oversight of AI, transforming the development practices and 
responsibilities of individuals interacting with AI. In this paper, we 
present the SPATIAL architecture, a system that augments modern applications 
with capabilities to gauge and monitor trustworthy properties of AI inference 
capabilities. To design SPATIAL, we first explore the evolution of modern 
system architectures and how AI components and pipelines are integrated. With 
this information, we then develop a proof-of-concept architecture that 
analyzes AI models in a human-in-the-loop manner. SPATIAL provides an AI 
dashboard for allowing individuals interacting with applications to obtain 
quantifiable insights about the AI decision process. This information is then 
used by human operators to comprehend possible issues that influence the 
performance of AI models and adjust or counter them. Through rigorous 
benchmarks and experiments in real-world industrial applications, we 
demonstrate that SPATIAL can easily augment modern applications with metrics 
to gauge and monitor trustworthiness, however, this in turn increases the 
complexity of developing and maintaining systems implementing AI. Our work 
highlights lessons learned and experiences from augmenting modern applications 
with mechanisms that support regulatory compliance of AI. In addition, we also 
present a road map of on-going challenges that require attention to achieve 
robust trustworthy analysis of AI and greater engagement of human oversight.


Pre-camera PDF 

IEEE Library Access

@INPROCEEDINGS{Ottun:ICDCS2024, 
author={Ottun, Abdul-Rasheed and SPATIAL, Team and Ding, Aaron Yi and Flores, Huber}, 
booktitle={44th IEEE International Conference on Distributed Computing Systems (ICDCS)}, 
title={The SPATIAL Architecture: Design and Development Experiences from Gauging and Monitoring the AI Inference Capabilities of Modern Applications},
year={2024}
}

Abdul-Rasheed Ottun, SPATIAL Team, Aaron Yi Ding, Huber Flores, "The SPATIAL Architecture: Design and Development Experiences from Gauging and Monitoring the AI Inference Capabilities of Modern Applications", in Proceedings of the 44th IEEE International Conference on Distributed Computing Systems (ICDCS), 2024.