AI-DAPT: AI-Ops Framework for Automated, Intelligent and Reliable Data/AI Pipelines Lifecycle with Humans-in-the-Loop and Coupling of Hybrid Science-Guided and AI Models

Today, Artificial Intelligence (AI) has paved a long way since its inception and has started experiencing exponential growth across various industries and shaping our world in ways that were once thought impossible. As AI transitions from research to deployment, leveraging the appropriate data to develop and evaluate AI models has evolved into one of its greatest challenges. Data are in fact the raw material and the most indispensable asset fuelling much of today’s progress in AI, generating previously unattainable insights, assisting more evidence-based decision-making, and bringing tangible business/economic benefits and innovation to all involved stakeholders. However, despite their instrumental role in determining performance, fairness, and robustness of AI systems, data are paradoxically characterised as the most under-valued and de-glamorised aspect of AI while a data-centric focus is typically lacking in the current AI research.

AI-DAPT aims to deliver an innovative and impactful research agenda that will provide tangible benefits to a variety of stakeholders that struggle with making AI services. Seeking to reinstate the pure data-related work in its rightful place, and reinforcing the generalizability, reliability, trustworthiness, and fairness of Al solutions, AI-DAPT vision relies on the implementation of an AIOps framework to support and automate AI pipelines that continuously learn and adapt based on their context. It enables proper purposing, collection, documentation, (bias) valuation, annotation, curation and synthetic generation of data, while keeping humans-in-the-loop across five axis: (i) Data Design for AI, (ii) Data Nurturing for AI, (iii) Data Generation for AI, (iv) Model Delivery for AI, (v) Data-Model Optimization for AI.

AI-DAPT brings forward a two-fold data-centric mentality in AI:
(a) Data: AI-driven automation for data pipelines based on Explainable AI (XAI) techniques as well as synthetic data generation and observability.
(b) Model: Automation on AI model building and hybrid science-AI solutions, bringing together data-driven AI models and science-based (first-principles) models that build on high-quality data.

Bridging the gap between data-centric and model-centric AI, AI-DAPT will turn over a new leaf in trustworthy AI and will nurture an ecosystem involving all AI and data value-chain stakeholders. The aim is to enhance their prosperous collaboration in order to deliver and apply innovative AI-driven methods that rely on smart and dynamic end-to-end automation of data, AI training/inference pipelines in the cloud-edge computing continuum.

To demonstrate the actual innovation and added value that can be derived through the AI-DAPT scientific advancements, the AI-DAPT results will be validated in two ways:
(a) By applying them to tackle real-world challenges in four key industries: (4) Health, Robotics, Energy, and Manufacturing.
(b) By integrating them into various AI solutions, whether open source or commercial, already present in the market.