Europe launches ELLIOT to develop open, multimodal AI foundation models for real-world applications

Funded through Horizon Europe, the four-year initiative brings together 30 partners to build general-purpose AI models trained on diverse data for research, industry, and public use.

A European consortium has launched ELLIOT(European Large Open Multi-Modal Foundation Models for Robust Generalization on Arbitrary Data Streams), an initiative designed to develop open-access, general-purpose AI models capable of processing multiple data types. The project is funded by Horizon Europe with a €25 million grant and will run for four years.

Led by the Information Technologies Institute at the Centre for Research and Technology Hellas (CERTH-ITI), the project includes 30 organizations across 12 countries. ELLIOT aims to develop AI systems that can learn from and generalize across various data streams, including video, text, satellite imagery, sensor readings, and industrial time series.

Focus on open infrastructure and reproducibility

ELLIOT’s models are designed to go beyond current limitations in generalization and data modality. The initiative prioritizes open access to both datasets and model architecture, enabling full reproducibility and adaptation for sector-specific applications. Key collaborators include open-source networks such as LAION and open-sci.

The models will be trained using both real and synthetic data drawn from public sources and European Data Spaces. High-performance computing infrastructure, including EuroHPC supercomputers such as LUMI, JUPITER, Leonardo, and MareNostrum, will be used for large-scale training.

Dr Yiannis Kompatsiaris, coordinator of ELLIOT at CERTH-ITI, says: “Through the development of a Virtual Innovation Lab, ELLIOT will not only ensure the widespread adoption of its results by the AI research community but also the uptake by end users in various application domains. For example, multimedia content editors and journalists in media organizations will be able to use ELLIOT models for new media production and live fact-checking.

“Other end users targeted by ELLIOT’s use cases include researchers and scientists working on Earth observation and climate modeling, workers in robot-assisted industrial processes, manufacturers of self-driving cars, engineers dealing with software development and hardware design, researchers that analyze extensive collections of rich media documents and more.”

ELLIOT will explore use cases in sectors such as media, Earth observation, autonomous systems, robotics, and software development. A dedicated Virtual Innovation Lab will support testing and real-world deployment of domain-specific models.

The initiative also includes a framework for addressing regulatory and ethical issues, covering privacy, bias, explainability, and alignment with EU law.

Dr Kompatsiaris adds: This uptake of the fine-tuned domain-specific ELLIOT models by the public and industry will ensure a two-way engagement with society.”