Experimental Evaluation in IR: Decades of Tradition not only to Steer Generative AI but also Quantum Computing
In this talk, I will go through the evolution of experimental evaluation in Information Retrieval (IR), namely the system-oriented approach based on the Cranfield paradigm, especially through the lenses of large-scale evaluation initiatives, such as CLEF (Conference and Labs of the Evaluation Forum, https://www.clef-initiative.eu/), in order to show how evaluation is fundamental to drive innovation and development in the field and foster the growth of a vibrant and interdisciplinary community. I will then present some challenges that experimental evaluation has to face when it comes to Generative Artificial Intelligence (AI), the hottest topic everyone is discussing about today. I will also discuss how experimental evaluation can allow us to explore other ground-breaking technlogies and apply them to IR, such as Quantum Computing (QC), by presenting the experience of QuantumCLEF (https://qclef.dei.unipd.it/), in order to remind us that our field should pursue as many innovative directions as possible for its evolution.
Nicola Ferro is Full Professor in Computer Science at the Department of Information Engineering at the University of Padua, Italy. His main research interests are information retrieval, data management and representation, and their evaluation. He chairs the Steering Committee of CLEF, the European evaluation initiative on multimodal and multilingual information access systems, and the Steering Committee of ESSIR, the European Summer School on Information Retrieval. He is Senior PC Member in top-tier conferences, like ECIR, ACM SIGIR, ACM CIKM, WSDM. He is General Co-Chair of SIGIR 2025 and of CIKM 2026. He was General Chair of ECIR 2016 and Associate Editor for ACM TOIS. He was awarded the SIGIR Academy in 2023 and the Tony Kent Strix Award in 2024.
Data Management Systems in the Age of Machine Learning
Database management systems have experienced a very successful story. Conceived initially to support Relational data and SQL queries it was soon extended to support generic UDF functions and different data models. We are now witnessing a new class of applications that integrate data with machine learning and AI models. However, building these ML applications is not easy. In this talk, I will argue that many of the functionalities needed to support ML applications are at the very heart of databases such as query optimization, data transparency, data heteroneity, views etc. As ML applications advance to more complex use of AI, as is the case of GenAI, more opportunities arise to revisit, explore and extend database techniques.
Fabio Porto is a Senior Researcher at the National Laboratory of Scientific Computing, where he coordinates the Data Extreme Lab (DEXL). He holds a DSc and MSc in Informatics from PUC-Rio, and a Mathematics-Informatics undergraduate degrees from UERJ. He was a posdoc at the École Polytéchniqué Fédérale de Lausanne, Swizerland (2004-2008). He currently holds an Inria International Chair (2024-2028) and coordinates the LNCC AI Institute. His main research interests are in ML Systems, Knowledge Graphs and Big Data flows.
European Data Spaces: Catalysts of the EU Data Economy
In recent years, with the aim of boosting and promoting the data economy, the European Union has invested in the creation of Common European Data Spaces. Initiatives such as the DSSC (Data Spaces Support Center) and the CoE-DSC (Centre of Excellence – Data Sharing and Cloud) have been launched to support the construction of Data Spaces. To make this vision viable, reference architectures, protocols and standards (such as the Dataspace Protocol), and various tools are being developed. The goal is to establish a solid foundation for the creation of Data Spaces, addressing crucial challenges such as data interoperability, access and identity management, and data sovereignty. In addition to these technical aspects, organizational, legal and business elements are also taken into account. In parallel, the interaction of Data Spaces with other data-sharing applications—such as DPP (Digital Product Passport), PETs (Privacy Enhancing Technologies) and Digital Twins—has been explored. Across Europe, numerous efforts have been undertaken to ensure that Data Spaces represent an effective solution, offering clear benefits to their participants. In this presentation, we will explore examples of mature European Data Spaces, covering reference architectures, building blocks and essential tools for their creation. We will focus on aspects such as interoperability, the establishment of trust and the preservation of data sovereignty. Finally, we will discuss current challenges and research opportunities in the field.
Bernadette Farias Lóscio is a senior scientist in the Data Ecosystems Department at TNO, an independent Dutch organization and a leader in applied science and technology. She also works as a scientist at the CoE-DSC (Centre of Excellence for Data Sharing & Cloud). Currently, she conducts research and participates in projects in the area of Data Ecosystems, with a focus on Trust Frameworks, Interoperability, and Federation of Data Spaces.
Previously, for 15 years, she was a professor at the Center for Informatics of the Federal University of Pernambuco (UFPE), where she coordinated projects and supervised numerous students in topics such as Data Ecosystems, Semantic Web, and Data Integration. Through consulting, training, authorship of articles, and international recommendations, she has made significant contributions to the advancement of Open Data in Brazil and Latin America.
Bernadette holds a Ph.D. in Computer Science from UFPE and a Master’s degree in Computer Science from the Federal University of Ceará (UFC). Her academic and professional journey reflects a continuous commitment to innovation and the advancement of knowledge in areas related to data use and management.
