Call for Posters / Demos

In total, we received 27 submissions for the Call for Posters / Demos. The programme committee descided to accepted 22 submissions. All abstracts have been reviewed by the programme committee / review board based on the following review criteria – rating from 0 (very low) to 10 (very high):

• Topicality (15%)
• Thematic Relevance (15%)
• Practical relevance (15%)
• Presentation and Language (15%)
• Overall recommendation (40%)

The possible maximum average score is 100.

(1) RADAR – Enhancing FAIR Research Data Management with AI Support
Felix Bach, Kerstin Soltau, Stefan Hofmann
Organization(s): FIZ Karlsruhe – Leibniz-Institut für Informationsinfrastruktur
Average reviewers ratings: 95.7 (out of 100)

Abstract:

RADAR, developed and operated by FIZ Karlsruhe, is a well-established research data repository supporting secure archiving, publication, and long-term preservation of data across disciplines. Since its launch in 2017, RADAR has continuously evolved to meet the growing demands of open science. It offers comprehensive metadata support, persistent identifiers, semantic enrichment (e. g. Schema.org, FAIR Signposting), discipline-specific terminologies via TS4NFDI, and integration with platforms such as GitHub and GitLab. Flexible deployment options (RADAR Cloud, RADAR Local) and tailored services (e. g. RADAR4Chem, RADAR4Memory) ensure broad usability and community alignment.

As part of our ongoing innovation efforts, we are currently exploring AI-driven enhancements that further support FAIR data practices. These include:

• AI-assisted metadata review and enrichment, enabling e. g. the automatic extraction of relevant metadata from documents linked within submissions;
• AI-assisted FAIRness checks, offering feedback and suggestions to improve the FAIRness of datasets.

These developments aim to help researchers meet growing expectations for quality metadata and data stewardship while reducing manual effort. A live notebook demo will be available to trial our AI-enhanced features, currently in testing. Our poster, which uses a timeline to illustrate RADAR´s evolution and feature set, invites discussion on AI, FAIR data, open science, and future-ready repository services.

Review 1
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 8 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 97
The integration of AI-assisted metadata as well as the FAIRness check is described sound and overall it seems comprehensible.

Review 2
Topicality 8 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 90
The poster and demonstration of the practices by the RADAR team to improve FAIRness and enriched metadata in an AI-assisted approach can be very relevant and beneficial for the community as showcase for such solutions.

Review 3
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 100
Your contribution is more than welcomed as practical tools to help FAIRifying data are really needed by the community.

(2) Helping SSH Researchers to Explore and Exploit Knowledge Graphs through Artificial Intelligence: the GRAPHIA Project
Matteo Romanello (1), Luca De Santis (2), Julien Homo (3), Stefano de Paoli (4), Sy Holsinger (5)
Organization(s): 1: Odoma; 2: Net7; 3: FoxCub; 4: Abertay University; 5: OPERAS
Average reviewers ratings: 92.0 (out of 100)

Abstract:

Many research institutions worldwide have invested over the years in the creation of knowledge graphs (KGs) with deep semantic descriptions of their data. Despite the semantic richness of this information, its accessibility and usefulness for researchers remain limited due to the complexity of their ontologies and the specialized query languages required to retrieve information (e.g., SPARQL) [1]. Helping researchers in social sciences and humanities (SSH) to more easily access information contained in KGs is one of the aims of the GRAPHIA project, a recent EU-funded project, which aims not only to create a comprehensive KG for SSH, but also to exploit large language models (LLMs) as a means to access and analyse the information contained in them.

In GRAPHIA, we will expand the current implementations of AI done in the context of GoTriple, a massive repository holding more than 19 million SSH publications. The first AI implementation is the GoTriple ChatBot [2], a RAG system (Retrieval Augmented Generation) fed with the full text of selected documents to answer user prompts. As a complete RAG indexation of GoTriple turned out to be too costly, a new implementation allows users to create notebooks starting from GoTriple documents, and then pass them to the RAG system for AI-assisted analysis. Finally, GRAPHIA partners’ experiments with AI include the usage of LLMs to convert natural language questions into structured queries against KGs, and the usage of the Model Context Protocol (MCP) to empower AI-assisted interactions with massive KGs.

[1] http://dx.doi.org/10.3233/SW-223117 

[2] http://dx.doi.org/10.5281/zenodo.10977163 

Review 1
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 100
The abstract is clear and well structured. The topic is well placed in the intersection between OS and AI. In addition, the topic is relation to disciplines (SSH) that often are undervalued when discussing OS practices.

Review 2
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 100
This poster addresses an important and timely sociotechnical problem: how to render the rich but complex infrastructures of knowledge graphs accessible and meaningful to SSH researchers through AI. The topic speaks directly to ongoing debates in Open Science about equitable access to data and tools, and fits well with the conference’s focus on infrastructures and the synergy between AI and Open Science. The practical implementations (and limitations) outlined — from the RAG chatbot to MCP-mediated KG querying — promise a hands-on demonstration of how such infrastructures can be navigated and appropriated by researchers. It would thus also be valuable if the poster could critically reflect on the current limitations and challenges of these approaches, such as cost, scalability, or epistemic assumptions embedded in the tools, to foster a more nuanced discussion.

Review 3
Topicality 8 (15%)	Thematic Relevance 8 (15%)	Practical relevance 10 (15%)	Presentation and Language 6 (15%)	Overall Recommendation 7 (40%)	Total points (out of 100) 76
While I understand the problem set out in the first paragraph, I don’t yet fully understand the status quo presented presented in the second paragraph. It suggests that there is already an AI implemented in the GoTriple context (also by the poster cited from 2024). But I can’t find the Chatbot online. Nevertheless, the demonstration/poster presents a good opportunity to discuss the case and the intermediate results with the community and to discuss the options of linking available (not always clean data in GoTriple) to AI-supported knowledge analysis and particularly the potentials of knowledge graphs exploited by AI. Stefan Skupien, Open Science Coordinator, Berlin University Alliance

(3) AI-assisted research data annotation in biomedical consortia
Felix Engel, Gita Benadi, Claudia Giuliani, Harald Binder, Klaus Kaier
Organization(s): Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center – University of Freiburg, Germany
Average reviewers ratings: 90.5 (out of 100)

Abstract:

Annotation of research data is a key element of Open Science and has gained additional value as training input for artificial intelligence. However, developing metadata schemas poses a series of challenges, including optimisation and securing both complete coverage and constant completeness and quality. We employ large language models (LLMs) to address some of these challenges while keeping researchers in the loop to ensure reliability of annotations.

Our research data management group currently supports seven biomedical research consortia. We develop customised metadata schemas together with consortium members, drawing on established controlled vocabularies (Engel et al. 2025). Schemas are implemented on the fredato research data platform developed at the IMBI (Watter et al. 2023). Schemas are documented and published as knowledge graphs adhering to the Resource Description Framework (RDF), relating metadata to research processes as modelled by commonly used ontologies.

LLMs are employed to develop initial schema drafts from related research literature and to predict dataset annotations from scientific papers (Giuliani et al. 2025). The models have proved to perform well with these tasks, supporting researchers with improving metadata coverage in their consortia.

References

Engel, F. et al. (2025). Development of Metadata Schemas For Collaborative Research Centers. FreiData. https://doi.org/10.60493/K1XE3-NPC10 

Giuliani, C. et al. (2025). Identifying biomedical entities for datasets in scientific articles – A 4-step cache-augmented generation approach using GPT-4o and PubTator 3.0. medRxiv. https://doi.org/10.1101/2025.03.04.25323310 Watter, M. et al. (2023). Standardized metadata collection in a research data management tool to strengthen collaboration in Collaborative Research Centers. E-Science-Tage, Heidelberg. https://doi.org/10.11588/HEIDOK.00033131

Review 1
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 96
The authors address a core issue in making research data FAIR  – the annotation by metadata. The presentation of their solution (based on Large Language Models and Knowlege Graphs) can be very beneficial for the community. I would assume even beyond of the biomedical field as the challenges are universal in research.

Review 2
Topicality 8 (15%)	Thematic Relevance 10 (15%)	Practical relevance 8 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 83
Overall, the abstract addresses an important topic at the intersection of open science and AI tools. I do believe the proposal could be strengthened by clearly articulating what someone would learn by visiting the poster or what question the poster seeks to answer, beyond a workflow description. I’m sure your team has lots of lessons learned that attendees would love to hear about!

Review 3
Topicality 8 (15%)	Thematic Relevance 8 (15%)	Practical relevance 8 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 83
The abstract is clear and well written. The topic is relevant for the conference and cover issues related to OS and AI. In the presentation, it will be important to showcase the challenges and the barriers in implementing this annotation system, especially in relation to OS.

Review 4
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 100
A very strong submission and highly relevant to the theme of this conference. The authors describe an improved process to use LLMs to annotate large datasets. The improvement comes from the LLM output being both constrained (limited to specific scientific papers that are provided as input together with the prompt) and verified by human experts. For this reviewer (not an expert in AI/ML), the references (pre-prints) were very useful and provided some good supporting information. Perhaps one suggestion for the final presentation would be to include some of the longer, step-by-step and more non-expert-friendly explanations of key aspect of this work, to help the audience grasp both the mechanism and the impact.

(4) A Human-Centered Open Source Platform for Image and Text Processing in Humanities Research
Ari Häyrinen
Organization(s): University of Jyväskylä
Average reviewers ratings: 90.5 (out of 100)

Abstract:

We present an open source application designed to support non-technical users in the fields of humanities and social sciences by enabling intuitive processing of visual and textual data. Functioning as an ETL-like pipeline tool, the platform integrates multiple OCR engines and supports the use of large language models (both open source and proprietary) to analyze and enrich image and text datasets. Built with simplicity and openness at its core, the tool fosters accessible, transparent, and reproducible research workflows.

This contribution addresses the intersection of AI and Open Science by (1) lowering the technical barrier for deploying LLMs in scholarly analysis, (2) supporting the use of open source models and tools, and (3) encouraging data sharing and methodological transparency. By prioritizing usability and interoperability, the application contributes to training and capacity-building efforts in digital humanities and enhances the responsible adoption of AI technologies within open research infrastructures.

The poster will demonstrate real-world use cases, showcase how the platform enables explainable AI workflows for humanistic inquiry, and invite collaboration for extending its modular design.

Github: https://github.com/OSC-JYU/MessyDesk 

Review 1
Topicality 8 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 94
From what I get in the concise description of the abstract, the poster addresses a topic which is highly relevant for the conference and the job seems well documented.

Review 2
Topicality 8 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 86
This submission is of relevance for the conference and clearly written. I am wondering how you define reproducibility for LLM based descriptions and annotations. From what I understand, repeat prompting would change the output. How would you capture current states to reproduce the same output? Or are you drawing the line of reproducibility somewhere else? Process reproducibility would in any case be increased as far as I can tell. On your github repo/ the README, you have a little graph showing how a description is added to a picture. You might want to consider using a different LLM as example or making sure people understand at first glance that you dont have to integrate gemini but that there are options (including open source models). Please consider spelling out ETL and OCR in the final version of the abstract: “Functioning as an ETL-like pipeline tool, the platform integrates multiple OCR engines” Good luck, Daniela Gawehns NLRN

Review 3
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 96
This poster addresses a timely question: how to support humanities and social science researchers with an “intuitive” open source platform for AI-based text and image processing, while maintaining reproducibility, openness, and transparency. The aim to lower technical barriers and foster accessible workflows is clearly relevant to the conference themes. I am curious to learn more about what is meant by “intuitive” in this context — how the design ensures that non-technical scholars retain enough critical room to make informed decisions and maintain transparency over the process. Clarifying how the tool balances ease of use with methodological rigor and interpretability would strengthen the contribution and make it even more compelling – and discussing the limitations!

Review 4
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 6 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 86
Interesting and unorthodox applicaiton of LLMs to support reproducible science

(5) Enhancing Discovery, Policy, and Practice: A Hands-On Demo of the European Open Science Resources Registry
Tereza Szybisty, Natalia Manola, Stefania Martziou, Antonis Lempesis
Organization(s): OpenAIRE AMKE
Average reviewers ratings: 88.5 (out of 100)

Abstract:

The EOSC Open Science Observatory is a policy intelligence platform that tracks and visualizes the progress of Open Science across Europe. It offers stakeholders—from policymakers to practitioners—access to trustworthy data on Open Science policies, practices, and impacts.

This demo will offer a hands-on exploration of the EOSC Open Science Observatory dashboard, guiding participants through its latest data visualization features. Participants will learn to navigate the platform’s new features that visualize key OS dimensions—from Open Access to FAIR data, infrastructure, and skills.

A highlight of the session will be the European Open Science Resources Registry, an integrated component of the EOSC Open Science Observatory. This AI-enhanced registry curates essential documents—including policies, strategies, and best practices—and supports advanced search and classification using Natural Language Processing (NLP) and Machine Learning (ML).

Participants will discover how these technologies automate document retrieval, extract metadata, classify content, and produce summaries—enhancing discoverability and understanding of Open Science efforts across Europe.

After this demo, participants will:

• learn to explore and interpret Open Science indicators using the EOSC OS Observatory
• understand how the European Open Science Resource Registry streamlines access to key policy resources
• get involved in the discussion how AI tools support smarter, more connected Open Science governance

This demo bridges technology and transparency, showcasing how AI and open infrastructures can reinforce the core principles of Open Science.

Review 1
Topicality 8 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 90
In its commendable efforts to improve the user experience of the EOSC Observatory, the team proposes demonstrating how AI and registry approaches can be integrated to provide quick, comparable overviews of different categories and content. However, the demonstration should also include a short section on the provenance and quality of the data used, as well as an explanation as to why the latest data is from 2022 despite annual surveys being planned as part of the methodology. The demonstration could also facilitate discussion on the use of open-source AI models in conjunction with open infrastructures (the abstract does not specify the AI model used). Stefan Skupien, Open Science coordinator, Berlin University Alliance

Review 2
Topicality 8 (15%)	Thematic Relevance 8 (15%)	Practical relevance 8 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 87
It is very well explained what the learning goals of the demo should be. AI aspects does not seen the main focus but supporting the approach.

(6) Bridging Open Science and large language models: Enhancing research accuracy through Knowledge graphs
Nicolaus Wilder, Marie Alavi, Julia Priess-Buchheit
Organization(s): Kiel University
Average reviewers ratings: 88.3 (out of 100)

Abstract:

Open Science (OS) emphasises reproducibility, factual accuracy and originality to promote responsible conduct of research, share reliable data, minimise resource waste, and foster innovation.

In contrast, large language models (LLMs) process vast amounts of (non-) scientific data by probabilistic modeling, prioritising quantity over reliability of data.

As LLMs are increasingly used in research, a critical question arises: How can the two different logics (1)—efficiency through openness and efficiency through volume—coexist and be utilised responsibly in research? We expand this question and argue for knowledge-augmented systems to enhance LLMs’ accuracy and reliability, proposing a combination with an OS knowledge graph (KG).

The poster visualises a triangular relationship among OS, KG, and LLM, highlighting two workflows.

(A) OS—KG—LLM: OS resources serve as the knowledge base structured within a KG ontology (2), connected upstream of an LLM, providing a constantly updated reference that guides the LLM’s data selection through specific nodes (entities) and branches (relationships), mitigating hallucinations and delivering contextually relevant content aligned with the prompt.

(B) OS—LLM—KG: OS resources are automatically prepared with the help of an LLM, which identifies nodes and relationships, extracts relevant data from the OS pool, and transforms publications into KG-compliant data. Thus, LLMs can assist in creating an ontology represented in a KG based on the principles, data, and results of OS.

References:

(1) https://doi.org/10.5281/zenodo.11562117 

(2) https://doi.org/10.48550/arXiv.2406.08223

Review 1
Topicality 8 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 90
The poster touches the relationship of large language models and knowledge graph in the light of Open Science principles and by that fits perfectly into the overall topic of the conference.

Review 2
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 8 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 90
The submission outlines an interesting and impactful way to combine Open Science, Knowledge Graphs, and Large Language Models. The submission/poster may further reflect on the circumstances under which each workflow would be preferable and explore how the two proposed workflows could complement each other.

Review 3
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 6 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 83
I think you are asking exactly the right questions but it is not entirely clear to me what you poster achieves beyond that — maybe highlight that concrete practical contribution a little more if it is there. Also, to avoid confusion, I would rather state your conclusion as “with the current amounts of open access material and the current practices we do not have enough training material” because a devil’s advocate could always say something like “maybe you have just not found the right way to make use of the amounts you have” etc.

Review 4
Topicality 8 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 90
This poster makes the point that the accuracy of LLMs in research can be improved by combining them with OS knowledge graphs. This is a dense topic that requires quite some previous knowledge on LLMs and knowledge graphs (KG). The strength of this poster is the idea to improve LLM use and reliability by the combination with OS KGs that will hopefully address critical points that arise when this kind of AI is used in research. As LLMs are not trained on accurcary and retaining correct information I wonder how KGs are helpful in shaping the statistics in favour of correct information output. Question I have are: will new (or already existing?) LLMs be trained as proposed by (A)? How does quality control look like for (A) and (B)? What is the time frame for such an endeavour? I liked the links that point to further in-depth reading to the topic and hope to see more resources on the final poster itself.

(7) Open Data and Open LVLMs: How to Explore Scientific Collections Differently
Iris Vogel (1), Florian Schneider (2), Narges Baba Ahmadi (2), Niloufar Baba Ahmadi (2), Chris Biemann (2), Martin Semmann (2), Kai Wörner (1)
Organization(s): 1: Center for Sustainable Research Data Management, University of Hamburg; 2: Hub of Computing and Data Science, University of Hamburg
Average reviewers ratings: 85.5 (out of 100)

Abstract:

We exemplify how multimodal agentic chatbots based on Large-Vision-Language-Models (LVLMs) can be leveraged as a tool for interactive and engaging exploration of scientific collections from diverse disciplines. Thereby, we demonstrate a potential beyond digital showcases. Specifically, with our tool we aim to make interesting scientific artifacts, often hidden behind complex search interfaces, more accessible and engaging, for non-experts and the general public.

In bringing together the vast amount of openly available scientific collections with open source LVLMs, we showcase how open science leverages hidden potentials in academic institutions. Our contribution focuses on the perspective of scientific collection of data as data, which not only serves the research community but also the interested public. By using more explorative layers for data linkage and retrieval through application of state-of-the-art Artificial Intelligence technology, we open up new and more accessible entry points for the data and its underlying relations

Search via conventional portals requires some expertise in the field. Enhancing it with an interactive chat interface, which can answer questions about the scientific collection portal in general, their collections, and single objects within, opens up an intuitive approach for all kinds of users of the collection portal.

https://fundus-murag.ltdemos.informatik.uni-hamburg.de

Review 1
Topicality 10 (15%)	Thematic Relevance 8 (15%)	Practical relevance 8 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 87
The poster presents a state-of-the-art agentic chatbot system that answers questions based on a collection of data underlying the system. It showcases the potential of agentic systems to interactively explore multimodal data collections in exchange with an LLM. It is both relevant for the AI and open science topic and shows the potential of such systems to support analysis of and interaction with (research) datasets.

Review 2
Topicality 8 (15%)	Thematic Relevance 8 (15%)	Practical relevance 8 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 84
This sounds like a very generic, and at the same time very promising, application of AI methods to an interesting Open Science problem.

(8) Embracing Transparency: A Study of Open Science Practices Among Early Career HCI Researchers
FeliTatiana Chakravorti (1), Sanjana Gautam (2), Sarah Rajtmajer (1)
Organization(s): 1: Pennsylvania State University; 2: University of Texas at Austin
Average reviewers ratings: 84.4 (out of 100)

Abstract

Many fields of science, including Human-Computer Interaction (HCI), have heightened introspection in the wake of concerns around reproducibility and replicability of published findings. Notably, in recent years the HCI community has worked to implement policy changes and mainstream open science practices. Our work investigates early-career HCI researchers’ perceptions of open science and engagement with best practices through 18 semi-structured interviews. Our findings highlight key barriers to the widespread adoption of data and materials sharing, and preregistration, namely: lack of clear incentives; cultural resistance; limited training; time constraints; concerns about intellectual property; and data privacy issues. We observe that small changes at major conferences like CHI could meaningfully impact community norms. We offer recommendations to address these barriers and to promote transparency and openness in HCI. While these findings provide valuable and interesting insights about the open science practices by early career HCI researchers, their applicability is limited to the USA only. The interview study relies on self-reported data therefore it can be subject to biases like recall bias. Future studies will include the scope to expand HCI researchers from different levels of experience and different countries allowing more justifiable examples.

Review 1
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 100
The abstract is clear and well structured. The topic focuses on the challenges of ECRs in the adoption of open science practices. In addition, the presentation aims to provide practical recommendations to overcome possible barriers to promote open science.

Review 2
Topicality 8 (15%)	Thematic Relevance 4 (15%)	Practical relevance 2 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 65
The work is of relevance for the HCI and Open Science Community.   I am afraid that the rubric for the scoring of abstracts is weighted high on the AI/Open Science topic. Which might mean that the abstract will miss out, not because it is of poor quality, but more because it isnt on the topic of this years conference. I like that you worked on concrete tips for a CHI and hope the organisers of the next edition will read them and maybe even put them into practice. I also like that you are presenting an interview study. We should work on more of those, I think they can give us a lot of insight into motivations, barriers and enablers of change. Good luck! Daniela Gawehns – NLRN

Review 3
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 8 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 97
The study addresses Human-Computer Interaction early-career researchers’ perceptions and Open Science practices. It highlights some key barriers to shared data adoption, which are highly relevant and call for action. In this direction, the authors propose some recommendations to address this pitfalls. Further steps are proposed as well, to deepen the understanding on this matter. Congratulations to the authors for this valuable proposal! Maria Paula Paragis

Review 4
Topicality 10 (15%)	Thematic Relevance 6 (15%)	Practical relevance 4 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 77
The submission addresses a question of high relevance regarding the practical adoption (or non-adoption) of open science practices by early-career researchers, focusing on the field of human-computer interaction. The submission aims to report findings from a qualitative interview study conducted in the United States. The submitted contribution addresses a well- defined research question and draws on a plausible methodology. The specific thematic relevance to the conference is slightly reduced by the fact that the submission does not address a question specifically related to the use of AI in open science (other than through the subject-area focus of the study). That is, its overall relevance (which I consider very high) is higher than its specific relevance. What could be elaborated more clearly is why the authors consider the applicability of their findings to be limited to the US (rather than, for example, the filed of human-computer interaction). Otherwise, the mentioned limits of the study as well as the suggested agenda for further research are very plausible.

(9) An open challenge to develop automated assessment of research finding
Timothy Errington
Organization(s): Center for Open Science
Average reviewers ratings: 84.8 (out of 100)

Abstract

Assessing the validity and trustworthiness of research claims is a central, ongoing, and labor-intensive part of the scientific process. Confidence assessment strategies range from expert judgment to aggregating existing evidence to systematic replication efforts, requiring substantial time and effort. What if we could create automated methods that achieve similar accuracy in a few seconds? The Predicting Replicability Challenge is a public competition to advance automated assessment of research claims. The challenge invites teams to develop algorithmic approaches that predict the likelihood of research claims being successfully replicated. Participants will have access to training data drawn from the Framework for Open and Reproducible Research Training (FORRT) database that documents 3,000+ replication effects. New research claims will then be used to test the algorithmic approaches’ ability to predict replication outcomes. The first set of held out social-behavioral claims will be shared with participating teams in August 2025. Teams will have one month to submit confidence scores, which will be evaluated in October 2025 with prizes awarded for the top-performing teams. A tentative second round will be held in October 2025, and the final round in February 2026. The initiative encourages innovation and interdisciplinary collaboration, including partnerships between AI/ML experts and domain specialists in social-behavioral sciences. The challenge is open to participants from academic and non-academic spaces worldwide.

Review 1
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 6 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 87
This poster/demo will introduce the Predicting Replicability Challenge, a team competition to develop algorithms that shall aid in predicting replicability of research claims. It appears to be a well-timed challenge with prizes for the top-performing teams. The deadline for team submissions is before the OSC though so I assume we will get a sneak peek into the performance of teams? That is unclear. I am missing the details on what will be presented at the conference and in what form – poster or demo? Additional linked resources would have been nice to have for further reading and more context on this submission. Linking to the website would be useful for readers: https://www.cos.io/predicting-replicability-challenge The topic is super interesting and very relevant, I hope to be able to catch it at the OSC.

Review 2
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 100
This is an important contribution, which aims to mobilise AI/ML experts and topic experts to develop algorithmic approaches to predict the reproducibility of research claims. Given the ongoing reproducibility challenges, together with the increasingly high burden on the peer review system, the development of new supporting automated/AI tools will be key. Access to high-quality training-data and clearly definend development and test phases further strengthen the approach. I therefore recommend acceptance of this proposal and am very interested to see the first outcomes. Mathijs Vleugel

Review 3
Topicality 8 (15%)	Thematic Relevance 8 (15%)	Practical relevance 6 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 6 (40%)	Total points (out of 100) 69
It remains a bit unclear to me what the poster aims to show or which research results it will present. However, it matches with the conference topic.

Review 4
Topicality 8 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 6 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 83
Really interesting project and looking forward to seeing the results it will yield! I put a lower score on the “Presentation and Language” for a couple of reasons. While I think most people who will attend the conference are aware of what replicability is, I think the abstract could have benefited from laying down the context a bit more. Why is it important in science? What is the difference between reproducibility and replicability? Explain what confidence scores are in more plain english. So that even people who are not familiar with the topic can follow and not fear of asking ‘stupid’ questions. Another thing, while I understand that your contribution is more to ‘promote’ the project, I wish you were more explicit to what you will present at the conference. Will it be the same as this abstract, a presentation of the timeline? Or will you provide other information?

(10) Improving Research Data findability with FAIR Signposting: implementation insights from KonsortSWD Data Centers
Janete Saldanha Bach, Brigitte Mathiak, Yudong Zhang, Peter Mutschke
Organization(s): GESIS – Leibniz-Institut für Sozialwissenschaften
Average reviewers ratings: 83.7 (out of 100)

Abstract:

The FAIR Principles are open to interpretation, resulting in varying assessments of compliance by different FAIR assessment tools, such as F-UJI. FAIR Signposting plays a critical role in standardizing these assessments, reducing inconsistencies, and enabling a more consistent interpretation across platforms. This is especially relevant in a pilot project by the KonsortSWD consortium of the German National Research Data Infrastructure (NFDI), which addresses challenges in evaluating and improving the FAIRness—particularly the findability—of research data.

The project implements the FAIR Signposting standard, a set of machine-readable, HTTP-based link relations that standardize metadata exposure and support automated FAIR assessments. It uses relation types (e.g., cite-as, described by, license, author, item, and collection) embedded in HTML headers, HTTP responses, or standalone linkset documents to guide automated agents such as search engines to metadata, persistent identifiers (PIDs), and related resources.

The two-part strategy included the deployment of a prototype at GESIS – Leibniz Institute for the Social Sciences and partner data centers such as LIfBi, DIPF, DIW/SOEP, and DZHW; and the creation of a best practices document based on implementation experiences. The application of FAIR Signposting led to significant improvements in FAIRness scores (e.g., GESIS: 43% to 79%).

This project demonstrates that embedding standard link relations enhances metadata interoperability, discoverability, and machine-readability. Tools like F-UJI were used to measure these improvements. The contribution offers practical guidance, implementation examples, and FAIR Signposting validation tools to support broader adoption across research data centers.

Review 1
Topicality 8 (15%)	Thematic Relevance 10 (15%)	Practical relevance 4 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 81
The demonstration covers the highly relevant topic of ensuring that research data can be found by automated search engines through links. Furthermore, it embeds research in broader contexts by allowing links to authors and other relevant resources, thus creating more interlinked data. While this is only implicit in the abstract, it should also be relevant for machine approaches, such as AI models. I therefore recommend the demonstration. Another reason for my recommendation is my curiosity about how the GESIS model can be transferred to other repositories with similar data, or how it compares to similar approaches. Stefan Skupien, Open Science Coordinator, Berlin University Alliance

Review 2
Topicality 8 (15%)	Thematic Relevance 6 (15%)	Practical relevance 6 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 7 (40%)	Total points (out of 100) 70
The FAIR principles play a centrol role in promoting Opens Science principles. Therefore, project described in this proposal, that set out to implement the FAIR signpositing standard, is central to the overall topic of Open Science. The proposal seems to hinge on two thoughts: 1) to implement a standardized method to accurately assess FAIRness and 2) to by doing that enhance things like meta-data interoperability. Linking these two would be very valuable, but from the abstract it is not entirely clear to me whether and how this will be done in the proposed session. The proposal would benefit from including a concise set-up of the proposed session. Addionally, although AI processes must play a role in the FAIR assessments, the link between OS and AI could be made more explicit.

Review 3
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 100
Very relevant contribution. The authors are encouraged to discuss the applicability across various scientific domains and the limitations or challenges of the proposed solution

(11) Gaining Transparency of LLM Usage in Academic Writing via Change Tracking
Po-Chun Tseng
Organization(s): FIZ Karlsruhe – Leibniz-Institut für Informationsinfrastruktur
Organization(s): Department of Conservative Dentistry and Periodontology, University Hospital, LMU Munich, Munich, Germany
Average reviewers ratings: 82.5 (out of 100)

Abstract:

The use of large language models (LLMs) is rapidly increasing in the academic setting. Although authors are required to declare the use of generative AI tools in their submissions for scientific publications, the description is always generic and unspecific. The effects of these uses cannot be fully characterized unless the use is to be summarized and disclosed in detail. In this project, relevant guidelines on the academic usage of generative AI tools will be briefly reviewed to identify the unmet needs. Based on the identified needs, a framework to track the LLMs’ contribution will be proposed, enabling an effective yet efficient record of the AI intervention. A prototype will be presented to showcase a potential solution. The ultimate goal is to develop an interface that empowers the responsible use of generative AI tools in academic writing for scientific rigor.

Review 1
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 8 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 90
This is a timely and commendable contribution tackling the transparency and accountability of LLM usage in academic writing. As the use of generative AI tools expands, your proposed solution offers a responsible way forward that aligns with Open Science values. Strengths: Clear and relevant problem definition Strong alignment with ethics, integrity, and AI transparency Concrete proposal for a framework and prototype Suggestions for Improvement: Consider including examples of how the change-tracking system works in practice Expand briefly on the type of guidelines reviewed and how they shaped your framework Consider connecting your proposal with specific publisher policies or institutional frameworks for added impact This is a promising initiative that addresses an urgent need in scholarly communication.

Review 2
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 8 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 93
The project’s intention is well described and the topic is highly relevant. Outcomes may prove valuable not only for scientific publications but also for funding proposals and for enhancing the overall understanding of AI usage in science, ultimately advancing reliable and innovative research and scientific writing. The intersection of AI and OS is recognizable. Creating a framework for tracking LLMs contributions aligns with OS emphasis on transparency, responsibility and research integrity, ensuring clear communication of (AI) tools used and is highly relevant. I still recommend emphasizing the context to OS more strongly in the poster/demo. The intended framework–reaffirmed by existing guidelines–is very promising to improve the understanding of AI’s role in research, identify gaps, reasonable and unjustified use, and foster a culture of openness. It is not clear, how “the unmet needs” will be identified, what their definition is and if they are the only indicators to solve the research question. The intention to present a practical solution (prototype) is worth highlighting as it engages the conference attendees. The feasibility of this demonstration is not clearly elaborated. Idea: The author would further support OS when sharing the outcomes throughout the project with the academic community. Furthermore, leveraging the audience’s richness of knowledge and involving them to generate further insights might support the project’s progress (these could be discussions stimulated by the poster/demo, qr code to tools for further collaboration, etc.).

Review 3
Topicality 8 (15%)	Thematic Relevance 6 (15%)	Practical relevance 6 (15%)	Presentation and Language 4 (15%)	Overall Recommendation 6 (40%)	Total points (out of 100) 60
The submitted contribution addresses a topic of high relevance, namely how to ensure transparent use of AI tools (LLMs)  in research. Sufficient transparency, in turn, is a key prerequisite for ensuring and safeguarding responsible conduct of research. This challenge, however, is not particular to open science but rather a challenge the entire research community faces. While this does not decrease the general relevance of the contribution, it may be a good but not ideal fir for the conference, depending on how evaluators weigh the specificity criterion. The specific content of the submitted contribution has a relatively high degree of practical relevance as it aims to introduce a framework and prototype to enable and facilitate transparent reporting of how AI tools/LLMs were used. However, two questions remain open: 1) How are the needs that the proposed solution aims to meet identified? 2) On what basis are existing guidelines assessed as “always generic and unspecific” so that the pressing need to develop a new framework from scratch (rather than reforming existing approaches) arises? A further relatively minor point: The prototype could have been described in slightly more detail as it remains rather unclear what type of tool it actually is and whether there are any limitations to its usage (e.g., re. applicability to different scientific disciplines). A slightly more detailed elaboration of the contribution could have addressed the questions and made it even more compelling and relevant.

Review 4
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 8 (15%)	Presentation and Language 6 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 87
Very interesting project, and indeed something that is very relevant to the current academic publishing landscape. I scored the “Presentation and Language” lower because of a couple of issues. Try and avoid absolute statements (e.g. the ‘always’ in “the description is always generic and unspecific”), especially if you have not yet ‘quantified’ the missing elements within descriptions. The abstract would have also benefited from just a bit more details (e.g. listing a couple of characteristics that are usually missing and that the framework will help with). This would have made the description of the project less abstract. Reviewer: Jonathan England

(12) Beyond the Numbers: Using AI to measure the impact of Open Science through data reuse
Felix Bach, Kerstin Soltau, StefaAgata Morka (1), Parth Sarin (2), Tim Vines (2), Iain Hrynaszkiewicz (1)
Organization(s): 1: PLOS; 2: DataSeer
Average reviewers ratings: 81.7 (out of 100)

Abstract:

Can language models help us capture how open science practices translate into real-world impact? PLOS and DataSeer collaborated on a project exploring how AI can address a critical gap in Open Science metrics: measuring data reuse. While data sharing is a widely encouraged Open Science practice, the reuse of shared data—one of the most meaningful impacts of Open Science—remains difficult to detect and quantify. Our project developed an AI-based methodology to tackle this challenge, combined with a consultation process with key scholarly communications stakeholders.

Following the community consultation, we refined definitions and criteria for identifying data reuse and fine-tuned a large language model (Llama-3.1-8B-Instruct) using proximal policy optimization (PPO) to classify instances of data reuse. The model was trained on a curated set of 421 annotated research articles and applied to a broader corpus of 4,328 PLOS publications. Results showed that 47% of the articles demonstrated data reuse, although persistent identifiers (e.g., DOIs, accession numbers) were only detected in a small fraction (176 cases). Importantly, the model’s reasoning summaries were adapted with RLHF and iterative feedback from researchers, editors, and policymakers to produce interpretable insights about how data were reused.

This work highlights that current bibliometric methods likely underestimate data reuse and demonstrates that AI can offer scalable, nuanced assessments of Open Science’s impacts. By shifting the focus from practices (such as data sharing) to measurable impacts (such as data reuse), our new indicator offers a more meaningful way to evaluate and incentivize Open Science practices.

Review 1
Topicality 6 (15%)	Thematic Relevance 8 (15%)	Practical relevance 4 (15%)	Presentation and Language 6 (15%)	Overall Recommendation 4 (40%)	Total points (out of 100) 52
The problem of finding data citations from the fulltext of articles without expliclty mentioning is not new, see e.g. [1] but more extensive literature search should be performed as well. The authors should acknowledge that and also compare their LLM methods with the ones done 10+ years ago, or explain why they think it is worth to use an LLM approach instead. The description of the technical details are good (interesting enough for the ones interested in the technical stuff, but not overwhelming for everyone else). I miss any numbers from an evaluation of the approach. Usually part of the annotated set is used to learn and the rest for evaluation (e.g. precision and recall). Whether the approach gives the correct answer in 95% or only 70% is a huge difference and an absolute must to estimate in such a work. Without such an evaluation and also no possibility mentioned how it could be reused for anyone else in the open science community, I am inclined to reject the submission. [1] Boland, K., Ritze, D., Eckert, K., & Mathiak, B. (2012). Identifying references to datasets in publications. In Theory and Practice of Digital Libraries: Second International Conference, TPDL 2012, Paphos, Cyprus, September 23-27, 2012. Proceedings 2 (pp. 150-161). Springer Berlin Heidelberg.

Review 2
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 93
Very interesting approach. It would have been nice to also learn about the accuracy of the model to determine the data citation/reuse contexts and about the types and definitions of data reuse you used.

Review 3
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 100
The contribution outlines a collaborative project between PLOS and DataSeer that addresses a significant gap in Open Science metrics: the measurement of data reuse. The project developed an AI-based methodology to assess this issue and its results highlight that current bibliometric methods likely underestimate data reuse. The contribution is highly valuable, as it presents a practical tool to measure impacts of Open Science practices regarding data sharing. Congratulations to the authors for this impactful initiative!

(13) The power of open polyglot plain text tooling for reproducible AI research
Alva Seltmann (1,2), Christian Eggeling (1,2)
Organization(s): 1: Institute for Applied Optics and Biophysics, Friedrich Schiller University Jena, Jena, Germany; 2: Leibniz Institute of Photonic Technology, Jena, Germany
Average reviewers ratings: 80.8 (out of 100)

Abstract:
Methods reproducibility is the ability to record and implement all experimental and computational procedures of the experiment with the same data and tools, obtaining the same results. It is a crucial step for applied Artificial Intelligence (AI) research in an Open Science context. While AI-specific reproducibility tools exist, the typical researcher employs AI in a larger scientific context. Here, we explore synergies of existing plain-text-based, polyglot tooling to capture this whole context. First, literate programming using Org-mode allows executing, capturing and annotating all research steps. It includes research hypotheses, dataset creation, study protocols, the training and evaluation process, and the model application with analysis code and reproducible figures and papers. We detail Org-mode’s polyglot capabilities of combining arbitrary programming languages and concepts like tangling or transclusion, which integrate source and artifact files in the literate document. Second, the Nix package manager allows declaring the full polyglot environment programmatically, making it reproducible on arbitrary machines. Third, recording every change with DataLad, built on Git and git-annex, provides provenance of all results and entry points to reproduce experiments. This Open Science approach to AI experiments integrates computational reproducibility with context readability for machines and humans.

Review 1
Topicality 8 (15%)	Thematic Relevance 10 (15%)	Practical relevance 8 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 87
Unfortunately, I lack the technical expertise to properly review this article. However, the topic of the reproducibility of methods in the context of AI tools seems relevant and timely in the context of open science. The poster/demonstration presentation appears to be scientifically interesting. Suggestions: • Briefly explain the basis of your understanding of plaintext or plaintext-based polyglot tools. • The intention “We explore synergies of existing plaintext-based polyglot tooling to capture this whole context” needs to be explained and clarified. • Could you highlight the research question, its relevance, the methods, and the key findings/results (if available) and conclusion more clearly?

Review 2
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 8 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 90
A very interesting demo that covers an important topic – tools for reproducibility of AI research. I recommend acceptance.

Review 3
Topicality 8 (15%)	Thematic Relevance 8 (15%)	Practical relevance 8 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 84
This abstract presents a well-structured approach to reproducibility in AI research by combining Org-mode, Nix, and DataLad. While the tool descriptions are clear, the abstract assumes familiarity with these technologies and lacks explicit mention of the intended audience. Clarifying who would benefit and highlighting real-world impact would improve its accessibility and appeal. Well done. Seán Lacey

Review 4
Topicality 8 (15%)	Thematic Relevance 8 (15%)	Practical relevance 8 (15%)	Presentation and Language 4 (15%)	Overall Recommendation 5 (40%)	Total points (out of 100) 62
The idea is valuable, but the abstract is not very well written; it implies knowledge in the field (literate programming, Org-mode, Nix package); it’s unclear if the idea is concrete or just a plan (there is no link to code or URL to showcase the tools is concrete)

(14) When Measures Become Targets: Lessons from Open Science and Machine Learning on the Fragility of Reform
Moritz Herrmann
Organization(s): Munich Center for Machine Learning, LMU Munich
Average reviewers ratings: 80.5 (out of 100)

Abstract:
Building on the position paper “Why We Must Rethink Empirical Research in Machine Learning” [1], which examines methodological and epistemic challenges of machine learning (ML) as an empirical science, we highlight parallels between the replication crisis in the applied sciences and current issues in ML/AI research. Both have long been subject to epistemic critiques: warnings about weak inferential foundations, misapplied statistical reasoning, and overreliance on performance metrics. Yet these concerns are routinely overlooked in practice, sidelined by institutional incentives and publication pressures. Moreover, because research communities function as social systems, measures of scientific quality—such as prediction performance or statistical significance—can themselves become targets, prone to corruption pressures. This dynamic also affects core practices promoted by the Open Science movement: code and data sharing, computational reproducibility, and preregistration are promising tools for improving research integrity, but as alternative measures of scientific quality they are not immune to the same social and systemic pressures. Without critical reflection, they risk being treated as shallow technical fixes rather than as components of a deeper epistemic shift. By drawing these connections, we advocate for a more epistemologically grounded approach to Open Science—within AI in particular, and across scientific practice more broadly.

[1] Herrmann, M., Lange, F. J. D., Eggensperger, K., Casalicchio, G., Wever, M., Feurer, M., Rügamer, D., Hüllermeier, E., Boulesteix, A.-L., & Bischl, B. (2024). Position: Why We Must Rethink Empirical Research in Machine Learning. Proceedings of the 41st International Conference on Machine Learning (ICML 2024), Proceedings of Machine Learning Research, 235, 18228–18247. https://proceedings.mlr.press/v235/herrmann24b.html 

Review 1
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 100
A critically pointing to new measures just because they are “open” is a true risk, so your reflection can be really useful for the community.

Review 2
Topicality 8 (15%)	Thematic Relevance 4 (15%)	Practical relevance 4 (15%)	Presentation and Language 6 (15%)	Overall Recommendation 4 (40%)	Total points (out of 100) 49
This contribution aims to highlight methodological challenges in ML/AI research, such as warnings about weak inferential foundations, misapplied statistical reasoning, and overreliance on performance metrics. The authors argue that these are systematically being overlooked as a result of pressures and incentives within the academic system. Likewise, the authors draw attention to the fact that broader standards for good scientific practice (e.g., research integrity and open science standards) are under threat by the current incentive system. In their contribution, the authors aim to make connections beween these two issues. It is increasingly recognised, that the current incentives and rewards systems negatively impacts on the quality, reproducibility, and impact of research, and there are important growing international efforts to reform this. Nevertheless, the link between (1) methodological quality of ML/AI research and (2) uptake of open science practices in this contribution seem indirect: they both impact on reproducibility and can be considered a consequence of the current assessment system, but this can be argued for many challenges and flaws within the research system. Therefore, I am afraid that the contribution may be beyond the intended scope of the conference. Mathijs Vleugel

Review 3
Topicality 10 (15%)	Thematic Relevance 8 (15%)	Practical relevance 8 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 90
The accompanying paper is well written and well researched, including many references as related work so it is not just a position paper in the sense of an opinion statement. The authors have clearly put in the work and I think the aspects discussed here would be very valuable food for thought for the community — all the more so because the paper brings a particularly interdisciplinary perspective (computer science, sociology, information science, science of science) to the table.

Review 4
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 8 (15%)	Presentation and Language 6 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 83
The author approaches a very important topic: reflection on the use of machine learning/artificial intelligence, the involved methods and the consequences on science and, in particular, on open science. The argument, which is build in the abstract, is easy to follow at first, beginning with the parallels between the replication crisis in applied sciences and current issues in AI research up to the pressures of institutional incentives and publication pressure. The good line of argumentation breaks somehow when it is stated, that research communities function as social systems. While true, this is also a significant point of the afore mentioned publication pressures or even a root cause of misapplied statistical reasoning. The point is made how this impacts open science, but the build of of the argument could have been shorter and more crisp, repetition like “corruption pressures”, “institutional incentives”, “research integrity”, “scientific quality” and more, should be avoided. I am all for the notion of a more “grounded approach to” open science, with AI in particular. The author states at the end, that they will advocate for such a thing. Especially for an abstract I would have liked to read more about the proposed systems or solutions they are planing to advocate for. Nevertheless, an important topic rarely seen in this time of AI hype.

(15) Small-scale Domain-specific Web Crawling for Complementing Established LLM Data Sources
Thomas Eckart (1), Frank Binder (2), Erik Körner (1), Christopher Schröder (2,3), Felix Helfer (1)
Organization(s): 1: Saxon Academy of Sciences and Humanities in Leipzig; 2: Institute for Applied Informatics (InfAI); 3: Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig
Average reviewers ratings: 79.5 (out of 100)

Abstract:

The multilingual Leipzig Corpora Collection (LCC) [1] operates its own crawling infrastructure since 2003. Despite the availability of established datasets for Large Language Model (LLM) training, we aim to continue this small-scale infrastructure as it guarantees complete control over source selection and thematic focus and still provides a significant gain in LLM training material over established resources.

For German, we compare LCC’s annual news and web crawls against the German subset of the popular OSCAR corpus of the same time [2]. Using identical content extraction and sentence-level deduplication on both resources and considering their levels of content density, we establish solid estimates of their degree of complementarity.

Accordingly, the 396-billion-raw-token German LCC crawl from 2023 yields at least 35.9 billion tokens of cleaned and deduplicated document-level text data, with less than 15% overlap with the German OSCAR subset and less than 28% overlap with the LCC data from 2021-2022. Thus, for 2023, our sovereign crawling infrastructure and content extraction gained over

22.1 billion complementary tokens in high-quality document text, to be freely used in research contexts.

Within the scope of the UrhBiMaG [3], we use this data to investigate effects of data quality and training schemes on the quality and performance of neural language models of different sizes and architectures in the ongoing CORAL project [4].

[1] https://wortschatz-leipzig.de

[2] https://oscar-project.org/

[3] https://www.bmj.de/SharedDocs/Gesetzgebungsverfahren/DE/2020_Gesetz_Anpassung-Urheberrecht-dig-Binnenmarkt.html

[4] https://coral-nlp.github.io/

Review 1
Topicality 8 (15%)	Thematic Relevance 8 (15%)	Practical relevance 6 (15%)	Presentation and Language 6 (15%)	Overall Recommendation 7 (40%)	Total points (out of 100) 70
The abstract is providing information on a web crawler project. It is however not made clear enough how this pertains to open science practices. Also a clear focus on a research question/gap that needs closing is missing. The apporaches are definitively important contributions and touch on the OSC theme, but its specificity may not cater to a broad audience at OSC.   Ulf Toelch BIH QUEST Center for Responsible Research

Review 2
Topicality 8 (15%)	Thematic Relevance 6 (15%)	Practical relevance 6 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 74
This abstract presents clear quantitative comparisons that suggest the stated dataset offers non-trivial additions to existing language corpora. When LLM is first stated, please use the long name with the abbreviation in brackets. Consider adding a sentence to clarify the intended audience and relevance to the conference theme, as the abstract’s relevance may not be immediately clear to readers beyond those focused on LLM or corpus collection. Seán Lacey

Review 3
Topicality 10 (15%)	Thematic Relevance 8 (15%)	Practical relevance 10 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 94
The authors highlight the advantages of maintaining a small-scale, controlled open source crawling infrastructure to enhance the quality and diversity of training data for LLMs. The authors further investigated the impacts of data quality and training schemes using propriety data of CORAL project as an interesting example. To help readers understand better the authors could elaborate on what makes the OSCAR corpus a relevant benchmark. In particular why LLC’s annual news crawl is compared with OSCAR corpus. A relevance statement on how these findings influence future developments in LLM training will enhance the wider outreach of this work.

Review 4
Topicality 8 (15%)	Thematic Relevance 8 (15%)	Practical relevance 8 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 80
A timely and relevant submission from experts in obtaining, managing, and improving web data for AI/ML models. But the abstract is quite technical. The final presentation might benefit from a bit more non-expert-friendly explanation of the process and results, and a stronger connection to open science. Also, the importance of high-quality and domain-specific data for AI/ML applications is clear. But what about the importance of the resulting AI/ML applications for open science? What are the uses of these applications to researchers and research support staff? How about the public wanting to learn about research?

(16) Enhancing Open Cosmology with Emulator Packaging
Ian Harrison, Hidde Jense
Organization(s): Cardiff University
Average reviewers ratings: 79.3 (out of 100)

Abstract:

In the field of cosmological astrophysics there is growing adoption of AI emulators to speed up numerical calculations necessary for inferring properties of the Universe, such as the behaviours of dark energy, dark matter and the theory of gravity. In our work we have leveraged OS principles to enhance this use in research, by creating a standard framework for the testing, training, accuracy validation, use and – importantly – sharing of these emulators. This involved the creation of a standard “packaging” of the necessary files and metadata, as well as extending existing popular frameworks to make use of this packaging. This standardisation and packaging improves reproducibility and reduces wasted effort and resources due to duplication of otherwise un-reusable emulators, which are often created at great computational expense on HPC clusters. It also reduces barriers to entry for new members of the community in providing ready-made tools which can be used with confidence on a laptop, allowing more diverse sets of analyses of new and interesting cosmological models.

Review 1
Topicality 8 (15%)	Thematic Relevance 6 (15%)	Practical relevance 6 (15%)	Presentation and Language 6 (15%)	Overall Recommendation 7 (40%)	Total points (out of 100) 67
This submission presents a packaging approach designed to encapsulate AI emulators, making them easier to share and helping to ensure the reproducibility of related computations. It’s always valuable to see practical implementations at the conference. Examples that others can explore and potentially adopt in other communities working with AI emulators. A link to a concrete example would have been helpful. The submission also leaves open the question of whether the packaging approach itself is based on open technologies, and if so, which ones. Additionally, I would have appreciated more clarity on whether the AI emulators themselves are open. That would have made the submission even more aligned with the focus of the conference.

Review 2
Topicality 6 (15%)	Thematic Relevance 6 (15%)	Practical relevance 8 (15%)	Presentation and Language 6 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 71
Making highly integrated AI emulators reproducible seems to be fairly interesting and relevant. Derived knowledge might very well apply to fields other than astrophysics.

Review 3
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 100
Adoption of open science principles to create a standardised framework to enhance research reproducibility in cosmological astrophysics is impressive and commendable. The tool/workflow reduces barrier to entry for new members into cosmological astrophysics.

(17) The Intelligence Behind the OpenAIRE Graph: Linking Science with AI
Stefania Amodeo (1), Paolo Manghi (1,2), Natalia Manola (1)
Organization(s): 1: OpenAIRE AMKE; 2: ISTI-CNR
Average reviewers ratings: 76.3 (out of 100)

Abstract:

The OpenAIRE Graph stands at the forefront of research infrastructure innovation, combining cutting-edge AI techniques with Open Science principles to process and analyze 400M+ research records monthly, including 290M+ publications, 82M+ datasets, and 1M+ software entries. More than a metadata aggregator, the OpenAIRE Graph fuses diverse sources into a richly linked, machine-actionable research ecosystem, powered by an advanced AI-driven analytical workflow that elevates data quality, connectivity, and usability through:

• automated metadata enrichment of persistent identifiers (e.g. ORCID, ROR), Fields of Science classifications, Open Access status, licensing terms, and semantic types using Natural Language Processing;
• entity recognition and disambiguation using ML models to connect authors, institutions, projects, and funders across heterogeneous sources;
• knowledge graph embeddings and similarity scoring to detect and link conceptually related research artefacts, enabling cross-disciplinary exploration and contextualization;
• relationship extraction and network mapping, to uncover latent connections among research outputs, such as citations, co-authorships, and funding dependencies.

These mechanisms are continuously refined using feedback loops, benchmarking datasets, and community input, ensuring the Graph remains a trusted foundation for Open Science monitoring, research assessment, and discovery.

Our demonstration will show how these AI capabilities operationalize the FAIR principles, support evidence-based policymaking, and streamline research workflows. This session will offer practical insights for those exploring AI-enhanced infrastructures for scholarly communication and assessment.

Review 1
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 8 (15%)	Presentation and Language 6 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 83
Sounds very relevant but your abstract is a bit underspecified — I would have liked one or two sentences on the context in which the OpenAIRE graph is emerging, and also in _which way_ your contribution will offer practical insights: will people be able to try things out hands-on?

Review 2
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 97
The submitted contribution presents a technically and conceptually strong demonstration of the OpenAIRE Graph as an AI-enhanced infrastructure for Open Science. It addresses a highly topical challenge at the intersection of AI and open scholarly communication and convincingly demonstrates how state-of-the-art machine learning and natural language processing techniques can be harnessed to enrich, link, and contextualize vast amounts of scholarly metadata.

Review 3
Topicality 6 (15%)	Thematic Relevance 8 (15%)	Practical relevance 6 (15%)	Presentation and Language 2 (15%)	Overall Recommendation 4 (40%)	Total points (out of 100) 49
OpenAIRE is an important infrastructure on an European level and its technical workflows for enriching the data and enhancing its quality is in principle interesting. Moreover, as some of steps also uses AI technology, the connection to the current topic of the conference is given. But the whole abstract reads for me like a sale pitch with some bold statement, e.g. “forefront of research infrastructure innovation”. The statement that OpenAIRE is “operationaliz[ing] the FAIR principles” is a stretch for me, as it cannot do much to “accessibility” and “reuse” because that lies on the data repository or researchers themselves. I miss any statement about how good the AI-based workflows are working in practice or other reflection on the methods. Are there any evaluation proving that the data quality evaluated by how much and what about potential newly introduced errors by them? It also seems that with NLP, ML, KGE some “older” AI technics are used instead of the omnipresent LLMs and it would be interesting to hear, why that is the case or whether they also tried out some other approaches etc. Overall, it is hard to see for me from the abstract, what the audience can then learn from such a poster/demonstration besides OpenAIRE as a platform (which probably a lot of people at the conference already know).

(18) Powering Open Science Across Borders: A Live Demonstration of the EOSC EU Node
Maja Dolinar (1), Natalia Manola (1), Spiros Athanasiou (2)
Organization(s): 1: OpenAIRE AMKE; 2: Athena Research Center
Average reviewers ratings: 75.5 (out of 100)

Abstract:

This demonstration introduces the EOSC EU Node—a European-level operational node of the European Open Science Cloud Federation—showcasing its role in accelerating transparent, reproducible scientific practices across disciplines and borders. The EOSC EU Node serves as a federated infrastructure for researchers, enabling access to FAIR-compliant data, computational tools, and collaborative workspaces in line with Open Science values.

In this live demo, we will explore how users can log in using institutional credentials, search and access open datasets through the EOSC Resource Hub, launch compute-intensive workflows via interactive notebooks or use any of the other available services and tools—entirely within a secure, GDPR-compliant environment. Special focus will be given to AI-supported services that facilitate data annotation, automated metadata generation, and reproducible pipelines, all aligned with the EOSC Interoperability Framework and the FAIR principles.

We will illustrate how the EOSC EU Node lowers barriers for smaller institutions, citizen science projects, and cross-disciplinary teams by integrating open tools into reusable research workflows. Attendees will engage with real-time interfaces, receive guidance on service onboarding, and explore how their organizations can benefit from and contribute to the EOSC ecosystem.

This session invites researchers, infrastructure providers, and policy actors to reflect on how federated infrastructures and responsible AI can converge to support a more equitable and innovative Open Science landscape. Join us to experience the EOSC EU Node in action and see how your research community can take part in building the future of science.

Review 1
Topicality 8 (15%)	Thematic Relevance 2 (15%)	Practical relevance 2 (15%)	Presentation and Language 4 (15%)	Overall Recommendation 2 (40%)	Total points (out of 100) 32
The author are planing to introduce the EOSC EU Node in a demo. Since this infrastructure is not that long available, yet, this is a good endeavor and logically the first paragraph of the abstract is describing the EOSC EU Node briefly. The authors then further describe what the demo will entail, from testing the login with institutional credentials up to launching compute-intensive workflows, task that are part of the basic description of the EOSC EU Node. It is stated that special focus will be given to AI-supported services, with struck me as odd. Since the goal and mission of the EOSC EU Node does not encompass such services, these things have to be the contribution of the authors, and those have to declared in the abstract. It could have been a more appealing abstract with the AI-tools not mentioned in line, seemingly belonging to the build in services of the EOSC EU Node – this is misleading. Especially since the next paragraph of the abstract contains not much content but repeats statements from the EOSC EU Node mission and vaguely summarizes how beneficial the use of the EOSC EU Node is. No concrete measure, tool or protocol is described at all. In the Last paragraph suddenly AI is mentioned again, without any context, functionality or concrete application. From this abstract I understand that the authors want to demonstrate the functionality of the EOSC EU Node, which I am all for. But they do not make clear that the AI part, they mention briefly, is not integral part of the EOSC EU Node. They further do not make it clear where the demo of infrastructure of the EU ends and their presentation of tools and application for that begins. Therefore I can not recommend this abstract highly.

Review 2
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 8 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 86
The establishment of  EOSC EU Node is a very relevant step for the development of EOSC as an European infrastructure – due to this this poster will be of high relevance for the community. The abstract also indicates that the connection to responsible AI building upon EOSC will be made.

Review 3
Topicality 10 (15%)	Thematic Relevance 8 (15%)	Practical relevance 8 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 87
I think this abstract outlines an important project in the European landscape of working with FAIR data and code in a uniform (and hopefully easy to use) manner. The European Open Science Cloud has been an absract notion for many, and now we learn in practice what it is and how we can use it. The proposal is clearly written in itself, although I would have liked to see a concise plan of how the session will be structured. There is a link between OS and AI, that the authors will stress during the session.

Review 4
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 8 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 97
Highly relevant contribution. The topic is suitable for an open discussion and community consultation

(19) Initiatives and networks- Irelands National research landscape
Ruth Patricia Moran (1), Jose Ignacio Flores (2)
Organization(s): 1: Atlantic Technological University; 2: UNIR, Madrid
Average reviewers ratings: 74.8 (out of 100)

Abstract:

Ireland’s focus on areas like AI ethics, Open science and research integrity has grown collaboratively in the last number of years.

The National Research integrity forum (NRIF) established in 2015 is a collaborative organisation that promotes good practices in areas like ethical use of AI, the responsible use of AI in areas like publications and drives the research integrity agenda in Ireland.

Ireland’s research eco-system also established the National Open research forum (NORF) to drive open research practices in Ireland.

National Academic integrity network (NAIN) established in 2019 to provide training and education around how to avoid any breaches of academic integrity such as data fabrication, text plagiarism using AI algorithms.

Ethics, research integrity and open science and the evolving change landscape of AI place a large demand on researchers, students and educators. NRIF, NORF and NAIN encourage researchers in Ireland to use AI tools honestly, in a transparent manner. As AI evolves, continuous education of the ethical use of AI policies and practices need to be continuously updated to reflect the ever-changing landscape.

References:

• ALLEA – The European Code of Conduct for Research Integrity- The European Code of Conduct for Research Integrity – ALLEA
• National Academic Integrity Network (NAIN) (2023) Generative Artificial Intelligence: Guidelines for Educators. Quality and Qualifications Ireland (QQI). NAIN Generative AI Guidelines for Educators 2023.pdf (accessed: 05 May 2025)
• National Open research forum (accessed: 5^th May 2025) NORF – Digital Repository of Ireland
• National Policy Statement on Ensuring Research Integrity in Ireland

Review 1
Topicality 8 (15%)	Thematic Relevance 8 (15%)	Practical relevance 6 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 77
The networks and topics presented seem relevant for the conference, the goal for the poster was not entirely clear to me, is it to present existing networks and initiatives in Ireland related to OS? Or training initiatives? Or both?

Review 2
Topicality 8 (15%)	Thematic Relevance 8 (15%)	Practical relevance 6 (15%)	Presentation and Language 6 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 74
This abstract offers a valuable overview of Ireland’s coordinated efforts in research integrity, open science, and AI ethics. It is institutionally grounded and informative, making it a useful contribution to the conference. However, it lacks analytical depth, specificity in the AI–Open Science nexus, and clear practical demonstration. With improvements in narrative flow and inclusion of concrete examples, the contribution could be significantly strengthened.

Review 3
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 8 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 90
The abstract is well written and clear, Hovewer, it presents mainly a presentation of the  the Irish landscape, without provinding clear information about what will presented during the conference. Hovewer, I think it is import to showcase what Irish institutions are doing in terms of education, OS and AI. This can give important inputs to institutions that are still trying to navigate the topic.

Review 4
Topicality 6 (15%)	Thematic Relevance 10 (15%)	Practical relevance 4 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 4 (40%)	Total points (out of 100) 58
The Irish initiatives are really interesting and I know a lot of work is done in terms of Open Science. However, based on your abstract, I failed to see what exactly presenting these initiatives would bring to the conference attendees. You listed the initiatives but didn’t really explain why they are relevant and what  take-home information you will provide. It would have been interesting to focus more on the part about using AI transparently (“NRIF, NORF and NAIN encourage researchers in Ireland to use AI tools honestly, in a transparent manner”). This recommendation emerged from discussions I suppose, and the thought and decision-making process could be an interesting angle, relevant to others that are still making decisions about their own AI policies. You also mention that “practices need to be continuously updated”, another interesting angle could be of how these three initiatives help deal with these uncertainties in practical terms. Reviewer: Jonathan England

(20) OS Policies – Vanguard of a Cultural Shift or Institutional Window Dressing?(20) OS Policies – Vanguard of a Cultural Shift or Institutional Window Dressing?
Verena Weimer (1), Tamara Heck (1), Florian Papilion (1), Tim Höffler (2), Kerstin Hoenig (3), Guido Scherp (4)
Organization(s): FIZ Karlsruhe – Leibniz-Institut für InformationsinfrastrukturOrganization(s): 1: DIPF | Leibniz Institute for Research and Information in Education; 2: Leibniz-Instituts für die Pädagogik der Naturwissenschaften (IPN); 3: German Institute for Adult Education – Leibniz Centre for Lifelong Learning; 4: ZBW – Leibniz Information Centre for Economics
Average reviewers ratings: 73.8 (out of 100)

Abstract:

Open Science (OS) has emerged as a normative ideal in research, yet the institutional uptake is highly uneven. Nosek [1] places policy at the top of his pyramid for achieving cultural change toward OS, characterizing it with the imperative to “make it required”. However, the impact of policies lies in the detail of the actual translation of OS practices. Developing institutional policies and determining concrete commitments and measures is a complex endeavour, framed by disciplinary contexts and their underlying practices.

In the project IvOS, we raise the question: Do Open Science policies truly drive cultural transformation in research, or do they serve primarily symbolic functions under the guise of compliance? This study addresses this tension by empirically investigating the content and perceived impact of institutional OS policies within a highly interdisciplinary and heterogeneous research network.

The database consists of OS policy documents (n=92) of Leibniz Institutions. These are coded applying qualitative content analysis (results of a pre-study are available [2]). The analysis focus lies on the OS dimensions (Open Access, Open Data, OER, etc.) [3], the practices addressed and their binding character [4], the consideration of discipline-specific concepts and inclusion [5], the implementation into institutional strategies and good scientific practice [6] as well as the monitoring to measure the policy impact [7].

The results provide information on the function of OS policies, whether they have the potential to act as vanguard of a cultural shift and how well they translate the principles to concrete research practices.

References

[1] Nosek, B. (2019). Strategy for Culture Change. Center for Open Science. URL: https://www.cos.io/blog/strategy-for-culture-change 

[2] Weimer, V., Heck, T., Scherp, G., Hoenig, K., & Höffler, T. (2024). Open Science Policy Documents of the Leibniz Institutions. Open Science Festival 2024, Mainz. Zenodo. https://doi.org/10.5281/zenodo.13862317 

[3] Leibniz Association (2022). Leibniz Open Science Policy. URL: https://www.leibniz-gemeinschaft.de/fileadmin/user_upload/Bilder_und_Downloads/Forschung/Open_Science/Open_Science_Policy.pdf 

[4] SPARC. (2018). An Analysis of Open Data and Open Science Policies in Europe. https://sparceurope.org/download/3674 

[5] Chtena, N., Alperin, J. P., Morales, E., Fleerackers, A., Dorsch, I., Pinfield, S., & Simard,M.-A. (2023). The neglect of equity and inclusion in open science policies of Europe and the Americas. In SciELO Preprints. https://doi.org/10.1590/SciELOPreprints.7366 

[6] Barcelona Declaration on Open Research Information, Kramer, B., Neylon, C., & Waltman, L. (2024). Barcelona Declaration on Open Research Information (1.0). Zenodo. https://doi.org/10.5281/zenodo.10958522 [7] European Commission: Directorate-General for Research and Innovation, Wouters, P., Ràfols, I., Oancea, A., Kamerlin, S. C. L. et al., Indicator frameworks for fostering open knowledge practices in science and scholarship, Publications Office of the European Union, 2019, https://data.europa.eu/doi/10.2777/445286

Review 1
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 0 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 10 (40%)	Total points (out of 100) 85
I highly recommend this poster/demonstration, even though it appears to have no connection to the conference’s AI topic. Open Science policies are currently being developed and reflected upon at several institutions, and arguments stemming from discussions on this topic can be useful during the agenda-setting and formulation phases of Open Science policy-making cycles. While the non-binding nature of many such policies in Germany is well known, it remains to be explored how OS policies can exert soft power in connection to other legal opportunities and how experiences from Leibniz institutions can be transferred to universities and other research institutions. At the same time, current Open Science policy-making can provide an adequate forum for reflecting on the impact of AI on the desired openness of the research process and for designing appropriate responses. Stefan Skupien, Open Science coordinator, Berlin University Alliance

Review 2
Topicality 8 (15%)	Thematic Relevance 4 (15%)	Practical relevance 0 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 7 (40%)	Total points (out of 100) 61
The abstract presented is well written and addresses a topic very important to Open Science. However, as part of the review process, I am asked to assess submissions within the context of the conference’s primary theme—the intersection of AI and Open Science. From this perspective, I believe the proposed topic is not directly within the scope of the current conference. That said, it could be adapted to explore the connection between policy making and AI in institutional settings, which would better align with the conference focus.

Review 3
Topicality 10 (15%)	Thematic Relevance 2 (15%)	Practical relevance 2 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 69
The contribution is about the impact of Open Science policies and shows results from a ongoing project IvOS. This is a highly relevant topic, as it questions a well-known connections for culture shift towards Open Science and the impact of policies in practice. Methodologically they are conducting a qualitative content analysis of policy documents. No AI methods are mentioned, but I don’t necessarily expect that for such an analysis. Actually, I think it is even good, that they didn’t try to make any tenuous connection to AI just for the conference submission. The text is well-written, outlines the context of the problem and gives references as well as some links to pre-study results. Unfortunately all references/links [1]-[7] are not resolved. That should be fixed for the final abstract. (I tried to find the results of the pre-study myself with some web search but wasn’t successful.) Some more context especially for the international audience could be given when speaking about the Leibniz institutions. E.g. mention that these are non-university research institutes in Germany. Moreover, it might be interesting to note, that Leibniz association had some process/strategy (?) to let the institutions make such Open Science policies, which then also lead to this high number of policies from them. From my personal experience with policies: Even if the impact on researchers is manageable, other important goals can be achieved with a policy. First, it gives the opportunity to discuss the topic in several rounds or create such a policy in a participative process. Moreover, someone could be made responsible for the topic, or a new position could be created for it. These roles can then work continuously to improve Open Science within the institution in the future.

Review 4
Topicality 10 (15%)	Thematic Relevance 8 (15%)	Practical relevance 6 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 80
The goal of the research described, evaluation of effectiveness of OS policy and an analysis of OS practices, seems very relevant for this conference, although the topic of AI is not discussed in relation to OS in this poster abstract

(21) Reproducibility the New Frontier in AI Governance
Israel Mason-Williams (1,2), Gabryel Mason-Williams (3)
Organization(s): 1: Kings College London; 2: Imperial College London; 3: Queen Mary University of London
Average reviewers ratings: 71.5 (out of 100)

Abstract:

Policymakers for AI are responsible for delivering effective governance mechanisms that can provide oversight into safety concerns. However, the information environment offered to policymakers is characterized by an unnecessarily low signal-to-noise ratio, favouring regulatory capture and creating deep uncertainty and divides on which risks should be prioritized from a governance perspective. We posit that the current speed of publication in AI, combined with the lack of strong scientific standards via weak reproducibility protocols, effectively erodes the power of policymakers to enact meaningful policy and governance protocols. Our paper outlines how AI research could adopt stricter reproducibility guidelines to assist governance endeavours and improve consensus on the risk landscapes posed by AI. We evaluate the forthcoming reproducibility crisis within AI research through the lens of reproducibility crises in other scientific domains and provide a commentary on how adopting preregistration, increased statistical power and negative result publication reproducibility protocols can enable effective AI governance. While we maintain that AI governance must be reactive due to AI’s significant societal implications, we argue that policymakers and governments must consider reproducibility protocols as a core tool in the governance arsenal and demand higher standards for AI research.

Review 1
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 6 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 7 (40%)	Total points (out of 100) 82
This poster focuses on the forthcoming (- i would say already existing and in full effect  -) reproducibility crisis in AI research and its implications for effective “governance”. The approach of learning from reproducibility crises in other fields and applying similar protocols (preregistration, higher statistical power, publishing negative results) to AI is very valuable and definitely thought-provoking. What remains unclear, however, is how the authors envision the translation from improved scientific governance (within the research community) to policy governance (in which domains, including regulators?) — and how these two forms of governance interact or differ. Clarifying what the authors mean by “governance and also strenghtening this link between the different types and domains of governance would strengthen the contribution and help articulate its practical implications for policymakers more concretely.

Review 2
Topicality 8 (15%)	Thematic Relevance 6 (15%)	Practical relevance 2 (15%)	Presentation and Language 2 (15%)	Overall Recommendation 2 (40%)	Total points (out of 100) 35
I am afraid that this abstract is more about open science practices in AI research than on AI for more open research. I needed three attempts to understand the second sentence, please consider re-writing it in easier language. I am fully with you demanding higher reproducibility standards for AI systems that inform policy making (this is how I understand the abstract), but I couldnt extract information on what kind of research you base this call. Daniela Gawehns – NLRN

Review 3
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 6 (15%)	Presentation and Language 10 (15%)	Overall Recommendation 9 (40%)	Total points (out of 100) 90
The abstract addresses a current challenge: the gap between AI research practices and policy making. It highlights how weak reproducibility standards in AI hinder effective governance and risk assessment, which is a critical issue given the rapid progress of AI and its potential for societal change. The paper focusses on reproducibility, which is a central aspect of Open Science. A clear link is made between Open Science practices and the need for sound policy making. This is well aligned with the focus of the conference. The language is clear, concise and easy to read. However, no specific poster or demo component is described in the abstract. The submission would benefit from clarifying what exactly is to be presented or visualised.

Review 4
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 6 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 7 (40%)	Total points (out of 100) 79
Mason-Williams and Mason-Williams address an important topic in in AI reseacrh, namely governance of AI development and ensuring transparency and safety. This is not the first attempt to propose soultions to the described problems. Examples are CONSORT-AI and other reporting guidelines. The abstract scratches here at the surface what the exact solution in this project is. It does not clearly present a tangible practical application. The project desceibed is thus topical and interesting for the cimmunity, but lacks some concrete application. Ulf Toelch BIH QUEST Center for Responsible Reseacrh

(22) Enhancing transparency and reusability through Diamond publishing model:
Françoise Gouzi (1), Francesco Gelati (2), Anne Baillot (1), Toma Tasovac (1)
Organization(s): 1: Digital Research Infrastructure for the Arts and Humanities (DARIAH); 2: University of Hamburg
Average reviewers ratings: 69.0 (out of 100)

Abstract:

Researchers often face financial issues or legal constraints when it comes to publishing their research output, may it be an article, a dataset or a piece of code.

In 2024, DARIAH (Digital Research Infrastructure for the Arts and Humanities) launched the Diamond open access, overlay journal Transformations: A DARIAH Journal (https://transformations.episciences.org/) in order to support Social Sciences and Humanities (SSH) scholars to better embrace Open Science practices. This new serial provides a trusted, non-commercial platform for documenting innovative research activities in the SSH. Submission types include data papers; workflows; pieces of software or code; training materials, as well as traditional scholarly papers. Such a focus on ephemeral or experimental outputs not only offers scholars more options to share efficiently and FAIRly their work; it also aims to promote (and give a greater visibility to) human labour-intensive, data-driven, genuine research compared to traditional scholarly papers.

At a time when it gets harder and harder to identify machine (and especially artificial intelligence)-enhanced forgeries and AI-powered bad scientific practices, a Diamond open access journal offering high openness and transparency (in both editorial and management processes) may be a safe space for good research.

Review 1
Topicality 4 (15%)	Thematic Relevance 4 (15%)	Practical relevance 6 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 5 (40%)	Total points (out of 100) 53
It remained a bit unclear to me what is the goal of the poster: is it just a presentation of the journal? Then I struggle to understand how this connects to the conference topic.

Review 2
Topicality 8 (15%)	Thematic Relevance 8 (15%)	Practical relevance 6 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 77
The abstract does not state whether it is for a poster or demo. Otherwise, it clearly outlines the relationship between Transformations and open science. It also hints at the potential relationship to AI – this could be strengthened. In the poster/ demo itself, it would be useful to address at least some of the following points not covered in the abstract: – sustainability of the financing – technical setup & limitations (e.g. I could not find information about XML versions of the articles or anything related to AI) – the rationale to go for an overlay journal rather than another possible setup – the scope of what Transformations can be overlaid on – applicability of the approach to other fields – interactions between Transformations and support structures like the European Diamond Capacity Hub or Germany’s Servicestelle Diamond Open Access

Review 3
Topicality 6 (15%)	Thematic Relevance 4 (15%)	Practical relevance 4 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 6 (40%)	Total points (out of 100) 57
The abstract describes an overlay journal with built-in tools and policy to promote open science practices. Authors note that the openness and transparency of the journal may help to fraudulent AI-generative work. However, it’s unclear to what extent (if any) the journal makes use of AI as a tool or enables/promotes responsible use of AI tools in research. It’s therefore unclear if this abstract meets the conference aim to explore the intersection between Open Science and Artificial Intelligence Reviewed by: Eileen Clancy

Review 4
Topicality 10 (15%)	Thematic Relevance 10 (15%)	Practical relevance 10 (15%)	Presentation and Language 8 (15%)	Overall Recommendation 8 (40%)	Total points (out of 100) 89
Very relevant project to present at this conference, the goal for the poster is clear.