Danielle Agnello is a Marie Curie doctoral fellow within the Health CASCADE project. Her research focuses on investigating methods that are most appropriate for evidence-based co-creation.
Quentin Loisel is a Marie Curie doctoral fellow within the Health CASCADE project. His research focuses on bridging the gap between humans and technology in co-creation.
Over the past year, Danielle and Quentin have coordinated the building of a novel co-creation database. They have worked hard alongside others in the Health CASCADE network to create a repository of co-creation knowledge. In this blog post, they reflect on why they set out to build the database, how well it captures the target literature, and provide instructions on how you can start using it!
We did it! We managed to collate all scientific articles about co-creation in all fields of research in one database. This is the first essential step in our journey to develop a more evidence-based methodology to make co-creation trustworthy.
We left no stone unturned, and after eight months of working with the entire Health CASCADE team, aided by several artificial intelligence co-researchers, we reviewed over 160,000 scientific publications and created a quality assured open access database! Quentin describes the process and the scale of the challenge in his blog post “How to climb a Giant: An innovative approach to reviewing co-creation literature.” It was a big investment, but it was worth the effort.
WHY MAKE A DATABASE?
Firstly, because as Danielle shared in her blog “Attempting to Co-create at the World Health Organization: Addressing wicked problems in public health,” finding evidence about co-creation to guide practice is challenging. Co-creation is used in many different fields, from tourism, the software industry, to environmental sciences — so knowledge is fragmented. There are also multiple definitions and different terminologies such as co-production, co-design which are not clearly delineated. We also needed to consider where participatory methodologies fit into this ‘co-creation’ picture.
Secondly, because we needed an invaluable resource to mine efficiently to advance methodological research about co-creation. For instance, we aimed to synthesize the existing knowledge about co-creation, to understand what the biggest gaps in knowledge are, as well as find the success stories and best practices to build from.
In the process of making the database, we have collectively learned a lot about new practices in knowledge synthesis and how artificial intelligence (AI) can be effectively used and is transforming the way we can conduct more robust systematic reviews. For instance, we used a reliable AI co-researcher that can support the screening of a massive volume of papers called ASReview. You can read more about it here: Van de Schoot et al., 2021.
Additionally, the Health Cascade consortium is collectively conducting several systematic reviews about different aspects of co-creation in our database — as all the knowledge is now at our fingertips! You can do so as well since the Health CASCADE Co-creation Database is open access (see details below).
After going through this novel and systematic curation process, we strongly believe this database represents a unified concept of co-creation in a diverse set of fields and includes a mixture of primary studies, reviews, and methodological papers. In this way, a researcher can not only find examples of when co-creation was used, but also investigate the theories, ethics, and methodological underpinning of co-creation.
THE DATABASE IN NUMBERS
Number of Articles: The final database consists of 13,501 citations. This is the entirety of the relevant scientific literature about co-creation.
Quality in numbers: Since we created this database to be the foundation of our work at Health CASCADE, we wanted to ensure that it is of the highest quality possible. Therefore, we ran multiple quality checks, resulting in a false positive rate of 20.36% — meaning potentially about 20% are papers that could be removed from the database. This aligned well with our goal of over-including articles, so we didn’t miss anything. We also have a false negative rate of 9.37%, which is on average lower than or similar to a human (~10%; see Wang et al., 2020), meaning it’s likely that 9% of relevant papers were excluded from this database. These results reveal that most of the references we were looking for are indeed in the final database. This is good news for those that want to use the database for their own studies!
Keywords in numbers: Since we used a diverse set of keywords to capture all the literature that is about our concept of co-creation, it was important to visualize the outcomes, post-screening, to see how the terminology stood up against each other. To no surprise, the predominant terms in the database are co-creation, co-production, user involvement and co-design, comprising 34% of the final database. However, the biggest hitter is participatory, coming in at 60% of the database — as it was often used alongside the other three terms.
You can read more about the analysis of the database, and additional metadata, in our publication (see more below).
YOU CAN USE IT TOO!
So why are we telling you all about this open access database? Well, we at Health CASCADE are aware that we are not the only people passionate about co-creation. We also know that countless researchers are dedicating themselves to using and perfecting its various techniques. Therefore, we’d like to invite you to use the database, and help us build it further to begin to shape our collective understanding of co-creation. The database is open to access and available here on ZENODO (version 1.5): https://doi.org/10.5281/zenodo.6773028. We are also preparing a scientific article about the development of this database, so keep an eye out on our social media channels for when it’s available.
If you’d like to hear about updates and follow the evolution of our work, please sign up for our newsletter here. Meanwhile, let us know what you think about our database by reaching out and engaging with us on social media with the hashtags #HealthCASCADE #HC-CCDB #CoCreationDatabase.
Twitter: https://twitter.com/health_cascade
Instagram: https://www.instagram.com/healthcascade.eu/
LinkedIn: https://www.linkedin.com/company/health-cascade-eu/mycompany/
Website: https://healthcascade.eu/
**If you have any questions about using the database, you can reach out to our team here: https://healthcascade.eu/contact/
Acknowledgements: The work on the HC Co-creation database was co-led by Danielle Agnello (ESR3) and Quentin Loisel (ESR6), under the supervision of Professor Sebastien Chastin. We’d like to thank the amazing Health CASCADE colleagues who took part in the challenging screening process (in alphabetical order): George Balaskas, Giuliana Longworth, Janneke De Boer, Katrina Messiha, Kunshan Goh, Lea Rahel Delfmann, Lauren McCaffrey, Maria Gine Garriga, Mira Vogelsang, Qingfan An, and Rabab Chrifou, and to Ukachukwu Abaraogu for his input on screening and the methodology.