AI-Powered Mixed-Methods Approach for Methodological Literature Reviews

In the Generative AI (GenAI) era, social researchers have explored different tools for different phases of research projects. Despite AI and GenAI being highly relevant topics in current literature, few researchers have addressed the question of how to explore GenAI tools based on human-AI collaboration to improve the quality of methodological literature reviews. Based on a real project, “a competence framework for social researchers integrating GenAI”, we propose a detailed analytical model that triangulates evidence from both quantitative and qualitative data analyses. We explore a mixed-methods approach with greater focus on the qualitative dimension (QUAL-quant), integrating the bibliometric analysis into the framework analysis method. The steps we follow are: (1) bibliometric analysis of Scopus publications (434 with the primary string; 84 with the secondary string from 2021-2024); (2) qualitative analysis of 32 publications using inclusion/exclusion criteria to develop initial categories via Chain-of-Thought prompting with Gemini and ChatGPT; (3) indexing using webQDA; (4) pattern interpretation and categorisation using ChatGPT and webQDA; and (5) mapping and interpretation to identify patterns and connections.
The outcomes of this study were the collaboration between social researchers and AI, which led to the identification of the potential and limitations of incorporating GenAI in different steps of methodological literature review. During the construction of the model, the researchers identified as a limitation the fact that they did not collect data from all meetings so that their analysis could improve the framework presented.