AI vs Human. Sentiment analysis of children's stories on living with cochlear implant.

Introduction

The study was designed to examine the ability of artificial intelligence (AI) to evaluate the sentiment of written statements. The utterances came from interviews conducted with children with congenital deafness who received cochlear implants in the prelingual period. A key submission criteria for admission to interview was having experience with the implant for at least five years.

The interviews were intended to bring the children's perspective on their daily functioning. This knowledge allows to anticipate easy and difficult emotionally or intellectually situations. This leads to the description of typical and atypical (emotionally difficult) situations and the development of ways to cope with them, which are then passed on to educators, teachers, psychologists supporting children with cochlear implants. Therefore, it is particularly important to correctly identify the emotions expressed by children in the stories they present.

Goals and Methods

Study was carried out through in-depth interviews. Recordings ware transcribed and saved as text files (in Polish). Research material was subjected to a threefold analysis (triangulation). Sentiment evaluation of selected utterances was performed by a human coder, by a multilingual AI (ChatGPT 4.0), and by AI built based on Polish corpus (AI that ‘understands’ contexts of utterances in Polish - BIELIK-11B-v2). The aim of this triangulation was to explore the usefulness of AI-supported analyses vs analyses performed by a human coder to improve data driven decision processes.

Results

High-level conclusions about the usefulness and limitations of AI-supported sentiment analysis versus human coding will be provided. Significant similarities and differences in the sentiment evaluation done by multilingual AI, single-language rooted AI, human coder analysis will be identified.

Conclusions

The aim of this study is to develop recommendations for sentiment analyses supported by the two types of AI mentioned, as well as to raise awareness of the differences of AI vs human coder work.