Automated Coding of Communication Data Using ChatGPT: Consistency Across Subgroups

ArXi:2510.20584v2 Announce Type: replace-cross Assessing communication and collaboration at scale depends on a labor intensive task of coding communication data into categories according to different frameworks. Prior research has established that ChatGPT can be directly instructed with coding rubrics to code the communication data and achieves accuracy comparable to human raters. However, whether the coding from ChatGPT or similar AI technology perform consistently across different graphic groups, such as gender and race, remains unclear. To address this gap, we