Communication Arts: Faculty Publications

Developing a ChatGPT-Based Text Extraction Model to Analyze Effective Communication Elements in Pandemic-Related Social Q&A Responses

Document Type

Conference Proceeding

Publication Date

2-19-2024

Publication Title

6th International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2024

DOI

10.1109/icaiic60209.2024.10463227

ISBN

9798350344349

Abstract

The present study attempts to use the large language model (LLM) to create a model that identifies Aristotle's rhetorical principles-ethos (source credibility), pathos (emotional appeal), and logos (logic)-in response to COVID-19 information on a question-And-Answer community (social Q&A platform). The model differentiates between the most upvoted and random answers to analyze the presence of subdimensions of these rhetorical principles. The research utilized answers to COVID-19 questions on Naver Knowledge-iN, the most popular social Q&A platform in South Korea. A set of 193 answer pairs was randomly selected for training (135 pairs) and testing (58 pairs). These answers were coded for the three rhetorical principles and their subdimensions by researchers, which were used to refine models based on GPT 3.5 technology. The F1 scores were improved to. 88 (ethos),. 81 (pathos), and. 69 (logos). The fine-Tuned models were employed to analyze 128 newly drawn answer pairs of the most upvoted answers and random answers. The paired sample t-Tests indicated that rhetorical elements of logos such as factual information and logical reasoning were positively associated with health consumers' preference of information (answers) while the other rhetorical principles of ethos and pathos were not associated with consumer preference of health information. By utilizing the LLM for the analysis of persuasive content, which has been typically conducted manually with much labor and time, this study not only demonstrates feasibility of using the LLM in studies of the humanities and social sciences, but also contributes to expanding the horizon in the field of AI text extraction.

Comments

Georgia Southern University faculty member, Beom Jun Bae, co-authored, "Developing a ChatGPT-Based Text Extraction Model to Analyze Effective Communication Elements in Pandemic-Related Social Q&A Responses."

Copyright

This work is archived and distributed under the repository's Standard Copyright and Reuse License (opens in new tab). End users may copy, store, and distribute this work without restriction. For all other uses, permission must be obtained from the copyright owners or their authorized agents.

Link to Full Text

Share

COinS