AUTHOR=Han Tianyi , Li Dechao , Ma Xingcheng , Hu Nan 

TITLE=Comparing product quality between translation and paraphrasing: Using NLP-assisted evaluation frameworks

JOURNAL=Frontiers in Psychology

VOLUME=Volume 13 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2022.1048132

DOI=10.3389/fpsyg.2022.1048132

ISSN=1664-1078

ABSTRACT=Translation and paraphrasing have been considered effective learning methods in second language acquisition (SLA). While many studies have investigated their similarities and differences in a process-oriented approach, little attention has been paid to the correlation in product quality between them, probably due to difficulties in assessing the quality assessment of translation and paraphrasing. Current quality evaluation methods tend to be either subjective and one-sided or lack consistency and standardization. To address these limitations, we proposed preliminary evaluation frameworks for translation and paraphrasing by incorporating indices from natural language processing (NLP) tools into teachers’ rating rubrics. The indices contained textual features from lexical and syntactic levels extracted from TAASSC and TAALES, and the rating rubrics consisted of both holistic and analytical assessments. Twenty-nine translators were recruited to perform a translation task (translating from Chinese to English) and a paraphrasing task (paraphrasing in English). Their output products were recorded by key-logging technique and graded by three professional translation teachers by using a ten-point Likert Scale. We identified indices that effectively predicted product quality and combined them with expert evaluation rubrics to establish NLP-assisted evaluation frameworks for translation and paraphrasing. By using these frameworks, we found a closely related performance between the two tasks, evidenced by several shared predictive indices in lexical sophistication and strong positive correlations between translated and paraphrased text quality according to all the rating metrics. These similarities suggest a shared language competence and mental strategies in different types of translation activities and perhaps in other forms of language tasks. Meanwhile, we also observed differences in the most salient textual features between translations and paraphrases, mainly due to the different processing costs required by the two tasks. These findings enrich our understanding of the shared ground and divergences in product quality between translation and paraphrasing and shed light on the pedagogical application of translation activities in classroom teaching. Moreover, the proposed evaluation framework can be seen as a new tool for translation quality assessment. It can also bring insights into the development of standardized evaluation frameworks in translation and paraphrasing in the future.