.JPG "Mutrah (2).JPG")
New Directions in Cognitive and Empirical Translation Research
Riitta Jääskeläinen, University of Eastern Finland
Séverine Hubscher-Davidson, Aston University, Birmingham
Isabel Lacruz, Kent State University, Ohio, USA
Cognitive research in translation and interpreting has reached a critical threshold of maturity that is triggering rapid expansion along several innovative paths. Some of these might be grouped under three areas which have recently grown in popularity: (1) innovative methodologies, (2) translation expertise, and (3) translation utility. This panel, which includes contributions that investigate new directions in these areas, is intended as a platform to explore and exchange views on the future development of cognitive and empirical translation research.
Innovative methodologies: One key driver of expansion in the field is the development and use of diverse empirical and experimental methodologies, which are often borrowed or adapted from other disciplines. New approaches, which increasingly involve multiple data sources, sometimes both qualitative and quantitative, are beginning to provide very rich information on all aspects of translation as a cognitive activity, including the roles of affect and metacognition. Large scale standardization of research instruments including questionnaires and on-line tools for extraction and manipulation of shared data will enable researchers to move beyond the limitations of case studies to carry out research at scale that allows for substantial generalization.
Translation expertise: More effective methodologies in cognitively oriented translation research are yielding deeper understanding of the nature and acquisition of translation competence and the development of translation expertise. This new knowledge will help guide the training of future translators as they develop into fully-fledged professionals. It will also positively impact the work of translators as they strive to effectively meet workplace needs and expectations.
Translation utility: Prior research has often focused on evaluating translations in terms of how accurately they transfer information between languages, according to semantic, syntactic, stylistic, cultural, and other similar standards. Increasingly, however, the basis of evaluation is moving toward judgment of utility – in other words, measuring how effortful it is for end-users to use a translation and how well it meets their needs. This change in focus is partly driven by increased need for post-editing of machine translations and the desirability of tuning machine translation output to minimize effort for post-editors rather than maximize formal accuracy. For this it is especially important to understand and measure the nature of cognitive effort in various aspects of the translation process, a theme that is explored by several contributors to this panel.
For informal enquiries: [riittaDOTjaaskelainenATuefDOTfi]
Riitta Jääskeläinen, (University of Eastern Finland) is Professor of English (translation and interpreting). Her teaching includes practical translation as well as research-related courses and thesis supervision. Her research interests in the area of translation process research include methodology and translators' expertise. She has published overview articles on think-aloud (Routledge Encyclopedia of Translation Studies, Benjamins Handbook of Translation Studies, vol. 1), translation process research (Oxford Handbook of Translation Studies) and translation psychology (Benjamins Handbook of Translation Studies, vol. 3).
Séverine Hubscher-Davidson (Kent State University) is Assistant Professor of Translation Studies. She teaches doctoral courses on empirical research methods for translation, and translation and cognition, as well as master level translation practice courses. Lacruz' current research interests include investigation of the mental processes involved in translation and post-editing. She has published theoretical and empirical articles on cognitive aspects of translation and post-editing.
Isabel Lacruz (Aston University, Birmingham) is a Lecturer in Translation Studies. She teaches both translation theory and practice, and her research interests are in the areas of translation process research, translators' personalities and emotional intelligence, as well as individual differences more generally. She has published articles mainly on psychological aspects of the translation process, and is currently working on a book on the topic of emotionality in translation.
Each paper is allocated with a 20 minutes time slot + 10 minutes discussion.
Discussion time at the end of each paper
Introduction: Riitta Jääskeläinen (10 minutes)
PART 1: Translation expertise/Innovative methodologies
Chair: Isabel Lacruz
PAPER 1:
Title: Tolerance for Ambiguity and the Translation Profession: A New Direction for Empirical Research in Translation
Speaker: Séverine Hubscher-Davidson, Aston University
PAPER 2:
Title: The role of expertise in emotion regulation: Exploring the effect of expertise on translation performance under emotional stir
Speakers: Ana Mª Rojo López and Marina Ramos Caro, University of Murcia
PAPER 3:
Title: The mind behind - Attributive metacognition in translation and its effect on the translator
Speaker: Annegret Sturm, University of Geneva
PAPER 4:
Title: Translation process differences between literary and technical translators
Speakers: Kristian Tangsgaard Hvelplund, University of Copenhagen, and Barbara Dragsted, Copenhagen Business School
PART 2: Translation utility
Chair: Séverine Hubscher-Davidson
PAPER 5:
Title: How editors read: An eye-tracking study of the effects of editorial experience and task instruction on reading behavior
Speaker: Melanie Ann Law, North-West University
PAPER 6:
Title: Studying the dynamics of term creation in European equally authentic texts in 24 official language versions
Speaker: Rita Temmerman, VrijeUniversiteit Brussel
PAPER 7:
Title: Cognitive Effort in Machine Translation Post-Editing: A Mixed-Method Approach
Speaker: Lucas Nunes Vieira, Newcastle University
PAPER 8:
Title: Cognitive effort in human translation and post-editing: an analysis of pupil dilation and fixation duration on metaphors
Speaker: Arlene Koglin, Federal University of Minas Gerais
PAPER 9:
Title: Source text features and their relationship to cognitive demand and cognitive effort in post-editing
Speaker: Isabel Lacruz, Kent State University
Wrap-up session (10 minutes): Riitta Jääskeläinen
PAPER TITLES, ABSTRACTS AND BIONOTES
PAPER 1:
Title: Tolerance for ambiguity and the translation profession: A new direction for empirical research in translation
Speaker: Séverine Hubscher-Davidson, Aston University
Abstract:
There is a growing need to describe the profile of participants in translation process research and to draw inferences between individual traits and translation competences (Saldanha and O'Brien 2013: 146). Muñoz Martín argues that scores from an intelligence test could become a predictor of translation success (2010: 92), but recent evidence suggests that personalities and other individual differences may also have some explanatory power for success in translation (e.g. Hubscher-Davidson 2009). The potential links between participant profiles and translation quality are interesting to investigate as they increase our understanding of the different psychological mechanisms at play in translation, and could help students assess their own strengths and weaknesses as translators (Jääskeläinen 2012: 194). In this paper, the Tolerance for Ambiguity (TA) personality trait will be discussed in the context of professional translation, and the following research questions will be explored: does TA increase with professional experience in translation? Does TA predict translators' job satisfaction or job success? Is TA an important trait for personality profiling in translation?
Due to the very nature of translation, tolerating ambiguity is clearly a key skill. Translators are continuously faced with having to make difficult translation decisions and, more often than not, there is no right answer but many possible alternative solutions to the translation of a ST segment. Benjamin (2012: 40) acknowledged this tricky aspect of a translator's work: "Precisely because the meaning of the original formulation, the one to be translated, is not singular, translation begins with the 'ambiguous'". In addition, tolerance for ambiguity is said to be positively related to "performance in the global work environment and in cross-cultural settings" (Herman et al 2010). Therefore, being able to perceive ambiguous situations as desirable, or at least non-threatening, and reacting well to unfamiliar and complex stimuli, are clearly essential components of successful translation performance. It is therefore interesting to investigate the relationship between professional translators' performance and their ability to cope with change, ambiguity, and conflicting perspectives.
In this interdisciplinary study, a number of professional translators were contacted over the space of 3 months in 2014, and asked to fill in (1) a background questionnaire and (2) the Tolerance for Ambiguity Scale (TAS). A total of 91 professional translators took part in the study. The TAS is new to empirical translation research, though it has been successfully employed in other multilingual contexts (e.g. Dewaele and Wei 2013). Both questionnaires were internet-based, as this is said to reduce social desirability and enabled the participation of a large number of translators. The aim was to explore, with an innovative psychometric instrument, an under-researched aspect of translators' personalities and its relationship with work performance. The TAS contains items reflecting ambiguous stimuli commonly experienced in multilingual contexts, and can therefore be used in cross-cultural research and practice. In this paper, I will present the main results of the study, explain the methods employed, and discuss the potential importance of TA as a new direction for personality profiling in translation process research.
Bionote:
Séverine Hubscher-Davidson is a Lecturer in Translation Studies at Aston University, Birmingham. She teaches both translation theory and practice, and her research interests are in the areas of translation process research, translators' personalities and emotional intelligence, as well as individual differences more generally. She has published articles mainly on psychological aspects of the translation process, and is currently working on a book on the topic of emotionality in translation.
PAPER 3:
Title: Investigating emotional aspects of the translation process: an interdisciplinary methodological framework
Speaker: Caroline Lehr, University of Geneva
Abstract:
Translation processes have only over the past decades become the subject of systematic empirical studies and methodologies in process-oriented research need to be developed further. As the improvement of methodology continues to be an important topic, translation researchers are particularly concerned with finding methods that enable deeper insights into the nature of decision-making processes during translation. In psychology, numerous research has demonstrated the importance of emotion for human decision-making, however, their importance for the translation process has to date scarcely been considered and process research lacks appropriate methods for the investigation of emotional aspects.
To address this gap, the present paper briefly outlines how emotions are seen within the componential view of emotion and how emotions can exert influences on human decision-making. It then focuses on how these influences can be tested empirically by presenting an interdisciplinary methodological framework which integrates current methods used in translation process research and methods used in psychological research to induce and measure emotions. The framework is situated within the mixed-methods paradigm, allowing for a flexible integration and weighting of both quantitative and qualitative methods, and comprises four groups of methods: 1) emotion induction procedures, 2) online measures of emotion, 3) offline measures of emotion, and 4) assessments of inter-individual differences. Whereas the first group includes emotion induction procedures used in psychological research, such as bogus performance feedback, social interaction, or music; the second and third groups are based on online- and offline-methods used in translation process research, which are complemented by psychological assessments of emotional responding and are subdivided accordingly into the three categories of emotion measures: self-reports of emotional experience (valence/arousal scales, assessments of discrete emotions), physiological measures (brain states, activity of the autonomic nervous system), and behavior (behavioral expression of emotion, assessments of task performance during an emotional state). In the fourth group, relevant psychometric assessments, as for example instruments to assess emotional intelligence or emotion regulation strategies, complement assessments of inter-individual differences already employed in translation process research. The methods included in the four groups are outlined and their strengths and weaknesses are assessed, as well as their suitability for studies emphasizing ecological validity or experimental control and research addressing basic or more applied questions. Finally, the paper exemplifies how the different methods can be combined for designing experimental and correlational studies in translation process research that wish to empirically address the multiple questions that arise from the fact that emotion is a central organizing construct of human cognition.
Bionote:
Caroline Lehr studied translation at the University of Geneva and the University of Heidelberg. After graduation, she became a teaching and research assistant at the University of Geneva and was responsible for translation classes at Bachelor and Masters level. For her PhD, which she received in 2014, she conducted interdisciplinary research in collaboration with the Swiss Center for Affective Sciences.
PAPER 2:
Title: The role of expertise in emotion regulation: Exploring the effect of expertise on translation performance under emotional stir
Speakers: Ana Mª Rojo López and Marina Ramos Caro, University of Murcia
Abstract:
The influence of emotions on the translation process has been so far barely explored in translation process research. But new views on cognition adopted in recent years have cleared the ground to explore the role that psychological and emotional factors play in the translation process. Regarding emotion, the work by Lehr (2013) has recently proposed an empirical approach to research the impact of emotion on translation performance. Her research suggests that positive emotions may enhance facets of creativity in translation, in particular on idiomatic expressions and stylistic adequacy, whereas negative emotions may foster accuracy in translating terminology. Results of her work indicate that positive and negative emotions may trigger different processing styles. Other pioneering work that has also argued that emotion regulation may affect translation performance is that of Hubscher-Davidson (2013). She has shown that personality traits like intuition or emotional intelligence also play a role in regulating translators' behaviour and can lend support to more successful translating. But the question still remains as to the role that professional expertise may play in the process of emotion regulation and the final quality of their performance.
The present study aims to investigate the influence that expertise may exert on emotion regulation and its consequences for translation performance. Our study replicates Lehr's methodology, but also explores the influence of personality factors and level of expertise on the induced emotional impact. The experiment compares the performance of translation students with that of professional translators and assesses trait variation in the participants' psychological resilience (Block and Kremen's ego-resiliency scale (ER89)) and creativity (CREA, Torrance Test of Creative Thinking). Participants were asked to provide a translation of an emotional text, which was rated for accuracy and creativity. Later, they were randomly assigned to a positive or negative feedback group and received false feedback on their performance. Immediately afterwards they were asked to translate a second text, whose ratings for accuracy and creativity were compared to those from their first translation. A retrospective interview was finally carried out to obtain data on the participants' subjective feelings. Results of the study suggest that personality factors and level of expertise play a decisive role in regulating emotion and guiding translational behaviour and may foster enhanced translation performance even in emotionally stirring situations.
Bionote:
Ana Rojo is Senior Lecturer in Translation at the University of Murcia (Spain), where she has been Head of the Translation and Interpreting Department for five years and is currently Coordinator of the Master of Editorial Translation and President of the PhD Commission. Her main areas of research are the fields of Translation and Cognitive Linguistics. She has authored and co-edited several books and monograph issues and written many scholarly articles which have appeared either in specialised national and international journals or as book chapters published by several national and international publishing houses
PAPER 3
Title: The mind behind - Attributive metacognition in translation and its effect on the translator
Speaker: Annegret Sturm,University of Geneva
Abstract:
In the context of translation, the term "metacognition" is most often used to refer to the translator's monitoring processes (Angelone 2010) or his/her awareness of the his/her own knowledge during the translation process (Hurtado Albir 2010). However, this type of self-referring metacognition (evaluative metacognition, Proust 2013) is not the only form of metacognition involved in translation. Even more central is the representation of other minds (Sperber 2000), also called "attributive metacognition" (Proust 2013). Although different branches of translation studies agree that translation is in itself a form of metarepresentation (Gutt 2000, Wolff 2002, Sturge 2007), one of the major implications of this fundamental claim has received little attention up to now, namely that translators are dealing with the content of other minds (Wilss 1992). Based on Hermans (2007), I shall argue that translation is a higher-order metarepresentation. Being the main agent in an "other-directed act" (Robinson 2001), the translator has to metarepresent two minds during the translation process, the source text author and the target audience. Its triadic nature makes translation a special form of communication. As a hybrid form of reported speech it can neither be classified as quotation nor as indirect speech. These distinctive features of translation should result in an enhanced metacognitive effort as compared to standard communication. Translators who are constantly operating on this higher metacognitive level should thus have a higher cognitive proficiency than non-translators. Developmental psychologists call our capacity of representing another person's mind 'Theory of Mind' (ToM). ToM develops throughout life (Kobayashi 2008) just as our pragmatic competence continues to evolve up to adulthood (Cummings 2007). Bilingualism is found to have an influence on ToM performance in children (Kovács 2009).
To test the hypothesis whether translation enhances metacognitive proficiency, I triangulated data of three experiments comparing students with two different levels of translation training (BA/ MA) using fMRI, eye tracking, key logging and translation quality analysis. Results of the fMRI study show that the metacognitive network is implicated in the translation condition. The behavioural data show that MA students have a clear advantage over BA students in terms of text comprehension and processing in the metacognitively demanding condition. However, the analysis of the translation products indicates that BA students develop strategies to compensate for their lack of metacognitive sensitivity in order to produce results which are comparable to those of MA students. Overall, this paper makes a point for transdisciplinarity in translation research. Translation does not only share common theoretical frameworks with other fields, but also contributes to them in a relevant way. As the world's population is becoming increasingly bilingual, life sciences encourage more research into translation (Abutalebi & Green 2007). Furthermore, psychologists have advocated for more ToM research in adults (Apperly 2014). Translation offers great potential for the study of the macrofeatures of understanding.
Bionote:
Annegret Sturm studied Interpreting, Translation and Pragmatics at the Universities of Leipzig and Geneva. She has been working as a professional translator in a think tank and specialized as a freelance translator in medical translation. Her research on the translator and translation competence is funded by the Swiss National Science Foundation.
PAPER 4:
Title: Translation process differences between literary and technical translators
Speaker: Kristian Tangsgaard Hvelplund, University of Copenhagen, and Barbara Dragsted, Copenhagen Business School
Abstract:
It is common that translators specialise in certain domains and prefer working with specific text genres. Focussing on a few familiar text types rather than working with many different genres and topics allows the translator to gain experience and build up specialised expertise for those specific types of text. Consequently, technical translators are for example reluctant to take on literary translation jobs, and literary translators generally stay clear of technical texts. The relationship between text genre preference and familiarity and cognitive processing of translation has received only little attention in Translation Studies (Sannholm 2010). Differences in processing behaviours between literary translation and non-literary translation have not been studied systematically, and it is not yet known how translators process text genres that they are not very familiar with. Addressing these issues could add a new dimension to the characterisation of different translation styles and profiles (Dragsted and Carl 2013), and generate new insights into how translators behave when faced with a non-familiar task requiring them to rely on a different set of strategies and competences (e.g. Shreve 1997).
This paper presents the findings from a joint research project carried out at the University of Copenhagen and the Copenhagen Business School. The project aims at identifying and comparing processing behaviours of two groups of translators: 10 translators specialising in literary translation and 10 translators specialising in technical translation. The translators in both groups translate both a literary text and a technical text, i.e. they carry out both a familiar and a non-familiar task. Data from eye tracking, keylogging and retrospective interviews from the two groups of translators are recorded in a series of data collection sessions. The project thus triangulates qualitative and quantitative methods. In addition, it uses a novel method of presenting process data by means of overlaying translation progression graphs (Carl et al. 2011). The quality of the quantitative process data are carefully and thoroughly assessed (Hvelplund 2014) and the data are analysed inferentially using linear mixed-effects regression modelling (LMER) (cf. e.g. Baayen 2009, Balling and Hvelplund forthcoming) in order to be able to make more confident generalizations. The specific research aims of the project are to: 1) identify processing behaviour and strategies shared by literary translators and technical translators; 2) identify processing behaviour and strategies, which are unique to either literary or technical translators working with familiar text genres; 3) compare the processing behaviour and strategies involved in the translation of familiar text genres with the behaviour and strategies involved in the translation of non-familiar genres. Thus, the project aims on the one hand to gain more insight into the special competences and expertise associated with literary and technical translation respectively, and on the other hand to identify behavioural characteristics which are not apparently genre-dependent. Furthermore, the study will investigate if and to what extent experienced translators fall back on behaviour and strategies generally associated with novices when working with non-familiar genres.
Bionote:
Kristian Tangsgaard Hvelplund is Assistant Professor in the Department of English, Germanic and Romance Studies at the University of Copenhagen. He teaches courses on literary translation and on cognitive aspects of translation, and his research focuses on cognitive processes in translation, including audiovisual translation, and processes in reading and writing. Barbara Dragsted is Associate Professor at the Department of International Business Communication, Copenhagen Business School, where she is a member of the CRITT center. She teaches business communication and specialised translation, and her research interests include cognitive processes in translation and LSP translation and communication.
PART 2: Translation utility
PAPER 5:
Title: How editors read: An eye-tracking study of the effects of editorial experience and task instruction on reading behavior
Speaker: Melanie Ann Law, North-West University
Abstract:
Eye-tracking has been used as a method to study reading and the online operations involved in reading for several decades. While the focus of this research has mostly been on the processing of words and strings of words, attention is now shifting to reading behaviour as it relates to whole texts. One of the questions that has received attention is how reading behaviour changes depending on task instruction and task experience. This question has been the focus of several recent studies in the fields of translation and proofreading (as a component of the writing process). However, although reading is a critical process in the work of editors, to date there is no research on the reading behaviour of professional editors.
This paper reports on the preliminary findings of a pilot study that made use of eye-tracking to compare the reading behaviour of two groups of participants: editors and non-editors. The aims of the study were to establish if editors' reading behaviour exhibited differences when compared to non-editors, and across two reading tasks (reading for comprehension and reading with the aim to edit). Existing research on translation and proofreading has demonstrated that task instructions influence reading behaviour (Schotter et al., 2014; Göpferich et al., 2008). Studies on reading behaviour in translation have also suggested that experience may account for differences in reading behaviour (Jakobsen & Jensen, 2008). Based on this, this study hypothesised that there would be differences in the way editor and non-editor participants read texts, that task instructions to read for different purposes would lead to a change in reading behaviour for both groups, and that the instruction to read with the aim to edit would particularly strongly influence editors' reading. To test these hypotheses, differences in eye-tracking measures were examined. Specifically, the dependent variables of fixation duration, total fixation count, saccade length, total saccade count, and total task time, as they relate to the independent variables of experience and task instruction, were investigated.
Bionote:
Melanie Law is a lecturer in Language Practice at the North-West University's Vaal Triangle campus. Melanie holds an MA in Language Practice and is currently enrolled for a PhD in Language Practice. Her most recent publication explores the various factors that influence the work and tasks of professional editors. Her current research focuses on the integration of process-oriented methodologies in the investigation of professional editorial work.
PAPER 6:
Title: Studying the dynamics of term creation in European equally authentic texts in 24 official language versions
Speaker: Rita Temmerman, VrijeUniversiteit Brussel
Abstract:
According to Shuibhne (2008) the European multilingual policy amounts to little more than a 'myth of equality' among languages. Most of the European information flow moves from an original draft in Euro-English to official translations into Euro-varieties of (in principle and at least) all the other 23 languages (Euro-Italian, Euro-Dutch, Euro-Maltese, Euro-Finnish, etc.). In earlier work the dynamics of terminological understanding and the impact of terminology creation in a socio-cognitive multilingual reality was described and it was demonstrated that translators are at the basis of many coinages in the target languages that are given equivalent status to neologisms in Euro-English. This means that translators are involved in what Sager (1990: 80) calls "secondary term formation". In the present contribution we discuss European secondary term formation in translations against the background of recent insights in several disciplines. We start from the concept of "interlingual uncertainty", which is --as Cao (2003) demonstrates--, a characteristic of all bilingual and multilingual legal texts. Then we go into the need for balance between precision and vagueness, a requirement for all legal documents. On one hand a legal text has to be maximally determinate and precise, on the other hand the text has to cover every relevant situation and therefore some vagueness is essential. Yet vagueness may cause problems in a setting of "equal authenticity" as explained by Schilling (2010). Europe pledges allegiance to the protection of legitimate expectations and to the non-discrimination principle. The mix of the EU's equal authenticity principle, conceptual divergence, cultural load of terminology combined with misinterpretations of translators can cause serious problems. It is quite common that equally authentic language versions of a Community Law have different interpretations if taken on their own. Yet a citizen has every reason and the right to trust his or her own language version. We will use examples to illustrate that how a term is interpreted may depend on several contextual factors, even though the rule of law forms part of a shared European cultural space. When translating European texts, translators should be aware of these factors in order to achieve optimal quality in secondary term formation. Insights from cultural terminology theory (Diki-Kidiri 2008) may benefit the quality of secondary term formation.
Bionote:
Rita Temmerman is Professor in translation, multilingual intercultural communication and terminology studies at Department of Applied Linguistics, VrijeUniversiteit Brussel (CVC). Her research contributed to the sociocognitive approach in terminology management. The research focus is on several issues related to the translation of special language in general and terminology in particular, such as: application-oriented terminology analysis, dynamic multilingual neology creation, terminological variation in a multilingual setting, multilingual terminology and cognition, terminology harmonisation within the EU, understanding terminology in cognitive, linguistic, situational and cultural contexts, metaphor studies, dynamic systems in language, culture-bound understanding, terminology engineering software (using natural language processing technology).
PAPER 7:
Title: Cognitive effort in machine translation post-editing: A mixed-method approach
Speaker: Lucas Nunes Vieira, Newcastle University
Abstract:
In view of the popularity of machine translation (MT) post-editing (PE) as a solution to the ever-increasing demands placed on human translation, PE effort and its measurement have quickly become common topics of investigation in Translation Studies and related fields, with research findings having applications that involve the estimation of pay rates in PE as well as a more robust prediction of raw MT output quality. In this talk, a mixed-method approach to the investigation of cognitive effort in PE will be presented in the context of a study involving an analysis of think-aloud protocols (TAPs) carried out through the lens of large-scale eye-tracking and key-logging data.
The study sets out to uncover qualitative information regarding the nature of mental processes taking place at task moments corresponding to different estimated levels of the effort experienced by participants, also providing insights into methodological aspects pertaining to a converging use of automatic logs and TAPs. At an initial stage (S1) of the investigation, nineteen subjects were asked to post-edit English machine translations of excerpts of two French news articles in tasks that involved eye tracking, key logging and a sentence level measurement of perceived mental effort based on a scale borrowed from the field of Educational Psychology. TAPs were not used at this stage to avoid any potential interference they could have with automatic logs and participants' perceptions. In a subsequent stage (S2), a new sample of nine participants, comparable to the previous sample in source language proficiency, previous experience, and sentiment towards MT, was asked to post-edit the same texts, but now under a think-aloud condition. This dual setting enabled a contrast of large-scale information gathered in S1 with in-depth think-aloud data gathered in S2. Preliminary results are indicative of potential connections between different levels of cognitive effort and the different aspects of the activity participants focus on, such as grammar, lexis, and readership-specific issues. A preliminary analysis of the data is also suggestive of the potential of TAPs when used as a tool to carry out qualitative analyses informed by large-scale quantitative data. Despite having come under criticism in recent years, TAPs have the advantage of providing information that lies at a deeper level in comparison with automatic logs. Automatic logging methods, on the other hand, are certainly attractive in view of their relative objectivity and less invasive nature. In view of the advantages that are inherent to both methods, this talk is hoped to provide insights into how TAPs and automatic logs can be combined in a strategy that tentatively avoids a trade-off scenario which at first sight may seem inevitable. Further details regarding potential links between TAPs and eye tracking as indices of cognitive effort in PE are also provided, constituting, to the best knowledge of the author, the first study where information of this kind is made available in the context of PE, with findings that are hoped to also inform cognitive investigations in translation as well as traditional revision.
Bionote:
Lucas Nunes Vieira is a PhD student in Translation Studies at Newcastle University, in the UK. After completing a Linguistics and Modern Languages degree in Brazil, he did a joint MA in Natural Language Processing and Human Language Technology at the University of Algarve in Portugal, and the University of Franche-Comté in France. In the past seven years he has also worked as an editor and translator for publishers in Brazil and in the UK.
PAPER 8:
Title: Cognitive effort in human translation and post-editing: an analysis of pupil dilation and fixation duration on metaphors
Speaker: Arlene Koglin, Federal University of Minas Gerais
Abstract:
The combination of temporal, technical and cognitive effort has been proposed as metrics to evaluate the feasibility of post-editing (Krings, 2001). Whereas temporal and technical effort are easier to measure, cognitive effort is more complex and therefore may require a combination of measures in order to have more reliable results, as well as a deeper understanding of cognitive processing. Translation process research usually relies on fixation duration and fixation count to measure cognitive effort; however, more recent studies have tested pupil size as an indicator of cognitive processing. Experimental evidence from Interpreting Studies (Hyönä, Tommola & Alaja, 1995) suggests that pupil size reflects momentary variations in processing load during a translation task. Their findings also indicate that words that are more difficult to translate induced higher levels of pupil dilation than easily translatable words. These results are encouraging with respect to the use of pupil dilation as an indicator of variation in cognitive processing, but further translation-process-driven studies are necessary to test the validity of pupil dilation as a standard measure of cognitive effort.
This presentation reports on an empirical study analyzing cognitive effort required to translate from scratch in comparison to post-editing a machine-translated output. More specifically, the study has two objectives. First, we aim at investigating the cognitive effort required to post-edit machine-translated metaphors compared to the translation of metaphors by analyzing differences between fixation duration and pupil dilation in areas of interest (AOIs) in source and target texts. The second aim is to test whether the pupillary response can be applied to study cognitive processing by correlating it with fixation duration, which is a well-established measure of cognitive effort in translation process studies. We hypothesized that a) translating metaphors from scratch would require more cognitive effort than post-editing them, and b) pupil dilation would have a positive correlation with fixation duration. In order to test these hypotheses, an experiment was carried out at the Laboratory for Experimentation in Translation (LETRA) under two different conditions. The control group was asked to translate a 224-word newspaper text whereas the experimental group was asked to post-edit a machine- translated output of the same source text. The multimethod data collection included eye tracking, key logging and retrospective protocols. For the purposes of this paper, eye-tracking data related to fixation duration and pupil size during the translation and post-editing of five metaphors are analyzed. Preliminary results indicate that the cognitive effort required to post-edit conventional metaphors is lower than translating them from scratch. However, creative metaphors are more cognitively demanding to be post-edited in comparison to translating them from scratch. Similarly to Hvelplund's (2014) findings, our initial analysis provides indications that fixation duration and pupil dilation are positively correlated. The corroboration of this trend in subsequent analyses may contribute to the validation of this measure as a standard indicator of cognitive processing.
Bionote:
Arlene Koglin is a PhD candidate (expected date of completion: March 2015) in Translation Studies at Federal University of Minas Gerais (Brazil). Her PhD thesis focus on the cognitive effort required to post-edit metaphors and to translate them from scratch.
PAPER 9
Title: Source text features and their relationship to cognitive demand and cognitive effort in post-editing
Speaker: Isabel Lacruz, Kent State University
Abstract:
Our objective is to contribute to identify source text features that are associated with increased levels of cognitive effort during post-editing of machine translation (MT) output. Recent work (Lacruz et al. 2012; Lacruz & Shreve 2013) has identified pause metrics as indicators of cognitive effort in post-editing. The simplest of these is the pause to word ratio (PWR), the number of pauses per word in a post-edited segment. Processing rate is another intrinsic measure of cognitive effort (Koponen et al. 2012). Cognitive effort by post-editors is the result of cognitive demand placed on them by errors that need to be corrected in MT output. It is commonly measured by quality metrics.
MT quality has routinely been measured by subjective human judgments and by automatic metrics that measure in various ways how close MT output is to a reference translation. One commonly used automatic metric is HTER (Snover et. al. 2006), which is a form of edit to word ratio, using the post-edited product as a reference translation. Both human quality judgments and HTER are extrinsic measures of MT quality, and so of cognitive demand. Recently, Lacruz et al. (2014), building on work of Lacruz and Muñoz (2014), proposed error to word ratio (EWR), the number of errors per word in an MT segment, as an intrinsic measure of MT quality. For post-editing of Spanish to English MT output, they found strong correlations between PWR, EWR, HTER, and human judgments of MT quality.
The current work examines which source text features correspond to different levels of cognitive demand imposed by MT output on post-editors, and of cognitive effort made by post-editors. We triangulate data from multiple modalities, including eye tracking, mouse tracking, and keystroke logging. In particular, we revisit the relationship between negative translatability indicators and cognitive demand and effort (e.g. O'Brien 2006). Using experimentally controlled source segments to reduce the noise that is inevitable in ecological studies, we also identify source text features that are associated with two categories of MT errors: transfer errors (errors that require the post-editor to refer to the source text) and mechanical errors (errors that can be successfully edited without reference to the source text) (Koby & Champe 2013). Transfer errors are more strongly correlated with cognitive demand (EWR and HTER) and cognitive effort (PWR) than are mechanical errors (Lacruz et al. 2014). We also map linguistically classified errors back to source text features, building on findings of Koponen et al. (2012) and Lacruz et al. (2014). Koponen et al., working with modifications of Temnikova's (2010) taxonomy of MT errors, and Lacruz et al., working with modifications of ATA grading rubrics (Koby & Champe 2013), both found that different linguistic categories of MT error are associated with different levels of cognitive demand and effort.
Bionote:
Isabel Lacruz is Assistant Professor of Translation Studies at Kent State University. She teaches doctoral courses on empirical research methods for translation, and translation and cognition, as well as master level translation practice courses. Lacruz' current research interests include investigation of the mental processes involved in translation and post-editing. She has published theoretical and empirical articles on cognitive aspects of translation and post-editing.