(a) Data collection and technical processing

A complete set of all texts published in three common official languages (English, French and Spanish) by the four main EU institutions, the UN (and the ICJ, only in English and French) and the WTO in 2005, 2010 and 2015 were compiled in cooperation with the relevant organizations and IT specialists. This involved several technological adaptations and verifications to build LETRINT’s first set of corpora (“LINST”, of 1.71 billion tokens), including: developing software for bulk downloading of documents from several repositories, as well as a script and pipeline for corpus compilation and tagging in the three core languages of the project; and verification of metadata and bulk conversion of text formats.

(b) Corpus design

All translated texts from LINST were classified into the following main legal functional categories, building on the framework for the LETRINT project (presented in Prieto Ramos, 2014a): (1) law- and policy-making (including hard and soft law); (2) implementation monitoring; (3) adjudication; and (4) administrative functions. The last group was added to the first three monolingual subcorpora of the “LETRINT 0” corpus (283.44m tokens as per source text word counts) in order to quantify all texts covered by institutional translation. In turn, this was the basis for the selection of representative text groups through stratified sampling for alignment of a parallel trilingual corpus of key genres (“LETRINT 1”, of 7,914 texts and 7.72m tokens) and subsequent downsizing for annotation of discourse features, translation patterns and quality indicators (“LETRINT 1+”, of 256 texts and 752,061 tokens). Our qualitative classifications and quantitative results were also verified against translation statistics obtained from international organizations. Simultaneously, a comparative study of tools for parallel corpus analysis was conducted, and cooperation was undertaken with an IT developer to adapt a concordancer for the exploitation of the LETRINT 1 and 1+ corpora. The resulting query tool, “LETRINT-Q”, is available under “Corpora and tools” .

(c) Legal contextualization of processes of multilingual text production and interpretation

The legal procedures and communicative conditions that frame the production of multilingual texts were examined in line with the abovementioned text classification. The resulting taxonomies of translated genres and their legal contextualization underpinned the mapping process, including the qualitative analysis of legal hierarchies (of high relevance for translation quality assurance (QA) policies in each organization). As regards multilingual text interpretation, focus has been placed on instances of comparison of language versions by international courts (International Court of Justice, Court of Justice of the EU and WTO’s Appellate Body). Patterns of comparative interpretation were analyzed in the case-law of each body.

(d) Analysis of institutional translation policies and practices

A comprehensive review of translation policies and practices, with special emphasis on QA and institutional legal translation, was conducted through interviews at all the relevant organizations, including the main EU institutions in Brussels and Luxembourg, and the main UN language services in Geneva, New York and Vienna (and the ICJ in The Hague). A total of 45 language professionals (including service directors, language unit heads and quality managers or advisers) took part in 33 structured interviews. These addressed aspects such as workflow conditions and constraints, legal hierarchies, profiles and recruitment, quality control and assessment, outsourcing practices, QA guidelines and resources. A considerable amount of background material was compiled during the interviews. A large-scale survey was also conducted in order to map institutional translators’ profiles and their use and perceptions of resources for translation in general and legal translation in particular (see also WS2 and WS3).

The parallel textual and contextual mapping carried out in WS1 has produced a significant number of results and innovations:

• The first data-driven taxonomies of translated texts of international and supranational law from a comparative perspective (Prieto Ramos, 2017a) and a text categorization matrix composed of the abovementioned primary categories and several subcategories of interrelated genres (Prieto Ramos, 2019).

Prieto Ramos, F. (2017a). Global Law as Translated Text: Mapping Institutional Legal Translation. Tilburg Law Review, 22(1-2), 185-214. doi:10.1163/22112596-02201009.

Prieto Ramos, F. (2019). Implications of text categorisation for corpus-based legal translation research: The case of international institutional settings. In Ł. Biel, J. Engberg, R. Martín Ruano & V. Sosoni (Eds.), Research Methods in Legal Translation and Interpreting: Crossing Methodological Boundaries (pp. 29-47). London: Routledge. doi:10.4324/9781351031226-3.

• The LETRINT corpora are a key contribution of the project, and can be used for further research in the field. More information on the design and features of these corpora can be found under “Corpora and tools”, including an infographic and an overview of the various datasets, as well as the “LETRINT-Q” query tool. The innovative stratified sampling techniques applied to build our corpora (Prieto Ramos, Cerutti and Guzmán, 2019), and ensure representativeness and balance, can be of added value for other studies in corpus linguistics and corpus-based translation studies. The technological adaptations required for corpus analysis (e.g. comparative analysis of fourth-generation concordancers in Cerutti [2017]) can also be enlightening for other projects of a similar nature.

Cerutti Benítez, G. (2017). Evaluating tools for legal translation research needs: The case of fourth-generation concordancers. In A. Liimatainen, A. Nurmi, A. Viljanmaa, M. Kivilehto, M. Wallace & L. Salmi (Eds.), Legal Translation and Court Interpreting: Ethical Values, Quality, Competence Training (pp. 355-389). Berlin: Frank & Timme. https://archive-ouverte.unige.ch/unige:105028.

Prieto Ramos, F., Cerutti Benítez, G. & Guzmán, D. (2019). Building representative multi-genre corpora for legal and institutional translation research: The LETRINT approach to text categorization and stratified sampling. Translation Spaces, 8(1), 93-116. doi:10.1075/ts.00014.pri.

• The mapping process has also contributed to empirically defining the scope and features of institutional legal translation, including important commonalities as regards the interconnection between legal text-types and functions, as well as differences that reflect the nature of each institutional legal order, in particular the prominence of hard law-making at the EU (with a high proportion of drafts and input documents) as opposed to larger translation volumes in monitoring procedures at the UN and the WTO (Prieto Ramos and Guzmán, 2021). The triangulation with survey results corroborates a higher level of legal specialization in the CJEU and the EU law-making institutions, in comparison to the UN and other intergovernmental organizations (Prieto Ramos, 2020a; see WS2).

Prieto Ramos, F. & Guzmán, D. (2021). Examining institutional translation through a legal lens: A comparative analysis of multilingual text production at international organizations. Target, 33(2), 258-281. doi:10.1075/target.21003.pri.

• The results also confirm a clear relationship between the number of official languages and the variations of multilingualism as implemented through translation policies and practices in each legal setting. The data obtained provide evidence of the underlying legal, textual and linguistic hierarchies, together with pragmatic and political factors, that prevail in translation directionality, strategies and QA, and in multilingual text interpretation. These aspects serve to reveal the “depth” of institutional multilingualism beyond the “breadth” of language coverage (Prieto Ramos, 2020b). More specifically, as regards patterns of comparative interpretation of equally authentic language versions by international courts (Prieto Ramos, 2018), our results point to a correlation between frequency and feasibility of explicit comparison in light of the complexity of each language regime.

Prieto Ramos, F. (2018). Comparative Interpretation of Multilingual Law in International Courts: Patterns and Implications for Translation. In F. Prieto Ramos (Ed.), Institutional Translation for International Governance: Enhancing Quality in Multilingual Legal Communication (pp. 181-201). Bloomsbury Advances in Translation. London: Bloomsbury. doi:10.5040/9781474292320.0023.

Prieto Ramos, F. (2020b). Translation at International Organizations: The Legal and Linguistic Hierarchies of Multilingualism. In M. Ji & S. Laviosa (Eds.), The Oxford Handbook of Translation and Social Practices (pp. 455–477). New York: Oxford University Press. doi:10.1093/oxfordhb/9780190067205.013.13.

• The contextual mapping is completed with an output on a previously unexplored aspect of translation process conditions and QA from the perspective of translation service management: the key role of translation service heads in QA implementation (Prieto Ramos, 2017b). The data compiled in the interviews and triangulated with vacancy announcements served to outline an inventory of their main QA-related functions, and raised questions on their impact and desirable skills in dealing with three key elements: competence management, workflow supervision and input into translation process conditions. The common challenges identified by service managers are: resource availability and productivity pressures; external contracting conditions; workflow changes and expectations deriving from technological developments.

Prieto Ramos, F. (2017b). The evolving role of institutional translation service managers in quality assurance: Profiles and challenges. In T. Svoboda, Ł. Biel & K. Łoboda (Eds.), Quality aspects in institutional translation (pp. 59–74). Berlin: Language Science Press. doi:10.5281/zenodo.1048188.

(a) Analysis of terminological features and translation difficulty levels

Terminology is used as a key indicator of the thematic diversity and hybridity of institutional texts, as well as of competence and quality in translation decision-making. Legal terms have been annotated and classified into various groups:

1. terms created in the international legal order and recognized as established terminology within the scope of competence of a particular organization (e.g. “universal periodic review” at the UN) (LEG-INT);
2. terms used to refer to concepts that are perceived as generic and common to multiple national and international legal systems (e.g. “appeal”, “legislation”) (LEG-GEN);
3. terms designating singular concepts that are specific to national legal systems or traditions (e.g. “magistrates’ court”, “Chancery Division”) (LEG-NAT-SIN);
4. names of national, regional or local bodies, instruments or positions that are common to multiple legal systems and traditions (e.g. “Parliament”, “Constitution”) (LEG-NAT-GEN).

Institutional titles and phraseology were also annotated separately, as well as other terminology in the following areas: economic (ECO); financial and budgetary (FIN); political, social and administrative (POL); scientific (SCI) and technical (TEC). All annotations have been associated to levels of translation difficulty by LETRINT's team of annotators with a view to (1) examining the density and hybridity of terminology per institution and textual legal function, and (2) supporting the analysis of decision-making patterns and the selection of “rich points” for quality evaluation, with a focus on legal terminology. Several outputs, including a PhD thesis (Cerutti, 2021), have been produced to present the results on the density and distribution of terminological and phraseological features per institution (as indicators of hybridity) and the levels of translation difficulty they represent:

Prieto Ramos, F. & Cerutti, C. (2022). Terminological hybridity in institutional legal translation. A corpus-driven analysis of key genres of EU and international law. Terminology. International Journal of Theoretical and Applied Issues in Specialized Communication, 29(1), 45-77. doi:10.1075/term.21047.pri.

Prieto Ramos, F. & Cerutti, C. (2021). Terminology as a source of difficulty in translating international legal discourses: an empirical cross-genre study. International Journal of Legal Discourse, 6(2), 155-179. doi:10.1515/ijld-2021-2052.

(b) Analysis of adequacy and consistency of legal terminological decision-making

The correlation between degrees of legal singularity and methodological gaps, established as hypothesis (see e.g. Prieto Ramos, 2014b), has been tested through a mixed-method approach that combines lexicometric (quantitative) and acceptability (qualitative) analysis to measure consistency and adequacy levels of terminological decisions, as well as their adherence to recommendations found in the corresponding institutional terminological resources.

After a first case study on the translation of “due process” (Prieto Ramos and Guzmán, 2018), the systematic analysis of translations’ accuracy and consistency levels has been expanded to other representative terms of the abovementioned categories. The findings suggest significant correlations between legal singularity and translation accuracy levels, and between intertextual consistency and accuracy fluctuations (Prieto Ramos, 2020c; Prieto Ramos and Guzmán, 2023a). A PhD thesis (Guzmán, 2019) on the consistency of English-Spanish and French-Spanish translations of 30 names of judicial bodies from six national jurisdictions in UN texts on human rights treaty implementation, as well as a subsequent study on the corresponding translation techniques and adequacy levels, confirm the same patterns (Guzmán and Prieto Ramos, 2021). Another PhD thesis (Bona-Pellissier, 2023) was devoted to the in-depth analysis of adequacy and intratextual consistency in the case of the English-French translations of five terms (“court of appeal”, “due process”, “high court”, “magistrates’ court” and “prima facie evidence”) in LETRINT 0 texts. The findings of a comparison of human and neural machine translations (NMT) of the same terms into French and Spanish (Guzmán and Prieto Ramos, 2024) point to similar difficulties for three NMT systems, especially as regards intratextual consistency.

Consistency in institutional translation was also the focus of a study on the role of international organizations as terminological agents in dealing with neologisms from English (Prieto Ramos and Morales Moreno, 2019). A diachronic and comparative analysis of patterns of translation of four terms representative of international law areas (“governance”, “tariff peak”, “tariff escalation” and “hedge fund”) in the three settings of the project suggests a direct relationship between degrees of fragmentation of terminological work, perceived linguistic authoritativeness of each institution and levels of terminology harmonization.

Prieto Ramos, F. & Guzmán, D. (2018). Legal Terminology Consistency and Adequacy as Quality Indicators in Institutional Translation: A Mixed-Method Comparative Study. In F. Prieto Ramos (Ed.), Institutional Translation for International Governance: Enhancing Quality in Multilingual Legal Communication (pp. 81-101). Bloomsbury Advances in Translation. London: Bloomsbury. doi:10.5040/9781474292320.0015.

Prieto Ramos, F. & Morales Moreno, A. (2019). Terminological innovation and harmonization in international organizations: Can too many cooks spoil the broth? In I. Simonnæs & M. Kristiansen (Eds.), Legal Translation: Current Issues and Challenges in Research, Methods and Applications (pp. 87-110). Berlin: Frank & Timme. https://archive-ouverte.unige.ch/unige:115687.

Prieto Ramos, F. (2020c). Ensuring Consistency and Accuracy of Legal Terms in Institutional Translation: The Role of Terminological Resources in International Organizations. In F. Prieto Ramos (Ed.), Institutional Translation and Interpreting Assessing Practices and Managing for Quality (pp. 128-149). New York: Routledge. doi:10.4324/9780429264894-10.

Guzmán, D. & Prieto Ramos, F. (2021). Assessing Legal Terminological Variation in Institutional Translation: The Case of National Court Names in the Human Rights Monitoring Procedures of the United Nations. Translation and Translanguaging in Multilingual Contexts, 7(2), 224-247. doi:10.1075/ttmc.00067.guz.

Prieto Ramos, F. & Guzmán, D. (2023a). Measuring the quality of legal terminological decisions in institutional translation. A comparative analysis of adequacy patterns in three settings. In Ł. Biel & H. J. Kockaert (Eds.), Handbook of Terminology. Volume 3: Legal Terminology (pp. 375-396). Amsterdam: John Benjamins Publishing Company. doi:10.1075/hot.3.mea1.

Guzmán, D. & Prieto Ramos, F. (2024). Human and machine translation of legal terminology in international institutional settings: A case study. De Europa. European and Global Studies Journal, special issue, 61-81. https://www.deeuropa.unito.it/content/de-europa-special-issue.

(c) Analysis of the use and suitability of terminological resources for legal terminology management

These aspects are regarded as two sides of the same coin. They are analyzed primarily by examining: (a) the congruity between terminological decisions and the relevant terminological entries in institutional databases; and (b) the suitability of these resources from the perspective of legal lexicography and as perceived by translation practitioners. Interviews on institutional lexicographical approaches have also been conducted with terminologists from each of the surveyed organizations.

The findings of the above lexicometric and survey analyses have revealed a low adherence of translations to the (generally limited) guidance of institutional terminological resources on legal system-specific terms (Prieto Ramos, 2020c), as well as concomitant indicators of lower relevance and reliability of institutional databases and other internal resources (such as previous translations and glossaries) in dealing with legal terminology as opposed to other specialized terminology (Prieto Ramos, 2020a). The results point to gaps identified in a preliminary analysis of legal terminological records in line with legal translators’ needs (Prieto Ramos, 2014b). The strengths and limitations of specific term banks (UNTERM and EU’s IATE) for the translation of legal terms have been examined in greater detail by triangulating (1) the evaluation of records for 30 terms from the perspective of legal translation needs, (2) institutional informants’ inputs, and (3) translators’ perceptions of reliability and changes needed (Prieto Ramos and Guzmán, 2023b).

Prieto Ramos, F. (2020a). The use of resources for legal terminological decision-making: patterns and profile variations among institutional translators. Perspectives, 29(2), 278-310. doi:10.1080/0907676X.2020.1803376.

Prieto Ramos, F. (2020c). Ensuring Consistency and Accuracy of Legal Terms in Institutional Translation: The Role of Terminological Resources in International Organizations. In F. Prieto Ramos (Ed.), Institutional Translation and Interpreting Assessing Practices and Managing for Quality (pp. 128-149). New York: Routledge. doi:10.4324/9780429264894-10.

Prieto Ramos, F. & Guzmán, D. (2023b). Translating Legal Terms at International Organisations: Do Institutional Term Banks Meet Translators’ Needs?. In J. Zhao & D. Li, V. L. C. Lei (Eds.), New Advances in Legal Translation and Interpreting (pp. 243-264). Singapore: Springer. doi:10.1007/978-981-19-9422-7_13.

(d) Analysis of translation-triggered corrigenda issued by international organizations

These formal corrections provide a most relevant source for the examination of translation errors and quality control processes. A first-of-its-kind output on these correction patterns (Prieto Ramos, 2020d) reviews the relevant procedures in the selected institutions and presents the results of the quantitative and qualitative analysis of translation-triggered corrigenda in two target languages, French and Spanish, per setting, year, genre, error type and severity. A distinction is made between content reformulation corrections and minor formal corrections for the comparison of diachronic changes and semantic impact levels of corrected errors between the institutions considered. The findings confirm that minor formal errors may have meaning-distorting effects that are as serious as content reformulation errors; when this is not the case, they rarely trigger single-correction corrigenda. The UN recourse to “reissues for technical reasons” for translation corrections and the growing number of corrigenda to EU legal acts and their implications for translation QA and legal certainty are further contextualized and discussed drawing on both corpus analysis and consultations with institutional informants.

Prieto Ramos, F. (2020d). Facing translation errors at international organisations: What corrigenda reveal about correction processes and their implications for translation quality. Comparative Legilinguistics — International Journal for Legal Communication, 41, 97-133. doi:10.14746/cl.2020.41.5.

(a) Analysis of the competences required and the impact of machine translation

In order to gain a better understanding of recent trends, a mixed-method study (Prieto Ramos and Guzmán, 2022) compared the duties and competences required for translator and reviser positions in a corpus of 224 vacancy notices (for 290 competitions) published by institutions of several sizes and domain specializations between 2005 and 2020. It revealed similar competence expectations through varying recruiting approaches, as well as the gradual integration of technological skills in job descriptors, even if these remained secondary to the core language and translation competences required. The comparative analysis points to three clusters of profile features in line with the organizations’ varied nature: the larger UN and EU translation services (with a broad-based approach to qualifying conditions), the international courts (more focused on legal translation profiles) and medium-sized intergovernmental organizations (with tailored specialization requirements and more emphasis on computer-assisted translation (CAT) tools).

A subsequent examination of patterns of human-machine interaction among institutional translators (Prieto Ramos 2024a) revealed a landscape of widespread “augmented translation”. Among the broad range of technological tools employed, translation memories remain the most popular due to their suitability for verifying the relevance and reliability of translation sources, while machine translation (MT) suggestions are integrated and edited approximately two thirds of the time, or slightly less frequently in the case of legal translation. In this context, the description of sub-competences used for LETRINT studies on translator competence (Prieto Ramos, 2011) was updated to include MT and post-editing, and to cover legal and institutional translation more broadly. It was validated through a survey of translation professionals of 24 international organizations (Prieto Ramos 2024b). Profiles with a stronger specialization in legal translation or more experience in institutional translation showed higher awareness of the relevance of all the sub-competences, especially the core language, strategic and thematic competences, and even more so for translating texts of a legal or administrative nature.

Based on the updated taxonomy, the survey also examined translators’ perceptions of post-editing and the impact of neural machine translation (NMT) in particular on their working methods, the quality of translations and the relevance of the various components of translation competence in international institutional settings (Prieto Ramos 2024c). The quantitative and qualitative findings illustrate the increasingly hybrid mix (and common core) of translational actions (including editing translation memory matches and MT suggestions) in augmented translation environments, the cognitive effort and risks associated with NMT, and the reinforced need for solid translation competence and subject matter knowledge to ensure translation quality. In turn, the results highlight the persistent relevance of translation expertise and specialized translator training.

Prieto Ramos, F. & Guzmán, D. (2022). Institutional translation profiles. A comparative analysis of descriptors and requirements. In T. Svoboda, Ł. Biel & V. Sosoni (Eds.), Institutional Translator Training (pp. 49-72). New York: Routledge. doi:10.4324/9781003225249-5.

Prieto Ramos, F. (2024a). Patterns of Human-Machine Interaction in Legal and Institutional Translation: From Hype to Fact. Polissema, 24, 1-27. https://parc.ipp.pt/index.php/Polissema/article/view/5750.

Prieto Ramos, F. (2024b). Revisiting translator competence in the age of artificial intelligence: the case of legal and institutional translation. The Interpreter and Translator Trainer, 18(2), 148-173. doi:10.1080/1750399X.2024.2344942.

Prieto Ramos, F. (2024c - Forthcoming). Embracing new translation technologies at international organizations: translators’ perceptions of neural machine translation and its impact on institutional translation.

(b) Relationship between legal translation competence and quality assurance

The analysis of translation issues and inadequacies identified during the annotation process of LETRINT 1+, and as part of studies on translation adequacy and corrigenda in WS2, contributed to testing and refining a Framework for Translation Quality Metrics that was used for quality assurance purposes. To this end, LETRINT’s holistic approach to quality (Prieto Ramos, 2015) was the basis to develop a taxonomy of quality markers, error types and severity levels tailored to institutional translation. The framework also considers the practices analyzed in WS1 and a review of professional, academic and institutional approaches and standards.

The suitability of the framework was corroborated in the last part of the project, which focused on measuring the impact of legal translation competence (as per specialized training and professional experience indicators) on legal translation quality assurance (revision performance indicators). A study was conducted to compare the performance of several groups of institutional translators (44 in total) according to their qualifications (translation graduates with or without legal translation specialization, law graduates and other translators) and experience levels (more or less than three years) (Prieto Ramos and Guzmán, 2024). In order to test the common core translation competence required to revise or post-edit texts and ensure translation quality, the revision of a low quality translation of a report on legal implementation issues was proposed without revealing the human or machine origin of the translation. In fact, two thirds of participants believed that the revised translation was raw MT output.

The scores of justified, missing and over-corrections of legal content and other errors obtained per profile group, among other indicators, suggest that the impact of legal translation training and subject-matter specialization on quality assurance is more significant than that of professional experience. The gradual benefits of experience to fill certain initial gaps were more tangible among groups without legal translation training (linguistic corrections among law graduates, legal content corrections among translation graduates without legal specialization, or both in the case of graduates from other fields). Qualified translators with a legal specialization stood out as the most efficient revisers, followed by law graduates, translation graduates without a legal specialization and other translators. The findings point to interdisciplinary legal translator training as a most adequate pathway to professional legal translation. They also challenge the rationale behind ISO 20771:2020 qualification requirements.

Prieto Ramos, F., & Guzmán, D. (2024). The impact of specialised translator training and professional experience on legal translation quality assurance: an empirical study of revision performance. The Interpreter and Translator Trainer, 18(2), 313-337. doi:10.1080/1750399X.2024.2344948.