The JRC-Acquis multilingual parallel corpus and Eurovoc (v. 3.0)
What are the Acquis Communautaire and the JRC-Acquis
The Acquis Communautaire (AC) is the total body of European Union
(EU) law applicable the the EU Member States. This collection of legislative
text changes continuously and currently comprises selected texts written between
the 1950s and now. At the beginning of the year 2007, the EU has 27 Member
States and 23 official languages. The Acquis Communautaire texts exist in
these languages, although Irish translations are not currently available.
The Acquis Communautaire thus is a collection of parallel texts in the following
22 languages: Bulgarian, Czech, Danish, German, Greek, English,
Spanish, Estonian, Finnish, French, Hungarian, Italian, Lithuanian, Latvian,
Maltese, Dutch, Polish, Portuguese, Romanian, Slovak, Slovene and Swedish.
The Language Technology
group of the European Commission's Joint
Research Centre did not receive an authoritative list of documents
that belong to the Acquis Communautaire. In order to compile the document
collection distributed here, we selected all those CELEX documents (see below)
that were available in at least ten of the twenty EU-25 languages (the official
languages of the EU before Bulgaria and Romania joined in 2007) and that additionally
existed in at least three of the nine languages that became official languages
with the Enlargement of the EU in 2004 (i.e. Czech, Estonian, Hungarian, Lithuanian,
Latvian, Maltese, Polish, Slovak and Slovene). The collection distributed
here is thus an approximation of the Acquis Communautaire which we
call the JRC-Acquis. The JRC-Acquis must not be seen as a legal reference
corpus. Instead, the purpose of the JRC-Acquis is to provide a large parallel
corpus of documents for computational linguistics research purposes.
The linguistic research interest of the JRC-Acquis
In (computational) linguistics, parallel corpora are useful resources that
are used for many applications and purposes. Most parallel corpora exist for
a small number of languages. To our knowledge, the JRC-Acquis with its 22
languages and its approximately 23,000 documents per language is the largest
existing parallel corpus, if we take into account both its size and the number
of languages covered.
The AC and other Community legislation is publicly available on the European
Commission's web sites. The Language Technology team of the Joint Research
Centre (JRC, http://langtech.jrc.ec.europa.eu/)
in Ispra, Italy, has attempted to identify the documents that are part of
the AC, has downloaded them and converted them to XML format. The Bulgarian
and Romanian documents were processed by the Romanian Academy of Sciences
(http://www.racai.ro/).
In further processing steps, the texts were cleaned of their footers and annexes,
and they were sentence-aligned. Instead of using a single pivot language,
all possible language pair combinations were aligned individually. This is
useful due to the n-to-n relationship between aligned sentences, which often
differs depending on the language pair involved.
For some of the documents, only preliminary translations were available. For
the online texts in some of the languages, only the title has been translated,
but the text displayed is English. An automatic language recognition tool was
therefore used to filter out those texts that are displayed as being one
language, but which are actually English. No manual check was carried out.
The European Commission's Office for Official Publications OPOCE
manages the
distribution rights of this aligned multilingual parallel
corpus. OPOCE agreed that the corpus can be given to research partners
for non-commercial use. See the section on licensing issues,
below.
The JRC-Acquis corpus (version 3.0) is currently available in 22
languages with the following distribution:
All documents were downloaded from the websites http://europa.eu.int/
and http://ccvista.taiex.be. See the
publication at LREC'2006 for details. The texts in the official EU languages
(as of 2006) on http://europa.eu.int/
were found in html format, while the Bulgarian and Romanian translations on
http://ccvista.taiex.be were found in
MS-Word format.
All documents have a numerical identifier called the CELEX code
(see http://eur-lex.europa.eu/).
This code helps to find the same text in the various languages.
Conversion from HTML and MS-Word to XML
After having downloaded the HTML documents (see Section 3),
the documents were converted to XML. The title and body text were isolated,
the paragraph breaks (<P> HTML tags) were kept. All texts were uniformly
encoded with UTF-8.
Identification of footers / annexes
A list of rules was used to detect the beginning of the documents'
annexes and signatures (repetitive and frequently multilingual text strings
ending the documents) and to separate the text body from the less useful text
parts. As the rules were hand-written by developers who do not speak the 22
languages, some signatures and annexes may have been missed and some may have
been recognised wrongly.
Document format / DTD
The documents have the format as illustrated below. The DTD
for this format is also provided with the distribution.
<TEI.2 id="jrcCELEX-LG" n="CELEX"
lang="LG">
<teiHeader
lang="en" date.created="DATE">
<fileDesc>
<titleStmt>
<title>JRC-ACQUIS CELEX LANGUAGE</title>
<title>Document Title</title>
</titleStmt>
<extent>nb_of_paragraphs paragraph segments</extent>
<publicationStmt>
<distributor>
<xref url="http://wt.jrc.it/lt/acquis/">http://wt.jrc.it/lt/acquis/</xref>
</distributor>
</publicationStmt>
<notesStmt>
....
</notesStmt>
<sourceDesc>
<bibl>Downloaded from <xref url="Downloading_URL">Downloading_URL</xref>
on <date>Downloading_DATE</date></bibl>
</sourceDesc>
</fileDesc>
<profileDesc>
<textClass>
<classCode scheme="eurovoc">Eurovoc_Code</classCode>
.....
</textClass>
</profileDesc>
</teiHeader>
<text>
<body>
<head n="1">Document Title</head>
<div type="body">
<p n="paragraph_number">... TEXT...</p>
.......
</div>
<div type="signature">
<p n="paragraph_number">... signature
text...</p>
....
</div>
<div type="annex">
<p n="paragraph_number">... annex text...</p>
....
</div>
</body>
</text>
</TEI.2>
Note that the title, body text, signature and annex further contain <p>...</p>
tags. Each tag contains as attribute (n) its sequential number in the document,
which is used in the paragraph alignment.
Strictly speaking, the corpus is currently aligned at the paragraph
level, as it was the <P> elements that were being aligned.
However, the paragraphs of the AC Corpus are usually short and do usually
contain one sentence, or even only part of a sentence.
The alignment was done using two different alignment programs:
Vanilla and HunAlign. Vanilla was written by Pernilla Danielsson
and Daniel Ridings. It implements the widespread Church and Gale / Dynamic
Time Warping algorithm. The C source and documentation of the program
are available at http://nl.ijs.si/telri/Vanilla/.
HunAlign was described by Varga, Halácsy, Kornai, Nagy, Németh & Trón.
Having two alignments for the same bilingual corpora allows
comparison and benchmarking of alignment tools and algorithms for multiple
language pairs. To our knowledge, no alignment tools have been tested on so
many different language pairs.
We decided to align the sentences of each language pair separately,
instead of using one pivot language. As the corpus exists in twenty-two languages,
there are 231 possible language pair combinations (462 language pair directions).
For each individual language pair, we thus produced files containing the language
pair-specific alignment information. These files contain, for each document
identifier (CELEX number), pointers to the paragraphs (in the"n"
attribute) that are translations of each other. The format used is that of
the Text Encoding Initiative (TEI).
Due to the size of the corpus and the number of language pairs,
the files do not contain the text itself. If you want to produce the parallel
corpus for a specific language pair, you thus need to generate this corpus
on the basis of the monolingual corpora (which all contain paragraph identifiers
in the <p> tags) and the alignment information, using the tool provided
by the JRC.
See the alignment directory for
further information on how to generate such an aligned corpus.
Most CELEX (EU) documents have been manually classified according
to the subject domains to which they belong. The classification scheme used
is the Eurovoc thesaurus (http://eurovoc.europa.eu/),
which is a multilingual wide-coverage conceptual thesaurus. The European Parliament,
large parts of the European Commission and about twenty national and regional
European parliaments use Eurovoc for the classification of their documents.
The Eurovoc thesaurus consists of over 6,000 descriptor terms (classes) that
are organised hierarchically into up to eight levels, using the relationships
Broader Term - Narrower Term (BT-NT) to describe the hierarchical relationship,
and Related Term (RT) to link descriptors that are related but not linked
hierarchically. Additionally, synonyms and near-synonyms for some of the descriptors
are listed, marked with the Use-For (UF) tag. Eurovoc exists in over twenty
languages and is maintained actively. As the descriptors are defined precisely
with Scope Notes, each descriptor has exactly one translation in each of the
languages. Numerical descriptor IDs link the various language versions. This
feature makes Eurovoc an ideal means for cross-lingual search and retrieval
applications and more.
While the parliaments use professional human indexers to classify
their documents manually, the JRC has been working on automating this task.
For details, see http://langtech.jrc.ec.europa.eu/Eurovoc.html.
Most Acquis Communautaire texts have been classified manually
with Eurovoc descriptors. The file celex-EurovocId.txt contains the
lists of numerical descriptor IDs that have been assigned to each of the AC
documents. As the AC documents have been written over a period of about fifty
years and the Eurovoc thesaurus keeps evolving, the documents are indexed
with different Eurovoc versions. The Eurovoc descriptor codes for documents
older than 1995 are not currently available. Furthermore, a small number of
newer documents also seems not to have been Eurovoc-indexed, so that Eurovoc
descriptor codes are not available for all AC documents.
With this distribution, we provide the numerical Eurovoc
descriptor codes. Should you be interested in the descriptor text (the class
name in any of the EU languages), you will need to get the licence for
Eurovoc from OPOCE.
Acquis Communautaire corpus
According to an agreement with the European Commission's Office
for Official Publications OPOCE, the AC corpus can be used and distributed
for research purposes, but the following usage conditions must be adhered
to:
The European Communities consider legislative and quasi-legislative
documents published in the Official Journal of the European Union and
related COM and SEC series as well as charters and treaties and ECJ case-law
to be in the public domain. Prior written permission is thus not required
for their reproduction/translation, and they may be reproduced/translated
freely without restriction, including for the purpose of further non-commercial
dissemination to final users, subject to the condition that appropriate
acknowledgment is given to the European Communities and to the source,
and provided that the additional guidelines set out below are respected.
(1) Whenever a document is reproduced verbatim from
a source other than the printed version of the Official Journal of the
European Union, a prominently positioned disclaimer should read:
'Only European Community legislation printed in the paper
edition of the Official Journal of the European Union is deemed authentic.'
(2) For the reasons stated in the disclaimer above,
it is advisable to ensure that translations are made from the printed,
authentic version of the Official Journal. This precaution, while minimizing
the risk of error, does not confer any legal status whatsoever to the
translated text. The following notice shall accompany the translated text,
printed below the acknowledgment:
'Originally published in the official languages of the European
Union in the Official Journal of the European Union by the Office for
Official Publications of the European Communities. Responsibility for
the translation into [specify language] from the original [specify language]
edition lies entirely with [name of translation copyright holder].'
Moreover, please note that we do not consider a "further
commercial dissemination" the inclusion, as reference material for consultation
purposes, of small amounts of relevant legislative texts in articles/thesis/studies/reports/books
issued by third-party authors or publishers, whatever the means, and disseminated
subject to payment.
Eurovoc thesaurus
Unlike the JRC-Acquis corpus, the Eurovoc thesaurus (http://eurovoc.europa.eu/)
must not be used or disseminated without prior written permission from the
European Commission's Office for Official Publications OPOCE. If you
want to get the rights to use Eurovoc and to receive a copy of the multilingual
thesaurus, please contact OPOCE at OP-INFO-COPYRIGHT@publications.europa.eu,
mentioning the file reference number 2005-COP-395. To our knowledge, the licence
is free of charge for research purposes and a commercial licence costs 500
Euro. To obtain a commercial licence, please contact OPOCE.
The JRC Workshop on Exploiting multilingual parallel corpora
(26-27 September 2005) was dedicated to exploring methods to exploit the Acquis
Communautaire and similar corpora. You find more information on the workshop
web page http://langtech.jrc.europa.eu/0509_EU-Enlargement-Workshop.html.
A description of version 2.2 of the Acquis Communautaire corpus
was published in the paper below. Please use this publication as a reference
when you mention the JRC-Acquis. You may want to check the web site http://langtech.jrc.europa.eu/ for
more up-to-date publications on the subject.
Steinberger
Ralf, Bruno Pouliquen, Anna Widiger, Camelia Ignat, Tomaž Erjavec,
Dan Tufiş, Dániel Varga (2006). The JRC-Acquis: A multilingual
aligned parallel corpus with 20+ languages. Proceedings of the 5th
International Conference on Language Resources and Evaluation (LREC'2006).
Genoa, Italy, 24-26 May 2006. Available at http://langtech.jrc.europa.eu/.
The following persons have contributed to the gathering, preparation
and publication of the aligned Acquis Communautaire corpus:
The Bulgarian and Romanian versions were compiled by Alexandru Ceausu, Radu
Ion and Dan Stefanescu from the Romanian Academy of Sciences.
To obtain a licence of the Eurovoc thesaurus, please contact the
European Commission's Office for Official Publications OPOCE at opoce-info-copyright@cec.eu.int,
mentioning the file reference number 2005-COP-395 (see above).
For information about the AC corpus and related work, please contact Ralf Steinberger
or another member of the JRC's Language Technology team (see http://langtech.jrc.europa.eu/JRC_staff.html)
at the email address of the format Firstname.Lastname@jrc.ec.europa.eu. The postal address
is shown below. We would be pleased to hear how you use the corpus.
