Abstract

This document describes the module for frequency, attestation and corpus information of the OntoLex Lexicon Model for Ontologies (OntoLex-Lemon) developed by the W3C Community Group Ontology-Lexica. The module is targeted at complementing dictionaries and other linguistic resources containing lexicographic data with a vocabulary to express

• corpus-derived information (frequency and cooccurrence information, collocation analysis),
• pointers from lexical resources to corpora and other collections of text (attestations, examples),
• the linking of corpora and linguistic primary data with lexical information (dictionary linking), and

This document is an official report of the OntoLex community group. It does not represent the view of single individuals but reflects the consensus and agreement reached as part of the regular group discussions. The report should be regarded as the official specification of lemon.

If you wish to make comments regarding this document, please send them to public-ontolex@w3.org (subscribe, archives).

Introduction

OntoLex-Lemon provides a core vocabulary to represent linguistic information associated with ontology and vocabulary elements. The model follows the principle of semantics by reference in the sense that the semantics of a lexical entry is expressed by reference to an individual, class or property defined in an ontology. The OntoLex module for Frequency, Attestations and Corpus-Based Information (OntoLex-FrAC) complements OntoLex-Lemon with the capability of including information drawn from or found in corpora and linguistic primary data.

In particular, the model’s primary motivation is to provide a means to link lexical resources to corpora and other collections of text, and to express the relationship between lexical information and the primary data from which it is derived. As such this module will:

1. Extend the use of OntoLex-Lemon to support digital lexicography,
2. Improve application and applicability of OntoLex-Lemon in natural language processing,
3. Contribute to the integration of lexicography, AI and human language technology communities,
4. Provide a method of representing this information in a way that is compatible with the existing OntoLex-Lemon model.

@prefix frac: <http://www.w3.org/ns/lemon/frac#> .

@prefix ontolex: <http://www.w3.org/ns/lemon/ontolex#> .
  @prefix synsem: <http://www.w3.org/ns/lemon/synsem#> .
  @prefix decomp: <http://www.w3.org/ns/lemon/decomp#> .
  @prefix vartrans: <http://www.w3.org/ns/lemon/vartrans#> .
  @prefix lime: <http://www.w3.org/ns/lemon/lime#> .
  @prefix lexicog: <http://www.w3.org/ns/lemon/lexicog#> .

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.
  @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>.
  @prefix owl: <http://www.w3.org/2002/07/owl#>.
  @prefix xsd: <http://www.w3.org/2001/XMLSchema#>.
  @prefix lexinfo: <http://www.lexinfo.net/ontology/3.0/lexinfo#>.
  @prefix dct: <http://purl.org/dc/terms/>.
  @prefix oa: <http://www.w3.org/ns/oa#>.
  @prefix dcterms: <http://purl.org/dc/terms/> .

Observations and Observables

OntoLex-FrAC provides the necessary vocabulary to express observations obtained from a language resource about any linguistic or conceptual entity that can be observed in a corpus (“observable”). By observable, we mean

• any lexical entity that can be described with OntoLex (including, but not limited to OntoLex core classes ontolex:LexicalEntry, ontolex:Form, ontolex:LexicalSense or ontolex:LexicalConcept), as well as
• any ontological entity from a knowledge graph (corresponding to the object of an ontolex:denotes, ontolex:reference or ontolex:isConceptOf property).

The top-level concepts of OntoLex-FrAC are thus frac:Observable and frac:Observation, complemented by a designating where the observation has been frac:observedIn.

For OntoLex, we assume that frequency, attestation and corpus information can be provided about every linguistic content element in the OntoLex-Lemon core model and in existing or forthcoming OntoLex modules. This includes ontolex:Form (for token frequency, etc.), ontolex:LexicalEntry (frequency of disambiguated lemmas), ontolex:LexicalSense (sense frequency), ontolex:LexicalConcept (e.g., synset frequency), lexicog:Entry (if used for representing homonyms: frequency of non-disambiguated lemmas), etc. (cf. Fig. 1). In particular, we consider all these elements to be countable, annotatable/attestable. For this reason, we introduce frac:Observable as a top-level element within the FrAC module that is used to define the rdfs:domain of any properties that link lexical and corpus-derived information.

Observations as understood here are empirical (quantitative) observations that are made against a corpus, a text, a document or another type of language data. Observations can be made in any kind of (collection or excerpt of) linguistic data at any scale, structured or unstructured, regardless of its physical materialization (as an electronic corpus, as a series of printed books, as a bibliographical database or as metadata record for a particular corpus).

Frequency

Lexicographers use (corpus) frequency and distribution information while compiling lexical entries, as a qualitative assessment of their resources. In this module, we focus on absolute frequencies, as relative frequencies can be derived if absolute frequencies and totals are known. Absolute frequencies are used in computational lexicography, and they are an essential piece of information for NLP and corpus linguistics.

A frequency should have a unit that specifies the segmentation unit of the frequency count. This can be, for example, “tokens”, “types”, “lemmas”, “sentences”, “paragraphs”, etc.

Examples of values of frac:unit include string literals such as "tokens", "sentences", etc. If a future community standard provides reference URIs for such datatypes, frac:unit should be used as a datatype property. Until such a convention has been established, it is recommended to be used as a datatype property.

The definition above only applies to absolute frequencies. For expressing relative frequencies, we expect the associated data source (frac:observedIn) object to define a total of elements contained (frac:total). In many practical applications, it is necessary to provide relative counts, and in this way, these can be easily derived from the absolute (element) frequency provided by the Frequency class and the total defined by the underlying corpus. If the real absolute values are unknown and only relative scores are provided, data providers should use percentage values for both the Frequency rdf:value and for the frac:total (i.e., 100%) of the associated corpus.

A simple example of indicating the frequency of a word in a corpus is given below:

The following example illustrates word and form frequencies for the Sumerian word a (n.) “water” from the Electronic Penn Sumerian Dictionary and the frequencies of the underlying corpus.

The example shows an orthographic variation (in the original writing system, Sumerian Cuneiform sux-Xsux, and its Latin transcription sux-Latn). It is slightly simplified insofar as the ePSD2 provides individual counts for different periods and only three of six orthographical variants are given. Note that these are orthographical variants, not morphological variants (which are not given in the dictionary).

An example of the use of frac:total is given below:

Attestations and Citations

Collocations

A collocation is a sequence of words or terms that co-occur more often than would be expected by chance. Often, collocations are idiomatic expressions, but they can also be more general, such as “strong tea” or “heavy rain”.

Collocation analysis is an important tool for lexicographical research and instrumental for modern NLP techniques. It has been the mainstay of 1990s corpus linguistics and continues to be an area of active research in computational philology and lexicography.

Collocations are usually defined on surface-oriented criteria, i.e., as a relation between forms or lemmas (lexical entries), not between senses, but they can be analyzed on the level of word senses (the sense that gave rise to the idiom or collocation). Indeed, collocations often contain a variable part, which can be represented by a ontolex:LexicalConcept.

Collocations can involve two or more words, they are thus modelled as an rdfs:Container of frac:Observabless. Collocations may have a fixed or a variable word order. Where fixed word order is required, the collocation must be defined as a sequence (rdf:Seq), otherwise, the default interpretation is as an ordered set (rdf:Bag).

Collocations obtained by quantitative methods are characterized by their method of creation (dct:description), their collocation strength (rdf:value), and the corpus or data source used to create them (frac:observedIn). Collocations share these characteristics with other frac:Observations and thus, these are inherited from the frac:Observation class.

Collocations are collections of frac:Observables, and formalized as rdfs:Container, i.e., rdf:Seq or rdf:Bag. The elements of any collocation can be accessed by rdfs:member. In addition, the elements of an ordered collocation (rdfs:subClassOf rdf:Seq) can be accessed by means of numerical indices (rdf:_1, rdf:_2, etc.).

By default, frac:Collocation is insensitive to word order. If a collocation is word order sensitive, it should be defined as rdfs:subClassOf rdf:Seq. Collocation analysis typically involves additional parameters such as the size of the context window considered. Such information can be provided in human-readable form in dct:description.

FrAC collocations can be used to represent collocations both in the lexicographic sense (as complex units of meaning) and in the quantitative sense (as determined by collocation metrics over a particular corpus), but that the quantitative interpretation is the preferred one in the context of FrAC. To mark collocations in the lexicographic sense as such, they can be assigned a corresponding lexinfo:termType, e.g., by means of lexinfo:idiom, lexinfo:phraseologicalUnit or lexinfo:setPhrase. If explicit sense information is being provided, the recommended modelling is by means of ontolex:MultiWordExpression and the OntoLex-Decomp module rather than frac:Collocation. To provide collocation scores about a ontolex:MultiWordExpression, it can be linked via rdfs:member with a frac:Collocation.

Collocations are frac:Observables, i.e., they can be ascribed frac:frequency, frac:attestation, frac:embedding, they can be described in terms of their (embedding) similarity, and they can be nested inside larger collocations.

Collocations can be described in terms of various collocation scores. If scores for multiple metrics are being provided, these should not use the generic rdf:value property, but a designated subproperty of frac:cScore:

LexInfo defines a number of popular collocation metrics as sub-properties of frac:cScore:

• lexinfo:relFreq (relative frequency): $RF(x,y|x)=\frac{f_{xy}}{f_x}(=R_x)$ (asymmetric, requires frac:head)
• lexinfo:pmi (pointwise mutual information, sometimes referred to as MI-score or association ratio, cf. Church and Hanks 1990, via Ewert 2005: $\mathrm{PMI}(x,y)=lo{g}_2\frac{f_{xy}N}{f_x{f}_y}$
• lexinfo:pmi2 (PMI²-score): ${\mathrm{PMI}}^2(x,y)=lo{g}_2\frac{f_{xy}^{2}N}{f_x{f}_y}$
• lexinfo:pmi3 (PMI³-score, cf. Daille 1994 in Ebert 2005, p.89): ${\mathrm{PMI}}^3(x,y)=lo{g}_2\frac{f_{xy}^{3}N}{f_x{f}_y}$
• lexinfo:pmiLogFreq (PMI.log-f, salience, formerly default metric in SketchEngine): $PMI.log-f(x,y)=lo{g}_2\frac{f_{xy}N}{f_x{f}_y}\times log{f}_{xy}$
• lexinfo:dice (Dice coefficient): $Dice(x,y)=\frac{2f_{xy}}{f_x\times f_y}$
• lexinfo:logDice (default metric in SketchEngine, Rychly 2008): $LogDice(x,y)=lo{g}_{2}Dice(x,y)$
• lexinfo:minSensitivity (minimum sensitivity, cf. Pedersen 1998): $MS(x,y)=min(R_x,R_y)$

with

• $x,y$ the (head) word and its collocate
• $f_x$ the number of occurrences of the word x
• $f_y$ the number of occurrences of the word y
• $f_{xy}$ the number of co-occurrences of the words x and y
• $R_y=\frac{f_{xy}}{f_y}$ relative frequency of y
• $N$ the total number of words in the corpus, this should be documented in dct:description

In addition to collocation scores, also statistical independence tests can be employed as collocation scores:

• lexinfo:logLikelihood (log likelihood, G² Dunning 1993, via Ewer 2005)
• lexinfo:tScore (Student’s t test, T-score, cf. Church et al. 1991, via Ewert 2005, p.82 ): $T(x,y)=\frac{f_{xy}-\frac{(f_x{f}_y)}{N}}{\sqrt{f_{xy}}}$
• lexinfo:chi2 (Person’s Chi-square test Manning 1999 ): ${\chi }^2(x,y)=\frac{N(O_{11}{O}_{22}-O_{12}{O}_{21}{)}^2}{(O_{11}\times O_{12})(O_{11}\times O_{21})(O_{12}\times O_{22})(O_{21}\times O_{22})}$

with

• $O_{11}=f_{xy}$
• $O_{12}=f_y-f_{xy}$
• $O_{21}=f_x-f_{xy}$
• $O_{22}=N-f_x-f_y\times 2f_{xy}$
• $N$ - the total number of words in the corpus

In addition to classical collocation metrics, as established in computational lexicography and corpus linguistics, related metrics can also be found in different disciplines and are represented here as subproperties of frac:cScore, as well. This includes metrics for association rule mining. In this context, an association rule (collocation) $x\to y$ means that the existence of word x implies the existence of word y

• lexinfo:support (the support is an indication of how frequently the rule appears in the dataset): $support(x\to y)=\frac{f_{xy}}{N}$ (with N the total number of collocations)
• lexinfo:confidence (the confidence is an indication of how often the rule has been found to be true): $confidence(x\to y)=\frac{f_{xy}}{f_x}$
• lexinfo:lift (the lift or interest of a rule measures how many times more often x and y occur together than expected if they are statistically independent): $lift(x\to y)=\frac{f_{xy}}{f_x{f}_y}$
• lexinfo:conviction (the conviction of a rule is interpreted as the ratio of the expected frequency that x occurs without y, i.e., the frequency that the rule makes an incorrect prediction, if x and y are independent divided by the observed frequency of incorrect predictions): $conviction(x\to y)=\frac{(1-f_y)f_x}{f_x-f_{xy}}$

Many of these metrics are asymmetric and distinguish the lexical element they are about (the head) from its collocate(s). If such metrics are provided, a collocation should explicitly identify its head:

As an example, the relative frequency score is the number of occurrences of a collocation relative to the overall frequency of its head.

The function of the property frac:head is restricted to indicate the directionality of asymmetric collocation scores. It must not be confused with the notion of “head” in certain fields of linguistics, e.g., dependency syntax.

The following example illustrates collocations as provided by the Wortschatz portal (scores and definitions as provided for beans, spill the beans, etc.

Corpus Annotation (non-normative)