Knowledge organization

The spectrum of structure

Knowledge organization systems differ mainly in how much they say about the relationships between terms. The field recognizes a spectrum. Most systems need a thesaurus and believe they need an ontology.

A controlled vocabulary is a list of approved terms, each standing for one concept, with the variants people might use mapped to it. It says nothing about how the terms relate. Its whole value is that “customer,” “client,” and “account holder” resolve to one thing.

A taxonomy adds hierarchy. Each term has a parent, and the relationship down the tree is “is a kind of”: a refund request is a kind of support ticket, which is a kind of customer contact. A taxonomy answers “what kinds of X are there” and “what is X a kind of,” and nothing else.

A thesaurus adds association: this term is related to that one without being above or below it. ISO 25964, the international standard for thesauri, codifies three relationships, broader, narrower, and related, on top of the equivalence a controlled vocabulary already provides. Library subject headings and most enterprise tagging schemes are thesauri whether they call themselves that or not.

A faceted scheme stops forcing every subject into one tree. It describes a subject by combining values from several independent dimensions, so a document can be about a product, a process, a jurisdiction, and a period at once. A document store that can be filtered by source system, document type, and date range is faceted, whether or not the team calls it that.

An ontology adds formal definitions: classes with properties, constraints on what values those properties may take, and rules a machine can use to infer facts that were never stated. Tom Gruber defined one in 1993 as “an explicit specification of a conceptualization,” and the field has not improved on the sentence. The W3C’s OWL (2004) is the common language for writing one.

The spectrum is not a ladder to climb. Each step up adds expressive power and adds maintenance cost in the same proportion. A controlled vocabulary that is kept current beats an ontology that nobody updates.

Where it came from

The discipline has two roots: libraries that needed to shelve books, and documentalists who needed to index everything else.

Melvil Dewey published the Dewey Decimal Classification in 1876, dividing all knowledge into ten classes and each class into ten more. The scheme was enumerative: every subject had to be listed in advance, in one place in the tree. A book on the chemistry of printing was either chemistry or printing. Dewey decided for you.

Paul Otlet and Henri La Fontaine took Dewey’s numbers as the base for the Universal Decimal Classification, whose first full edition appeared between 1904 and 1907, and added a syntax for combining them. A document could now be classed under a subject and a place and a form by joining codes together. It was the first widely used scheme to treat a subject as something you compose.

S. R. Ranganathan made composition the whole method. His Colon Classification (1933) built every class from facets, and in later editions he generalized them into five fundamental categories: Personality, Matter, Energy, Space, and Time. The Classification Research Group, founded in London in 1952, spent the following decades turning facet analysis into a general procedure, and it is the procedure behind most modern taxonomy work.

Thesauri arrived with post-war technical documentation, when engineering reports outgrew the shelf and had to be indexed by subject. ANSI Z39.19 and ISO 2788, both published in 1974, standardized the construction rules. ISO 25964 replaced ISO 2788 in 2011; its second part (2013) covers how a thesaurus maps to other vocabularies.

Ontologies came from a different direction, artificial intelligence research in the 1980s and 1990s, where the problem was giving a program explicit knowledge of a domain. Gruber’s definition dates from that period. When the web needed a way to share such definitions, the W3C standardized OWL in 2004 and SKOS, a simpler model for thesauri and taxonomies, in 2009. The two traditions, library classification and AI knowledge representation, now use the same standards and argue about the same problems.

Principles that have survived

Four rules recur in every generation of the literature. They are the ones to check a structure against.

Build from warrant. E. Wyndham Hulme coined “literary warrant” in 1911: a class should exist because documents exist that need it. The modern counterpart is user warrant: a class should exist because people ask for it. Warrant also expires. A class nobody has assigned anything to in a year has lost its warrant and should go. A taxonomy built from an org chart or a product roadmap, with no warrant from the collection or the queries, is the most common failure in enterprise work.

One concept, one preferred term. Every concept gets a single name used everywhere, and every variant people actually use is recorded and mapped to it. Librarians call the record of this decision an authority file. Without it, the same thing is indexed under three names and found under one.

Hierarchy means “is a kind of.” The parent-child relationship in a taxonomy has to be genus and species. “Invoices” under “Finance” is correct; “Invoices” under “Monthly tasks” is an association, not a hierarchy, and it breaks every inference drawn from the tree. Thesauri exist partly to give that second kind of relationship somewhere legitimate to live.

Classes should not overlap, and together should cover the domain. Where a subject genuinely belongs in two places, that is the signal to split the tree into facets. Filing it twice postpones the split and doubles the maintenance.

Why it matters for AI

A language model has no stable concept of “customer.” What it has is a distribution over how the word has been used. Given a document about a “client,” it will treat the two as the same thing or different things depending on context, and it will not tell you which it chose. The system around the model has to supply the definition, and that definition is knowledge organization.

Retrieval filters need categories, and a filter is only as reliable as the taxonomy behind it. An agent’s tools are a vocabulary of actions, and the entities those actions accept form a class hierarchy whether or not anyone drew it. A knowledge graph cannot be traversed sensibly unless its “is a kind of” edges are actually hierarchical. Embeddings, which many teams use in place of all this, measure similarity. Identity is a different question: two chunks about different customers with the same complaint will sit close together, and nothing in the vectors says they are different people. Libraries built name authority files for exactly this problem; the Library of Congress still maintains one so that two authors with the same name stay two people.

The failures are the classic ones with new names. Entity confusion is a missing authority file. An agent that follows a “related to” link as if it were “part of” has found a hierarchy nobody checked. None of these is solved by a better model, because the model is not where the definition lives.

Common questions

Is a taxonomy an ontology? No. A taxonomy is a tree of “is a kind of” relationships. An ontology adds properties, constraints, and rules a machine can reason with. Most systems described as ontologies in product documentation are taxonomies or thesauri, and that is usually fine.

Should I reuse an existing scheme? Where one exists for your domain, start from it. Schema.org, with roughly 800 types descending from a single root called Thing, encodes warrant you would otherwise have to rediscover, and so do the published industry ontologies and public thesauri. Extend them where your questions are not covered. Do not rebuild them.

How big should it be? As small as still answers the questions the system is asked. Count the classes that have been assigned something in the last year. That is the real size.

Where to go next

The Ontologies topic takes this material into current practice: how to scope a structure and write it in a form a model can read. Knowledge Graphs covers what happens when the structure is populated with facts.

• Ontologies
• Knowledge Graphs