TDWG2026
Home / Talks / SAL A
talk · Friday 25 September · 08:48–⁠09:00 · SAL A

SCRIBE: Structured Collection Record Interpretation and Bio-entity Extraction

Arianna Salili-James · Large Language Models for Biodiversity Data Discovery, Integration, and Curation. Part A: AI- enabled knowledge creation and extraction

Recording time 1:12:21–1:24:29Open on Vimeo ↗

Summaries and key moments are machine-produced. Check the recording for the speaker’s exact words.

The short versionSCRIBE aims to replace many collection-specific extraction workflows with one LLM and computer-vision platform that turns any uploaded record images into structured data.

Overview

What this was about

Arianna Salili-James reviewed the NHM's earlier data-extraction projects: ALICE label imaging, video label tracking, egg-card structuring with Google Vision (over 90% accuracy), and LLM extraction from registers, bird-skin labels, herbarium sheets and double-sided cards. Each needed its own workflow and prompts. SCRIBE, funded by DCMS and a private donor, is meant to replace these with one platform where users upload images and get structured output. It clusters mixed datasets, samples each cluster to build customised prompts, runs LLM extraction and cleans the output towards Darwin Core. The project started recently (data scientist hired in August) and runs to March 2028.

Why it matters. A generic, hosted extraction service could let DiSSCo UK institutions without in-house AI capacity turn labels, cards and registers into structured data, and the output would feed the Planetary Knowledge Base.

Key ideas

In the room

  • Previous NHM extraction projects each had separate workflows, training and prompts, which was inefficient.
  • SCRIBE targets information that can be output in a structured form (e.g. locality, habitat), not free-text transcription of books.
  • Users say whether an upload is homogeneous or mixed. Mixed datasets are clustered (tests with DBSCAN and PCA) before prompts are built.
  • Samples from each cluster go through a quick LLM entity check to customise a base prompt, then extraction runs on the full dataset, followed by cleaning and Darwin Core formatting.
  • Pre-processing includes auto-rotation and auto-cropping.
  • DiSSCo UK partners will get free token allowances; others can add their own API key. Shared outputs may feed the PKB.
  • Tests found handwritten and printed/typed text did equally well with LLM extraction.
Jump in

Notable moments

In their words

Transcript

Automatically generated captions can contain mistakes, especially in names and technical terms. Times are relative to the room recording.

Read the transcript ↓
Loading transcript…
Keep exploring

Related talks.

See the map ↗
talk · 22 Sept · 16:10–⁠16:15 · SAL B

Rediscovering Archives - Using LLM's for the Transcription and Extraction of Ecological Data from Historical Archives

Off-the-shelf multimodal LLMs can transcribe varied handwritten archives well, with closed models more robust on structured tables.

Phoebe Santos
talk · 22 Sept · 11:30–⁠11:45 · SAL B

From Images to Structured Data: A Scalable AI Workflow for Natural History Museums

A simple 'just do it' AI transcription pipeline with schema prompts, an intent step and full provenance can make label digitisation scale.

Anne Koivunen
talk · 22 Sept · 15:00–⁠15:15 · SAL B

Closing Gaps in Specimen Metadata: A Collaborative Annotation and Feedback Platform for Natural History Collections

A national collaborative annotation platform with AI-assisted transcription can close metadata gaps across many Taiwanese collections.

Szu-Hsien Lee
Programme abstract

SCRIBE: Structured Collection Record Interpretation and Bio-entity Extraction

Arianna Salili-James

There are over 120 million objects housed in natural history collections in the UK. With DiSSCo UK, 90+ institutions will be coming together, in order to make over 50% of UK natural history collections digitally available and accessible. As well as specimen imaging, a crucial goal is the building of datasets extracted from the millions of textual records. These records can include labels and index cards associated with specimens, collection registers and more. Our aim is to provide a service to aid the automatic extraction of structured data from collection records. SCRIBE (Structured Collection Record Interpretation and Bio-entity Extraction) is a new platform, being developed by a team at the NHM UK for automatic data extraction. The proposed software has the potential to revolutionise data extraction, for curation, as well as ecological analysis, by utilising Artificial Intelligence (AI), and providing an easy-to-use interface, to help unlock a plethora of data. While traditionally this task would involve manual transcription, or traditional OCR methods, taking up days and weeks, the proposed tool will for some collections, reduce this to minutes. The tool follows a computer vision and LLM based workflow that utilises customised prompts depending on visual differences within images uploaded by users, enabling users to upload datasets from varied collections concurrently. SCRIBE is a DCMS (UK Department for Culture, Media, Sport) funded and externally-funded program that will sit within the DiSSCo UK workflows. The software will be shared with institutions and evaluations will be jointly collected to further improve the online tool. Finally, the SCRIBE online tool will be supported in-house, helping to scale AI-assisted digitisation across museum and heritage sectors.

Submitted to the conference programme before the talk, so the talk itself may differ.