TDWG2026
Home / Talks / SAL A
talk · Tuesday 22 September · 11:27–⁠11:39 · SAL A

HerbAudit: Validating AI-Driven Herbarium Transcriptions

Dilara Ağacık · AI for Biodiversity Data

Recording time 2:57:07–3:11:07Open on Vimeo ↗

Summaries and key moments are machine-produced. Check the recording for the speaker’s exact words.

The short versionHerbAudit shows AI herbarium transcription reaching ~94% accuracy and provides a fair, field-aware way to benchmark it.

Overview

What this was about

Dilara Ağacık presented HerbAudit, a tool that both transcribes herbarium labels with a single multimodal AI call (after LeafMachine2 label detection) into Darwin Core, and evaluates transcriptions against manual annotations or GBIF records using field-specific scoring that avoids wrongly penalising AI: taxonomic synonym checks, order-independent collector matching, date-distance scoring, ISO country normalisation and transliteration, word-based locality matching and Haversine distance for coordinates. On a manually annotated benchmark of 90 specimens from 63 institutions, AI transcription averaged 94% accuracy; a Gemini 'Lite' model (ASR 'Gemini 3.5 Lite') offered the best accuracy-to-cost trade-off, the local Gemma model came close at zero API cost, and HerbAudit outperformed other tools with full extraction coverage. Handwritten labels scored lowest and most variably.

Why it matters. The community lacked a clear benchmark for AI transcription; field-aware evaluation and evidence that local models perform well help institutions adopt AI responsibly.

Key ideas

In the room

  • Manual databasing at NYBG averages 10 specimens per hour (130,000 staff hours for 1.3 million specimens).
  • GBIF records can be incomplete or wrong and are an imperfect ground truth.
  • Single-call transcription (no separate OCR) reduces time and cost and logs metadata such as model, tokens and cost.
  • Benchmark: 90 specimens, 63 institutions, handwritten/typed/mixed categories, annotated in Label Studio.
  • Interactive HTML reports show per-field and per-specimen scores; synonyms and doubtful taxonomy are excluded from scoring and flagged.
  • Error sources: hard-to-read cursive surnames, low resolution, crossed-out redeterminations and multiple labels.
Jump in

Notable moments

In their words

Transcript

Automatically generated captions can contain mistakes, especially in names and technical terms. Times are relative to the room recording.

Read the transcript ↓
Loading transcript…
Keep exploring

Related talks.

See the map ↗
talk · 22 Sept · 11:51–⁠12:03 · SAL A

AI-assisted georeferencing a posteriori of herbarium specimens: a case study on Italian herbaria

Combining LLM parsing and LLM judging with gazetteers more than doubles correct georeferences for historical herbarium localities compared with Google Maps, even with a local model.

Matteo Conti
talk · 22 Sept · 15:00–⁠15:15 · SAL B

Closing Gaps in Specimen Metadata: A Collaborative Annotation and Feedback Platform for Natural History Collections

A national collaborative annotation platform with AI-assisted transcription can close metadata gaps across many Taiwanese collections.

Szu-Hsien Lee
talk · 22 Sept · 11:30–⁠11:45 · SAL B

From Images to Structured Data: A Scalable AI Workflow for Natural History Museums

A simple 'just do it' AI transcription pipeline with schema prompts, an intent step and full provenance can make label digitisation scale.

Anne Koivunen
Programme abstract

HerbAudit: Validating AI-Driven Herbarium Transcriptions

Dilara Ağacık

Millions of herbarium specimens are now digitized and accessible online, yet transforming specimen label images into structured, queryable data remains a major bottleneck in biodiversity informatics. While OCR and LLM technologies are increasingly applied to this challenge, the community lacks a shared framework to measure tool performance and identify data gaps. To fill this gap, we developed HerbAudit, an open source tool combining herbarium label transcription with systematic quality evaluation, to be publicly available by September 2026. We also compiled a gold standard benchmark dataset of 90 typed and handwritten specimens from 63 institutions worldwide, each manually transcribed using Label Studio. HerbAudit consists of two core components. The transcription component builds on VoucherVision logic, processing images via LLMs like Google Gemini and OpenAI to output Darwin Core formatted data. It includes a resolution adjustment submodule and reports estimated cost and carbon footprint metrics per image. The evaluation component offers two distinct pathways. First, OCR tool outputs are compared directly against the gold standard manual annotations. Second, AI tool predictions in Darwin Core format are cross checked against GBIF and POWO, incorporating a built-in synonym check. Fields are scored using domain specific methods, such as geographic coordinates via the Haversine formula and author names via Character Error Rate. Users can apply transcription and AI evaluation to their own datasets, with the latter requiring records on GBIF to pinpoint errors and visualize where tools fall short. All results are compiled into an interactive HTML report, giving developers and researchers a shared foundation to benchmark, compare, and improve digitization pipelines.

Submitted to the conference programme before the talk, so the talk itself may differ.