From Sensor to Species: AI-Assisted Segmentation, Deep Learning Workflows, and Persistent Biodiversity Records from Earth Observation Systems
Kit Lewers · From Mobilizing Data to AI-Ready Knowledge: Infrastructure for Multimodal Biodiversity Data
Summaries and key moments are machine-produced. Check the recording for the speaker’s exact words.
The short versionPersistent organism identifiers for remotely sensed individual trees could turn repeat Earth-observation imagery into longitudinal biodiversity records, but only if uncertainty and model provenance travel with every record.
What this was about
Kit Lewers argued that remote sensing has a longitudinal biodiversity problem: repeat airborne and satellite acquisitions are processed as separate images, so the same tree can appear in many datasets without a connected history. GBIF would not link such records a year apart. In a proof of concept at NEON's SJER oak savanna, they used off-the-shelf DeepForest crown delineation on RGB imagery, IoU matching across 2023 and 2024, and species candidates constrained by NEON vegetation records from GBIF. The pipeline output Darwin Core occurrences with organismID, identifiedBy 'AI' and basisOfRecord MachineObservation. The weak results (rocks and shadows detected as crowns, a generic cloud model giving one species) were a 'useful failure' showing that model provenance and uncertainty at every step must travel with records. Next steps use NASA G-LiHT's 15-year multi-sensor archive.
Why it matters. Remote sensing could add millions of tree-level records to biodiversity infrastructures, and linking detections of the same individual over time is needed to capture growth, disturbance and loss.
In the room
- Most remote-sensing workflows treat each acquisition separately, so one organism can appear in 10 datasets without an accumulated history.
- GBIF fuzzy matching would probably not link observations of the same organism made a year apart (per conversation with Tim Robertson).
- Test site NEON SJER (San Joaquin Experimental Range), an oak savanna deliberately outside DeepForest's typical use case.
- Pipeline: high-resolution RGB → DeepForest crown delineation → candidate species constrained with NEON field records pulled via pygbif → provenance-rich occurrence export.
- Detected crowns went from 222 in 2023 to 232 in 2024. IoU matching found overlaps but also persistent false detections of rocks and shadows.
- A generic off-the-shelf cloud model identified all trees as one type; biodiversity constraints helped but also hurt a general model.
- Output Darwin Core occurrences used occurrenceID, organismID, scientificName, identificationRemarks, identifiedBy = AI and basisOfRecord = MachineObservation.
Notable moments
Transcript
Automatically generated captions can contain mistakes, especially in names and technical terms. Times are relative to the room recording.
Read the transcript ↓
Related talks.
Closing Gaps in Specimen Metadata: A Collaborative Annotation and Feedback Platform for Natural History Collections
A national collaborative annotation platform with AI-assisted transcription can close metadata gaps across many Taiwanese collections.
Open Digital Curation and Round-Tripping Biodiversity Data Enhancements Across Collection Infrastructures: Six Years and 50 Million Links in Bionomia
Round-tripping is much harder than expected, and data managers must fully populate identifiedBy so taxonomists' expertise is credited.
Are plant image datasets ready for AI-enabled community assessment? A systematic review of bias, representativeness and annotation quality
Current plant image datasets are not yet ready for AI-enabled ecological monitoring, especially for spatially explicit tasks and annotation quality.