AI-Enabled Clinical NLP for Federal Regulatory Review
Replacing manual SNOMED CT classification at a large federal regulatory agency with a responsible, Human-in-the-Loop AI workflow: 12x faster, fully auditable, and built without custom model training. The platform continues to support active regulatory review workflows at the agency, with no model retraining required since initial deployment.
12×
Faster Search
HITL
Human-in-the-Loop
S508
Section 508 Compliant
Zero
Custom Model Training Required
The Challenge
A Manual Process at the Heart of Regulatory Review
At a large federal regulatory agency, clinical reviewers were responsible for mapping unstructured medical text from pharmaceutical submissions to standardized SNOMED CT terminology codes. The workflow was entirely manual, dependent on specialized clinical knowledge, slow to execute, and difficult to standardize.
The result: a bottleneck at the center of mission-critical regulatory workflows, with no scalable path forward and growing pressure to modernize without compromising accuracy or oversight.
Subject matter experts manually interpreted thousands of unstructured clinical indications from regulatory submission forms, a process entirely dependent on scarce specialist knowledge
Each SNOMED CT terminology search took up to 2 minutes per query, creating bottlenecks in mission-critical regulatory review workflows
Manual classification introduced variability in outcomes: different reviewers could reach different mappings for the same clinical text
No standardized scoring or audit trail meant outputs were difficult to validate, trace, or defend in a regulatory context
Scaling the process required adding more expert reviewers: an expensive, slow, and unsustainable path
Our Approach
Evaluate First. Build with Confidence.
We didn't start by building. We started by understanding, evaluating multiple AI approaches before committing to a path aligned with the agency's constraints and mission.
Discovery & Technology Evaluation
We evaluated three categories of AI before choosing: (1) Transformer-based LLMs including BERT, rejected due to insufficient agency-available training data and domain complexity; (2) Classical ML models, rejected for similar reasons and contextual limitations; (3) Domain-specialized clinical NLP services, selected. Amazon Comprehend Medical was purpose-built for clinical entity extraction and avoided the need for custom model training, annotation, or ongoing maintenance.
Proof of Concept & Compliance Validation
We ran a controlled PoC against restricted datasets, documenting traceability and having outputs verified by agency reviewers. This phase confirmed accuracy, interpretability, and alignment with federal acquisition standards before any broader rollout.
Incremental Expansion
Only after the PoC demonstrated consistent accuracy and compliance did we expand to full implementation, integrating with the agency's existing workflow platform and enabling the complete SNOMED CT adjudication workflow end-to-end.
“This phased, acquisition-aligned approach ensured responsible, transparent AI adoption, delivering meaningful operational improvements while maintaining the governance and accountability required in a federal regulatory environment.”
The Solution
An Intelligent, Human-Centered Adjudication Workflow
We built an AI-enabled SNOMED CT adjudication application that accelerates the reviewer's work without replacing their judgment.
Clinical Entity Extraction
Amazon Comprehend Medical analyzes unstructured clinical text and extracts medical entities (conditions, procedures, anatomy, medications) with associated confidence scores.
SNOMED CT Terminology Integration
Real-time integration with SNOMED CT Terminology APIs validates and enriches extracted concepts dynamically, mapping entities to standardized medical codes.
Weighted Confidence Ranking
A composite scoring algorithm combines entity confidence and concept confidence to surface the most relevant SNOMED CT candidates: ranked, explainable, and auditable.
Human-in-the-Loop (HITL) Controls
AI functions strictly as an advisory capability. All final SNOMED CT selections remain with trained human reviewers, preserving accountability and regulatory defensibility.
Workflow Platform Integration
Seamlessly integrated into the agency's existing Appian/OneNexus platform, including automated data pre-population, status tracking, assignment management, and comment audit trails.
Explainable, Auditable Outputs
Every AI recommendation includes its confidence score, source entities, and ranking rationale, making outputs transparent, traceable, and compliant with federal responsible AI standards.
The Impact
Faster. More Consistent. Fully Auditable.
12×
Faster terminology search
From 2 minutes to under 10 seconds per query
↑
Mapping consistency
Standardized AI recommendations reducing inter-reviewer variability
0
Custom model training
No annotation, no retraining, no ongoing ML maintenance required
100%
Human oversight retained
AI advises; reviewers decide. Every time.
What 12× means in practice
Before this system, FDA subject matter experts manually searched SNOMED CT classification hierarchies for every clinical indication in a regulatory submission, each lookup taking up to 2 minutes of specialist time. A single review session of 30–50 queries consumed 60–100 minutes of expert attention. With AI-ranked shortlists surfaced in under 10 seconds, that same session now takes under 10 minutes, returning roughly 90 minutes of SME time per session to higher-judgment work.
60–100 min
manual · per session
< 10 min
AI-assisted · same session
AWS Partnership Impact
During delivery, our team identified gaps and improvement opportunities in Amazon Comprehend Medical and partnered directly with AWS to report them. Our engagement contributed to improving the service for the broader healthcare AI community.
Responsible AI in Practice
Every AI recommendation in this system includes its confidence score, source entities, and ranking rationale. The AI never decides; it advises. This design reflects our commitment to building AI that augments human expertise rather than replacing it.
Technologies Used
The Stack Behind the Solution
AI / NLP
- Amazon Comprehend Medical
- SNOMED CT Terminology APIs
- Custom NLP Orchestration (Python)
- Weighted Ranking & Confidence Scoring
Platform
- Appian / OneNexus
- Low-code Workflow Integration
- Assignment & Status Management
Compliance
- Section 508 / WCAG 2.0 AA
- Federal Acquisition Standards
- Responsible AI / HITL Architecture
- Explainable AI Output Design
Have a Similar Challenge?
Whether it's a manual process that needs AI, a legacy workflow that needs modernization, or a compliance requirement that needs architecture. We'd love to hear about it.