Summary
Compare precision/recall of different entity extraction configurations to quantify accuracy/performance tradeoffs.
Details
Compare three configurations on the 18 evaluation cases:
- 49-term curated thesaurus (current
snomed_thesaurus.json)
- 1.4M-pattern full UMLS automaton (current
umls_automata.bin.zst)
- Expanded curated set (500+ terms, from expanded thesaurus work)
Metrics to capture:
- Precision (correct extractions / total extractions)
- Recall (correct extractions / expected extractions)
- F1 score
- Extraction latency (ms per case)
- Memory usage per configuration
Acceptance Criteria
Priority: P3
Summary
Compare precision/recall of different entity extraction configurations to quantify accuracy/performance tradeoffs.
Details
Compare three configurations on the 18 evaluation cases:
snomed_thesaurus.json)umls_automata.bin.zst)Metrics to capture:
Acceptance Criteria
Priority: P3