Lensa ML
Lensa ML

Named Entity Recognition

Step 1 of 6

Entity Types

PER, ORG, LOC, and more

Cursor position0
SarahPERworksOatOGoogleORGinOLondonLOC"Sarah" → Person (PER)Named person — could be a first name, last name, or full nameEntity types:PER = PersonORG = OrganizationLOC = Location
TOKEN
"Sarah"
ENTITY TYPE
PER
ENTITIES FOUND
3

"Sarah" is a Person entity (PER). NER models learn to recognize these from context — "Sarah" appears in a position and context typical of person names.

Step 1 of 6: Entity Types

PER, ORG, LOC, and more

Cursor position0
SarahPERworksOatOGoogleORGinOLondonLOC"Sarah" → Person (PER)Named person — could be a first name, last name, or full nameEntity types:PER = PersonORG = OrganizationLOC = Location
TOKEN
"Sarah"
ENTITY TYPE
PER
ENTITIES FOUND
3

"Sarah" is a Person entity (PER). NER models learn to recognize these from context — "Sarah" appears in a position and context typical of person names.

Step 2 of 6: BIO Tagging

Begin, Inside, Outside

Reveal tags (token position)0
B = Begin entity | I = Inside entity | O = OutsideTimB-PERCook?leads?Apple?from?Cupertino?Why BIO, not just entity type?"Tim Cook" → B-PER I-PER (one 2-token person entity)Without B/I: "Tim" PER "Cook" PER — is that 1 person or 2?
REVEALED
1/6
B-TAGS
1
I-TAGS
0
O-TAGS
0

"Tim" gets tag B-PER. B (Begin) marks the start of a new entity.

Step 3 of 6: Context Windows

How much context helps?

Context window size1
TimCookleadsApplefromCupertinowindow ±1Prediction confidence for "Apple" → ORG:50%Context clues:✓ "leads" → subject is likely an ORG (company role)
FOCUS
"Apple"
WINDOW
±1
CONTEXT
3 tokens
CONFIDENCE
50%

With ±1 context: "leads Apple from". The verb "leads" and nearby tokens help disambiguate — Apple is likely an organization because someone "leads" it. Confidence: 50%.

Step 4 of 6: Sequence Labeling

RNN hidden states → entity tags

Sequence position0
Timh_0B-PERCookh_1leadsh_2Appleh_3fromh_4Cupertinoh_5Hidden state h_0 activation:0.830.410.630.590.530.54RNN reads left-to-right, accumulating context in the hidden state vector.Each position outputs a tag prediction based on all prior tokens.
POSITION
Tim
HIDDEN DIM
6
PREDICTED
B-PER

Position 0: "Tim" — the RNN has no prior context. It uses only the word embedding of "Tim" to predict B-PER. Common first names are strong signals even without context.

Step 5 of 6: Confidence Thresholds

When to label, when to skip

Confidence threshold0.50
Tim72%Cook45%leads51%Apple79%from90%Cupertino60%Threshold: 50% — entities below this are rejectedPrecision vs Recall trade-off:Precision:100%Recall:75%F1 Score:86%
THRESHOLD
50%
PRECISION
100%
RECALL
75%
F1
86%

Threshold 50%: balanced precision (100%) and recall (75%). F1 = 86%. This is the precision-recall trade-off — tighter thresholds mean fewer but more confident predictions.

Step 6 of 6: Entity-Level Evaluation

Exact span matching vs. partial credit

Match strictness0.00
Gold (ground truth):TimCookleadsApplefromCupertinoPredicted:TimCookleadsApplefromCupertinoMatch results (Exact span):Tim Cook [PER]✗ MissApple [ORG]✓ MatchCupertino [LOC]✓ MatchPrecision: 67%Recall: 67%F1: 67%Mode: Exact span — spans must match exactly
STRICTNESS
Exact span
PRECISION
67%
RECALL
67%
F1
67%

Exact span matching: "Tim" ≠ "Tim Cook" — the predicted span is too short, so it's a miss. Only "Apple" and "Cupertino" match exactly. This is the strictest evaluation used in NER benchmarks like CoNLL.