FIND JOBS IN AI GUIDE
Data annotation guide: practise labels and ambiguous cases
Data annotation means adding labels or other structured information to material such as text, images, audio or video. The project’s instructions define what counts as a correct label. This guide uses a small, fictional text exercise to show how to apply those instructions, handle uncertainty and explain a decision.
By Find Jobs in AI · Updated

What does a data annotator produce?
The deliverable depends on the task. Text work might involve a category for each message or a marked span of words. An image task might ask for an object label and its boundary. Audio work might involve a transcript or speaker labels, while video work might require action labels with start and end times.
These are different skills. Read the actual task instructions and required tools before assuming that experience with one format covers another. The exercise below is about choosing text labels; it does not test transcription, drawing boundaries or specialist knowledge.
Read the label rules first
For this exercise only, label the customer’s requested action using one of three labels. DELIVERY means the message asks for the parcel’s location or arrival time. CANCELLATION means it asks to cancel the order. NEEDS_REVIEW means it requests both actions, requests neither action, or does not give enough information to choose.
Use only the message shown. Classify what the customer asks for, not every topic mentioned. A request that is explicitly negated does not count: “do not cancel” is not a cancellation request. Return one label and a short quotation supporting it. These are practice rules, not a claim about how an employer labels its data.
Try these six fictional messages
A. “Where is my parcel?”
B. “Please cancel my order.”
C. “The parcel is late. Please cancel the order.”
D. “It still has not arrived.”
E. “Where is my parcel, and can you cancel the order?”
F. “Do not cancel my order; tell me when it will arrive.”
Choose a label for each message before reading the worked answers below. For each decision, check whether the quoted words show the requested action or only describe the situation.
Keep uncertainty separate from a guessed answer
A complaint about a late parcel does not automatically state whether the customer wants a delivery update, cancellation or something else. Under these exercise rules, an unstated request goes to NEEDS_REVIEW. Do not invent missing context to make the example fit a label.
A real project might have an “other” label, a way to flag ambiguity, or a rule for handling multiple requests. Follow that system. If its instructions do not cover a case, use the project’s review route rather than silently making up a new label or rule.
Check consistency before submitting
Re-read the label definitions, then compare similar cases. Did you label the requested action in both A and C, or did the word “parcel” lead you to label them the same way? Did you notice the negation in F? A useful check targets the reason for a decision, not just whether every row has a label.
If an instruction changes, identify which earlier decisions it affects. Keep notes about unclear cases within the project’s approved tools, and include only the evidence needed for a reviewer to understand the issue.
Connect the practice to an application
Describe relevant experience honestly: following a written classification policy, checking records, transcribing audio, or recording visible actions may be useful when it matches the vacancy. Completing this exercise is personal practice, not professional annotation experience or an employer certification.
Compare the role’s data format, required language, equipment, location, pay basis and assessment terms. A text exercise cannot establish whether you meet a video, audio or specialist annotation role’s requirements.
Worked answers: classify the requested action
A is DELIVERY: “Where is my parcel?” requests its location.
B is CANCELLATION: “Please cancel my order” requests cancellation.
C is also CANCELLATION: “Please cancel the order” is the requested action; the parcel being late is background.
D is NEEDS_REVIEW: “It still has not arrived” states a problem but does not request either action.
E is NEEDS_REVIEW because it requests both a location update and cancellation.
F is DELIVERY: “tell me when it will arrive” asks for an arrival time, and “Do not cancel” explicitly rules out cancellation.
These answers follow the three label definitions in this exercise; another project may use different definitions.
Use a disagreement to check the rule
If another reviewer gives C the DELIVERY label, compare the evidence with the instruction to label the requested action. The word “parcel” is not enough on its own. If reviewers disagree about a case the instructions do not settle, the useful next step is clarification from the project, not an unsupported claim that either answer is certain.
Questions to ask before you start
- What is the unit being labelled: a whole message, a word span, an image region or a time segment?
- Which labels are allowed, and can more than one be applied?
- What is the approved route for missing context or a case the instructions do not cover?
- Which tools, data formats, languages and equipment does the role require?
- How are assessments, accepted tasks, review time and any revisions paid?