What to Prepare Before Annotating Data for an Image Model

Published: November 5, 2026

With a new batch of photos in hand, many teams want to start labeling right away. But if the task and standards aren't clear, they often discover halfway through that they have to start over. Before you begin, check these seven things.

1. Decide what question the model should answer

The model's task determines what to label and how long it will take:

• Classification: what category does this photo belong to?
• Object detection: where is the target? Usually marked with a box.
• Key points: what are the coordinates of specific positions?
• Segmentation: which pixels belong to the target? Usually the most time-consuming.

Choose the simplest approach that solves the problem. If you only need to know whether something is present, you may not need to label every pixel.

2. Check that the data reflects real use

Models usually learn only the situations they've seen. If users will take photos on many phones and in different light, the training data should cover those differences too. Watch the balance between categories, and make a point of collecting rare but important cases.

3. Handle privacy and permissions first

Confirm how the data was obtained and how it may be used before annotating. If images include faces, name tags or other identifying details, de-identify them first and limit who can access the data. For data involving children or health, confirm consent and legal requirements first.

4. Write annotation guidelines

When standards live only in people's heads, different people label differently. Useful guidelines include at least:

• A definition of each category
• Examples of correct and incorrect labels
• How to handle edge cases
• What to do when something is unclear, such as marking it "cannot determine" rather than forcing an answer

5. Run a small pilot first

Pick a small batch covering a range of situations, have two people label it independently, and compare the results. Where they disagree is usually where the guidelines aren't clear enough. Fixing them before full annotation saves a lot of rework.

6. Plan quality checks

Decide up front who reviews, how much to sample and how errors get fixed. If you use AI pre-labeling, have people confirm the results rather than treating them as answers. Also keep a separate test set, and don't let similar data from the same source appear in both training and test sets, or results will look better than they really are.

7. Decide on delivery format and versions

Confirm the format training needs, such as JSON, COCO or YOLO, and how coordinates are defined. Keep versions of both data and guidelines, so you can trace the cause when model performance changes.

What we learned from annotating pencil-grip images

While developing Little Hands, a single image often needed classification, areas and key points, and some photos were hard even for professionals to judge. So we record uncertainty rather than treating guesses as correct answers. For a model, an honest "not sure" is usually more useful than a complete-looking wrong label.

Pre-annotation checklist

• The model task and annotation type are decided
• The data reflects real use
• Privacy, permissions and access are handled
• Guidelines are written and cover edge cases
• The pilot is done and guidelines are updated
• Review methods and test data are planned
• Delivery format and versioning are confirmed