Skip to main content

​
What is a Labeling Queue?

  • A labeling queue is a collection of items that need to be labeled.
  • Labeling queue is an actual queue with FIFO (first in, first out) order.
  • Items in the queue have exactly the same shape as datapoints in a dataset.
  • Labeling operation in this context means writing a data to the target field of a datapoint.
Screenshot of a trace visualization

​
How to Use the Labeling Interface

When you open a labeling queue, you’ll see a split-screen interface designed for efficient labeling:

​
Payload view

The left panel shows you the full JSON payload of the current item you’re labeling. Payload is a JSON object with the same shape as a datapoint in a dataset. It has data and target fields.

​
Target Editor

This is where you do the actual labeling work:
  • Edit the JSON in the target editor to correct, improve, or write new data to the target field.
  • Use proper JSON formatting - the editor will help you with syntax highlighting
As you type in the target editor on the right, watch how the "target" section in the left payload updates in real-time. This helps you see exactly what will be saved to your dataset.

​
Save Preferences

  • Select your target dataset from the dropdown to choose where completed items should go
  • Click “Complete” to save the current item to the dataset and move to the next item in the queue.
  • Check the item counter (“Item 5 of 11”) to see how many items you’ve completed and how many remain
  • Use the navigation buttons to move through your queue:
    • Click “Skip” if you want to pass on the current item without making changes
    • Use “Prev” and “Next” to move between items (helpful for comparing similar cases)
    • Click “Complete” when you’re satisfied with your labeling

​
Push items to the queue

​
From Span View

You can push individual spans directly to a labeling queue for labeling. This is particularly useful when you want to label specific model outputs for evaluation. Span input will be added to the data field of the datapoint, and span output will be added to the target field.
Screenshot of a trace visualization

​
From Dataset View

You can also push existing datapoints from datasets into a labeling queue. You can either push individual datapoint or select a subset of datapoints in the dataset view.
Screenshot of a trace visualization
When pushing items to a queue, they maintain the same JSON structure as datapoints in datasets, ensuring consistency between your labeling workflow and final datasets.