Images
- The Smart Tool with Segment Anything 3, SAM 2.1 or HQ-SAM
- One SAM 3 box segments every similar object in the image
- Pre-labels from the Predict App, with your models or foundation models
- Live Training for boxes and masks, on Enterprise
AI-assisted labeling
with SAM 3 and your own models.
AI-assisted labeling in Supervisely puts neural networks inside the labeling tools. The Smart Tool turns a box and a few clicks into a precise mask with Segment Anything 3, and deployed models pre-label whole projects, so people review labels instead of drawing each one.
Segment anything with the Smart Tool and SAM 3.
The Smart Tool is the interactive segmentation tool of the Supervisely image, video and DICOM labeling tools. It runs Segment Anything 3 or another interactive model and is class-agnostic: it outlines objects it was never trained on.
-
Draw a box
Box the object with about 10% of padding, and the model predicts its mask.
-
Click to correct
Click on parts the mask missed and Shift+click on parts it should leave out. Each point updates the mask.
-
Press Space
The mask is saved as an object. Brush, Eraser and Pen fix the last details by hand.
One box, every similar object
With SAM 3, turn on reference bounding box prompts and box a single example: SAM 3 finds and segments every visually similar object in the image, from products on a shelf to cells under a microscope.
Your choice of model
The model ID button in the toolbar switches between the interactive models running in your team: Segment Anything 3, SAM 2.1, HQ-SAM or a SAM 3 you fine-tuned.
On your own GPU
Run the Serve Segment Anything 3 app on a GPU machine connected as an agent. On a private instance, administrators choose the models the Smart Tool offers.
Free to start
Community teams get a Smart Tool that is already running, with 500 requests per day. Pro gets 5,000 requests per day with the Image Max add-on, and Enterprise has no limit.
Pre-label whole projects with the models you deploy.
Auto annotation in Supervisely runs through the Predict App: it applies a neural network to a project or dataset of images or videos, and annotators review and correct the predictions instead of starting from empty images.
-
Pick the data
Start the app from the Neural Networks menu of an images or videos project or dataset, or from the Ecosystem.
-
Pick the model
Connect to a model that is already running, deploy a pretrained model from the Ecosystem, or deploy one of your own from your experiments.
-
Preview
Keep the classes you need, set the confidence threshold and other inference settings, and preview the predictions on a random image or video.
-
Run
Predictions go to a new project, or into the source project, merged with its labels or replacing them.
Videos with tracking
On videos, a detector labels every frame and the enhanced BoT-SORT tracker links its boxes into tracks with object IDs.
Text prompts
Foundation models such as SAM 3, Florence-2 and Grounding DINO detect objects you describe in a text prompt.
Models in a chain
A detector can pass its boxes to Segment Anything for masks, to a pose model for keypoints or to a classifier for finer classes.
Pre-labeled data goes into labeling jobs like any other data: reviewers accept or reject every item, and the labels export to COCO, YOLO, Pascal VOC or your own format.
AI assistance in every labeling toolbox.
Masks on images, tracks in videos, cuboids in LiDAR point clouds and segmentations of DICOM volumes: each toolbox brings the models that fit its data.
Videos
- Auto Tracking picks a tracker for the shape: MixFormer for boxes, Segment Anything for masks, CoTracker for keypoints
- Auto mode extends a track as you near its end and re-tracks an object when you change its label
- Detect off-screen ends a track when the object leaves the frame
- Tracking by detection starts tracks for new objects as you move forward
3D point clouds
- Circle an object with the 3D Smart Tool, and the 3D AI assistant places a cuboid around it
- Click an object with the Cuboid tool, and the AI fits a cuboid around it
- Ground segmentation in one click, with Patchwork++, GndNet and three more algorithms
- Cuboid tracking across frames, and 3D objects from boxes and masks drawn on camera images
DICOM
- The Smart Tool segments a structure on a slice in a few clicks
- Slices in the axial, sagittal and coronal planes, each in its place in the volume
- Label key slices only: the Volume Interpolation app fills in the slices between them
- Preview and edit the interpolated object in 3D
Train a model while you label with Live Training.
Live Training fine-tunes an object detection or segmentation model in parallel with labeling. After the first few images, it pre-labels each new image, learns from your corrections and gets more accurate as you go.
-
Start it in the labeling tool
Click Auto Labeling in the top bar of the image labeling tool, choose Live Training and launch the app for the project.
-
Label the first images
Label an image as usual and press Finish & Next. After 2 to 5 images (5 to 10 for masks), the model starts to predict.
-
Review and correct
Every new image opens with the model’s labels. Fix them, or discard them and label by hand: the model learns from the result.
- A Model Quality score in the floating panel shows how closely the predictions matched your final labels, updated after every image.
- Checkpoints are saved to Team Files and listed on the Experiments page, so a session can resume later.
- When the project is done, you have both a labeled dataset and a trained model ready to deploy.
Live Training is part of the Enterprise edition, with one app for object detection and one for segmentation.
Use your own models, or fine-tune SAM 3 on your data.
A model you train in Supervisely labels data too: deploy its checkpoint with a serving app or straight from the Predict App, and it pre-labels the next projects for your team.
Train on your labels
Train YOLO v8 - 26, DEIMv2, RT-DETRv2 and other training apps learn from the projects you labeled, and every run is saved on the Experiments page.
Fine-tune Segment Anything 3
Fine-tune SAM 3 on your objects with the Fine-tune Segment Anything 3 app, then serve the checkpoint and select it in the Smart Tool.
Bring your architecture
Wrap any model in the Inference class of the open-source Python SDK, and it works in the Predict App and the labeling tools. Custom model integration is part of Enterprise.
Frequently asked questions.
What is AI-assisted labeling in Supervisely?
It is the set of AI tools built into the Supervisely labeling tools. The Smart Tool segments objects from a box and a few clicks with Segment Anything 3, the Predict App pre-labels whole projects with deployed models, Auto Tracking follows objects through videos, the 3D AI assistant places cuboids in point clouds, and Live Training learns from your labels as you work. People review and correct the results instead of drawing every label.
How do I label images with Segment Anything (SAM 3)?
Run the Serve Segment Anything 3 app on a GPU machine connected as an agent. In the image labeling tool, select the Smart Tool, click the model ID button and choose SAM 3. Draw a box around the object with about 10% of padding, click to add areas or Shift+click to remove them, and press Space to save the mask. See the Smart Tool guide.
Can I auto-annotate a whole dataset with my own model?
Yes. The Predict App applies a pretrained model, a model you trained in Supervisely or a model that is already running to a project or dataset of images or videos. You choose the classes, check a preview and save the predictions to a new project or to the source one. Read more about training and deploying models.
Is the Smart Tool free?
Yes, on every plan. Community teams get a Smart Tool that is already running, with 500 requests per day; Pro gets 5,000 requests per day with the Image Max add-on, and Enterprise has no limit. To run SAM 3 or another model on your own hardware, connect a GPU machine as an agent: the free plan connects one. Compare plans.
Does AI-assisted labeling work for videos, 3D point clouds and DICOM?
Yes. In videos, Auto Tracking follows boxes, masks and keypoints across frames and can start tracks for new objects. In 3D point clouds, the 3D AI assistant places and fits cuboids, segments the ground, tracks cuboids and turns labels drawn on camera images into 3D objects; it comes with Enterprise, and with Pro through its Cloud Max add-on. In DICOM volumes, the Smart Tool segments slices and the Volume Interpolation app fills in the slices between them.
What is Live Training?
Live Training trains an object detection or segmentation model while you label. After a few labeled images, it predicts labels for each new image and learns from every correction you make, and its checkpoints are saved on the Experiments page. Live Training is part of the Enterprise edition. See the Live Training quickstart.