Model-assisted labeling

Let a model do the first pass: built-in detectors, click-to-segment Smart Select, your own Core ML models, and batch runs. Everything runs on your Mac.

Assist

Open Assist (the wand) in the editor toolbar and pick a detector. Each proposal appears as a dashed box or polygon labeled with its class and confidence, e.g. player (92%). Hover one on the canvas for quick accept / reject buttons, or work from the inspector's Proposals section: its confidence slider hides weak proposals live, and Accept All (⌘↩) keeps the visible ones. Accepted proposals become normal annotations, and a new label is created for any class you don't already have.

Built-in detectors

Detect Multiple (RF-DETR) (⌃⌘M): finds 80 common object classes (people, vehicles, animals, everyday objects) with a bundled RF-DETR Nano model.
Detect People (Boxes / Polygons), Animals, Faces, and Text: powered by Apple's Vision framework.
Subjects (Polygons): outlines the main foreground objects of any kind, labeled subject for you to reassign. A real segmentation to start from instead of a blank canvas.

Custom models

Bring your own object detector for your own classes: a Core ML .mlmodel or .mlpackage. One-time setup, offline afterwards:

Get a model. Search Hugging Face for Core ML object detectors, train your own with Create ML on a dataset you export from yaAnnotator, or convert an existing model with coremltools. For RF-DETR, the open-source RF-DETR to Core ML converter produces a model yaAnnotator decodes directly, class names included.
In Settings ▸ Detection, click Choose Model… and pick the file.
If the model is a raw-tensor detector whose class names aren't in its metadata, enter them comma-separated in Settings ▸ Detection, in the model's output order. Otherwise proposals come back as class_1, class_2, …

Run it from Assist ▸ Detect with "your model", or ⇧⌘D.

Detect across many images at once

In the list or grid, select several images (⌘-click, ⇧-click, or ⌘A), then pick any command from Tools ▸ Detect. It runs that detector over every selected image and accepts all proposals automatically. There's no per-image review, so the app confirms first, and by default skips images that already have annotations. Because a batch can add a lot at once, you remove one by re-selecting those images and deleting.

Tuning (Settings ▸ Detection)

Three filters apply to the next run of any detector, built-in or custom: Minimum confidence, Max detections, and Minimum box size. Defaults are permissive; tighten them to cut false positives.

Smart Select

Smart Select turns a single click into an object outline. Choose the sparkle-lasso tool (S) and click an object; it traces a polygon for you to label and refine like any other annotation. The first click on an image takes a moment to prepare; clicks after that are quick.

It ships with SAM 2.1 Tiny, a small, fast Apache-2.0 Core ML model, so it works at install with nothing to download. For more accuracy, point it at a bigger variant: download one of Apple's coreml-sam2.1-small / baseplus / large releases from Hugging Face, then use Load Custom Model… in Settings ▸ Smart Select. A compatible model is a Core ML build of SAM 2 split into its encoder / prompt-encoder / decoder parts, named as in Apple's releases.

Tip: click the whole subject to select all of it, or a distinct part (a jersey, a stick) to select just that part; the Click selects setting biases which you get.

Tuning lives in Settings ▸ Smart Select: Detail (how closely the outline follows), Output (polygon, pixel mask, or bounding box), Edge (tighten or loosen at the boundary), and Click selects.

Pixel masks

With Output set to Pixel mask (creation is Pro; viewing, converting, importing, and exporting masks is free), Smart Select keeps the model's selection exactly: every pixel, including holes and separate pieces a polygon outline can't hold.

Refine with clicks: while the mask is selected and Smart Select is active, click to add the region under the cursor, ⌥-click to remove it. Each click is one undo step; press Return when you're done.
Segment Selected Box (Tools menu): re-runs Smart Select inside a selected box annotation, a quick way to turn a detector's box into a tight outline or mask.
Convert one-way: Convert Mask to Polygon traces the mask's largest region as an editable polygon (lossy; undo is the way back). Convert to Box keeps just the bounding box.
View ▸ Canvas Layers hides whole kinds on the canvas, e.g. hide boxes and polygons to review just the masks.

Annotating frames and bursts

Tools ▸ Paste Annotations from Previous Image (⌥⌘V) copies the previous image's annotations onto the current one, ready to nudge into place. It pairs well with video frames import.