Segment-anything-2

​

Overview

The segment-anything-2 pipeline provides direct access to the Segment Anything 2 pipeline developed by Meta AI Research. In its current version, it supports only image segmentation, enabling it to segment any object in an image. Future versions will also support direct video input, allowing the object to be consistently tracked across all frames of a video in real-time. This advancement will unlock new possibilities for video editing and enhance experiences in mixed reality. The pipeline is powered by the latest diffusion models from HuggingFace’s facebook/sam2-hiera-large.

​

Models

​

Warm Models

The current warm model requested for the segment-anything-2 pipeline is:

• facebook/sam2-hiera-large: The largest model in the Segment Anything 2 model suite, designed for the most accurate image segmentation.

​

On-Demand Models

The following models have been tested and verified for the segment-anything-2 pipeline:

​

Basic Usage Instructions

To generate an image with the segment-anything-2 pipeline, send a POST request to the Gateway’s segment-anything-2 API endpoint:

curl -X POST http://<gateway-ip>/segment-anything-2 \
    -F model_id="facebook/sam2-hiera-large" \
    -F point_coords="[[120,100],[120,50]]" \
    -F point_labels="[1,0]" \
    -F image=@<PATH_TO_IMAGE>/cool-cat.png

In this command:

• <gateway-ip> should be replaced with your AI Gateway’s IP address.
• model_id is the diffusion model for image generation.
• The point_coords field holds the coordinates of the points to be segmented.
• The point_labels field holds the labels for the points to be segmented.
• The image field holds the absolute path to the image file to be transformed.

For additional optional parameters, refer to the AI Subnet API Reference.

After execution, the Orchestrator processes the request and returns the response to the Gateway:

{
  "masks": "[[[2.84, 2.83, ...], [2.92, 2.91, ...], [3.22, 3.56, ...], ...]]",
  "scores": "[0.50, 0.37, ...]",
  "logits": "[[[2.84, 2.66, ...], [3.59, 5.20, ...], [5.07, 5.68, ...], ...]]"
}

​

API Reference