Trouble using segment_anything_2_image in Label Studio

[mxochicale]

Hi HumanSignal team

First off, I really appreciate the open-source version of Label Studio. Amazing work!

Unfortunately, I haven’t been able to get SAM2 for image segmentation working, even after following the steps from your guide here:
🔗 segment_anything_2_image example

My implementation is available here:
👉 https://github.com/mxochicale/labelstudio

I thought the problem might be related to:

• LABEL_STUDIO_URL and LABEL_STUDIO_API_KEY, but fixing those didn’t help
• I also tried adding showInline="true" to the labeling configuration
• The ML backend logs look fine (though I’m not sure if I’m missing something):

ml-backend  | [2025-10-25 00:14:03,387] [DEBUG] [hydra.core.utils::set::279] Setting JobRuntime:name=UNKNOWN_NAME
ml-backend  | [2025-10-25 00:14:03,387] [DEBUG] [hydra.core.utils::set::279] Setting JobRuntime:name=app
ml-backend  | [2025-10-25 00:14:05,293] [INFO] [root::_load_checkpoint::174] Loaded checkpoint sucessfully
ml-backend  | [2025-10-25 00:14:10,488] [DEBUG] [label_studio_ml.api::log_request_info::185] Request headers: Host: localhost:9090
ml-backend  | User-Agent: curl/7.81.0
ml-backend  | Accept: */*
ml-backend  | 
ml-backend  | 
ml-backend  | [2025-10-25 00:14:10,488] [DEBUG] [label_studio_ml.api::log_request_info::186] Request body: b''
ml-backend  | [2025-10-25 00:14:10,489] [DEBUG] [label_studio_ml.api::log_response_info::191] Response status: 200 OK
ml-backend  | [2025-10-25 00:14:10,489] [DEBUG] [label_studio_ml.api::log_response_info::192] Response headers: Content-Type: application/json
ml-backend  | Content-Length: 41
ml-backend  | 
ml-backend  | 
ml-backend  | [2025-10-25 00:14:10,489] [DEBUG] [label_studio_ml.api::log_response_info::193] Response body: b'{"model_class":"NewModel","status":"UP"}\n'

Here’s what I see in the UI:

Do you have any recommendations or know what I might be missing?
Thanks a lot for your help!

[heidi-humansignal]

Hello,

Here’s a quick, reliable checklist to get masks appearing in real time.

1. Enable interactive mode in two places

• Project settings → Machine Learning → 3‑dot menu on your SAM2 model → turn ON “Interactive preannotations”
• In the labeling UI bottom bar → toggle ON “Auto‑annotate” (magic‑wand icon)

Thank you,
Abu

Comment by Abubakar Saad
Workflow Run

[vvwcap]

你好
我跟你是一样的问题，没有模型预测，这个例子的docker是否没有做好，以下是我的日志输出，仅仅显示了模型版本

[2025-11-10 02:06:11,011] [DEBUG] [label_studio_ml.api::log_response_info::191] Response status: 200 OK
[2025-11-10 02:06:11,011] [DEBUG] [label_studio_ml.api::log_response_info::192] Response headers: Content-Type: application/json
Content-Length: 26

[2025-11-10 02:06:11,011] [DEBUG] [label_studio_ml.api::log_response_info::193] Response body: b'{"model_version":"0.0.1"}\n'

[heidi-humansignal]

Hi there,

My apologies for the delayed response. If you are still having trouble generating model predictions, please share the following:

• What steps did you take to connect the model? Can you share a screenshot of your model connection settings in Label Studio?
• Are you setting environment variables for LABEL_STUDIO_URL and LABEL_STUDIO_API_KEY as described here?
• What steps did you take in the UI to generate the annotation? Did you use a smart tool to start labeling?
• Have you confirmed that your project’s label configuration matches the output produced by your ML backend?
• Please share your labeling configuration and your predictions structure as well.

Regards,

Cassie C.
Support Engineer
HumanSignal

Comment by Cassie Champange
Workflow Run

[heidi-humansignal]

Oops, looks like the links I shared did not come through, here they are again:

• LABEL_STUDIO_URL and LABEL_STUDIO_API_KEY information: https://labelstud.io/guide/ml#Allow-the-ML-backend-to-access-Label-Studio-data
• Smart tool documentation: https://docs.humansignal.com/guide/ml#Smart-tools

Cassie C.
Support Engineer
HumanSignal

Comment by Cassie Champange
Workflow Run

[yemsb]

I had the same issue as @mxochicale: I can connect the SAM2 model from the Docker container and get some logging info that looks correct, but the smart tool isn't doing anything.

The last question of @heidi-humansignal gave me the important hint that you need to adjust your labelling configuration (under Settings->Labeling Interface) and paste the code given in https://labelstud.io/guide/ml_tutorials/segment_anything_2_image#Labeling-configuration; adjust the label names, and then you can start using the tools as shown in the video.