[FEAT] Importing segmentation and annotation labels

[rushelby123]

Currently your library/UI/API doesn't allow a clear way to import and load pre-existing labels, annotations or segmentation. I have an agent that automatically generates labels, annotations and segmentations and I only have to control they are correct, I would like to use your UI but this missing action on your system prevents me to make it work with my pipelines.

Additionally I would like to automate the imports of newly generated dataset as they are generated, how can I import it by python script?

[IgorSusmelj]

Hey @rushelby123, thanks a lot for the feedback. We’re still working on improving the docs.

It’s already possible to use the Python API directly to add annotations to a single image, for example. There is an example here:
https://docs.lightly.ai/studio/dataset_setup/image_dataset/#imagesample-class

A more complex example can be found here:
https://github.com/lightly-ai/lightly-studio/blob/main/lightly_studio/src/lightly_studio/examples/example_evaluation_classification.py

It distinguishes between annotations and predictions. If you would like to keep using only annotations and add new annotations to the same set, you can simply reuse the annotation source.

Let me know if that helps. We’re updating the docs in the coming days and will incorporate your input!

[rushelby123]

Thank you! This worked for me and I also found the examples to upload from s3 the datasets in coco format. I still have a problem to address to fully integrate you frontend.

I'm running into a related problem while trying to automate sync between an S3-backed dataset store and the Lightly Studio UI via a custom proxy server (using ImageDataset.load_or_create() + add_samples_from_coco() on a periodic background thread).

The core issue is that I can't sync an existing session. Once a session/dataset is already loaded and being served by ls.start_gui() (or ls.start_gui_background()), there appears to be no supported way to push updates (new/changed/removed samples or annotations) into that already-running session and have the UI reflect them. Each add_samples_from_coco() call operates on the dataset object, but it's unclear whether/how those changes propagate to a session that's already open in the browser, and there's no documented method to remove samples that were deleted from the upstream source of truth (S3, in my case) from an existing session.

Could the maintainers clarify the intended pattern for syncing an already-running session, or confirm if this is currently unsupported and planned?