babamul-skyportal-plugin

A SkyPortal external micro-service that ingests alerts from the babamul broker (Kafka/Avro on kaboom.caltech.edu) into a SkyPortal instance.

How it's wired

SkyPortal's external-service framework clones this repo and runs main.py as a supervised service when the block below is present in the SkyPortal config under services.external.babamul (see config.yaml.defaults). main.py connects to the babamul Kafka topics, decodes each Avro alert, and writes to the DB via SkyPortal's models + add_external_photometry.

What it ingests

babamul publishes raw ZTF alerts (no filter results, no cutouts), optionally cross-matched to LSST. Per alert the plugin:

• creates/uses the Obj (objectId, ra/dec from candidate);
• ingests ZTF photometry from candidate + prv_candidates + fp_hists + prv_nondetections (detections, forced photometry tagged alert_fp, and upper limits; psfFlux/psfFluxErr in nJy → SkyPortal flux with per-survey zeropoint; S/N < 3 nulled; exact-duplicate points within an alert are deduped to satisfy SkyPortal's upsert key);
• registers it as a Candidate under each configured filter_ids (this is how raw alerts become scannable — leave filter_ids empty to ingest Objs+photometry only);
• stores the alert properties blob as an Annotation (origin: babamul);
• for survey_matches.lsst, ingests the LSST match Obj + its photometry and links the two objects via a SuperObj.

Cutouts/thumbnails are not produced (babamul alerts carry no cutouts).

Configure

In the SkyPortal config, set the babamul Kafka credentials and the ingest targets:

services:
    external:
        babamul:
            params:
                kafka:
                    host: kaboom.caltech.edu
                    port: 9093
                    group_id: <your group id>
                    username: <babamul username>
                    password: <babamul password>
                    sasl_mechanism: SCRAM-SHA-512
                    topics: [babamul.ztf.no-lsst-match.hosted, babamul.ztf.lsst-match.hosted]
                ingest:
                    filter_ids: [<skyportal filter ids>]   # alerts become Candidates under these
                    group_ids: [1]
                    ztf_stream_ids: []                      # optional, per-survey photometry streams
                    lsst_stream_ids: []
                    ztf_instrument_name: ZTF
                    lsst_instrument_name: LSST

The ZTF (and LSST, if matching) instruments must exist in the instance, and the listed filter_ids must reference filters in the groups you want to scan from.