Custos is a security middleware for science gateways and HPC research computing, developed under the Apache Airavata umbrella. It provides identity and access management, credential storage, federated authentication, and resource allocation services through a language-independent API.
The project is currently being rebuilt around an HPC allocation management focus.
Custos is composed of pluggable pieces a deployment site mixes and matches.
airavata-custos/
├── core/ # Shared contracts and domain models
├── connectors/ # Adapters to external allocation systems (ACCESS-CI, SLURM, ...)
├── extensions/ # Node-side components a site may opt into (PAM, SSH cert signer)
└── dev-ops/ # Local compose stack, Terraform, Ansible
| Area | Purpose | Examples |
|---|---|---|
core/ |
Go interfaces and shared domain types that connectors and extensions depend on | accountprovisioning.Provisioner |
connectors/ |
Protocol adapters that bring external state into Custos | ACCESS/AMIE-Processor, SLURM/Association-Mapper |
extensions/ |
Independent services that run alongside Custos to extend HPC node behavior | CILogon-SSH-PAM, SSH-Certificate-Signer |
dev-ops/ |
Local dev stack and deployment automation | compose/, terraform/, account-provisioning/ |
Every audit row in the system lives in the core audit_events table. Core, every connector, and any future extension write to it via the same shape: event_type, entity_type, entity_id, details, source, and the OpenTelemetry trace columns (trace_id, span_id, parent_span_id). The trace-view API at /audit/traces* reads from this one table.
When a connector needs to attach connector-specific references to an audit row (for example AMIE keeps packet_id and event_id so it can fetch from a packet down to its audits), those references go in a separate <connector>_audit_extras table owned by the connector. The extras table has audit_event_id as its primary key with a ON DELETE CASCADE foreign key to audit_events(id), plus whatever connector-specific columns it needs. The connector writes the row into its own extras table, inside the same transaction as the audit_events insert.
Connector-specific endpoints live under /connectors/{name}/... and join audit_events with the connector's extras table. The unified trace view never reads the extras tables.
The shape generalizes: a new connector that needs to record connector-specific references creates its own <connector>_audit_extras table and follows the same pattern. The core audit_events table stays neutral and never grows connector-shaped columns.
- Go 1.24+
- Docker and Docker Compose
protocandprotoc-gen-go(only needed when regenerating proto sources)
Clone the repository:
git clone https://github.com/apache/airavata-custos.git
cd airavata-custosStart the backing services (MariaDB, Prometheus, Grafana, Vault):
cd dev-ops/compose
docker compose up -dBuild and test a connector, e.g. ACCESS-CI AMIE:
cd connectors/ACCESS/AMIE-Processor
go build ./...
go test ./...See each connector's and extension's README for run and configuration details.
- Open a GitHub issue
- Subscribe to the Custos mailing list:
custos-subscribe@airavata.apache.org
@inproceedings{10.1145/3311790.3396635,
author = {Ranawaka, Isuru and Marru, Suresh and Graham, Juleen and Bisht, Aarushi and Basney, Jim and Fleury, Terry and Gaynor, Jeff and Wannipurage, Dimuthu and Christie, Marcus and Mahmoud, Alexandru and Afgan, Enis and Pierce, Marlon},
title = {Custos: Security Middleware for Science Gateways},
year = {2020},
isbn = {9781450366892},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3311790.3396635},
doi = {10.1145/3311790.3396635},
booktitle = {Practice and Experience in Advanced Research Computing},
pages = {278–284},
numpages = {7},
location = {Portland, OR, USA},
series = {PEARC '20}
}
@inproceedings{10.1145/3491418.3535177,
author = {Ranawaka, Isuru and Goonasekara, Nuwan and Afgan, Enis and Basney, Jim and Marru, Suresh and Pierce, Marlon},
title = {Custos Secrets: A Service for Managing User-Provided Resource Credential Secrets for Science Gateways},
year = {2022},
isbn = {9781450391610},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3491418.3535177},
doi = {10.1145/3491418.3535177},
booktitle = {Practice and Experience in Advanced Research Computing},
articleno = {40},
numpages = {4},
location = {Boston, MA, USA},
series = {PEARC '22}
}
This project is funded by the National Science Foundation (NSF).
We are grateful to Trusted CI for conducting the First Principles Vulnerability Assessment (FPVA) for this software and providing security architecture guidance and improvements.