Skip to content

Commit 2e44809

Browse files
committed
feat: per-group status pages, monthly SLA reports, hora doctor
Lightweight multi-tenancy for operators hosting client services: - /status/{group}: one display group's monitors, the overall badge recomputed from them, maintenance banners filtered to the group, no peers section. server.group_tokens entries reveal the full view of their group - private monitors included - and nothing else (never accepted as a global token); validated at load (length, and the group must exist). Unknown or fully-private-anonymous groups answer 404, revealing nothing. - /report/2026-05 + hora report [YYYY-MM] (default last month): a printable (print-first, "Save as PDF") monthly SLA report - uptime per monitor and group, incidents, downtime clipped to the month, MTTR over incidents resolved in the month, SLO verdict, error-budget consumption (conservative: covered time assumed fully monitored; the running month judged against elapsed time only). ?group= scopes the report to one group, with the group token accepted - the report an agency hands its client. - hora doctor: runtime diagnostics judged against what the config needs - database writable (BEGIN IMMEDIATE probe, no side effects), listen port (busy = warn: the daemon is probably running), IPv4/IPv6 routes (UDP connect, no packets), the unprivileged ICMP datagram socket, a real resolver lookup. Exits non-zero only when a *needed* capability is missing (no IPv6 route fails only with dual_stack monitors configured).
1 parent def66ea commit 2e44809

19 files changed

Lines changed: 1402 additions & 26 deletions

File tree

CHANGELOG.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,28 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
99

1010
### Added
1111

12+
- **Per-group status pages** (`/status/{group}`): one display group's
13+
monitors, nothing else - lightweight multi-tenancy for an operator hosting
14+
several clients on one Hora. A per-group token (`server.group_tokens`)
15+
reveals that group's full view - private monitors included - and nothing
16+
else (it is never accepted anywhere as a global token); the maintenance
17+
banners are filtered to windows touching the group, and the peers section
18+
stays off client pages. An unknown group, or a fully private one viewed
19+
anonymously, answers 404.
20+
- **Monthly SLA reports** (`/report/2026-05` and `hora report [YYYY-MM]`,
21+
default last month): a printable, print-first page - uptime per monitor
22+
and group, incidents, downtime (clipped to the month), MTTR (incidents
23+
resolved within the month), SLO verdict and error-budget consumption.
24+
`?group=` scopes the report to one group, with the group token accepted -
25+
the report an agency hands its client. Works as far back as the one-year
26+
aggregate retention; the running month is judged against elapsed time only.
27+
- **`hora doctor`**: runtime environment diagnostics - the companion of
28+
`hora check`. Database writable, listen port free (busy is a warning: the
29+
daemon is probably just running), IPv4/IPv6 routes (no packets sent), the
30+
unprivileged ICMP datagram socket, and a real system-resolver lookup. Each
31+
finding is judged against what the *current config needs* - no IPv6 route
32+
only fails when a `dual_stack` monitor needs one - and the process exits
33+
non-zero on any missing needed capability.
1234
- **Weekly digest** (`[digest]`): a recap of the last seven days through the
1335
notification channels - "99.97% overall, 2 incidents" plus one line per
1436
monitor with uptime, incidents and the error budget left when an SLO is

README.md

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,12 @@ Full guides for everything below live in the
7171
at 9am" at a glance.
7272
- **Private monitors** behind a viewer token - one Hora for a public status page
7373
*and* your internal services. Per-IP API rate limiting.
74+
- **Per-group status pages & SLA reports** (lightweight multi-tenancy) -
75+
`/status/{group}` shows one group only, and a **per-group token**
76+
(`server.group_tokens`) reveals that group's full view and nothing else.
77+
Monthly, printable **SLA reports** (`/report/2026-05`, optionally
78+
`?group=...`): uptime per monitor/group, incidents, MTTR, error budget -
79+
"here is your May report, 99.95%" for operators hosting client services.
7480

7581
**Operations**
7682

@@ -80,7 +86,9 @@ Full guides for everything below live in the
8086
kept a year; the database never grows forever. `hora backup` snapshots it in
8187
one statement.
8288
- **Uptime Kuma import** (`hora import kuma backup.json`), `hora check` for CI,
83-
`hora test-alert` to verify the notification chain before the first incident.
89+
`hora test-alert` to verify the notification chain before the first incident,
90+
and **`hora doctor`** to diagnose the runtime environment (IPv6 route, ICMP
91+
socket, DNS resolver, listen port, database) against what the config needs.
8492
- Single self-contained binary - migrations and templates compiled in.
8593

8694
## Quick start (Docker)
@@ -127,12 +135,14 @@ and on the container: `-e HORA_TELEGRAM_TOKEN=123:abc`. Only `HORA_BIND`,
127135
```sh
128136
hora # run the monitor
129137
hora check # validate the config; non-zero exit on error (CI-friendly)
138+
hora doctor # diagnose the runtime environment (IPv6, ICMP, DNS, port)
130139
hora test-alert # send a test down + recovered through every channel
131140
hora test-alert website # ... through the channels routed for monitor "website"
132141
hora silence api,web 10m "deploying" # mute alerts ad hoc (checks keep recording)
133142
hora silence list # show the active silences
134143
hora silence clear # remove every silence
135144
hora digest # print the weekly digest (dry run of [digest])
145+
hora report 2026-05 # print the monthly SLA report (default: last month)
136146
hora incidents # list recent incidents with their ids
137147
hora annotate last "fiber cut" # attach a note to an incident (shown on /history)
138148
hora backup /mnt/nas/hora-backup.db # consistent snapshot of the database (VACUUM INTO)
@@ -205,6 +215,8 @@ rate-limit settings are read once at startup and still require a restart.
205215
| `GET /metrics` | Prometheus metrics (text exposition format). |
206216
| `GET /history` | Incident history page (HTML). |
207217
| `GET /history.atom` | Incident history as an Atom feed. |
218+
| `GET /status/{group}` | Status page restricted to one display group. A `server.group_tokens` entry reveals that group's full view (and nothing else). |
219+
| `GET /report/{YYYY-MM}` | Printable monthly SLA report: uptime per monitor/group, incidents, MTTR, error budget. `?group=` scopes it (group token accepted). |
208220
| `GET /api/summary` | All monitors: status, 24h uptime (per-mille), p50/p95/p99 latency, cert days left, daily history; plus active incidents. |
209221
| `GET /api/monitors/{id}/latency?hours=24` | Latency samples `[{ "t", "latency_ms" }]` (404 if unknown). |
210222
| `POST /api/push/{id}` | Record a heartbeat for a push monitor. Send the token as an `X-Push-Token` header (preferred - it stays out of proxy access logs) or as `?token=…`. Optional `status=up\|down\|degraded`, `msg`, `ping`. 401 on a wrong token, 404 if not a push monitor. |

config.example.toml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,12 @@ rate_limit_burst = 30
3232
# /history and /history.atom; without it those endpoints only show public
3333
# monitors. Read live: a change applies on the next reload.
3434
# auth_token = "${HORA_AUTH_TOKEN}"
35+
# Per-group viewer tokens for the /status/{group} pages (lightweight
36+
# multi-tenancy): the token reveals the FULL view of its group - private
37+
# monitors included - and nothing else. Also accepted by the group-scoped
38+
# SLA report (/report/2026-05?group=...). The global auth_token always works.
39+
# [server.group_tokens]
40+
# "Clients ACME" = "${ACME_TOKEN}"
3541

3642
# --- Notification channels ------------------------------------------------
3743
# Each channel has a unique `name` and a `type`. Several channels may share a

crates/hora-core/src/config.rs

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,12 @@ pub struct Server {
164164
/// Sent as `Authorization: Bearer <token>` or `?token=` query parameter.
165165
#[serde(default)]
166166
pub auth_token: Option<Secret>,
167+
/// Per-group viewer tokens for the `/status/{group}` pages (lightweight
168+
/// multi-tenancy): a token here reveals the *full* view of its group -
169+
/// private monitors included - and nothing else. The global `auth_token`
170+
/// always works too.
171+
#[serde(default)]
172+
pub group_tokens: std::collections::HashMap<String, Secret>,
167173
}
168174

169175
/// A named notification channel. Several channels may share a `type` (e.g. two
@@ -1074,6 +1080,18 @@ fn validate_token(label: &str, token: Option<&Secret>) -> anyhow::Result<()> {
10741080

10751081
fn validate(config: &Config) -> anyhow::Result<()> {
10761082
validate_token("server.auth_token", config.server.auth_token.as_ref())?;
1083+
for (group, token) in &config.server.group_tokens {
1084+
validate_token(&format!("server.group_tokens[{group:?}]"), Some(token))?;
1085+
// A token for a group no monitor belongs to guards nothing - that is
1086+
// a typo'd group name, not a future plan.
1087+
anyhow::ensure!(
1088+
config
1089+
.monitors
1090+
.iter()
1091+
.any(|monitor| monitor.group.as_deref() == Some(group)),
1092+
"server.group_tokens: no monitor belongs to group {group:?}"
1093+
);
1094+
}
10771095
let channel_names = validate_channels(config)?;
10781096

10791097
for window in &config.maintenance {

crates/hora-core/src/doctor.rs

Lines changed: 264 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,264 @@
1+
//! `hora doctor`: runtime environment diagnostics - the companion of
2+
//! `hora check`. The config can be perfectly valid while the *environment*
3+
//! can't honour it: no IPv6 route for a `dual_stack` monitor, an ICMP socket
4+
//! forbidden by `net.ipv4.ping_group_range` in rootless Docker, a busy listen
5+
//! port, an unreachable resolver. Each finding says whether the current
6+
//! config actually needs the capability, so a failure is actionable, not
7+
//! noise.
8+
9+
use std::time::{Duration, Instant};
10+
11+
use crate::config::{Config, Kind};
12+
13+
/// Severity of one finding. `Fail` means a capability the *current config
14+
/// needs* is missing - `hora doctor` exits non-zero on any of these.
15+
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
16+
pub enum Status {
17+
Ok,
18+
/// Worth knowing, not blocking (e.g. the listen port is busy because the
19+
/// daemon is already running, or IPv6 is absent but nothing needs it).
20+
Warn,
21+
Fail,
22+
}
23+
24+
impl Status {
25+
#[must_use]
26+
pub fn label(self) -> &'static str {
27+
match self {
28+
Self::Ok => "ok",
29+
Self::Warn => "warn",
30+
Self::Fail => "FAIL",
31+
}
32+
}
33+
}
34+
35+
/// One diagnostic finding.
36+
#[derive(Debug)]
37+
pub struct Finding {
38+
pub name: &'static str,
39+
pub status: Status,
40+
pub detail: String,
41+
}
42+
43+
impl Finding {
44+
fn new(name: &'static str, status: Status, detail: impl Into<String>) -> Self {
45+
Self {
46+
name,
47+
status,
48+
detail: detail.into(),
49+
}
50+
}
51+
}
52+
53+
/// Run every diagnostic against `config`'s requirements.
54+
pub async fn run(config: &Config) -> Vec<Finding> {
55+
vec![
56+
database(config).await,
57+
listen_port(config).await,
58+
ip_route("ipv4", "8.8.8.8:53", true, config),
59+
ip_route(
60+
"ipv6",
61+
"[2001:4860:4860::8888]:53",
62+
needs_ipv6(config),
63+
config,
64+
),
65+
icmp_socket(config),
66+
dns_resolver().await,
67+
]
68+
}
69+
70+
/// Whether any monitor needs working IPv6 on the probing host.
71+
fn needs_ipv6(config: &Config) -> bool {
72+
config
73+
.monitors
74+
.iter()
75+
.any(super::config::Monitor::dual_stack)
76+
}
77+
78+
/// The database: openable and writable when it exists; merely announced when
79+
/// it does not (the daemon creates it on first start - doctor must not).
80+
async fn database(config: &Config) -> Finding {
81+
let path = &config.server.database_path;
82+
if path != ":memory:" && !path.starts_with("file:") && !std::path::Path::new(path).exists() {
83+
return Finding::new(
84+
"database",
85+
Status::Ok,
86+
format!("{path} does not exist yet - created on first start"),
87+
);
88+
}
89+
let options = sqlx::sqlite::SqliteConnectOptions::new()
90+
.filename(path)
91+
.busy_timeout(Duration::from_secs(2));
92+
let pool = match sqlx::sqlite::SqlitePoolOptions::new()
93+
.max_connections(1)
94+
.connect_with(options)
95+
.await
96+
{
97+
Ok(pool) => pool,
98+
Err(err) => return Finding::new("database", Status::Fail, format!("{path}: {err}")),
99+
};
100+
// A write-lock probe without writing anything: BEGIN IMMEDIATE takes the
101+
// reserved lock (fails on a read-only mount), ROLLBACK releases it.
102+
let writable = sqlx::raw_sql("BEGIN IMMEDIATE; ROLLBACK;")
103+
.execute(&pool)
104+
.await;
105+
pool.close().await;
106+
match writable {
107+
Ok(_) => Finding::new("database", Status::Ok, format!("{path} is writable")),
108+
Err(err) => Finding::new(
109+
"database",
110+
Status::Fail,
111+
format!("{path} not writable: {err}"),
112+
),
113+
}
114+
}
115+
116+
/// The listen address: bindable, or busy (which usually just means the daemon
117+
/// is already running - a warning, not a failure).
118+
async fn listen_port(config: &Config) -> Finding {
119+
let bind = &config.server.bind;
120+
match tokio::net::TcpListener::bind(bind).await {
121+
Ok(_listener) => Finding::new("listen", Status::Ok, format!("{bind} is free")),
122+
Err(err) if err.kind() == std::io::ErrorKind::AddrInUse => Finding::new(
123+
"listen",
124+
Status::Warn,
125+
format!("{bind} already in use - is Hora already running?"),
126+
),
127+
Err(err) => Finding::new("listen", Status::Fail, format!("cannot bind {bind}: {err}")),
128+
}
129+
}
130+
131+
/// Whether the host has a route for one IP family. `UdpSocket::connect` only
132+
/// consults the routing table - no packet is sent - so this is instant and
133+
/// quiet. The classic catch: Docker's default bridge networks have no IPv6,
134+
/// which silently breaks `dual_stack` monitors.
135+
fn ip_route(name: &'static str, probe_addr: &str, needed: bool, config: &Config) -> Finding {
136+
let local = if name == "ipv6" {
137+
"[::]:0"
138+
} else {
139+
"0.0.0.0:0"
140+
};
141+
let routed = std::net::UdpSocket::bind(local)
142+
.and_then(|socket| socket.connect(probe_addr))
143+
.is_ok();
144+
match (routed, needed) {
145+
(true, _) => Finding::new(name, Status::Ok, "route to the public internet"),
146+
(false, true) => {
147+
let dual = config
148+
.monitors
149+
.iter()
150+
.filter(|monitor| monitor.dual_stack())
151+
.count();
152+
Finding::new(
153+
name,
154+
Status::Fail,
155+
format!(
156+
"no route - {dual} dual_stack monitor(s) need it \
157+
(in Docker, enable IPv6 on the network or use host networking)"
158+
),
159+
)
160+
}
161+
(false, false) => Finding::new(name, Status::Warn, "no route (no monitor needs it)"),
162+
}
163+
}
164+
165+
/// The unprivileged ICMP datagram socket - exactly what the icmp probes open.
166+
/// In rootless Docker this hinges on `net.ipv4.ping_group_range` covering the
167+
/// process's gid (or `CAP_NET_RAW`).
168+
fn icmp_socket(config: &Config) -> Finding {
169+
let needed = config
170+
.monitors
171+
.iter()
172+
.filter(|monitor| monitor.kind == Kind::Icmp)
173+
.count();
174+
let ping_config = surge_ping::Config::builder()
175+
.kind(surge_ping::ICMP::V4)
176+
.sock_type_hint(socket2::Type::DGRAM)
177+
.build();
178+
match surge_ping::Client::new(&ping_config) {
179+
Ok(_client) => Finding::new("icmp", Status::Ok, "unprivileged datagram socket available"),
180+
Err(err) if needed > 0 => Finding::new(
181+
"icmp",
182+
Status::Fail,
183+
format!(
184+
"socket unavailable ({err}) - {needed} icmp monitor(s) need it; \
185+
widen net.ipv4.ping_group_range or grant CAP_NET_RAW"
186+
),
187+
),
188+
Err(err) => Finding::new(
189+
"icmp",
190+
Status::Warn,
191+
format!("socket unavailable ({err}) (no icmp monitor configured)"),
192+
),
193+
}
194+
}
195+
196+
/// The system resolver, exercised with a real lookup - the probes' DNS path.
197+
async fn dns_resolver() -> Finding {
198+
let resolver = match hickory_resolver::TokioResolver::builder_tokio()
199+
.map(hickory_resolver::ResolverBuilder::build)
200+
{
201+
Ok(Ok(resolver)) => resolver,
202+
Ok(Err(err)) | Err(err) => {
203+
return Finding::new("dns", Status::Fail, format!("resolver setup failed: {err}"));
204+
}
205+
};
206+
let started = Instant::now();
207+
match tokio::time::timeout(Duration::from_secs(5), resolver.lookup_ip("example.com.")).await {
208+
Ok(Ok(_)) => Finding::new(
209+
"dns",
210+
Status::Ok,
211+
format!(
212+
"system resolver answered in {}ms",
213+
started.elapsed().as_millis()
214+
),
215+
),
216+
Ok(Err(err)) => Finding::new("dns", Status::Fail, format!("lookup failed: {err}")),
217+
Err(_elapsed) => Finding::new("dns", Status::Fail, "lookup timed out (5s)".to_owned()),
218+
}
219+
}
220+
221+
#[cfg(test)]
222+
mod tests {
223+
use super::*;
224+
225+
fn config(toml: &str) -> Config {
226+
crate::config::parse(toml).expect("config")
227+
}
228+
229+
#[tokio::test]
230+
async fn missing_database_is_announced_not_created() {
231+
let config = config(
232+
r#"
233+
[page]
234+
[server]
235+
database_path = "/tmp/hora-doctor-test-does-not-exist.db"
236+
"#,
237+
);
238+
let finding = database(&config).await;
239+
assert_eq!(finding.status, Status::Ok);
240+
assert!(finding.detail.contains("created on first start"));
241+
// Doctor must be side-effect free.
242+
assert!(!std::path::Path::new("/tmp/hora-doctor-test-does-not-exist.db").exists());
243+
}
244+
245+
#[test]
246+
fn ip_route_severity_depends_on_need() {
247+
// A guaranteed-unroutable family: probing a v6 address from a v4-only
248+
// socket fails at connect; with no dual_stack monitor it only warns.
249+
let quiet = config(
250+
r#"
251+
[page]
252+
[server]
253+
[[monitors]]
254+
id = "web"
255+
name = "Web"
256+
target = "https://example.com"
257+
interval_secs = 60
258+
"#,
259+
);
260+
let finding = ip_route("ipv6", "0.0.0.1:1", false, &quiet);
261+
// Whatever the host's networking, "not needed" can never be a Fail.
262+
assert_ne!(finding.status, Status::Fail);
263+
}
264+
}

0 commit comments

Comments
 (0)