Monitoring

Master switch for the monitoring subsystem. Note: a configured Redis (config.redis) is a HARD precondition — the entire block (MonitoringService construction, collectors, alerts and the HTTP endpoints) is gated on `redis && monitoring.enabled`, so with no Redis wired the subsystem silently does not start even when enabled is true. (config.liveMonitoring is gated independently on its own enabled flag and needs no Redis.)

Host metrics (on by default). collectInterval sets the sampling cadence (default 10s — the 30s in the sample above is just an override example); metrics toggles cpu, memory, disk and process (all on by default) plus network (opt-in, defaults false — collected only when explicitly set true, whereas the other four are collected unless explicitly set false).

In-process metrics (on by default). metrics toggles requests, responseTime, errors and rateLimits — your real-time picture of API behaviour.

Postgres health — OPT-IN (default off). metrics toggles connections and queryTime, with slowQueryThreshold (default 100ms — the 500ms in the sample above is just an override example) marking the boundary for the slow-query count.

Toggle collection of Redis health metrics — OPT-IN (default off).

History storage configuration.

Threshold-based alerting.

HTTP surface for metrics.

A simple status + timestamp probe for an uptime badge.

Pull the recent live in-memory activity feed (memory/CPU/Dapr/WebSocket/request) — this is the live-monitoring feed, not persisted metric history for charts.

Read or tune which feeds are captured and at what cadence, from an admin panel.

SystemCollector reads CPU/memory/disk/network/process (network via /proc/net/dev on Linux); ApplicationCollector tallies requests/response-time/errors/rate-limit blocks (fed by recordRequest / recordRateLimitBlock from the middleware); DatabaseCollector queries pg_stat_activity/pg_stat_database for connections and query timings/slow-query count (avg query time on PG14+); RedisCollector parses INFO for memory/clients/ops and hit rate (direct connection mode only — skipped under Dapr). Each enabled metric is flattened into dotted MetricPoints (e.g. system.memory.usagePercent).

Every snapshot is written to monitoring:{appId}:latest (1h TTL) and appended to a monitoring:{appId}:history list trimmed to the last hour — so the stream/snapshot/history endpoints get instant recent data without touching the database. (This Redis cache is separate from the live-monitoring GET_LOGS feed, which reads its own in-memory store, not these keys.)

Collected MetricPoints buffer in memory and are flushed in batches into the monitoring_metrics table on flushInterval (default 1m); a failed flush re-queues them. A daily prune deletes rows older than retentionDays.

After each collect, AlertService compares the snapshot against thresholds (cpu 80%, memory 85%, disk 90%, error-rate 5%, response-time 1000ms, rate-limit blocks 100/min by default). A breach raises an alert and, if configured, emails the recipients — then a cooldown (default 5m) suppresses re-alerting of that type for the window. The cooldown throttles the ENTIRE re-alert, not just email: within the window createAlert returns null before building the event, so neither the activeAlerts entry is refreshed nor an email is sent. Streaming persists purely because the alert entry is never auto-cleared, so getActiveAlerts (backing GET /alerts and the SSE 'alerts' event) keeps returning it for the process lifetime even after the metric normalizes. Note the retained entry is NOT frozen at the first breach: each breach outside the cooldown window overwrites the map entry with a fresh value/timestamp (createAlert sets lastAlertTime[type] and activeAlerts.set(type, alert)), while breaches inside the window early-return null without updating. So the surviving entry reflects the LAST breach before normalization — equal to the first breach only when the metric normalizes within a single cooldown window. Acknowledge flips acknowledged:true but does not remove it, and clearAlert (the only method that deletes an entry) is unused.