Guardrails for AI agents that run while you sleep

The failures are supervision failures

Every one of these happened on our own machines, running real jobs:

• A media pipeline filled 86 GB of disk overnight. Nothing crashed. The disk was just full.
• A broadcast bot got a phone number banned by sending about 1,100 messages in a day. It was doing exactly what it was told.
• A listings bot re-sent items it had already sent, because the file it read from changed under it.
• A subprocess hung, the exit trap around it never fired, and the “finished” notification never came.
• Log files grew to 400 MB because nothing was watching them.

None of these is a model problem. A better prompt would not have prevented any of them.launchd and systemd will restart a process, but they will not notice that it has spent $40, written 2 GB or sent the same message thirty times. So we built the supervisor that does, and published it: nightshift, MIT licensed, with zero runtime dependencies.

npm install -g nightshift-cli   # Node 20+, macOS or Linux

nightshift run --budget 5usd --max-runtime 2h --idle-timeout 15m \
  --report telegram -- claude -p "fix the failing tests and open a PR"

The npm package is nightshift-cli; the command it installs is nightshift. It wraps anything you can start from a terminal. For Claude Code, Codex CLI and OpenClaw it also reads the agent’s structured output and meters tokens and dollars live.

Budgets that end the run

A warning at 3 a.m. is a message nobody reads. In nightshift every limit is a kill. The agent gets SIGTERM, a grace period, then SIGKILL, and the report says which limit fired and why.

Three details matter more than the list itself.

Spend is measured while it happens

Claude Code only reports its dollar cost at the end of a run. A budget that fires after the money is spent is not a budget. So nightshift estimates spend live from tokens at list prices, then reconciles against the reported figure when the run ends. The report shows both. Output, token and dollar limits are checked on every chunk the agent writes, so a flood is stopped within one write, not on the next polling tick.

Unknown means expensive

A model with no list price is counted at the most expensive rate. The budget is never blind to a new model id. For Claude Code, the dollar budget is also passed on as --max-budget-usd, so the agent stops itself first and nightshift is the second line.

A limit that cannot be measured refuses to start

If you set --budget on a command nightshift cannot meter, such as your own shell script, it will not run unless you pass --allow-unmetered. A limit that silently does nothing is worse than none.

A kill that stops the whole tree

Most supervisors send one signal to one process and hope. Agents spawn shells, which spawn tools, which spawn servers. Kill the parent and the server keeps running. nightshift uses five separate ways to find what belongs to a run:

1. The process group. The agent leads its own group, so one signal reaches the tree.
2. The descendant map. A process-tree walk at spawn, on every chunk of output and on every tick. Each pid is stored with its start time, so a recycled pid is never signalled by mistake.
3. The environment marker. Every process the run starts inherits NIGHTSHIFT_RUN_ID. Anything still carrying it is ours.
4. The stray check. Anything that appeared after the run began, was re-parented to init, and still has the run’s working directory is ours too.
5. The cgroup, on Linux. Cgroup membership is inherited on fork and cannot be given up without write access, so it lists every descendant regardless of what the process did to itself.

The grace period is measured from the SIGTERM, not from when the direct child happened to exit. After the agent exits, the same checks sweep for orphans and kill them. Anything that survives all of it is listed by pid under Survivors, in bold, because a kill that quietly did not kill is the worst outcome.

The README also documents the gap. On macOS, a process can still escape if a silent parent detaches it into a new session and exits before the first tree walk, and the process also hides its environment and leaves the run’s working directory. Every step has to be deliberate. The test suite includes exactly that case. On Linux with the cgroup active it is caught; elsewhere it is reported as not applicable, never as a pass.

That suite is how we keep the claims honest. It lists 26 deliberately broken agents: one that hangs, one that ignores SIGTERM, one that floods stdout, one that fills the disk, one that sends 21 messages. A case passes only if the report says the right thing and no process from the case is left alive. It runs in CI on macOS and Linux.

Side effects that happen once

Two of the incidents above were the same bug: an agent sent something it had already sent. The fix is a ledger the agent must write to before any side effect, with a rate limit on the scope.

nightshift ledger claim --scope telegram --key "listing-8812" --limit 40/day \
  && send_listing 8812

A key is claimed once per scope, ever. The same key again is refused as a duplicate. The 41st claim in a sliding day is refused as capped. For Claude Code there is a hook, so the agent itself does not change: matching shell commands are claimed before they run, and a refusal comes back as a reason the model reads. If the ledger cannot record a claim, for example because the disk is full, the send is denied. A side effect that cannot be recorded cannot be deduplicated. The refusals come from the ledger, never from a model.

Councils: every answer is challenged

Budgets bound how much an agent does. They say nothing about whether its answer is right. For that we use a second pattern: fan out several independent attempts, try to refute each result, and keep only what survives. We packaged the orchestration as consilium, a small typed library with no runtime dependencies. It works with any model through a single Runner function.

const verdict = await council.refute({
  claim: "startOf('day') is timezone-safe here",
  context: theCode,
  voters: 3,
  threshold: "majority",
});
if (!verdict.survives) drop(finding);

Each voter is prompted to refute, not to agree, and defaults to refuted when unsure. Around that sit a few more primitives: fanout and pipeline for concurrent work, panel to judge one item through several lenses, and loopUntilDry to keep searching until a set number of rounds in a row find nothing new.

The design rule is that control flow is code, not a model. Loops, fan-out and conditionals are ordinary deterministic code you can step through. The model decides content, never orchestration. That makes the council itself testable with a mock runner and no API key. Budgets live here too: maxCalls caps every attempt, retries included, and a failed item becomes null instead of sinking the batch.

We are careful about what this proves. Surviving a vote is a workflow result, not proof that a finding is correct. Models can share the same mistakes. Whether a council beats a single call depends on the model, the prompts and the task, and needs evaluating on real examples.

nightshift itself went through this process before its first release. Four independent reviewers examined the code, and every finding was re-verified adversarially by two more; 31 confirmed defects were fixed. Three red-team agents then attacked the result. Two escaped the kill through a silent detach, which is why the fourth net exists. Every escape became a case in the suite.

The morning report

A run you cannot read about in the morning did not happen. Every nightshift run writes report.md, report.json and the full output log, including runs that were killed, runs whose command did not exist, and runs stopped with Ctrl-C. Reports can be delivered to Telegram, Discord or a webhook.

Two details make the report trustworthy. First, postconditions: --require out/report.json turns an agent that exits 0 without producing the file into a failed run, exit code 3. Nothing downstream should trust “exit 0” from an agent. Second, redaction: delivery credentials are kept out of the agent’s environment, and anything the agent prints that looks like a token or key is redacted before it is stored or sent.

What this does not do

nightshift is not a sandbox. It bounds how much and how long, not what. To bound what an agent may touch, use the agent’s own permission modes, a sandbox or a container. It is not a scheduler either, and a ledger refusal is a rule, not a human approval.

That last point is the one we hold to. As our homepage puts it, our engineers work with agents, but a person decides what gets built and reads every change, and nothing reaches a client until the automated tests pass and the record shows where each change came from. The guardrails make overnight work safe to leave running. They do not replace the review in the morning. Our AI disclosure sets out how we use AI and the oversight we keep.