Archived: This repository has moved to https://git.yongbeom.com/dernbu/orca.

agent

agent is a local task-orchestration daemon. It loads YAML task definitions, starts isolated Run workspaces from cron, one-off schedule, or Lark message triggers, and executes each task as an ordered series of Bash, file, messaging, or reusable runner steps.

The service is intended for trusted automation: task and runner configuration can execute arbitrary commands with the daemon user's permissions.

How it works

Each task defines three lifecycle phases:

trigger -> global pre_init -> init -> idle
                                      |  ^
                 matching Lark reply |  | pre_resume -> resume
                                      v  |
                                     idle
                                      |
                                      | idle TTL expires
                                      v
                                   expired
                                      |
                                      v
                              workspace + row removed

• A trigger creates a Run with its own directory under $AGENT_CONFIG_DIR/runs.
• Local pre_init hooks run once, followed by the task's init steps.
• Replies in a Lark thread associated with the Run execute local pre_resume hooks and then task resume steps, resetting the idle TTL.
• When the idle TTL expires, reconciliation classifies the Run as expired, then removes its workspace and metadata without depending on current task config. Infrastructure cleanup failures retry with capped in-memory backoff; a daemon restart forgets that state and retries immediately. The daemon expiry path does not yet execute configured end steps; TD-074 tracks that contract.
• Steps within one phase run in order and stop at the first exhausted failure. Init and resume operations share a process-wide worker pool, with at most AGENT_EXEC_WORKER_COUNT executions active at once. Operations for one Run execute one at a time in queue order; unrelated Runs may bypass a blocked same-Run item. Submission admission and preparation are synchronous and are not worker-pool work.

Run metadata, pending execution work, Lark thread mappings, Lark ingress deduplication claims, and cron/schedule watermarks are stored in $AGENT_CONFIG_DIR/db.sqlite3. Idle Runs, their workspaces, and queued lifecycle work can therefore survive daemon restarts. Before opening SQLite, the daemon takes a nonblocking exclusive lock on $AGENT_CONFIG_DIR/daemon.lock; a second daemon targeting that state root exits with the lock path and the owning PID when available. The persistent file is only a rendezvous point and PID diagnostic—the kernel-held lock is the source of ownership and is released even if the process crashes. There is no exact crash recovery for a command that was already in flight; an unacknowledged queue row is retried after restart and may repeat side effects.

Requirements

• Go 1.26.4 or the version declared in service/go.mod
• just
• Bash and Git
• macOS only for the included launchd integration
• Lark app credentials only when using Lark triggers or lark-message-send steps
• Any executable referenced by a custom runner, such as codex, agent, or opencode
• jq when using the checked-in Codex, Cursor Agent, or OpenCode runners

Quick start

1. Create the local environment file:

   cp .env.example .env
   

2. Leave the optional values empty for a local-only setup, then validate and test:

   just validate
   just service_test
   

3. Start the daemon in the foreground in one terminal:

   just service_run
   

4. In another terminal, submit the disabled example task once. Debug execution ignores enabled, returns after durable queue admission, and leaves the Run lifecycle and workspace cleanup to the daemon:

   just debug_task test_task
   

Always use the just recipes. The root Justfile loads .env; running go run directly does not.

Configuration

The checked-in configuration lives under .config:

.config/
├── global-hooks.yaml
├── runners/
│   └── *.yaml
└── tasks/
    └── *.yaml

The checked-in codex, agent, and opencode runners use jq to extract a harness session ID from machine-readable init output. Each corresponding task step must set SESSION_ID_FILE_PATH to a path relative to its Run workspace. An init invocation that exits zero atomically publishes the extracted ID (or an empty file when no ID can be extracted), and later resume steps pass a nonempty stored ID to the harness's exact-session option. A missing, unreadable, or empty file falls back to that harness's --last or --continue behavior. Init and resume steps use the same path to share a session; different paths allow several harness sessions in one workspace.

A minimal task looks like this:

name: "Heartbeat"
id: heartbeat
enabled: true
ttl: 24h
on:
  - cron: '*/5 * * * *'
    mode: create_new

init:
  steps:
    - type: shell
      run: |
        set -euo pipefail
        date -u

resume:
  steps: []

end:
  steps: []

See Configuration reference for the complete task, trigger, phase, step, runner, global-hook, environment, and merge syntax. Checked-in examples are available in .config/tasks and .config/runners.

Optional remote configuration

Set ADDITIONAL_CONFIG_GIT_REPO to a comma-separated list of Git URLs to merge additional configuration roots with the checked-in .config:

ADDITIONAL_CONFIG_GIT_REPO=git@example.com:team/agent-config.git,git@example.com:team-2/agent-config.git

Each repository must put tasks/ and runners/ at its root. The daemon clones or refreshes each repository's default branch into its own stable checkout under $AGENT_CONFIG_DIR/remote-config/<name>. Task IDs and runner step_type values must remain unique across all roots. Global hooks are local-only. Repositories refresh in env-var order. Each fetched revision is validated together with the local configuration and repositories already accepted earlier in that pass; a cross-repo duplicate rejects the later checkout and rolls it back through first-parent history to the newest valid ancestor, which remains checked out across daemon restarts. If no ancestor validates, Run admission is blocked rather than falling back to local-only config. When the remote is unreachable but a usable clone already exists on disk, refresh degrades instead of blocking: the last on-disk checkout is re-validated and served until a successful refresh, and daemon logs and debug report per-repository revisions, served-offline flags, and served-ancestor flags; debug derives a served-offline flag and the resolved revision of the last-refreshed repository from them. Empty entries and duplicated URLs in the variable are rejected and block config reads.

Validate one or more remote configuration checkouts together with the built-in configuration before publishing them:

just validate ../agent-config ../team-2-agent-config

An invalid file or cross-root collision invalidates the combined configuration and prevents task execution.

Built-in config mirror

Before every init and resume Turn, the checked-in local hooks invoke a compiled Go built-in to copy the complete repository .config tree to ./builtin-config-reference/ in the Run workspace. Remote-config authors can inspect this mirror to discover local task IDs, runner step_type values, and the complete definitions against which their changes will be merged.

The mirror is deliberately not sanitized. It includes committed built-ins and ignored machine-local files and may contain operationally sensitive configuration. It is intentionally available to every Run step and must not be treated as a secret-filtered or public interface. Removing write bits is only an accidental-edit guardrail: a process running as the daemon user can restore them, and this is not a security boundary. Workspace edits never propagate back to the live repository; the hook replaces the mirror on the next init or resume.

Refresh uses a staged complete copy and fails the Turn before task steps if the source is missing, copying fails, or any symlink or special filesystem entry is found. Supporting explicitly bounded symlinks can be reconsidered if a concrete use case appears. The mirror is not refreshed before end, so a refresh problem cannot block workspace cleanup. Live local config remains authoritative for validation and can change after a mirror is copied.

Environment

just reads the ignored .env file. Supported service settings are:

Variable	Purpose	Default
AGENT_CONFIG_DIR	State, remote config clone, database, and Run workspace root	~/.agent-config
AGENT_EXEC_WORKER_COUNT	Maximum concurrent lifecycle workers; must be a positive integer	12
ADDITIONAL_CONFIG_GIT_REPO	Optional comma-separated Git URLs for additional config roots	empty
LARK_CHAT_ID	Fallback destination for task-produced Lark messages when the Run has no stored chat	empty
LARK_APP_ID	Lark app ID for the listener and messaging client	empty
LARK_APP_SECRET	Lark app secret for the listener and messaging client	empty
DEFAULT_ALLOWED_LARK_USER	Live Open ID allowed for inbound Lark operations when access_control.lark is absent	empty; affected operations fail closed

The daemon process environment is inherited by step scripts. It also injects Run, task, step, and Lark-message metadata; the full contract and precedence rules are documented in Environment available to steps.

Do not commit .env. Runner scripts and shell steps are trusted code and can read the daemon's environment.

Lark setup and behavior

Lark integration is optional. Without both LARK_APP_ID and LARK_APP_SECRET, the daemon logs a warning, skips the WebSocket listener, and continues to run cron and schedule tasks.

To use it:

1. Configure a Lark app for long-connection IM message events and grant the permissions needed to receive messages and send/reply to them. Reaction feedback additionally needs one write scope (im:message or im:message.reactions:write_only) plus im:message.reactions:read (or the broader im:message:readonly) so interrupted feedback can be reconciled.
2. Add the bot to each conversation it should handle.
3. Set LARK_APP_ID, LARK_APP_SECRET, and a concrete DEFAULT_ALLOWED_LARK_USER Open ID in .env.
4. Add an enabled lark_bot trigger with an exact chat ID or "*", then put any explicit user policy at task level under access_control.lark.allowed_users. Omit the Lark policy to use the live default, list concrete Open IDs to replace it, or use ["*"] to make the task explicitly public to valid Lark senders.

The dispatcher filters chat/regex matches through their task-level user policies. One authorized task match starts a Run; several authorized matches produce the existing conflict reply. A raw match with no authorization gets a generic denial and runs nothing. A reply in an already-associated Lark thread is re-authorized against the task's current explicit policy, or the live default when no explicit policy exists. This applies even if a non-Lark trigger started the Run. A denial never falls through to a new command. The built-in lark-message-prepare and lark-message-send accept an optional mode. thread_reply is the backward-compatible default: it replies in the Run's associated thread or, if none exists, starts the first message in LARK_CHAT_ID. new_message always starts a fresh top-level message in the authoritative chat and moves the Run binding to the returned message metadata, so replies to the superseded thread no longer resume the Run.

Accepted daemon lifecycle work, including Lark work, waits in an unbounded SQLite-backed queue. Admission returns only after its row commits. The queue/store serializes work FIFO within each Run and allows unrelated Runs to use free workers. Before returning work to a worker, it atomically changes the row from queued to in_flight only if that Run has no other in-flight row. Only then does the worker report Started and invoke the executor. Startup resets in_flight rows to queued before replay.

Succeeded, failed, and rejected work ACKs only the delivered row. It does not yet cancel already-queued work for the same Run. Interrupted work is retained for restart recovery. An ACK failure is logged and also leaves the row for restart recovery.

Before forwarding a validated inbound Lark message, the adaptor atomically claims its message ID in SQLite. Another delivery of the same pending ID does not reach the dispatcher and returns a retryable error; after the owner succeeds, a retry observes success and is acknowledged. Claim blocks on any existing success row regardless of age; hourly reconciliation removes expired successes, which is the only way an ID is re-enabled. A downstream admission error releases the claim and is returned so Lark can retry. Startup removes pending claims inherited from a prior daemon before starting the listener. A failed release is retried when the next pending redelivery arrives. A crash between durable downstream acceptance and the success transition remains deferred under TD-082 in TODOs.md.

The durable payload includes the task/runner/hook snapshot and invocation environment (including Lark chat, message, thread, and sender values). Daemon configuration environment and lifecycle callbacks are not persisted. Recovered work snapshots the former from the new daemon and has no callback. Lifecycle callbacks remain the internal completion path for schedule watermarks and cleanup; cron watermarks advance synchronously after durable queue admission. Callbacks do not send Lark feedback. Debug submission returns at durable queue admission and does not install a completion callback.

The execution service submits user-facing feedback independently for admitted, started, and terminal execution states. The dispatcher resolves the target from the turn's persisted LARK_MESSAGE_ID, so recovered work can still update its original message. Turns without a message ID are submitted and become no-ops in the dispatcher. Pre-admission failures use a raw Lark message ID; thread IDs are not accepted as feedback targets.

For handled Lark messages, feedback adds OneSecond while work is queued, replaces it with Typing when a worker starts execution, and removes the managed reaction on success. It leaves a CrossMark on failures and pre-execution rejections. Existing authorization, multiple-match, and expired Run replies flow through the same dispatcher. V1 delivery is process-local, fire-and-forget, unbounded, unordered, and not retried or drained at shutdown; delivery failures are logged and never change the task result. Reaction ownership is stored in SQLite before Lark is mutated; after restart, the listener waits for stale owned reactions to be reconciled before accepting messages.

Trigger delivery notes

• Cron: every cron entry requires mode: create_new | resume; resume additionally requires a task-unique workspace_key. Each trigger contributes at most one missed firing per scan and owns a watermark keyed by a global identity containing its task ID, index, and complete-trigger hash. Editing or reordering a trigger makes it new. A create_new firing initializes a fresh Run and a resume firing continues its workspace's Run (starting a first-use init when no binding exists). A per-trigger process mutex covers eligibility, durable admission, and watermark persistence; the watermark advances at admission, so later execution failure does not retry that occurrence. Queue delivery serializes work for the same continuing Run. This is a hard cutover: migration 015 drops every legacy task-level cron watermark, so every configured trigger fires once as new after rollout. The old reader rejects the new fields and the new validator rejects old cron entries, so stop scheduling while publishing the new remote config and binary/local config, validate them together with the new binary, then restart.
• Schedule: due timestamps are evaluated with a per-task SQLite watermark. A timestamp missed during downtime is run after restart. The watermark is saved only after successful task initialization, favoring duplicate delivery over loss.
• Watermark cleanup: after each successful daemon configuration refresh (at startup and once per minute), cron rows whose exact trigger identity is absent and schedule rows whose task ID is absent are removed from the complete validated configuration. Current disabled triggers/tasks retain their rows. Each table cleans atomically and independently; failures are logged without blocking dispatch and are retried after the next refresh. Cleanup is eventually consistent with dispatch already in flight.
• Lark: message regular expressions use Go syntax. Zero raw matches do nothing; unauthorized matches are denied before Run admission; multiple authorized task IDs produce an error reply and run nothing. These application outcomes are acknowledged and mark the inbound message ID successful. A durable queue-admission failure releases the pending ID and is returned to Lark so it may redeliver. A pending duplicate is also returned for retry without re-running selection or admission; a successful duplicate is acknowledged.

Use idempotent task actions whenever a trigger can be delivered more than once.

Commands

Command	Description
just / just lint	Run the target internal import-architecture lint
just service_run	Run the daemon in the foreground
just service_test	Run all Go tests
just service_arch_lint	Check the target internal import architecture
just validate [config_dir ...]	Run the architecture lint, then validate local config plus optional additional roots
just debug_task <task_id>	Refresh config and queue one task's initialization through the running daemon, even when disabled
just daemon_ping	Ping the running daemon over its private control socket
just daemon_install	Render, lint, and load the macOS LaunchAgent
just daemon_start	Restart the loaded LaunchAgent
just daemon_status	Print LaunchAgent state
just daemon_uninstall	Unload and remove the LaunchAgent

just validate runs the architecture lint as part of required validation. Fix reported imports instead of broadening the target allowlist solely to make the check pass.

Successful shell-step stdout is not printed. On failure, the service logs the captured stdout and stderr.

Running with launchd

On macOS:

just daemon_install
just daemon_status

The generated LaunchAgent runs run.sh, restarts automatically, and writes stdout.log and stderr.log in the repository root. After changing the plist template or environment, reinstall it. Remove it with:

just daemon_uninstall

The generated plist captures PATH, while run.sh enters the repository and uses just, which loads .env.

The checked-in launchd module installs com.yongbeom.local-agent-dev as a second, local-development job. It does not unload or remove the stable com.yongbeom.local-agent job. When both jobs run, their checkouts must provide different AGENT_CONFIG_DIR values and different Lark app credentials. The installer does not inspect or pin those values: every restart reloads the current checkout's .env. Separate state roots isolate SQLite, Run workspaces, the remote-config cache, and control sockets; they do not prevent both daemons from using the same remote task configuration. If two jobs are accidentally configured with the same state root, the later process fails before opening SQLite because the first process holds that root's daemon lock.

State and generated files

With the default AGENT_CONFIG_DIR, runtime state is:

~/.agent-config/
├── db.sqlite3       # Runs, execution queue, Lark metadata, trigger watermarks
├── daemon.lock      # persistent kernel-lock rendezvous and diagnostic owner PID
├── daemon.sock      # private control endpoint while the daemon is running
├── remote-config/   # one checkout per additional config repo
└── runs/            # active Run workspaces

Do not edit any remote-config/<name>/ checkout directly. It is a disposable clone: refresh follows the origin's current default branch and removes all tracked, untracked, and ignored checkout drift. Invalid repositories are removed and cloned again at their own derived path. If the update steps fail but a usable clone already exists on disk, refresh degrades and serves that on-disk fallback until a successful refresh; only a recovery with nothing usable on disk blocks config reads. Validation may leave a checkout at a validating first-parent ancestor. Older installations may retain an unused repos/ cache from the former worktree implementation; the current service does not read it. The state database and Run workspaces can contain task inputs, Lark message contents, command output, secrets, and the unsanitized builtin-config-reference/ mirror. At startup, the daemon repairs the state root and runs root to mode 0700 and the SQLite database to 0600; a symbolic link at the final runs path is rejected. Periodic reconciliation removes old managed workspace directories only when neither a Run row nor an active init request owns them; a one-hour grace period protects the creation interval.

Development

The main code is under service/:

service/cmd/daemon             daemon and one-minute dispatch loop
service/cmd/utils              validation and daemon-control clients, including debug submission
service/internal/config        YAML models, readers, validation, config refresh
service/internal/dispatch      cron, schedule, debug, and Lark admission
service/internal/exec          Run lifecycle, environments, and step execution
service/internal/adaptor       filesystem, Git, Lark, and SQLite integrations

Before submitting a change:

just validate
just service_test

Contributor-specific architecture and debugging notes live in AGENTS.md. Ownership boundaries are documented in docs/ownership.md, and active work is tracked in TODOs.md.

Troubleshooting

Task not found for a remote task

Run through just. Direct go run does not load .env, so ADDITIONAL_CONFIG_GIT_REPO is empty and the remote config is not fetched.

Lark listener is disabled

Set both LARK_APP_ID and LARK_APP_SECRET. Cron and schedule dispatch remain available without them.

Configuration validates locally but not with the remote root

Run just validate /path/to/remote-checkout. Look for duplicate task IDs, duplicate runner step_type values, or a custom runner environment requirement that no step or supported service setting provides.

A debug submission appears silent

That is expected after successful queue admission: just debug_task returns without waiting for execution or printing a task result. Inspect the running daemon's logs and the task's configured side effects. Redirecting the debug command captures only client-side socket, refresh, validation, lookup, or admission errors.

A schedule or cron action ran twice

Delivery is at least once in several failure and restart paths. Make the action idempotent; see Trigger delivery notes.