Operator Workflow
Status: Canonical
Date: 2026-08-11
This document is the shortest supported operator path for the current Home AI Cluster architecture. It defines three distinct modes:
- ordinary local-only operation;
- ordinary explicit static multi-node operation;
- explicit historical two-machine proof operation.
Use the command reference for current syntax and option lookup; this document remains the procedural sequence.
The ordinary workflows below assume the published package is installed and use
hac. From a repository checkout, contributors can prepare the locked
environment with uv sync --locked and use uv run hac ... for development.
Local-only is the default, shortest, and least operationally complex path. All external runtimes remain operator-owned. Home AI Cluster does not start, stop, supervise, repair, or discover runtimes or remote machines.
Daily-use overview
The ordinary process is started once and can serve repeated requests. chat,
code, summarize, and classify are finite clients of that already-running
process; they do not start it.
Local-only (the shortest default path):
external runtime
-> hac local
-> repeated hac chat / hac summarize / hac classify / hac code requests
-> stop hac local
Explicit static cluster:
receiver runtime + receiver hac local
-> caller preflight/status when useful
-> caller hac static-cluster
-> repeated caller hac chat / hac summarize / hac classify / hac code requests
-> stop caller, then receiver
The second path adds an explicit caller, receiver, and saved declaration file; it does not replace the local-only default. A declared remote does not guarantee remote execution because ordinary routing is local-first.
For repeated ordinary static-cluster startup, HAC-managed retained configuration can establish a local baseline:
hac config local --runtime ollama --execution-limit 2
hac config node summary-node --base-url http://192.0.2.10:25042 --capability summarize
hac config show
hac static-cluster
hac config show reports those retained facts; it does not observe runtime
health or prove that a cluster is running. HAC-managed retained configuration
provides the ordinary static-cluster startup baseline. A complete explicit
topology source replaces retained topology for that invocation; runtime
composition follows its own retained or explicit source-selection rules.
The retained local HAC execution limit is also consumed by ordinary
retained-baseline hac local and hac static-cluster. It limits overlapping
HAC-owned execution intervals, not runtime concurrency, and one process shares
the same limit for originating local work and receiver-side work.
Inspection remains separate: bare hac preflight is local-only,
while hac preflight --declaration <DECLARATION_PATH> inspects a selected static
topology and hac status --declaration <DECLARATION_PATH> observes the selected
declared cluster.
Inspection commands are finite observations, not mandatory prerequisites for every startup or request:
hac preflight --declaration <DECLARATION_PATH>checks static declaration coherence, not runtime or network availability.hac healthobserves the selected machine's local runtime composition, not declared remote nodes.hac status --declaration <DECLARATION_PATH>makes one bounded observation of the caller's local node and its declared remotes.
Capability lists shown by preflight and health belong to different scopes and need not match: preflight reports static topology routing eligibility, while health declares the ordinary local execution composition. Neither list is discovered from a runtime; health's live observation concerns adapter health.
None starts, supervises, repairs, or guarantees later request success. Historical proof runbooks and retained proof records are supporting evidence, not required steps in either ordinary daily path.
Static order and local HAC execution limits
Each machine's operator configures its own limit locally. For example:
Node A: local HAC execution limit = 1
Node B: local HAC execution limit = 1
Caller static order: A -> B
If A is already executing one HAC-owned interval, the caller still contacts A in normal static order. A may refuse before adapter invocation; after that exact safe refusal, the caller may consider B. The caller does not pre-query or know A's current work, limit, active interval count, or remaining allowance.
With Node A execution limit = 2, Node B execution limit = 1, and static
order A -> B, A can accept two overlapping HAC executions before a later
request is refused to the next candidate. This remains deterministic static
ordering, not least-loaded routing; it makes no fairness promise.
Mode 1: Ordinary local-only operation
1. Prepare the external local runtime
Install and start the runtime using its own supported procedure. Ensure its required model is locally available. Home AI Cluster does not own this process.
2. Run local-only preflight
hac preflight
This checks only that every adapter declared by an ordinary local node resolves in the ordinary local adapter registry. A coherent report does not prove that a runtime, model, or application is available.
Preflight, health, and status results are human-readable by default. Automation that needs their structured reports must request them explicitly:
hac preflight --json
hac health --json
hac status --declaration <DECLARATION_PATH> --json
This incremental output change applies to preflight, health, and status. Their
plain-text output is for ordinary operators; the explicit --json form retains
the compact structured output for automation.
3. Run local health
hac health
This observes the configured local runtime adapter. If it is not usable, repair or start the external runtime, confirm the required model, then rerun health.
The default health report keeps declared state separate from adapter
observations: Declared state is configured node metadata, while Adapter
observations report what the runtime adapter observed during this invocation.
Declared available and healthy can therefore truthfully appear beside an
adapter unavailable; when diagnosing current runtime usability, use the
adapter observation rather than treating the declared fields as a live probe.
A completed snapshot may show an unavailable, missing, or probe-failed
adapter observation while the command itself still completes successfully. Use
hac health --json when automation needs the existing compact structured
snapshot.
4. Start the ordinary local-only application
hac local
The native endpoint is:
http://127.0.0.1:25042/v1/chat
This existing ordered-message endpoint carries both ordinary chat and code;
there is no /v1/code endpoint.
The same process also exposes the native bounded summarize endpoint:
http://127.0.0.1:25042/v1/summarize
It also exposes the native bounded classification endpoint:
http://127.0.0.1:25042/v1/classify
With this ordinary default exact host, the same process also serves the fixed
browser page at http://127.0.0.1:25042/. Its navigation is Chat, Code, Image,
Summarize, Classify, and Configuration, which make same-origin calls to the
existing native endpoints. Image is the visible label for the
image-generation capability. Configuration remains loopback-only. Loopback
Chat may expose page-local automatic and separate explicit External Information
surfaces.
Chat is memory-only and shows discreet attribution per assistant response. The
page provides accessible active feedback for each request. One explicitly
selected Summarize or Classify file is read locally with strict UTF-8 decoding
and populates that view's editable text area; the current textarea value is
submitted through the existing JSON text contract. Classify labels remain in
displayed order, and no multipart data or filename is submitted. This loopback
page is not a dashboard, operator inspection surface, or compatibility
interface; its authority remains distinct from the trusted-LAN browser.
The page is attached to the loopback-only native authority. Receiver activation
does not add browser routes to its trusted-LAN receiver authority. Separately,
an operator may use hac local --lan-browser-host <LAN_IP> for the
capability-only trusted-LAN browser. It is a distinct listener and authority:
it exposes Chat, Code, Image, Summarize, and Classify, but excludes
Configuration, Workspace, and External Information authority. It is plain HTTP
with no TLS or client authentication and is appropriate only where reachable
peers and the network path are trusted.
5. Send one native request
Replace <OPERATOR_SUPPLIED_MESSAGE> at invocation time. Do not retain the
supplied prompt or generated response in documentation or proof records.
hac chat --message "<OPERATOR_SUPPLIED_MESSAGE>"
This is the ordinary one-shot native client of the already running process. A
successful result includes cluster-owned node attribution. The native endpoint
remains POST /v1/chat for lower-level use when needed.
To summarize one bounded supplied text through that same process, use the ordinary root client:
hac summarize --text "<OPERATOR_SUPPLIED_TEXT>"
The client also accepts one bounded UTF-8 source from stdin
when no explicit source is supplied, or one bounded strict-UTF-8 regular file
through --file <PATH>. --text and --file are mutually exclusive; either
explicit source ignores stdin. The client does not start or inspect the process.
To classify one bounded source against one ordered operator-supplied label set, use the ordinary root client:
hac classify --text "The invoice is due tomorrow." --label invoice --label personal
classify uses the same bounded source selection as summarize: --text,
--file, or stdin; explicit --text and --file are mutually exclusive and
ignore stdin. Labels use repeated ordered --label options. The result contains
one exact selected label and cluster-owned node attribution; the executor, not
the adapter, validates exact membership. See the command reference for bounds
and output forms.
To request bounded textual code assistance, use the ordinary root client:
hac code --message "<OPERATOR_SUPPLIED_CODE_REQUEST>"
code is explicit: the client constructs capability=code, accepts exactly
one non-blank message, and limits its aggregate message content to 65,536 UTF-8
bytes without truncation. It is topology-blind and uses the existing native
POST /v1/chat path, timeout, and output conventions. Its generated text is
not executed and grants no filesystem, repository, shell, Git, testing, tool,
agent, or execution authority.
For one slow-but-valid ordinary request, chat, code, summarize, and classify accept one
per-invocation --timeout-seconds SECONDS value, for example:
hac summarize --timeout-seconds 300 --text "<OPERATOR_SUPPLIED_TEXT>"
SECONDS is a base-10 integer from 1 through 3600; omission keeps the
120-second default. The value is the one-shot HTTP client's
pool/connect/write/read scalar timeout, not a total deadline. It adds no retry
or cancellation and does not change routing, remote, receiver, or runtime
timeout ownership. A timeout does not prove work has stopped elsewhere; avoid
immediately repeating a timed-out command on slow hardware unless additional
work is acceptable.
6. Stop manually
Stop the ordinary application with normal process interruption. Leave the external runtime running or stop it manually according to operator policy.
Mode 2: Ordinary explicit static multi-node operation
Roles:
- receiving machine or machines: each runs an ordinary Home AI Cluster application plus an externally owned local runtime;
- calling machine: ordinary static multi-node process with one local node and one or more explicitly declared remote nodes.
The calling machine and every receiving machine must use compatible published HAC versions and remain on the same trusted LAN for the first reproduction. The existing one-receiving-machine path remains the simple supported special case.
1. Prepare the calling machine and every receiving machine
Install the current published HAC package on the calling machine and every receiving machine. Confirm compatible published versions. On every receiving machine, prepare and start the external runtime and ensure the required model is locally available. Home AI Cluster does not own that runtime.
2. Run static preflight and health on every receiving machine
On every receiving machine:
hac preflight
hac health
Preflight checks local static declaration coherence. Health observes each receiving machine's configured local runtime adapter. Neither result proves LAN reachability from the calling machine.
3. Start each receiving application
On every receiving machine represented by the declaration:
hac local --receiver-host <RECEIVER_ADDRESS>
This starts one foreground HAC process: its ordinary native/local authority
remains on 127.0.0.1:25042, and the explicit receiver authority binds
<RECEIVER_ADDRESS>:25042. Use --receiver-port <PORT> only when the receiver
needs a different port; --port controls only the native/local listener.
<RECEIVER_ADDRESS> must be a concrete non-loopback, non-wildcard IP address;
there is no 0.0.0.0 default or hostname resolution. The receiver exposes only
RFC-0109's internal request and status routes, remains unauthenticated plain
HTTP under the trusted-LAN assumption, and does not provide authentication or
confidential transport. Restrict any firewall allowance to the trusted LAN and
remove it after use.
4. Select or create one saved declaration file
On the calling machine, select or create one explicit, operator-owned declaration at a stable local path:
DECLARATION="<DECLARATION_PATH>"
This shell variable is ordinary shell convenience, not a Home AI Cluster contract. The project defines no default path, the CLI does not discover this file, and the operator supplies its path explicitly. Do not treat the declaration as disposable proof material, and do not commit private addresses or machine-specific values.
For one remote, use these root keys:
remote_node_id = "<DECLARED_REMOTE_NODE_ID>"
remote_base_url = "http://<RECEIVER_ADDRESS>:25042"
local_capabilities = ["chat"]
remote_capabilities = ["chat", "summarize"]
For multiple remotes, use ordered tables:
[[remote_nodes]]
node_id = "<DECLARED_REMOTE_NODE_A_ID>"
base_url = "http://<RECEIVER_A_ADDRESS>:25042"
capabilities = ["chat"]
[[remote_nodes]]
node_id = "<DECLARED_REMOTE_NODE_B_ID>"
base_url = "http://<RECEIVER_B_ADDRESS>:25042"
capabilities = ["summarize"]
To keep chat caller-local while allowing summarize only on an eligible remote, use this healthy-operation specialization:
local_capabilities = ["chat"]
[[remote_nodes]]
node_id = "summary-remote"
base_url = "http://<RECEIVER_ADDRESS>:<PORT>"
capabilities = ["summarize"]
For one inline remote, repeat the closed capability option as needed:
hac static-cluster \
--remote-node-id <DECLARED_REMOTE_NODE_ID> \
--remote-base-url http://<RECEIVER_ADDRESS>:25042 \
--local-capability chat \
--remote-capability summarize
local_capabilities and --local-capability define caller-local routing
capabilities: they control only which capabilities the caller-side static-cluster
router may consider locally. They do not disable adapters, change runtime health,
remove endpoints, configure hac local, change receiver behavior, verify remote
runtime capability, select a target node, or create scheduling or preference.
The accepted explicit names are chat, summarize, classify, and code.
Omitted local or remote capability fields and options retain only chat plus
summarize; classify and code eligibility are explicit. Explicit sets cannot be empty,
duplicated, or unknown, and order has no priority meaning. Declaration and
inline topology modes are mutually exclusive. Declaration order remains
meaningful for the existing ordered remote behavior; declarations do not probe
remote capability or schedule requests.
The healthy-operation specialization above has this eligibility result:
chat -> local
summarize -> eligible remote
No local failure is needed for summarize because the local candidate is
ineligible. The same caller-local routing capability set is projected by static
preflight, which remains network-free. hac local remains unchanged.
Do not add merging, include files, aliases, schema versions, environment
expansion, lookup precedence, or automatic discovery.
To keep the caller-local node eligible for chat and summarize while making an eligible declared remote handle classification directly, use this specialization:
local_capabilities = ["chat", "summarize"]
[[remote_nodes]]
node_id = "classification-remote"
base_url = "http://<RECEIVER_ADDRESS>:25042"
capabilities = ["classify"]
For a classify request, the caller-local candidate is ineligible and the
declared remote is eligible, so capability-centered remote selection occurs
without local runtime failure. This is distinct from the accepted bounded
fallback, which applies only when an eligible local candidate becomes
unavailable before transmission.
5. Run static preflight
On the calling machine:
hac preflight --declaration "$DECLARATION"
This validates static declaration coherence and performs no remote network observation. Run it before status or startup. An unknown key is an invalid declaration; compare the saved declaration file with the accepted single-remote or multi-remote shape rather than reconstructing its schema from memory.
For the equivalent inline topology, provide the same complete inline pair and capability values to preflight; it projects the caller-local routing capability set without network activity:
hac preflight \
--remote-node-id <DECLARED_REMOTE_NODE_ID> \
--remote-base-url http://<RECEIVER_ADDRESS>:25042 \
--local-capability chat \
--remote-capability summarize
6. Inspect the declared static cluster
Run one finite, read-only inspection from the calling machine:
hac status --declaration "$DECLARATION"
The command validates the declaration before local or remote observation. Coherent declaration validation does not prove live reachability. Status reports separate local runtime status, remote application reachability, and remote runtime availability. The fixed local node is first; each declared remote is observed in declaration order. This operation does not start or stop runtimes, repair services, mutate declarations, poll, or watch.
Status is human-readable by default. Use
hac status --declaration "$DECLARATION" --json when
automation needs the compact structured result. unreachable, request-failed,
invalid-response, unavailable, observation-failed, and unknown are
normalized result data in a completed status result, not whole-command failures.
If a receiver is unreachable, first check the saved declaration file and receiving process. Do not interpret that result automatically as a network fault. Correct a wrong address or stale operator value in the saved declaration file, rerun preflight, then rerun status; do not delete and recreate the declaration merely to repeat the workflow.
7. Optionally observe the calling machine's local runtime
hac health
This remains local health only; it does not inspect the remote node. It matters because ordinary routing is local-first, so a usable local path normally wins.
8. Start the ordinary static multi-node process
On the calling machine:
hac static-cluster --declaration "$DECLARATION"
It binds the calling machine's native endpoint to:
http://127.0.0.1:25042/v1/chat
The same process also exposes the separate native /v1/summarize and
/v1/classify endpoints. Its existing /v1/chat ordered-message endpoint carries ordinary
chat and explicit code; no /v1/code endpoint exists.
The process owns only its HTTP client and application lifecycle; it does not start, stop, supervise, repair, or discover the remote machine or runtime.
9. Send one ordinary request
hac chat --message "<OPERATOR_SUPPLIED_MESSAGE>"
The same process also accepts an explicit bounded textual code request through
hac code --message "<OPERATOR_SUPPLIED_CODE_REQUEST>". A declared code-capable
node can be selected only when code is explicit in its static declaration;
omission remains exactly chat plus summarize. This does not expand the
Chat-only OpenAI-compatible process.
A usable eligible local candidate has precedence. A declared remote can be selected directly when the local candidate is ineligible for the requested capability, as in the classification specialization above. Separately, the accepted narrow fallback can select the next eligible candidate only when an eligible local runtime fails before transmission with the accepted connection-unavailable condition. There is no direct node targeting, retry loop, balancing, scoring, scheduling, or discovery. A declared remote node does not guarantee that the first request uses the remote path. Do not retain the supplied message or generated response in proof records.
10. Stop in canonical order
- stop the calling static multi-node process;
- stop the receiving ordinary application;
- remove any temporary firewall allowance;
- leave or stop external runtimes manually according to operator policy.
The calling process and each receiving process remain foreground-bound. The measured one-receiver exercise used a maximum of three simultaneously used terminals: one receiver process, one caller process, and another available caller terminal for finite inspection and the request. That is an observation, not a universal terminal requirement. Normal process interruption is the current stop mechanism; no PID file, detached mode, stop command, supervision, or service manager is implied.
The retained daily workflow evidence records one successful native two-machine exercise. Its only measured recovery event was an invalid declaration; repeated same-day creation and deletion of declarations was broader workflow friction. Foreground operation and normal interruption were sufficient in that exercise, without establishing production readiness or eliminating possible future lifecycle needs.
Mode 3: Explicit historical two-machine proof operation
This preserves the historical distributed architecture proof. It is not the ordinary static multi-node operating mode.
The home-ai-cluster-static-proof launcher was retired by accepted RFC-0075.
The command below is retained as historical evidence, not a current installation
instruction. Exact reproduction requires the historical repository revision
that still installed it. Use the ordinary hac static-cluster
workflow for current operation.
uv run home-ai-cluster-static-proof http://<receiving-lan-address>:8000
It used explicit declared-remote-only selection for the historical proof. For
the detailed historical runbook, see docs/static-two-machine-proof.md.
Mode comparison
| Mode | Calling process | Selection behavior | Preflight | Intended use |
|---|---|---|---|---|
| Local-only | ordinary app | local only | local declarations | normal simplest use |
| Static multi-node | hac static-cluster |
local-first, narrow fallback | local + declared remote static declarations | ordinary explicit two-node operation |
| Historical proof | retired launcher; historical revision only | declared remote only | ordinary local preflight only unless separately invoked | retained architecture evidence |
Failure-layer lookup
Successful preflight does not imply runtime or network success.
| Layer | Owning surface |
|---|---|
| Static declaration coherence | hac preflight |
| Local runtime health | hac health |
| Declared local and remote live observations | hac status --declaration <path> |
| Process startup and port conflict | Invoked process and operating system |
| Trusted-LAN reachability | Explicit trusted-LAN request |
| Receiving endpoint availability | Receiving application and explicit request |
| Routing and fallback execution | Existing request and explanation surfaces |
| Optional request history | History inspection and clearing commands |
Do not reinterpret one layer's failure as another layer's result.
Process and port ownership
| Process | Purpose | Accepted port and exposure | Ownership |
|---|---|---|---|
| External AI runtime | Model execution | Runtime-specific | Operator-owned |
| Ordinary Home AI Cluster application | Native local or receiving endpoint | 25042; loopback by default, trusted-LAN bind only when explicitly started that way |
Home AI Cluster process, manually started |
| Static multi-node process | Calling-machine ordinary multi-node endpoint | 25042 on the calling machine loopback |
Home AI Cluster process, manually started |
| Historical static proof process | Historical calling-machine proof endpoint | 8000 on the calling machine loopback |
Historical repository revision only |
| OpenAI-compatible process | Optional compatibility access | 8001; loopback only |
Separate optional Home AI Cluster process |
This table does not imply supervision or automatic lifecycle management.
Recovery guidance
Use only supported manual actions:
- for an unknown declaration key, compare the saved declaration file with the accepted single-remote or multi-remote shape before rerunning preflight;
- for a wrong address or stale operator value, correct the saved declaration file, rerun preflight, then rerun status;
- do not delete and recreate a declaration merely to rerun the workflow;
- do not interpret an unreachable receiver as a network fault before checking the saved declaration file and receiving process;
- start or repair the external runtime before rerunning health;
- ensure the required model is locally available;
- stop a conflicting process when an accepted fixed port is occupied;
- rerun the failed inspection step before repeating a request;
- stop Home AI Cluster processes with normal process interruption;
- clear optional request history explicitly when desired.
Do not infer automatic repair, retries, service restart, remote shutdown, configuration mutation, or process supervision from this workflow.
Declaration and lifecycle boundary
Declaration placement remains operator-owned. The CLI accepts an explicit path; it does not automatically discover declarations, define a project default path, apply lookup precedence, or merge configuration. Declarations must not contain secret values. They do not grant remote lifecycle authority, and Home AI Cluster does not own external runtimes. Any future project-defined declaration location, automatic lookup behavior, or changed lifecycle surface requires an RFC.
Privacy boundary
Do not retain in repository documentation or proof records real private LAN addresses, prompts, generated responses, authorization values, credentials, filesystem paths, raw exceptions, machine names, hardware details, personal account details, or secrets. Use placeholders for operator-specific values.
Detailed references
README.mddocs/static-two-machine-proof.mddocs/phase-8-ordinary-static-multi-node-proof.mdRFC/RFC-0036-static-operator-preflight.mdRFC/RFC-0037-canonical-operator-workflow.mdRFC/RFC-0038-ordinary-static-multi-node-mode.mdRFC/RFC-0041-explicit-static-cluster-status.mddocs/daily-operator-workflow-evidence-result.md
This document remains the canonical shortest operator sequence.