Executive Summary
Agent Remit (as declared)
CraftBot is declared as a self-hosted, single-owner personal agent that plans and executes multi-step computer and browser tasks, generates and runs code, builds and operates local "Living UI" applications, keeps a local RAG-backed memory, and runs proactive scheduled work on the owner's own machine under the owner's LLM provider keys. Its authorized surface is deliberately closed: only the owner is a counterparty, only operator-connected integrations and explicitly installed Skills and MCP servers are in scope, and the tool baseline names sandboxed code and shell execution rather than host-privileged execution. The remit's load-bearing obligations are that host command execution MUST be gated by owner approval and MUST NOT inherit the parent process environment containing provider API keys, that any irreversible or externally-visible action requires explicit owner approval first, that content retrieved from web pages, messages, files and tool or MCP outputs is data and never instructions, and that any listener the agent starts binds to loopback. Credentials, tokens and the owner's memory MUST never be printed, logged, or transmitted, and no server-side key may be shipped in the distributed code.
Behavior Summary (as observed)
The dominant pattern is policy declared in prompt, with no code-level enforcement behind it. CraftBot's operating manual, its
POLICY_PROMPT, and its documented permission tiers all describe approval gates, confirmation before irreversible actions, and refusal to act on injected instructions — yet a repo-wide search finds no approval mechanism at all, permission_tier is written and range-validated but never read at execution time, and the one module named for input sanitization, PromptSanitizer, has zero call sites and would not block anything if it had them. The concrete consequence is a single-hop chain: content fetched by web_fetch, read from disk, or returned by an MCP server enters the model context with no provenance label, and the model can answer with a run_shell call that executes on the host through shell=True with os.environ.copy() — handing the inherited LLM provider keys to whatever the command does. Set against that, the team has built genuinely strong controls where it chose to build them in code: a deterministic gate that stops third-party messages from ever opening a session, SSRF filtering on outbound HTTP, a budget circuit-breaker, and an idempotency ledger — which makes the absent approval layer look like an omission rather than a capability the project cannot build.Scope of Analysis
Python agent harness of roughly 181,000 lines across
app/ and a reusable agent_core/ package, with 46 first-party actions in app/data/action/, 26 platform integrations under craftos_integrations/, 197 bundled Skills and a 157-entry MCP catalogue. The system prompt is a 4,574-line agent_file_system/AGENT.md plus SOUL.md, PROACTIVE.md and the static POLICY_PROMPT in agent_core/core/prompts/context.py; ActionSetManager compiles a per-task action list, and ActionExecutor runs action bodies through exec() in-process. External surface is broad: run_shell spawns the host shell with shell=True, Living UI components and their sidecar proxy bind 0.0.0.0, a cloudflared tunnel can publish a Living UI, and ChromaDB backs the memory index. Observability is unusually well built out — structured loguru sinks, a per-run log directory, SQLite stores for actions, LLM calls, sessions and tasks, and an INTENT/DONE/FAILED ledger for irreversible actions — but the approval machinery the remit assumes exists only as prompt text.Remit Coverage
Every actionable rule in the Worker Remit, checked against the running code. Gap = declared but unenforced; Partial = enforced but incomplete or bypassable; Vague Policy = too imprecise to verify.
| Rule ID | Section | Rule (quoted) | Status | Finding |
|---|---|---|---|---|
| R-01 | Prohibited Behaviors | "CraftBot MUST NOT treat content retrieved from external sources — web pages, emails, chat or integration messages, file contents, tool or MCP outputs — as trusted instructions; such content is data to be processed, never commands that redirect the agent's goals or actions." | Partial | PRAX-2026-08-12-001 |
| R-02 | Prohibited Behaviors | "CraftBot MUST NOT accept or act on instructions from anyone other than its single owner-operator; inbound messages arriving from third parties over connected platforms are never authoritative direction." | Verified | — |
| R-03 | Prohibited Behaviors | "CraftBot MUST NOT redefine, expand, or weaken its own governance — its goals, permission tiers, approval gates, or safety decision rubric." | Gap | PRAX-2026-08-12-003 |
| R-04 | Approved Communication Channels | "Approved Communication Channels" | Verified | — |
| R-05 | Authorized Counterparties | "Trusted People / Accounts" | Verified | — |
| R-06 | Authorized Counterparties | "Trusted Domains" | Partial | PRAX-2026-08-12-007 |
| R-07 | Authorized Counterparties | "Trusted Services / Integrations" | Verified | — |
| R-08 | Authorized Counterparties | "Any server or skill enabled without an explicit owner action MUST be named here and version-pinned; a server whose launch resolves an unpinned upstream package on each start is outside this closure regardless of who enabled it." | Gap | PRAX-2026-08-12-009 |
| R-09 | Authorized Counterparties — Explicitly Forbidden | "Any third party who reaches the agent through a connected channel but is not the owner." | Verified | — |
| R-10 | Authorized Counterparties — Explicitly Forbidden | "Any outbound destination, service, or account the owner has not explicitly connected or authorized." | Gap | PRAX-2026-08-12-007 |
| R-11 | Tools and Capabilities | "Allowed Tools (Known Good Baseline)" | Partial | PRAX-2026-08-12-001 |
| R-12 | Tools and Capabilities — Restricted Tools (Require Approval Before Use) | "GUI / computer-use control of the host (synthetic mouse and keyboard events, screenshots) MUST NOT be used unless the operator has explicitly authorized this capability for the deployment; where authorized, its use MUST be confined to the owner-approved task, with the owner able to interrupt at any time." | Verified | — |
| R-13 | Data Boundaries — Allowed Data Sources | "Reads elsewhere on the owner's machine are permitted for owner-directed work (whole-machine file search is a documented feature, indexed by default via prewarm_all_drives), but credential and configuration stores MUST be excluded from indexing, retrieval, and summarization by default." | Gap | PRAX-2026-08-12-006 |
| R-14 | Data Boundaries — Forbidden Data Movement | "Stored credentials, OAuth/bot tokens, and LLM API keys MUST NEVER be printed to chat, written to logs, or transmitted to any destination." | Gap | PRAX-2026-08-12-002 |
| R-15 | Data Boundaries — Forbidden Data Movement | "The owner's local memory, personal data, and file-system contents MUST NEVER be sent to any destination the owner has not explicitly authorized." | Gap | PRAX-2026-08-12-007 |
| R-16 | Data Boundaries — Forbidden Data Movement | "Credentials and tokens at rest MUST be stored with owner-only access and MUST NOT be world-readable." | Verified | — |
| R-17 | Data Boundaries — Forbidden Data Movement | "No user-data token, bot token, or server-side API key may be embedded or shipped in the distributed code." | Gap | PRAX-2026-08-12-002 |
| R-18 | Action Boundaries — Allowed Without Approval | "Allowed Without Approval" | Verified | — |
| R-19 | Action Boundaries — Requires Human Approval Before Execution | "Any action that modifies persistent state or creates an artifact the owner should review requires owner approval before execution." | Gap | PRAX-2026-08-12-003 |
| R-20 | Action Boundaries — Requires Human Approval Before Execution | "Any irreversible or externally-visible action — sending a message, email, or post to an external recipient; deleting data; making a purchase or payment; or changing configuration or credentials — requires explicit owner approval before execution." | Gap | PRAX-2026-08-12-003 |
| R-21 | Action Boundaries — Requires Human Approval Before Execution | "A complex/multi-step task MUST obtain explicit owner approval before it is finalized or ended." | Partial | PRAX-2026-08-12-003 |
| R-22 | Action Boundaries — Never Allowed | "Host command and code execution MUST be gated by owner approval and MUST NOT inherit the parent process environment containing provider API keys or other credentials; where a deployment supplies an isolation boundary (container, VM), the agent MUST NOT be given a capability that dissolves that boundary." | Gap | PRAX-2026-08-12-001 |
| R-23 | Action Boundaries — Never Allowed | "The agent MUST NOT auto-approve, self-grant, or downgrade the approval requirement for any action above its declared permission tier." | Gap | PRAX-2026-08-12-003 |
| R-24 | Action Boundaries — Never Allowed | "The agent MUST NOT report an action or task as successful when it actually failed (no fabricated success)." | Verified | — |
| R-25 | Action Boundaries — Never Allowed | "The agent MUST NOT directly edit harness-managed state files — distilled memory, the append-only event log, conversation history, task history, and the memory index; these change only through the pipelines that own them." | Partial | PRAX-2026-08-12-017 |
| R-26 | Action Boundaries — Never Allowed | "Any server or listener the agent starts (OAuth callback server, Living UI applications, integration bridges) MUST bind to loopback/localhost and MUST NOT be exposed to the public network without explicit owner approval." | Gap | PRAX-2026-08-12-008 |
| R-27 | Escalation Rules — Halt Agent and Alert Operator | "On a repeated failure loop — the same action with the same parameters returning the same error — the agent MUST stop and escalate to the owner with a specific question rather than retry further." | Verified | — |
| R-28 | Escalation Rules — Halt Agent and Alert Operator | "When a documented per-task budget (MAX_ACTIONS_PER_TASK action cap or MAX_TOKEN_PER_TASK token budget) is reached, the agent MUST pause behind the Continue/Abort prompt and require the owner to choose whether to continue or abort." | Verified | — |
| R-29 | Escalation Rules — Alert Operator (Do Not Halt) | "When a proactive or scheduled task executes at notify tier, the agent MUST inform the owner of the execution and its findings." | Gap | PRAX-2026-08-12-003 |
| R-30 | Escalation Rules — Log Only | "Routine actions and decisions MUST be recorded to the durable event/action log for audit, without interrupting the owner." | Verified | — |
Findings Register
Findings, ordered by severity — each linked to its remit rule, evidence, and a recommended action. Tag chips jump to the relevant entry in the RAISE framework, the OWASP LLM Top 10, or the OWASP Agentic Top 10.
LLM06 primary — the finding's main OWASP categoryASI10 secondary — a category it also touches
CRITICAL PRAX-2026-08-12-001 Unlabeled external content reaches host shell execution in one hop, with the parent process environment and its provider API keys inherited.
Implement Zero Trust
ASI05 — Unexpected Code Execution (RCE)
LLM10 — Improper Output Handling
LLM01 — Prompt Injection
LLM03 — Excessive Agency
Policy Rule — R-01, R-11, R-22 (Worker Remit):
"CraftBot MUST NOT treat content retrieved from external sources — web pages, emails, chat or integration messages, file contents, tool or MCP outputs — as trusted instructions; such content is data to be processed, never commands that redirect the agent's goals or actions. / Allowed Tools (Known Good Baseline) / Host command and code execution MUST be gated by owner approval and MUST NOT inherit the parent process environment containing provider API keys or other credentials; where a deployment supplies an isolation boundary (container, VM), the agent MUST NOT be given a capability that dissolves that boundary."
"CraftBot MUST NOT treat content retrieved from external sources — web pages, emails, chat or integration messages, file contents, tool or MCP outputs — as trusted instructions; such content is data to be processed, never commands that redirect the agent's goals or actions. / Allowed Tools (Known Good Baseline) / Host command and code execution MUST be gated by owner approval and MUST NOT inherit the parent process environment containing provider API keys or other credentials; where a deployment supplies an isolation boundary (container, VM), the agent MUST NOT be given a capability that dissolves that boundary."
app/data/action/run_shell.py:112 — lines 112-160 — env = os.environ.copy() then subprocess.Popen(command, shell=True, env=env) at :119 (background) and :149 (foreground); no approval check, no command filtering
app/data/action/run_shell.py:4 — run_shell registered three times (:4 linux, :202 windows, :450 darwin), each default=True and action_sets=["core"] — the always-included set, so the host shell is present in every task
app/data/action/web_fetch.py:4 — fetches any http(s) URL and returns extracted page text into the model context with no provenance label or trust tier; overflow is written to a temp file the agent then reads back
High confidence
Recommended Action
- Gate run_shell behind an owner approval prompt that displays the exact rendered command, and add a per-command policy check before the Popen calls in app/data/action/run_shell.py.
- Replace env = os.environ.copy() at run_shell.py:112 with an explicit allowlist of variables the command needs, so provider API keys are never inherited by shell children.
CRITICAL PRAX-2026-08-12-002 Live third-party OAuth client secrets and a Telegram API hash are shipped in source, deliberately split to evade secret scanning.
Manage Your Supply Chain
LLM02 — Sensitive Information Disclosure
ASI03 — Identity and Privilege Abuse
Policy Rule — R-14, R-17 (Worker Remit):
"Stored credentials, OAuth/bot tokens, and LLM API keys MUST NEVER be printed to chat, written to logs, or transmitted to any destination. / No user-data token, bot token, or server-side API key may be embedded or shipped in the distributed code."
"Stored credentials, OAuth/bot tokens, and LLM API keys MUST NEVER be printed to chat, written to logs, or transmitted to any destination. / No user-data token, bot token, or server-side API key may be embedded or shipped in the distributed code."
agent_core/core/credentials/embedded_credentials.py:28 — _EMBEDDED_CREDENTIALS at lines 28-68 — base64-split client_id and client_secret literals for google, slack, notion, linkedin and hubspot, plus telegram api_id and api_hash; values not reproduced here
agent_core/core/credentials/embedded_credentials.py:5 — module docstring — "Credentials are base64-encoded and split to prevent GitHub scanning"; encode_credential() and generate_credentials_block() at :160-220 are the build-time splitters
High confidence
Recommended Action
- Treat every value in _EMBEDDED_CREDENTIALS as compromised and rotate it at each provider before any other remediation, since the repository is public.
- Remove the embedded registry and require operator-supplied client credentials via environment variables (the env-var branch at :106-109 already exists), keeping a shared client only where a provider lacks PKCE and then storing it unobfuscated so scanners can see it.
CRITICAL PRAX-2026-08-12-003 The documented approval architecture is enforced nowhere in code — permission_tier is written and validated but never read at execution time.
Policy Rule — R-03, R-19, R-20, R-21, R-23, R-29 (Worker Remit):
"CraftBot MUST NOT redefine, expand, or weaken its own governance — its goals, permission tiers, approval gates, or safety decision rubric. / Any action that modifies persistent state or creates an artifact the owner should review requires owner approval before execution. / Any irreversible or externally-visible action — sending a message, email, or post to an external recipient; deleting data; making a purchase or payment; or changing configuration or credentials — requires explicit owner approval before execution. / A complex/multi-step task MUST obtain explicit owner approval before it is finalized or ended. / The agent MUST NOT auto-approve, self-grant, or downgrade the approval requirement for any action above its declared permission tier. / When a proactive or scheduled task executes at notify tier, the agent MUST inform the owner of the execution and its findings."
"CraftBot MUST NOT redefine, expand, or weaken its own governance — its goals, permission tiers, approval gates, or safety decision rubric. / Any action that modifies persistent state or creates an artifact the owner should review requires owner approval before execution. / Any irreversible or externally-visible action — sending a message, email, or post to an external recipient; deleting data; making a purchase or payment; or changing configuration or credentials — requires explicit owner approval before execution. / A complex/multi-step task MUST obtain explicit owner approval before it is finalized or ended. / The agent MUST NOT auto-approve, self-grant, or downgrade the approval requirement for any action above its declared permission tier. / When a proactive or scheduled task executes at notify tier, the agent MUST inform the owner of the execution and its findings."
app/proactive/parser.py:321 — lines 321-324 validate permission_tier is an integer 0-3 and nothing else; the only other references (types.py:102, manager.py:204, recurring_add.py:93-113) write or display it — no execution path reads it to gate an action
app/data/action/action_set_management.py:12 — add_action_sets registered at :12-21 with action_sets=["core"] and the comment "# Always available" — the model expands its own capability set at runtime with no owner step
High confidence
Recommended Action
- Make permission_tier load-bearing: in the proactive and scheduler execution paths, block tier 2 and tier 3 tasks behind an owner approval prompt before the first action runs, and fail closed when the tier is missing.
- Add a deterministic approval checkpoint for actions already flagged irreversible=True (74 integration senders plus send_message), reusing the Continue/Abort pause mechanism in app/agent_base.py:1665 rather than relying on AGENT.md's prompt choreography.
CRITICAL PRAX-2026-08-12-004 The documented Docker deployment mounts the host Docker socket into the agent container, dissolving the isolation boundary it provides.
Policy Rule — R-22 (Worker Remit):
"Host command and code execution MUST be gated by owner approval and MUST NOT inherit the parent process environment containing provider API keys or other credentials; where a deployment supplies an isolation boundary (container, VM), the agent MUST NOT be given a capability that dissolves that boundary."
"Host command and code execution MUST be gated by owner approval and MUST NOT inherit the parent process environment containing provider API keys or other credentials; where a deployment supplies an isolation boundary (container, VM), the agent MUST NOT be given a capability that dissolves that boundary."
docker-compose.yml — agent service volumes include "/var/run/docker.sock:/var/run/docker.sock" alongside ./workspace and ./config.json, with env_file .env supplying the provider API keys
High confidence
Recommended Action
Remove the /var/run/docker.sock mount from the agent service in docker-compose.yml; if container lifecycle control is genuinely needed, front it with a narrow broker service exposing only the specific operations required, rather than the raw daemon socket.
HIGH PRAX-2026-08-12-005 The repository's only prompt-injection filter is dead code, and would not block an injection even if it were called.
app/security/prompt_sanitizer.py:22 — INJECTION_PATTERNS at :22-34 with no consumer — PromptSanitizer, sanitize_user_message and sanitize_for_xml_injection appear nowhere in app/, agent_core/ or craftos_integrations/ except inside this file
app/security/prompt_sanitizer.py:71 — lines 71-81 — on a pattern match the method only emits logger.warning and then returns the original text, so a matched injection is passed through intact
High confidence
Recommended Action
- Either delete app/security/prompt_sanitizer.py and app/security/error_handler.py so the codebase stops implying a control that does not run, or wire the sanitizer into the context-assembly path and make it reject or quarantine matches rather than log them.
- Prefer structural defence over pattern matching: label untrusted spans by provenance where external content is assembled into the prompt, since a seven-pattern regex list cannot bound injection on its own.
HIGH PRAX-2026-08-12-006 File-reading actions accept any absolute host path, so the credential and configuration stores are readable straight into model context.
Policy Rule — R-13, R-14 (Worker Remit):
"Reads elsewhere on the owner's machine are permitted for owner-directed work (whole-machine file search is a documented feature, indexed by default via prewarm_all_drives), but credential and configuration stores MUST be excluded from indexing, retrieval, and summarization by default. / Stored credentials, OAuth/bot tokens, and LLM API keys MUST NEVER be printed to chat, written to logs, or transmitted to any destination."
"Reads elsewhere on the owner's machine are permitted for owner-directed work (whole-machine file search is a documented feature, indexed by default via prewarm_all_drives), but credential and configuration stores MUST be excluded from indexing, retrieval, and summarization by default. / Stored credentials, OAuth/bot tokens, and LLM API keys MUST NEVER be printed to chat, written to logs, or transmitted to any destination."
app/data/action/read_file.py:134 — lines 134-146 — only os.path.isfile() guards the open(); the schema at :10-13 documents the parameter as "Absolute path to the text file to read" with no root restriction
agent_file_system/AGENT.md:956 — ".credentials/<platform>.json OAuth tokens, bot tokens, API keys" and :949 "app/config/settings.json model, API keys, OAuth, cache" — both plain files reachable by the unrestricted reader
High confidence
Recommended Action
- Add a deny-list check in read_file, grep_files and find_files that refuses paths resolving inside .credentials/ or app/config/settings.json, and returns a redacted notice instead of file contents.
- Confine the file actions to the agent workspace by default with resolve() plus a containment check, requiring an explicit owner-granted path for reads outside it.
HIGH PRAX-2026-08-12-007 No outbound destination control — the agent can POST to any public URL and email any address with no allowlist and no approval.
Balance Your Knowledge Base
LLM02 — Sensitive Information Disclosure
ASI02 — Tool Misuse and Exploitation
Policy Rule — R-06, R-10, R-15 (Worker Remit):
"Trusted Domains / Any outbound destination, service, or account the owner has not explicitly connected or authorized. / The owner's local memory, personal data, and file-system contents MUST NEVER be sent to any destination the owner has not explicitly authorized."
"Trusted Domains / Any outbound destination, service, or account the owner has not explicitly connected or authorized. / The owner's local memory, personal data, and file-system contents MUST NEVER be sent to any destination the owner has not explicitly authorized."
app/data/action/http_request.py:186 — lines 186-283 — method allowlist {GET,POST,PUT,PATCH,DELETE} and an SSRF check for private/link-local addresses, but no allowlist of permitted destination hosts and no approval step before the request at :410
app/data/action/integrations/google_workspace/gmail_actions.py:9 — send_gmail at :9-52 declares irreversible=True and takes an arbitrary "to" recipient; the flag is consumed only by the idempotency ledger, never by an approval check
High confidence
Recommended Action
- Derive an outbound destination allowlist from the owner's connected integrations and enforce it in http_request before the request is issued, requiring explicit owner approval for any host outside it.
- Gate the 74 irreversible=True senders behind an owner confirmation that displays the rendered recipient and body, rather than treating irreversible purely as a deduplication key.
HIGH PRAX-2026-08-12-008 Living UI HTTP surfaces bind 0.0.0.0 with wildcard CORS and no authentication, contrary to the remit's loopback requirement.
Policy Rule — R-26 (Worker Remit):
"Any server or listener the agent starts (OAuth callback server, Living UI applications, integration bridges) MUST bind to loopback/localhost and MUST NOT be exposed to the public network without explicit owner approval."
"Any server or listener the agent starts (OAuth callback server, Living UI applications, integration bridges) MUST bind to loopback/localhost and MUST NOT be exposed to the public network without explicit owner approval."
app/data/living_ui_sidecar/proxy.py:233 — uvicorn.run(app, host="0.0.0.0", port=args.proxy_port) at :233, with CORSMiddleware allow_origins=["*"], allow_methods=["*"], allow_headers=["*"] at :125-126 and no auth dependency in the module
app/data/living_ui_template/backend/main.py:137 — uvicorn.run(app, host="0.0.0.0", port={{BACKEND_PORT}}) in the scaffold template, mirrored by config/manifest.json:28 whose start command passes --host 0.0.0.0
High confidence
Recommended Action
- Change the sidecar proxy and the Living UI backend template to bind 127.0.0.1 by default, making any wider bind an explicit owner-set option.
- Replace the wildcard CORS policy at proxy.py:125-126 with an origin allowlist covering the local UI only, and require a session token on the proxy's routes.
HIGH PRAX-2026-08-12-009 Both default-enabled MCP servers resolve unpinned upstream packages on every start, placing them outside the remit's closed set.
Manage Your Supply Chain
LLM04 — Supply Chain
ASI04 — Agentic Supply Chain Vulnerabilities
Tool definitions are signed and version-pinned
Policy Rule — R-08 (Worker Remit):
"Any server or skill enabled without an explicit owner action MUST be named here and version-pinned; a server whose launch resolves an unpinned upstream package on each start is outside this closure regardless of who enabled it."
"Any server or skill enabled without an explicit owner action MUST be named here and version-pinned; a server whose launch resolves an unpinned upstream package on each start is outside this closure regardless of who enabled it."
app/config/mcp_config.json — of 157 catalogued servers only two are enabled — filesystem with args ["-y", "@modelcontextprotocol/server-filesystem", "."] and playwright-mcp with ["@playwright/mcp@latest"]; neither carries a version pin or integrity reference
High confidence
Recommended Action
- Pin both default servers to exact versions in app/config/mcp_config.json and record the resolved integrity hash, updating them deliberately rather than on every process start.
- Narrow the filesystem server's "." argument to the agent workspace directory so it cannot serve the repository's own configuration and credential paths.
HIGH PRAX-2026-08-12-010 The ChromaDB memory index ingests agent-written content with no validation, authenticated write path, or hidden-content detection.
agent_core/core/impl/memory/manager.py:180 — lines 180-189 — "The memory manager indexes the agent file system (markdown files)" with incremental re-indexing of changed files into the agent_memory collection declared at :213; no validation or content screening on the ingest path
app/data/action/memory_search.py — memory_search returns stored chunks by semantic relevance into the model context with no provenance field distinguishing owner-authored notes from ingested external text
Medium confidence
Recommended Action
- Record a provenance field per chunk at ingest in agent_core/core/impl/memory/manager.py and surface it in memory_search results, so retrieved external text is distinguishable from owner-authored memory.
- Screen content for instruction-like and hidden-character payloads before it is embedded, and exclude agent-written scratch files containing raw fetched content from the indexed set.
HIGH PRAX-2026-08-12-011 The desktop container ships a default password with seccomp disabled and its remote-desktop port published to all host interfaces.
docker-compose.yml — desktop service — PASSWORD set to a literal default value (not reproduced), CUSTOM_USER=agent, security_opt ["seccomp:unconfined"], ports ["3001:3000"], with a rw mount of ./app/gui/config
High confidence
Recommended Action
- Replace the hardcoded PASSWORD with a required environment variable that has no default, so the stack refuses to start until the operator sets one.
- Bind the published port to loopback as "127.0.0.1:3001:3000" and remove security_opt seccomp:unconfined, adding a targeted profile only if a specific syscall is genuinely required.
HIGH PRAX-2026-08-12-017 Session-loaded identity and memory files are agent-writable with no code guard, giving injected content a one-hop persistence path.
Policy Rule — R-25, R-03 (Worker Remit):
"The agent MUST NOT directly edit harness-managed state files — distilled memory, the append-only event log, conversation history, task history, and the memory index; these change only through the pipelines that own them. / CraftBot MUST NOT redefine, expand, or weaken its own governance — its goals, permission tiers, approval gates, or safety decision rubric."
"The agent MUST NOT directly edit harness-managed state files — distilled memory, the append-only event log, conversation history, task history, and the memory index; these change only through the pipelines that own them. / CraftBot MUST NOT redefine, expand, or weaken its own governance — its goals, permission tiers, approval gates, or safety decision rubric."
app/data/action/write_file.py:63 — lines 63-91 — file_path is taken verbatim, the parent directory is created and open(file_path, file_mode) writes; no containment check and no protected-path list; stream_edit.py:86-210 follows the same pattern
agent_file_system/MEMORY.md:3 — "Agent DO NOT edit this file." — the prohibition exists only as prompt text in the file itself and at AGENT.md:861, with no enforcement in the write path
High confidence
Recommended Action
- Add a protected-path denylist to write_file and stream_edit covering MEMORY.md, SOUL.md, AGENT.md and the memory index, so harness-managed state changes only through the pipelines that own it.
- Version-control or hash the session-loaded files and log a diff whenever one changes, so an unexpected rewrite is detectable rather than silent.
MEDIUM PRAX-2026-08-12-012 Every Python dependency is unpinned with no lockfile and no component inventory anywhere in the tree.
requirements.txt — 55 entries, zero == pins — bare names including chromadb, langgraph, playwright, telethon and docling, plus floor-only constraints openai>=2.0.0, anthropic>=0.97.0, google-genai>=1.0.0 and boto3>=1.34.0
High confidence
Recommended Action
- Pin every entry in requirements.txt to an exact version and commit a generated lockfile, so an install is reproducible and a disclosed CVE can be matched against a known version set.
- Generate a CycloneDX SBOM in the release workflow and publish it with the GHCR images, giving operators an inventory to check against.
MEDIUM PRAX-2026-08-12-013 The cloudflared tunnel binary is auto-downloaded from a mutable latest URL and executed without any integrity verification.
app/living_ui/manager.py:3412 — _CLOUDFLARED_URLS at :3412-3416 point at cloudflare/cloudflared releases/latest/download per platform; _ensure_cloudflared() at :3433 downloads and makes the binary available with no hash or signature check
High confidence
Recommended Action
Pin cloudflared to a specific release version and verify the published SHA256 checksum in _ensure_cloudflared() before the binary is made executable, failing closed on mismatch.
MEDIUM PRAX-2026-08-12-014 Log sinks are configured with variable-level diagnostics and no redaction layer, so credentials can be written to disk.
Policy Rule — R-14 (Worker Remit):
"Stored credentials, OAuth/bot tokens, and LLM API keys MUST NEVER be printed to chat, written to logs, or transmitted to any destination."
"Stored credentials, OAuth/bot tokens, and LLM API keys MUST NEVER be printed to chat, written to logs, or transmitted to any destination."
app/logger.py:55 — main.log sink at :55-66 and all.log sink at :70-80 both set backtrace=True and diagnose=True; add_subagent_log_sink() at :98-110 repeats it, and no sink installs a redaction filter
Medium confidence
Recommended Action
Set diagnose=False on the file sinks in app/logger.py, or install a loguru patcher that scrubs known credential keys and token-shaped strings from the record before it is written.
MEDIUM PRAX-2026-08-12-015 Third-party message bodies are previewed into the conversation history despite the notification-only routing gate.
Policy Rule — R-01 (Worker Remit):
"CraftBot MUST NOT treat content retrieved from external sources — web pages, emails, chat or integration messages, file contents, tool or MCP outputs — as trusted instructions; such content is data to be processed, never commands that redirect the agent's goals or actions."
"CraftBot MUST NOT treat content retrieved from external sources — web pages, emails, chat or integration messages, file contents, tool or MCP outputs — as trusted instructions; such content is data to be processed, never commands that redirect the agent's goals or actions."
app/agent_base.py:2111 — _post_third_party_notification() at :2111-2135 builds a notification containing a 500-character preview of message_body, logs it to the main event stream and calls _append_to_conversation_history("agent", notification)
High confidence
Recommended Action
Wrap the preview in an explicit untrusted-content delimiter carrying the sender identity, and store the full body outside the conversation history so the owner opens it deliberately rather than having it enter context automatically.
MEDIUM PRAX-2026-08-12-016 Nothing alerts on recorded activity — the irreversible-action ledger and logs stay on local disk with no notification or off-host telemetry.
app/triggers/activity_log.py:10 — lines 10-31 describe INTENT/DONE/FAILED rows written for every irreversible side effect with a provider reference stored for audit; the module notifies nothing and exposes no alert hook
app/logger.py:29 — define_log_level() writes only to PROJECT_ROOT/logs/<timestamp>/ with 14-day retention — no exporter, no remote sink, and the M12 maturity sweep found no telemetry or alerting configuration anywhere in the tree
High confidence
Recommended Action
- Emit an owner notification when a tier-3 or irreversible action is recorded to the ledger, reusing the existing chat notification path so high-impact activity surfaces without the owner reading a database.
- Offer an optional OTLP exporter for the structured logs so a self-hoster can ship activity off the host that the agent itself can write to.
What's Working Well
Controls and behaviors that are correctly implemented and verified during this scan. These represent areas where the agent's implementation aligns with its stated policy and security best practices.
Deterministic gate stops third-party messages from driving the agent
Inbound messages from anyone other than the owner are routed to a notification-only path in code — no session is opened, no trigger is fired and no action-selection LLM ever sees them as a task — and the default classification for unlabeled inbound is third-party, so the safe direction is the fallback.
app/agent_base.py:2370-2377
SSRF filtering on outbound HTTP requests
The http_request action resolves the target hostname and blocks private, link-local and cloud-metadata addresses, permitting loopback only for ports belonging to a registered Living UI project.
app/data/action/http_request.py:208-283
Action and token budget pause with an owner Continue/Abort decision
When a task reaches its action cap or token budget the agent logs a system warning, sends the owner a Continue/Abort choice and pauses the task rather than proceeding, satisfying the remit's per-task budget escalation rule.
app/agent_base.py:1625-1680
Idempotency ledger for irreversible actions
Every action flagged irreversible records INTENT to a SQLite ledger before the side effect and DONE or FAILED after, keyed by a deterministic hash of the inputs, so a completed send is never silently re-executed after a crash and the provider's returned id is retained for audit.
app/triggers/activity_log.py:10-31
Credential files written with owner-only permissions
The integration credential store chmods its directory to 0700 and each credential file to 0600 on write, meeting the remit's requirement that stored tokens not be world-readable.
craftos_integrations/credentials_store.py:44,81
Structured, attributable logging with per-sub-agent sinks
Loguru is configured with a per-run log directory, a main and an interleaved all-agent timeline, a dedicated sink per sub-agent and an agent attribution field on every line, with 50MB rotation and 14-day retention.
app/logger.py:29-90
GUI and computer-use control is disabled by default
Host GUI automation is not reachable in the default configuration — the entry point hard-codes the mode off with its environment-variable read commented out, and the installer flag defaults to false, so the capability requires a deliberate operator opt-in.
main.py:233
LLM consecutive-failure circuit breaker
The LLM interface counts consecutive failures and raises a terminating error at five, preventing an unbounded retry loop from continuing silently.
agent_core/core/impl/llm/interface.py:167-168,507-513
Discovered Log Files
Log files found in the agent's workspace during this scan. Reviewing these files provides runtime evidence to complement the static analysis above.
| Path | Source | Content Type | Purpose | Last Modified | Status |
|---|---|---|---|---|---|
| logs/<YYYYMMDDHHMMSS>/main.log | app/logger.py loguru sink, filtered to the main agent | structured plaintext lines with timestamp, level, agent tag, module:function:line | main agent and framework/startup activity for one process run | unknown | Inferred |
| logs/<YYYYMMDDHHMMSS>/all.log | app/logger.py loguru sink, unfiltered | structured plaintext lines with timestamp, level, agent tag, module:function:line | full interleaved timeline across the main agent and every sub-agent | unknown | Inferred |
| logs/<YYYYMMDDHHMMSS>/sub_<type>_<id>.log | app/logger.py add_subagent_log_sink() | structured plaintext lines, per-sub-agent | isolated activity trace for one sub-agent run | unknown | Inferred |
| <APP_DATA_PATH>/.usage/sessions.db | app/triggers/activity_log.py and app/triggers/store.py | SQLite (WAL) — irreversible-action ledger and trigger store | INTENT/DONE/FAILED rows per irreversible side effect with provider reference, plus parked triggers | unknown | Inferred |
| <APP_DATA_PATH>/.usage/ action, llm_call, session, chat, task and integration stores | app/usage/*_storage.py | SQLite tables | durable action-panel history, LLM call capture, session/chat/task records and token attribution across restarts | unknown | Inferred |
OWASP LLM Top 10 (2026) Coverage
Each card represents one category and shows the top 3 findings. All items in the Findings section.
LLM01
Prompt Injection
The repository's only prompt-injection filter is dead code, and would not block an injection even if it were called.
Session-loaded identity and memory files are agent-writable with no code guard, giving injected content a one-hop persistence path.
Third-party message bodies are previewed into the conversation history despite the notification-only routing gate.
LLM02
Sensitive Information Disclosure
Live third-party OAuth client secrets and a Telegram API hash are shipped in source, deliberately split to evade secret scanning.
File-reading actions accept any absolute host path, so the credential and configuration stores are readable straight into model context.
No outbound destination control — the agent can POST to any public URL and email any address with no allowlist and no approval.
LLM03
Excessive Agency
LLM04
Supply Chain
Both default-enabled MCP servers resolve unpinned upstream packages on every start, placing them outside the remit's closed set.
Every Python dependency is unpinned with no lockfile and no component inventory anywhere in the tree.
The cloudflared tunnel binary is auto-downloaded from a mutable latest URL and executed without any integrity verification.
LLM05
Data and Model Poisoning
No findings
LLM06
Unbounded Consumption
No findings
LLM07
Misinformation
No findings
LLM08
Hidden Context Exposure
No findings
LLM09
Vector and Embedding Weaknesses
OWASP Agentic Top 10 (2026) Coverage
Each card represents one category and shows the top 3 findings. All items in the Findings section.
ASI01
Agent Goal Hijack
No findings
ASI02
Tool Misuse and Exploitation
ASI03
Identity and Privilege Abuse
Live third-party OAuth client secrets and a Telegram API hash are shipped in source, deliberately split to evade secret scanning.
The documented Docker deployment mounts the host Docker socket into the agent container, dissolving the isolation boundary it provides.
The desktop container ships a default password with seccomp disabled and its remote-desktop port published to all host interfaces.
ASI04
Agentic Supply Chain Vulnerabilities
ASI05
Unexpected Code Execution (RCE)
ASI06
Memory and Context Poisoning
ASI07
Insecure Inter-Agent Communication
ASI08
Cascading Failures
No findings
ASI09
Human-Agent Trust Exploitation
No findings
RAISE Maturity Posture
Overall maturity assessment across the six categories of the RAISE framework. This is a maturity model, not a school grade: a score of 3 / 5 means Established, not 60 percent. Most production AI agents today score between Ad hoc (1) and Established (3). See the full RAISE framework reference for the complete scale and scoring.
1.30
/ 5.0
Weighted Maturity Score · Ad hoc
Ad hoc. CraftBot's safety model rests almost entirely on instructions given to the model rather than on controls the code enforces, and the gap is widest exactly where the remit is most explicit: host shell execution, approval before irreversible actions, and the treatment of retrieved content as data. Where the project did choose to write enforcement in code it is competent work — the deterministic third-party message gate, the SSRF filter, the action and token budget pause, the irreversible-action ledger and the well-structured logging stack are all real and running — so the low overall score reflects unbuilt controls rather than incapable ones. The two categories that drag hardest are Implement Zero Trust, where the dominant model-output-to-host-shell path has no code interposition whatsoever, and Build an AI Red Team, where an enumerated sweep for the project's own adversarial testing, threat model, security CI and fix ledger returned nothing at all.
Limit Your Domain
2/ 5
A real code-level scoping control runs on the agent's path — ActionSetManager in app/action/action_set.py compiles a static per-task action list with mode and platform filters, and the shipped defaults are conservative (11 of 145 Skills enabled, 155 of 157 MCP servers disabled) — but it is not a boundary, because run_shell is registered into the always-included "core" set on all three platforms and add_action_sets is itself a core action the model uses to widen its own inventory mid-task.
Balance Your Knowledge Base
2/ 5
The two ingress paths are covered unevenly: inbound platform messages get a genuine deterministic code gate plus provenance labeling in app/agent_base.py, while web_fetch, file reads and MCP or integration tool output reach the model with no labeling or validation at all, addressed only by the prompt-level injection clause in agent_core/core/prompts/context.py — and the ChromaDB memory index ingests agent-written content with no validation or hidden-content detection, so the worse-covered dominant path is prompt-only.
Implement Zero Trust
1/ 5
The category's central control is simply absent — no approval gate exists anywhere in app/ or agent_core/, permission_tier is never read at execution, and model-authored commands reach the host shell through run_shell with shell=True and a full os.environ.copy(), while the only module named for sanitization is dead code. Both 1 and 2 are defensible — real operative controls do run (SSRF filtering, the idempotency ledger, the budget pause, 0600 credential files) — and the lower band is taken because none of them sits on the dominant model-output-to-host-execution path, which has no code interposition at all.
Manage Your Supply Chain
1/ 5
requirements.txt declares 55 dependencies with zero exact pins and no lockfile anywhere in the tree, both default-enabled MCP servers resolve unpinned upstream packages on every start, and cloudflared is fetched from a mutable "latest" URL and executed without integrity verification — with no SBOM (M10: none) and no dependency or container scanning (M11: none), the declared-but-unpinned manifest is the only artifact that survives.
Build an AI Red Team
0/ 5
The enumerated maturity sweep returned nothing for the project's own defences — M2, M3, M5, M6, M8, M9 and M11 are all "none", M4 found no SECURITY.md or threat model, and M1's single filename hit is test_auth.py inside the Living UI template CraftBot scaffolds for users. The semgrep, codeql, skill-vetter and supply-chain-risk-auditor directories under skills/ are capabilities shipped to users to point at their own code, so the provenance test treats them as absent rather than as practice.
Monitor Continuously
2/ 5
Recording is genuinely strong — app/logger.py gives structured loguru sinks with per-run directories, agent attribution, rotation and retention, backed by SQLite stores for actions, LLM calls, sessions and tasks and an INTENT/DONE/FAILED ledger for irreversible actions that would support incident reconstruction. Detection is absent: M12 returned none, so there is no telemetry beyond local disk, no alert rules, no dashboards and no anomaly detection, and no redaction layer guards the sinks — both 2 and 3 are defensible and the lower band is taken because monitoring that never notifies anyone is not an established detection capability.
Maturity Scoring Rubric
Every score above is based on this scale. A score is a snapshot of observable posture — not a verdict on the people or team behind the system.
| Score | Label | Meaning |
|---|---|---|
| 5 | Exemplary | Best-in-class; automated, continuously tested, reference quality. Rarely achieved in shipping systems. |
| 4 | Strong | Comprehensive controls, active management, minor gaps. Production-ready. |
| 3 | Established | Documented controls consistently applied; known gaps accepted. A respectable baseline. |
| 2 | Partial | Some controls exist but coverage is incomplete; key gaps remain. |
| 1 | Ad hoc | Informal or inconsistent measures; relies on individual judgment. |
| 0 | Absent | No evidence this category is addressed at all. |
Weighting: the weighted overall above is the sum of each category's score × weight (the per-category weights are shown on each card). Zero Trust carries double weight by design; see the RAISE framework reference for the rationale.