AutoGen Code Executor Analysis Report
Completed July 11, 2026 · 16 artifacts examined
11Findings
1Critical
5High
5Medium
RAISE maturity 2.00 / 5.0
Executive Summary
Agent Remit (as declared)
The AutoGen code-executor family is the execution tier of a generator/executor pattern: `CodeExecutorAgent` extracts fenced Python and shell code blocks from paired-agent or group-chat messages and runs them, returning structured exit status and captured output. Execution is expected to happen inside an isolated environment — the Docker command-line executor is the recommended production path, with Jupyter, Azure dynamic-sessions, and a local-host executor as alternatives. The remit confines file I/O to a configured working directory, requires a wall-clock timeout on every execution, forbids the executor from initiating its own outbound messaging or network egress, and requires operator approval before using the local host executor in production, mounting extra host volumes, exposing GPUs, or raising resource ceilings.
Behavior Summary (as observed)
The dominant pattern is isolation with the shape but not the substance of a guarantee: the container executors provide real host isolation, per-execution timeouts, and an approval hook, but the safe path is neither the default nor guaranteed. `create_default_code_executor` downgrades to unisolated host execution (`LocalCommandLineCodeExecutor`) on nothing more than a `warnings.warn` when Docker is absent, `approval_func` defaults to `None` so LLM-generated code auto-executes, and the local executor hands the executed code the parent process's full environment through `os.environ.copy()`. Compounding this, the Docker executor exposes no network-isolation or resource-limit controls by default and no executor writes a durable structured record of what it ran, so an untrusted code block can reach unapproved host execution with open network egress and no audit trail. The local executor's own docstring even claims a dangerous-command sanitizer that does not exist anywhere in the code.
Scope of Analysis
A Python library spanning `autogen-agentchat` (the `CodeExecutorAgent` in `_code_executor_agent.py`) and `autogen-ext` (the executor implementations under `code_executors/`). `CodeExecutorAgent` optionally drives an LLM `model_client` to generate, execute, reflect, and retry, and an optional `approval_func` gates each block. Executors share a common code-writing and `silence_pip` path in `_common.py`: `LocalCommandLineCodeExecutor` and `JupyterCodeExecutor` run on the host, `DockerCommandLineCodeExecutor` and `DockerJupyterCodeExecutor` run in containers, and `ACADynamicSessionsCodeExecutor` posts code to an Azure endpoint. `create_default_code_executor` prefers Docker but silently falls back to the local host executor when Docker is unavailable, working-directory confinement is enforced only for an optional `# filename:` header, and no executor emits a durable structured audit record of what it ran.
Remit Coverage

Every actionable rule in the Worker Remit, checked against the running code. Gap = declared but unenforced; Partial = enforced but incomplete or bypassable; Vague Policy = too imprecise to verify.

Verified: 12 Gap: 4 Partial: 10 Vague Policy: 0 Enforcement Not Possible: 1 Total Rules: 27
Rule ID Section Rule (quoted) Status Finding
R-01 Job Description "Extract code from properly formatted markdown code blocks (triple-backtick fences) in incoming messages and execute only code in supported languages (Python and shell for command-line executors; Python only for the Jupyter and Azure executors)." Verified
R-02 Job Description "Execute each code block inside the configured execution environment, in the order received, and return a structured result containing exit status and captured output." Verified
R-03 Job Description "Confine all file reads and writes to a configured working directory (for the Azure executor, `/mnt/data`)." Partial PRAX-2026-07-11-010
R-04 Job Description "Enforce a configured wall-clock timeout on every execution and terminate any process that exceeds it." Verified
R-05 Job Description "When configured with an approval function, obtain approval for each code block before executing it." Verified
R-06 Job Description "Surface all errors (syntax, runtime, timeout, out-of-memory) back to the caller in structured form without silently discarding them." Verified
R-07 Non-Goals (Out of Scope) "Sending email, SMS, webhooks, or any outbound message on its own behalf." Verified
R-08 Approved Communication Channels "The executor must not initiate its own network traffic beyond a configured allow-list." Partial PRAX-2026-07-11-004
R-09 Authorized Counterparties "Any code source other than the paired agent's generated code blocks and code passed explicitly by operator application logic — code arriving from a file, HTTP endpoint, message queue, or memory store must not be accepted by the executor interface." Verified
R-10 Authorized Counterparties "The paired AutoGen code-generator agent (the LLM component that produces code); code from this counterparty is the only accepted execution input." Partial PRAX-2026-07-11-008
R-11 Forbidden Tools "Any mechanism that loads or executes code fetched from a remote URL or other unverified source on the LLM's behalf." Verified
R-12 Forbidden Tools "Any capability that persists execution results to long-term storage or carries state across unrelated sessions." Verified
R-13 Forbidden Data Movement "Reading or modifying the parent process's environment, credentials, or state from within an execution." Partial PRAX-2026-07-11-006
R-14 Forbidden Data Movement "Moving files or data outside the configured working directory." Partial PRAX-2026-07-11-010
R-15 Action Boundaries "Executing code on the local host executor in a production deployment (default production deployments must use a containerized executor)." Gap PRAX-2026-07-11-001
R-16 Action Boundaries "Enabling any network egress from the execution environment beyond a configured allow-list." Gap PRAX-2026-07-11-004
R-17 Action Boundaries "Mounting host volumes into a container executor at any path other than the configured working directory." Partial PRAX-2026-07-11-009
R-18 Action Boundaries "Any change to resource limits (CPU, memory, timeout) that raises the ceiling." Partial PRAX-2026-07-11-007
R-19 Action Boundaries "Exposing host GPUs or other host devices to a container executor." Partial PRAX-2026-07-11-009
R-20 Action Boundaries "Executing code with host-level privileges when a less-privileged option achieves the same task." Partial PRAX-2026-07-11-001
R-21 Action Boundaries "Loading or executing code from a remote URL or unverified source on the LLM's behalf; the paired agent's generated code is the only accepted input." Verified
R-22 Action Boundaries "Silently swallowing or transforming error output before returning it to the caller." Verified
R-23 Action Boundaries "Connecting to services, databases, or networks not explicitly permitted by configuration." Partial PRAX-2026-07-11-004
R-24 Behavioral Expectations "Actions that should never be retried: executions terminated for exceeding the timeout must be terminated, not re-run in a loop; approval-denied code must not be retried without new approval." Verified
R-25 Escalation Rules "Every execution is recorded with timestamp, executor kind, language, source agent, working directory, timeout, exit status, and a digest (not the body) of the executed code." Gap PRAX-2026-07-11-005
R-26 Escalation Rules "The local host executor is used in a production deployment without operator approval." Gap PRAX-2026-07-11-001
R-27 Escalation Rules "A code execution attempts to read or write outside the working directory, or to escape the container/sandbox." Enforcement Not Possible
Findings Register

Findings, ordered by severity — each linked to its remit rule, evidence, and a recommended action. Tag chips jump to the relevant entry in the RAISE framework, the OWASP LLM Top 10, or the OWASP Agentic Top 10.

LLM06 primary — the finding's main OWASP categoryASI10 secondary — a category it also touches

CRITICAL PRAX-2026-07-11-001 create_default_code_executor silently downgrades to unisolated host execution with only a warning, defeating the required container-approval gate.
Policy Rule — R-15, R-20, R-26 (Worker Remit):
"Executing code on the local host executor in a production deployment (default production deployments must use a containerized executor). / Executing code with host-level privileges when a less-privileged option achieves the same task. / The local host executor is used in a production deployment without operator approval."
python/packages/autogen-ext/src/autogen_ext/code_executors/__init__.py:58 — create_default_code_executor falls through to LocalCommandLineCodeExecutor with only a warnings.warn (lines 58-80) when _is_docker_available() is False — no approval, no error, no halt. python/packages/autogen-ext/src/autogen_ext/code_executors/local/__init__.py:341 — _execute_code_dont_check_setup runs each block as a host subprocess (cwd=work_dir) with no container isolation — the fallback target of the silent downgrade.
Recommended Action
  • In `create_default_code_executor`, do not silently fall back to `LocalCommandLineCodeExecutor`; raise unless the caller passes an explicit opt-in flag (e.g. `allow_local_fallback=True`) acknowledging unisolated host execution.
  • Route any local-host execution in a non-ephemeral deployment through the `approval_func` path so it cannot run without an approval response.
HIGH PRAX-2026-07-11-002 approval_func defaults to None, so all LLM-generated code is auto-executed with no human-in-the-loop gate.
python/packages/autogen-agentchat/src/autogen_agentchat/agents/_code_executor_agent.py:458 — approval_func default None triggers only warnings.warn (lines 458-467); execute_code_block runs the approval check only `if self._approval_func is not None` at line 691, otherwise executes directly.
Recommended Action
  • Consider making `approval_func` required (or defaulting to a deny-by-default gate) for non-interactive deployments, rather than auto-approving on a bare warning.
  • Document and expose a supported "approval mandatory" configuration so operators can enforce human review without writing their own gate.
HIGH PRAX-2026-07-11-003 LocalCommandLineCodeExecutor docstring claims dangerous-command sanitization that is not implemented anywhere in the code.
python/packages/autogen-ext/src/autogen_ext/code_executors/local/__init__.py:57 — Docstring claims "Command line code is sanitized using regular expression match against a list of dangerous commands"; a repo-wide grep for sanitize/dangerous/denylist returns only this comment line — no implementation. python/packages/autogen-ext/src/autogen_ext/code_executors/local/__init__.py:352 — _execute_code_dont_check_setup loop applies only silence_pip and get_file_name_from_content — no command sanitization before writing and executing the code.
Recommended Action
Either implement the documented dangerous-command sanitizer or remove the sanitization claim from the docstring so operators do not over-trust the local executor.
HIGH PRAX-2026-07-11-004 DockerCommandLineCodeExecutor creates containers with default networking and exposes no egress control, so executed code can reach the network.
Policy Rule — R-08, R-16, R-23 (Worker Remit):
"The executor must not initiate its own network traffic beyond a configured allow-list. / Enabling any network egress from the execution environment beyond a configured allow-list. / Connecting to services, databases, or networks not explicitly permitted by configuration."
python/packages/autogen-ext/src/autogen_ext/code_executors/docker/_docker_code_executor.py:537 — client.containers.create (lines 537-550) sets volumes, working_dir, extra_hosts and device_requests but no network_mode — the container gets default bridge networking with open egress.
Recommended Action
Default the container to `network_mode="none"` and expose an explicit allow-list parameter so egress is opt-in per the remit.
HIGH PRAX-2026-07-11-005 No executor writes a durable, structured audit record of executions, leaving the remit's Log Only requirement unmet.
Policy Rule — R-25 (Worker Remit):
"Every execution is recorded with timestamp, executor kind, language, source agent, working directory, timeout, exit status, and a digest (not the body) of the executed code."
python/packages/autogen-agentchat/src/autogen_agentchat/agents/_code_executor_agent.py:623 — Execution is surfaced only as a yielded CodeExecutionEvent (line 624) and the module's event_logger (line 47) is never called to persist an execution record. python/packages/autogen-ext/src/autogen_ext/code_executors/docker/_docker_code_executor.py:362 — _execute_command returns output/exit_code with no structured audit write; surrounding logging calls are debug/error diagnostics, not an action-level execution log.
Recommended Action
Emit a structured (JSON) execution record per block — executor kind, language, source, work_dir, timeout, exit status, and a SHA-256 digest of the code — to a durable logger the operator can route.
HIGH PRAX-2026-07-11-006 LocalCommandLineCodeExecutor passes the parent process's full environment to executed code via os.environ.copy().
Policy Rule — R-13 (Worker Remit):
"Reading or modifying the parent process's environment, credentials, or state from within an execution."
python/packages/autogen-ext/src/autogen_ext/code_executors/local/__init__.py:397 — env = os.environ.copy() then passed as env= to create_subprocess_exec (lines 397-434) — executed code receives all parent env vars including secrets.
Recommended Action
Build the subprocess environment from an explicit allow-list (PATH plus any operator-approved variables) instead of copying the full parent environment.
MEDIUM PRAX-2026-07-11-007 Docker executor sets no CPU or memory limits, bounding executions only by wall-clock timeout.
Policy Rule — R-18 (Worker Remit):
"Any change to resource limits (CPU, memory, timeout) that raises the ceiling."
python/packages/autogen-ext/src/autogen_ext/code_executors/docker/_docker_code_executor.py:537 — containers.create (lines 537-550) passes no mem_limit/nano_cpus/pids_limit; the only bound is the "timeout" command wrapper at line 360.
Recommended Action
Expose and default sensible `mem_limit`, CPU, and PID limits on the container, requiring operator approval to raise them per the remit.
MEDIUM PRAX-2026-07-11-008 The sources counterparty filter defaults to None, so code from any group-chat agent is executed rather than only the paired coder.
Policy Rule — R-10 (Worker Remit):
"The paired AutoGen code-generator agent (the LLM component that produces code); code from this counterparty is the only accepted execution input."
python/packages/autogen-agentchat/src/autogen_agentchat/agents/_code_executor_agent.py:681 — extract_code_blocks_from_messages executes blocks when "self._sources is None or msg.source in self._sources" — default sources=None accepts code from any source.
Recommended Action
Document `sources` as a security-relevant control and consider requiring it (or defaulting to the paired agent) in multi-agent group chats.
MEDIUM PRAX-2026-07-11-009 extra_volumes and device_requests are accepted and applied with no gate or warning, allowing host mounts and GPU exposure beyond the working directory.
Policy Rule — R-17, R-19 (Worker Remit):
"Mounting host volumes into a container executor at any path other than the configured working directory. / Exposing host GPUs or other host devices to a container executor."
python/packages/autogen-ext/src/autogen_ext/code_executors/docker/_docker_code_executor.py:546 — volumes={bind_dir: /workspace, **self._extra_volumes} and device_requests=self._device_requests passed to containers.create (lines 546-549) with no validation or approval.
Recommended Action
Log a warning (or require an explicit acknowledgement flag) when `extra_volumes` or `device_requests` extend the container beyond the working directory, so the approval requirement is visible.
MEDIUM PRAX-2026-07-11-010 Working-directory confinement is enforced only for the optional # filename: header; the code body is unconfined on host executors.
Policy Rule — R-03, R-14 (Worker Remit):
"Confine all file reads and writes to a configured working directory (for the Azure executor, `/mnt/data`). / Moving files or data outside the configured working directory."
python/packages/autogen-ext/src/autogen_ext/code_executors/_common.py:96 — get_file_name_from_content only validates the workspace boundary for a leading "# filename:" line; code without that header is written and executed unchecked. python/packages/autogen-ext/src/autogen_ext/code_executors/local/__init__.py:426 — The subprocess runs with cwd=work_dir but no filesystem sandbox, so absolute-path or ../ file access in the code body escapes the working directory on the host.
Recommended Action
Document that host executors do not confine the code body and recommend a container/Jupyter-in-container executor whenever working-directory confinement is a requirement.
MEDIUM PRAX-2026-07-11-011 The default Docker execution image python:3-slim is a floating tag with no digest pin, exposing executions to base-image drift.
python/packages/autogen-ext/src/autogen_ext/code_executors/docker/_docker_code_executor.py:71 — image: str = "python:3-slim" (config line 71, constructor default line 160) — a floating tag pulled without a digest pin in start()'s images.pull.
Recommended Action
Pin the default image by digest (or document a pinned recommended image) and support an operator-supplied approved-image allow-list.
What's Working Well

Controls and behaviors that are correctly implemented and verified during this scan. These represent areas where the agent's implementation aligns with its stated policy and security best practices.

Container executors provide genuine host isolation

DockerCommandLineCodeExecutor and the Docker-Jupyter executor run code inside a container that mounts only the working directory to /workspace, giving real filesystem and process isolation on the recommended production path.

python/packages/autogen-ext/src/autogen_ext/code_executors/docker/_docker_code_executor.py:546

Per-execution wall-clock timeout enforced across all executors

Every executor enforces a configurable timeout and terminates the process/kernel/request that exceeds it, bounding runaway executions.

python/packages/autogen-ext/src/autogen_ext/code_executors/docker/_docker_code_executor.py:360

Approval hook available and honored before execution

When an approval_func is supplied, execute_code_block calls it with the code and context and returns an error result without running the code if approval is denied; the behavior is unit-tested.

python/packages/autogen-agentchat/src/autogen_agentchat/agents/_code_executor_agent.py:691

Errors surfaced, not swallowed

Execution results carry the real exit code and captured stderr/stdout back to the caller, matching the remit requirement that errors never be silently discarded.

python/packages/autogen-agentchat/src/autogen_agentchat/agents/_code_executor_agent.py:720

Docker-Jupyter server generates a random auth token by default

DockerJupyterServer defaults token to a 32-byte secrets.token_hex value and binds the published port to 127.0.0.1, so the kernel gateway is authenticated unless the operator explicitly disables the token.

python/packages/autogen-ext/src/autogen_ext/code_executors/docker_jupyter/_jupyter_server.py:353

Core dependency pinned to an exact version

autogen-ext pins autogen-core==0.7.5 exactly, reducing supply-chain drift on the primary runtime dependency.

python/packages/autogen-ext/pyproject.toml:18
Discovered Log Files

Log files found in the agent's workspace during this scan. Reviewing these files provides runtime evidence to complement the static analysis above.

Path Source Content Type Purpose Last Modified Status
(stderr / configured logging handlers) autogen_ext.code_executors.docker._docker_code_executor and local executor unstructured Python logging (debug/info/error) diagnostic messages for container lifecycle, pip install, and stop/cancel errors — not an execution audit trail unknown Inferred
(in-memory message stream) autogen_agentchat CodeExecutorAgent ephemeral CodeGenerationEvent / CodeExecutionEvent objects per-turn generation and execution events yielded to the caller UI; not persisted unknown Inferred
OWASP LLM Top 10 (2025) Coverage

Each card represents one category and shows the top 3 findings. All items in the Findings section.

OWASP Agentic Top 10 (2026) Coverage

Each card represents one category and shows the top 3 findings. All items in the Findings section.

RAISE Maturity Posture

Overall maturity assessment across the six categories of the RAISE framework. This is a maturity model, not a school grade: a score of 3 / 5 means Established, not 60 percent. Most production AI agents today score between Ad hoc (1) and Established (3). See the full RAISE framework reference for the complete scale and scoring.

2.00 / 5.0
Weighted Maturity Score · Partial
Partial (2.00). The executor family has genuine, operative Zero Trust primitives — real container isolation on the recommended path, a wall-clock timeout on every execution, an approval hook, and errors surfaced rather than swallowed — which keep it above Ad hoc. But the secure configuration is opt-in rather than default: silent host fallback, auto-approval, open container egress, a parent-environment leak, and unrestricted host mounts pull Zero Trust down to Partial, while the near-total absence of structured execution logging and of adversarial testing leaves Monitor Continuously and Build an AI Red Team at Ad hoc. Supply-chain management and domain scoping are the strongest areas, held back respectively by floating base-image tags and a default-open code-source filter.
Limit Your Domain
3/ 5
Confidence: High  |  Weight: 15%  |  Weighted: 0.45
The component is narrowly scoped to code execution — `CodeExecutorAgent` extracts only fenced blocks in `supported_languages` (default `python`/`sh`) via a compiled regex and the terminal description forbids other action — but the `sources` counterparty filter defaults to `None`, so in a group chat any agent's code is executed rather than only the paired coder's.
Balance Your Knowledge Base
2/ 5
Confidence: Medium  |  Weight: 15%  |  Weighted: 0.30
Untrusted LLM- or agent-generated code is the executor's primary input and is run with no validation of its source or content; there is no RAG/memory surface, but the default-open `sources` filter means code from any group-chat participant enters the execution path unchecked.
Implement Zero Trust
2/ 5
Confidence: High  |  Weight: 25%  |  Weighted: 0.50
Real controls exist and run — container isolation on the Docker path, a per-execution timeout, an approval hook, and `# filename:` workspace confinement — but the defaults are permissive: `create_default_code_executor` silently falls back to host execution, `approval_func` defaults to auto-approve, the local executor leaks the parent environment via `os.environ.copy()`, and the Docker executor ships no network-isolation or resource-limit controls.
Manage Your Supply Chain
3/ 5
Confidence: Medium  |  Weight: 15%  |  Weighted: 0.45
The core dependency is pinned exactly (`autogen-core==0.7.5`) and the package is a mature, provenance-known monorepo, but executor extras are floor-pinned with `>=`/`~=` (`docker~=7.0`, `nbclient>=0.10.2`, `aiohttp>=3.11.16`) and the default Docker execution image `python:3-slim` is a floating tag with no digest pin.
Build an AI Red Team
1/ 5
Confidence: Medium  |  Weight: 15%  |  Weighted: 0.15
The test suites are correctness-focused — an approve/deny approval test and a single `# filename:` outside-workspace rejection test exist in `test_commandline_code_executor.py` and `test_code_executor_agent.py` — but there is no adversarial red-team program, sandbox-escape/injection corpus, or evidence that security testing drove architectural change.
Monitor Continuously
1/ 5
Confidence: High  |  Weight: 15%  |  Weighted: 0.15
No executor emits a durable, structured, action-level record of executions (executor kind, language, source agent, exit status, code digest); only ad hoc Python `logging` debug/error calls and ephemeral yielded `CodeExecutionEvent` objects exist, so the remit's Log Only audit requirement is unmet and detection of escape/egress attempts is impossible.

Maturity Scoring Rubric

Every score above is based on this scale. A score is a snapshot of observable posture — not a verdict on the people or team behind the system.

Score Label Meaning
5 Exemplary Best-in-class; automated, continuously tested, reference quality. Rarely achieved in shipping systems.
4 Strong Comprehensive controls, active management, minor gaps. Production-ready.
3 Established Documented controls consistently applied; known gaps accepted. A respectable baseline.
2 Partial Some controls exist but coverage is incomplete; key gaps remain.
1 Ad hoc Informal or inconsistent measures; relies on individual judgment.
0 Absent No evidence this category is addressed at all.
Weighting: the weighted overall above is the sum of each category's score × weight (the per-category weights are shown on each card). Zero Trust carries double weight by design; see the RAISE framework reference for the rationale.