Agentic LLM for autonomous data science with code-execution runtime and API
github.com/ruc-datalab/DeepAnalyze · 2026-10-03 · f04a1c3
Defense-in-depth score
2.7 / 10
As shipped in the README's first WebUI, DeepAnalyze executes every block of Python the model writes directly on the host, as your user, with your environment variables, full network access, and no approval, sandbox, round limit, or log. The backend's network exposure and access control, and that of its workspace file server, are not locked down. The newer WebUI v2 in the same repo is substantially safer by default (hardened per-session Docker sandbox with no network, run budgets, stop, execution records); deployers should use it, bound to localhost.
The default WebUI runs model-written Python on your host as your user, with no approval, sandbox, round limit or log, and its network access is not locked down. The project's WebUI v2 ships a hardened Docker sandbox, run budgets and execution records on by default, so the safest setup uses v2 and builds the identity and network boundaries around it yourself.
| Setup | How | What it gives you |
|---|---|---|
| A dedicated, credential-free account | Start the backend as a dedicated OS user whose environment and home directory hold no API keys, cloud or SSH credentials | Whatever code runs, it reaches only what that account holds, not your own credentials C1 |
| Reachable only by you | Bind the backend and file server to localhost, and use an SSH tunnel or an authenticating reverse proxy for remote access | Only you can start analyses or read the workspace C1 |
| WebUI v2's Docker sandbox | Run WebUI v2 (demo/chat_v2) with the default DEEPANALYZE_EXECUTION_MODE of docker and DEEPANALYZE_ALLOW_UNSAFE_LOCAL_EXECUTION left false | Model code runs in a per-session container with no host environment and only the session workspace mounted C4 |
| The sandbox's hardening kept | Keep the shipped DEEPANALYZE_DOCKER_NETWORK_MODE of none, DEEPANALYZE_DOCKER_USER of 1000:1000 and DEEPANALYZE_DOCKER_READ_ONLY of true in the environment and .env | The container stays offline, non-root and read-only, so data has no direct path out C4 |
| Run budgets | Keep DEEPANALYZE_CHAT_MAX_ROUNDS at 12 and DEEPANALYZE_CHAT_MAX_DURATION_SEC at 900, or lower | Every analysis ends after a fixed number of rounds and minutes C10 |
| A fresh session per dataset | Start a new session for each dataset from outside, and remove leftover files you didn't add before the next request | Files written during one analysis can't steer the next one C6 |
| Its own VM | Run the whole stack in a VM whose network reaches only the model server and the hosts the task needs | An outer boundary around model code and a narrow path for data to leave C5 |
| A lasting execution record | Copy the .session_state directory beside the workspace to storage the backend can't modify | A record of every executed script and its output that outlives the 200-run limit C9 |
0.00 / 1.00
The WebUI backend runs model-written Python as the same operating-system user that started it and hands that code a full copy of the server's environment variables, so whatever credentials the operator has (environment keys, ~/.aws, ~/.ssh, cloud CLIs) are available to the model. The API's network exposure and access control are not locked down. There is no agent-specific identity and no notion of which user is asking.
0.00 / 1.00
There is no human approval anywhere. Every <Code> block the model emits is extracted and executed immediately in a loop until the model writes an answer, and the same code-execution engine is exposed directly over HTTP. Model code can delete or overwrite workspace and host files, make network requests, and run any program, with nothing to review first and no undo.
0.00 / 1.00
The agent has one tool: run arbitrary Python. It is not narrowed or validated in any way, and input handling on the surrounding HTTP endpoints is not a strict boundary.
0.50 / 1.00
In the scored WebUI, model-generated Python runs as an ordinary child process on the host, as the same user, with the server's full environment and unrestricted network and filesystem access; the only limit is a 120-second timeout. The project's newer WebUI v2 ships a much better design that is on by default there: each session gets a Docker container with all capabilities dropped, no-new-privileges, a non-root user, a read-only root filesystem, no network, CPU/memory/PID limits, and only the session workspace mounted, and it refuses to fall back to host execution unless an explicitly named unsafe flag is set. Because that sandbox is only used if the deployer chooses WebUI v2 instead of the default WebUI, it is credited as an opt-in mechanism.
0.00 / 1.00
DeepAnalyze's job is to read data files supplied by the user, which may come from anywhere, and code output from those files flows straight back into the model's context with the same standing as everything else. Nothing limits what a hijacked session can do: it can run any code, delete files, and send data out, either directly from the code (full network access) or through other unconfined paths.
0.10 / 1.00
DeepAnalyze has no long-term memory, vector store, or auto-loaded instruction files, and it does not load a .env from the working directory. The persistent state that does exist is the session workspace: files written by model code stay there across conversations in the same browser session, and the names of top-level files are inserted into every new user message as the '# Data' section, so a poisoned session can plant text that later runs see as part of the user's request and act on with code. Session separation is not enforced server-side.
N/A · full credit 1.00 / 1.00
The scored WebUI backend loads no plugins, MCP servers, downloadable tools, or model files at runtime; the model is served by a separately run vLLM process. Model-written code could still install packages itself, but that is a consequence of unsandboxed code execution and is scored under code-execution isolation. The separate Jupyter demo connects to jupyter-mcp-server and the README's vLLM commands pass --trust-remote-code; neither is part of the scored backend.
0.10 / 1.00
The WebUI holds no API keys of its own (the local vLLM key is the literal 'dummy') and sends no telemetry. But it does nothing to keep sensitive material contained: model code receives a full copy of the server's environment, nothing is redacted anywhere, and access to uploaded data through the workspace file server is not locked down.
0.50 / 1.00
The scored WebUI keeps no record of what the agent did: there is no logging module, no execution log, and the conversation transcript lives only in the browser. After an incident you could not reconstruct which code ran or who triggered it. WebUI v2 is better: it saves every executed script to the workspace and appends a structured execution record (run id, source agent or manual, timestamps, output, artifacts) to a session-state file outside the container-visible workspace, though it has no actor identity and keeps only the last 200 runs.
0.50 / 1.00
Each code execution in the scored WebUI is killed after 120 seconds, and each model call is limited to 32,768 new tokens, but the agent loop itself runs until the model writes an answer, with no cap on rounds, total time, or cost, and there is no stop button on the server side. WebUI v2 adds real budgets on by default (12 model rounds, 15 minutes per analysis, a response-size cap, per-execution timeouts bounded by remaining time) and a stop endpoint that closes the model stream and stops the session's container, though code can leave background processes running in the container until it is reaped.
SafetyCard