Self-hosted web platform with a data agent (Python/SQL code interpreter), a plugins agent (200+ third-party API plugins), and a web-browsing agent driven through a Chrome extension.
github.com/xlang-ai/openagents · 2026-10-04 · ff2e464
Defense-in-depth score
1.8 / 10
As shipped, OpenAgents runs model-written Python directly inside its backend process, with the operator's OpenAI key in the environment and no approval step. Any content that hijacks the model can execute code as root in the container and read every user's data; access control on the backend API is not locked down. The optional Docker code interpreter helps, but it is off by default and not hardened in this repo.
OpenAgents ships few protections on by default: model-written Python runs inside the backend with no approval step, and access to the backend API is not locked down. The one mechanism to turn on is the Docker code interpreter, so the safest setup pairs it with scoped keys, a private deployment and containment you add yourself.
| Setup | How | What it gives you |
|---|---|---|
| Dedicated, low-limit keys | Give the deployment its own OpenAI key with a spending limit set at the provider, a Kaggle account used for nothing else, and save only the plugin keys you need, scoped or read-only where the service allows | A hijacked agent reaches only the spend and services you set aside for it, not your own accounts C1 |
| A private, authenticated entry point | Keep the backend and frontend ports on a private network behind an authenticating proxy, with one deployment per trusted group of users | Only people you chose can reach the agent, and stored conversations and files stay within that group C1 |
| The Docker code interpreter | Uncomment the code_interpreter service without its added capabilities and set CODE_EXECUTION_MODE=docker for the backend | Model-written Python runs in a separate kernel container that never holds the OpenAI key C4 |
| Only the tools needed | In the data agent's tool picker, select Python only for tasks that need code | Conversations that don't need code never run model-written Python C3 |
| The iteration cap | Keep the default max_iterations of 5 | Every agent turn ends after a small, fixed number of steps C10 |
| A separate browser profile | Install the web agent's extension in a browser profile signed in to nothing you would not hand to the agent | Clicks and form fills the agent sends land only in sessions you set aside for it C2 |
| A pre-fetched embedding model | Download the embedding model at a revision you have checked into the backend's model cache before first start, and run with Hugging Face downloads turned off | The backend loads only model weights you chose, and they can't change between restarts C7 |
| Its own VM with limited egress | Run the whole stack on a VM that holds no other data, with outbound traffic limited to OpenAI and the plugin hosts you use | An outer boundary around in-process code and plugins, and a narrow path for data to leave C5 |
SafetyCard
runtime.log and container output like the conversations they hold.0.05 / 1.00
Access control on the backend API is not locked down. Every tool runs with the operator's own OpenAI key, Kaggle key, unauthenticated MongoDB/Redis, and any third-party plugin API keys users saved, all as root in the backend container. There is no per-request authorization of tool use.
0.00 / 1.00
There is no human approval step anywhere in the backend. Model-generated Python and SQL run as soon as the model emits them, plugin API calls (including Zapier actions and a Netlify deploy endpoint) fire directly, and the web agent returns click and type actions for the user's live browser without a confirmation step in the server. The most powerful action, arbitrary code execution, is ungated by default.
0.25 / 1.00
The data agent's main tools accept arbitrary model-written Python and SQL with no argument validation. Plugins are narrower: each one only calls fixed endpoints on a fixed host, and the model's chosen endpoint must exist in the plugin's table; web-agent actions are type-checked. Python is selected by default in the shipped frontend, and the tool set is chosen per request by the client rather than restricted to read-only.
0.40 / 1.00
In the default configuration (CODE_EXECUTION_MODE=local, set in both app.py and docker-compose.yml) model-generated Python runs through IPython's run_cell in the backend's own worker process. A code-safety check exists but is not a strict boundary. The code therefore runs as root in the backend container with the OpenAI key in its environment, full network access, unauthenticated MongoDB/Redis, and write access to the application source and every user's files. An optional Docker code-interpreter mode sends code to a separate kernel container, but it is commented out in the compose file, uses an image not in this repo, adds NET_ADMIN/SYS_PTRACE capabilities, and shares the backend data volume.
0.00 / 1.00
The agents read plenty of content their user did not write: uploaded files and databases, Kaggle datasets, third-party plugin API responses, and arbitrary web pages in the web agent. None of it is marked or treated differently; tool results go straight back into the model context. A hijacked session can run arbitrary Python with network access and the OpenAI key, call write-capable plugins, and drive the user's browser, with no human in the loop; frontend output rendering is not hardened either.
0.07 / 1.00
Conversation history, including tool outputs that may carry injected text, is stored in MongoDB/Redis and reloaded as chat memory each time the conversation continues, with no validation or provenance. Uploaded and generated files persist in the backend data folder and can be reused as grounding data later. Per-user isolation of stored history and files is not a complete boundary. There are no auto-loaded instruction or config files from a workspace.
0.07 / 1.00
The 232 plugins are vendored Python modules shipped in the repository and loaded in-process with importlib; they are part of the pinned code rather than downloaded at runtime, though each one sends data to a third-party API. At startup the plugin selector downloads the hkunlp/instructor-large embedding model from Hugging Face with no pinned revision or hash, and (through the sentence-transformers/InstructorEmbedding loader) deserializes its weights in the backend process. Runtime pip installs of helper packages are also unpinned. Nothing runs third-party code in a separate or confined process.
0.15 / 1.00
Secrets come from environment variables and files: the OpenAI key is set in the compose file, the Kaggle key is written into the image at build time, and third-party plugin API keys are stored in plaintext in Redis/MongoDB. The backend API's handling of stored plugin keys is not locked down. Full request bodies and generated code are logged at DEBUG/TRACE by default, and the only protection is loguru's diagnose=False on file sinks. Model-run code can read every secret from the process environment.
0.33 / 1.00
Each finished chat turn is written to MongoDB with the intermediate steps (tool used, generated code, results) and the final answer, and loguru writes runtime logs. This gives a usable transcript, but it is written only when the stream completes, carries no real actor identity, and lives in an unauthenticated database and log directory that the model's own code can modify.
0.50 / 1.00
The agent loop stops after 5 iterations by default and runs in a child process that is terminated after a period without output (90 s for the data agent) or when the user hits stop. There is no wall-clock limit on the loop itself, no token or cost budget, no rate limiting, and the web agent's step loop is driven by the browser extension with no server-side cap. Termination kills the worker process but not anything it spawned.