# Luxonis Agent Toolkit The Luxonis Agent Toolkit brings current Luxonis knowledge and OAK-specific development workflows into your coding agent. It packages eight specialist skills with the Luxonis MCP server. Install the Toolkit in Claude Code, Codex, or Cursor, then work in the coding environment you already use. For developers using a compatible coding agent, the Luxonis Agent Toolkit is the recommended way to get started with OAK and begin building an application. It brings camera setup, current Luxonis knowledge, and direct testing into one workflow, so you can move from a new camera to a working application without learning every tool first. Start with a normal request. You do not need to choose a skill or follow a fixed sequence: ```text Build an OAK application that scans barcodes on packages moving along a conveyor. ``` The primary `luxonis` skill determines what the request needs, retrieves current Luxonis documentation, examples, and model information through the Luxonis MCP server, and brings in a specialist when useful. > **Beta** > The Toolkit's skills and the project files they create may change between releases. Use the Toolkit to accelerate development and evaluation, but review the generated plan, code, and evidence before relying on the result. ## What you can do with the Toolkit Use the Toolkit to: * Ask OAK and DepthAI questions or choose a camera for a use case. * Unbox and set up a new OAK camera, then verify it is configured and ready for development in your environment. * Start from scratch and build the working foundation of a complete OAK application for your use case. * Record a real scene and replay it while developing without the camera. * Inspect frames, depth, detections, and other output from a running pipeline. * Diagnose an application that fails, hangs, or produces the wrong result. * Convert and integrate a custom model for the target OAK platform. For application work, the goal is not only to make a process run. The agent gathers direct evidence from the pipeline and verifies the output you actually requested, such as a file, API response, MQTT message, terminal event, or user interface. ## Install the Toolkit The Toolkit's full plugin includes eight skills and the Luxonis MCP server at `https://mcp.luxonis.com/mcp`. The MCP server gives the agent current Luxonis documentation, examples, platform context, and model information. Choose the installation path that fits how you work. Each full-plugin path installs the same Luxonis skills and MCP server. ### Ask your agent to install it This is the simplest path when your coding agent can run commands or guide you through plugin settings. Copy the prompt below into the agent. Review any permission requests before approving them. ```text Install the Luxonis Agent Toolkit in this coding agent. The toolkit is the "luxonis" plugin from the luxonis/skills marketplace (https://github.com/luxonis/skills). It adds the Luxonis skills and the Luxonis MCP server at https://mcp.luxonis.com/mcp. 1. Work out which harness you are running in (Claude Code, Codex, Cursor, or something else). 2. Read https://docs.luxonis.com/software-v3/agent-tools/agent-skills and follow the install steps for that harness. 3. Verify the skills are available and the luxonis MCP server is connected, then tell me what you installed and anything that still needs my approval. ``` ### Install with commands Use the commands below when you prefer to install the plugin yourself. Run each command separately and wait for it to finish before entering the next one. #### Claude Code Add the Luxonis marketplace: ```text /plugin marketplace add luxonis/skills ``` Install the plugin: ```text /plugin install luxonis@luxonis ``` Reload plugins: ```text /reload-plugins ``` Claude Code may ask you to approve the `luxonis` MCP server the first time it loads. You can verify the connection from a terminal: ```bash claude mcp list ``` The output should list `luxonis` at `https://mcp.luxonis.com/mcp` as connected. #### Codex Add the Luxonis marketplace from a terminal: ```bash codex plugin marketplace add luxonis/skills ``` Install the plugin: ```bash codex plugin add luxonis@luxonis ``` Start a new Codex session so it loads the skills and MCP tools: ```bash codex ``` You can also open `/plugins` in Codex, select the `luxonis` marketplace, and install or enable the plugin there. #### Cursor The plugin is not yet listed in the Cursor Marketplace. Clone the repository into Cursor's local plugin directory, then restart Cursor: ```bash git clone https://github.com/luxonis/skills ~/.cursor/plugins/local/luxonis ``` On Cursor Teams or Enterprise, an administrator can add `luxonis/skills` as a team marketplace from Dashboard → Plugins → Add Marketplace → Import from Repo. Plugin installation includes the Luxonis MCP server. #### Skills only The following options install the skill instructions but do not install the Luxonis MCP server. With the `npx` installer: ```bash npx skills@latest add luxonis/skills ``` For a Cursor remote rule, use: ```text luxonis/skills ``` Use a full Toolkit installation when possible so the skills can retrieve current Luxonis information instead of relying only on local context. ### Install in the app Use the app interface when you do not want to run installation commands. The labels may move as the apps change, but the flow is the same: add the `luxonis/skills` marketplace, install the `Luxonis` plugin, and connect the `luxonis` MCP server. #### Codex app Open Plugins, select Add, then choose Add a marketplace. Enter `luxonis/skills` as the source. Keep the Git ref as `main`, set the sparse path to `plugins/codex`, then select Add marketplace. Search for `luxonis`, then install the Luxonis plugin. Start a new Codex session after installation so the skills and MCP tools load. #### Claude app Open Customize → Plugins, select Add, then choose Add marketplace and Add from a repository. Enter `luxonis/skills` as the repository URL, leave automatic sync enabled, and select Sync. Open the Luxonis plugin after it appears and make sure the plugin is enabled. Open the plugin's Connectors tab and install the `luxonis` connector. Approve the connection when Claude asks. In a new chat, open the Connectors menu and confirm Luxonis is enabled before using the Toolkit. ## Start with your goal Open your coding agent in the folder where you want the work to live, then describe the outcome in plain language. For example: ```text Which OAK camera should I use for outdoor license-plate reads? ``` ```text Get this OAK camera working with this project. ``` ```text Build a people-counting application that publishes one MQTT event when a person crosses this doorway. ``` Compatible agents can select the right skill automatically. To invoke the primary skill explicitly in Claude Code, use: ```text /luxonis:luxonis ``` In Codex, use: ```text $luxonis ``` ## How application work proceeds The amount of process follows the request. A small first run, such as showing depth or streaming RGB, does not require a full product brief. A new product application or material redesign gets more structure. The agent reads the project, asks only for missing decisions that would change the build, and records the use case in `docs/brief.md` when the work needs a product brief. For substantial application work, the agent writes a dated plan under `docs/plans/`. The plan can include the pipeline, device and model choices, host-versus-device placement, user interface or final output, recording strategy, and validation checks. Review consequential decisions before implementation continues. The agent implements the smallest useful vertical slice. When useful, it records the camera and IMU sources under `recordings/` so development can continue by replaying representative input. The agent uses bounded evidence under `evidence/` to check what the running pipeline actually produces, then verifies the requested final output separately. It reports what was observed and what remains unproven. ## Toolkit skills The Toolkit includes the specialists below. You can invoke one directly when you have a narrow task. Otherwise, start with `luxonis` and let the agent route the work. | Skill | What it does | | --- | --- | | `luxonis` | Answers Luxonis questions, helps choose a device, checks the working folder, and routes implementation work. | | `luxonis-workspace` | Prepares the folder for Luxonis development, including project guidance, oakctl, the DepthAI environment, and a glossary. | | `luxonis-app` | Builds or changes an OAK application and verifies the requested behavior. | | `luxonis-device-setup` | Gets OAK hardware working for development and records reusable setup notes in `docs/device.md`. | | `luxonis-record` | Captures or replays a holistic recording of the real camera and IMU sources. | | `luxonis-inspect` | Inspects frames, depth, detections, topics, and other output from a running pipeline. | | `luxonis-troubleshoot` | Finds and repairs the first reproducible failing layer in an existing application. | | `luxonis-model` | Converts or integrates a custom model for a specific OAK target. | ## Project context across sessions The Toolkit preserves useful state in the project instead of depending on one long chat. Files are created only when the request needs them. | Path | Purpose | | --- | --- | | `AGENTS.md` | Always-on project guidance and pointers for compatible coding agents. | | `docs/glossary.md` | Project vocabulary and definitions. | | `docs/brief.md` | The living product problem and acceptance behavior. | | `docs/device.md` | Reusable host and device setup notes. Live device state still takes precedence. | | `docs/plans/` | Dated implementation plans and a pointer to the current plan. | | `recordings/` | Real camera and IMU inputs that can be replayed during development. | | `evidence/` | Bounded frames, messages, and other runtime observations. | ## Scope and limitations The Toolkit can help build and observe an OAK or DepthAI application, including supported custom-model conversion and integration. It does not autonomously train a model, construct a labeled training dataset, invent a proprietary SLAM system, deliver a complete ROS solution, or certify an application for production. When a use case depends on one of those systems, the agent should identify the boundary clearly and continue the supported OAK work when that remains useful. ## Related ### Toolkit repository Browse the source, report an issue, and inspect the full skill definitions. [Open on GitHub](https://github.com/luxonis/skills) ### oakctl CLI The host toolchain used for device setup, application runs, and pipeline inspection. [Install oakctl](https://docs.luxonis.com/software-v3/oak-apps/oakctl.md) ### OAK Examples Known-good DepthAI applications your coding agent can inspect and adapt. [Browse examples](https://github.com/luxonis/oak-examples)