Qwen-MM-Plugins
English · 中文
Native multimodal plugins for Qwen models. Make any agent harness multimodal-native.
Contents
- 🧩 Capabilities
- 🏗 Architecture
- 📦 Installation
- 🔧 Dependencies
- 🔑 Configuration
- 🚀 Quick Start
- 🧪 Development
🧩 Capabilities
Each capability is installed separately — a skill (so the model knows the toolset exists) plus an optional MCP server (the tools themselves).
We ship cookbooks of Qwen3.8-Max + these plugins in action — each capability's cookbook (linked in the table below) has its full tool listing, setup, and worked cases. Enjoy!
| Capability | What it does | Install name | Cookbook |
|---|---|---|---|
| core | Local I/O plugin: read images and video in dynamic resolution, and visualize any file (e.g. docs, 3D, and more) — plus some image tools (crop, annotate, extract frames) | qwen-mm-plugins-core |
link |
| api | Cloud APIs for understanding media, by model family: VL (vision chat, OCR, grounding), Omni A/V (timestamped captioning, ASR / multi-speaker diarization, temporal grounding, event counting), plus ASR and segmentation (SAM3); currently supports DashScope | qwen-mm-plugins-api |
link |
| search | Web + reverse-image search to confirm facts: web search, page extraction, reverse image search; currently supports Serper | qwen-mm-plugins-search |
TBD |
| video-memory | Long-video memory: a hierarchical graph memory that powers QA over very long videos | qwen-mm-plugins-video-memory |
TBD |
| video-edit | Video editing + generation: editing workflows + image / video / audio generation | qwen-mm-plugins-video-edit |
TBD |
| blender | Blender 3D modeling: drive a running Blender via Python (thin client, 22 tools) — modeling / materials / lighting / rendering | qwen-mm-plugins-blender |
TBD |
| freecad | FreeCAD parametric CAD: drive a running FreeCAD (thin client, 14 tools) — modeling, property edits, STEP/STL import/export, FEM analysis | qwen-mm-plugins-freecad |
TBD |
| edu-agent | Educational tutorial videos: turn a math/science problem or an image into a step-by-step Chinese explainer video / interactive page (skill-only, no MCP server) | qwen-mm-plugins-edu-agent |
TBD |
🏗 Architecture
📦 Installation
A capability = a skill (so the model knows the tools exist) + an optional MCP server (the tools themselves, launched on demand by uvx — needs uv, no manual pip).
Recommended: the guided installer
One script handles install · configure · verify · uninstall across every harness it supports (Claude Code · Codex · Qoder · OpenClaw · Qwen Code · Gemini CLI). It drives each harness's own native install under the hood — nothing reinvented — and writes a single shared config file (~/.qwen-mm-plugins/config) that GUI and terminal harnesses both read, so you set things up once:
curl -fsSL https://raw.githubusercontent.com/QwenLM/Qwen-MM-Plugins/main/install.sh | bash
Or run one action at a time — bash install.sh install / configure / verify / uninstall (what configure and verify do is detailed under Configuration and Dependencies).
Windows x64: use WSL2 (Ubuntu recommended) and clone the repository inside your WSL
home directory (for example ~/code), rather than under a mounted Windows drive such as
/mnt/c. Then run the same commands there. WSL2 is currently the only supported Windows
environment; native Windows has not yet been validated. See the concise
Windows notes.
By hand (per-harness)
Prefer your harness's own commands — or you're on opencode / pi / QwenPaw, which the installer doesn't cover? Register the skill + MCP yourself.
Plugin-marketplace harnesses (Claude Code · Qoder · Codex · OpenClaw · Qwen Code) — add the marketplace, then install a capability (replace <cap> with core / api / search / video-memory / video-edit / blender / freecad). Install core by default — it's the local-I/O base every other capability builds on — plus whichever others you need:
# Claude Code
claude plugin marketplace add https://github.com/QwenLM/Qwen-MM-Plugins.git
claude plugin install qwen-mm-plugins-<cap>@qwen-mm-plugins
# Qoder
qodercli plugins marketplace add https://github.com/QwenLM/Qwen-MM-Plugins.git
qodercli plugins install qwen-mm-plugins-<cap>@qwen-mm-plugins
# Codex
codex plugin marketplace add https://github.com/QwenLM/Qwen-MM-Plugins.git
codex plugin add qwen-mm-plugins-<cap>@qwen-mm-plugins
# OpenClaw
openclaw plugins install qwen-mm-plugins-<cap> --marketplace https://github.com/QwenLM/Qwen-MM-Plugins.git
# Qwen Code
qwen extensions install https://github.com/QwenLM/Qwen-MM-Plugins.git:qwen-mm-plugins-<cap> --consent
marketplace add also accepts a local repo path; re-running is safe. On codex, marketplace add does not refresh an already-added marketplace, so run codex plugin marketplace upgrade qwen-mm-plugins before plugin add to pick up newly-published capabilities.
Other harnesses (Gemini CLI · opencode · pi · QwenPaw · …) register the skill + MCP in their own config — exact per-harness blocks are in docs/en/installation.md. Easiest of all: just ask the agent — "install qwen-mm-plugins-<cap>".
🔧 Dependencies
uvx installs the Python dependencies for the chosen profile on first launch — no manual pip. The only things you install yourself are system tools: ffmpeg (video / audio), plus optional libreoffice / blender / texlive / chromium for visualize. Run bash install.sh verify to self-test what's installed — it confirms your API key and reports any missing system tools (fetching each capability's env and running --check-system under the hood). Full system-tool table, the edu-agent (skill-only) setup, and the blender/freecad thin-client notes: see docs/en/installation.md.
🔑 Configuration
The API-based tools need a key — native image / video / document reading doesn't:
DASHSCOPE_API_KEY—vision_chat/ocr/grounding/transcribe_audio/ Omni audio-video understanding / generation / video-memory buildSERPER_API_KEY—web_search/web_extractor/image_search
Export them in your shell, or persist them to ~/.qwen-mm-plugins/config (read whenever a var isn't already in the environment — so GUI-launched harnesses pick them up too). The guided installer's Configure step writes that file for you:
bash install.sh configure
For non-interactive/automation setup and the full environment-variable catalog, see docs/en/installation.md.
🚀 Quick Start
Once a capability is installed, reference a file in your harness and just ask — the model picks the right tool automatically. Reading is dynamic-resolution: every image, video frame, and document page is auto-scaled to the VL model's patch grid, so a 4K screenshot's fine print and a tiny thumbnail both come in at the detail they need — no manual resizing.
# core — read images / video / docs / 3D models (local, dynamic-resolution)
@dashboard-4k.png Read every number in this dashboard.
@report.pdf Summarize page 3. # api — cloud VL + Omni APIs: caption / OCR / grounding / segmentation / ASR, plus Omni audio-video understanding
@receipt.jpg OCR this and total the line items.
@street.jpg Draw a box around every car in the scene. # grounding
@meeting.mp4 Transcribe this with speaker labels and timestamps. # omni
@sports-clip.mp4 Count every completed pass and list when each one occurs. # omni
@song.mp3 Tag the genre, mood, instruments, key, and vocal profile. # omni
# search — web + reverse-image search to confirm what's on screen
@place.jpg Where was this photo taken? # image_search + web_search
# video-memory — QA over long videos; the first query auto-builds memory
@lecture-2h.mp4 What are the main points, with timestamps?
# video-edit — image / video / audio generation + editing workflows
Generate a 1024×1024 image of a red panda coding at night.
@/path/to/media Help me edit this video down to about 3 minutes.
# blender — drive a running Blender to model / texture / light / render (thin client, 22 tools)
Model a low-poly wooden stool, add a warm key light, and render it.
# freecad — parametric CAD in a running FreeCAD (thin client, 14 tools; STEP/STL, FEM)
Model an M6 hex bolt 30 mm long and export it as STEP.
# edu-agent — turn a math/science problem into a step-by-step Chinese explainer video (skill-only)
@geometry-problem.png Explain how to solve this as a narrated video.
See each capability's 🍳 cookbook for every tool, setup, and a worked case.
🧪 Development
Development setup, contribution guidelines, and verification commands are in
CONTRIBUTING.md. Detailed guides: local development
· adding a capability · testing.
📄 License
Apache-2.0 — see LICENSE. The Blender and FreeCAD capabilities vendor third-party MIT-licensed code; see src/capabilities/blender/NOTICE.md and src/capabilities/freecad/NOTICE.md for attribution.
