Add drawing tools, Mermaid support, and MCP server bridge

- Convert flat project to Cargo workspace (crates/agcanvas, crates/agcanvas-mcp) - Add drawing layer: rectangles, ellipses, lines, arrows, text with select/move/resize - Add Mermaid diagram rendering via mermaid-rs-renderer - Add agcanvas-mcp: MCP server bridge for Claude Code, OpenCode, and Codex - Add toolbar UI with keyboard shortcuts (V/R/E/L/A/T) and shape interaction - Add example MCP configs for Claude Code, OpenCode, and Codex - Update README with full feature docs, MCP setup, and updated architecture
2026-02-08 22:49:24 +01:00
parent 732e205943
commit d248864ee2
32 changed files with 2833 additions and 733 deletions
--- a/README.md
+++ b/README.md
@@ -1,38 +1,49 @@
 # agcanvas

-A system-level interactive canvas for agent-human collaboration. Paste SVGs from Figma, get structured understanding, iterate with AI agents.
+A system-level interactive canvas for agent-human collaboration. Draw shapes, paste SVGs from Figma, render Mermaid diagrams, and let AI agents understand your designs via MCP or WebSocket.

 ## What is this?

-agcanvas bridges the gap between visual design and code generation. It's not a design tool—it's a **feedback tool** for rapid iteration between humans and AI agents.
+agcanvas bridges the gap between visual design and code generation. It's a **collaboration surface** for humans and AI agents — draw your ideas, paste designs, and let agents see exactly what's on the canvas.

 ```
-┌─────────────────────────────────────────────────────────┐
-│  Figma                    agcanvas                      │
-│  ┌─────┐   Copy SVG    ┌──────────────────────────────┐ │
-│  │     │ ───────────►  │  Canvas (pan/zoom)           │ │
-│  │Frame│               │  ┌────────┐  ┌────────┐     │ │
-│  │     │               │  │ Parsed │  │ Agent  │     │ │
-│  └─────┘               │  │  Tree  │  │ Server │     │ │
-│                        │  └────────┘  └───┬────┘     │ │
-│                        └──────────────────┼──────────┘ │
-│                                           │            │
-│  AI Agent ◄───── WebSocket (JSON) ────────┘            │
-│  - Sees structure                                      │
-│  - Describes semantically                              │
-│  - Generates code                                      │
-└─────────────────────────────────────────────────────────┘
+┌──────────────────────────────────────────────────────────────┐
+│  Figma / Draw / Mermaid         agcanvas                     │
+│  ┌─────┐  ┌─────┐  ┌─────┐   ┌──────────────────────────┐   │
+│  │ SVG │  │Shapes│  │Merm.│   │  Canvas (pan/zoom)       │   │
+│  │Paste│  │ Draw │  │ Diag│   │  ┌────────┐ ┌────────┐  │   │
+│  └──┬──┘  └──┬───┘  └──┬──┘   │  │ Parsed │ │ Agent  │  │   │
+│     └────────┴─────────┘      │  │  Tree  │ │ Server │  │   │
+│              │                │  └────────┘ └───┬────┘  │   │
+│              ▼                └──────────────────┼──────┘   │
+│         Visual Canvas                           │          │
+│                                                 │          │
+│  AI Agent ◄──── MCP (stdio) ◄── agcanvas-mcp ──┘          │
+│  AI Agent ◄──── WebSocket (JSON) ───────────────┘          │
+│  - Sees structure (element tree)                           │
+│  - Reads drawing shapes                                    │
+│  - Generates code                                          │
+└──────────────────────────────────────────────────────────────┘
 ```

 ## Features

+### Canvas & Drawing
 - **SVG Paste** — Copy frames from Figma, paste directly (Cmd+V)
- **Structure Parsing** — SVG → typed element tree (groups, rects, circles, paths, text, images)
- **Semantic Description** — Auto-generates human-readable structure description
- **Agent Protocol** — WebSocket server for AI agents to query and understand the canvas
- **Code Generation** — Stubs for React, HTML, Tailwind, Svelte, Vue
+- **Shape Drawing** — Rectangles, ellipses, lines, arrows, text directly on canvas
+- **Selection & Editing** — Select, move, resize shapes with corner handles
+- **Mermaid Diagrams** — Write Mermaid syntax, render as SVG on canvas
+- **Sessions/Tabs** — Multiple canvases in tabs, each with independent state
 - **Pan/Zoom** — Smooth canvas navigation

+### AI Agent Integration
+- **MCP Server** — `agcanvas-mcp` bridge for Claude Code, OpenCode, and Codex
+- **WebSocket Protocol** — Direct JSON API on `ws://127.0.0.1:9876`
+- **Structure Parsing** — SVG → typed element tree (groups, rects, circles, paths, text, images)
+- **Semantic Description** — Human-readable canvas description for LLMs
+- **Bidirectional Events** — Push notifications to agents when canvas state changes
+- **Code Generation** — Stubs for React, HTML, Tailwind, Svelte, Vue
+
 ## Installation

 ### From source
@@ -41,9 +52,12 @@ agcanvas bridges the gap between visual design and code generation. It's not a d
 git clone https://github.com/yourusername/agcanvas.git
 cd agcanvas
 cargo build --release
-./target/release/agcanvas
 ```

+This builds two binaries:
+- `target/release/agcanvas` — The desktop app
+- `target/release/agcanvas-mcp` — The MCP server bridge
+
 ### Requirements

 - Rust 1.70+
@@ -55,35 +69,93 @@ cargo build --release

 1. **Open agcanvas**
   ```bash
-   cargo run --release
+   cargo run --release -p agcanvas
   ```

-2. **Copy SVG from Figma**
-   - Select a frame in Figma
-   - Right-click → Copy as SVG (or Cmd+C)
+2. **Draw shapes** — Select a tool from the toolbar (or press a shortcut key) and drag on the canvas

-3. **Paste into agcanvas**
-   - Cmd+V (or File → Paste SVG)
+3. **Paste SVG from Figma** — Copy a frame in Figma, then Cmd+V

-4. **Navigate**
-   - **Pan**: Middle-click drag, or Cmd+drag
-   - **Zoom**: Scroll wheel
-   - **Reset**: Cmd+0
+4. **Render Mermaid** — Click the Mermaid button in the toolbar, write your diagram, click Render

-5. **Inspect**
-   - View → Element Tree (hierarchical structure)
-   - View → Description (semantic text)
+5. **Navigate** — Pan (middle-click drag or Cmd+drag), Zoom (scroll wheel), Reset (Cmd+0)

 ### Keyboard shortcuts

 | Action | Shortcut |
 |--------|----------|
+| Select tool | V |
+| Rectangle tool | R |
+| Ellipse tool | E |
+| Line tool | L |
+| Arrow tool | A |
+| Text tool | T |
+| Delete selected | Delete / Backspace |
+| Cancel / back to Select | Escape |
 | Paste SVG | Cmd+V |
+| New Tab | Cmd+T |
+| Close Tab | Cmd+W |
 | Reset zoom | Cmd+0 |

-## Agent Protocol
+## MCP Server (AI Agent Integration)

-agcanvas exposes a WebSocket server on `ws://127.0.0.1:9876` for AI agents to interact with the canvas.
+`agcanvas-mcp` is a standalone MCP server that bridges AI coding tools to the agcanvas desktop app. It communicates with agcanvas over WebSocket and exposes canvas data as MCP tools.
+
+### Setup for Claude Code
+
+Add to your project's `.mcp.json` (or `~/.claude/mcp.json` for global):
+
+```json
+{
+  "mcpServers": {
+    "agcanvas": {
+      "command": "agcanvas-mcp",
+      "args": ["--port", "9876"]
+    }
+  }
+}
+```
+
+### Setup for OpenCode
+
+Add to your `opencode.json`:
+
+```json
+{
+  "mcpServers": {
+    "agcanvas": {
+      "command": "agcanvas-mcp",
+      "args": ["--port", "9876"]
+    }
+  }
+}
+```
+
+### Setup for Codex
+
+Same MCP config format — add the `agcanvas` entry to your Codex MCP configuration.
+
+> **Note:** Make sure `agcanvas-mcp` is in your PATH, or use the full path to the binary (e.g., `/path/to/target/release/agcanvas-mcp`). agcanvas must be running for the MCP tools to work.
+
+See [`examples/mcp-configs/`](examples/mcp-configs/) for ready-to-copy configuration files.
+
+### MCP Tools
+
+| Tool | Description |
+|------|-------------|
+| `list_sessions` | List all open tabs/sessions in agcanvas |
+| `get_element_tree` | Get the full parsed SVG element tree (structured JSON) |
+| `describe_canvas` | Get a human-readable description of the canvas |
+| `get_element_by_id` | Look up a specific element by ID |
+| `get_elements_at_point` | Find elements at an (x, y) coordinate |
+| `get_drawing_elements` | Get all user-drawn shapes (rects, ellipses, lines, arrows, text) |
+| `generate_code` | Generate code stubs (html, react, tailwind, svelte, vue) |
+
+All tools accept an optional `session_id` parameter. If omitted, the active session is used.
+
+## WebSocket Protocol
+
+agcanvas also exposes a direct WebSocket server on `ws://127.0.0.1:9876` for custom integrations.

 ### Connecting

@@ -96,45 +168,56 @@ ws = websocket.create_connection("ws://127.0.0.1:9876")

 ### Requests

-#### Get full element tree
+All requests support an optional `session_id` parameter. If omitted, the active session is used.
+
+#### List sessions

 ```json
-{"type": "GetTree"}
+{"type": "ListSessions"}
 ```

 Response:
 ```json
 {
-  "type": "Tree",
-  "tree": {
-    "root": {
-      "id": "frame-1",
-      "kind": {"type": "Group", "name": "Login Form"},
-      "bounds": {"x": 0, "y": 0, "width": 400, "height": 600},
-      "children": [...]
-    },
-    "metadata": {
-      "source": "svg_paste",
-      "width": 400,
-      "height": 600,
-      "element_count": 15
-    }
-  }
+  "type": "Sessions",
+  "sessions": [
+    {"id": "session-1", "name": "Tab 1", "has_svg": true, "element_count": 15},
+    {"id": "session-2", "name": "Tab 2", "has_svg": false, "element_count": null}
+  ],
+  "active_session": "session-1"
 }
 ```

+#### Get full element tree
+
+```json
+{"type": "GetTree"}
+{"type": "GetTree", "session_id": "session-2"}
+```
+
 #### Get semantic description

 ```json
 {"type": "Describe"}
 ```

-Response:
+#### Get drawing elements (user-drawn shapes)
+
 ```json
-{
-  "type": "Description",
-  "text": "- Group 'Login Form'\n  - Rectangle (400x600) fill=#ffffff\n  - Text 'Welcome Back' (24px)\n  - Rectangle (320x48) fill=#f0f0f0\n  - Text 'Email' (14px)\n  ..."
-}
+{"type": "GetDrawingElements"}
+{"type": "GetDrawingElements", "session_id": "session-1"}
+```
+
+#### Get element by ID
+
+```json
+{"type": "GetElementById", "id": "button-primary"}
+```
+
+#### Query elements at point
+
+```json
+{"type": "GetElementsAtPoint", "x": 150.0, "y": 200.0}
 ```

 #### Generate code
@@ -145,33 +228,25 @@ Response:

 Targets: `html`, `react`, `tailwind`, `svelte`, `vue`

-Response:
-```json
-{
-  "type": "Code",
-  "code": "// Generated from SVG...\nexport function Component() {\n  return (\n    <div className=\"container\">\n      {/* TODO: Implement based on structure */}\n    </div>\n  );\n}",
-  "target": "react"
-}
-```
-
-#### Query elements at point
-
-```json
-{"type": "GetElementsAtPoint", "x": 150.0, "y": 200.0}
-```
-
-#### Get element by ID
-
-```json
-{"type": "GetElementById", "id": "button-primary"}
-```
-
 #### Ping

 ```json
 {"type": "Ping"}
 ```

+### Events (GUI → Agent)
+
+agcanvas pushes events to all connected agents when state changes:
+
+| Event | Trigger |
+|-------|---------|
+| `Connected` | Agent connects (includes current session state) |
+| `SessionCreated` | New tab created |
+| `SessionClosed` | Tab closed |
+| `SessionActivated` | User switches tabs |
+| `SvgLoaded` | SVG pasted or loaded |
+| `SvgCleared` | Canvas cleared |
+
 ### Element types

 ```rust
@@ -190,20 +265,33 @@ enum ElementKind {
 ## Architecture

 ```
-src/
-├── main.rs              # Entry point, window setup
-├── app.rs               # Main application state, UI rendering
-├── element_tree.rs      # Structured element representation
-├── clipboard.rs         # System clipboard integration
-├── canvas/
-│   ├── state.rs         # Pan/zoom transformation state
-│   └── interaction.rs   # Mouse/keyboard input handling
-├── svg/
-│   ├── parser.rs        # SVG → ElementTree conversion
-│   └── renderer.rs      # SVG → pixels (resvg/tiny-skia)
-└── agent/
-    ├── protocol.rs      # JSON message types
-    └── server.rs        # WebSocket server
+crates/
+├── agcanvas/                 # Desktop app
+│   └── src/
+│       ├── main.rs           # Entry point, window setup
+│       ├── app.rs            # Main app state, UI, toolbar, drawing interaction
+│       ├── session.rs        # Session/tab state management
+│       ├── element_tree.rs   # Structured element representation
+│       ├── clipboard.rs      # System clipboard integration
+│       ├── mermaid.rs        # Mermaid → SVG rendering
+│       ├── drawing/
+│       │   ├── element.rs    # DrawingElement, Shape, ShapeStyle, hit testing
+│       │   ├── tool.rs       # Tool enum, DragState, ResizeHandle
+│       │   └── render.rs     # Shape rendering via egui Painter
+│       ├── canvas/
+│       │   ├── state.rs      # Pan/zoom transformation state
+│       │   └── interaction.rs # Mouse/keyboard input handling
+│       ├── svg/
+│       │   ├── parser.rs     # SVG → ElementTree conversion
+│       │   └── renderer.rs   # SVG → pixels (resvg/tiny-skia)
+│       └── agent/
+│           ├── protocol.rs   # JSON message types
+│           └── server.rs     # WebSocket server
+└── agcanvas-mcp/             # MCP server bridge
+    └── src/
+        ├── main.rs           # CLI entry point, stdio transport
+        ├── bridge.rs         # WebSocket client → agcanvas
+        └── tools.rs          # MCP tool definitions
 ```

 ### Dependencies
@@ -213,19 +301,26 @@ src/
 | `eframe`/`egui` | GUI framework |
 | `usvg` | SVG parsing |
 | `resvg`/`tiny-skia` | SVG rendering |
+| `mermaid-rs-renderer` | Mermaid → SVG |
 | `arboard` | Clipboard access |
-| `tokio-tungstenite` | WebSocket server |
+| `tokio-tungstenite` | WebSocket (both server and client) |
+| `rmcp` | MCP server SDK (Anthropic official) |
 | `serde`/`serde_json` | Serialization |

 ## Roadmap

+- [x] Multi-frame support (sessions/tabs)
+- [x] Shape drawing (rectangle, ellipse, line, arrow, text)
+- [x] Selection, move, resize with handles
+- [x] Mermaid diagram rendering
+- [x] MCP server bridge for AI coding tools
 - [ ] Real code generation (not just stubs)
- [ ] Element selection on canvas
 - [ ] Agent draw commands (modify canvas from agent)
- [ ] Multi-frame support
 - [ ] Export to file
 - [ ] Diff view (before/after agent changes)
 - [ ] Plugin system for code generators
+- [ ] Undo/redo
+- [ ] Multi-select and group operations

 ## License