agcanvas/README.md

agcanvas

A system-level interactive canvas for agent-human collaboration. Paste SVGs from Figma, get structured understanding, iterate with AI agents.

What is this?

agcanvas bridges the gap between visual design and code generation. It's not a design tool—it's a feedback tool for rapid iteration between humans and AI agents.

┌─────────────────────────────────────────────────────────┐
│  Figma                    agcanvas                      │
│  ┌─────┐   Copy SVG    ┌──────────────────────────────┐ │
│  │     │ ───────────►  │  Canvas (pan/zoom)           │ │
│  │Frame│               │  ┌────────┐  ┌────────┐     │ │
│  │     │               │  │ Parsed │  │ Agent  │     │ │
│  └─────┘               │  │  Tree  │  │ Server │     │ │
│                        │  └────────┘  └───┬────┘     │ │
│                        └──────────────────┼──────────┘ │
│                                           │            │
│  AI Agent ◄───── WebSocket (JSON) ────────┘            │
│  - Sees structure                                      │
│  - Describes semantically                              │
│  - Generates code                                      │
└─────────────────────────────────────────────────────────┘

Features

• SVG Paste — Copy frames from Figma, paste directly (Cmd+V)
• Structure Parsing — SVG → typed element tree (groups, rects, circles, paths, text, images)
• Semantic Description — Auto-generates human-readable structure description
• Agent Protocol — WebSocket server for AI agents to query and understand the canvas
• Code Generation — Stubs for React, HTML, Tailwind, Svelte, Vue
• Pan/Zoom — Smooth canvas navigation

Installation

From source

git clone https://github.com/yourusername/agcanvas.git
cd agcanvas
cargo build --release
./target/release/agcanvas

Requirements

• Rust 1.70+
• macOS / Linux / Windows

Usage

Basic workflow

1. Open agcanvas

   cargo run --release
   

2. Copy SVG from Figma

   • Select a frame in Figma
   • Right-click → Copy as SVG (or Cmd+C)

3. Paste into agcanvas

   • Cmd+V (or File → Paste SVG)

4. Navigate

   • Pan: Middle-click drag, or Cmd+drag
   • Zoom: Scroll wheel
   • Reset: Cmd+0

5. Inspect

   • View → Element Tree (hierarchical structure)
   • View → Description (semantic text)

Keyboard shortcuts

Action	Shortcut
Paste SVG	Cmd+V
Reset zoom	Cmd+0

Agent Protocol

agcanvas exposes a WebSocket server on ws://127.0.0.1:9876 for AI agents to interact with the canvas.

Connecting

import websocket
import json

ws = websocket.create_connection("ws://127.0.0.1:9876")

Requests

Get full element tree

{"type": "GetTree"}

Response:

{
  "type": "Tree",
  "tree": {
    "root": {
      "id": "frame-1",
      "kind": {"type": "Group", "name": "Login Form"},
      "bounds": {"x": 0, "y": 0, "width": 400, "height": 600},
      "children": [...]
    },
    "metadata": {
      "source": "svg_paste",
      "width": 400,
      "height": 600,
      "element_count": 15
    }
  }
}

Get semantic description

{"type": "Describe"}

Response:

{
  "type": "Description",
  "text": "- Group 'Login Form'\n  - Rectangle (400x600) fill=#ffffff\n  - Text 'Welcome Back' (24px)\n  - Rectangle (320x48) fill=#f0f0f0\n  - Text 'Email' (14px)\n  ..."
}

Generate code

{"type": "GenerateCode", "target": "react", "element_id": null}

Targets: html, react, tailwind, svelte, vue

Response:

{
  "type": "Code",
  "code": "// Generated from SVG...\nexport function Component() {\n  return (\n    <div className=\"container\">\n      {/* TODO: Implement based on structure */}\n    </div>\n  );\n}",
  "target": "react"
}

Query elements at point

{"type": "GetElementsAtPoint", "x": 150.0, "y": 200.0}

Get element by ID

{"type": "GetElementById", "id": "button-primary"}

Ping

{"type": "Ping"}

Element types

enum ElementKind {
    Group { name: Option<String> },
    Rectangle { rx: Option<f32>, ry: Option<f32> },
    Circle { cx: f32, cy: f32, r: f32 },
    Ellipse { cx: f32, cy: f32, rx: f32, ry: f32 },
    Path { d: String },
    Text { content: String, font_size: f32 },
    Image { href: String },
    Line { x1: f32, y1: f32, x2: f32, y2: f32 },
}

Architecture

src/
├── main.rs              # Entry point, window setup
├── app.rs               # Main application state, UI rendering
├── element_tree.rs      # Structured element representation
├── clipboard.rs         # System clipboard integration
├── canvas/
│   ├── state.rs         # Pan/zoom transformation state
│   └── interaction.rs   # Mouse/keyboard input handling
├── svg/
│   ├── parser.rs        # SVG → ElementTree conversion
│   └── renderer.rs      # SVG → pixels (resvg/tiny-skia)
└── agent/
    ├── protocol.rs      # JSON message types
    └── server.rs        # WebSocket server

Dependencies

Crate	Purpose
eframe/egui	GUI framework
usvg	SVG parsing
resvg/tiny-skia	SVG rendering
arboard	Clipboard access
tokio-tungstenite	WebSocket server
serde/serde_json	Serialization

Roadmap

• Real code generation (not just stubs)
• Element selection on canvas
• Agent draw commands (modify canvas from agent)
• Multi-frame support
• Export to file
• Diff view (before/after agent changes)
• Plugin system for code generators

License

MIT