Initial commit: agcanvas - interactive canvas for agent-human collaboration
- SVG paste from Figma with structure parsing (usvg) - Element tree representation (groups, rects, paths, text, images) - Canvas rendering with pan/zoom (egui + resvg + tiny-skia) - WebSocket agent protocol on port 9876 - Semantic description generation - Code generation stubs (React, HTML, Tailwind, Svelte, Vue) - Cross-platform Rust implementation
This commit is contained in:
David Ibia
232
README.md
Normal file
232
README.md
Normal file
@@ -0,0 +1,232 @@
|
||||
# agcanvas
|
||||
|
||||
A system-level interactive canvas for agent-human collaboration. Paste SVGs from Figma, get structured understanding, iterate with AI agents.
|
||||
|
||||
## What is this?
|
||||
|
||||
agcanvas bridges the gap between visual design and code generation. It's not a design tool—it's a **feedback tool** for rapid iteration between humans and AI agents.
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Figma agcanvas │
|
||||
│ ┌─────┐ Copy SVG ┌──────────────────────────────┐ │
|
||||
│ │ │ ───────────► │ Canvas (pan/zoom) │ │
|
||||
│ │Frame│ │ ┌────────┐ ┌────────┐ │ │
|
||||
│ │ │ │ │ Parsed │ │ Agent │ │ │
|
||||
│ └─────┘ │ │ Tree │ │ Server │ │ │
|
||||
│ │ └────────┘ └───┬────┘ │ │
|
||||
│ └──────────────────┼──────────┘ │
|
||||
│ │ │
|
||||
│ AI Agent ◄───── WebSocket (JSON) ────────┘ │
|
||||
│ - Sees structure │
|
||||
│ - Describes semantically │
|
||||
│ - Generates code │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Features
|
||||
|
||||
- **SVG Paste** — Copy frames from Figma, paste directly (Cmd+V)
|
||||
- **Structure Parsing** — SVG → typed element tree (groups, rects, circles, paths, text, images)
|
||||
- **Semantic Description** — Auto-generates human-readable structure description
|
||||
- **Agent Protocol** — WebSocket server for AI agents to query and understand the canvas
|
||||
- **Code Generation** — Stubs for React, HTML, Tailwind, Svelte, Vue
|
||||
- **Pan/Zoom** — Smooth canvas navigation
|
||||
|
||||
## Installation
|
||||
|
||||
### From source
|
||||
|
||||
```bash
|
||||
git clone https://github.com/yourusername/agcanvas.git
|
||||
cd agcanvas
|
||||
cargo build --release
|
||||
./target/release/agcanvas
|
||||
```
|
||||
|
||||
### Requirements
|
||||
|
||||
- Rust 1.70+
|
||||
- macOS / Linux / Windows
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic workflow
|
||||
|
||||
1. **Open agcanvas**
|
||||
```bash
|
||||
cargo run --release
|
||||
```
|
||||
|
||||
2. **Copy SVG from Figma**
|
||||
- Select a frame in Figma
|
||||
- Right-click → Copy as SVG (or Cmd+C)
|
||||
|
||||
3. **Paste into agcanvas**
|
||||
- Cmd+V (or File → Paste SVG)
|
||||
|
||||
4. **Navigate**
|
||||
- **Pan**: Middle-click drag, or Cmd+drag
|
||||
- **Zoom**: Scroll wheel
|
||||
- **Reset**: Cmd+0
|
||||
|
||||
5. **Inspect**
|
||||
- View → Element Tree (hierarchical structure)
|
||||
- View → Description (semantic text)
|
||||
|
||||
### Keyboard shortcuts
|
||||
|
||||
| Action | Shortcut |
|
||||
|--------|----------|
|
||||
| Paste SVG | Cmd+V |
|
||||
| Reset zoom | Cmd+0 |
|
||||
|
||||
## Agent Protocol
|
||||
|
||||
agcanvas exposes a WebSocket server on `ws://127.0.0.1:9876` for AI agents to interact with the canvas.
|
||||
|
||||
### Connecting
|
||||
|
||||
```python
|
||||
import websocket
|
||||
import json
|
||||
|
||||
ws = websocket.create_connection("ws://127.0.0.1:9876")
|
||||
```
|
||||
|
||||
### Requests
|
||||
|
||||
#### Get full element tree
|
||||
|
||||
```json
|
||||
{"type": "GetTree"}
|
||||
```
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"type": "Tree",
|
||||
"tree": {
|
||||
"root": {
|
||||
"id": "frame-1",
|
||||
"kind": {"type": "Group", "name": "Login Form"},
|
||||
"bounds": {"x": 0, "y": 0, "width": 400, "height": 600},
|
||||
"children": [...]
|
||||
},
|
||||
"metadata": {
|
||||
"source": "svg_paste",
|
||||
"width": 400,
|
||||
"height": 600,
|
||||
"element_count": 15
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Get semantic description
|
||||
|
||||
```json
|
||||
{"type": "Describe"}
|
||||
```
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"type": "Description",
|
||||
"text": "- Group 'Login Form'\n - Rectangle (400x600) fill=#ffffff\n - Text 'Welcome Back' (24px)\n - Rectangle (320x48) fill=#f0f0f0\n - Text 'Email' (14px)\n ..."
|
||||
}
|
||||
```
|
||||
|
||||
#### Generate code
|
||||
|
||||
```json
|
||||
{"type": "GenerateCode", "target": "react", "element_id": null}
|
||||
```
|
||||
|
||||
Targets: `html`, `react`, `tailwind`, `svelte`, `vue`
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"type": "Code",
|
||||
"code": "// Generated from SVG...\nexport function Component() {\n return (\n <div className=\"container\">\n {/* TODO: Implement based on structure */}\n </div>\n );\n}",
|
||||
"target": "react"
|
||||
}
|
||||
```
|
||||
|
||||
#### Query elements at point
|
||||
|
||||
```json
|
||||
{"type": "GetElementsAtPoint", "x": 150.0, "y": 200.0}
|
||||
```
|
||||
|
||||
#### Get element by ID
|
||||
|
||||
```json
|
||||
{"type": "GetElementById", "id": "button-primary"}
|
||||
```
|
||||
|
||||
#### Ping
|
||||
|
||||
```json
|
||||
{"type": "Ping"}
|
||||
```
|
||||
|
||||
### Element types
|
||||
|
||||
```rust
|
||||
enum ElementKind {
|
||||
Group { name: Option<String> },
|
||||
Rectangle { rx: Option<f32>, ry: Option<f32> },
|
||||
Circle { cx: f32, cy: f32, r: f32 },
|
||||
Ellipse { cx: f32, cy: f32, rx: f32, ry: f32 },
|
||||
Path { d: String },
|
||||
Text { content: String, font_size: f32 },
|
||||
Image { href: String },
|
||||
Line { x1: f32, y1: f32, x2: f32, y2: f32 },
|
||||
}
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
src/
|
||||
├── main.rs # Entry point, window setup
|
||||
├── app.rs # Main application state, UI rendering
|
||||
├── element_tree.rs # Structured element representation
|
||||
├── clipboard.rs # System clipboard integration
|
||||
├── canvas/
|
||||
│ ├── state.rs # Pan/zoom transformation state
|
||||
│ └── interaction.rs # Mouse/keyboard input handling
|
||||
├── svg/
|
||||
│ ├── parser.rs # SVG → ElementTree conversion
|
||||
│ └── renderer.rs # SVG → pixels (resvg/tiny-skia)
|
||||
└── agent/
|
||||
├── protocol.rs # JSON message types
|
||||
└── server.rs # WebSocket server
|
||||
```
|
||||
|
||||
### Dependencies
|
||||
|
||||
| Crate | Purpose |
|
||||
|-------|---------|
|
||||
| `eframe`/`egui` | GUI framework |
|
||||
| `usvg` | SVG parsing |
|
||||
| `resvg`/`tiny-skia` | SVG rendering |
|
||||
| `arboard` | Clipboard access |
|
||||
| `tokio-tungstenite` | WebSocket server |
|
||||
| `serde`/`serde_json` | Serialization |
|
||||
|
||||
## Roadmap
|
||||
|
||||
- [ ] Real code generation (not just stubs)
|
||||
- [ ] Element selection on canvas
|
||||
- [ ] Agent draw commands (modify canvas from agent)
|
||||
- [ ] Multi-frame support
|
||||
- [ ] Export to file
|
||||
- [ ] Diff view (before/after agent changes)
|
||||
- [ ] Plugin system for code generators
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
Reference in New Issue
Block a user