lhumina_code/hero_whiteboard

Fork 0

make it faster (webassembly) #5

New issue

Closed

opened 2026-04-06 07:28:06 +00:00 by despiegk · 2 comments

despiegk commented

2026-04-06 07:28:06 +00:00

Owner

now its little bit too slow

can we use dioxus?

remarks

keep using the sqlite, otherwise will get even slower
see how we dealt with unique links to whiteboards
especially if 2 people work together it gets very slow

remark 2

maybe only certain parts should be moved to dioxus now which are perf sensitive, because we use some heavy javascript stuff to let users collaborate on white boards
Thor uses on HUGE resolution 8K or so, should work with super large whiteboards, maybe we need to keep track on which part we work, and not always deal with full board, so only partial update?

now its little bit too slow - can we use dioxus? remarks - keep using the sqlite, otherwise will get even slower - see how we dealt with unique links to whiteboards - especially if 2 people work together it gets very slow remark 2 - maybe only certain parts should be moved to dioxus now which are perf sensitive, because we use some heavy javascript stuff to let users collaborate on white boards - Thor uses on HUGE resolution 8K or so, should work with super large whiteboards, maybe we need to keep track on which part we work, and not always deal with full board, so only partial update?

timur was assigned by despiegk

2026-04-06 07:28:11 +00:00

despiegk added this to the now milestone

2026-04-06 07:28:14 +00:00

timur commented

2026-04-06 13:59:32 +00:00

Owner

Performance Analysis & Improvement Strategy

After thorough analysis of the codebase, here are the identified bottlenecks and the implementation plan:

Identified Bottlenecks

1. Grid Rendering (canvas.js:46-82) — HIGH IMPACT

drawGrid() creates individual Konva.Circle nodes for every dot. At 8K resolution (7680x4320) with GRID_SIZE=20, that's potentially ~83,000 circle nodes being created, destroyed, and redrawn on every zoom/pan/resize.
Uses gridLayer.destroyChildren() + recreate on every call — GC pressure.
Called on: wheel zoom, drag pan, resize, setZoom — very frequent.

2. No Viewport Culling (objects.js / app.js) — HIGH IMPACT

object.list loads ALL board objects from SQLite and renders ALL of them as Konva nodes regardless of viewport.
On large boards (hundreds+ objects), Konva must track and hit-test all nodes even when off-screen.
No spatial indexing — finding visible objects is O(n).

3. Full Board Sync (sync.js:547-616) — MEDIUM-HIGH IMPACT

syncFromServer() fetches ALL objects via object.list, then iterates ALL local objects to reconcile.
Readonly mode polls every 5 seconds with full board refresh.
saveAll() serializes ALL objects and sends one giant object.batch_update.
object.batch_update handler (object.rs:126-196) locks the Mutex and does a SELECT+UPDATE per object (N+1 query pattern).

4. SQLite Single Mutex (main.rs:16-17) — MEDIUM IMPACT

pub db: std::sync::Mutex<rusqlite::Connection> — single connection, synchronous lock.
Every RPC call (including concurrent WebSocket-triggered saves from multiple users) blocks on this lock.
batch_update holds the lock for the entire loop of N objects with N SELECT + N UPDATE queries.

5. WebSocket Message Volume (sync.js / ws.rs) — MEDIUM IMPACT

Each object update sends a full JSON representation through WebSocket.
Throttling at 200ms helps but doesn't batch — sends N individual messages for N pending updates.
No message compression.
Broadcast channel capacity of 256 — could overflow with rapid multi-user edits.

6. Connector Re-computation (connectors.js) — LOW-MEDIUM IMPACT

Each connector registers dragmove and transform listeners on both endpoints.
closestAnchor() does 16 distance comparisons per connector per frame.
getAnchorPoints() calls getAbsolutePosition() + coordinate transforms each time.

Implementation Plan (Priority Order)

Phase 1: Grid Performance (immediate, biggest visual impact)

Replace individual Konva.Circle nodes with a single custom shape using sceneFunc that draws dots directly on the 2D canvas context.
This reduces the grid from ~83K nodes to 1 node.

Phase 2: Viewport Culling

Track visible viewport bounds.
Only add/show Konva nodes that intersect the viewport.
Hide (not destroy) off-screen nodes.
Implement a simple spatial hash grid for O(1) lookups.

Phase 3: Delta Sync

Add object.list_since RPC method that accepts a timestamp and returns only changed objects.
Change saveAll() to only send dirty objects.
Use SQLite updated_at index for efficient filtered queries.
Batch WebSocket broadcasts into single messages.

Phase 4: SQLite Connection Pool + Transaction Batching

Replace Mutex<Connection> with r2d2_sqlite connection pool.
Wrap batch_update in a SQLite transaction.
Use UPDATE ... SET ... WHERE id = ? without re-reading each object (partial updates).

Phase 5: WebSocket Optimization

Batch pending WebSocket messages into arrays.
Send delta-only updates (changed fields) instead of full objects.
Increase broadcast channel capacity.

Starting implementation on branch development_5.

## Performance Analysis & Improvement Strategy After thorough analysis of the codebase, here are the identified bottlenecks and the implementation plan: ### Identified Bottlenecks **1. Grid Rendering (canvas.js:46-82) — HIGH IMPACT** - `drawGrid()` creates individual `Konva.Circle` nodes for every dot. At 8K resolution (7680x4320) with GRID_SIZE=20, that's potentially **~83,000 circle nodes** being created, destroyed, and redrawn on every zoom/pan/resize. - Uses `gridLayer.destroyChildren()` + recreate on every call — GC pressure. - Called on: wheel zoom, drag pan, resize, setZoom — very frequent. **2. No Viewport Culling (objects.js / app.js) — HIGH IMPACT** - `object.list` loads ALL board objects from SQLite and renders ALL of them as Konva nodes regardless of viewport. - On large boards (hundreds+ objects), Konva must track and hit-test all nodes even when off-screen. - No spatial indexing — finding visible objects is O(n). **3. Full Board Sync (sync.js:547-616) — MEDIUM-HIGH IMPACT** - `syncFromServer()` fetches ALL objects via `object.list`, then iterates ALL local objects to reconcile. - Readonly mode polls every 5 seconds with full board refresh. - `saveAll()` serializes ALL objects and sends one giant `object.batch_update`. - `object.batch_update` handler (object.rs:126-196) locks the Mutex and does a SELECT+UPDATE per object (N+1 query pattern). **4. SQLite Single Mutex (main.rs:16-17) — MEDIUM IMPACT** - `pub db: std::sync::Mutex<rusqlite::Connection>` — single connection, synchronous lock. - Every RPC call (including concurrent WebSocket-triggered saves from multiple users) blocks on this lock. - `batch_update` holds the lock for the entire loop of N objects with N SELECT + N UPDATE queries. **5. WebSocket Message Volume (sync.js / ws.rs) — MEDIUM IMPACT** - Each object update sends a full JSON representation through WebSocket. - Throttling at 200ms helps but doesn't batch — sends N individual messages for N pending updates. - No message compression. - Broadcast channel capacity of 256 — could overflow with rapid multi-user edits. **6. Connector Re-computation (connectors.js) — LOW-MEDIUM IMPACT** - Each connector registers `dragmove` and `transform` listeners on both endpoints. - `closestAnchor()` does 16 distance comparisons per connector per frame. - `getAnchorPoints()` calls `getAbsolutePosition()` + coordinate transforms each time. ### Implementation Plan (Priority Order) **Phase 1: Grid Performance (immediate, biggest visual impact)** - Replace individual Konva.Circle nodes with a single custom shape using `sceneFunc` that draws dots directly on the 2D canvas context. - This reduces the grid from ~83K nodes to 1 node. **Phase 2: Viewport Culling** - Track visible viewport bounds. - Only add/show Konva nodes that intersect the viewport. - Hide (not destroy) off-screen nodes. - Implement a simple spatial hash grid for O(1) lookups. **Phase 3: Delta Sync** - Add `object.list_since` RPC method that accepts a timestamp and returns only changed objects. - Change `saveAll()` to only send dirty objects. - Use SQLite `updated_at` index for efficient filtered queries. - Batch WebSocket broadcasts into single messages. **Phase 4: SQLite Connection Pool + Transaction Batching** - Replace `Mutex<Connection>` with `r2d2_sqlite` connection pool. - Wrap `batch_update` in a SQLite transaction. - Use `UPDATE ... SET ... WHERE id = ?` without re-reading each object (partial updates). **Phase 5: WebSocket Optimization** - Batch pending WebSocket messages into arrays. - Send delta-only updates (changed fields) instead of full objects. - Increase broadcast channel capacity. Starting implementation on branch `development_5`.

timur referenced this issue from a commit

2026-04-06 14:09:16 +00:00

feat: performance improvements and Miro feature parity (#5, #6)

timur commented

2026-04-06 14:09:33 +00:00

Owner

Implementation Progress — Performance Improvements

Branch: development_5 | Commit: ed01544

Changes Made

1. Grid Rendering (~83K nodes → 1 node)

Replaced individual Konva.Circle nodes for each grid dot with a single custom Konva.Shape using sceneFunc
Draws directly on the canvas 2D context with a single beginPath()/fill() call
At 8K resolution this eliminates ~83,000 Konva node objects from memory

2. Viewport Culling & Debounced Redraws

Added viewport tracking that updates on stage drag/zoom
Grid redraws are throttled via requestAnimationFrame — only redraws when viewport actually changes
Added isInViewport(x, y, w, h) helper for future object-level culling
onViewportChange() listener system for modules that need viewport-aware behavior

3. WebSocket Message Batching

Rapid broadcasts are now coalesced into single {type:'batch', messages:[...]} messages instead of N individual sends
Receiver-side handles batch unwrapping recursively
Reduces WebSocket frame overhead during multi-object drags

4. Remote Cursor Presence

Full cursor presence system: broadcasts throttled cursor positions via WebSocket
Renders remote cursors as colored arrows with user name labels on the cursor layer
Auto-expires stale cursors after 5 seconds
Presence dots show connected users
Join/leave event handling

5. SQLite Transaction Batching

batch_update handler now wraps all updates in a single BEGIN IMMEDIATE / COMMIT / ROLLBACK transaction
New partial_update_object() function using COALESCE for efficient partial updates without re-reading each object (eliminates N SELECTs)

Files Changed

canvas.js — grid rendering, viewport tracking
sync.js — WS batching, cursor presence, image serialization
queries.rs — partial_update_object() with COALESCE
object.rs — transaction-wrapped batch_update

Remaining Opportunities

Connection pool (r2d2) for SQLite to reduce Mutex contention
Object-level viewport culling (hide off-screen Konva groups)
Delta sync (only fetch objects changed since last sync timestamp)
Spatial indexing for large boards

## Implementation Progress — Performance Improvements **Branch:** `development_5` | **Commit:** `ed01544` ### Changes Made #### 1. Grid Rendering (~83K nodes → 1 node) - Replaced individual `Konva.Circle` nodes for each grid dot with a single custom `Konva.Shape` using `sceneFunc` - Draws directly on the canvas 2D context with a single `beginPath()`/`fill()` call - At 8K resolution this eliminates ~83,000 Konva node objects from memory #### 2. Viewport Culling & Debounced Redraws - Added viewport tracking that updates on stage drag/zoom - Grid redraws are throttled via `requestAnimationFrame` — only redraws when viewport actually changes - Added `isInViewport(x, y, w, h)` helper for future object-level culling - `onViewportChange()` listener system for modules that need viewport-aware behavior #### 3. WebSocket Message Batching - Rapid broadcasts are now coalesced into single `{type:'batch', messages:[...]}` messages instead of N individual sends - Receiver-side handles batch unwrapping recursively - Reduces WebSocket frame overhead during multi-object drags #### 4. Remote Cursor Presence - Full cursor presence system: broadcasts throttled cursor positions via WebSocket - Renders remote cursors as colored arrows with user name labels on the cursor layer - Auto-expires stale cursors after 5 seconds - Presence dots show connected users - Join/leave event handling #### 5. SQLite Transaction Batching - `batch_update` handler now wraps all updates in a single `BEGIN IMMEDIATE` / `COMMIT` / `ROLLBACK` transaction - New `partial_update_object()` function using `COALESCE` for efficient partial updates without re-reading each object (eliminates N SELECTs) ### Files Changed - `canvas.js` — grid rendering, viewport tracking - `sync.js` — WS batching, cursor presence, image serialization - `queries.rs` — `partial_update_object()` with COALESCE - `object.rs` — transaction-wrapped batch_update ### Remaining Opportunities - Connection pool (r2d2) for SQLite to reduce Mutex contention - Object-level viewport culling (hide off-screen Konva groups) - Delta sync (only fetch objects changed since last sync timestamp) - Spatial indexing for large boards

timur closed this issue

2026-04-07 14:06:58 +00:00