Assistant and voice on the tester cockpit and the admin dashboard #259

New issue

Open

opened 2026-06-07 00:21:42 +00:00 by mik-tf · 0 comments

mik-tf commented

2026-06-07 00:21:42 +00:00

Owner

End state:

Tester cockpit: it already has the voice bar top right. Add the assistant there too, so a tester can chat or speak to it and it can act on their own apps.
Admin dashboard: add both the assistant and the voice bar (the voice bar is the same drop-in piece already on the cockpit). Here the assistant can run deployer actions for the operator, for example "add a tester and set them up on node 0001" or "show me which machines have an update available".

Safety: the assistant runs in one of two modes, like the assistant we already ship. In confirmation mode, the default on the admin, it asks before any action that changes something or costs money, showing exactly what it is about to do so a mishearing is caught before anything happens. In auto mode it runs actions without asking. Reading information is always allowed without asking.

What we already have: the assistant is already a reusable web component with the ask-before-acting mode and the ability to use tools through its tool interface (MCP). The voice bar is already a drop-in. So the work is wiring and verification, not building.

Open questions to settle in the first session:

Does the assistant component embed cleanly into another page top right, or does it need a compact mode.
Does it show the confirmation prompt in that embedded form, and can we set the default mode per surface.
Exposing the deployer actions to the assistant as tools through the same tool interface it already uses.

Related: #256

Signed-by: mik-tf mik-tf@noreply.invalid

Goal: put the Hero assistant on both the tester cockpit and the admin dashboard, top right, so a person can ask it to do things in plain language or by voice. This reuses the assistant we already ship to testers, so it is an assembly job rather than a new build. End state: - Tester cockpit: it already has the voice bar top right. Add the assistant there too, so a tester can chat or speak to it and it can act on their own apps. - Admin dashboard: add both the assistant and the voice bar (the voice bar is the same drop-in piece already on the cockpit). Here the assistant can run deployer actions for the operator, for example "add a tester and set them up on node 0001" or "show me which machines have an update available". Safety: the assistant runs in one of two modes, like the assistant we already ship. In confirmation mode, the default on the admin, it asks before any action that changes something or costs money, showing exactly what it is about to do so a mishearing is caught before anything happens. In auto mode it runs actions without asking. Reading information is always allowed without asking. What we already have: the assistant is already a reusable web component with the ask-before-acting mode and the ability to use tools through its tool interface (MCP). The voice bar is already a drop-in. So the work is wiring and verification, not building. Open questions to settle in the first session: - Does the assistant component embed cleanly into another page top right, or does it need a compact mode. - Does it show the confirmation prompt in that embedded form, and can we set the default mode per surface. - Exposing the deployer actions to the assistant as tools through the same tool interface it already uses. Related: https://forge.ourworld.tf/lhumina_code/home/issues/256 Signed-by: mik-tf <mik-tf@noreply.invalid>