OrbitPilot chat widget — streaming, movable, mobile & voice

What it is

OrbitPilot is now a global floating chat widget for logged-in users: a 🪐 button in the corner opens a panel you can use from any page. The panel is non-blocking (keep browsing while it's open), draggable, collapsible, and it remembers its place and your open conversation as you move around the site.

New abilities

• Streaming answers — the reply appears word-by-word instead of after a wait.
• Stop — halt the AI mid-answer. Stopping saves whatever was written so far and does not use up an ask from your monthly allowance.
• Copy / Download the whole chat — buttons at the top and bottom of the panel (download is a .txt transcript).
• Copy a single message — hover any bubble for a copy icon.
• Maximize the window, browse your chat history, and start a New conversation.
• Voice input — tap the microphone and talk; your words are typed for you.

Guests (not logged in) keep the original home-page chat pop-up.

On your phone (mobile)

• The panel docks to the bottom of the screen and never spills past the edges, in any orientation — rotating the phone re-fits it automatically.
• It respects the notch and the home-indicator (safe areas), and uses the live screen height so the browser's address bar can't clip the Send button.
• After you speak and send, the box is emptied for your next question — tap the microphone again to talk for the next turn.

For developers

Streaming requires an async view under ASGI (a sync generator buffers before the first byte). orbitpilot.views.orbitpilot_ask_stream is an async def returning StreamingHttpResponse over an async generator that emits Server-Sent Events (meta → token… → done). All ORM/gating runs in two SYNC helpers (_prepare_ask_stream / _finalize_ask_stream) via sync_to_async(thread_sensitive=True). Stop = the client aborts the fetch → the async generator is cancelled → the partial answer is saved and the quota row is NOT written. Frontend lives in templates/_orbitpilot_widget.html + static/css/orbitpilot-widget.css + static/js/orbitpilot-widget.js, included in base.html for authed users only.

Mobile responsiveness (Workstream E): the whole widget is box-sizing: border-box so the 1px border counts inside the width (the old iPhone overflow was width:100vw + border). The @media (max-width:575px) rule drops 100vw for left/right: env(safe-area-inset-*) + width:auto, sizes with dvh (with a vh fallback), and pads the composer by env(safe-area-inset-bottom). In JS, a saved desktop drag position is not applied on a mobile viewport, and a resize/orientationchange listener re-clamps the position via applyPos(). Voice-clear: the speech module keeps a finalized saved buffer separate from the input; send() now calls a shared resetVoice() (stop recognition, clear the silence timer, blank saved) right where it clears input.value, so the next turn starts empty.

Chat font size + mobile Send fix (2026-07-27)

• A− / A+ in the widget header shrink/grow the chat text. Your choice is remembered (saved in the browser via localStorage key op_widget_font) across pages and sessions. Size is clamped to 12–22px.
• Send button on phones: the composer row now keeps the Send (and mic) buttons visible next to the text box on narrow screens — previously the text box could push Send off the right edge.

For developers: markup templates/_orbitpilot_widget.html, behavior static/js/orbitpilot-widget.js (applyFont()/bumpFont(), restored on init), styling static/css/orbitpilot-widget.css. The buttons bump a --op-chat-font CSS var read by the message body + input; the composer uses min-width:0 on the textarea and flex:0 0 auto on the buttons so Send never gets shoved off-screen.

Hamburger menu + full-page mode (2026-07-27)

The chat widget header now has a ☰ menu and a ⤢ full-page button:

• Your instructions — standing notes the AI always considers (like ChatGPT custom instructions). Saved to OrbitPilotSettings.custom_user_prompt via orbitpilot_custom_instructions; plan-gated by the custom_prompt feature (a locked message + upgrade link appears if the plan doesn't allow it; the server also enforces it with a 403).
• Downloads — now clearly two options: “This conversation only” (orbitpilot_conversation_download) vs “All my chat history” (new orbitpilot_history_download). The toolbar button is relabelled “⭳ This chat”.
• Memory → Rebuild my memory now — a full rebuild (orbitpilot_rebuild_memory) behind a confirm cost-warning (it re-reads everything and re-embeds, which uses AI credits); it won't stack a rebuild while one is genuinely already running. (BL-404, 2026-09-07) If that "already running" build actually crashed or was killed by a server restart — stuck in planned or running well past a safe window (10 minutes with the background worker on, 3 minutes without it) — pressing the button now clears it automatically and starts a fresh rebuild, using the same reaper the Booklet Inspector has always used (orbitpilot.services.reap_stuck_builds). The response says a stuck build was cleared rather than pretending nothing happened. Before this fix a dead build in that state blocked Refresh forever, with no way to unstick it short of an admin opening the Inspector.

Full-page mode (⤢) opens /orbitpilot/chat/ in a new tab, a ChatGPT/Gemini-style full-screen chat that reuses the SAME widget (via body class op-fullpage) — so switching between the small widget and the full page keeps the same conversation (shared op_widget_conv localStorage + a storage event listener). Also: A− / A+ change the chat font (remembered), and on a phone the Send button stays visible next to the mic.

For developers: markup templates/_orbitpilot_widget.html, behavior static/js/orbitpilot-widget.js, styling static/css/orbitpilot-widget.css; full-page view orbitpilot.views.orbitpilot_chat → orbitpilot/templates/orbitpilot/chat_full.html.

Quick update vs Full rebuild (soft/hard memory, 2026-07-27)

The widget's 🧠 Memory menu now offers two rebuilds:

• ⚡ Quick update (soft) — re-reads and re-embeds only the items you've changed since the last build; unchanged items keep their existing embeddings, removed items are dropped. Much cheaper, because the OpenAI embedding cost scales with the delta, not the whole corpus.
• 🔄 Full rebuild (hard) — re-reads and re-embeds everything from scratch (uses more AI credits). Use it only if something looks wrong.

For developers: run_memory_build(user, settings, build, incremental=True) diffs the new chunks against the stored ones by the (source_key, source_id, chunk_index, content_hash) key before calling embed_chunks, so only new/changed chunks are embedded; stale rows are deleted in the same atomic block; unchanged rows are left untouched. The mode is carried on OrbitPilotMemoryBuild.incremental (migration orbitpilot/0025) and set by orbitpilot_rebuild_memory from mode=soft|hard. If incremental is omitted, run_memory_build reads it from the build record. The full/hard path keeps the exact prior atomic delete-all + re-embed behaviour.

Chat-driven fixes (2026-07-28)

Analysing real (PII-redacted) OrbitPilot conversations surfaced four fixes:

• Broken AI links (your-app-link): the model invented a domain when handed relative paths from a stale booklet. Now (a) a System Health check warns if SITE_URL is localhost in production, and (b) orbitpilot.services.heal_links() rewrites any placeholder/localhost link base to the real site host in every answer (stream/blocking/guest) — so links work even before a rebuild. The permanent fix (set SITE_URL + rebuild) is on the Admin Setup Checklist.
• “List my Blinks” now works: pipeline/retrieve.py add_list_intent_sources() injects ALL of a source's chunks when the query is an enumerate request ("list/show/all my blinks/tasks/events"), and the workspace snapshot samples Blink titles — so the AI lists actual items, not just the count.
• Clickable next-actions: actions are now {label, url}. A real navigation link renders as a clickable button (opens in a new tab); otherwise it's a pre-fill chip. Fixed across the widget, the guest home modal (which previously dropped actions), and the full-page result. Inline answer links also open in a new tab now.
• Prompt polish: the core answer prompt stops over-repeating the user's headline goal and handles "what changed since yesterday" gracefully (it works from the current snapshot, not day-over-day history).

For developers: orbitpilot/services.py (heal_links, request_base, split_answer_actions now dict-returning, guest next_actions layer), pipeline/retrieve.py (add_list_intent_sources), adminapp/diagnostics.py (_check_site_url), widget JS + home.html (action render + DOMPurify target=_blank hook), aihub/prompt_registry.py (core + next_actions defaults; refreshed on untouched seeded Defaults via aihub/0007).

One toolbar, not two (2026-08-28)

What changed. The widget used to draw the same row of buttons — New · History · Copy · This chat — twice: once directly under the purple header, and again below the message box. There is now one, below the message box.

Why. Report #136, from a phone. Two rows of chrome inside an already short window left the conversation with what was over. The report was about how little room the conversation had.

Why the bottom row is the one that stayed. It is the one a thumb reaches on a phone, it sits beside the box you type in — which is where "start a new chat" and "copy this chat" belong — and it already carried every button the other one had. The top row's tooltips were the clearer wording, so they came down with it.

Nothing became unreachable. The header keeps its own ✕, so a collapsed widget still closes; and Close stays outside the sign-in gate, so a guest can still shut the widget. Guarded by orbitpilot/test_widget_toolbar.py.

The microphone: one intent, one session (BL-447, 2026-09-09)

The owner reported the chat microphone as "the first time working, then the second time it's not working, and sometimes it's stucking". Production was already carrying the previous fix when he said it, so this was what remained underneath it, not a missed deploy. Three things were wrong, and they explain the three different symptoms.

1. The button could mean the opposite of what it showed

The click handler decided what to do by reading one variable (wantListening, the user's request) while the red pulsing state was painted from a different one (listening, the engine's actual state). The browser's stop() call is asynchronous, so after the six-second silence timeout and after every Send there is a window where the request is already off while the engine is still running — and the button still looks active. A press meant as stop was therefore read as start, and the previous fix (correctly, for its own case) kept that press alive so the engine restarted. Pressing stop turned the microphone back on. The mirror image of the same gap made a second press cancel the user's own first press, so the microphone never started at all — that is the "second time it's not working".

Now there is one variable. micOn is what the user asked for; the button is drawn from it and the click reads it, so they cannot disagree.

2. One recognition object served the whole page

A single SpeechRecognition was built when the page loaded and reused forever. That is the only reason starting could fail with an "invalid state" error at all, and an object wedged by a network blip or a microphone held by another application stayed wedged until the page was reloaded — "works once, then never again". Every other microphone on the site (the homepage capture box, the bug reporter, admin dictation) already builds a fresh one per press. The chat widget now does the same, so each press gets a clean engine.

3. Closing the window did not release the microphone

close() never stopped recognition and nothing stopped it when the page was left, so closing the chat mid-dictation left the engine running behind a hidden panel: the browser's recording indicator stayed lit, the restart loop kept re-arming, and words kept landing in a box nobody could see. That is the "stucking". Closing the window and leaving the page both release the microphone now.

What else changed that you can see

• Failures say what happened. Blocked permission, no microphone found, and "that press did not start anything" used to look identical — a grey button and the ordinary placeholder. Each now writes its own short message in the message box.
• A broken engine gives up instead of spinning. Dictation still restarts itself through a thinking pause, which is the behaviour every microphone here shares, but a session that keeps dropping without ever hearing speech stops after a handful of tries and says so. Real speech refills that budget, so ordinary dictation never runs out.
• The chat box only takes the keyboard when it is on screen. Dictation ending used to focus the input unconditionally, which could pull the keyboard to a window that had just been closed.

For developers. The dying session is orphaned — the current-session reference is set to null and every handler checks its own identity before touching anything — so a late final result after Send is ignored because it belongs to a session nobody owns any more. That keeps the guarantee the two earlier fixes were reaching for without a flag that can be left armed. The logic lives in static/js/orbitpilot-widget.js (the initVoice block) and the shape is pinned by adminapp/test_widget_mic.py and adminapp/test_voice_parity.py. Browser speech cannot run in the test suite; a real-Chromium smoke test is a separate proposed item.