SnapBuddy
Sign inGet early access

Knowledge base

AI assist

The three AI features in the web app: what each one does, what leaves and to whom, who may ask, where the answers appear, and what happens when no provider is configured.

None of the shipped strings quoted here contains an em dash, so nothing is substituted; they are verbatim.

Three features, all of them in the web app

  • AI bug draft. A QA card written from what this site already holds about one capture, a screenshot's picture included: a title, a summary, steps, expected and actual (apps/web/src/lib/ai/bug-draft.ts).
  • AI title and summary. What the capture shows, from the same facts and the same picture. It is deliberately not a diagnosis: the system prompt tells the model to say what the capture shows and not to write steps to reproduce a defect (apps/web/src/lib/ai/summary.ts:63-72).
  • Help with a note. A rough note turned into what happened, what was expected, and how to see it again (apps/web/src/lib/ai/composer.ts).

Nothing else in the product reaches an AI provider. The desktop app and the extension reach none: the desktop's structured document never leaves the machine (DECISIONS.md, "core.js's structured document never leaves the machine"), and the extension reaches exactly one address, the SnapBuddy app, for exactly two things - signing in, and a picture you upload. Nothing is uploaded unless you press Upload on a card, and every other card stays in this browser. Neither of those two is a model: no capture, note or developer log is ever put in front of an AI provider from that surface (extension.md).

⚠ This paragraph used to deny the extension any network traffic whatsoever, a claim it had inherited from extension.md and which had been false for a release by the time anyone read it. The sentence above is now held word for word against the extension's own Settings row by an equality join in scripts/check-promises.mjs, rather than restated here in words of its own that nothing checks.

Exactly three modules under apps/web/src call the AI service, and scripts/check-promises.mjs counts them on every run and prints the list when the count disagrees with the privacy notice (AI_COUNT_SENTENCE and the module scan beside it - cited by construct, not by line, because the line numbers that used to stand here had drifted onto unrelated rules; joined-to-code).

The note helper is pressed from the Note panel on an asset's page, where the note editor mounts it (apps/web/src/components/note-editor.tsx).

Nothing is sent until somebody presses a button

There is no workspace setting that turns any of this on, nothing runs on upload, and no capture is sent because it was opened. Each feature is one press on one capture.

The draft and the summary are plain form submissions from the dashboard asset page, so they work with JavaScript off (the draftBugReport and summarizeAsset forms in apps/web/src/app/dashboard/a/[id]/page.tsx, both actions in apps/web/src/lib/ai/actions.ts). The buttons read "Write an AI draft" (apps/web/src/app/dashboard/a/[id]/page.tsx) and "Write an AI summary" (apps/web/src/app/dashboard/a/[id]/page.tsx), and "Write another AI draft" (apps/web/src/app/dashboard/a/[id]/page.tsx) or "Write another AI summary" (apps/web/src/app/dashboard/a/[id]/page.tsx) once one exists.

The note helper sends only from its own click handler. There is no effect, no timer, no blur handler and no keystroke handler in the component (apps/web/src/components/composer-assist.tsx:60 and :104; joined-to-code: apps/web/src/lib/ai/composer.test.ts:384-393 reads the component source and fails the build if a second call site, a useEffect, a setTimeout, a setInterval, an onBlur, an onChange or an onKeyUp appears).

Its disclosure sits above the button and describes the press before it happens, verbatim: "Nothing is sent until you press this. Pressing it sends the note exactly as you typed it, along with what this capture already stores, to the AI provider named in the privacy notice." (apps/web/src/components/composer-assist.tsx:94-101; joined-to-code: composer.test.ts:395-402 fails if that sentence moves below the button).

What leaves, and to whom

The provider is Anthropic. The request goes to https://api.anthropic.com/v1/messages and is made only when ANTHROPIC_API_KEY is set (packages/services/src/ai.ts:23, :56-59).

All three features are shown the same details, because there is one column list and one prompt builder and each feature uses both. What is sent about a capture: its title, type, status and triage state, duration, frame size, the marks that were drawn with the labels the person wrote on them, and, for a recording, whether a camera and system audio were switched on (loadCaptureFacts, apps/web/src/lib/ai/bug-draft.ts:308-350; buildCaptureContext, :202-231; used by summary.ts:91-93 and composer.ts:110-125).

A screenshot's picture goes too, for two of the three. In the privacy notice's own words: "For a screenshot, the bug draft and the title and summary also send the picture itself, with anything you redacted or greyed out already removed from it, so that what comes back can say what the picture shows." (apps/web/src/app/privacy/page.tsx) And: "Anything you blurred or pixelated is sent as it looks: Blur and Pixelate obscure what they cover without removing it." (apps/web/src/app/privacy/page.tsx) Help with a note does not send the picture.

The picture is read from storage with the store's own credentials, re-encoded as a JPEG that fits inside 1568 by 1568 pixels (which also drops metadata such as where a photo was taken), and sent as its bytes, never as an address for the provider to fetch (loadCapturePicture, apps/web/src/lib/ai/picture.ts). If it cannot be read, the press goes ahead from the details above alone, exactly as it did before pictures were sent. Redactions and grey-outs are already gone because the product bakes a screenshot's marks into the picture before it is stored, and the two API routes that could not check that refuse them: PUT /api/v1/assets/{id}/annotations takes marks for a recording only, and POST /api/v1/captures refuses a redact, grey out, blur, pixelate or filled rectangle mark (joined-to-code: scripts/check-promises.mjs fails the build when the notice's picture sentence and the code that sends the picture disagree, when any third module sends one, or when the request carries an address instead of bytes).

What is never sent, by any of the three: a video and its audio, an email address, a share link or token, a storage key or any other address of a stored file, a signed URL, an API key. The select in loadCaptureFacts names eight columns and omits storage_key, stream_uid, thumbnail_key and owner_id, and the input type has no field for any of them (bug-draft.ts, loadCaptureFacts and BugDraftInput; joined-to-code: bug-draft.test.ts, summary.test.ts and composer.test.ts each assert it against the built prompt string, not against the type). The picture's own read names storage_key to fetch the bytes, and the key goes no further.

On a recording, the position of a redact, grey out, blur or pixelate mark is withheld and the prompt says so, because those marks are painted over the player rather than burned into the file and the pixels are still there. On a screenshot the same marks are baked in, so their positions are stated (markLines in bug-draft.ts; joined-to-code: bug-draft.test.ts, "withholds an effect position on a recording and states it on a screenshot"). See annotating.md.

No caption text is sent by anything today, although the prompt can carry it: the WebVTT is not parsed into a stored transcript, so no caller has any (bug-draft.ts:98-106, :348).

The note is the exception, and it is the one to know about

Help with a note sends the details above, never the picture, plus the note itself, as typed. It is not summarised, trimmed to a topic or rewritten first; newlines are preserved because a note is written in lines (apps/web/src/lib/ai/composer.ts:110-125; joined-to-code: composer.test.ts:180). It is bounded at 4,000 characters and the prompt says "Truncated." when it hits that (apps/web/src/lib/ai/composer.ts).

Pressing the button with an empty note sends nothing and says so: "There is nothing to work from yet, so nothing was sent. Write a note first." (apps/web/src/lib/ai/composer.ts).

The suggestion is never stored. It sits beside the note until it is kept or discarded, and both answers keep the person's own words: discard returns the note byte for byte, and keep appends the suggestion underneath it rather than replacing it, so a mis-click costs a paragraph to delete rather than the note (composer.ts:238-246; joined-to-code: composer.test.ts:412-419, which was mutation-tested against a version that replaced the note).

Who may ask

An editor or above, and the check runs before the provider is called, not after. RLS would refuse the write anyway, but that refusal arrives on the insert, which is after the prompt has been built, sent and paid for, so a viewer holding the button would spend a model call per press (the gate in draftBugReport and in summarizeAsset, apps/web/src/lib/ai/actions.ts, and apps/web/src/lib/ai/composer.ts:163; joined-to-code: actions.test.ts:195 and :257, composer.test.ts:312, all three asserting nothing reached the model).

A viewer is told: "You do not have permission to write an AI draft here." (apps/web/src/lib/ai/actions.ts), "You do not have permission to write an AI summary here." (apps/web/src/lib/ai/actions.ts), or "You do not have permission to ask for help with a note here." (ASSIST_FORBIDDEN in apps/web/src/lib/ai/assist-text.ts).

Neither button appears on a capture in the trash (canWriteAi in apps/web/src/app/dashboard/a/[id]/page.tsx).

Where answers appear: workspace members only, never a share link

AI output is member-only by owner ruling (DECISIONS.md, "AI summary visibility: member-only"). A model's guess about a bug, on a page a stranger opens from a link with no member present to correct it, is not revocable once it has unfurled into a chat.

Both panels live in the dashboard asset page file, behind getSessionContext, and in no shared component the share viewer could import (the AI summary and AI draft panels in apps/web/src/app/dashboard/a/[id]/page.tsx). AI_DRAFT_REF in scripts/check-promises.mjs fails the build if ai_artifacts, bug-draft, BugDraft or AI_DRAFT appears in apps/web/src/app/s/[token]/page.tsx or apps/web/src/lib/share-access.ts (joined-to-code). See sharing.md for what a share link does show.

Every answer says a machine wrote it

One constant per feature, exported from the module that generates it and imported wherever it renders, so no surface can describe the same artifact in a different register. The badge and the sentence come before the model's own title, so the first thing read is that a machine wrote it.

  • Bug draft, badge "AI draft": "AI draft from this capture's own stored data. Check it before you file it." (DRAFT_NOTE, apps/web/src/lib/ai/bug-draft.ts:43; rendered in the AI draft panel of apps/web/src/app/dashboard/a/[id]/page.tsx).
  • Summary, badge "AI summary": "AI summary from this capture's own stored data. Check it before you rely on it." (SUMMARY_NOTE, apps/web/src/lib/ai/summary.ts:40; rendered in the AI summary panel of the same page).
  • Note suggestion, badge "AI suggestion": "AI suggestion, written from this capture and from the note you wrote. Check it before you file it." (ASSIST_NOTE, apps/web/src/lib/ai/assist-text.ts:47-48; rendered at composer-assist.tsx:120-125; joined-to-code: composer.test.ts:400 fails if the component stops rendering the constant).

The third one is a separate sentence rather than a reuse on purpose: the other two say the text came from the capture's own stored data, which for the note helper would be an understatement in the one direction that misleads, because the words the person typed went too (composer.ts:41-54). It is appended into the note when a suggestion is kept, not printed only on the panel, so the marker survives a paste into a ticket (composer.ts:244).

Beside each stored panel the page prints who wrote it and when, read from the row rather than baked into the sentence: "Written by", the stored name, "on", then the row's timestamp in UTC. The stored name is the service and not the model id, so it reads anthropic, because the AIService interface does not expose a model id (bug-draft.ts:420-422, and the Written by line in each panel of the asset page). A row written before a name was stored prints "an unnamed model" (apps/web/src/app/dashboard/a/[id]/page.tsx).

When no provider is configured

With no ANTHROPIC_API_KEY the service is a stub that answers {"stub":true} to any JSON request. No feature offers its button in this state: each prints its refusal where the button would have been, so nobody presses a button that can only fail (the asset page, apps/web/src/app/dashboard/a/[id]/page.tsx, for the draft and the summary; the note editor, apps/web/src/components/note-editor.tsx, for the note helper):

  • Bug draft: "No AI provider is configured here, so there is no draft to show." (apps/web/src/lib/ai/bug-draft.ts)
  • Summary: "No AI provider is configured here, so there is no summary to write." (SUMMARY_UNCONFIGURED, apps/web/src/lib/ai/summary.ts)
  • Note helper: "No AI provider is configured here, so nothing was sent and there is no suggestion to show." (ASSIST_UNCONFIGURED in apps/web/src/lib/ai/assist-text.ts)

If the model answers and the answer is not usable, nothing is stored and the sentence says so: "The model answered with something that is not a draft, so nothing was saved." (apps/web/src/lib/ai/bug-draft.ts), with its summary equivalent in summary.ts. If the request fails, the vendor's own error text is never shown, because it can carry the prompt back into a page: "The model did not answer, so there is no draft. Try again." (apps/web/src/lib/ai/bug-draft.ts), which summary.ts says the same way for the summary. The note helper says "The model did not answer, so there is no suggestion. Your note is untouched." (apps/web/src/lib/ai/composer.ts)

What is stored

A draft is one ai_artifacts row of kind repro; a summary is one row of kind summary, holding the title and the summary together so the two cannot drift apart (bug-draft.ts:423-435, summary.ts:191-203). Neither kind needed a migration: both values already exist in the ai_artifact_kind enum (supabase/migrations/0001_foundation.sql:15). Asking again appends a row and the panel shows the newest, so the buttons read "Write another" rather than replacing anything (apps/web/src/app/dashboard/a/[id]/page.tsx). A note suggestion is stored nowhere (composer.ts; joined-to-code: composer.test.ts:281).

The sentence a support agent can paste

"Nothing about your capture is sent to an AI provider unless somebody presses one of the AI buttons on it, and asking for help with a note also sends the note exactly as it was typed. What comes back is labelled as written by AI, is only visible to people signed in to your workspace, and never appears on a share link."