Progress

Living status of the build. Updated as work moves between buckets. For what shipped per release, see ../CHANGELOG.md. For why a decision was made, see decisions/.

Last updated: 2026-05-07 · Current version: v0.2.0-rc.1 · post-rc1: V0.2 mobile slice + audio transcode worker on Modal. Voice talk-back paused per ADR-0010. E2E pipeline test green (PR #10 merged 2026-05-06). Topic graph shipped end-to-end across web + mobile per ADR-0012 (PR #11 merged 2026-05-07).

Currently shipping

Summarize + topics: Gemini Flash primary, Anthropic Haiku fallback, Groq Llama 3.3 70B third fallback (ADR-0011 Layer 3). Backfill via supabase/functions/backfill-summaries/ enqueues summarize jobs for memories with null summary, null topics, or null topics_meta (the last clause picks up legacy pre-β.1.2 rows per ADR-0012).

Paused / deferred

Voice talk-back (web + mobile) — paused 2026-05-05 per ADR-0010. Pipecat service + /talkback web page stay in the repo behind a paused notice; resumption is a single revert PR. Reactivate when EAS unblocks mobile voice and the speech-to-speech-vs-Pipecat question has a concrete deciding signal.
MCP wiring on voice service — paused with the parent feature. MCP server itself is unaffected (web /api/chat still uses it).

Just shipped

Mobile Universe — Skia renderer + Obsidian-style layout (γ.1) — 2026-05-06 (feature/mobile-universe, 4 commits).

Renderer migration: react-native-svg → @shopify/react-native-skia. SVG was hitting a perf cliff around 150 bridged native views; Skia is GPU-backed via a single canvas surface. Bundled in Expo Go SDK 53+, no dev build needed (Expo Go constraint preserved).
Force model adopted from Obsidian’s graph view: continuous d3-force simulation with alphaTarget drift floor (never freezes), forceX/forceY for soft center pull, forceCollide so dots never overlap, node radius base + mult * sqrt(degree) for hub emphasis. Tuning derived from the “make my graph look better” Reddit thread (linkDistance 25, charge -50, center 0.2) plus experimentation: ambient peaceful drift (alphaTarget 0.03, velocityDecay 0.75) instead of jittery vibration.
Touch interactions: long-press to drag a node — pins via d3-force fx/fy, simulation re-heats so connected nodes follow via the link force (real physics, not faked animation). Tap a node to focus → 1-hop neighbours stay full-opacity, everything else dims (Obsidian’s signature trick). Tap again to navigate. Pinch has rubber-band overshoot via Reanimated withSpring. Filter selection persists across sessions via AsyncStorage.
“You” anchor: a real SimNode pinned to canvas centre via fx/fy, with faint cosmetic spokes radiating to every memory (no link force; render-only). Long-press-drag releases the pin temporarily; release re-pins to centre.
Globe silhouette: forceRadial pulls only orphan nodes (degree 0) toward an outer ring, leaving connected clusters undisturbed. At demo data scale (~150 connected + 50 orphans), this gives the recognisable Obsidian “core + ring” shape.
First-paint flow: 60 sync warm-up ticks bring the layout near equilibrium, then a single snap-fit places the bbox in the viewport. No animated re-fit later (one snap reads as “the graph appeared”, not “the graph zoomed”). Double-tap re-fits.
Demo data toggle synthesizes 8 clusters × 18 connected nodes
- 50 orphans for visual evaluation when real data is too sparse.

Topic graph end-to-end (ADR-0012 β.1 + β.2) — 2026-05-07 (PR #11, feature/topic-graph, 19 commits, ~3000 LOC).

β.1 — backend pipeline. topics + memory_topics schema with kind enum (person/place/project/theme/event) + transitional memories.topics_meta jsonb. New link-topics-step runs hybrid pg_trgm + pgvector resolution via resolve_topic SQL function. compute-edges-step writes topic-shared edges (jaccard) alongside semantic; memory_edges.kind distinguishes them via compound unique constraint. backfill-summaries extended to pick up legacy topics_meta IS NULL rows.
β.2 — navigation surfaces (web + mobile). /topic/[id] detail, /topics index, kind-colored chip strips on Today rows, Universe header edge-kind breakdown, topic:Foo and topic:"Foo Bar" search operator with intersection semantics, ?topic=<id> URL pivot.
Pipeline poke — enqueue() fires a non-blocking pipeline-tick call after each enqueue so the chain runs in seconds instead of the cron’s 30s baseline. Cost analysis in commit d0c5d1f.
Hotfixes along the way: verify_jwt=false on link-topics-step
- backfill-summaries (sb_secret_* gateway 401), security_invoker=true on pipeline_stuck_jobs view (security advisor), text[] cast in the memory_edges constraint-rename block, ILIKE substring search on web (parity with mobile, fixes “danie” → “Daniel” surfacing).
Diagnostic scripts/debug-link-topics.ts — the script that found the verify_jwt gap is now permanent alongside debug-tick.ts.

E2E pipeline test green on deployed Supabase — 2026-05-06 (PR #10, feature/e2e-pipeline-test). Full chain validated end-to-end: record → transcribe → diarize → reid → summarize → embed → compute-edges. The cheapest-pre-demo drift catch on the queue, now off the queue. Going forward, any pipeline schema change re-runs this before merge.

V0.2 mobile follow-ups — flow continuity, Realtime, Resend, offline browse — 2026-05-04 (feature/mobile-expo-followups).

Realtime correctly delivers postgres_changes to the new sb_publishable_* key model. Without realtime.setAuth(jwt) called before .subscribe(), the WS connected anonymously and RLS-scoped broadcasts (memories, recordings) silently never arrived — manual refresh was the only way to see new entries. Wired in both apps/web/lib/supabase/browser.ts and apps/mobile/lib/supabase.ts with await getSession() → setAuth → .subscribe() ordering in the consumers (memory-list-live.tsx, app/index.tsx).
Resend SMTP live for magic-link delivery. Sender: noreply@send.arcive.io. Wired into Supabase Auth → SMTP Settings (host smtp.resend.com, port 465). Free tier 3k/mo covers V0.2 + early prod. Dev-skip button still in place for routine local iteration.
Recording row never disappears. Recorder always enqueues to the SQLite queue (even when online) and triggers a foreground flush; the pending row appears in Today the moment the user stops recording, transitions through “Uploading” → “Transcribing”, and is dismissed in the same Realtime tick that the real memory row arrives. Two new columns on pending_uploads (recording_id, uploaded_at) bridge the upload → transcript window.
Each recording persists to a unique path. Earlier recorder.uri was a single reusable temp slot — recording 3 clips offline made all 3 queued rows point at the same file (the latest), so when network came back the queue uploaded the latest audio 3 times. Now copied to documentDirectory/arcive/pending/<unique>.<ext> immediately after stop; deleted after successful upload.
Offline browse + on-device search. Last unfiltered fetch is cached in memory; on network failure (airplane mode, server 5xx) the feed falls back to filtering that cache locally with the same ILIKE behaviour. Subtle red banner indicates locally-cached results. Solves the “search filter applied + airplane mode → can’t clear back to full feed” stuck state.
Detail screen stops cross-memory audio. Tapping play on Today row A and navigating to A’s detail keeps audio playing (the user- liked “listen-while-reading” flow); navigating to a different memory’s detail now stops A so the user isn’t hearing audio that doesn’t match what they’re looking at.
Auto-network recovery via 30-second foreground retry loop in _layout.tsx. NetInfo would push-trigger this without polling but adding the dep mid-V0.2 wasn’t worth the bundler restart.
Subtle animations (built-in Animated + LayoutAnimation — no Reanimated/worklets dep, works in Expo Go): row fade/slide on feed updates, dot pulse on Uploading→Transcribing transition, breathing record button while capturing.
Code-review fixes from a multi-agent review pass: empty-string sentinel in playback_storage_path swapped for playback_content_type = 'missing' so consumers’ || fallback to storage_path works cleanly; periodic-save interval reads from a status ref so it actually fires (deps no longer include currentTime); inline / detail player identity moved from URL to memoryId so signed-URL refreshes don’t mis-identify the active row; flushPendingUploads has a top-level guard preventing concurrent drains from minting duplicate recordings rows.

PR: #6.

Audio transcode worker live on Modal — 2026-05-04. Cross-platform audio playback is now solved end-to-end.

backend/workers/audio-transcode/main.py deployed at https://sujith--audio-transcode.modal.run. ingest-audio fires every new upload at the webhook; a 10-minute cron drain_pending sweeps anything missed.
Worker is “always-transcode” — drops the unreliable storage.objects mimetype lookup and just runs every input through ffmpeg -c:a aac -ac 1 -b:a 64k -movflags +faststart into m4a. Inputs already in AAC re-encode cheaply (~1s per 30s clip). All output containers are audio/mp4 which iOS AVPlayer accepts (the bare audio/m4a MIME doesn’t, an Apple quirk).
Source-missing rows (storage object deleted but recording row retained) are flagged with playback_storage_path = '' so the cron doesn’t keep retrying. Cleanup is a separate concern.
Backfilled all 8 existing recordings on first deploy: 7 transcoded, 1 marked source-missing.
App-side: both apps/mobile/lib/audio-url.ts and apps/web/.../recordings/[id]/audio/route.ts already prefer playback_storage_path when set — apps automatically picked up the transcoded copies on next play with no further changes.
Logs visible in Modal dashboard; per-row status keys are transcoded / already-processed / source-missing / ffmpeg-failed / upload-failed.
MODAL_TRANSCODE_URL documented in .env.example. README in the worker dir has concrete modal deploy + supabase secrets set steps.

Mobile slice 3 — resume position, share-to-space, transcode scaffold — 2026-05-04 (feature/mobile-expo-followups).

Per-recording playback position persisted in AsyncStorage; player resumes mid-clip after navigating away or relaunching. Saves on pause/stop and every 5s while playing. Tracks finishing within 1.5s of duration are not saved.
Share-to-space mobile counterpart of the web component. Collapsible card on memory detail, RLS-protected memory_spaces insert/delete.
Server-side transcode scaffold:
- Migration 20260504000003_recordings_playback_path.sql adds playback_storage_path + playback_content_type (nullable).
- apps/mobile/lib/audio-url.ts and the web’s audio route both prefer playback_storage_path when set, fall back to the original — so once the worker lands, no client changes needed.
Mini-player + ±15s skip buttons removed per UX feedback. Memory detail player kept the drag-to-seek progress bar with thumb, single play/pause button, elapsed/total times.

Mobile slice 2 — audio playback, offline queue, search — 2026-05-04 on feature/mobile-expo-followups. Audio playback on Today rows (lazy-loaded inline player) and Memory detail (full player with elapsed/duration + progress bar) via supabase.storage.createSignedUrl on the audio bucket. Recorder now has tap debounce + real upload progress via XMLHttpRequest + five labelled phases. Failed uploads land in an expo-sqlite queue (pending_uploads) and auto-retry on app foreground; pending count surfaces as a banner on Today. Today gained a Postgres FTS search input (transcript_tsv / websearch), and Memory detail now supports pull-to-refresh.

Dropped local-Supabase / Docker dependency — 2026-05-04. Removed Path A (local stack via Docker) from the README; hosted Supabase is now the only path. Removed db:start / db:stop / db:reset / functions:serve from root package.json; kept db:push, db:diff --linked, and functions:deploy. Updated .env.example, apps/mobile/.env.example, apps/mobile/README.md, CI build env, and 01_SOFTWARE_PLAN §1.7 to point at hosted projects only. Mobile no longer needs the LAN-IP workaround. Historic migration that seeded host.docker.internal as a vault placeholder is left intact (already neutralised by the v0.3 cleanup migration).

Expo mobile app — first slice — 2026-05-04. apps/mobile scaffolded on Expo SDK 53 + Expo Router 4, wired into the pnpm workspace (Metro config watches the workspace root). Magic-link sign-in, session-gated routing, read-only Today (RLS-scoped query + Realtime subscription), Memory detail, and press-to-record dictation via expo-audio → POST /ingest-audio. Deferred to follow-ups: background/always-on capture (iOS background-audio mode, Android foreground service), BLE pairing, SQLite offline queue, audio playback on detail. Unblocks the V0.2 closeout per 01_SOFTWARE_PLAN §7 Phase 2.

Per-role system prompt lookup in voice service — 2026-05-04. Voice service now resolves the system prompt from the roles table by role_id query param (RLS scopes to built-in + own roles), appends the shared VOICE_MODE_ADDENDUM, and falls back to the Reviewer default if no role is supplied or the lookup fails. Closes the V0.2 dangling thread; voice channel can now run any built-in or user-authored role without code changes.

Post-rc1 hardening (on branch) — production-readiness pass.

pgmq DLQ + max-retry (ADR-0008).
Rate limits on Stripe checkout/portal + ingest-audio.
Explicit Sentry captures on Edge Functions + chat/Stripe routes.
dev_pass default flipped to false; UI hidden in production; vault placeholder cleaned up with a loud warning.
MCP server now type-checked in CI (was silently skipped).
Universe view paginates via ?before=<iso> keyset.

Post-rc1 (on branch) — MCP wiring + consent gate.

/api/chat now routes retrieval through the internal MCP server when MCP_SERVER_URL is set; falls back to inline otherwise. Brings the MCP server out of scaffold status into the active path.
Agent retrieval is consent-gated: respect_consent=true excludes memories with any non-granted participant. The data owner’s /today browsing is unchanged.
ADR-0007 documents the design.

v0.2.0-rc.1 — V0.2 web slice + voice scaffold. See CHANGELOG → 0.2.0-rc.1.

Headlines:

Claude Agent SDK driver replaces stateless RAG with tool-using agent.
Three new built-in roles: Tutor, Brainstorm Partner, Caregiver.
Family tier + Spaces UX (members, invites, share-to-space).
Pipecat voice talk-back service + /talkback web client.
Internal MCP server scaffold with search_memories, get_person, timeline_window tools.
dev_pass per-user flag — bypass tier gates during development while keeping the implementation intact.

Sizing convention

A single S/M/L isn’t enough — different slices fail for different reasons. Each row in this doc carries three signals plus free-text notes:

Field	Values	What it tells you
Size	XS / S / M / L / XL	Rough diff size. XS < 50 LOC, S 50-200, M 200-800, L 800-3000, XL 3000+. Doesn’t claim wall-clock hours.
Validate	`local` / `cloud` / `device` / `users` / `ext`	Where the work has to run for you to know it works. `local` = dev machine alone. `cloud` = needs Supabase/Stripe/etc. configured. `device` = real iOS/Android/dev kit. `users` = only humans can judge (e.g. voice naturalness). `ext` = needs a third-party service deployed.
Risk	`low` / `med` / `high`	What happens if it’s wrong. `low` = env-gated, easy revert. `med` = schema or external state changes; revert needs care. `high` = destructive or hard-to-undo (data loss, billing, public release).

Hardening slices skew XS-S, cloud-validated, high-risk — small diff, real state changes. Features skew M-L, local-validated, low-risk — bigger diff, mostly reversible. The schema makes that distinction visible.

Next up (top of queue)

Phases α, β, and γ.1 are done (PRs #10, #11, and feature/mobile-universe). The mobile force-graph is now live with Skia + d3-force, Obsidian-style layout, long-press drag, focus-dim, and a pinned “You” anchor. Full rationale and what’s deferred from ADR-0012: 2026-05-06/07 entries in 04_JOURNAL.md, working notes in discussions/2026-05-06_graph_tagging_strategy.md.

Priority	Item	Size	Validate	Risk	Phase	Notes
α.1	~~Real end-to-end pipeline test~~	S	cloud	low	—	Done 2026-05-06 (PR #10).
α.2	~~Decide ADR-0012~~	XS	local	low	V0.3	Accepted 2026-05-06.
α.3	~~DLQ pruning cron~~	XS	cloud	low	prod prep	Done 2026-05-07 (`feature/post-merge-cleanup`). 30-day TTL via daily pg_cron job.
α.4	Remove dev-skip path	XS	local	low	prod prep	Deferred to release prep — Skip button is current dev workflow per project memory; pulls when Apple Developer / public launch lands.
β.1	~~Topic-graph slice — schema + extraction + edges~~	M	cloud	med	V0.3	Done 2026-05-07 (PR #11).
β.2	~~Transcript viewer inline highlights + chip row~~	S	local	low	V0.3	Done 2026-05-07 (PR #11).
γ.1	~~Universe / graph view on mobile~~	M	local	low	V0.3	Done 2026-05-06 (`feature/mobile-universe`). Skia + d3-force, Obsidian-style layout, long-press drag, focus dim, pinned “You” anchor.
γ.2	Role conversations on mobile (text)	M	local	low	V0.3	Mobile parity for `/roles/[id]/chat`. Reuses `/api/chat` + MCP.
γ.3	MCP topic surface — `memories_by_topic`, `related_topics` RPCs	S	cloud	low	V0.3	Lets the chat agent navigate the graph instead of just embed-searching. Schema is in place; this is wiring.
δ.1	Multi-modal ingest groundwork	L	cloud	med	V0.3+	Per `discussions/2026-05-04_multimodal_expansion.md`: `memories.kind` enum + `assets` table, universal `/ingest` endpoint, PWA `share_target`, email-in. Vision/OCR topics flow into β’s `memory_topics`.
ε.1	Apple Developer account + EAS dev build	S	ext	med	release	Deferred to release prep per 2026-05-04 decision. Gates background-audio capture, BLE pairing, TestFlight, store submission. $99/yr + admin overhead.
—	~~Group conversation mode (LiveKit)~~	L	ext	med	V0.3	Paused with voice talk-back per ADR-0010.
—	~~Voice talk-back client on mobile~~	—	—	—	—	Paused per ADR-0010. Mobile blocked by Expo Go (Pipecat client-js needs `react-native-webrtc` → EAS); deferred until EAS unlocks AND the speech-to-speech-vs-Pipecat question has a deciding signal.
—	~~MCP wiring on voice service~~	—	—	—	—	Paused with parent feature (ADR-0010).

Backlog

V0.2 closeout

V0.3

Wire /api/chat to MCP server (replace inline Voyage). 2026-05-03.
Consent gate on agent retrieval (ADR-0007). 2026-05-03.
DLQ + max-retry policy on pgmq (ADR-0008). 2026-05-03.
Group conversation mode — LiveKit transport + interject().
DoA fusion with Pyannote re-ID for higher-confidence speaker labels.
MCP list_topics + get_thread tools.

Voice quality (paused — see ADR-0010)

Step 1 — spoken-register system prompt (ADR-0006).
[~] Step 2 — TTS swap (ElevenLabs v3 audio tags / Sesame CSM). Paused.
[~] Step 3 — prosody-controller FrameProcessor. Paused.

Hardware track (untouched)

firmware/ repo skeleton — ESP32-S3 + XVF3800 + PlatformIO.
BLE GATT server with placeholder UUIDs replaced.
Pairing flow (QR → BLE write → device joins WiFi).
OTA scaffolding (esp_https_ota + manifest poll).
I2S capture → 30s WAV chunk → POST /ingest-audio.

Hardening / quality

Production prep

Migration to flip dev_pass default to false + bulk-update existing rows. 2026-05-03.
Hide <DevPassSection /> behind NODE_ENV !== "production". 2026-05-03.
Replace vault.create_secret('replace-with-real-…') placeholder. 2026-05-03.
Sentry: explicit captures on critical paths. 2026-05-03.
Rate limits on Stripe + ingest routes. 2026-05-03.
Add backend/mcp/arcive-memory-mcp/ to CI typecheck. 2026-05-03.
pg_cron pruning job for pipeline_dead_letters (30-day TTL, policy set in ADR-0008). 2026-05-07. Daily at 03:00 UTC.
Resend SMTP for auth emails. Live as of 2026-05-04. Sender noreply@send.arcive.io (verified subdomain via SPF + DKIM + MX). Supabase Auth → SMTP Settings host smtp.resend.com, port 465. Free tier (3k/mo) covers V0.2 + early prod; bump to paid before public launch.
Remove dev-skip path. Delete app/login/dev-skip-action.ts, drop the “Skip (dev only)” button in login-form.tsx, remove DEV_USER_EMAIL / DEV_USER_PASSWORD from .env.example. Keep magic-link as the only entry point.

V1.0

Public MCP server (Cloudflare Workers).
Role marketplace UI + Stripe Connect (70/30 payouts).
B2B admin dashboard.
SOC 2 Type 1 prep.
Hardware retail (FCC/CE skipped — research units).

V1.1+

Vertical packages (Caregiving, Education, Therapy).
Memory-as-a-Service API tier.
Smartwatch app (Apple Watch / Wear OS).
Multi-language (ES, FR, DE).

Team infra (deferred — weekend project)

Self-hosted Outline on Raspberry Pi 5 (8 GB) + Cloudflare Tunnel + Cloudflare Access. Replaces Notion thin layer when we hit the 1000-block cap (~12 mo runway) or get a free weekend. Hardware ~$145 one-time, ~$10/year electricity, real inline block editor (Outline-grade), unlimited everything, MCP keeps Claude edit access via mcp-outline. Full plan + reliability checklist + migration path: discussions/2026-05-07_team_wiki_tooling.md.

Open questions

Member display in spaces — surface emails to other members (privacy-questionable), require display_name on signup, or server-side function returning a sanitized “Joe S.” form?
Caregiver consent enforcement — filter at retrieval ( search_memories) only, or also block in agent prompt-building, or both?
TTS swap timing — ship Cartesia by default and hold ElevenLabs/ Sesame for measurable pain, or jump now?

How to update this file

Two rules:

Move things between buckets, don’t rewrite history. When an item ships, move it from “Backlog” → “Next up” (briefly) → “Just shipped” (with a CHANGELOG link). Don’t delete; the audit trail is the value.
Keep “Open questions” honest. When a question is resolved, answer it inline and move it down to a new ADR if it shaped the architecture.