Live snapshot of the first ever WallSpace.Studio Liver performance with the app at Gray Area Noise Pop show in SF

Continuation of https://app.daydream.live/creators/AEYESTUDIOS/crt-wall-manager project page focused on AI Video Cohort 3. Will be posting daily releases here for beta group members as well as release notes on new features and bug fixes. What's New

Current Latest Build:

3/14/2026

https://github.com/jackmo650/crt-wall-controller/releases/tag/v2.2.10

WallSpace.Studio v2.2.10 — Experience Viewer, Multi-Source Transcription, A.EYE.STUDIOS Platform

Latest jackmo650released this36 minutes ago v2.2.10 9f9877d

🎬 Experience Viewer — Recorded Multi-Camera Playback

Standalone 3D viewer for recorded multi-camera experiences at wallspace.studio/view/.

• experience.html + experience-viewer.js — loads experience.json config
• Builds 3D room from SceneDescription format (spaces, walls, geometry)
• Per-wall <video> elements with THREE.VideoTexture
• Shared playback clock syncs all videos, re-syncs on drift >0.15s
• Transport controls (play/pause, seek, timeline scrubber)
• VTT caption support with multilingual display
• OG meta tags + CSS variables refactor for consistent theming

🗣️ Multi-Source Transcription & Translation

• Multi-source concurrent transcription — transcribe multiple audio sources across scenes simultaneously
• Translation layer styling — independent font, size, color, position controls for translations
• Dual text block rendering — show original + translated text, or replace mode
• Live translation wiring — real-time translation renders alongside transcription
• Web Speech engine — ESM import fix for renderer (was crashing with require())
• Fix captions broadcast — captions now send to all scenes, not just active scene

🔧 Output & Rendering Fixes

• Fix output3/4 state — destructured from useOutputState correctly in App.tsx
• Per-slot L2 grid override — display controls for outputs 3 & 4
• Per-CRT Syphon Out — batch selection creates one output per CRT slice
• Fix Zone 2 toolbar — always shows layer details, not CRT/wall/slice panels
• Guard IPC frame sends — prevents disposed render frame errors
• Fix audio feedback loop — don't route transcription mic back to speakers

🎵 Audio & MIDI

• Audio waveform for all layers — show waveform section for all audio-capable layers
• Guard MIDI dispatch — prevent MIDI errors on non-MIDI layers

🤖 A.EYE.STUDIOS AI Media Platform

• aeyestudios.ai — AI media generation web app deployed on Cloudflare Pages
• 37+ AI models: image generation (Flux, Seedream, Qwen), video, speech, text
• RunPod text/audio models added to catalog
• Text result display + "Send to Text Layer" button for LLM/STT models
• A.EYE.STUDIOS branding: logo (eye/@ icon), favicon, OG meta tags

🌐 Cloudflare Deployment & Custom Domains

Two web properties deployed on Cloudflare Pages (free tier, unlimited bandwidth):

URLPurposeaeyestudios.aiAI media generation platform — generate images, video, speech, text with 37+ modelswallspace.studioLive 3D show viewer, recorded experience playback, fan camera join

• Both hosted on Cloudflare Pages with R2 for media storage
• DNS managed via Cloudflare (nameservers from Namecheap)
• Zero egress fees on all media delivery

3/13/2026

A.EYE.ECHO v0.4.0 — Android Preview

Pre-release jackmo650released thisyesterday v0.4.0-android 4c1e396

A.EYE.ECHO v0.4.0 — Android Preview (Build 7)

Download: `a-eye-echo-v0.4.0.apk` (118MB)Requires: Android 8.0+ (API 26)

What Is A.EYE.ECHO?

A real-time captioning app for deaf and hard-of-hearing users. Point your phone's mic at any sound source — conversations, lectures, TV, live events — and get live captions on screen. Built by WallSpace.Studio under Neem LLC.

Everything In This Build (21 commits in 24 hours)

Core Transcription Engine

• Replaced whisper.rn with expo-speech-recognition — whisper.rn had a fatal JSI bug (issue #299, hung after first segment). Now uses Apple SFSpeechRecognizer on iOS and Google SpeechRecognizer on Android for stable, streaming live transcription
• Streaming partial results — captions appear word-by-word as you speak, not in delayed chunks
• Auto-restart sessions — iOS has a 60s speech recognition limit; engine auto-restarts at 55s seamlessly. Android uses Google's natural silence-based stopping with auto-restart on end
• Silence-based segmentation — 1.5s pause finalizes a caption segment, then starts fresh for the next utterance
• Removed all whisper.rn dead code — deleted whisperEngine.ts, audioStreamAdapter.ts, ModelDownloadModal.tsx, WHISPER_MODELS constants

Android Support

• Full Android platform support — platform conditionals for iOS-only audio session config, platform-specific WebView user agents
• minSdkVersion 26 — required for vision-camera-face-detector (covers 97%+ of active Android devices)
• Clean Gradle build — removed react-native-live-audio-stream (native module causing build failure), simplified audio capture to cross-platform stubs

Translation (Real-Time)

• DeepL translation — primary engine with API key injection via EAS environment variables (key never in git)
• LibreTranslate fallback — free fallback when DeepL quota exceeded
• MyMemory web API — replaced crashing ML Kit native translate module
• 11+ target languages — translate live captions in real-time
• Visual indicator — original text shown in blue italic below translated caption

URL/Video Mode

• Paste any URL — play video/audio in WebView, transcribe via mic
• WebView navigation — back/forward buttons, URL bar
• Auto-resume media — JS injection keeps video playing across speech recognition restarts
• Platform-specific user agents — Safari for iOS WebView, Chrome for Android WebView
• mixWithOthers audio — speech recognition doesn't pause WebView video playback

Speaker Detection & Camera

• Face tracking — react-native-vision-camera + MLKit face detection for speaker identification
• Speaker colors — auto-assigned color palette for different speakers
• Face-anchored captions (Phase 4D) — AR-style: captions position near the detected speaker's face instead of center screen. Toggle in Settings > Speaker Detection
• ASL hand landmarks — Apple Vision VNDetectHumanHandPoseRequest for 21-joint hand pose detection (iOS). Geometry-based fingerspelling classifier using finger extension states
• Independent face/hand detection — separate try blocks in frame processor so one failing doesn't break the other

Speech Intensity & Prosody (Phase 2)

• Speech intensity indicator — prominent color-coded bar showing Silent (gray), Quiet (green), Normal (blue), Loud (red) with text label. Helps deaf users understand emotional emphasis
• Caption prosody visualization — font size and spacing adapt to speech pace. Fast speech = compressed text, slow/emphatic = larger highlighted text, pauses = animated ellipsis
• Vibration grammar — distinct haptic patterns for punctuation: question marks (two short + one long), exclamations (three rapid), ellipsis/pauses (gentle fade). Helps feel the rhythm of speech

Quick Rewind Buffer (Phase 2)

• 5-second rewind — tap to show the last 5 seconds of captions you missed. Designed for when you're looking around or talking to someone and miss what was said

Accessibility Presets (Phase 2)

• Cinema — large text, bottom position, dark background, 2-line max
• Conference — medium text, center position, speaker colors, 4-line display
• Stage — extra large text, high contrast, 1-2 lines
• Classroom — medium text, speaker labels, full transcript mode

Conversational Chat Layout (Phase 4B)

• Chat bubble view — Matt's request: "Use chat layout A: <blah> B: <blah>". First speaker aligns left, others align right
• Speaker grouping — consecutive messages from same speaker grouped with tighter spacing
• Toggle views — switch between Timeline and Chat view in Transcript tab
• Translation in bubbles — shows translated text + original in chat bubbles

Speaker Diarization (Phase 4A)

• Audio-based speaker detection — detects speaker changes from pause timing, energy shifts, and question/answer patterns
• Auto-assigned speaker IDs — each detected speaker gets a unique ID and color
• Combines with camera — uses face detection when camera active, falls back to audio-only when off

Caption Networking (Phase 4C)

• Share captions between devices — one phone captures audio and broadcasts to others via WebSocket
• Room codes — 6-digit code to join a caption sharing session
• Share tab — new navigation tab with CaptionSharePanel UI

Transcript Timeline (Phase 4E)

• Tap to edit — tap any segment to correct transcription errors inline
• Search — live filtering with result count
• Export to 5 formats — TXT, SRT, VTT, JSON, Markdown via native share sheet (expo-file-system + expo-sharing)

Adaptive Power Management (Phase 3)

• Battery monitoring — expo-battery tracks charge level
• Three modes — Full (>30%), Balanced (15-30%), Saver (<15%)
• Auto-switch — automatically reduces features at low battery
• Saver mode — disables amplitude metering, stops camera to extend battery life
• Power badge — visual indicator when not in Full power mode

Video Streaming to WallSpace.Studio

• Camera to CRT wall — WebSocket client sends JPEG camera frames to WallSpace's signaling server for live compositing
• Auto-reconnect — exponential backoff on connection drops
• Binary + base64 transport — supports both frame encoding modes

Open Source

• MIT License — open source under Neem LLC
• Full README — features, setup guide, contributing instructions, project structure, tech stack
• Powered by WallSpace.Studio

Build & Infrastructure

• EAS Build — cloud-based Android APK generation
• Secure secrets — DeepL API key via EAS environment variables, secrets.default.ts with empty keys committed as template
• expo-build-properties — clean plugin-based build config
• Comprehensive dev log — architecture docs, technical decisions, issues faced and solved

Install Instructions

1. Download `a-eye-echo-v0.4.0.apk` below
2. On your Android device, enable "Install from unknown sources" if prompted
3. Open the APK to install
4. Grant microphone permission when asked
5. Start captioning!

https://github.com/jackmo650/a-eye-echo/releases/tag/v0.4.0-ios

A.EYE.ECHO v0.4.0 — iOS (Face-Anchored Captions + Open Source)

Pre-release jackmo650released this10 minutes ago v0.4.0-ios 4c1e396

A.EYE.ECHO v0.4.0 — iOS (Face-Anchored Captions + Open Source)

Platform: iOS (local Xcode build)Date: 2026-03-13 to 2026-03-14

What's New Since v0.3.0

Face-Anchored Captions (Phase 4D)

• AR-style subtitle positioning — when camera is active, captions optionally position near the detected speaker's face instead of center screen
• Settings toggle — Settings > Speaker Detection > "Anchor captions near face"
• Speaker service tracks face bounds — `getActiveSpeakerBounds()` returns face bounding box for caption positioning
• Smooth positioning — captions appear just below the face with max-width constraint to prevent overflow

Open Source

• MIT License — open source under Neem LLC
• Full README — features, download links, setup guide, contributing instructions, project structure, tech stack
• "Powered by WallSpace.Studio" — attribution in README and release notes
• Repository fields — license and repository added to package.json
• Copyright updated — Neem LLC across all files
• .gitignore hardened — build artifacts, caches, ASL model outputs excluded

Caption Networking Wired (Phase 4A + 4C)

• Speaker diarization — `audioDiarization.analyze()` called on each segment, auto-attaching speakerId
• Caption sharing — WebSocket-based broadcast between devices with 6-digit room codes
• Share tab — new CaptionSharePanel navigation tab

Transcript Timeline (Phase 4E)

• Inline editing — tap any segment to correct transcription errors
• Search — live filtering with match count
• Export — 5 formats (TXT, SRT, VTT, JSON, Markdown) via native share sheet

WallSpace.Studio Integration

• Video stream service — WebSocket client sends JPEG camera frames to WallSpace signaling server (port 8765) for live compositing onto the CRT wall
• Auto-reconnect — exponential backoff on connection drops

Build Fixes

• iOS deployment target 16.0 — set via expo-build-properties for MLKit/VisionCamera compatibility
• Removed temporary resetOnboarding() — onboarding now shows once as intended
• expo-doctor suppression — silenced false positive for expo-av

Commits (12)

```e09caac Phase 4A+4C: Wire speaker diarization + caption networking1273ab8 Phase 4E: Transcript timeline with search, edit, and export5941de4 Remove system audio + live-audio-stream to fix Android buildfdc4e80 Fix Android build: add expo-constants, fix dep versions, remove deploymentTargetc1d4b62 Remove temporary resetOnboarding() call from layoutf91ce6e Add video stream service for sending camera to WallSpace.Studio549a1e3 Fix Android build: raise minSdkVersion to 26 for vision-camera-face-detector940469f Add MIT license, README, and open-source prepecfec8d Update copyright to Neem LLC, add Powered by WallSpace.Studio3e2540a Add face-anchored captions (Phase 4D) — AR-style subtitle positioning4c1e396 Bump version to 0.4.0```

Full Feature List (cumulative)

Everything from v0.3.0-ios plus:

• Face-anchored AR captions
• Speaker diarization wired into pipeline
• Caption networking (device-to-device sharing)
• Transcript timeline (edit, search, export)
• WallSpace.Studio video streaming
• Open source (MIT, Neem LLC)

See v0.4.0-android for the full combined feature list.

MIT License | Neem LLC | Powered by WallSpace.Studio

https://github.com/jackmo650/a-eye-echo/releases/tag/v0.3.0-android

A.EYE.ECHO v0.3.0 — Android Preview

Pre-release jackmo650released thisyesterday· 4 commits to main since this release v0.3.0-android 549a1e3

A.EYE.ECHO v0.3.0 — Android Preview (Build 6)

Download: `a-eye-echo-v0.3.0.apk` (118MB)Requires: Android 8.0+ (API 26)

First Android build. Core features:

• Live mic transcription (Google SpeechRecognizer)
• URL/video mode (paste URL, transcribe via mic)
• Real-time translation (DeepL + LibreTranslate fallback)
• Speaker detection via camera face tracking
• Transcript export (SRT, VTT, TXT, JSON, Markdown)
• ASL hand landmark detection (iOS only)
• Accessibility presets (Cinema, Conference, Stage, Classroom)
• Dark theme UI

See v0.4.0 for the latest build with all new features.

MIT License | Neem LLC | Powered by WallSpace.Studio

Previous Builds

WallSpace.Studio v2.2.9 — WebRTC Video Bridge, 3D Scene Streaming, Unreal/Unity Integration

 jackmo650released thisyesterday· 16 commits to main since this release v2.2.9 21beb5d

⚠️ Pre-release — Not Yet Tested

This release contains significant new infrastructure that has not been tested in production. Features are functional but may have edge cases. Please report issues.

🌐 WebRTC Video Bridge (Phases A-F)

Live video streaming between WallSpace.Studio, mobile devices, and web browsers.

• WebSocket signaling server (port 8765) — peer management, SDP/ICE relay
• HTTP web portal (port 8766) — browser-based camera send and output viewing
• Full WebRTC with audio+video — hardware H.264/VP8 encoding, adaptive bitrate
• Remote Camera layer source — select connected feeds in the compositor
• JPEG-over-WebSocket fallback — for mobile clients without WebRTC
• Output streaming — stream compositor output to remote viewers via captureStream()
• A.EYE.ECHO integration — mobile video stream service for the companion app

New Files

• `src/main/videoSignalingServer.ts` — WebSocket signaling + HTTP portal server
• `src/renderer/services/remoteCameraService.ts` — WebRTC/JPEG video receiver
• `src/renderer/services/outputStreamService.ts` — Output canvas → WebRTC stream
• `assets/web-portal/index.html` — Self-contained web portal SPA

🏠 3D Scene Streaming (Phases 0-7)

Stream the creator's 3D CRT wall room to web/VR viewers with per-user customization.

Phase 0: Scene Description Protocol

• Lean serializable format for remote room reconstruction
• Wall texture atlas composites all wall canvases for efficient streaming
• RTCDataChannel alongside video for scene metadata

Phase 1: 3D Portal Viewer

• Three.js viewer in web portal (CDN + offline fallback)
• Room geometry reconstructed from SceneDescription
• CRT screens UV-mapped to atlas VideoTexture for live content
• OrbitControls for free-look navigation

Phase 2: Per-User Scenes

• Each viewer gets independent skybox, lighting, floor/ceiling colors
• Preferences persisted in localStorage
• Overrides applied locally without affecting other viewers

Phase 3: Image Upload Rooms

• Panorama upload as equirectangular skybox with reflections
• Custom room shell with adjustable dimensions (width/depth/height)
• Wall photo upload with texture application

Phase 5: WebXR VR Support

• Immersive VR on Meta Quest and other WebXR headsets
• Room scale matches real-world feet units
• "Enter VR" button shown when WebXR is available

Phase 6: Motion Tracking + Avatars

• Device orientation capture → quaternion → peer avatar meshes
• Cylinder+sphere avatars with name labels for each connected peer
• 10fps motion broadcast via signaling server

Phase 7: Blueprint Ingestion

• Floor plan parser: trace walls on an image, auto-generate room geometry
• Auto-detect rectangular rooms from simple floor plan images
• Scale factor (pixels per foot) for accurate dimension conversion

New Files

• `src/renderer/utils/sceneDescriptionBuilder.ts`
• `src/renderer/services/wallTextureAtlas.ts`
• `assets/web-portal/viewer3d.js` — Three.js 3D viewer module
• `assets/web-portal/lib/three.min.js` + `OrbitControls.js` — offline fallback
• `src/renderer/utils/floorPlanParser.ts`

🎮 Unreal Engine & Unity Integration (Phase 4)

Export per-wall textures to external 3D engines for immersive VR/XR experiences.

WallSpace Side

• wallTexturePublisher.ts — publishes each wall as individual NDI source + Syphon/Spout server
• Per-wall WebRTC streaming — subscribe to specific wall textures via signaling
• NDI sources appear as: `WallSpace - Front Wall`, `WallSpace - Left Wall`, etc.

Unreal Engine

• `WallSpaceNDIReceiver.h/.cpp` — Actor template for receiving wall textures
• Supports NDI, Spout, and WebRTC connection methods
• Dynamic material texture application to any mesh

Unity

• `WallSpaceReceiver.cs` — MonoBehaviour for single wall NDI texture
• `WallSpaceSceneBuilder.cs` — Auto-generates complete room from SceneDescription
• Works with KlakNDI, VR-ready (Quest, PCVR, Vision Pro)

🔊 Audio Analysis Pipeline Fix

Fixed three cascading bugs that prevented audio waveforms from ever displaying:

1. AudioEngine.layerNodes was private — analyzer couldn't access AnalyserNodes. Added `getLayerNodes()`.
2. AudioAnalyzer.start() was never called — the RAF analysis loop never ran. Now starts with AudioEngine.
3. Performance mode bug — `skipAudioAnalysis` wasn't gated behind `_pmEnabled`, disabling analysis even when performance mode was off.

Audio waveforms (oscilloscope, spectrum, 7-band meter), kick detection, and RMS/peak levels should now work for all audio-enabled layers.

📝 Additional Changes

• OSC caption bridge — receive/send live captions via UDP (port 9000)
• Web Speech API engine — browser-based speech recognition as Whisper alternative
• Caption export service — SRT/VTT/TXT/JSON/MD export formats
• Multi-language Whisper — configurable `--language` for 19 languages
• Translation service — DeepL + LibreTranslate integration
• `remote-camera` source type — new layer source with feed picker in LayerEditor
• `ws` package — added as direct dependency for WebSocket server

https://github.com/jackmo650/a-eye-echo/releases/tag/v0.3.0-ios

A.EYE.ECHO v0.3.0 — iOS (Major Rewrite + All Features)

Pre-release jackmo650released this7 minutes ago v0.3.0-ios 4c1e396

A.EYE.ECHO v0.3.0 — iOS (Major Rewrite + All Features)

Platform: iOS (local Xcode build)Date: 2026-03-12 to 2026-03-13Name change: Renamed from CaptionCast to A.EYE.ECHO in this version

Massive Release — 19 Commits, Complete Feature Set

This version is a near-complete rewrite. The app went from a basic Whisper transcription demo to a full-featured accessibility captioning platform. Every major subsystem was either rewritten or added.

Engine Rewrite: whisper.rn → expo-speech-recognition

The biggest change. whisper.rn had a fatal JSI bug (GitHub issue #299) — the native context would hang after the first transcription segment, making it unusable for continuous captioning.

Replaced with expo-speech-recognition:

• Apple SFSpeechRecognizer on iOS — streaming, low-latency, built-in
• Streaming partial results — captions appear word-by-word as you speak
• Auto-restart at 55s — handles Apple's 60s session limit seamlessly via generation counter
• Silence-based segmentation — 1.5s pause finalizes a segment, starts fresh
• No more WAV files — direct streaming replaces the PCM→WAV→Whisper batch pipeline

App Rename & Expansion

• CaptionCast → A.EYE.ECHO — new name, new identity
• 30 languages — full multi-language support via SFSpeechRecognizer locale mapping
• Onboarding walkthrough — first-launch tutorial explaining all features
• Settings persistence — AsyncStorage-backed settings that survive app restarts
• Error boundary — graceful crash handling

Real-Time Translation

• DeepL Free API — primary translation engine (high quality)
• LibreTranslate fallback — free fallback when DeepL quota exceeded
• MyMemory web API — third fallback, replaced crashing ML Kit native module
• 11+ target languages — Spanish, French, German, Japanese, Chinese, Arabic, etc.
• Visual indicator — original text in blue italic below translated caption
• Secure API keys — DeepL key via EAS environment variables, never in git

Camera & Speaker Detection

• react-native-vision-camera + MLKit — face detection with tracking IDs
• Speaker colors — auto-assigned color palette per detected speaker
• Lip-sync correlation — audio amplitude mapped to mouth landmarks
• iOS deployment target 16.0 — required for MLKit compatibility
• babel worklets-core plugin — frame processor support

ASL Sign Language Recognition

• Apple Vision framework — VNDetectHumanHandPoseRequest for 21-joint hand pose detection
• Geometry-based classifier — finger extension states (extended/curled/half) instead of ML model
• Fingerspelling support — static ASL alphabet poses recognized
• Independent detection — face and hand detection in separate try blocks so failures don't cascade
• HandLandmarksPlugin — Swift VisionCamera frame processor plugin

URL/Video Mode

• Paste any URL — play video/audio in WebView, transcribe via mic
• WebView navigation — back/forward buttons, editable URL bar
• Auto-resume media — JS injection keeps video playing across speech recognition restarts
• mixWithOthers audio — speech recognition doesn't pause WebView playback
• Platform-specific user agents — Safari for iOS WebView

Speech Intensity & Prosody (Matt's Phase 2)

• Speech intensity indicator — color-coded bar: Silent (gray), Quiet (green), Normal (blue), Loud (red) with text label
• Caption prosody visualization — font size adapts to speech pace (fast → compressed, slow → larger + highlighted, pauses → animated ellipsis)
• Vibration grammar — distinct haptic patterns: questions (two short + one long), exclamations (three rapid), ellipsis (gentle fade)

Quick Rewind Buffer

• 5-second rewind — tap to show last 5 seconds of captions you missed

Accessibility Presets

• Cinema — large text, bottom position, dark background, 2-line max
• Conference — medium text, center position, speaker colors, 4 lines
• Stage — extra large, high contrast, 1-2 lines
• Classroom — medium, speaker labels, full transcript mode

Conversational Chat Layout (Matt's Phase 4B)

• Chat bubble view — first speaker left-aligned, others right-aligned
• Speaker grouping — consecutive same-speaker messages grouped
• Toggle views — switch between Timeline and Chat in Transcript tab
• Translation in bubbles — translated + original text in chat bubbles

Speaker Diarization (Phase 4A)

• Audio-based detection — speaker changes from pause timing, energy shifts, question/answer patterns
• Auto-assigned speaker IDs — each speaker gets unique ID and color
• Camera + audio fusion — face detection when camera active, audio-only fallback

Caption Networking (Phase 4C)

• WebSocket caption sharing — one phone captures, broadcasts to others
• 6-digit room codes — join a session by entering the code
• Share tab — new navigation tab with CaptionSharePanel UI

Transcript Timeline (Phase 4E)

• Tap to edit — correct transcription errors inline
• Search — live filtering with result count
• Export to 5 formats — TXT, SRT, VTT, JSON, Markdown via native share sheet

Adaptive Power Management (Phase 3)

• expo-battery monitoring — tracks charge level in real-time
• Three modes — Full (>30%), Balanced (15-30%), Saver (<15%)
• Auto-switch — reduces features at low battery thresholds
• Saver mode — disables amplitude metering, stops camera

Cleanup

• Removed whisper.rn — dead code, JSI bug made it unusable
• Removed whisperEngine.ts, audioStreamAdapter.ts, ModelDownloadModal.tsx
• Removed WHISPER_MODELS constant and model selector from settings
• Removed react-native-live-audio-stream — replaced by expo-speech-recognition
• Removed system audio capture — deferred (Broadcast Upload Extension stubbed)

Video Streaming to WallSpace.Studio

• Camera to CRT wall — WebSocket client sends JPEG frames to WallSpace signaling server
• Auto-reconnect — exponential backoff on connection drops
• Binary + base64 transport — dual encoding support

Build & Infrastructure

• EAS Build configured — preview and production profiles
• App icons — generated from A.EYE.ECHO logo
• Splash screen — configured with dark background
• EAS environment variables — secure DeepL API key injection
• secrets.default.ts — template for local development
• .npmrc — legacy-peer-deps for build compatibility

Commits (19)

```d212681 Rename to A.EYE.ECHO, add multi-language, translation, sign language, URL ingest, system audio capture21facb2 Add app icons from A.EYE.ECHO logo and configure EAS Build1fd4883 Fix package dependencies for EAS Build398d3a2 Configure EAS project and fix dependencies for cloud buildf27ebd2 Fix EAS Build: add .npmrc, remove unused deps, pin worklets-corec5a8a79 Re-add ajv dependency required by expo-router config plugin9cb9dc9 Add expo-linking dependency required by expo-routerf250f94 Remove native-heavy deps for managed Expo build compatibility69cb185 Replace whisper.rn with expo-speech-recognition for stable live transcription8f44e28 Add camera face tracking, fix translation crash, web-based translation, UI cleanupada739d ASL sign language recognition via Apple Vision + DeepL translation6ffe27a URL video transcription, Broadcast Upload Extension, UI improvements6e6d899 Add comprehensive development log documentationd8fa5f3 Add Android platform support via platform conditionals6d7e4e0 Phase 2: Speech intensity, vibration grammar, prosody, rewind buffer, accessibility presetscc78bc4 Phase 3: Remove whisper.rn dead code, add adaptive power management96745b0 Phase 4B: Conversational chat bubble layout for transcriptsbb140f3 Fix EAS build: secrets fallback for CI, add audio diarization servicec7dfcc9 Secure DeepL API key injection via EAS Environment Variables```

This was the last iOS-only version. Android support was added in this version's commits and the first Android APK was published as v0.3.0-android.

MIT License | Neem LLC | Powered by WallSpace.Studio

3/12/2026

https://github.com/jackmo650/a-eye-echo/releases/tag/v0.2.0-ios

A.EYE.ECHO v0.2.0 — iOS (Native Whisper + Camera)

Pre-release jackmo650released this6 minutes ago v0.2.0-ios 4c1e396

A.EYE.ECHO v0.2.0 — iOS (Native Whisper + Camera)

Platform: iOS (local Xcode build only)Date: 2026-03-12Name at this point: CaptionCast (renamed to A.EYE.ECHO in v0.3.0)

Second Build — Real Native Integrations

Replaced all placeholder services with working native implementations. First time the app actually transcribed speech.

What Was Built

Whisper Engine (whisper.rn)

• GPU/Metal/CoreML acceleration — on-device inference using Apple's Neural Engine
• HuggingFace model download — automatic download of Whisper models with progress UI
• File-based pipeline — mic → PCM accumulate → WAV temp file → Whisper inference → emit segment
• Multiple model sizes — tiny, base, small, medium (user-selectable)
• ModelDownloadModal — first-time setup UI showing download progress

Note: whisper.rn was later replaced in v0.3.0 due to a fatal JSI bug (GitHub issue #299)

Live Audio Capture

• react-native-audio-pcm-stream — raw 16kHz mono PCM from device microphone
• Int16→Float32 conversion — format conversion for Whisper input
• Permission handling — iOS microphone permission flow

Camera Face Detection

• react-native-vision-camera — camera preview with frame processors at 15fps
• MLKit Face Detection — face landmarks, tracking IDs, smile/eye probability
• Lip-sync correlation — map mouth movement to audio amplitude for speaker identification
• CameraFaceDetector component — small PiP camera preview in corner of live screen

UI Updates

• Live screen — camera preview overlay, model download progress, auto-save to SQLite
• Real transcription — first time seeing actual speech-to-text on the live screen

Technical Decisions

• Chose whisper.rn for on-device privacy (no cloud API needed)
• Chose MLKit over MediaPipe for face detection (better Expo compatibility)
• 15fps frame processor to balance accuracy vs battery

Commits

• `008b0ef` Phase 2: Native Whisper engine, mic capture, camera face detection

https://github.com/jackmo650/a-eye-echo/releases/tag/v0.1.0-ios

A.EYE.ECHO v0.1.0 — iOS (CaptionCast Initial Build)

Pre-release jackmo650released this5 minutes ago v0.1.0-ios 4c1e396

A.EYE.ECHO v0.1.0 — iOS (CaptionCast Initial Build)

Platform: iOS (local Xcode build only)Date: 2026-03-12Name at this point: CaptionCast (renamed to A.EYE.ECHO in v0.3.0)

First Build — Architecture & Foundation

This was the initial scaffold — the entire app architecture ported from WallSpace.Studio's transcription pipeline into a standalone React Native + Expo mobile app.

What Was Built

Core Architecture

• TranscriptionService — PCM audio capture, 16kHz resampling, silence detection, Whisper inference pipeline
• VibrationService — haptic feedback patterns for speech start/end and speaker changes
• SpeakerService — camera-based face detection with lip-sync correlation (interfaces defined, wired in v0.2.0)
• ExportService — transcript export in 5 formats: TXT, SRT, VTT, JSON, Markdown
• Database — SQLite storage for sessions, segments, and speakers via expo-sqlite

State Management

• useTranscriptStore — Zustand store for live transcript segments and partial text
• useSettingsStore — Zustand store for all app settings with AsyncStorage persistence

UI Components

• CaptionDisplay — large, styled live caption text with theme support
• TranscriptList — scrollable transcript with timestamps and speaker labels
• 4-tab navigation — Live, Transcript, Sessions, Settings (expo-router)

Accessibility-First Design

• 5 built-in high-contrast themes
• Large default font sizes
• Dark mode throughout
• Haptic feedback for speech events

Types & Data Model

• TranscriptSegment — text, timestamps, speaker, confidence
• Speaker — id, label, color, face tracking metadata
• AppSettings — language, theme, font size, vibration, camera toggle
• ExportFormat — txt, srt, vtt, json, md
• VibrationConfig — enable/disable, pattern selection

Commits

• `74ef779` Initial scaffold: CaptionCast mobile live captioning app

This was an iOS-only local build. No APK/IPA published.

Neem LLC | Powered by WallSpace.Studio

3/11/2026

https://github.com/jackmo650/crt-wall-controller/releases/tag/v2.2.8

WallSpace.Studio v2.2.8

Major release introducing Metal GPU compute effects, live audio transcription via whisper.cpp, deep architecture refactoring (R3 Domain Contexts), a new Scope effects node plugin, and significant UI cleanup across the application.

Metal GPU Compute Effects (webgpuEffects.ts)

• New WebGPU compute shader pipeline for real-time per-pixel effects using Apple Metal backend
• Two rendering APIs: Canvas API (async, for preview) and Texture API (sync, for compositor pipeline)
• 16 GPU-accelerated effects: chromakey, edge detect, threshold, invert, RGB gain, scanlines, brightness, contrast, saturation, blur, glow, color quantize, Canny, lineart, softedge, depth
• Effects run entirely on-GPU with zero CPU readback — chained via the WebGPU compositor ping-pong buffer system
• Automatic device/pipeline caching with context-loss recovery
• Integrated into the existing effects.ts and compositor.ts pipelines

Live Audio Transcription System

Whisper Bridge (whisperBridge.ts)

• New main-process child process manager for whisper.cpp (C++ inference)
• Auto-downloads Whisper models from HuggingFace on first use: tiny.en (~75MB), base.en (~142MB), small.en (~466MB)
• Manages whisper.cpp subprocess lifecycle (spawn, kill, restart)
• PCM → WAV conversion for whisper.cpp input
• Relays transcripts to renderer via IPC bridge

Transcription Service (transcriptionService.ts)

• AudioWorklet-based PCM capture — avoids MediaRecorder crashes in Electron
• Captures Float32 PCM from any audio source, resamples to 16kHz mono
• Configurable chunk duration and accumulation window
• Sends audio chunks to Whisper via IPC, receives transcript text back
• Three audio source modes: system audio (loopback), microphone, layer audio

Caption Renderer (captionRenderer.ts)

• Renders live transcript text onto a cached HTMLCanvasElement (compositable as a layer source)
• Full styling support: font family/size, text color, outline, background, positioning
• Word-wrap with configurable maxLines limit
• Performance-optimized: only re-renders when text or style actually changes

macOS Audio Permissions

• Info.plist patched at postinstall for NSAudioCaptureUsageDescription, NSMicrophoneUsageDescription, NSScreenCaptureUsageDescription
• Electron builder extendInfo config for production builds

Architecture: R3 Domain Contexts Refactoring

Major refactoring of App.tsx (reduced from ~8,471 to ~7,466 lines) by extracting state into 4 React domain contexts:

SelectionContext

• Tracks user selections across all panels (CRTs, scenes, layers, slices, projectors)
• Separated from scene data to avoid unnecessary re-renders during high-frequency selection changes
• Manages drag state for slices and corner pins

SceneDataContext

• Single source of truth for all persistent scene graph data
• Owns: spaces, walls, CRTs, scenes, projectors, active IDs, canvas/wall config
• Provides derived values: activeSpace, activeWall, layers, enabledLayers, etc.
• initialState.ts extracted for independent provider initialization

OutputContext

• Manages output window state, display configurations, compositor refs
• Absorbs useOutputState hook into a shareable context
• Owns canvas refs, quality settings, output window lifecycle

ScopeContext

• Manages Scope/Daydream AI pipeline state
• RunPod GPU state, plugin serverless service configuration
• Phase A: state variables and refs in context; complex handlers remain in App.tsx hooks

Provider tree: SceneDataProvider > SelectionProvider > OutputProvider > ScopeProvider > App20 components now consume contexts directly via hooks (useSelection, useSceneData, useOutput, useScopeContext), eliminating deep prop drilling.

Scope WallSpace Effects Node (New Plugin)

• New standalone Scope plugin: scope-wallspace-effects
• GPU-accelerated effects deployable as preprocessor or postprocessor in the Daydream Scope pipeline
• Two pipeline configurations: wallspace-effects-pre and wallspace-effects-post
• 16+ real-time effects powered by PyTorch: brightness, contrast, saturation, chromakey, edge detect, blur, glow, scanlines, Canny, lineart, softedge, depth, color quantize, and more
• Python 3.12+, no additional model downloads required
• Complementary scope-wallspace-upscaler plugin for AI upscaling in the Scope pipeline

Centralized Logging System (from v2.2.7+)

• LogAggregator in main process: ring buffer, JSONL file output, port registry, IPC push
• createLogger(source, service) for main process, createAppLogger(source, service) for renderer
• Console interceptor captures React warnings + unmigrated console calls → Logs panel
• Port registry with 60s orphan detection (PID-based cleanup)
• Scope Local server auto-stops when browser window closes
• 459 console calls migrated across 13 files
• LogPanel in Zone 2 "Logs" tab with filter tabs (All/Errors/Ports/Scope/Perf)

Compositor & Rendering Improvements

• Compositor pipeline updated to route effects through WebGPU compute when available, with Canvas2D fallback
• perspectiveWarp.ts: WebGL context caching with size-based invalidation and context-loss detection (fixes WebGL context exhaustion at 30fps)
• crtHelpers.ts: Updated slice rendering and batch CRT optimizations
• webgpuCompositor.ts: Ping-pong buffer refinements for effect chaining
• Performance template types updated for new GPU profiling metrics

UI Cleanup & Component Updates

• LayerEditor: +188 lines — transcription controls UI, caption layer source type, audio source selection, Whisper model picker
• HierarchyItemRow: New caption/transcription indicators in the hierarchy tree
• BatchCRTEditPanel / CRTEditPanel / LayerListPanel: Minor type alignment fixes
• WebRTC Import Service: Extended for audio stream capture alongside video
• Audio Engine: +106 lines — system audio loopback capture, AudioWorklet integration, PCM extraction for transcription
• ControlNet Sync: Extended sync capabilities for new effect parameters
• Plugin params types: +80 lines — new effect parameter definitions for Scope effects node

Type System Updates

• pluginParams.ts: New effect parameter interfaces for all 16 GPU effects
• scope.ts: +36 lines — Scope pipeline types for effects node integration
• electron.d.ts: +13 lines — IPC type definitions for Whisper bridge
• gpuConsole.ts: Updated GPU performance metric types
• index.ts: Extended layer/source types for caption layers

Testing & Tooling

• Storybook: Vitest setup for portable story testing (.storybook/vitest.setup.ts)
• Vitest shims: Global type shims for test environment (vitest.shims.d.ts)
• Render profiling infrastructure from R3 refactoring

3/10/2025

https://github.com/jackmo650/crt-wall-controller/releases/tag/v2.2.7

v2.2.7 — Architecture Refactoring, GPU Pipeline, WebCodecs Transport & UI Polish

Latest jackmo650released this2 minutes ago v2.2.7 c246543

v2.2.7 — Architecture Refactoring, GPU Pipeline, WebCodecs Transport & UI Polish

This release represents a major multi-session effort using Claude Code (everything-claude-code workflow) to systematically refactor the application architecture, optimize the GPU rendering pipeline, introduce hardware-accelerated frame transport, and polish the canvas UI.

Architecture Refactoring (R1/R2)

Significant structural improvements to the monolithic App.tsx, reducing it from 8,770 → 8,143 lines (−627 lines, −7.2%):

• IPC Bridge (utils/ipcBridge.ts): 48 fire-and-forget window.electronAPI.* calls migrated to a typed ipc.* wrapper organized by domain (output, preview, projector, mapping, gpu, config, media, camera, recording, etc.). Provides compile-time safety and cleaner call sites throughout the app.
• Layer Builder (utils/layerBuilder.ts): Extracted buildLayers(), buildLayersFromList(), and compositeExtraSources() from the render loop into a standalone utility module with a LayerBuildContext interface (~300 lines). Eliminates closure dependencies and enables future testability.
• useOutputState Hook (hooks/useOutputState.ts): 14 output state vars, 6 3D preview vars, 2 output configs, and 9 compositor refs extracted into a dedicated hook. Accepts initialOutputDisplayConfigs parameter.
• ServiceRegistry (services/serviceRegistry.ts): Centralized lazy service lifecycle management for 6 services (Syphon/Spout, NDI, Plugin Serverless, Scope Controller, WebRTC Import, RunPod). Services are created on-demand with proper disposal on unmount.
• useUIState & useLayerManager Hooks: 32 UI state variables + 7 effects and 12 layer management handlers fully wired into App.tsx.
• R3 Domain Contexts Plan (docs/R3_DOMAIN_CONTEXTS.md): Documented future architecture plan for 5 React Contexts (Selection, SceneData, Compositor, Output, Scope) to replace 80+ prop drilling on CanvasView. Includes Storybook setup strategy and render profiling plan.

GPU Rendering Pipeline (Phases 1–3)

Complete output performance stabilization from plan through implementation:

• Phase 1 — Quick Wins: Fixed BeatDetector/AudioAnalyzer startup race (competing RAF loops), staggered output encodes (even→output1, odd→output2 for 50% CPU reduction), output FPS feedback via IPC (cyan badge in OutputsTab), RAF timing stabilization with EWMA smoothing.
• Phase 2 — Pipeline Optimization: Half-resolution OffscreenCanvas encode (~4× faster than full-res JPEG), encode timing instrumentation with EWMA metrics, lazy service initialization (Syphon/Spout, NDI, Scope, Plugin Serverless only init when needed).
• Phase 3B — WebGPU Compositor: Ping-pong render technique with WGSL shaders, 13 blend modes (multiply, screen, overlay, darken, lighten, etc.), per-pixel RGB gains/gamma/black-white point/zoom/pan/flip. Metal backend on macOS for full M3 GPU utilization. Feature-detects navigator.gpu, falls back to Canvas 2D.
• Phase 3C — Direct Frame Transport: MessageChannelMain for direct renderer-to-renderer communication (zero-copy ArrayBuffer transfer, bypasses main process). Falls back to JPEG IPC path if frame port not established.
• Feature Onboarding: First-run modal for feature selection (inputs, audio, outputs, AI, MIDI). Stored in localStorage, gates service initialization for simpler setups.

WebCodecs Frame Transport (New in v2.2.7)

Hardware-accelerated video compression for cross-window frame delivery:

• WebCodecs VideoEncoder integration with H.264/VP8 codec support (8 Mbps bitrate, realtime latency mode, prefer-hardware)
• EWMA timing instrumentation for readback, transfer, and total frame metrics via EWMATracker utility
• FrameSender.sendFrameEncoded() with codec-init handshake, keyframe interval (every 30 frames), and automatic codec configuration
• Backward-compatible: FrameReceiver handles both encoded (codec) and raw RGBA messages — falls back gracefully when WebCodecs unavailable
• Compositor integration: useWebCodecsTransport flag in compositor defaults to true when VideoEncoder is available

WallSpace Upscaler Plugin

• New Scope plugin for AI-powered upscaling of CRT wall output
• Serverless GPU Docker container with enhancement pipeline
• Scope template bumped to v0.1.7

UI Fixes & Polish

• Render Engine panel now auto-collapses when navigating to the Outputs tab (starts collapsed on mount)
• Quick Add sidebar auto-collapses when a CRT or projector is selected in the canvas, reducing panel clutter
• Bottom panel (DetailPanelRouter): Removed CRT Details and Projector/Slice Details from the bottom bar — those panels are already available in the canvas right sidebar. The Layer Editor now stays visible when CRT or projector is selected, so you can adjust layers without switching selections.
• Compact Wall/Space selector: Redesigned from two rows into a single compact bar — wall tabs on the left, space selector on the right. Inline badges for 3D transform, source region, and scene assignment. "All Walls" view automatically hidden when only 1 wall exists in the active space.
• SelectionTagBar component: Unified tag display for wall, CRT, projector, and scene selections in the toolbar
• ScenePlaybackQueue component: Timeline-based scene sequencing

Claude Code Workflow

This release was developed across multiple sessions using the everything-claude-code methodology:

• Architectural analysis with dependency mapping and risk assessment
• Phased refactoring plan (R1 hooks → R2 services/IPC → R3 contexts) with incremental verification
• Performance plan with 11 prioritized tasks across 3 phases, each verified against output FPS metrics
• Persistent memory files for cross-session continuity (render-pipeline.md, refactoring.md)
• Type-checked after every change (npx tsc --noEmit) with zero regressions

3/09/2026

https://github.com/jackmo650/crt-wall-controller/releases/tag/v2.2.6

WallSpace.Studio v2.2.6

 jackmo650released this4 days ago· 9 commits to main since this release v2.2.6 faf5740

What's New

• Scenes & Layers panel stays open — The panel now defaults to expanded when the app loads, no longer auto-collapses when switching tabs
• Windows support — First portable .exe release for PC users
• Windows disk space check — Fixed low-storage warning to work on Windows (uses wmic instead of Unix-only df)
• Build improvements — macOS native helpers (Syphon, NDI) properly packaged as extra resources; native files excluded from Windows builds

Downloads

FilePlatformWallSpace.Studio-2.2.6.dmgmacOS installerWallSpace.Studio-2.2.6-mac.zipmacOS zipWallSpace.Studio 2.2.6.exeWindows portable (no install needed)

Notes

• macOS: If you get a Gatekeeper warning, right-click → Open, or go to System Settings → Privacy & Security → Open Anyway
• Windows: The portable exe runs directly — no installation required. Your GPU is used automatically for WebGL rendering.