feat(dashboard): Parallax Engram Field — cursor-reactive 3D/4D text

The dashboard text is now a volumetric field: each glyph floats at a depth
set by its trust, stroke weight set by FSRS retention, blur by confidence,
glow by salience. Move the cursor and nearby glyphs lift toward the camera,
lean, and ripple (SDF domain-warp); the field parallaxes like a diorama.
Every motion is a real memory fact — Math.random would visibly break it.

v1: PARAMS 16→20 cursor uniform (pre-divide NDC + velocity + off-canvas
sentinel), depth/weight packed into the free info.zw glyph lanes, vertex
depth→clip.z + Gaussian cursor lift + lean + pulse parallax, fragment SDF
warp + retention weight bias + depth-of-field + depth glow. Zero new render
passes. Deterministic (params.time/frame, never wall clock).

Built by GPT-5.5 against a source-verified scour spec (8-agent frontier
research combining W-perspective depth, cursor Gaussian fields, SDF warp,
and weight-as-data-gauge). Conductor live-GPU audit caught + fixed two
visual bugs: z clamped positive so cursor-lift never clips glyphs behind the
near plane; resting depth-of-field and glow floor raised so the field is
readable and lit at rest, sharpening toward the cursor. Verified live: text
glows at rest, brightens and lifts toward the pointer. check 0 errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Sam Valladares 2026-07-09 13:32:15 -07:00
parent a4a36ee220
commit 0d50eac624
7 changed files with 228 additions and 16 deletions

View file

@ -0,0 +1,52 @@
# PARALLAX ENGRAM FIELD — interactive-text system spec
## "the memory looks back at you" · per-glyph MSDF · zero DOM
Produced by the bleeding-edge scour workflow wf_ed4da324-bdd (5 frontier-angle agents + novelty-skeptic + buildability-judge + synthesizer that READ THE REAL SOURCE and corrected the scour). Build v1 against THIS.
## THE CONCEPT (signature wow)
Move the cursor over a wall of black text and it becomes a 3D depth-sorted brain that leans, ripples, and focuses toward the pointer — important memories physically closer to your face. Fuses 4 techniques into ONE grammar: (a) per-glyph W-perspective/clip-z depth, (b) cursor Gaussian z-lift + tilt (diorama parallax), (c) small SDF atlas-UV domain-warp near cursor (liquid-mercury letterforms), (d) SDF-threshold weight + AA-band defocus as the data channel. v2 adds spring-physics compute pass + true 4D w-axis hover punch.
## VERIFIED ENGINE FACTS (from real source — do NOT re-derive)
- info.zw is FREE (packGlyph writes 0,0 at float offsets 14-15). out.clip.z is pinned 0.0 (msdf-text.wgsl.ts line 67) — depth pre-wired, unused.
- PARAMS_FLOATS=16 and ALL 16 lanes claimed (last 4 by live bridge). Cursor uniform requires GROWING the struct to 20 (still 16-byte aligned = 80 bytes). This is the ONE upfront cost.
- Glyph anchors live in PRE-aspect-divide NDC (item.x,item.y). Shader divides pos.x/=max(aspect,1), pos.y*=min(aspect,1). Cursor MUST be written in the SAME pre-divide space: cursor_pre.x = ndcX*max(aspect,1), cursor_pre.y = ndcY/min(aspect,1) (inverse of pickAt lines 187-193) or the field skews.
## DATA BINDING (discipline test — Math.random visibly breaks each)
| visual | real fact | source field |
| depth z (front↔back) | trust/centrality | ObservatoryNode.isCenter + inverse suppression |
| stroke weight (SDF threshold bias) | FSRS retention (live-recomputed) | retention (from stability+lastAccessed) |
| DoF blur (AA-band width) | trust % / contradiction | trust |
| brightness (bloom eligibility) | salience/activation | activation |
| cursor lift/ripple/focus | user's attention (honest: the lens, not a data fact) | cursor uniform |
| w-punch on hover | the engram being inspected | pickAt |
| recall shock on click | a real recall event | click → route pick path |
Rule: resting field = data readout; cursor = lens that perturbs it. Nothing moves that isn't a memory fact OR the user's attention.
## BUILD PLAN — v1 (ZERO new render passes)
1. **Cursor uniform (one-time cost):** types.ts PARAMS_FLOATS 16→20, extend PARAM_IDX + WGSL struct Params with cursor_x[16], cursor_y[17], cursor_vx[18], cursor_vy[19] (pre-divide NDC space + smoothed velocity). Engine writes it in the per-frame param block (one writeBuffer). Velocity = smoothed (cursor - prevCursor).
2. **Pack data into info.zw:** in packGlyph, offsets 14-15 become info.z=engramDepth (0..1 trust), info.w=engramWeight (0..1 retention). Add optional {depth?, weight?} to TextLayerItem, thread from ObservatoryNode (trust→depth, retention→weight). Backward-compat: default mid value.
3. **Vertex (vs_text, replace lines 60-67):**
- resting depth: z = mix(0.55, 0.05, depth) (near=0.05, far=0.55)
- cursor Gaussian: d=distance(anchor, cursor_pre); R=0.45; w=exp(-d*d/(R*R))
- z -= w*0.30 (lift toward camera); pos += normalize(cursor-anchor)*w*0.015 (lean)
- pulse parallax: pos += vec2(sin(time*0.6),cos(time*0.5))*((1-depth)*0.006)*pulse
- KEEP the existing aspect transform unchanged
- wclip = 1.0 + z (fake perspective, far shrinks); out.clip = vec4f(pos, z, wclip)
- out.info = vec4f(info.x, info.y, w, depth) (pass cursor-influence w + depth to FS)
4. **Fragment (fs_text, ~lines 77-88):**
- SDF domain-warp near cursor: uv += vec2(sin(uv.y*40+time*3),cos(uv.x*40+time*3)) * (cursor_w*0.010) (SMALL — heavy warp mushes MSDF)
- weight bias: (weight-0.5)*0.18 added to (dist-0.5) → bold high-retention, thin low
- DoF: dof=(1-depth)*(1-cursor_w); screen_range_dof = screen_range/(1+dof*3) (far+unhovered=soft)
- glow: mix(0.6,1.35,depth) + cursor_w*0.5 → near/focused glyphs push rgb>1.0 → bloom for FREE in the existing HDR chain
5. **Hover (existing pickAt):** CPU drives picked run's info.z forward (crisp+bright hero). **Click:** one-shot recall-shock flare (per-glyph shock-phase counter, deterministic).
## v2 (trophies)
6. Spring-physics COMPUTE pass — clone node-renderer.ts recall-sim template (createComputePipeline/beginComputePass/dispatchWorkgroups already proven). Widen Glyph +1 vec4f phys lane (pos_xy, vel_xy), critically-damped spring toward anchor, cursor w injects impulse. capture_mode[11] freezes physics for ?frame=N loops.
7. True 4D w-axis hover punch: dedicated w-coord lane, xw-plane rotation + double perspective divide, hovered engram swells and passes THROUGH neighbors.
8. Deterministic demo-cursor Lissajous path under capture_mode for hands-free hero clips.
9. Velocity-directional chromatic aberration on fast sweeps (cursor_vx/vy already plumbed).
## DETERMINISM + BLOOM + ZERO-DOM
Every animated term rides params.time/frame/pulse (fixed sim clock, NOT wall clock) + per-glyph instance_index seed — never Math.random. ?frame=N&loop=1 reproduces frame-for-frame. Cursor is the only live input; for clips, freeze cursor or script a demo-cursor Lissajous under capture_mode. All glow = pushing rgb>1.0 into the existing rgba16float→bloom scene (no bloom-chain changes). Everything = glyph-instance data + 2 shader edits + 1 uniform write. No per-letter DOM, no three.js, one canvas.
## SOURCES COMBINED
W-perspective 4D→3D→2D (tesseract 6-plane rotation + W-camera divide); cursor Gaussian z-lift + magnetic tilt (Olivier Larose magnetic lineage); SDF atlas-UV domain-warp (Xor GM Shaders SDF, warp the sample not the quad, small-amplitude); SDF-threshold weight + DoF as gauge (Helsingin Sanomat Climate Crisis "type IS the gauge" + uncertainty→blur); spring physics compute (Codrops interactive text destruction, clone node-renderer.ts).

View file

@ -76,6 +76,8 @@
let paused = $state(false);
let userSetPause = $state(false);
let ready = $state(false);
let cursorSmoothed: { x: number; y: number } | null = null;
let focusedChromeRun: string | null = null;
const CYAN = [...rgb01(CAUSAL.forward), 1] satisfies [number, number, number, number];
const DIM_GREEN = [...rgb01(RETENTION.recall), 0.58] satisfies [number, number, number, number];
@ -239,16 +241,44 @@
};
}
function writeCursorLens(ndc: { x: number; y: number }) {
if (!canvasLayerEl || !engine) return;
const rect = canvasLayerEl.getBoundingClientRect();
if (rect.width === 0 || rect.height === 0) return;
const aspect = Math.max(0.0001, rect.width / Math.max(1, rect.height));
const raw = {
x: ndc.x * Math.max(aspect, 1),
y: ndc.y / Math.min(aspect, 1)
};
const prev = cursorSmoothed ?? raw;
const next = {
x: prev.x + (raw.x - prev.x) * 0.35,
y: prev.y + (raw.y - prev.y) * 0.35
};
cursorSmoothed = next;
engine.setCursorPreNdc(next.x, next.y, next.x - prev.x, next.y - prev.y);
}
function handlePointerMove(e: PointerEvent) {
const ndc = pointerToNdc(e);
if (!ndc) return;
writeCursorLens(ndc);
const navHit = navPass?.setHoverFromNdc(ndc.x, ndc.y);
const chromeHit = chromeText?.pickAt(ndc.x, ndc.y);
const nextFocus = chromeHit?.id ?? null;
if (nextFocus !== focusedChromeRun) {
focusedChromeRun = nextFocus;
chromeText?.setRunDepth(nextFocus, 1.0);
}
if (canvasLayerEl) canvasLayerEl.style.cursor = navHit || chromeHit || onpick ? 'crosshair' : 'default';
}
function handlePointerLeave() {
navPass?.clearHover();
focusedChromeRun = null;
chromeText?.setRunDepth(null);
cursorSmoothed = null;
engine?.setCursorPreNdc(999, 999, 0, 0);
if (canvasLayerEl) canvasLayerEl.style.cursor = 'default';
}

View file

@ -13,7 +13,7 @@
*/
import { DemoClock } from './demo-clock';
import { PARAMS_FLOATS, demoModeId, type DemoMode } from './types';
import { PARAMS_FLOATS, PARAM_IDX, demoModeId, type DemoMode } from './types';
import { PostChain, SCENE_FORMAT } from './post/post-chain';
import { VOID_CLEAR_HDR } from './post/tone-reference';
@ -132,6 +132,7 @@ export class ObservatoryEngine {
this.clock.framesPerLoop
: null;
this.params[8] = 1; // brightness default — the void must never eat the field
this.setCursorPreNdc(999, 999, 0, 0);
}
get status(): EngineStatus {
@ -187,6 +188,18 @@ export class ObservatoryEngine {
return this.clock.state.totalFrames;
}
/**
* Cursor lens for Parallax Engram text. x/y are in the text layer's
* pre-aspect-divide NDC space (the inverse of TextLayerPass.pickAt's
* screen-space transform), and velocity is in that same space.
*/
setCursorPreNdc(x: number, y: number, vx = 0, vy = 0): void {
this.params[PARAM_IDX.cursorX] = Number.isFinite(x) ? x : 999;
this.params[PARAM_IDX.cursorY] = Number.isFinite(y) ? y : 999;
this.params[PARAM_IDX.cursorVx] = Number.isFinite(vx) ? vx : 0;
this.params[PARAM_IDX.cursorVy] = Number.isFinite(vy) ? vy : 0;
}
/**
* Freeze/unfreeze the ambient motion (prefers-reduced-motion or the on-page
* pause control). Frozen = the clock stops advancing, so the orbit + force

View file

@ -16,6 +16,10 @@ struct Params {
live_frame: f32,
live_energy: f32,
projection_days: f32,
cursor_x: f32,
cursor_y: f32,
cursor_vx: f32,
cursor_vy: f32,
};
struct Glyph {
@ -31,6 +35,7 @@ struct VSOut {
@location(0) uv: vec2f,
@location(1) @interpolate(flat) info: vec4f,
@location(2) @interpolate(flat) color: vec4f,
@location(3) @interpolate(flat) weight: f32,
};
@group(0) @binding(0) var<uniform> params: Params;
@ -58,34 +63,63 @@ fn vs_text(@builtin(vertex_index) vi: u32, @builtin(instance_index) ii: u32) ->
let uv_max = glyph.uv_rect.zw;
let aspect = max(0.0001, params.viewport_w / max(1.0, params.viewport_h));
var pos = anchor + quad_offset + corner * size;
let depth = clamp(glyph.info.z, 0.0, 1.0);
let cursor_pre = vec2f(params.cursor_x, params.cursor_y);
let cursor_delta = cursor_pre - anchor;
let d = distance(anchor, cursor_pre);
let R = 0.45;
let cursor_w = exp(-(d * d) / (R * R));
// Depth → clip z. Trust (depth~1) floats forward (small z), low-trust sinks back.
// Cursor lifts a glyph forward, but z MUST stay > 0 or clip.z<0 clips the quad
// behind the near plane and the glyph vanishes (v1 bug: cursor made text disappear).
var z = mix(0.42, 0.10, depth);
z = clamp(z - cursor_w * 0.20, 0.04, 0.6);
let lean_dir = select(vec2f(0.0, 0.0), normalize(cursor_delta), length(cursor_delta) > 0.0001);
pos = pos + lean_dir * cursor_w * 0.015;
pos = pos + vec2f(sin(params.time * 0.6), cos(params.time * 0.5)) * ((1.0 - depth) * 0.006) * params.pulse;
// Keep glyphs square in BOTH orientations: normalize by the longer axis.
// Landscape (aspect>1): narrow x. Portrait (aspect<1): shrink y instead —
// dividing x by aspect<1 would WIDEN x and push text off-screen.
pos.x = pos.x / max(aspect, 1.0);
pos.y = pos.y * min(aspect, 1.0);
let wclip = 1.0 + z;
var out: VSOut;
out.clip = vec4f(pos, 0.0, 1.0);
out.clip = vec4f(pos, z, wclip);
out.uv = vec2f(mix(uv_min.x, uv_max.x, corner.x), mix(uv_max.y, uv_min.y, corner.y));
out.info = glyph.info;
out.info = vec4f(glyph.info.x, glyph.info.y, cursor_w, depth);
out.color = glyph.color;
out.weight = clamp(glyph.info.w, 0.0, 1.0);
return out;
}
@fragment
fn fs_text(in: VSOut) -> @location(0) vec4f {
let atlas_px = vec2f(textureDimensions(atlas_tex, 0));
let msdf = textureSample(atlas_tex, atlas_sampler, in.uv).rgb;
let cursor_w = clamp(in.info.z, 0.0, 1.0);
let depth = clamp(in.info.w, 0.0, 1.0);
let weight = clamp(in.weight, 0.0, 1.0);
var uv = in.uv;
uv = uv + vec2f(sin(uv.y * 40.0 + params.time * 3.0), cos(uv.x * 40.0 + params.time * 3.0)) * (cursor_w * 0.010);
let msdf = textureSample(atlas_tex, atlas_sampler, uv).rgb;
let dist = median3(msdf);
let uv_width = max(fwidth(in.uv), vec2f(1.0 / max(atlas_px.x, 1.0), 1.0 / max(atlas_px.y, 1.0)));
let uv_width = max(fwidth(uv), vec2f(1.0 / max(atlas_px.x, 1.0), 1.0 / max(atlas_px.y, 1.0)));
let texels_per_px = max(length(uv_width * atlas_px), 0.0001);
let screen_range = max(0.5, 4.0 / texels_per_px);
let px_dist = screen_range * (dist - 0.5);
// Depth-of-field: far/un-hovered glyphs soften, cursor sharpens. Kept GENTLE so
// the resting field stays READABLE (v1 bug: *3 blurred rest into invisibility).
let dof = (1.0 - depth) * (1.0 - cursor_w);
let screen_range_dof = screen_range / (1.0 + dof * 1.1);
let weight_bias = (weight - 0.5) * 0.18;
let px_dist = screen_range_dof * (dist - 0.5 + weight_bias);
let coverage = clamp(px_dist + 0.5, 0.0, 1.0);
let reveal_span = max(1.0, in.info.y);
let reveal = clamp((params.frame - in.info.x) / reveal_span, 0.0, 1.0);
let alpha = coverage * in.color.a * reveal;
if (alpha < 0.001) { discard; }
let rgb = in.color.rgb * params.brightness;
// Glow floor raised so even mid-depth text is visibly lit at rest (v1 bug: 0.6
// floor left it near-black); depth + cursor push bright glyphs past the bloom line.
let glow = mix(1.0, 1.5, depth) + cursor_w * 0.6;
let rgb = in.color.rgb * params.brightness * glow;
return vec4f(rgb * alpha, alpha);
}
`;

View file

@ -20,6 +20,10 @@ export type TextLayerItem = {
revealSpan?: number;
maxWidthEm?: number;
maxLines?: number;
/** 0..1 data depth/trust channel: 1.0 renders closer/crisper/brighter. */
depth?: number;
/** 0..1 data weight/retention channel: higher biases the MSDF stroke bolder. */
weight?: number;
};
type TextRunRect = {
@ -31,6 +35,8 @@ type TextRunRect = {
y0: number;
y1: number;
payload: TextLayerItem;
glyphStart: number;
glyphCount: number;
};
const GLYPH_FLOATS = 20;
@ -47,6 +53,7 @@ export class TextLayerPass implements FramePass {
private glyphCount = 0;
private pendingItems: TextLayerItem[] = [];
private runs: TextRunRect[] = [];
private runDepths = new Map<string, number>();
private initPromise: Promise<void> | null = null;
constructor(engine: ObservatoryEngine) {
@ -114,6 +121,7 @@ export class TextLayerPass implements FramePass {
});
const color = item.color ?? DEFAULT_COLOR;
const runGlyphs = laid;
const glyphStart = glyphIndex;
let x0 = Number.POSITIVE_INFINITY;
let x1 = Number.NEGATIVE_INFINITY;
let y0 = Number.POSITIVE_INFINITY;
@ -130,16 +138,20 @@ export class TextLayerPass implements FramePass {
packGlyph(packed, item, glyph, color, size, glyphIndex++);
}
if (runGlyphs.length > 0) {
const id = item.id ?? `msdf-text:${itemIndex}`;
runs.push({
id: item.id ?? `msdf-text:${itemIndex}`,
id,
kind: item.kind ?? 'text',
text: item.text,
x0,
x1,
y0,
y1,
payload: item
payload: item,
glyphStart,
glyphCount: glyphIndex - glyphStart
});
this.runDepths.set(id, clamp01(item.depth ?? 0.5));
}
});
@ -198,6 +210,22 @@ export class TextLayerPass implements FramePass {
return null;
}
setRunDepth(id: string | null, depth = 0.5): void {
const device = this.engine.gpuDevice;
if (!device || !this.glyphBuffer) return;
for (const run of this.runs) {
const targetDepth = run.id === id ? depth : (run.payload.depth ?? 0.5);
const clamped = clamp01(targetDepth);
if (this.runDepths.get(run.id) === clamped) continue;
this.runDepths.set(run.id, clamped);
const one = new Float32Array([clamped]);
for (let i = 0; i < run.glyphCount; i += 1) {
const floatOffset = (run.glyphStart + i) * GLYPH_FLOATS + 14;
device.queue.writeBuffer(this.glyphBuffer, floatOffset * 4, one);
}
}
}
dispose(): void {
this.glyphBuffer?.destroy();
this.glyphBuffer = null;
@ -233,11 +261,15 @@ function packGlyph(
glyph.v + glyph.vh,
ageFrame,
revealSpan,
0,
0,
clamp01(item.depth ?? 0.5),
clamp01(item.weight ?? 0.5),
color[0],
color[1],
color[2],
color[3]
);
}
function clamp01(value: number): number {
return Math.min(1, Math.max(0, Number.isFinite(value) ? value : 0.5));
}

View file

@ -120,8 +120,12 @@ export const PATH_KIND = {
// sim, firewall shock rings the field, etc.)
// [15] projectionDays (Phase 1 forward-scrub: added to every node's real
// FSRS elapsed so decay is legible in one session)
export const PARAMS_FLOATS = 16;
export const PARAMS_BYTES = PARAMS_FLOATS * 4; // 64 (multiple of 16)
// [16] cursorX (cursor in pre-aspect-divide NDC; far offscreen at rest)
// [17] cursorY
// [18] cursorVx (smoothed pre-divide cursor velocity, per pointer sample)
// [19] cursorVy
export const PARAMS_FLOATS = 20;
export const PARAMS_BYTES = PARAMS_FLOATS * 4; // 80 (multiple of 16)
/**
* Live-event kinds written into params[12] (liveKind). The field's shaders
@ -154,7 +158,11 @@ export const PARAM_IDX = {
liveKind: 12,
liveFrame: 13,
liveEnergy: 14,
projectionDays: 15
projectionDays: 15,
cursorX: 16,
cursorY: 17,
cursorVx: 18,
cursorVy: 19
} as const;
export function demoModeId(mode: DemoMode): number {

View file

@ -6,8 +6,11 @@
let hostEl: HTMLDivElement | null = $state(null);
let textPass: TextLayerPass | null = null;
let engineRef: ObservatoryEngine | null = null;
let cursorSmoothed: { x: number; y: number } | null = null;
async function handleReady(engine: ObservatoryEngine) {
engineRef = engine;
const pass = new TextLayerPass(engine);
textPass = pass;
await pass.init();
@ -21,7 +24,9 @@
color: [0.14, 0.78, 0.87, 1],
startFrame: 0,
revealSpan: 20,
maxWidthEm: 18
maxWidthEm: 18,
depth: 0.51,
weight: 0.51
});
engine.addPass(pass);
engine.demoClock.reset();
@ -37,14 +42,52 @@
if (hit) console.info('[msdf-test] picked', hit.id);
}
function pointerToNdc(e: PointerEvent): { x: number; y: number } | null {
if (!hostEl) return null;
const rect = hostEl.getBoundingClientRect();
if (rect.width <= 0 || rect.height <= 0) return null;
return {
x: ((e.clientX - rect.left) / rect.width) * 2 - 1,
y: -(((e.clientY - rect.top) / rect.height) * 2 - 1)
};
}
function handlePointerMove(e: PointerEvent) {
if (!hostEl || !engineRef) return;
const ndc = pointerToNdc(e);
if (!ndc) return;
const rect = hostEl.getBoundingClientRect();
const aspect = Math.max(0.0001, rect.width / Math.max(1, rect.height));
const raw = {
x: ndc.x * Math.max(aspect, 1),
y: ndc.y / Math.min(aspect, 1)
};
const prev = cursorSmoothed ?? raw;
const next = {
x: prev.x + (raw.x - prev.x) * 0.35,
y: prev.y + (raw.y - prev.y) * 0.35
};
cursorSmoothed = next;
engineRef.setCursorPreNdc(next.x, next.y, next.x - prev.x, next.y - prev.y);
const hit = textPass?.pickAt(ndc.x, ndc.y);
textPass?.setRunDepth(hit?.id ?? null, hit ? 1.0 : 0.51);
}
function handlePointerLeave() {
cursorSmoothed = null;
engineRef?.setCursorPreNdc(999, 999, 0, 0);
textPass?.setRunDepth(null, 0.51);
}
onDestroy(() => {
textPass?.dispose();
textPass = null;
engineRef = null;
});
</script>
<!-- Transparent pointer host only; ObservatoryCanvas owns the single visible canvas. -->
<!-- svelte-ignore a11y_no_static_element_interactions -->
<div bind:this={hostEl} class="fixed inset-0 bg-[#020307]" onpointerdown={handlePointerDown}>
<div bind:this={hostEl} class="fixed inset-0 bg-[#020307]" onpointerdown={handlePointerDown} onpointermove={handlePointerMove} onpointerleave={handlePointerLeave}>
<ObservatoryCanvas demo="recall-path" seed="msdf-test-v1" onready={handleReady} />
</div>