ASCII Visual Engine

[Independent · 2026]

A real time ASCII rendering engine shipped as a TypeScript library. One grid, many inputs: patterns, motion, simulations, logos and type, video, audio, and MIDI drive which glyph lands in each cell. Host apps own the UI; the engine owns the grid.

Digital Forest look from the ASCII Visual Engine, dense green glyph blocks on black

Product brief

The ASCII Visual Engine turns a grid of characters into a live visual instrument. Anything that can produce a number per cell, a pattern, a motion field, a simulation, a logo, a webcam frame, an FFT band, a MIDI knob, decides which glyph lands in that cell and how it moves.

It is a library, not a product. Applications call one function with a canvas and a preset, get back a handle with a dozen methods, and build their own controls on top. Looks are JSON presets that the engine validates and interprets at runtime, so a new look is a file, not a fork.

It started as the motion language behind Ask Nate and is now the visual layer under Plantasonic, which is live at plantasonic.xyz. Hosts install it from a git tag; the harness at visual-engine.xyz exists to test it and show what it does.

The solution

01  /  06 1

One grid, many inputs

Every frame the engine resolves a grid of cells, each with a brightness, a glyph, a scale, and a phase. Patterns write the base field, motion behaviors move it, simulations and sources layer on top, effects and post passes reshape it, and the renderer draws it. Because every stage speaks the same grid, any input can drive any look.

Organic Bloom look, a breathing field of small green glyphs with scattered bright stars
02  /  06 2

Looks as JSON presets

A look is a preset: glyph set, density, speed, trails, glitch, and nested groups for motion, pattern, simulation, post, audio, input, and glyphs, every group optional. Thirty ship with the engine. Every pattern and effect declares its tunables with defaults, a look exports as JSON that loads back identically, and presets are validated on load and fail loudly with every problem listed. A morph blends one look into another over a chosen number of seconds and switches structure in one step partway through.

Minimal Zen look, an even lattice of at signs, percent signs, and plus signs
03  /  06 3

Plugins, motion, simulation

Visual behavior is composed from registered plugins, not monolithic rendering code. Six patterns, five effects, and a blendable motion system (flow field, organic growth, orbital, breathing, curl noise) sit beside eight simulations: particles, boids, cellular automata, reaction diffusion, L-systems, gravity, springs, fluid. A procedural glyph language with eleven libraries picks characters by semantic role and morphs between them.

Flow Field look, glyphs carried along curl noise currents
04  /  06 4

Logos and type

An SVG logo or a line of text rasterizes at the grid's own pixel size and becomes the brightness map. Each cell averages the source over its whole footprint, so a stroke thinner than a cell keeps its share of brightness instead of being hit or missed. In mask mode the wordmark becomes a window onto any look: every procedural stage runs, then the shape cuts it. Image, video, webcam, and canvas sources go through the same path, so a camera feed becomes glyphs in real time.

The word ASCII set in bold type as a mask, with the Digital Forest look showing through the letterforms
05  /  06 5

Built to perform live

Web Audio feeds an FFT, a feature extractor, and a beat detector with a BPM estimate; bands and beats map to any control or fire notes. Web MIDI brings notes, knobs, MIDI learn, device presets, and clock sync so pulse and breathing lock to tempo. Keyboard and pointer are input plugins on the same path. A seed and a fixed timestep make a show reproducible frame for frame, a take records every input and replays without the controller, and the canvas records to WebM or MP4 with the host's audio muxed in.

Corrupted Broadcast look, sparse glitching glyphs drifting across black
06  /  06 6

Host owned UI

No application chrome ships with the library. An app author calls createEngine with a canvas and gets a handle with about a dozen methods plus events; everything deeper is reachable through it. The harness is a test rig with one section per subsystem: preset, composition, source, renderer, audio, input, export, performance, scripting. Its sliders are generated from what the current preset actually reads, and a test fails if any control exists on one side and not the other.

The engine harness: the Preset section open with its sliders, morph seconds, and save and load, over the Organic Bloom look

One grid.
Any input.
Every look a file.

By the numbers

Square crop of the Digital Forest glyph grid

30

built in presets across seven families

362

automated tests with coverage floors, run on every push

16.7

milliseconds per frame is the budget; every hero look measures under it at 1080p

My role

I designed and built the engine end to end: the grid model, the plugin and event architecture, the motion and simulation systems, the glyph language, the source pipeline, the audio and MIDI mapping, the preset schema and validation, the export and recording paths, the facade, the harness, and the documentation and test suite that keep it honest.

Role

Designer, Engineer

Timeframe

2026

Disciplines

Creative Coding, Systems Design, Library API Design

Current state

Version 0.4.1, September 2026. Four releases in three weeks: 0.2.0 consolidated three drifting copies into one; 0.3.0 added the facade, nested presets, Retina scaling, pointer input, and the frame budget work that made WebGL unnecessary; 0.4.0 added beat detection, MIDI clock, seeds, morphing, video and input recording, and logos and type as sources; 0.4.1 hardened every entry point after the first host report. Canvas 2D is the renderer, with DOM as a fallback. Hosts pin a git tag. The remaining roadmap is demand driven: nothing starts until a host needs it.

Stack

TypeScript, Canvas 2D, Web Audio, Web MIDI, MediaRecorder, Vite, Vitest

Export

PNG, SVG, GIF, ASCII text, JSON scene, frame recording, WebM or MP4 video

Scripting

Safe public ScriptAPI over presets, controls, simulations, layers, and events

Platform

Any browser with canvas; zero framework dependency

Problems & decisions

Give several instrument projects one live ASCII visual layer without each of them carrying its own rendering code, and keep the engine fast enough for a laptop driving a screen.

Three projects wanted the same thing: live ASCII visuals that react to sound, input, and state. Building it three times would have meant three drifting copies. One engine with a stable contract lets each app spend its time on what makes it different.

Two users. The app author needs a canvas animating in a few lines and a way to reach deeper only when they need to. The look author needs to describe a visual in a file they can read, tweak, and share, without touching engine code. Both need it to hold 60 frames per second on an ordinary laptop.

Engine or product? The demo could have become the app, with its own chrome, routing, and controls. I kept it a harness. The library ships no UI, hosts own the shell, and the harness exists to test the engine and show what it does. The public surface had grown past a hundred methods because every host need was met by adding one, so 0.3.0 put a small facade on top rather than removing anything underneath: app authors see a dozen methods, look authors see nested presets where every group is optional, and the full engine stays reachable for whoever needs it.

Frame pipeline

  • Patterns write the base brightness field.
  • Motion behaviors blend by weight and priority and move the field.
  • Simulations and live sources layer on top; a source in mask mode cuts the finished frame to its shape.
  • Effects and post passes (feedback, smear, displacement, threshold, dither, scanline) reshape it.
  • The glyph language maps brightness and role to characters; a renderer draws the grid.

Systems

  • Typed event bus for notes, controls, presets, morphs, and custom events.
  • Plugin manager with pattern, effect, input, renderer, and utility types; every input device goes through one dispatch path and one mapper.
  • Multi layer compositing with blend modes and masks.
  • Named random streams behind a seed, and a fixed timestep, so the same take renders the same frames.
  • Frame profiler, quality presets with adaptive mode, object pooling, glyph cache, dirty region rendering.
  • Preset validation at load; scene documents for save and restore.

The target machine is a laptop driving a screen at 1080p, sometimes 4K, and the plan was to add a WebGL renderer when Canvas 2D ran out. It never did. The first measurement at 1920x1080 put three hero looks at 18 to 20 milliseconds a frame. Retina scaling behind a context transform, and a draw loop that sets the fill style once per brightness level instead of once per cell, brought all six to between 3 and 12. With the budget met, WebGL moved from next to demand driven, and the stub came out of the harness. The frame budget is re-measured and recorded with each release.

By September the engine existed in three places: the local tree with new validation and a reshaped demo, GitHub with CI and snapshot tests, and a copy vendored into a platform monorepo that had grown host driven methods for color, glyph override, and bass driven scale. Version 0.2.0 folded all three back into one branch with every test passing, and set the rule that hosts pin a git tag instead of carrying a copy.

The day Plantasonic went live, two of its five worlds went black about two seconds in while the engine still reported 60 frames per second. The host's adapter was passing a control value of NaN from an audio level read before the analyser had data, and one NaN cell reached the renderer's bucket lookup and threw every frame. The fix in 0.4.1 was not one guard but a boundary: every number that enters the engine is checked and a bad one is ignored with a warning that names it, the renderer treats a NaN cell as dark, and every preset is now run for four seconds in the test suite with the assertion that every cell stays finite. The warnings named the host's three bad controls the same afternoon.

The grid abstraction earned its keep: every feature since the first release, including morphing, input takes, and text as a mask, slotted in as a stage that reads and writes the same cells. The facade and the nested preset shape should have come first; a flat 59 field preset and a 105 method class were the honest cost of adding features one host at a time, and 0.3.0 paid it. The other lesson is about boundaries. A library that takes numbers from hosts has to assume some of them are wrong, and it took a live host to make that concrete.