Speculative Acoustics Lab · Nov 19, 2026

Metric Balls & Grain Clouds

Segmenting a live spectrum into Bregman balls under KL divergence, then drawing grains only from the current ball.

Granular synthesis usually decides what to play next from an envelope, a random walk, or a hand-drawn position in a corpus. This one makes that decision statistically: the spectrum is segmented online into metric balls under a Bregman divergence, and grains are drawn only from frames that belong to the active ball. The cloud then inherits the sound’s statistical structure instead of tracking its amplitude.

A ball is a region of statistical similarity

A Bregman divergence BF(p ∥ q)B_F(p \,\Vert\, q) measures how far a distribution pp is from a reference qq. For a regular exponential family there is a unique Bregman divergence whose sufficient statistics are the coordinates of the family, and that bijection is what lets us treat a family of distributions as a geometric object. It is also the basis of the Music Information Geometry framework that Cont, Dubnov and Assayag built for audio.

Take the family to be the multinomial over normalised magnitude spectra. Its Bregman divergence is the Kullback–Leibler divergence:

DKL(p ∥ q)=∑ipilog⁡piqi.D_{\mathrm{KL}}(p \,\Vert\, q) = \sum_i p_i \log \frac{p_i}{q_i} .

A metric ball is the set of spectra within a fixed divergence rr of a centre:

BF(θ^,r)={ θ:BF(θ ∥ θ^)≤r }.B_F(\hat\theta, r) = \{\, \theta : B_F(\theta \,\Vert\, \hat\theta) \le r \,\} .

Everything inside the ball is, by that measure, the same sound: one quasi-stationary region.

The centre — the right-sided Bregman centroid — is the point that minimises the average divergence to the ball’s members, and for KL it has a closed form. It is just the normalised arithmetic mean of the member spectra:

c^=1n∑i=1npi.\hat{c} = \frac{1}{n}\sum_{i=1}^{n} p_i .

That closed form is what makes the whole thing streaming. Keep a running sum of accepted spectra, divide by its own total to get the centroid, and accept the next frame if its divergence from the centroid is at most rr. No iterative solver, no lookahead, one pass.

ball 0pool 0/40grey = ball centroid · orange = live spectrum

Headphones recommended. The demo loop is four two-second chords; Radius is logarithmic from 0.01 to 5 nats, so pulling it down splits each chord into its own ball.

From balls to grains

The segmentation is published research. The granulation is the part I have not seen done before, and it takes three mappings:

  1. Ball membership defines grain eligibility. When a grain fires, its source frame is sampled from the frames currently in the ball. Every one of those frames was accepted because it was statistically close to the centroid, so the grain is timbrally consistent with the region — and as the centroid drifts, the pool drifts with it.

  2. Radius controls timbral spread. A small rr admits only frames very close to the centroid, so the cloud is narrow and focused. A large rr admits frames further away, so the cloud is broader and more heterogeneous. This is a geometric control over timbre, not a density or pitch parameter.

  3. A ball boundary is a structural trigger. When a frame exceeds rr, the ball closes, a new one opens, and the grain pool resets. That boundary marks a statistical rupture: a moment where the sound’s local organisation changed by more than the ball could absorb. In the toy it flashes the frame and resets the pool bar, so the segmentation’s own rhythm is visible.

What to listen for

Sweep Radius while the demo plays. The loop is four two-second chords with deliberately different harmonic profiles, and each one darkens as it rings. Radius is logarithmic, from about 0.01 to 5 nats, and it starts at the point where one chord is one ball — so the grey centroid under the live spectrum is a picture of the current chord. Pull it down and the ball counter starts jumping: first at chord changes, then mid-chord as the timbre decays, until near the bottom the balls come apart every frame or two. Push it to the top and the whole eight-second loop collapses into a single ball, so the cloud becomes one long average of everything. That contrast, between “the cloud is the current chord” and “the cloud is the whole phrase”, is the instrument in one slider.

Freeze mid-chord. The pool stops growing and the cloud locks to the region captured at that moment. Unfreeze and it starts accumulating again around whatever the sound is doing now. If freezing does nothing audible, the radius is already large enough that the phrase was one ball to begin with.

Push Spread to six or eight at a high density. Grains start up to several analysis frames away from the position that qualified them, which loosens the timbre while the pool still constrains the neighbourhood. The result is a smear that is geometrically bounded rather than arbitrary.

The microphone button makes the same geometry listen to the room instead of the loop. There is no dry monitor on that path, on purpose — you hear only the grains, so the segmentation is audible as a texture rather than buried under the source.

Notes on the instrument

The analysis uses an AnalyserNode with a 2048-point FFT, read once per 4096-sample ScriptProcessorNode block. That puts the descriptor rate at roughly 10.8 Hz at 44.1 kHz, which is fast enough to track chord changes and slow enough that a ball can contain a few frames of them. The same processor writes a four-second ring buffer, and grains are read out of that buffer with the ball’s frames as indices. Grains are scheduled with a 100 ms lookahead over the audio clock, with inter-onset times drawn from an exponential distribution, so density is a rate rather than a drum pattern.

ScriptProcessorNode is deprecated and an AudioWorklet would be the modern choice. The worklet needs its own module file loaded with addModule, which complicates a component that otherwise ships as one island; the processor keeps everything in one place. The buffer is fixed at 4096 samples, which is the price of that convenience and is fine at a 10 Hz descriptor rate.

The desktop version is the same ball geometry with three synthesis modes sharing one ball: granular, which sounds the pool; additive, which resynthesises the peaks of the centroid as sinusoidal partials; and spectral, which uses the scaled centroid as a vocoder-style filter over an STFT. Same topology, three different ways to ask the centroid what it knows.

Open questions

The segmentation half is validated research; the granulation half is, as far as I can find, unexplored territory. The directions I would push are the ones where the geometry supplies a decision rather than an interpolation: using the current divergence from the centroid as an anomaly signal, using boundary events to trigger non-granular material, and moving the descriptor from a multinomial to a multivariate Gaussian so the covariance matrix captures correlations between partials rather than only their marginal energies. That last one is the natural bridge back to harmonic structure, and it is where I would look next.

References

  • Banerjee, A., Merugu, S., Dhillon, I. S., & Ghosh, J. (2005). Clustering with Bregman divergences. Journal of Machine Learning Research, 6, 1705–1749.
  • Cont, A., Dubnov, S., & Assayag, G. (2011). On the information geometry of audio streams with applications to similarity computing. IEEE Transactions on Audio, Speech, and Language Processing, 19(4), 837–846.
  • Dessein, A., & Cont, A. (2011). Segmentation statistique de flux audio en temps réel dans le cadre de la géométrie de l’information. Actes des Journées d’Informatique Musicale.

How this page works: the instrument is a React island embedded in this MDX post with client:visible, so its audio and canvas code only load when you scroll to it.