Unfolding

A lattice finding the shape of things, with no one to teach it.

Most machine learning is taught. A network is shown example after example, each with the correct answer attached, and corrected until its guesses match — the way the other works in this series learn to read a hand, or to pull a tangle apart. Unfolding learns the other way. It is given points drawn from a hidden shape with no labels and no answers, and one rule only: pull each part of itself toward the data nearest it. From a crumpled knot, with no teacher, it unfolds — the sheet smoothing and draping over a form no one named — and at the last it colours the data with categories it discovered on its own. Order found, not given. When the sheet has settled it holds; then it dissolves, a new shape is drawn, and it begins again. No two are alike. The work is built to run for hours, and does not repeat.

Image

Read the net as a single surface. Each node carries a fixed colour set by its place in the lattice — a smooth two-axis gradient. While the net is a knot the colours are jumbled; as it organises, a coherent gradient resolves out of the tangle, so the order being found is made visible as colour coming into agreement. The faint grey field is the unlabeled data — no colours given. Fine lines reach from those points into the net: each one pulling the nearest part of the sheet toward itself, the only guidance there is. At the last, the once-blank points take on the colours of the regions the net found for them — categories the machine assigned with no teacher.

Sound

Sound is continuous and made in the moment. As each point finds its place in the net, a single note is rung, drawn from where in the lattice it landed, so the music sweeps across the sheet as it organises. Many notes while the net is still untangling, then fewer as it comes to rest. Nothing is sequenced and nothing is recorded. What is heard is a function of what the net is finding, right now.

The artist

Joshua Borsman is an artist working with sound, light, and real-time systems. His generative works give form to processes that ordinarily pass unseen — the orbits overhead, the weather of a coast, the machinery beneath the internet — and render them as continuous, non-repeating fields of image and sound. Unfolding is the third work in Hidden Layers, a series on the unseen interior of machine learning — the place where a system comes to know. Where the earlier works are taught, this one learns with no teacher at all.

Specification

Series: Hidden Layers, III
Form: Generative real-time installation
Process: In-browser self-organizing map · unsupervised
Image: Real-time canvas
Sound: Continuous; rooted in G
Edition: Browser; all platforms

← Return to the work