I should start with where I'm from, because everything in this project comes back to it. I grew up in Tijuana, on a border where technologies, languages and bodies are constantly being translated, filtered and policed. You learn early there that infrastructure, especially technological, is never neutral. A fence is a technology. A checkpoint is a membrane and an interface. When I started working with generative neural networks for sound, almost a decade ago now, I recognized the same logic operating at a planetary scale: enormous AI systems trained on data extracted from all of us, owned by a handful of companies, arriving in our hands as sealed products whose interiors we are never invited to understand. SEMILLA.AI began as my attempt to work otherwise. To build a neural audio instrument from the position of the musician, the small dataset, the community, and the seed.
I'm an autodidact. I never went through a formal computer science education, and I say this as a method, more than a biography. Everything I know about machine learning came from hacking open-source code, breaking it, sitting with the error messages, asking why. Workshops I've led over the years on demystifying neural networks all start from the same conviction: these systems are presented to us as inevitable and inscrutable, and they are neither. They are software, written by people, trainable on modest hardware, and bendable toward purposes their original authors never imagined. Including, as it turns out, casting seeds into a simulation and listening to what grows.
The seed and the zero
SEMILLA takes its name and its iconography from the Mayan seed glyph, the representation of zero. The Mayans were among the first cultures to formalize zero as a numerical and computational concept, and their version of it carries a meaning the Western tradition mostly lost: zero as potentiality and not emptiness; the seed from which everything emerges. That single inversion has guided the whole project. The dominant story of generative AI frames the model as an oracle producing finished culture and media on demand. I wanted an instrument where the model behaves more like soil. Something you plant in, tend, and listen to as it grows in sonic directions you can't fully predict.
© Private Collection / Bridgeman Images
Facsimile copy of codex Borbonicus, detail depicting the elaboration of the Oxomoco and Cipactonal calendar
There's a second lineage folded into the name. In Deep Learning practice, a "seed" is an integer used to initialize a pseudo-random number generator, the little ritual gesture at the start of every training and inference run. I drew a parallel between that and the Mixe divination practice of Mook pajk wëjwë, which loosely translates as maize seed divination: mook is maize, pajk is seed, wëjwë is to divine. Divination, whether the I Ching's three coins or a spread of Tarot cards or maize kernels cast across a cloth, is a technique for interpretative pattern recognition. You inject real-world randomness into a symbolic system and read what comes back. That is a pretty good description of what a musician does inside the latent space of neural synthesizers. The first hardware version of SEMILLA.AI, built during my residency at the Intelligent Instruments Lab in Iceland, made this a reality: physical maize seeds thrown into a circle, read by computer vision, their coordinates mapped directly into the latent dimensions of a neural decoder running on a small embedded computer. The throw of the seeds becomes the interface.
Part of what I'm after is a re-enchantment of machine learning culture, which is cold, statistical, and chasing optimization. Indigenous epistemologies offer other ways of relating to predictive systems, ways grounded in reciprocity and interpretation where the dominant paradigm offers extraction and control. Redirecting the research gaze with this in mind changes what you build.
Small data, mutual voices
The first public version of SEMILLA.AI used the RAVE architecture (the Realtime Audio Variational Autoencoder developed by Antoine Caillon and Philippe Esling at ACIDS-IRCAM) as its synthesis engine, already with a focus on the decoder as a generative instrument. It shipped with models trained on small, intimate corpora: salterio, cümbüş saz, darbouka and my own music. No scraped mass culture anywhere in the process. I started calling this approach "small data," and the politics of it became the core of MUTUALISMX: Becoming Sonic Networks, the collaborative album released on Nicolás Jaar's Other People in 2024.
© Hexorcismos
Hexorcismos - MUTUALISMX, (Other People, 2024)
The premise of MUTUALISMX was simple to state and slow to live through. I invited twelve musicians from across the Global Majority: KMRU, Ale Hop, Khyam Allami, Jessika Khazrik, Daniela Huerta, Debashis Sinha, Kloxii Li, El Irreal Veintiuno, MAF and others, and trained an individual neural audio model for each of them on their own archive, their own sonic fingerprint. Then everyone had a choice: work only with your own model, or share signatures with the others and let the identities blur into what we called a mutual voice, to engage in becoming sonic networks. An AI mutual, a cooperative of models, the opposite of a monolithic and centralized AI model. KMRU fed his immense catalog into the network and used the synthesizer almost like a mirror, pulling his own timbres and tendencies back out of it. Others went looking for the nebulous regions, the places where the model hallucinates between identities. Every artist kept ownership of their model. That detail mattered to me more than any sound on the record. Datasets are made by people, and when you engage directly with the musicians inside your dataset, the whole ethics of the technology changes shape.
When you work with the big pre-trained systems you are, in a sense, working with a collective intelligence of everyone whose data went into them, filtered through a machine's interpretation. That can be beautiful. It can also be a quiet form of expropriation, because nobody inside that collective was asked, and nobody inside it can opt out. MUTUALISMX was an attempt to build the consented version of that collectivity, at a scale where consent is possible.
SEMILLA.AI v2: from navigation to complex systems simulation
© SEMILLA.AI STUDIO
SEMILLA.AI v2
The version of SEMILLA I'm releasing now, v2, pushes the instrument somewhere more appropriate to the metaphor of seed-casting divination, and this is what I mostly want to describe here because it changed my own understanding of what these models are for.
The earlier paradigm of latent exploration in v1 was essentially cartographic. You moved a point through the model's latent space with knobs or seeds or XY pads, or you encoded a live input and let the model reinterpret it, which is the familiar magic of timbre transfer. v2 keeps timbre transfer available as one mode of play (you can still sing into a model trained on a saz and hear yourself come back as strings), but its core is something else entirely: decoder-only neural audio synthesis driven by a continuous physics simulation.
© SEMILLA.AI STUDIO
SEMILLA.AI (Hardware neural synthesizer, built at Intelligent Instruments Lab, Iceland)
The interface is organized around a large circular world-space. Into it you cast seeds, the same gesture inherited from the divinatory hardware instrument, except now each seed is an attractor inside a particle- and fluid-dynamics simulation, and the particles being pushed around are generating the latent codes themselves. The simulation panel exposes the physics directly. Gravity, friction and bounce govern how the particles fall, drag and rebound. Turbulence injects fluid-like instability into the field. Speed and range scale the temporal and spatial behaviour of the whole system. Double-clicking any seed opens its own Pull, Pole and Spin controls, so each attractor can drag, repel or swirl the latent codes in its vicinity, and seeds with different characters set up competing zones of influence across the space. You can see a video demo here.
On top of this sits an Orbit engine, which routes the seeds themselves along parametric paths, rose curves and related figures, with intensity and speed under your control and each seed's X and Y motion independently routable. So the attractors are themselves in motion, tracing flowers through the space while the particles stream and tumble between them. What you see on screen is a dense green mesh of interconnected trajectories, constantly reorganizing itself. What you hear is the decoder reading those trajectories in real time, frame by frame, as sound.
In this way of performing SEMILLA.AI there is no need for a neural audio encoder. No input audio being reconstructed, no reference signal. And this is the part I find genuinely important, beyond the fun of it: because the latent codes come out of the continuous simulation itself, never from encoding anything resembling the training material, the chance of ever reproducing the actual audio the models were trained on is vanishingly small. You aren't retrieving memories from the model. You're listening to the chaotic orbits and trajectories the simulation traces through the model's learned space. The training data shaped the terrain, the hills and valleys of timbre, and I think of the trained decoder as a landscape with a memory. The path across that landscape belongs to physics, chance, and to the performer. A model trained on a voice will produce something voice-like, breath-like, but the phrases it sings have never existed before and will never recur, because the simulation never visits the same state twice.
In practice the instrument loads two AI models at once, one per channel, so in a set I might run a model trained on my own musics on the left against a cümbüş saz model on the right, the same trajectory decoded through two different sonic intelligences simultaneously, drifting in and out of phase with each other. A Latent Parameters section gives per-dimension control through small polygonal editors: bias and scale for every latent dimension on each side, with randomize, reset and flip operations, so you can warp the geometry of the space the particles are moving through while they move through it. A States system captures full constellations of the instrument and morphs between them along an A/B axis, by hand or driven by an LFO, and multiple instances run side by side under MIDI, so the whole ecology can be played from hardware like any other synth. It took a long time to get the interface to a point where all this feels playable and not overwhelming.
The deeper shift in this new version is from generation to listening. The instrument augments rather than automates; there's no prompt box, no render button, no finished artifact dropping out the bottom. There's a complex chaotic system, and you're inside it, steering. Some nights the simulation finds a basin and circles it for minutes, producing these slow, respiring drones, and the right move is to do nothing and listen. That relationship to the machine, attentive, reciprocal and divinatory, is exactly what I hoped the seed-casting metaphor would bring to the software.
Provenance is sovereignty
There's an argument folded into all of this design that I want to make explicitly, because I think it's the decisive one for musicians right now.
Most of the public conversation about "ethical AI" in music concentrates on outputs: watermarking, detection tools, style protection, takedowns. Those debates matter, but the decisive question sits upstream, at data provenance. What was a model trained on, under what license, in whose interest? And here is the uncomfortable truth about the big general-purpose audio models: once a model has been trained and deployed, its provenance becomes practically unknowable. The dataset dissolves into hundreds of millions, or even billions, of weights. You can't audit it from the outside, the companies that could open the books rarely do and even well-intentioned documentation tends to arrive after the fact, reconstructed long after the fact instead of witnessed. Whatever extractions, biases and erasures went in are laundered into a smooth interface, and as users we inherit the consequences of decisions we can't inspect. The model becomes a sealed environment, culturally speaking. You can live inside it but you can't ask how it was built.
A sovereign approach inverts this. When a model is small enough to be trained by an artist or a community on their own archive, provenance stops being a forensic problem and becomes a relationship. I know what's inside the models I perform with because I recorded the material myself, or because the musician who did sat with me and gave their approval to train it, decided how it could be used, and kept the weights afterwards. In MUTUALISMX, every model belonged to the artist whose sound it learned. That's what I mean by artist-owned AI as sovereignty. Holding your own weights, knowing your own data, being able to retrain, retire, share or withhold the model on your own terms. Sovereignty here is also collective, mutualist instead of monopoly, and it's a separate question from open-sourcing; openness without provenance just distributes the mystery more widely.
The decoder-only architecture of v2 extends this ethic into the act of performance itself. Even within a fully consented model, I'm more interested in emergence, in treating the trained space as fertile ground and never as a disguised sample library. The physics simulation guarantees that what an audience and I hear together in a concert has never sounded before and won't sound again. The seeds are doing exactly what they were always supposed to do. Potentiality, the zero from which things emerge.
Plant your own
My hope for SEMILLA.AI is that the project demonstrates a viable counter-model, and that other artists feel motivated to build or train their own models and AI music instruments. Small data over scraped corpora. Augmentation over automation. Owned weights over rented APIs. Provenance as a relationship over provenance as a mystery. The shape of AI in music is still being decided, and it gets decided partly through which tools we choose to build, use and legitimize, in studios and on stages, long before any legislation catches up. I'd rather we plant seeds than harvest someone else's field.
