Sowing Points, Reaping Forms

The programmability of the computer is expressed over a very vast structurable space¹, making it possible to set in motion algorithmic processes governed by very simple rules, which can nevertheless produce graphic configurations of great complexity. Cellular automata and autonomous agents illustrate these processes well. It is through the creation and algorithmic manipulation of autonomous agents that the Spanish artist Toni Mitjanit built his film An Autonomous Agent Choreo (2016). Autonomous agents belong to the so-called behavioral modeling techniques, which appeared in the 1980s to address the limitations of cognitivist approaches². Whereas cognitivist modeling architectures respond to global symbolic representations³, behavioral models operate on the basis of the capacities of individual agents, each of which is able to react according to the data “perceived” – detected, captured, read – in an environment shared with other autonomous agents, in which they evolve by interacting.

Modeling the Living

The digital short film An Autonomous Agent Choreo is composed of animated sequences that follow one another through cross-fades. These sequences, each lasting 4 or 5 seconds, present enormous ensembles of autonomous agents performing complex choreographies. Toni Mitjanit obtained these choreographies through long and meticulous experimentation, which consisted in setting the parameters of the agents’ behavior and the characteristics of their environment, then letting the algorithmic system evolve until it reached visual renderings that the artist found interesting. These thousands of autonomous agents circulate and interact across the plane of the image, producing particular situations of dynamic equilibrium, which seem to be affected by geometric compartments defined within the frame. These compartments delimit zones of local behavior for the agents, which can nevertheless cross the borders to reach neighboring compartments. Thus each sequence of the film proposes a different division of the image into geometric compartments, taking the form of mosaics or of concentric structures arranged in various ways.

The autonomous agents that populate this short film take the form of points that move in oscillation, leaving trails that reveal their trajectories. Taken together, they evoke shoals of fish of unreal extent, or outsized groups of worms or snakes in motion. In reality, in designing his simulated creatures, the artist drew inspiration from the movement of the tentacles of sea anemones, in which he recognized a choreographic potential that he wished to transpose into an audiovisual piece.

The modeling technique used by Mitjanit was developed in the 1980s by the mathematician Craig Reynolds⁴, who was looking for an algorithmic model capable of reproducing the flight of flocks of birds, herds of animals and schools of fish. This organic signature is indeed present in An Autonomous Agent Choreo mainly at the level of the microstructure – at the level of each agent – while at the global scale of the image it manifests itself in the form of a moving, dynamic texture. This organic aspect is, however, counterbalanced by the geometric subdivision of the plane of the image, a segmentation made visible by the color of each block but also by the behavioral similarity of the agents within each compartment. Thus, to the organic profusion that never leaves the image is added a global geometric aspect that stabilizes it.

Cybernetics Revisited

In its conception and functioning, this work by Mitjanit lends itself particularly well to a reading through the prism of cybernetics, a discipline that has had fluctuating relations with the digital⁵, but which, in the face of the new advances of this technology, is drawing closer to the digital in a renewed way. Cybernetics, a very active field of study in the 1940s and 1950s, had as its project the development of a conceptual structure valid for both the living and the machine. Three key notions of cybernetics shed particular light on Mitjanit’s work, which, as we have seen, is also inspired by the living: the notions of homeostasis, of autopoiesis and, above all, of emergence.

As regards homeostasis – which can be understood as an organism’s capacity to adapt, without altering its nature, to disturbances coming from its environment – it is clear that each autonomous agent in Mitjanit’s work is able to adjust its trajectory according to the spatial arrangement of the agents around it, following behavioral functions of cohesion, separation and alignment⁶. However, the mechanism that ensures the dynamic stability of each autonomous agent does not manage to guarantee invariably that the system as a whole will evolve toward a stable state. Indeed, the global system can evolve either toward a regular distribution of the autonomous agents or, as is most often the case, toward a heterogeneous distribution of the agents, with overcrowded zones and empty ones⁷. Only actually running the simulation makes it possible to determine the evolution of the system according to the initial parameter settings, which is precisely the experimental work Mitjanit undertook in order to obtain relevant sequences for his film.

The phenomenon of autopoiesis, as described by Humberto Maturana and Francisco Varela, involves a stable structure that constantly regenerates itself, even in the face of disturbances coming from its environment, and this despite the constant replacement of its individual components: “[…] an autopoietic machine is a system with stable relations whose fundamental invariant is its own organization (the network of relations that defines it)⁸.” The system devised by Mitjanit, based on the programming of simple autonomous agents and the simulation of their behavior under different starting conditions⁹, sometimes tends toward a form of stability that makes us think of autopoiesis: a system that maintains its – graphic – structure despite the continuous renewal of the autonomous agents that form it. We could then say that Mitjanit’s film is a succession of autopoietic situations, stemming from a system whose stability is not guaranteed but remains plausible. This stability cannot be predicted by analytical means; it can only be discovered by trial – simulation – and error.

The Game of Life

When we add the notion of emergence to the reflection¹⁰, Mitjanit’s autonomous agents resonate with cellular automata, systems defined on a two-dimensional grid in which each cell can take two values – white or black, zero or one – within a pattern that evolves according to very simple rules, but which can lead to extremely complex, unexpected and sometimes surprising results. The emblematic example of cellular automata, proposed by the British mathematician John Conway in 1970, was named The Game of Life.

In the game, at each generation (t=0, t=1, t=2, …) there will be a “birth” – the cell turns black – if a white cell is surrounded by exactly three black cells, a “survival” – the cell stays black – if a black cell is surrounded by two or three black cells, and a “death” if a black cell is surrounded by fewer than two or more than three black cells (1, 4, 5, 6, 7 or 8). These extremely simple rules, depending on the initial state of the system (number and position of the white and black cells), give rise to very diverse situations. Indeed, depending on its initial conditions, the system can evolve toward the total disappearance of the black cells – and therefore the end of the game – or toward the appearance of configurations in which the black cells form stable “objects,” fixed or oscillating. These objects were discovered and named by the game’s explorers according to their resemblance to real objects, including the “block,” the “loaf,” the “boat,” the “pond,” the “blinker” and the “beacon,” among many others. Some configurations present mobile objects that leave the frame of the board, such as “gliders,” or, in the most complex cases, objects that move and seem to release smaller elements, such as “glider guns” and “puffers.” Beyond these objects, of which 3,798 have been identified to date¹¹, it is possible to “build,” in the Game of Life, metacells that reproduce the behavior of the basic cells at a much larger scale, following a fractal logic.

Moreover, it has been shown that the Game of Life allows the complete modeling of a Turing machine, since it would have “the capacity to carry out any calculation that can be done by a program¹².” The speculations of the mathematicians Elwyn Berlekamp, John Conway and Richard Guy, reported by Jean-Paul Delahaye, suggest the ultimate levels of complexity of which certain evolutions of the Game of Life would be capable – starting, let us recall, from basic rules of extreme simplicity: “it is probable that, if a large enough part of the infinite plane of the Game of Life is filled with a random configuration, then, after a while, intelligent self-reproducing beings will emerge and populate the space¹³.”

The complex plastic configurations triggered by Toni Mitjanit to generate the sequences of his short film echo the sophisticated evolutions that emerge from the Game of Life. When such formal complexity emerges from such a simple set of rules, the reductionist approach, which brings the global behavior of a system back to the principles governing its individual components, is called into question. In this framework, the notion of emergence takes on its full meaning, for it expresses the fact that the “whole” can be more than the sum of its parts. In other words, the emergent behavior of a system cannot be reduced to the laws governing its fundamental components.

Triggering Emergence(s)

Edmond Couchot proposes an interpretation of emergence specific to creative processes that mobilize computing means. He shows us the artist as the bearer of a particular attention, capable of isolating, among the forms produced (generated), those that can take part relevantly in the construction of the work. Indeed, for Couchot, the artist:

creates a spatio-temporal situation […] in which virtual objects […] maintain “emergent”-type relations among themselves; he lets them develop more or less freely while controlling their autonomy, and retains only the moments he judges the most interesting from an aesthetic point of view¹⁴.

These remarks by Couchot resonate deeply with the creative process followed by Mitjanit, and quite directly with several works we analyze in this chapter.

It is important, however, to distinguish, as Mark A. Bedau emphasizes, two types of emergence in the evolution of a system: a “weak” emergence, which remains explicable by the underlying processes (at the level of the individual components of the system), and a “strong” emergence, which manifests itself autonomously at the level of the global system, independently of the laws governing its basic processes¹⁵. So-called “strong” emergence, in Bedau’s terms – and we share his view – is “uncomfortably like magic¹⁶,” and it could not be envisaged within the framework of this research. On the other hand, the production of forms by cellular automata or by autonomous agents corresponds to what we mean by weak emergence. This remains explicable (a posteriori) on the basis of the laws that control the basic components: it is not only computable but computed, and therefore remains mechanistic in this context. Weak emergence nevertheless exceeds human capacities for intellectual and analytical prediction, while remaining, as this research senses, within reach of artistic intuition.

According to Henri Atlan, the notion of emergence has taken the place of vitalism, a concept in vogue in the nineteenth century¹⁷; yet while for vitalism it is “life” that bursts into the inorganic, the notion of emergence rather privileges the idea of “mechanism.” It is therefore legitimate to ask whether a mechanism can trigger emergence, produce an event, and finally, whether it is possible to make a work from a mechanism. One could not claim an absolute mechanism in an artistic context; on the contrary, machinic operations should be inscribed in a creative process whose singularity lies also outside the technical means mobilized. Indeed, mechanism can become creative, provided it is orchestrated by the artist, often in interaction with other mechanical processes, but always within the framework of a singular creative process, in which machinic and human instances are actualized. It is in this temporality that moments of groping, experimentation, decision and choice take place, from which the work is built. This process can be likened to a form of human-machine gesture¹⁸, but one whose initiative belongs exclusively to the human: it is the human who prepares the ground, who sows and, in happy cases, reaps.

In-formed Matter

Let us return to cybernetics to conclude this part, a science which, as we have seen, can be fruitfully related to artistic creation through digital means. This is explained by the fact that certain ideas formulated by cybernetics are only becoming materially feasible today¹⁹. An Autonomous Agent Choreo seems to align itself with this “return” of cybernetics²⁰, through which the notions of entropy and information are drawing closer to digital creation, on a new footing. What interests us here is that the latest technological advances tend to bear out the conception of information put forward by the “first cybernetics.” In this framework, the notion of information is apprehended from a material perspective: the image is to be approached as a concrete material entity, whose evolution and internal organization can be described in terms of order and disorder, in other words of entropy. The notion of entropy, however, must in turn be considered in its physical sense, which refers to an energetic aspect of matter rather than to the statistical distribution of some disembodied information.

Jean-Louis Boissier’s problematization of the notion of “capture” (saisie) supports our argument. For this artist-researcher, the data that make up databases are pieces of information that maintain, more or less directly, a contact with the real, like the plant specimens gathered by a botanist after conscientiously surveying the terrain²¹. These specimens are arranged in series and compared, in a process that actively calls upon human perception. While statistical models lend themselves to the description of these collections of data-objects, they are nonetheless deduced from very material samples, stemming from humans’ experiential contact with their environment. Thus, in resonance with our reflection, Boissier states:

Decidedly, after having said that a synthetic image “starts from nothing,” we shall say that it tends, on the contrary, to start from “everything²²”

In light of these remarks, we see Mitjanit’s autonomous agents not as entities stemming ex nihilo from mathematical processes, but as what they are in reality: the schematized result of a substantial process of capture and observation.

It is without forgetting these material and experiential origins of data and models that the notion of novelty should be approached within the framework of digital creation – a novelty that arises thanks to the artist’s intuition and intentionality, both of which are liable to be affected when the artist negotiates forms with the agency of matter.

Not disembodied information, but rather informed matter, the digital image is not to be excluded from the world of objects. Certainly, as a mechanically reproducible object, which can be multiplied far more easily than the analog image – think, for example, of silver-based photography – the digital image would be foreign to any aura. However, should not the Benjaminian condemnation to ubiquity be qualified once we recognize the importance of the spatial arrangement and rendering of the image when it is shown? Indeed, each presentation of the image is unique, including on screen: no two screens render exactly alike. In short, no two exhibition dispositifs are identical.

Despite this, reflection on the digital image is still subject to a tension, so much do discourses seem intent on problematizing it in its topological or logical state, namely as a list of I,J,R,G,B values. Yet the image we see with our own eyes is always concretely materialized. It has a spatial and object-like development. It is affected, depending on the case, by the ambient lighting, the size of the print, the material conditioning of the frame, the hanging height, the type of paper used to print a book, or the calibration of the screen or projection dispositif. In turn, the gestures we seek to characterize must be apprehended as interventions on the body of the digital image, on its physicality, or else on its physical and/or symbolic periphery – the latter also being fundamentally dependent on the materiality of forms.

After considering gestures that destructure the materiality of the image, we have, to sum up, approached gestures that produce a structural surplus, either by extending pre-existing structures or through the emergence of structures of a new nature. We have seen, with Menkman, Biermann and Arcangel, how creation could stem from a structural surplus on the periphery of the image – discursive, plastic or technical, respectively. We then entered the image to exhaust it – with John F. Simon and Jeffrey Thompson – to weave its threads algorithmically – with Jean-Michel Rolland – and, finally, we have just observed how the digital image can become fertile ground for Toni Mitjanit’s strange cellular automata. Through his experimental approach, this artist reveals the organic materiality from which the simulation algorithms he uses are built, in order to generate, on the plane of the image, animated textures of great graphic complexity.

The work of Simon Le Boggit, which we will see next, acts beneath the surface of visibility of the digital image, helping to further underline its stratified structure. Indeed, the visible surface of the image hides, just behind it, the logical level, where the interaction between data and code takes place.

1. We experienced the unimaginable extension of this space through the prism of Every Icon (1997) by John F. Simon and Every Possible Photograph (2013) by Jeffrey Thompson.↩︎

2. Miguel Rodriguez, Modélisation d’un agent autonome : Approche constructiviste de l’architecture de contrôle et de la représentation des connaissances. [s. l.] : Université de Neuchâtel, 1994.↩︎

3. Ibid.↩︎

4. See Flocks, Herds, and Schools: A Distributed Behavioral Model, Maureen C. Stone and Association for Computing Machinery (dir.), SIGGRAPH ’87 Conference Proceedings: July 27 - 31, 1987, Anaheim, California. vol. 21. New York, N.Y : ACM, 1987, p. 25-34.↩︎

5. Indeed, while cybernetics was not one of the main axes of reflection on the digital during the 1990s and 2000s, it should be recalled that the very first exhibition to stage the digital as an artistic means was entitled Cybernetic Serendipity. This famous, pioneering event took place in 1968 at the Institute of Contemporary Arts in London, with works by, among others, Vera Molnár, Frieder Nake and John Whitney.↩︎

6. See the explanatory document provided by the artist, entitled AAAC, An Autonomous Agents Choreo (2016).↩︎

7. Ibid.↩︎

8. Francisco J. Varela and Paul Bourgine, Autonomie et connaissance: Essai sur le vivant: Traduit de l’américain. Paris : Seuil, 1989, p. 45.↩︎

9. Number and location of the agents, starting directions and definition of zones of differential behavior.↩︎

10. Which we will now approach through an example before defining it more precisely.↩︎

11. See the popular science article by the mathematician and computer scientist Jean-Paul Delahaye, Le royaume du Jeu de la Vie, Pour la Science, no. 378, p. 86-91, April 2009.↩︎

12. Ibid.↩︎

13. See the popular science article by the mathematician and computer scientist Jean-Paul Delahaye, Le royaume du Jeu de la Vie, op. cit.↩︎

14. Edmond Couchot and Norbert Hillaire, L’art numérique. Paris : Flammarion, 2009, p. 101↩︎

15. Mark A. Bedau, «Weak Emergence ». Noûs [en ligne]. June 2008, vol. 31, p. 375–399.↩︎

16. Ibid.↩︎

17. Meeting with Henri Atlan, Qu’est-ce que l’émergence ?, Forum des images, 13 November 2019. Link to the video of the meeting: https://www.youtube.com/watch?v=dTlI4iDf2MI, accessed 17 February 2023.↩︎

18. For this instance we propose the notion of hypergesture.↩︎

19. Thus, for example, in the very recent and dizzying history of the neural networks used for the generative production of images, we find at work principles initially suggested by the North American cybernetic project. See in this regard the lecture Cybernétique, intelligence et imagination by Mathieu Triclot and Reza Negarestani, as part of ENS Postdigital, on 23/10/2017.↩︎

20. On the aesthetic plane, the first exhibition presenting works produced with computers, the iconic Cybernetic Serendipity, held in 1968 at the Institute of Contemporary Arts in London, already suggested an articulation between digital artistic creation and cybernetics.↩︎

21. See on this subject Jean-Louis Boissier, La relation comme forme: l’interactivité en art. Nouv. éd. augmentée. Genève [Dijon] : Musée d’art moderne et contemporain les Presses du réel, 2009, p. 31.↩︎

22. Ibid., p. 44.↩︎