Transposing the Sensible

Within the digital, visual data can be “read” to give form to a sound register and, conversely, it is possible to generate an image from a sound file (.wav, .mp3)¹. This property, already present in analog media², intensifies with the digital, making the possibilities of intermedial experimentation more immediate. Numerous visual essays and micro-works, shared on social networks or specialized forums³, have thus come into being, with diverse aesthetic consistencies. Most of the time, the passage of data between sound and image is carried out in the quest for glitches, potentially interesting anomalies to be discovered in the image or sound after forcing the data to transgress their respective medial boundaries, that is, by manipulating an image in sound-processing software and vice versa⁴. In the following paragraphs, however, we will look at a work that modulates the sound-image articulation within the digital differently, ensuring a form of coherence between the two registers without resorting, as do the works studied in the previous section – Destructurations – to an “off-program” functioning of the machine.

When Sound Tears the Image

With Entwining of Lost Souls, a 9m38s video made by Simon Le Boggit in 2022, we find ourselves facing a hybrid entity in which sound and image are intimately intertwined. On the screen, musical notes played on the viola materialize, taking the form of traces of highly saturated colors, covering the extremes of the chromatic spectrum and evolving in perfect synchrony with the sound. These notes, prolonged by long vibrati, follow one another without forming a melody, but create short intervals of varying expressiveness (dramatic, melancholic, mysterious…). Each new note tears the image in a different direction, imprinting on it a visual vibration that fades at the same time as the sound. Everything happens as if a brush of sound and light were acting on the frame, depositing matter while redistributing what is already there, like a watercolor briefly seized by an intense oscillation. The colored strokes are superimposed one on top of the other, animated when they are “deposited” but quickly frozen: as soon as “their” note has died away, each pictorial stroke melts into the colored background of the image, which itself reveals another background, black this time, in a visually stratified image. The new traces spread out, deforming the old ones, as in a sort of fluidified palimpsest. At moments, the traces of sound-light present themselves as part of flat surfaces crossing the frame obliquely, surfaces that redefine their spatial inclinations with each note. The image then struggles between the flatness of the frame and the illusion of spatial depth.

Entwining of Lost Souls mobilizes the digital image in a singular way, revealing its almost infinite elasticity. The work makes the image react in its liquid dimension, as if it were composed of flows running in every direction. Like several works in our corpus, Le Boggit’s reveals the magmatic flesh of the digital image, made of electrical signals that escape human perception.

The Image, Memory of Sound

Beyond the medial aspect, a few elements of context will allow us to grasp one of the main stakes of the work. Simon Le Boggit reconciles his artistic practice with a difficult personal situation. Indeed, this artist, who comes from the world of music, has suffered a progressive loss of his hearing, which may worsen in the future⁵. The artist thus takes advantage of the transcoding capacity of digital technology to transpose the sound universe into the visual register, preparing himself, in his own words, for the day when he will, unfortunately, have lost his hearing completely⁶.

In this context, we may ask ourselves a fundamental question: can this passage from sound to the visual take place transparently? Or will we have to anticipate alterations, slippages, that will affect sound forms in their passage toward the visible? From this question follows the obvious fact that the domains of sound and image have specific characteristics that make them fundamentally different, a reality clearly manifest in the work, despite the degree of intertwining that occurs in it between sound and image. Indeed, while sound establishes a privileged sensible connection, able to structure states of mind directly and instantaneously, the visual is more spatialized and seems to maintain a stronger link with the “intellectualizable” and with the symbolic.

However, as this research never ceases trying to underline, before it is populated by symbols, the visual is made up of traces, and it is these traces that ultimately operate within the work. It is through the visual that we gain access to the “past” of the image, but also of the sound in the case of this work, so close is the sound here to the visual. Indeed, while sound, especially in this atonal composition that ignores harmony, retains little memory of the preceding notes, the image preserves the visual impressions of past notes more durably. Thus it is at the level of the memorial transfer from sound to image, particularly legible within the digital, that one of the active fibers of the work lies. The work underlines, in a particular way, the synesthetic possibilities of the medium.

From “Language” to Trace

When we study the details of the creative process behind Entwining of Lost Souls, an essential aspect emerges that enriches our reflection. From a processual point of view, what is at stake in the work lies not so much in a passage from sound to image as in a simultaneous generation⁷. Indeed, in Entwining of Lost Souls, sound and image were created simultaneously, from the same data, by algorithmic means⁸. The work thus takes advantage of, while shedding light on, the characteristic proper to the digital mentioned at the beginning of these paragraphs: namely, the capacity of the digital to bring media together – sound, image and even, to a certain extent, text. Yet this characteristic is often wrongly presented as a major rupture with artistic tradition. In this respect, we can find in the literature:

[…] digital art calls into question the traditional categories of art. It is indeed founded on a single language (binary language) […]. Digital art calls into question the separation between the arts of space and the arts of time, for example, or the distinctions internal to the visual arts, between painting and sculpture⁹.

Can we really subscribe to this “specificity” of the digital, formulated in this way on the basis of a “binary language” whose contours and characteristics as a language remain to be specified¹⁰? There has never been any question of considering painting (or drawing) and music as belonging to the same “medium,” on the grounds that these two practices find a support of inscription of the same nature (canvas, paper score).

The specificity of the digital lies in the fact that, where we need a musician to read and perform a paper score, the computer can “read” the same traces both as image and as sound. This common formal substrate certainly contributes to the increased possibilities of synchronization between sound and image within the digital, but only because it is read with exactitude by the machine either as sound or as image. Thus it is not at the level of the support of inscription that the digital brings media together, but at the level of automatic reading. This distinction is essential¹¹.

The Digital Support and the Sensible

If the digital offers more fluid bridges between sound and image, bridges that Entwining of Lost Souls takes advantage of, does it really merge the categories of art? Nothing is less certain, for this must be assessed on the sensible plane, where sound and image remain what they have always been, even in the digital. From a perceptual point of view – what happens to us – the visual and the sonic continue to express themselves in the classical way, as in the analog: we perceive the image with our eyes, thanks to a visual signal – informed light – while sound is picked up by our hearing, thanks to a signal that takes the form of pressure waves in the air. We owe to Lev Manovich a distinction, which we have already mentioned, and which once again helps us clarify the reflection at this stage. Indeed, in the digital, sound and image share the same origin at the level of what Manovich calls the computer layer, while at the level of output, sound and image manifest themselves in their classical registers: they are perceived and interpreted by the spectator within what Manovich designates as the cultural layer. This distinction, for all its simplicity, sheds a great deal of light on the functioning of computers and makes it possible to qualify, by making them more precise, the discourses that seek to “homogenize everything” when it comes to the digital. Le Boggit’s work, by bringing sound and image into intimate dialogue, makes their common “genetic” origin sensible, while highlighting the expressive qualities proper to each of these two fundamental registers.

Entwining of Lost Souls helps us delimit precisely, within the computer layer, the entity correlative to a sensible object (sound, image, text) manifesting itself on the cultural layer. Indeed, it is not enough to confine ourselves to the data specific to each object¹²: if we wish to pin down the “active instance” of the sensible object (sound or image), we must also take into account the algorithmic component that “reads” these data. Schematically, one could say that the traces (data), specific to each sensible object, are read by a signal already formatted – preformatted – by a reading algorithm, which differs depending on whether it is software for playing sound or for rendering digital images.

Electricity as Material

More fundamentally, this work by Le Boggit helps illustrate the blind spots of the discourses that seek to level all media within the digital, to make them equivalent. These discourses with ontological ambitions seek to bring sensible objects closer to human language, and thereby to wrap them in an immaterial and transcendent guise. Entwining of Lost Souls bears witness to the fact that, in the light of artistic making, the functioning of the digital becomes clear in a natural way: the internal material domain of the digital image reveals itself. Moreover, every artist knows – even if they would never admit it – that every art is the art of something¹³.

Entwining of Lost Souls, in the end, reveals the unprecedented capacity of the digital to redirect flows of energy in an extremely complex yet controlled way, and that is precisely what algorithms do. Often presented as series of instructions stemming from “language,” they are better apprehended as a highly sophisticated ensemble capable of dividing, directing, combining and organizing an undifferentiated electrical flow that enters the computer through the power cable. Stricto sensu, and for all their complexity, computers do nothing else. Part of the electrical energy is transformed into light by the display cells of screens, another part is transformed into sound by the membranes of loudspeakers or headphones. In painting, there was a time when it was useful to remember, with Maurice Denis, that a canvas is first of all “a flat surface covered with colors assembled in a certain order¹⁴.” Likewise, in the digital, it is perhaps time to recognize that a computer is first and foremost an extremely sophisticated dispositif that manages to process an electrical signal in a highly elaborate way. Entwining of Lost Souls was built by taking advantage of this fundamental characteristic of computers.

When the electrical signal interacts with traces, even when it produces them, it is informed by them. The code that Le Boggit implements produces traces – data – that can be read both as sound and as image. A signal designed to read a sound can be directed toward reading the data belonging to an image, and vice versa, which succeeds in establishing, very concretely, a medial bridge. The data being identical, the nature of the form produced will depend on that of the signal that interprets these traces.

It is on the plane of these hyper-fast electrical flows, informed by the microstructure of the support, that the true mode of existence of digital technology should be sought. So that they can be controlled by users, these flows and traces are transposed into the form of code – or interfaces – and data: flows and traces are “mediated.” Le Boggit’s gesture, passing through the algorithm, seems to reach the fundamental bedrock of the computer’s functioning by transforming the very substance of the digital image. It is at this level that Jacques Perconte also intervenes, an artist who uses tools built by diverting compression codecs, but who, above all, composes with them from the primary fiber of the digital image. Under the artist’s aegis, and beneath words, the image begins to breathe and, at once, to tell. This happens through the modulation of a magmatic plasticity that seems to pierce all the layers of mediation to offer itself directly to human sensibility.

1. Likewise, most files can be opened as text files, even if the text will often remain incomprehensible to any reader. This principle is at work in one of the most popular effects, the WordPad Effect, which consists in opening an image file with the text editor WordPad, then saving it after modifying the encoded data of the image, displayed by WordPad in the form of ASCII characters. As a result of this operation, the image will be “glitched.” See in this regard http://blog.animalswithinanimals.com/2008/08/databending-and-glitch-art-primer-part.html visited 17 September 2024.↩︎

2. See for example Nicolas Maigret’s work Flat Earth Society (2008), in which a vinyl record was engraved so as to reveal an image of the terrestrial globe. Website of the work: http://peripheriques.free.fr/blog/index.php?/works/2008-earthtodisk/, visited 17 September 2024.↩︎

3. See for example the blog datamoshing.com.↩︎

4. Another similar effect consists in opening an image file (for example a .bmp) with Audacity, a sound-processing program, and then saving it after modifying it.↩︎

5. See the interview with the artist during his exhibition at Fabrica Gallery, UK, in 2021. Web link: https://www.youtube.com/watch?v=tn-K5q1d8Gs, accessed 22 July 2024.↩︎

6. Ibid.↩︎

7. The British experimental filmmaker Malcolm Le Grice (1940-2024) took an interest in this kind of simultaneous generation in his film Heads I win, tails you lose (1984). Thanks to a program written by the artist himself, he builds a dynamic collage in which sound and image are organically intertwined.↩︎

8. See in this regard the description of the work on the artist’s website: “An algorithmically composed duet for viola, accompanied by an abstract CGI animation video generated from the same data set as the music.” Web link: https://quantumsouporchestra.weebly.com/ accessed 22 July 2024.↩︎

9. Fred Forest, «Esthétique du numérique : Rupture et continuité: ». L’Observatoire [en ligne]. December 2010, vol. Hors-série 3, n° 3, p. 10–16.↩︎

10. We will see, in chapter 3, the need to draw a clear line between human language, on the one hand, and what is called machine language, on the other.↩︎

11. The distinction between reading software on the one hand and stored data on the other will help us to better describe the digital image in relation to its material conditioning, in the section xxxxx (processual / non-essentialist ontology of the digital image)↩︎

12. For example, a .wav file for a sound, a .jpeg file for an image, etc.↩︎

13. At the extreme, Tino Sehgal works with the human voice, language, movement and interactions, certainly, but interactions between concrete elements.↩︎

14. Jacques-Émile Blanche, Propos de peintre. Paris : Editions Séguier, 2013, preface, p. XXXII.↩︎