Hyper-Collages, Algorithm and Big Data

Cory Arcangel, born in 1978, is a major figure of digital art in the United States. Among his projects are several that illustrate the strategies addressed in the previous section, in which the production of forms stems from the malfunctions, whether spontaneous or provoked, of technological means. His work Super Mario Clouds (2002), for example, with which Arcangel achieved notoriety, results from the modification of a Mario Brothers game cartridge, in which the artist erased all the elements of the game except the animated background, made up of clouds that travel across the screen from right to left¹. We could also mention Self Playing Sony Playstation I Bowling (2008), in which Arcangel alters not the cartridge of a bowling game on Playstation but the console’s controller itself, so that the game’s character plays in a loop, systematically throwing the ball into the gutters that border the bowling lane².

The Algorithmic Exploit

However, the work by Cory Arcangel that particularly interests us in the framework of this research does not result from a technological malfunction or détournement. It is the video work Paganini Caprice No. 5, made by the artist in 2011 with the help of a small army of software programs³, combined with algorithms designed by the artist himself, without any détournement of the computer’s default functioning being discernible in his creative process.

In this 3m41s video, Arcangel recreates one of the most famous pieces by the virtuoso violinist Niccolò Paganini (1782-1840) through a succession of several hundred very short video sequences – most lasting well under a second – extracted from electric guitar tutorials of the kind found by the millions on the web. Thus each note of Caprice No. 5 corresponds, in the video, to a sequence of the same duration, at the precise moment when a guitarist – often an amateur – plays the same note in the course of their tutorial.

This process of searching for videos, characterizing them (according to the notes played) and assembling them (following the model of Paganini’s piece) results, on the sound level, in a fairly recognizable rendering of the original composition: indeed, the timbre of the electric guitar, and even the vibrato on the held notes, come close to certain performances of Paganini’s work on the violin. The visual register, on the other hand, suffers from a lack of legibility: it consists of a hyper-rapid succession of sequences whose brevity makes them difficult to apprehend, especially for the shortest notes.

From the outset, the video thus presents itself as unbalanced, with a “coherent” sound register, made up of the ascending and descending intervals of the melody, superimposed on a choppy and disruptive visual close to flickering, resulting from a machinic, non-human montage⁴. At times, however, the visual and sound registers seem to come together when, on the one hand, a gestural continuity in the guitarists’ hands emerges from the succession of micro-sequences while, on the other, the A minor melody evolves toward a succession of unresolved and vertiginous intervals of intermediate notes.

If X Then Y (Sensing the Medium)

Although the artist is presented as “post-conceptual⁵,” his work rather reflects an approach anchored in experimentation, in which the artist deploys considerable computing know-how. It is also important to note that, among the pillars of his career, Arcangel acknowledges his contact with experimental video artists in his hometown, Buffalo, where he studied art⁶. Thus, in Paganini Caprice No. 5, the artist’s strategy seems close to the “what if?” approach, that is, when an artist wishes to know the result of an action on the real, an action that is often offbeat, but in which they sense a formal potential. We find this approach at the origin of the video Around & About (1980) by Gary Hill, one of the artists who taught at the University at Buffalo. Hill made Around & About from an introspective text, the reading of which is accompanied by photographs of his office at the university, at the rate of one image per syllable⁷.

With Paganini Caprice No. 5, Arcangel seems to go further than the “what if?” strategy, transforming the question into a conditional implication, “If […] then […]”: if the iconosphere (the visual part of big data) is so vast and dense, then it should be possible to reproduce a song, even a very complex one, by assembling video excerpts that will have to be searched for and processed, but that must necessarily be available on the Internet. The artist achieves this by designing and implementing dedicated algorithmic procedures⁸, thus making perceptible the colossal volume of images available online.

A work of this type, resulting from a massive process of searching, cutting and re-pasting, owes its depth less to its sensible expression (the image and sound perceived by the spectator) than to its own creative process and its implications. Indeed, Caprice No. 5 is conceivable only thanks to the immense quantity of videos of this kind (tutorials and covers produced by amateur guitarists at home) available online⁹. Thus the work cannot be fully grasped without recognizing an artistic gesture that intuitively anticipates, and exploits, the new possibilities offered by the medium: the power of algorithms, the state and evolution of the Internet, and the totalizing tendency of big data.

A Network of Images

In light of Paganini Caprice No. 5, the digital image appears, on the one hand, characterized by a plasticity attainable only by algorithmic means and, on the other, inscribed in its natural environment: networked with millions of other images, within a gigantic collective database – humanity’s big data – which never ceases to expand and grow denser, thus flirting with formal continuity¹⁰. While the digital image will appear to us, through the prism of Jean-Michel Rolland’s work, as cohabiting with its distorted reflections and its ghosts, Arcangel’s work reveals it in its natural interconnection with a vast universe of other images. These images interact and transform themselves fruitfully, particularly when they become the raw material of the algorithm, directed by an artistic intention conscious of its medium, which is questioned in return.

Certainly, there would be aspects to consider by mobilizing “database aesthetics¹¹.” These, however, rest on the idea of the database as the fundamental unit of a real that tends to be reflected upon on the symbolic plane, the last resort of an approach to the digital that misses its appointment with the concrete, material conditioning of the medium.

The intentionality – here Arcangel’s – that scrutinizes the medium while weighing its creative potential in no way does so on a symbolic basis, according to which, in a database, any datum may be present or absent, and all data may be related to one another infinitely. On the contrary, the weighing that grasps the possibility of the creative gesture knows that this gesture must be delimited within its real possibility, that is, within algorithmic plausibility and the computational resources available. Symmetrically, the medium is scrutinized without neglecting the affective aspect, the attachment or indifference of humans toward certain types of images, which ultimately determine their massive presence or relative absence in the system.

Consequently, the structure resulting from the algorithm, itself mobilized by the artist’s gesture, gives rise to an experimental video, certainly, but it must also be apprehended with respect to its physicality: as a surplus of connectivity – physical (technical), and therefore sensible – built between images.

Cinema, Television, Internet. The Human Jostled.

Broadening the reflection prompted by this work by Arcangel, which stages the networked digital image, we can discern a trait common to our era. It is the rather rapid internalization – the naturalization – of the new technological modalities of representation and remote vision stemming from the increased interconnection achieved with the Internet. In this respect, the Internet certainly extends the effects of the technological revolutions in images that preceded it: cinema and television. Indeed, as Manovich points out¹², drawing on Walter Benjamin and Paul Virilio, cinema (according to Benjamin) and the transmission of images (according to Virilio) contribute to “compressing” time and space, which entails nothing less than a transformation of human nature itself.

In our framework of thought, in which the human and technology co-evolve, bound by a transductive relation, we interpret the effects of cinema and television as a displacement of this relation: the appearance of cinema and the increasing fluidity of image transmission materialize new technological structurations, which in turn condition the subsequent evolution of the human structuring of space-time. If the moving image (cinema), followed by the mobile camera¹³, then the transmission of television images induced a mutation in the human apprehension of time and space, this work by Arcangel makes tangible the extension of this tendency in which the human and technology structure each other. It echoes the ongoing formation of an immense worldwide archive of images, big data, which constitutes the foundation and condition of possibility of generative artificial intelligence, the next milestone to which the human-machine dialectic will have to adapt.

Thus it is by making use of big data, a specificity of the digital medium, that Arcangel builds Paganini Caprice No. 5. Within big data, images – like texts, sounds and code – are networked, that is, accessible from countless points of entry, but also interconnected in multiple and diverse ways. The artist creates his work by structuring a new form of interconnection: audiovisual flows that follow one another according to a pre-existing pattern, the notes of Paganini’s work.

Before Arcangel, whose gesture structures a “surplus of network,” we sought to grasp how Biermann produces his work by setting up a new plastic level, in which Hollywood imagery is algorithmically reassembled. And before Biermann, we saw how Menkman structures her work by producing a narrative surplus. These three works seem to act, as mentioned earlier, outside the images themselves, by extending symbolic structures – discursive, plastic or technical – stemming from the network.

It is now time to explore gestures that generate structures inside the image, along a path that highlights a stratified image, composed of active layers superimposed on one another. These images present themselves according to an easily apprehensible structure, but behind this apparent simplicity lies an infinite – or nearly infinite – universe of possible forms.

1. The artist has made the code that allows him to carry out this détournement available on the GitHub platform, thereby bearing witness to his adherence to the principles of sharing knowledge and methods characteristic of the Open Source community.↩︎

2. Arcangel modified the Playstation controller by clipping onto it a makeshift circuit extension, an intervention that extends the practice of circuit bending. The latter, popularized by artists such as Reed Ghazala (1953-), consists of manipulating the electronic circuits of simple devices, often toys emitting lights and sounds, in order to generate unforeseen effects. Circuit bending can be seen as a “hard-wired” version of writing and executing an algorithm, the latter associated with the temporary modification of a “plastic” electronic circuit. Circuit bending makes the gestural filiation of algorithms tangible.↩︎

3. The software used: ffmpeg, sox, g++ and MAX/MSP.↩︎

4. The editing in this work does not result from a sensible decision based on the images, but mechanically follows the succession of notes in Paganini’s melody.↩︎

5. The artist’s page on the Whitney Museum of American Art website. Web link: https://whitney.org/artists/8339 visited 28 August 2024.↩︎

6. Arcangel says he was a student of experimental video and film artists, notably Steina and Woody Vasulka, who had settled in Buffalo after leaving New York. See in this regard the interview with the artist during his exhibition at the Lisson Gallery in London. Website: https://www.youtube.com/watch?v=u7keMVNTTCs, accessed 28 August 2024.↩︎

7. The video, available on the artist’s website, exploits, at the same time as it reveals, the immense possibilities of variation that arise at the level of capturing and rendering images. Indeed, Gary Hill explores in depth changes of point of view, framing and focus during capture, while the images are rendered reframed and repositioned on screen in extremely varied ways. Web link to the work: https://garyhill.com/work/video/around-about.html, accessed 29 September 2024.↩︎

8. A principle of this kind is also at work in Postdigital Flipbook: “if such a volume of photographs of faces is present on the Internet, then it should be possible, through the succession of sufficiently close images, to create something akin to an animation.”↩︎

9. Arcangel had already produced a very similar work in 2009, taking advantage of another type of video proliferating on the Internet. In Arnold Schoenberg, op. 11 - I - Cute Kittens, Arcangel recomposes a piece by Schoenberg from the sequencing of hundreds of micro-sequences extracted from videos of cats walking on piano keyboards.↩︎

10. This explains the quasi-magical behavior of generative neural networks, whose effectiveness rests on interpolations across immense, hyperdense “data sets.”↩︎

11. These will be specified later, in the paragraphs that seek to reflect on these structuring works as a whole.↩︎

12. Lev Manovich, The language of new media, op. cit., p. 170-175.↩︎

13. As masterfully staged by Dziga Vertov in his iconic 1929 film, Man with a Movie Camera.↩︎