Yanguin (2023)

Title: Yanguin (2023)

Yanguin (2023) is a collaborative work by Dr. Mark Pilkington (Thought Universe Studio, Manchester) and Dr. Oliver Carman (University of Liverpool) investigating the integration of sound and image in the composition of electroacoustic music. Applying an animated graphic score to interact with electroacoustic sounds derived from the Chinese instrument the Yanguin - a hammered dulcimer.

Description of Sound materials
Please add Description of your sound processes - OLiver

Initial Steps
To begin with the sound sources were processed to create a sound library for the new work, always keeping the sketch in mind. Transformations tend to fall into a number of categories each moving further away from the initial source material (much inline with Smalley’s (add reference Link) gesture surrogacy concept) Transformation of pitch, reversing, equalistation, compression. (basic sound processing with DAW), compositing materials into sound objects for further processing. Cross synthesis (Cecelia) vocal sounds and FM sounds combined to create crossover between organic and digital. Extensive granular processing, filtering. Starting at step 1 again but with the newly processed materials.

Development
Selection and synchronisation of existing sound materials to a short 45 second video that has been created in Processing and provided by Mark. At this stage it is already clear to me that a number of materials stand out from the sound library that will be suitable for the video. Sound processing at this point tended to be modifications to make materials represent the image more effectively i/e extension of resonant properties to a sound through granulation, changes to envelope or panning to reflect placement or movement of an image. A subsequent 1 minute of material was then composed based on developing these opening materials. This was then sent back to Mark for the video to be created. This process continued back and to through the duration of the creative process.

1. Yangqin(2023) - audio

Text by Dr. Oliver Carman - Feb 2024

Visual Processes of Yangqin

A number of visual processes were used in Yangqin. A fine art sketch (fig. 1) provides the intial lmaterial, conceived without a specific concept purely expressing my own thoughts and feelings. The sketch forms an imaginary landscape containing architectural zones of interaction. Consieration is given the idea of social morhology; an interaction of space - the consideration of how we interact with sound and visual in the space in an urban enviromental setting. I see my work as a continuous process throughout the day in different mediums. where from waking to the sleeping, covering myself the ability to code the visuals allows many different intrepations of the final piece - easy to radical alter the structure.

Expresses an interchage of ideas and methods by the identification of algebriac correlation with geometrical locus. Numerical relationship can be expressed as spatial ones, lines are changed into numbers and numbers into lines; a mathmatical progression where knowlodge of the terms follows of necessity from knowlodge of another. Striving for a unification of art and science. The efficencey of the system - iRCAM researcher express concern over the lack of efficency in machine learning - quote.

Fig.1 The original hand drawn sketch for Yanguin by Mark Pilkington.

Animating with Code
The original sketch (fig. 1) is animated using the visual programming language Processing to form a temporal framework. This is then used to align the sonic treatments developed by Oliver. Processing is a flexible software sketchbook and a language for learning code within the context of the visual arts. By listening back to the audio I would begin to programmed a series of dynamic visual processes following the morphological structure. Through listening we become acquainted with movement as a perceptual correspondence. That signifies zones of elements weight and desnity Analyzing the audio through listening to the recording - computation analyses In regards to listening Michel Chion, states there are three modes of listening causal, semantic, and reduced (Chion, 1994. p. 25). The modes refer to different constituents of meaning-creation in the process of listening.

Firstly, the duration of Oliver's sound file was mapped to the resolution of the screen. Embedded into a Processing sketch using the minim API sound library. Visual cues were meticulously plotted by hand tracking the onset of each sound. The temporal framework produced provided a method to see the motion, revealing the spatial position of each sound entity. Parameters of the sound material, density, decay, onset, spectral morphology The process of coding continued until a state of equilibrium is reached between the visual and audio elements. Working in 2 dimensions highlighting the underlaying the visual elements in the hand drawn sketch. The piece exists as a self-contained application (fig. 2).

Fig 2. A screen grab of Yangqin (2023).

Live Performance
Yangqin (2023) can be performed as a standalone application (macOS, Windows and Linux) or linked to control additional sound devices. During a performance the algorithmic system is designed to be in a continuous state of flux, being interactive the piece can take on many forms and meanings depending on control input from a user.

Fig 3. Online performance video of Synergies for Art in Flux 2020.

Conclusion

Text by Dr. Mark Pilkington - Feb 2024

Fig 4. A p5js sketch.

Bibliography
Descartes, René. Trans., by Sutcliffe, F.E. Discourse on Method and Other Writings, England, 1968.
Chion, Michel, Audiovision: Sound on Screen, New York, 1994.
Watson, J, Guattari's Diagrammatic Thought - Writing between Lacan and Deleuze, Continuum, London and New York, 2009.