[Source: MOTT, IAIN. Zhong Shuo (People say)... . In: Ros. Bandt; Michelle. Duffy; Dolly. MacKinnon (Eds.); Hearing Places: Sound, Place, Time and Culture. p.389-401. Newcastle: Cambridge Scholars Publishing, 2007.]
In 2004 I began planning a narrative-focussed sound art project for China. The project has involved the creation of installations distributed in different parts of the country, that collect and replay site-specific stories from the public on change. Now in its final phase, the project is entitled Zhong Shuo (ÖÚËµ) which translates as ¡°People Say¡± in English. By creating installations in Beijing and Chongqing in 2005, the work strove to establish a discourse locally and between sites. Collected stories and environmental sound recordings were played back at each installation and shared between sites over the internet, enabling individuals to contribute stories and reflect on those of others across geographical divides. The project will culminate with an exhibition in Brisbane featuring translated text synchronised to the audio playback of stories. A fourth China-based exhibition will be staged in Shanghai to run concurrently with the Australian exhibition and will collect additional stories from the local people.
Background and Development
Zhong Shuo began early in 2004 when I visited Beijing to exhibit a binaural video installation Close (Mott, 2001) at the Dashanzi International Art Festival, and to investigate staging the then untitled narrative project in China. At the time, I had recently completed a project in the Netherlands, Summoned Voices (Mott, 2003) with designer Marc Raszewski, which involved cumulative verbal interactions by the public with three networked kiosks. The work set out to establish meaningful interaction with the public, whereby participants would contribute fresh content to the work as they engaged with it. Each interaction added audio material (voice, singing, mobile-phone noises, Walkman recordings) to a database of material, and a computer system was used to analyse each new recording and produce a playback synthesised from the most recent entry and fragments of previous entries selected on the basis of sonic similarity. Summoned Voices encouraged participants to engage in a form of vocal play, contributing content and listening to results. The kiosks were distributed in a large building but shared a central database of sounds and analysis information.
The experience of producing this work led to a desire to explore a new work focussed on narrative and less focussed on the sound of the voice. Where Summoned Voices broke sentences into de-contextualised fragments and produced a response to interaction in a form of musique concr¨¨te, the new project would find ways of mixing voices while maintaining the linearity and coherence of the narrative. The work would also encourage story telling by asking specific questions of participants. The distributed nature of Summoned Voices was originally created to amplify the presence of the work and to maximise access. As a viable technique, it posed an enticing challenge to work in a much larger geographical context and to explore notions of place and local identity. Distance and the isolation of individual installations would introduce regional variation in accents and dialect, along with divergent ideas and expression, resulting from the different environments.
In Dashanzi while exhibiting Close, I was introduced to the Long March Foundation (LMF) (Long March Project) as a potential collaborator and ultimately LMF invited me to participate in their Long March¡ªA Walking Visual Display project. The foundation is engaged in local and international curatorial projects, including this ongoing project, that consists of contemporary art installations and events staged along the route of the historic Long March. One of the main aims of the project is to counter a perceived shift of focus of contemporary art away ¡°from the broad masses of the people toward the elite, from private studios towards hierarchical structures ¡ and from China toward the world beyond China¡± (Lu & Qiu). This curatorial scope and sensibility offered great potential with existing venues and support networks throughout the country. The environment of China itself is also conducive to grand narrative, with its dramatic social upheavals, both past and present, widely varying landscapes, ethnic groups and social stratifications.
The visit helped to consolidate and focus the concept of what became Zhong Shuo and the activities of the LMF and its contributing artists served in part as inspiration. The early concept was presented at a conference in Beijing late in 2004 under the working title Chinese Whispers (I. Mott, ¡°Chinese¡±). A period of software development took place in Melbourne before I departed for Beijing for a four month artist residency hosted by the LMF in July 2005.
In Beijing I set up base at the LMF's Long March Space in Dashanzi in the north east of Beijing. I worked with the Director Lu Jie and International Director David Tung to define potential installation sites and artistic collaborators. Three sites were mooted, one at the Long March Space itself, a second in the city of Chongqing and a third virtual installation in Yanchuan County which would concentrate on story gathering rather than audio playback at an installation. Construction of the Beijing installation commenced first and served as a model for subsequent installations, both technically and aesthetically.
In Beijing I worked with the visual artist Ding Jie to create the installation. Later in Chongqing, I worked with the Li Chuan Group, an artist collective comprising Li Chuan, Ren Qian and Li Yong. The Yanchuan project unfortunately did not eventuate due to the planned visit coinciding with the harvest. In Beijing, Ding Jie's design established an environment for public interaction that offered a welcoming gesture to visitors. Created in the form of a Chinese garden, a kiosk¡ªsimilar in style to small food kiosks found in the outskirts of Beijing¡ªhoused a modified telephone for contributing stories and an adjoining pond played host to a small school of goldfish and water-plants. Distinctly Chinese, it was a peaceful setting, and offered a degree of privacy for those who wished to tell a story. Since we had no way of predicting how the local public would react to the work, a private space was considered prudent. Hidden away in the rocks and shrubs by the pond was a loudspeaker which played a steady stream of stories that were automatically recorded and edited by a computer during the course of the interactions. This sound, in addition to the voices, included ambient sound recordings collected locally and was edited in the style of radio documentary. Stories were interwoven usually with three narratives interspersed at any one time. Interwoven stories were played in their entirety from beginning to end.
Banners were included at the installation. One broadly described the project and another offered instructions on how to participate. The instructions were in Chinese and English and were as follows:
Use the phone in the kiosk to tell a story about Beijing. How do you experience the changes around you? How does change affect your hopes for the future? How is your life different from that of your parents¡¯ generation? Pick up the phone and listen for directions.
While an English translation was provided, all voice prompts over the telephone were in Chinese. In the case of Beijing, the dialect used was Putonghua (Mandarin). Chongqinghua was used subsequently in the installation in Chongqing.
The visitor's experience and demographics
The Long March Space in Beijing is situated in the Choayang District in the north east of the city. The site is part of the 1950's built 798 factory complex which today hosts a mix of art galleries (which are under domestic and foreign direction), artist studios, architects, designers and a variety of factories including electronic component manufacturers and a regional power station. The Dashanzi International Art Festival is held in Dashanzi each year and has helped to raise the profile of the district and secure its survival in the face of rampant land development. The increased attention to the area has led to the unfortunate result of increased rents, which has affected the viability of the district as a studio venue for less established and emerging artists. The development of the area as an artist community at first happened due to the suitability of the Bauhaus style buildings for studios and cheap rents. The increased costs of operation in Dashanzi now favour more commercially oriented enterprises.
Visitors to the Beijing installation included the local art-going public, factory workers and expatriate visitors and tourists seeking out contemporary Chinese art.
As people approached the courtyard of the Long March Space, they would hear voices coming from the direction of a kiosk under the courtyard's canopy. They would hear people speaking about Beijing, giving opinions on change, and their experiences of living in the city. Some visitors would choose to listen by the pond while looking at the plants and goldfish. They read the banners with instructions and would peer inside the kiosk through the blind. Inside were clean silver walls. The interior was lit from above and on a shelf was a telephone with a small plaque next to it repeating the instructions. The telephone rang periodically to encourage people to pick up the receiver although it could be used at any time. When someone picked up the phone, they would hear a number of questions about change spoken in Chinese. The voice advised they could speak for up to 10 minutes after the beep. Once a story was recorded a chime would sound from the garden and a recorded voice would acknowledge receipt of the story, stating it was scheduled for playback after 15 minutes.
Work in Chongqing with the Li Chuan group commenced after the opening of the Long March Space installation. Chongqing is an industrial city of six million set in mountains. Once part of Sichuan Province, Chongqing, with its new surrounding province by the same name, is a special municipality directly administered by the Central Government. Like Beijing, the city is booming, yet it is markedly different in character with narrow winding streets, a more regional ambience and in my own experience, with frenetic taxi drivers every bit as overheated as the local Sichuan food.
Work with the Li Chuan group commenced at break-neck speed. Li Chuan, Li Yong and Ren Qian have been working as a collective for over four years and are a tight creative unit. With the aid of an interpreter, Liu Wen Bo, we defined an installation strategy and secured the venue of the new Chongqing Planning Exhibition Gallery, all on the day of my arrival from Beijing. Our exhibition coincided with the opening of the building, which includes four levels and 30,000m2 of exhibition space dedicated to urban planning and real estate ventures. A more suitable if not ironic site for an installation on change could not have been wished for.
Chongqing is known locally as the ¡®Mountain City' and also ¡®Fog City' due to the entrapment of fog and smog by the surrounding mountains. Until recently the hillsides were dotted with traditional stilt houses clinging to sharp inclines. The group quickly decided upon these houses as a design model in answer to Ding Jie's kiosk. We originally planned to build the installation as a mezzanine against a supporting wall, with a ladder provided for public access. Unfortunately the spaces provided at the gallery proved unsuitable, so a bungalow style was chosen using a traditional bamboo construction.
Calligraphy was included on the kiosk and used characters chosen by Li Chuan's father Li Cheng Fu, a poet. The characters have a direct translation in English but I understand from the group they have multiple layers of meaning relating to Chongqing's past. The characters read: ¡°People Say House¡± (ÖÚËµ¾Ó), ¡°The machine will broadcast throughout the world¡± (»úÃùÌìÏÂ) and ¡°Bamboo symbolises good luck¡± (Öñ±¨Æ½°²). The group created a rock-garden next to the kiosk in which sat the loudspeaker. The garden was constructed with stones from the Yangzi River (Chang Jiang) which flows beside the exhibition space. The Chongqing installation had a nearly identical technical setup to that in Beijing and used similar instructive panels.
Unlike the Beijing installation where visitor numbers were moderate, the new Chongqing Planning Exhibition Gallery attracted thousands of visitors each day. The installation was at the gallery for one month and collected over six hundred entries in contrast to Beijing where just under five hundred stories were collected in two months. The visitors constituted a general rather than art-going audience and, from my observations and in my opinion, represented a fair crosssection of the community. Almost all the entries were spoken in Chinese and I have noticed only one recording in a European language. The gallery had the atmosphere of a high-tech shopping centre and our installation stood as a strangely antique reminder of old Chongqing in amongst the escalators, neon signs, plasma screens and projected flythroughs of proposed development projects.
Transfer, Transmission and Ambient Sound
Both the Beijing and Chongqing installations were connected to the internet and once the Chongqing installation opened to the public, stories began to be shared between sites through an automated transfer of audio and database files. The period in which the Chongqing installation operated coincided with the last month of operation of the Beijing installation and transfers occurred daily.
Intermixed with the voices were ambient field recordings that I made in both cities. The recordings were used as signifiers, as well as being mixed beneath voices, and were also used to introduce and contextualise a new story from a particular site. In Beijing I recorded not only streetscapes, but also advertisements, pop songs and soap operas from AM radio. I also included, with permission, music by the Beijing electronica composer Hz (aka Hu Zi). Perhaps the most iconic sound recorded was that of the local daily newspaper vendors; rhythmically looped megaphone announcements heard in almost every commercial street in Beijing. In Chongqing I recorded the city's newly constructed mono-rail (of which the population is very proud), fog-horns sounding from the river, street hawkers and restaurants.
For Chinese listeners, dialect is probably the greatest signifier of place in Zhong Shuo. As mentioned, all pre-recorded messages and announcements were recorded in local dialect, and this was done to encourage participants outside Beijing to speak in their native tongue and not the official Putonghua. I was told that most Chinese can understand most of what other Chinese are saying in most Chinese dialects. And it was for this reason that I persisted in encouraging regional voices.
To give Zhong Shuo an audience beyond of the installations, the sound of the Beijing installation was streamed to the internet in MP3 format. This internet radio channel was accessible from www.reverberant.com, www.longmarchspace.com and the www.shoutcast.com listings for the course of the Long March Space exhibition. Internet streaming will recommence during the Brisbane show and in future I plan to arrange a more permanent webcast with synchronised text translations. The entire database of stories and translations is available in browseable lists at www.reverberant.com/cw.
To help the reader understand how the 2005 installations of Zhong Shuo operated on a more detailed level, I include here a brief technical description. This system was partially developed by me in collaboration with hardware designer Jim Sosnin, prior to my departure to China. In the kiosks at each installation, the modified telephone enabled an on-site computer to record speech, play voice prompts on the receiver and control the ringer on the phone to attract attention. In addition to sending and receiving audio to and from the phone, the computer played audio over the loudspeaker outside the kiosk. This sound included stories and soundscapes, as well as prerecorded announcements such as those to acknowledge the receipt of a story.
When the receiver was lifted, the computer played a pre-recorded message on the earpiece in Chinese prompting the participant to speak after a beep. The computer began recording the story on the hard disk and once the receiver was replaced, the recording was stopped and information about the recording, including the date, time, duration and location (eg. Beijing) was written in a database. All recordings were compressed to facilitate fast transfer over the internet. All recordings longer than seven seconds were acknowledged by a voice message played over the garden speaker. Shorter messages were rejected and messages longer than ten minutes were truncated and the speakers warned by a voice prompt on the phone to finish the story.
Playback of stories on the loudspeaker was continuous, and generated algorithmically by the computer. Recorded story files were selected in a semirandom way and edited in an interwoven fashion. The interweaving of voices¡ªor cross-fading between different voices¡ªallowed up to three stories to be heard concurrently. Once a given selected story finished, a new story would be introduced replenishing the cycle. As mentioned above, site-specific ambient sound recordings were mixed with these stories and used to introduce each new speaker.
The computers in Beijing and Chongqing were relatively autonomous, and would automatically contact each other to retrieve new stories. Theoretically, in this way, any number of computers and installations could be deployed and share materials. The Beijing computer was solely used to stream sound to the internet.
As I do not speak Chinese, my observations up to this point on the quality of the stories have been based on the performance style of each speaker, the statistics retrievable through the database and on listening to recordings in English. In just over two months, the Beijing and Chongqing installations of Zhong Shuo collected over ten hours of useful material made up of 994 recordings. The recordings were on average 40 seconds long in Beijing and 35 seconds long in Chongqing.
After my residency in China, I created an online version of the Zhong Shuo database. This consists of a public web interface that facilitates browsing of individual sound files. In addition, I made a private translator's interface, which allows translators online to listen to each sound file and make annotations; transcriptions in Chinese and translations in English that then become accessible through the public interface.
At the time of writing, my translator in Chongqing, Liu Wen Bo, has just begun the task of translating the database in preparation for the Brisbane exhibition of Zhong Shuo. The results thus far are fascinating. The following text is reproduced in full and is a translation of a recording made in Beijing at 7:20PM on 23 August 2005:
I have just heard a woman say this phone booth is too big! She said that life nowadays is getting more and more difficult and people are getting poorer everyday. But my experience is the opposite. In my childhood my family was extremely poor. I couldn't afford to buy even an apple. My grandmother and I went to the streets to pick up cigarettes that other people threw on the ground. Why? Because she could not afford to buy cigarettes! My father's salary was only 100 RMB per month. He would bring home big bones to feed the family... Life was hard then... I used to be a soldier, used to be a merchant...
I think now there is so much money lying around here in Beijing that it's up to the individual to decide to pick it up or not. My character is such that I always want to win. I ponder why other people are so rich and ask: Why not me? Today the situation in Beijing is so good, but why are some people starving to death and others driving BMWs? I think we should never depend on God, Government or anything. We should depend solely on ourselves. To be valuable to society one must seize opportunities, and especially seize some of the ¡°dark¡± opportunities; then you are able to change your destiny.
If you go through the same routine as everybody else, such as going to university; after you graduate you are jobless in China! People work in factories for their entire life and in the end wind up with nothing! Therefore, I say that these people that are poor today are poor in their heads. They lack thoughts and imagination, not wallets! What is this thing called poverty? When you feel something is too expensive, you are poor! How to turn yourself in to a rich man? We should all learn from Li Jia Cheng.
I am over 50 years old. I will sing a song for all of you. The song is my favourite song from my childhood. The name is ¡°Listen to My Mother Tell a Story from Days Gone By¡±:
The moon is wandering in the flower-like clouds In the night the wind blows with happy songs We are sitting on a hill in the corn fields Listen to my mother tell a story from days gone by We are sitting on a hill in the corn fields Listen to my mother tell a story from days gone by Long ago my mother had no land of her own All the family depended on her bare hands She worked in the hot summer fields but she ate wild weeds and wasted food The storms of winter blew sounds like a wolf My mother wore a mere shabby cloth
Beijing nowadays is quite different from what it used to be 50 years ago. I think even though today's sky is no longer as blue as it once was and the water is no longer as clean, Beijing today is a great land for us to get rich! We are able to get richer everyday. We are able to give our parents a good life and we can travel the world. Just look at me. I am a rich man from a poor childhood. I drive fabulous cars and I have mansions in every big city on the planet. I am the owner of the 4 star hotel in Beijing named ¡°A Sa.¡±
Finally, to summarise I want to say: that where there is a will there is a way. I thank God for providing such good opportunities to earn money. I want to encourage those people who have lost faith to carry on their lives, especially those who have contemplated suicide. If you are not afraid of death, then why you are afraid of being alive?...That's it, Thank you all!
This recording always stood out to me as a passionate sounding account of Beijing and on reading the translation, it occurs to me the man could earn a lucrative second income on the motivational speaker circuit and perhaps he does. To me, the speaker epitomises the spirit of Beijing and the speech is emblematic of unshackled development in the new China. To a western ear some of his statements might appear mercenary, yet his personal account of poverty supplemented by the song (a song from the Revolution by all accounts), is a moving one. The tenor of his story strikes me as proud rather than boastful, and I am left in no doubt the man believes he is doing his best to benefit society and family. It is interesting to note that the now Chengdu-based translator Liu Wen Bo gave this entry the title ¡°Hope,¡± whereas a western translator may well have applied a more cynical tag. As the speaker suggests, a good number of stories protest about the cost of living in Beijing, one man even complaining about the cost of prostitutes. Some young speakers spoke optimistically about their future in Beijing but opinions in general tend to be critical. Many speakers lamented the loss of heritage, stress, traffic congestion and degradation of the environment:
I have so many stories to tell. I lived near the Yi He Yuan, and there was a place called Liu Lang village. And beside the village, there was a structure named Huo Qing. Huo Qing was the place where eunuchs lived. That was a beautiful place. I watched the place change, there used to be so many lakes. In the 50s, those lakes and bulrushes were destroyed. Once, we used to catch fish there. We would often catch lots of fish and my mother cooked the fish for us. In the winter, we would put branches into the water and catch so many loaches! It was my dream to live there. Now they have blocked the river and rebuilt it. The past is gone... It only remains in my memories. Back then Beijing was really amazing with fresh air. We could catch boats on the rivers and Yi He Yuan was connected with Yuan Ming Yuan. The water was so clean with plenty of fish. Due to the chemical pesticides, none of those good things have lasted. You have to go to Neimeng Province to see such beautiful scenery. What a shame! There was a temple there. In winter as children we could buy very cheap candy from the temple street and we could buy wonderful toys. Now, the feeling is long gone. This is the change that I see.
Isolation and loneliness is another recurrent theme and many newcomers to Beijing expressed disappointment about unreliable people. One woman told a story of a dog:
A little dog once had a lot of friends in his old neighbourhood. But one day, a group of merchants came and bought the land. Houses were torn down and the dog became lonely. He and his owner were moved into a skyscraper. He had never felt so lonely. All around him were concrete walls. The days when he and his friends went to the streets to find something to eat were gone. Day by day, the little dog became older. Eventually the dog died.
No story dealt with commercialism in Beijing as well or as humorously as this one:
Yesterday we went to a festival. A monk asked us if we would spend some money to get blessed by the Bodhisattva. We asked how much money would we have to spend. He first asked, ¡°Do you have 300 RMB?¡± We answered no. He continued, ¡°Do you have VISA card?¡± We said no. And he replied, ¡°You will never get blessed from the Bodhisattva since you are not willing to spend any money!¡± We stared at the monk. After that we decided that we will never believe in Buddhism. We feel disappointed! I would like to announce that we lost faith in Buddhism in Beijing!
Stories collected from Chongqing are in stark contrast to those collected in Beijing. Most deal directly with the topic of Chongqing's development and this is perhaps due to the contextualising effects of the site; the Chongqing Planning Exhibition Gallery. By far the majority of stories are optimistic, positive and times appear even jingoistic. Liu Wen Bo attributes the contrast between the two cities as being due to: higher expectations on the part of the Beijing populace due to the city¡¯s position as the seat of power in China; the recent primitive conditions of Chongqing; greater competition in Beijing leading to stress and disillusionment, particularly in new immigrants to the city; and significant loss of heritage buildings and environmental damage in Beijing.
The following story is typical of those collected in Chongqing:
Chongqing is open to all people and our lives here are wonderful. In the past we were tired of climbing steps all the time, now we can take buses and ships to cross the whole city! I am in love with the view of Chongqing's skyline at night! Chongqing is like a little Hong Kong! Views like Nanbing Road, Chaotianmen, Jiefangbei... beautiful! Chongqing has a bright future. As Chongqing people, we will keep up the good work after seeing the big changes and development (at this exhibition). The mountains and water here are full of the emotions of Chongqing people! We feel great, peaceful and happy! This little bamboo house reminds us of a more innocent time.
Like this story, many others express a deep affection for the city and surrounds. People often expressed satisfaction and gratitude for simple pleasures:
Today my daughter gave me a ticket to this exhibition! Hello, I am 85 years old and I feel satisfied. Our government takes good care of the elderly. I have entered the University for aged people and I can draw pictures. I feel excited to have seen all those new things in the exhibition hall. Ok, that's all.
This particular story is a good illustration of changes in family values:
My story is about my grandfather and me. My grandfather was a very traditional man, his thoughts so outdated. He would shave my head when I was a little girl, and he would not allow us to wear skirts... After the 70s our lives became better and better. We had our first black & white TV and due to this TV, my grandfather¡¯s mind was opened. In the 80s, we went to middle schools, and in the 90s we went to university, and then we came back home. My grandfather was a changed man. We arrived back with red and green hair and my grandfather accepted this! He even said that he liked the colours. And all people have accepted a new way of life. I hope Chongqing can have better traffic systems and be more environmentally friendly. I hope my child will live in a better Chongqing. Thanks.
As stated, around six percent of Beijing stories were spoken in English and these on the whole reflect the concerns of the expatriate worker or the tourist. One lengthy and fascinating account comes from a North American man of Chinese descent. Based in Beijing, the man first offers a detailed description of his appearance and then his reception by the local mainland Chinese. He delivers an almost tragic (although at times tragi-comic) narrative on identity and dashed expectations of acceptance. While the man does not deal directly with the topic of change in China, his experience is illustrative of China's conservatism in the face of change and foreign influence.
By the time this chapter is published I hope most of the remaining stories at www.reverberant.com will be translated.
The final phases
In recent weeks I have been preparing for an Australian exhibition of Zhong Shuo and this will be part of Multimedia Art Asia Pacific (MAAP) 2006 in Brisbane in late November. The Brisbane exhibition will consist of a replica of the Chongqing bamboo hut and is an opportunity to present collected stories to an Australian audience. Ostensibly a documentary exhibition, the installation will maintain partial functionality of the telephone by delivering instructive messages on pick-up of the receiver, however it will not accept new stories. The installation will involve continuous playback of interwoven voices, and mounted on a wall inside the hut will be a plasma screen. This will display text harvested from translations in the online database and will synchronise the display with playback of the corresponding story. Additional screens will be included at the installation to play a DVD of the project and to facilitate interactive browsing of audio files and their translations.
MAAP are also sponsoring a further China-based installation in Shanghai at the Zendai Museum of Modern Art in December 2006. This will involve an exhibition of the hut from Chongqing and will collect new stories from the perspective of Shanghai residents and visitors.
As an artwork Zhong Shuo was conceived as a modular yet deliberately incomplete system and requires the creative engagement of the public to fulfil its potential. Zhong Shuo solicits and presents stories pertaining to specific sites and moments in time. Directed at first towards the Chinese public and made in collaboration with Chinese artists, the project is now seeking exposure to broader English-speaking audiences internationally by way of translated materials presented at installations and over the internet.
Zhong Shuo was made possible by an Australia China Council Arts Fellowship which was brokered by Asialink. The project has received additional sponsorship from the Australian Government through the Australia Council, its arts funding and advisory body. Other generous support has been received from MAAP and the project received initial support through the Australian Network for Art and Technology (ANAT) Conference and Workshop Fund. I would also like to sincerely thank my Chinese collaborators and staff from the Long March Foundation and the Dashanzi International Art Festival for their support and friendship.
Huang, R. ed. Beijing 798: Reflections on Art, Architecture and Society in China. Hong Kong: Timezone 8 Ltd, 2004.
Lu, J. and Z. Qiu, Z. ¡°Curators¡¯ Words.¡± Long March - A Walking Visual Display. New York: Long March Foundation, 2003.
Mott, I. Close, www.reverberant.com/CL. 2001.
°Chinese Whispers.¡± In Proceedings of Qi and Complexity: Consciousness Reframed 2004, 270-276. Beijing, 2004.
Zhong Shuo (People say) is a sound installation made collaboratively by Australian sound artist Iain Mott, Beijing visual artist Ding Jie and the Chongqing art collective the Li Chuan Group comprising Li Chuan, Ren Qian and Li Yong. The work acts as a system for the collection and telling of stories and focuses on the rapid force of change in China. In 2005 two installations were created. The first was at the Long March Space in the Dashanzi district of Beijing and was launched on 20 August. The second was launched on 10 October at the Chongqing Planning Exhibition Gallery at Chaotianmen Square, Chongqing. Zhong Shuo received 3rd prize in the 2005 UNESCO Digital Arts Awards.
Zhong Shuo has been made possible by Iain Mott's Australia China Council Arts Fellowship which was brokered by Asialink. The project has received additional sponsorship by the Australian Government through the Australia Council, its arts funding and advisory body. Other generous support has been received from MAAP, Multimedia Art Asia Pacific and the project received initial support through ANAT's Conference and Workshop Fund. For their kind assistance, many thanks to the following: Penelope Aitken, An Rui, Heidi Angove, Berenice Angremy, 798 Audio, Ros Bandt, Thomas Berghuis, Chen Ni, Chen Shao Dong, Dong Xun, Han Nan, Tim Kreger, Lu Jie, Kim Machan, Po Shui, Padmini Sebastian, Suresh Sebastian, Song Yi, Jim Sosnin, David Tung, Wang Dong and others.
Summoned Voices acts as a living memory of people and place. It consists of a series of door installations each with an intercom, sound system and a computer that is networked to a central file and database server. The design metaphor of the door presents a familiar scenario, that of announcing oneself at a doorway and waiting for a response from persons unknown. Signage instructs the public to speak, make sounds or sing into the intercom. Their voice is stored and interpreted, and results in local playback composed of the individual's voice with those that have gone before. Summoned Voices acts as an interpreter of sound, a message board and an imprint of a community - a place for expression, reflection and surprise.
Summoned Voices was premiered at the Art In Output Festival in Eindhoven, Netherlands in February 2003 and is a collaborative project by Iain Mott and Marc Raszewski. It was initiated during Iain Mott's artist residency at the CSIRO Mathematical and Information Sciences in 1999/2000 working with the Digital Media Information Systems (DMIS) research group in Sydney and Canberra. The project was assisted by the New Media Arts Fund of the Australia Council, the Federal Government's arts funding and advisory body. The Studium Generale of the Technische Universiteit Eindhoven assisted the final realisation at Art In Output.
Squeezebox incorporates spatial sound, computer graphics and kinetic sculpture. Participants manipulate the sculpture to produce real-time changes to the spatial location and timbre of the sound, as well as to manipulate digitised images. The sound and images are presented as an integrated plastic object, a form which is squeezed and moulded by participants. The artwork consists of a frame supporting four sculpted pistons on pneumatic shafts. An interactive image is displayed on a monitor beneath a one-way mirror at the centre of the sculpture. Four loudspeakers are situated at the outer four corners.The cast hands of Squeezebox invite participation. Participants grasp and press down the sculpted pieces, working against a pneumatic back-pressure to elicit both sound and image. The interaction reveals a form which has visual, aural as well as physical properties. As participants press down on the hands a sound mass is shifted from one point of the sculpture to another by pressing down on alternate pistons. Music is produced algorithmically and is derived from a set of rules which respond to the spatial location of the sound mass. The system of rules however is never static. One spatial strategy gives way to another resulting in an evolution of sound, requiring a constant readjustment of focus in the listener.
Squeezebox is collabroration between Iain Mott, Marc Raszewski and artist Tim Barrass who designed the interactive graphics. It was first exhibited in "Earwitness", Experimenta '94, ether ohnetitel, Melbourne, 1994. The project was produced with the assistance of The Australia Council, the Federal Government's arts funding and advisory body.
The composition Pope's Eye by Iain Mott was an outcome of a two week artist residency undertaken by Ros Bandt and Iain Mott in 2004 at the Melbourne Aquarium. Bandt and Mott made hydrophone recordings at the aquarium as well as recordings at Pope's Eye in Port Phillip Bay, Victoria. The aquarium recordings were subject to high levels of noise from the filtration equipment and noise reduction software was used to isolate the marine sounds. The sounds that can be heard in this composition include feeding sounds of marine life (fish and crustaceans), the sounds of fish calls, the sounds of staff divers at the aquarium, a motor boat on the bay and gannets at Pope's Eye. Other than the noise reduction, very little audio processing was applied to the recorded sound. The sounds were simply edited into a narrative form.
The Great Call is an early composition by Iain Mott made at La Trobe University. It was originally composed for the University of Melbourne Guild Dance Theatre's production of Signals in 1989. It was later performed as part of the Astra concert program for 1990-10-05 at Elm St Hall, North Melbourne with the Bell & Whistle Company. The composition is based on recordings of the homonymous vocalisations of the white cheeked gibbon (Hylobates concolor) made at the Melbourne Zoological Gardens in March 1998 on analogue tape. In the studio a pitch to MIDI tracking device was used to control an Oberheim Xpander synthesiser and a sampler. Other synthesised sounds, percussion and vocalisations were improvised and recorded by the composer on multitrack tape.
Iain Mott Conservatorium of Music University of Tasmania
Jim Sosnin School of Arts and Media La Trobe University
This paper proposes an argument for the role of sound installation in addressing the physical relationship between music and the general public. The focus of the discussion is on an outdoor interactive music event titled Sound Mapping, which explores the issues raised. Sound Mapping is a site specific algorithmic composition to be staged in the Sullivan's Cove district of Hobart. Sound Mapping uses four mobile sound-sources each carried by a member of the public. These sound sources are played with respect to geographical location and participant interactions using a system of satellite and motion sensing equipment in combination with sound generating equipment and computer control. The project aims to assert a sense of place, physicality and engagement to reaffirm the relationship between art and the everyday activities of life.
1 Music and Physicality
Digital technology, for all its virtues as a precise tool for analysis, articulation of data, communication and control, is propelling society towards a detachment from physicality. In music, the progressive detachment of the populace from the act of making music has a long history with its roots in earlier technologies and cultural practice. Perhaps the most critical early development is that of the musical score which was born of a European paradigm that emphasised literary culture over oral culture. Not only does the score require the production of music by trained or professional musicians, the relationship between musical idea and sound is further abstracted with the consequential emergence of the composer, an individual distinct from performers.
The introduction of the phonograph and radio in the early Twentieth Century broke the physical relationship between performer and listener entirely. Talking films along with television have to a degree alleviated this crisis, however the gained relationship is purely visual and not physical; worse still is that performing musicians working in all such media, like screen actors (Benjamin, 1979), are denied direct interaction with their audience (and vice versa). The representation is dislocated from its origins in time and space (Concannon, 1990).
Advocates of telecommunication technology boast of the medium's ability to connect individuals with events they would otherwise not see. Live broadcasts are presented as legitimate live experience. In experiential terms however, distinctions between live and recorded broadcast are irrelevant without interaction. Even where interaction exists (eg the internet) the audience remains dislocated from the performance and experiences reduced channels of communication than they would customarily experience through a physical engagement with the event.
Issues relating to public interaction have been addressed with some success by digital technologies. Virtual reality attempts to remedy shortcomings of interaction with multi-sensory, multi-directional engagement. However, it is often performed in denial of the body, providing a substitute "virtual flesh" with which to engage with an imagined universe. It is arguable whether a truly disembodied experience is attainable, but it is nonetheless an ideal of many VR proponents (Hayles, 1996). Such disembodiment is tantamount to the postmodern vision with its obsessions of reproduction over production, the irrelevance of time and place and the interchangeability of man and machine (Boyer, 1996).
VR offers composers an extended language beyond traditional musical dimensions of pitch and texture. Issues of navigation, the relationship between sound and physical form and interaction represent a new and significant input to the musical experience. The question remains: what constitutes real experience? In an era where people are increasingly living an existence mediated by communications networks, distinctions between the real and the virtual become blurred (Murphie, 1990). While artists must engage with the contemporary state of society, they must also be aware of the aesthetic implications of pursuing digital technologies and should consider exploring avenues that connect individuals to the constructs and responsibilities of physical existence. In the words of Katherine Hayles: "If we can live in computers, why worry about air pollution or protein-based viruses?" (1996).
Problems associated with technology and the physical engagement of the public in active music-making goes beyond the mechanics of production. Monopoly share of music production by the mass media serves to reduce society's participation. Its sheer abundance is responsible for this and further, can discourage society from listening (Westerkamp, 1990).
Installation as a means to present music has the potential to address issues of time, physicality, access and engagement. In computer music, installation can strategically bridge the gap between a body of artistic research and the general public. Installation can offer tangibility and an environment in which individuals kinaesthetically engage with the work and with other individuals. Works have discrete physicality as well as a location within in the greater spatial environment. Music installation can reassert the matrix of time and space and has the capacity to anchor a potentially metaphysical musical object (pure sound) to the physical realities of life.
Sound Mapping is a site specific music event to be staged in the Sullivan's Cove district of Hobart in collaboration with the Tasmanian Museum and Art Gallery. The project is currently under development and will be presented by the Museum in January 1998. The project is a collaboration between composer Iain Mott, electronic designer Jim Sosnin and architect Marc Raszewski. The three have worked together previously on the Talking Chair (Mott, 1995), (Mott & Sosnin, 1996) and Squeezebox sound installation projects.
The project involves four mobile sound-sources each carried by a member of the public. Each source is a portable computer music module housed in a wheelable hard-cover suitcase. Groups of individuals will wheel the suitcases with a Museum attendant through a specified district of Sullivan's Cove (Figure 1) following a path of their choice. Each individual plays distinct music in response to location, movement and the actions of the other participants. In this way a non-linear algorithmic composition is constructed to map the footpaths, roadways and open spaces of the region and the interaction of participating individuals.
A schematic summary of the Sound Mapping communications system is provided in Figure 2. The group of modules consists of a single hub case and three standard cases. All the cases contain: battery power; a public address system; an odometer and two piezoelectric gyroscopes. The standard cases contain a data radio transmitter for transmission to the hub and an audio radio device to received a single distinct channel of music broadcast from the hub .
The hub case contains a Differential Global Positioning System (DGPS) receiver that generates spatial coordinates for the positioning of the group to an accuracy of 5 m. GPS is a worldwide radio-navigation system formed from a constellation of 24 satellites and their ground stations. GPS uses these satellites as reference points to calculate positions of GPS receivers anywhere on the earth. DGPS offers greater accuracy than standard GPS and requires an additional radio receiver that receives an error correcting broadcast from a local base station (Trimble, 1997). The Australian Surveying and Land Information Group (AUSLIG) offer subscriptions to such signals broadcast on the JJJ FM carrier signal.
The hub produces music on a four channel sound module managed by a lap-top computer in response to its location as well as motion data from itself and the three standard cases. Each channel of music is produced for a specific case. The music is broadcast to its target case by radio transmission. Motion data is generated by an odometer and gyroscopes in each case and is broadcast to the hub from the standard cases by data radio transmitters. Odometers measure wheel rotation in both directions and two gyroscopes measure tilt and azimuth.
The technical distinction between the hub and standard cases is designed to be transparent. Participants will be unaware of the leading role of the hub .
The channels and dynamics of communication are among the defining characteristics of interactive artworks (Bell, 1991). Sound Mapping is an attempt to explore modes of communication beyond those used in the Talking Chair and Squeezebox projects.
From the perspective of participants, communication will travel along four multi-dimensional pathways: a) that between participants and their respective suitcase b) between onlookers and participants c) between the environment and the group d) between the participants themselves (Figure 3).
In the first instance there exists a relationship between the kinaesthetic gestures of participants and resultant music. This communication occurs in mutual feedback.
The interaction between onlookers and participants is anticipated to be intense due to the very public nature of the space. The interaction will be musical, visual, and verbal as well as social in confronting participants with taboos relating to exhibitionism. This situation is likely to deter many people from participating but nonetheless it is hoped the element of performance will contribute to the power of the experience for both participants and onlookers.
The effect of the environment on the work marks a development from our previous installations in its capacity to dynamically signify musical elements. Music will react to the architecture and urban planning of Sullivan's Cove by means of GPS which will be used to correlate musical algorithms to specific urban structures.
Communication between participants will be verbal, visual, musical and like the relationship between the group and onlookers will feature social dimensions such as the interplay of personalities. Musical communication between individuals will be complex due to the interrelation of music algorithms specific to each case. Algorithms will use pooled musical resources (Polansky, 94) enabling participants to share structures and contribute information to elements such as rhythmic structures and timbre and pitch sets. Verbal and visual communication between participants in Sound Mapping is critical. Participants will need to establish the musical relationship that exists regarding each other and communicate to coordinate group music-making. Group decisions will need to be agreed on to seek out areas of interest within the space in relation to urban structures. It is likely participants will discuss the relationship of sound to both gesture and the environment.
4 Exploration of Music
The suitcases in Sound Mapping , as well as providing a practical solution to the transportation of music modules, serve as a metaphor for travel and exploration. This is particularly relevant to the way the composition is structured but also to the function of Sullivan's Cove. The Cove was once the major port of Hobart however the majority of heavy shipping has since left the district. The region now has dual purpose as a port and leisure district and is characterised by a major influx of tourists during the summer months (Solomon, 1976).
Composition of music for this project at present is in the planning stage however a basic structure has been devised. Navigation of the musical composition will occur on two levels: a location dependant global level and a gesture and pseudo-location dependant local level.
The global structure is determined by the DGPS receiver. As the group moves through the mapping zone the lap-top computer will use information from the DGPS to retrieve local algorithms written specifically for corresponding locations. One algorithm will be created for each of the four cases for every specified region within the zone. The borders of regions defined by the global structure will interconnect in a continuous fashion. Transitions between algorithms will occur at the global level. The rate and nature of the transitions will be decided upon regarding the aesthetics of adjoining regions. Sites contrasting sharply with adjoining regions will use rapid transitions to signify the demarcation. Adjoining regions displaying similarity will call for slow and fluid transitions.
DGPS has limited data resolution for the production of music at the local level. High resolution information is therefore needed if the system is to be responsive to the subtle gestures of participants. Local algorithms produce music directly and receive their prime input from motion sensing equipment along with transition data supplied by the global level. Motion sensors will quantify gesture but also to provide additional location information to modulate musical and gestural parameters. Gestural input will include tilt, azimuth and forward and backward velocity.
Automotive navigation systems use a combination of GPS, gyroscopes and odometers called dead reckoning to address problems of poor resolution and transmission failure (Andrew Corporation, 1996). It is possible such a system could be used to provide detailed location information for parameter mapping as well as a gestural interface supplemented by a tilt gyroscope. Dead reckoning will not be implemented however because it is difficult to guarantee standard starting conditions for all four cases and maintain accuracy over prolonged periods without greater cost in hardware development.
Instead a pseudo-location system has been contrived to track the movement of all four cases for each local region specified by the hub . While the hub remains within a given local region, the functional size of the region is determined by the actual distance standard cases stray from the hub . How can we map musical parameters to local movement if the functional size of the region is unknown? As a solution, two-dimensional motion data from the odometer and the azimuth gyroscope will be mapped to the surface of a spherical data structure with a surface area smaller then that of the local region concerned. Because this data is mapped to the continuous surface of a sphere, issues relating to boundaries become irrelevant.
5 Fact and Fiction
Sound Mapping is a mobile sound installation . This apparent oxymoron results from the fact that although physically independent of location, the work is dependant on location specific information. Sound Mapping constitutes a form of what Mark Weiser of Xerox terms embodied virtuality , a state where the body is both physical entity and a pattern of information (Hayles, 1996). In Sound Mapping , participants occupy a space that is at once both virtual sound-scape and physical environment. Their movements, as interpreted by the technology, are transmuted into digital information that interacts with their immediate location and gestures of their fellow participants.
In this way a convergence is created between the familiar home (or holiday destination) of the participant and a sonic fantasy mapping known territory. It is hoped this interaction between fact and fiction will prove resonant for participants who will interpret their home, perhaps for the first time, through a vision of sound.
The music of Sound Mapping will be constructed using concrete and synthesised sounds under the continuous control of participants and the environment. It will reflect the locations where it is produced and not necessarily represent those places. It does not function as a historical or cultural tour of a city per se but operates in reverse using the city as an energising surface. Music will on occasions strive to represent locations. It will however also be produced to contrast and challenge peoples perceived notions of place, time and motion.
Where historical references are made in the music, they will relate to history widely known by the general public. Reference will only be made if historical usage and events have a significant bearing on contemporary function.
6 Music and Architecture
Participant exploratory works employing diffuse sound fields in architectural space have been explored by sound artists such as Michael Brewster (1994) and Christina Kubisch in her "sound architectures" installations (1990). Recently composers such as Gerhard Eckel have embarked on projects employing virtual architecture as means to guide participants through compositions that are defined by the vocabulary of the virtual space (1996).
The interaction between music and architecture represents a fascinating area for new research. In contrast to architecture, music traditionally structures events in time rather than space to form what is a linear narrative. Architecture spatially arranges and compartmentalises functional components of society into integrated efficient structures. In doing so it employs visual structures that signify multiple meanings including functional purpose and social ideals.
Sound Mapping investigates a translation of both the organisational and the symbolic qualities of architecture. The relationship will be commensal rather than parasitic on architecture and will draw on the work of Young et al. (1993) which details terms of reference common to both disciplines in the areas of inspiration, influence and style. As such, music and architecture will share a complementary language of construction, elucidating functions common to both.
Signification of discrete elements of architecture is integral in the organisation of urban space. Just as people have an intuitive understanding urban design, the environment will act to aid participant's navigation and understanding of the associated musical landscape. Urban structures will suggest, among other things: musical linkages, demarcations, continuations and points of interest.
7 Hardware Implementation
The hardware for Sound Mapping comprises four separate but interdependent units and requires a distributed processing approach to design. Within the hub case, the lap-top computer that controls the four synthesiser voices must process the data from the DGPS system, the hub case gyroscope and odometer readings, plus three sets of gyroscope and odometer readings sent via RF (Radio Frequency) link from the standard cases; considerable preprocessing of these data is necessary to merge them into a consistent format suitable for the lap-top's serial input port. In addition, preprocessing of gyroscope and odometer readings within each standard case is required to merge them into a consistent format suitable for the RF link.
Th PIC 16C84/04 microcontroller, manufactured by the Microchip company, has been chosen for all these preprocessing tasks in the four cases. Whilst this microcontroller does not include any dedicated serial hardware or analog conversion capabilities on the chip itself, as some other microcontrollers do, it has the overwhelming advantage of having an easily reprogrammable EEPROM program memory. This reprogrammability, without the need for ultraviolet erasure required for reprogramming conventional EPROM, allows an efficient software development cycle in this project; the alternative, of using a software simulator, is expected to be much less efficient, given the distributed, real time problems to be encountered.
Within each case, the gyroscope, which is a piezoelectric, rather than a rotational type, provides an analog voltage output. This is converted to digital form by the PIC with an external ADC chip. The odometer comprises two hall-effect sensors that respond to magnets implanted into the case wheel; the hall-effect sensor outputs are amplified and read by the PIC directly, as an essentially digital signal, and processed to extract direction as well as displacement. The final task for the PIC is to send all this data as a serial data block; no extra hardware, such as a UART, is required for this, as the PIC is fast enough to simulate this process in software.
Within the hub case, a fifth PIC is dedicated to the task of merging the DGPS, local gyroscope and odometer data with the radio data from the standard cases; this also forms a serial data block, and is sent to the lap-top.
Serial data rates
Standard MIDI (31.25 KBits per sec) is used for the final, merged data connection feeding the lap-top. This allows a maximum transfer rate of roughly 3 KByte per sec, although the average rate is somewhat lower. Initially, for consistency, it had been planned to use MIDI for the intermediate serial data block transfers also, within each standard case, but this would have required a greater bandwidth from the RF links; this is rightly discouraged by the Australian SMA (Spectrum Management Agency), especially in this low power application, where predetermined short-range frequency allocations can be used, and a specific license application is not necessary.
The DGPS system sends its data, in bursts, at 9.6 KBits per sec, so this rate is now used within the standard cases also, and allows readily available modules to be utilised for the transmitters and receivers in the RF links.
Sound Mapping is a participatory work of sound art made for outdoor environments. The work is installed in the environment by means of a Global Positioning System (GPS), which tracks movement of individuals through the space. Participants wheel four movement-sensitive, sound producing suitcases to realise a composition that spans space as well as time. The suitcases play music in response to nearby architectural features and the movements of individuals. Sound Mapping aims to assert a sense of place, physicality and engagement to reaffirm the relationship between art and the everyday.
Sound Mapping is a collaborative project by Iain Mott, Marc Raszewski and Jim Sosnin. The premier exhibition was staged in Sullivan's Cove, Hobart by the Tasmanian Museum and Art Gallery (TMAG) on 29 January - 15 February, 1998. Sound Mapping was awarded an Honorary Mention in the Interactive Art category of Prix Ars Electronica in Linz, Austria. The project was exhibited as part of the Ars Electronica festival in Linz, Austria in September 1998. It is pictured on this page at the 2004 International Conference on Auditory Display (ICAD) at the Sydney Opera House.
This project is assisted by the New Media Arts Fund of the Australia Council, the Federal Government's arts funding and advisory body. Additional generous support from: Arts Tasmania, Vere Brown leather goods and luggage, Fader Marine, Salamanca Arts Centre, Hobart City Council and the Hobart Summer Festival.
Close is a multi-screen video projection installation by Iain Mott that blurs the separation between viewer and subject by means of 3-dimensional sound. The viewer wears headphones and hears sound from the perspective of the subject - an effect achieved through binaural recording methods. Close portrays a haircut as a kind of death. The hairdresser, in a gradual erasure of the subjects features, removes hair and eyebrows with scissors and razors. The viewer inhabits the body of the subject as if watching his or her own disappearance. The shared ritualistic experience explores territories of death and loss.
Production Credits Justin Brickle - director of photography Dominic Bromilow - gaffer/grip Stephen Dixon - editor/sound recordist Christiane Hanesch - art director Iain Mott - writer/director/sound editor
Produced with the assistance of Arts Victoria and the CSIRO Mathematical & Information Sciences. This project has been assisted by the Commonwealth Government through the Australia Council, its arts funding and advisory body. Videotaped at CSIRO Building Construction and Engineering, Melbourne. Offline edit at Open Channel - online edit at Fine Cut.
A voice means this: there is a living person, throat, chest, feelings, who sends into the air this voice, different from all other voices. A voice involves the throat, saliva, infancy, the patina of experienced life, the mind’s intentions, the pleasure of giving a personal form to sound waves. — Italo Calvino
Acoustic theatre for headphones of Italo Calvino's story A King Listens (Um Rei à Escuta in Portuguese) by the Laboratório de Performance e Teatro do Vazio (LPTV), research and extension group of the Instituto de Artes (IdA) of the Universidade de Brasília (UnB), linked to the Decanato de Extensão (DEX) and to the Departamento de Artes Cênicas, UnB.
Déborah Soares Felipe Fernandes Rogério Luiz
Direction, audio production and sound design:
Binaural processing and mixing performed with the Soundscape Renderer (SSR), Puredata, Ardour 3, Jconvolver and the Jack Audio Connection Kit.
CALVINO, Italo. Um Rei à Escuta. In: Nilson Moulin (Trad.); Sob o Sol-Jaguar [Sotto il sole giaguaro, 1986]. p.57–89. São Paulo: Companhia das Letras, 1995.
Spatial-acoustic research on the cerrado from the perspective of the performing arts
Professor responsible: Dr. Iain David Mott Departamento de Artes Cênicas (CEN) Universidade de Brasília (UnB)
Project approved by the MCTI/CNPq in the Edital Universal Nº 14/2013.
To study the acoustics and soundscape of the landforms and vegetation types of the national park Chapada dos Veadeiros and produce virtual acoustic environments from the investigations with applications in the performing arts. The project will emphasise the human voice in particular and its integration with the soundscape of the cerrado. Raw data from the field research will be used in the creation of new artworks and made available on the Internet to encourage dialogue between disciplines and new research. More generally, the project intends to feature the cerrado from an audio perspective. The project will welcome the participation of the public from the regional town of Alto Paraíso and will form part of the activities of research and extension of the collective Audiocena, part of the Laboratório de Performance e Teatro do Vazio (LPTV), of UnB. Cerrado Ambisônico will produce works of theatrical installation and radiophonic art, contributing to the education of undergraduate students of Artes Cênicas at the Universidade de Brasília from a interdisciplinary perspective.
The approach of the Canadian derived movement of acoustic ecology will be the conceptual base of the project, examining the acoustic elements of a designated place as a dynamic system of information exchanges. Impulse response measurements will be made to facilitate the simulation of the acoustics of the environment by way of an associated process of convolution. Moreover, the techniques of ambisonic recording and binaural recording will enable the reproduction of soundscapes and their acoustics on headphones or multichannel systems of loudspeakers.
* Ambisonic Cerrado. Cerrado is the general term for the savana-like vegetation of central Brazil. The cerrado is extensive, occupying an estimated 2 million km² of land and has great biodiversity.
O Espelho (The Mirror) is a theatrical sound installation inspired in part by the story of the same name by 19th century Brazilian writer Machado de Assis. It is by sound artist Iain Mott and performer Simone Reis and includes objects and paintings by visual artist Nelson Maravalhas and video by Alexandre Rangel. O Espelho investigates the games of deception we play with our identities using a variety of contemporary audio technologies, video and visual illusions from the 19th century, together with pre-recorded performance and electroacoustic composition. The installation is devised for a solitary viewer in keeping with the isolation experienced by the protagonist in the original story. Of great importance in this work its investigation of how sound, voice and listening can function ontologically side by side with visual representations of the self.
The characters played by Simone Reis are projected as "pepper's ghosts" in a false-room adjoining the publicly accessible installation space. This 19th century theatre technique involves angled glass positioned on a stage or diorama to provide the illusion that the space is occupied by a phantom figure. In our installation, the false-room is identical to the installation-space, however all furniture as well as pictures and objects on the walls by Nelson Maravalhas are arranged in mirror-image. The gallery visitor sees the projected video images and the false-room as if they were a reflection. This is by way of a false-mirror, a simple pane of glass in a frame. An "audio spotlight" (a special loudspeaker) is used to produce what are experienced as "in-head" sounds by the participant. These sounds principally correspond to the voice of the false-reflection played by Simone. Other sounds representing the memories, thoughts and fantasies of the characters (and by implication, the visitor), surround the seated listener and are projected in the space by what is known as an "ambisonic" sound system, consisting of multiple loudspeakers.
CCBB, Brasilia, 14/7 - 16/9, 2012 - Galeria de Artes Van Gogh, Sobradinho, 21/9 - 21/10, 2012 - Teatro Newton Rossi, SESC Ceilândia, 26/10/2012 - 26/1/2013.
O Espelho is by sound artist Iain Mott and actress/performer Simone Reis in collaboration with others including: video maker Alexandre Rangel, visual artist Nelson Maravalhas, playwright Camilo Pellegrini, lighting designer and arquitect Jamile Tormann and costume designers Cyntia Carla and Simone Reis. The production is by Alaôr Rosa and Arte Viva Produções with funding from the Fundo de Apoio à Cultura (FAC) of Brasilia and the Centro Cultural Banco do Brasil (CCBB).
Installation concept: Iain Mott Artistic directors: Iain Mott and Simone Reis Script: Simone Reis, Iain Mott and Camilo Pellegrini Curators: Simone Reis and Iain Mott Performance: Simone Reis Stage direction: Simone Reis and Iain Mott Composition and sound design: Iain Mott Video: Alexandre Rangel Computer programming: Iain Mott Lighting design: Jamile Tormann Paintings and objects: Nelson Maravalhas Photography: Mila Petrillo/Rayssa Coe/W. Hermusche Direction assistant: Luara Learth Make up: Raphael Balduzzi Assessoria de Comunicacao: Rodrigo Machado, Territorio Cultural Assessoria de Comunicacao Project coordination, production and director of production: Arteviva Produções/Alaôr Rosa e Fernanda Oliveira Executive producer: Nalva Sysnandes
The following people and groups gratefully acknowledged for their contribution toward the conception or development of the project:
Antonia da Silva Reis, Carlos Lin Silva, Celso Araújo, Cláudia Gomes, Dalton Camargos, Davi Reis, Departamento de Música e Departamento de Artes Cênicas do Instituto de Artes da UnB, Fons Adriaensen, Greg Schiemer, Guilherme Reis e Espaço Cena, Guillaume Potard, Jim Sosnin, Luís Augusto Jungmann Girafa, Marc and Elizabeth Raszewski, Marília Panitz, Núcleo de Vivência da UnB, Odila Bohrer, Olive Maureen Mott, Renata Azambuja, Ros Bandt, Stephen Barrass, Zé Celso Martinez Correia
Puredata with iem_ambi, iemlib, iemgui, pduino, ggee, zexy and room_sim_2d, as well as Jack, Ardour, Ambdec, Jconvolver, Qutecsound, Rosegarden, Xjadeo and Final Cut Pro.
Linux computer, LCD TV, Lanbox/dimmers/lights, Bose FreeSpace loudspeakers, Holosonics Audio Spotlight, RME Multiface, Netgear router, Arduino and pressure sensor.
* Published in Proceedings of the 1996 International Computer Music Conference.
Iain Mott Conservatorium of Music University of Tasmania
Jim Sosnin School of Arts and Media La Trobe University
This paper describes a new interface for interactive sound spatialisation by the public over three-dimensional (3-D) loudspeaker arrays. The interface uses ultrasound to implement an adaptation of F. R. Moore's General Model for Spatial Processing of Sounds and offers clear ergonomic and computational advantages over existing strategies. It is designed for use by non-trained individuals but could be equally useful in a performance context especially where the performer is remote from the projected sound field.
The design of expressive public interactive music installations obligates an approach centred on guiding the public through specific sets of activities. An activity-based approach is necessary to elicit appropriate music generating behaviour from participants. Physical restrictions on the actions of participants are generally needed to keep behaviour within functional bounds. In addition, where adequate restraint of behaviour is unfeasible, it may also be necessary for designers to rationalise the sensitivity of sound generating mechanisms.
First-time users of interactive installations may not fully recognise the relationship between their own actions and the resulting sound. Hence, designers of public interactive works must establish potent techniques to facilitate rapid cognition in participants. It is important that participants are aware of the consequences of their actions, as it is through informed behaviour that composers can best communicate aesthetic material in an interactive context.
This paper will discuss a new method for interactive sound spatialisation by the public. The system has been devised for The Talking Chair [Mott, 1995], an interactive sound installation that enables non-trained individuals to manipulate compositional algorithms and control the 3-D motion of a virtual sound object around their body. We have developed a design with strong spatialised visual reference points to aid the first-time user in interactive sound spatialisation. While the interface is designed to control both music generating algorithms and the spatialisation of sound objects, discussion of the compositional element is beyond the scope of this paper.
2 Earlier Design
Our current work is performed as part of an ongoing investigation into interactive music. In collaborations with designers and animators we have produced works exploring spatial sound manipulation by the public. Squeezebox was made after The Talking Chair in 1994 and was produced in part to examine the use of restricted movement in public interaction. Unlike The Talking Chair which allows unhindered spatial gestures within a given region, the Squeezebox interface utilises four pneumatic pistons to insure smooth gestures from the public.
The impetus for designing a new spatial controller came from a need to readdress human factors associated with The Talking Chair. The first 1994 version of the installation (Figure: 1) implemented John Chowning's  theories of moving sound simulation and used an ultra-sound wand device as an interface. A single seated participant moved the wand through a cubic zone above the lap to control the motion of sound. Ultra-sound pulses transmitted from the wand were picked up by three fixed receivers and gestures were mapped in the form of cartesian coordinates. A mirrored display cabinet was situated in front of the listener as a visual aid to the spatial navigation of sound. The viewing cabinet, using a magic box technique [Popular Mechanics Co., 1913], displayed the illuminated wand tip as a glowing ball moving about a reflected image of the participants head. The position of the ball relative to the head corresponded to the perceived position of sound in the space surrounding the listener.
Figure 1: 1994 version of The Talking Chair
Regrettably, while the magic box system worked effectively, its use required detailed written explanation and some practice. We found that few people in museum environments fully read the detailed diagrams and as a result, the cabinet often proved confusing and distracting. The complexity of the interface also lead a few individuals to believe that the cabinet itself was the object of the interaction! The magic box display, in addition, requires a degree of control over ambient lighting that is not alway practical in many public spaces. A simplification of approach was in order.
In attempting to remedy shortcomings in our original human-machine interface our two main aims were to a) efficiently map the spatial gestures of participants to factors simulating the spatial motion of sound and b) provide an interface design that explicitly informs the participant of the spatial relationship between their body and the virtual sound object.
We chose F. R. Moore's  General Model for Spatial Processing of Sounds as the basis for our interface design. The model is readily adaptable to a user interface using ultrasound technology and has qualities which are highly instructive in the interactive positioning of sound relative to the body.
The model was originally produced to address problems of listener perspective in multi-speaker systems by generating audio delays and amplitudes with respect to the loudspeakers rather than the listener. In this way listeners in different locations within a sound field will each receive a different perspective of the same spatial event. Loudspeakers in spatial arrays are modelled as windows in a room, the size and shape of which is determined by the number and placement of the speakers. This inner room is surrounded by a larger outer room in which illusory sonic objects may move freely. The audience is contained within the inner room and perceives virtual sound objects through the windows. Spatialisation of sound is simulated by measuring the distance of the direct and reflected sound paths from the virtual object to each window. The distances are used to calculate individual amplitudes and time delays for each loudspeaker channel.
We were attracted to the General Model, not because of issues of listener perspective but rather due to its potential to produce an interface design of great ergonomic clarity. By constructing a user interface as a physical analogue of the inner room we at once provide an efficient mechanism for real-time spatial control and strong visual tool to communicate the position of sound with respect to the listener.
4 Implementation Overview
The Sound Sphere
The interface design incorporates a small sphere suspended from a stalk in front of the listener. The surface of the sound sphere is impregnated with six ultrasound receivers each in positions directly corresponding to the position of the speakers of The Talking Chair. An alignment tube is included in the interface so participants can adjust the seat height to position their head at the centre of the sound field. An ultrasound emitting wand is moved around the sound sphere to position the sound relative to the speakers of the sculpture. Participants will be informed in a simple four stage instruction chart (Figure 2) how to interact with the sculpture. Instruction covers a) the use of the alignment device b) how to change sounds and c) how to position sound objects relative to the sculpture and consequently the body of the participant. The discovery of the remaining nuances of interaction is left to the investigation of participants.
Amplitudes and Delays
The interface uses ultrasound to measure the distance from the wand to each receiver. Distance measurements are used to determine both amplitude and delay time of sound for each loudspeaker channel. Audio amplitudes are to be attenuated with a linear rather than an inverse cubic relationship [Moore, 1989] to distance, as such methods are more useful for interactive control [Ballan et. al., 1994].
With our implementation of the General Model, only the direct (not reflected) paths from the wand to each receiver are measured. While the omission of reflected paths represents a simplification of the model, a further complexity results from the distortion of the sound field caused by the sound sphere.
In the General Model, windows with direct sound paths obstructed by walls receive signal levels of zero. In cases where a window is suddenly shadowed by an obstructive wall, signal levels decline sharply and it is necessary to interpolate between amplitude values in order to avoid audible clicks [Moore, 1989]. With our interface design part of the ultrasound signal is diffracted around the surface of the sound sphere. Smooth amplitude transitions occur as windows (receivers) become gradually shadowed. Distance measurements from the wand to shadowed receivers already include the extra path lengths resulting from the sound wave following the curve of the surface, and are thus a truer representation in this context.
While it is possible for us to achieve stable amplitude measurements at each receiver, we will not however map these readings to loud speaker amplitudes. Our transmitter is not fully omnidirectional and consequently models a sound object with a directional radiation pattern. In addition to the inverse square nature of the response, such directionality would render the interaction unsuitable to non-trained usage. A mechanism of this type is however worthy of continued investigation in a performance context.
Reverberation is to be implemented using off-the-shelf effects units and the level will be determined according to the distance of the wand from the sound sphere. As in the 1994 version of The Talking Chair we will implement John Chowning's local and global reverberation techniques  which we have found to be highly efficient and effective in simulating changes in distance.
The attenuation of individual audio channels is to be performed by a custom built MIDI controlled mixer. The device which uses analog circuitry, is capable of controlling four independent sound objects in a six loudspeaker array. The device has the capacity to control the signal level of audio sent to each loudspeaker as well as controlling the level of both local and global reverberation. The mixer also contains insert points for the six individually tapped delays of one virtual object.
At time of writing, the audio delay software required for the spatialisation is being implemented on a DSP56002 processor [Motorola Inc., 1993]. Whilst the processor is an overkill for the simple delay processing required (basically, memory lookup and interpolation), it was decided to pursue this method as it is planned, eventually, to shift all of the spatial processing required for The Talking Chair and other projects onto the DSP, and eliminate most of the analog circuitry currently being used.
Ultrasound distance measurements are performed by a PIC16C84 microcontroller [Edwards, 1994] [Microchip Technology Inc., 1994]. The unit produces its output as MIDI messages which will be received by a Macintosh computer running the FORMULA music language. In addition to controlling music generating hardware, the Macintosh will control the DSP56002 and the custom mixer via MIDI.
5 Ultrasound Specification
Choice of Frequency
Ultrasound transducers, of the type readily available to us, are highly directional devices. In order for them to provide useful distance information for interactive spatialisation it is necessary to enhance their omni-directional capacity. This is to ensure that for all angles, the signal strength of a direct path to a given receiver is of a higher intensity than that of reflected signals. The 1994 ultrasound implementation used 40KHz transducers. The wand housed the transmitting transducer (Tx) and was moved through a zone monitored by three receivers. We found that by fixing a 16mm (diameter) marble in front of the Tx, sufficient reflection occurred to allow distance (ie timing) measurements to work, no matter which direction the wand was pointed in.
The original wand would be impractical in the new implementation as it is necessary for the sonar wave to be diffracted, at least partially, around a sound sphere of perhaps 80mm diameter. The new wand uses 25KHz, which has a wavelength of around 13.5mm, and although this is still much smaller than 80mm, our tests showed that there was just enough diffraction around the sound sphere for useable distance (ie timing) measurements at all but the most distant receivers. A 40KHz signal is far less useable as its shorter wavelength (8.5mm) will result in far greater reflection.
This lower operating frequency also allows the use of small (6mm diameter) electret mics as the sonar receivers. Their small size results in them having a near-omnidirectional response pattern in free air (mic diaphragm diameter is smaller than wavelength), and thus a hemispherical response when they are mounted flush on the sound sphere surface. 25KHz is somewhat above the intended operating frequency of the electret mics, which are designed for audio applications, but with sufficient amplification and appropriate bandpass filtering they are quite useable.
Modifications to 25KHz Transducers
Attempts to improve the omni-directional response of 25KHz transducers using a marble to scatter the beam have been unsuccessful. This is possibly due partly to the design differences between the two transducers as well as differences in wavelength. The 25 KHz Tx radiates directly from the piezoelectric surface via a grille, the centre 9 mm of which is blocked off, allowing sound only through the surrounding annulus. An enhanced omni-directional response has been achieved through the use of a conical device (Figure 3) attached to the front of the Tx. In our design, the inner concentric cone directs the annular cross-section wave to the tip of an outer cone, where the final release to the surrounding air is via a 4mm diameter hole. This hole is much smaller than the 13.5 mm wavelength, so the radiation is nearly omnidirectional. With the ultrasound cone attached, the radiation in any direction is uniform within 6 dB, except for a shallow null of 9 dB at 165 degrees. This compares favourably with the radiation pattern measured originally on the bare transducer, which showed a variation of 26 dB (Figure 4).
The current implementation of the hardware uses a PIC16C84 microcontroller to generate the blips and process the received signals. It incorporates a manual adjustment of blip rate (30 to 60 mSec between blips) to allow use in different reverberant spaces. It also measures all 6 received signals near-simultaneously, so that a complete set of measurements is made at each blip, rather than having to cycle through 6 blips (and wait 6 times as long before a system response to a wand movement can be perceived).
The MIDI data byte output of the PIC is currently set as 7-bits and the resolution of distance measurements is 6mm. Data filtering techniques are to be applied in the Macintosh computer to smooth and interpolate between values.
The Tx is pulsed with bursts of 8 cycles of the 25KHz signal (each burst lasts for 0.32 mSec). This signal is generated in software, and drives the Tx from two PIC data lines via a differential summing amp. The final waveform is symmetrical, swinging ±20V, with zero average DC. Lower frequency clicks from the Tx, which had earlier been a problem, are thus almost inaudible, despite the higher power used to compensate for losses in the cone structure.
The new Talking Chair user interface has significant advantages over the original magic box device. The most notable advancement is the shift from a 2D frame of reference to a true 3D method. Freedom of movement is more limited with the new interface due to partial obstruction by the attachment stalk and the need to navigate the wand in close proximity to a solid surface. We believe however that the strong ability of the device to communicate notions of spatial location, far outweighs obstruction concerns in a context of public interaction.
[Ballan et. al., 1994] Oscar Ballan, Luca Mozzoni and Davide Rocchesso. Sound Spatialisation in Real Time by First-Reflection Simulation. Proceedings of the 1994 International Computer Music Conference, pp.475-476, 1994.
[Chowning, 1971] John Chowning. The Simulation of Moving Sound Sources, Journal of the Audio Engineering Society 19(1): pp.1-6, 1971.
[Edwards, 1994] Scott Edwards. The PIC Source Book, Scott Edwards, 1994.
[Moore, 1983] F. Richard Moore. A General Model for Spatial Processing of Sounds, Computer Music Journal 7(3): pp.6-15, 1983.
[Moore, 1989] F. Richard Moore. Spatialization of Sounds over Loudspeakers. In Max V. Mathews and John R. Pierce (Eds.): Current Directions in Computer Music Research, MIT Press, Cambridge Massachusetts, pp.89-103, 1989.
[Motorola Inc., 1993] DSP56002 Digital Signal Processor User's Manual, Motorola Inc. 1993.
[Mott, 1995] Iain Mott. The Talking Chair: Notes on a Sound Sculpture, Leonardo 28(1): pp.69-70, 1995.
[Popular Mechanics Co., 1913] An Electric Illusion Box in The Boy Mechanic, Volume I, Popular Mechanics Co., Chicago, pp.130-131, 1913.