MASTERS BY PROJECT PROPOSAL / _BEN JARRETT

TITLE
Heteropoetic Mediation – The Emergence of Biokinetic Interfacing

INTRODUCTION
This project may be seen as an extension of several design themes that I have engaged with over my undergraduate studies, and particularly within my final project. Despite design theoreticians positing that intuitive interfaces should be responsive and physically dynamic (Norman 43, Kankeinen 154), domestic audio electronics have yet to engage in this emergent field. It is my contention that current digital music carriers (Optical disc players & HDD based devices) disconnect listeners from meaningful interaction from a product that is ostensibly designed to enable emotional connection. By engineering nuance, dynamic change and subtlety into the primary user experience, I believe that an evocative, sustainable relationship can evolve between the designed object and us.

QUESTION
How can the application of biokinetic interfacing [1] upon audio components augment the poetics afforded by the domestic listening experience?

AIM
The intention of this post graduate research is to create a consequential post-screen interface, specifically including visual/haptic feedback, to control and ‘sense’ domestic audio equipment. Considering the interface between listeners, audio products and music as a dialogic communication informed by cultural, historical and social contexts; the project seeks to discover which forms of positive intertextual mediation can be elicited by biokinetic interfacing using affordance[2], metaphor and product semantics[3] as core design issues. By pursuing an action centred methodology based on creating and evaluating simulated and real models, this work seeks to educe, test and refine a coherent approach to generating post-screen interfaces; expanding several threads of contemporary design theory and practice.

OBJECTIVES
To investigate the validity of bio-kinetic interfaces for use within the audio domain
To develop and articulate phenomenographic [4] insight into biokinetic interfacing
To gain comprehension into the biomimetic and kinetic elements implicit relationship to affordance, metaphor and object semiotics
To create a body of work that explores, interprets and questions my emergent understanding
To codify a set of cohesive strategic and tactical guidelines for general implementation of biokinetic interfaces
To create a final prototype specifically designed for high-end audio equipment

OUTCOMES
THEORETICAL
1/ A transferable framework of conceptual and practical design stratagems for advocating and implementing biokinetic interfaces across a wide range of product experiences
2/ A deeper epistemological understanding of human/designed object relationships
PRACTICAL
1/ Generation of a large body of 2D concepts, consisting of hand-drawn image, as well as static and moving digital simulacra
2/ Creation of a considered group of 3D prototypes that explore various aspects of biokinetic interfacing. These moving and static models will be used to test and evaluate design hypotheses
3/ A design document for the general public describing the action based research approach taken, and insights uncovered.
4/ Writing of an exegesis detailing the post-graduate research undertaken
5/ Building of a working final prototype that distils the design knowledge accrued into a cohesive and compelling vehicle for biokinetic interfacing
6/ Exhibition of the final model and associated material at the Unitec Design Exhibition 2012

CONTEXT & RATIONALE
As the 20th century materialized, industrial design became a prominent discipline allied to the business of manufacturing, shaping the physical output. Louis Sullivan’s famous 1896 dictum – Form follows Function – informed the virtually the entire industrial design landscape until the latter part of the century, despite periodical forays outside modernism as the century progressed. Indeed the Bauhaus and Ulm schools’ minimal, functionalist aesthetic defined the ideals of Modernism in regards to consumer goods, and “designer ware” is still associated with Tectonic cool tones and stark lines – witness Apple computers. Regarding consumer goods, the theory and physical reality of Post Modernism arose as a counter move against the intended lack of historical reference/precedence set as a cornerstone of the Modernist movement. By referencing contemporary and historical ideas and devices, the Post Modernists sought to imbue their designs with layers of extra meaning; an embodied narrative that augments the experience of using/consuming the design, moving beyond mere functionality. Often early Post Modern pieces now appear haphazard and contrived, ironically due to the practitioners grounding in Modernist theory, and the use of stylistic contrivance without a deeper understanding. One could say that Form followed Idea.
Towards the end of the century new design thinking began to emerge from the shadows of these opposing movements. J.J. Gibson put forth the idea of affordance in his book “The Perception of the Visual World” (28). He posited that humans perceive objects in terms of what they may be used for, rather than objects in themselves. As Klaus Krippendorff writes:
“A careful reading of Gibson’s work suggests that his affordances, the perception of possible uses, the awareness of usabilities, equal the meanings of the artefacts in use.”
In conjunction with Reinhart Butter, Krippendorff himself forwarded the concept of product semantics – the investigation into, and deliberate use of, cultural symbols in design (12). They suggested that people map conscious and unconscious meanings from prior experience onto products instantaneously. Several designers working in the Post-Modernist vein, such as Marcel Wanders(7), have successfully utilized this higher level understanding of cultural significance to create works that seem to integrate intended narratives (often localized), in a way that eluded earlier designers.
Concurrent with these new insights, was the idea of Human Cantered Design (!). HCD aims to exploit natural mappings, allowing users to engage with new technology in an intuitive manner. This concept became more important with the event of Guide User Interfaces for computers. Early interfaces were designed for, and by, fellow programmers requiring specialist knowledge. As other users began to engage with personal computers it became clear that proper interface design was necessary. Human Computer Interface now has a mass of research and literature codifying well established conventions for accessing and manipulating data in a hierarchical environment observed via a monitor.
Entering the 21st Century, many new developments have occurred in contemporary design; however there are three threads that are particularly pertinent to this contextualization. Firstly the rise of the poetic notion within design circles as promulgated by Fiona Raby and Anthony Dunne (qtd. in Moggridge 595). Often political, poetic design seeks to ask questions of the user or recipient – it is the relationship between the user and object that is the focus of attention – indeed poetic design veers towards fine art practices, frequently creating works that are not intended for mass production, or in actuality, any production at all.
Janine Benyus is commonly considered the main proponent of a second important thread in contemporary design practice – biomimicry. By studying biology, innovative answers to design and technological problems may be solved, with the additional benefit of using aesthetic values that we seem to be pre-programmed to value as inherently beautiful. A seminal pre-theory example transpired when George de Mestral noticed the burrs attached to his dog and his socks after a walk, leading to the invention of Velcro.
Thirdly, the advent of computer generated design utilizing mathematical algorithms, again derived from nature, has begun to impact design, most notably in architecture. By incorporating generative algorithms, practitioners can incorporate unique data sets, and complex formations hitherto extremely difficult, if not impossible, to realise using traditional techniques. Many of these algorithms were originally written by scientists in order to predict and analyse biological phenomena.
These developments in design practice and theory are significant precursors to my intended research. My characterization of a biokinetic interface for audio equipment utilises two direct inputs to shape the physical output: the user’s interaction and the music playing. By combining the variance in user control input and dynamic analysis of the musical waveform in the form of digital algorithms, a microprocessor controlled physical output may be realised. This output is constantly moving – quickly or slowly, gently or violently, reflecting the mutable inputs. The interface also is sensitive to what has already passed; it has a historical reflex. Hans-Georg Gadamer argued that people have a ‘historically effected consciousness’ (wirkungsgeschichtliches Bewußtsein) (87); biokinetic interfacing attempts to model this concept within the context of an electronic component. By manipulating a dynamic feedback, moderated by past inputs, the interface may be said to be perpetually evolving.
The political aspect of the interface has interesting consequences: for example if Harry has played string quartets and John Fahey quietly all month, the interface will possess some sort of calm layout. When Sally comes home and cranks up Sepultura and Pantera by angrily stabbing at the media boundary, the interface will subjectively become brutal, due the intense interaction. Has Sally wrecked the meditative interaction that Harry has consciously, or unconsciously, been cultivating all month? Will Harry become agitated? Will he play more Pantera louder? Or will he try and disentangle the mess? Is it a mess?
Biokinetic interfacing potentially offers new semantic and instrumental mediation between users and electronic objects. In order to achieve tacit understanding, I believe that a suitable metaphor may be fire. Objectively we may use the fire as a heat source (sound), but we may also sit and gaze on the flickering light, basking in reverie, or contemplate in solitude. It may a central focus, or we may re-centre it to our periphery, conscious of its existence. Writing about kinetic interfaces (defined as screen-based applications using movement), Eikenes says:
(Screen-based) interfaces are constructed artefacts; they have to be designed by someone. Interface design plays an important role in shaping mediated human activity and enabling meaning making. There is a need for studies that focus on the (bio) kinetic interface as a mediating and cultural artefact in its own right, recognising its situatedness in social and cultural contexts. (13)
METHODOLOGY

This project fits into the category of “wicked problems” as identified by Horst Rittel (qtd. in Nelson and Stoltermann 15). These sorts of problems not quickly definable as “tame problems”, and are partly chararactorized by possessing some of the following qualities:
• Cannot be exhaustively formulated
• Every formulation is the statement of a solution
• No true or false
• Many explanations for the same problem
• Every problem is the symptom of another problem
• No immediate or ultimate test
• Every problem is essentially unique
In order to avoid the paralysis that might be experienced by pursuing a linear methodology as common in consumer product design:
Understand problem> Gather information > Analyse information > Generate solutions >Assess the solutions >Implement>Test>Modify>Present
I will instead pursue an action research based methodology that emphases the iterative formulation of models and prototypes as medium for testing, allowing time for critical reflection, necessitating the new information which in turn informs the next sequence of creation, and so on.

METHODS
Prior to beginning the initial phase of making, I need to acquire greater knowledge in several areas. Foremost is a deeper knowledge of the subject via further reading. In parallel with this activity I also need to raise the level of my slim ability to write Processing code. Processing is used in two critical parts of my project: the computer algorithm based generative graphics used for simulacra, and more importantly for coding to Arduino[5], a microprocessor platform which will allow me to build working models.
Parallel to, and accelerating from, completion of this phase I shall use a variety of methods to create, simulate, test, refine and build drawings, models and prototypes. These include: freehand drawing, ceramic work, rough prototyping, hand building, Solidworks modelling, Rhino>Grasshopper modelling, CNC milling, stop motion animation, video, rapid prototyping, microprocessor controlled models, etc.
User testing is an important part of this work in order to validate, or mitigate, my design assumptions. There are four types of user testing/awareness that will be targeted. Firstly, informal ad hoc testing, on students and facility members. I am hoping to set up a video booth, automatically filming when people engage with the models. This will allow me to watch the de facto working knowledge of participants developing over time, and gain a better phenomenographic insight into the interaction.
Secondly, I will conduct formal tests on test subjects (subject to approval of the ethics committee) to assess more technical issues related to control and tacit knowledge. Thirdly, in the later stages of the project I will conduct longer term user evaluation – placing models within participant’s dwellings to obtain knowledge of the interface in context. Lastly I will be maintaining a website www.metier.co to detail learning outcomes, and invite domestic and international dialogue.

LITERATURE REVIEW
There is a distinct paucity of literature directly addressing biokinetic interfaces: relevant texts tend to be short chapters in books addressing other topics. Therefore, in order to gain a theoretical and practical framework a wide variety of disciplines must be examined.
Design theorists often cite the phenomenological philosophers examining being-object relationships such as Husserl, de Serteau, Merleau-Ponty and especially Martin Heidigger, as the intellectual basis for many subsequent design concepts. Mihalyi Csikszentmihalyi has made a contribution with his concept of flow and consideration of happiness, as has Maslow with his hierarchies of human need.
Writers specifically associated with design looking at the underlying human-object relationship include Klaus Krippendorff writing about product semantics, J.J. Gibson and his concept of affordance, the venerable Donald Norman extending that idea into advocating usability, and Patrick Jordan extolling the principle of pleasurable products, among others. Contemporary collected works edited by the theorists Victor Margolin & Richard Buchanan, Vera Buhlmann, Pieter Desmet & Co., Ilpo Koskinen & Co. with many others provide rich sources of fragmented insight. These collected volumes of academic text also have excellent reference lists which will be invaluable for further reading.
There has been a lot of writing examining Human Computer Interaction, although usually targeted at screen based GUIs, their underlying systems analysis and discussion of natural mappings is an excellent basis for basic system interface design. Alan Cooper’s extension of the subject into the creation of personas will influence this research, as will John Maeda’s focus on the return to simplicity in HCI. Designing Interactions, edited by Bill Moggridge is a good primer on the topic, while Jon Kolko’s book Thoughts on Interaction Design is an excellent, pithy, work describing the scope of the subject.
Another way of approaching the subject of futuristic interfaces is the examination of Cybernetics and Artificial/Ambient Intelligence. The collection Prefiguring Cyberculture – An Intellectual History has several excellent chapters detailing trends and theories in this area. Futurists such as Kevin Kelly and Roger Penrose think and write about future technologies and subsequent ramifications, as do a host of science fiction writers such as Stanislaw Lem and Neal Stephenson.

KEY READINGS
Krippendorff, Klaus. The Semantic Turn – A new foundation for design. 2005 New York: CRC Press
The Semantic Turn is perhaps the definitive book on design theory as related to product semantics – the change of emphasis from functionality to the implicit meaning of the object. Krippendorff details affordance, semiotics and maps associated vocabulary and details coherent design practices for semiotic design.
Konecˇni, Vladimir. The Psychology of Music – Social interaction and musical preference. 1982
New York: Academic Press. Date of access: 8 April 2011

Konecˇni is one of the very few researchers engaged in the subject of listening to music within a domestic environment. In particular, he looks at the causal links between music listening and emotion.

Eikenes, Jon Olav Husabø. Navimation – A sociocultural exploration of kinetic interface design,
2010. 13 Oslo, Norway: ADORA database. Date of access: 22 March 2011
< http://brage.bibsys.no/aho/handle/URN:NBN:no-bibsys_brage_17205>

Eikenes’ contemporary work is his PHD thesis, written specifically about screen based interfaces. This work delves into several of the same issues (sociological context, semiotics, user testing, systems structure, etc.) as presented by biokinetic interfaces. Additionally the format and methods of investigation offer a template for entering this emergent subject.

BIBLIOGRAPHY
Alperson, Phillip. Eds. What is Music? An Introduction to the Philosophy of Music. 1987. Pennsylvania, USA: The Pennsylvania State University Press
Antonelli, Paula. Eds. Design and the Elastic Mind. 2008. New York: The Museum of Modern Art
Bang, Jens. Bang & Olufsen – From Spark to Ocean. 2005. Struer, Denmark: Bang & Olufsen
Barzun, Jacques. Eds. Pleasures of Music. 1951. London: Cassell & Company
Buchanan, Richard. Doordan, Dennis& Margolin, Victor. Eds. The Designed World – Images, Objects, Environments. 2010. New York: Berg
Buchanan, Richard. & Margolin, Victor. Eds. Discovering Design. 1995. Chicago: The University of Chicago
Buhlmann, Vera. & Wiedmar, Simon. Eds. Pre-Specifics – Some Comparatistic Investigations on Research in Design and Art. 2008. Zurich: JRP-Ringier
Caplan, Ralph. By Design – Why there are no locks on the bathroom doors in the Hotel Louis XIV and other object lesson. 2005. New York: Fairchild Publications
Cerbone, David. Understanding Phenomenology. 2006. Chesham: Acumen Publishing
Coles, Alex. Eds. Design and Art. 2007. London: Whitechapel Ventures Ltd
Desmet, Pieter. van Erp, Jeroen & Karlsson, Marie Anne. Eds. Design & Emotion Moves. 2008. Newcastle upon Tyne: Cambridge Scholars Publishing
Dormer, Peter. The Meaning of Modern Design – Towards the Twenty First Century. 2006. London: Thames & Hudson Ltd
Fisher, Phillip. Wonder, the Rainbow, and the Aesthetics of Rare Experiences. 1998. London: Harvard University Press.
Gadamer, Hans Georg. Truth and Method (2nd Edition). 2004. London: Continuum
Green, William & Jordan, Patrick. Eds. Human Factors in Product Design – Current practice and future trends. 2001. London: Taylor & Francis
Hall, Sean. This Means This. That Means That: A users guide to semiotics. 2007. London: Laurence King Publishing Ltd
Heskett, John. Design – A very short introduction. 2002. Oxford, UK: Oxford University Press
Kelly, Kevin. Out of Control: The new biology of machines. 1994. London: Fourth Estate Ltd
Kolko, Jon. Eds. Thoughts on Interaction Design. 2010. Massachusetts: ELSEVIER Inc.
Koskinen, Ilpo, Battarbee, Katja & Mattelmaki, Tuuli. Eds. Empathetic design – User Experience in Product Design. 2007. Edita, Finland: Edita Publishing Ltd
Lanham, Richard. The Economics of Attention. 2007. Chicago: The University of Chicago Press
Liu, Kecheng. Semiotics in Information Systems Design. 2000.
Lowgren, Jonas. & Stolterman, Eric. Thoughtful Interaction Design. 2004. Massachusetts: MIT Press
Luhmann, Niels. Social Systems. 1995. Translated by Bednarz Jr., J. California: Stanford University Press
Merleau Ponty, Maurice. The Visible and the Invisible. 1968. Evanston: Northwestern University Press
Moggridge, Bill. Eds. Designing Interactions. 2007. Massachusetts: MIT Press
Naylor, Maxine & Ball, Ralph. Form Follows Idea: An introduction to design poetics. 2005. London: Black Dog Publishing.
Nelson, Harold. & Stolterman, Eric. The Design Way. 2003. New Jersey: Educational Technology Publications
Norman, Donald. The Design of Future Things. 2007. New York: Basic Books
North Adrian. & Hargreaves, David. The Social and Applied Psychology of Music. 2008. Oxford: Oxford Press
Oei, Loan. & De Kegel, Cecila. Elements of design – Rediscovering Colors, Textures, Forms and Shapes. 2002. London: Thames & Hudson Ltd
Penrose, Roger. Shadows of the Mind. 1995. London: Vintage
Tofts, Darren. Jonson, Annemarie & Cavallaro, Alessio. Eds. Prefiguring Cyberculture – An Intellectual History. 2002. Massachusetts: The MIT Press
Virilo, Paolo. Unknown Quality. 2002. Nantes: Imprimerie Le Govic
Wanders, Marcel. Behind the Ceiling. 2009. Berlin: Die Gestalten Verlag GmbH & Co.

GLOSSARY

[1] Biokinetic Interfacing
A biokinetic interface is a form of dynamic and ever changing device that enables semantic and instrumental mediation between one or more users and an electronic component. Eschewing touchscreens, a biokinetic interface uses physical input from the user to control/manipulate the component and responds giving user visual/haptic feedback by moving all, or part, of itself by means of microprocessor controlled physical output. The basis of the movement is a biological algorithm that combines the variance in physical user input and the component’s functionality, regulated via basic interface guidelines. By manipulating dynamic inputs, moderated by past configurations, the interface may be said to be perpetually evolving.
[2] Affordance
Originally coined by J.J. Gibson, affordance is the quality of an object or environment that allows an individual to perform an action. He posited that humans perceive objects in terms of what they may be used for, rather than objects in themselves. Rather than the open ended possible utilities of any given object as suggested by Gibson, Donald Norman refined the term ‘perceived affordance’ to those possibilities that are readily perceived and intuited, particularly in regard to Human Computer Interfaces.
[3] Product Semiotics
Refers to the study of the intentional use of signs and meaning in the design of consumer and industrial products. Marking a change from an emphasis on functionality and efficiency, product semantics assumes these basic conditions are met, and seeks to investigate acquired and embodied meaning(s) within artefacts as integral to the final design outcome. The term was originally coined by Klaus Krippendorff and Reinhart Butter.
[4] Phenomenography
Phenomenography is an empirical research methodology that seeks to obtain and analyse quantitative data from the study of the different ways that people experience objects and situations. It is not the same as phenomenology, which is the philosophical study of experience, but both ideas are subjectivist – based on the assumption that there is one world and there are many ways to experience the same object or environment.
[5] Arduino
Arduino is an open source microprocessor board that is designed run on the open source language Wiring, which is in turn based on Processing developed by Casey Reas and Ben Fry of MIT. Arduino allows multiple inputs and the ability to manipulate multiple outputs in many different ways – dependant on the coder’s ability and imagination. Arduino can interface with almost any device capable of exporting or importing numerical/electrical data – such as LEDs, stepper motors, electromagnets, light sensors, touchscreens, etc. Arduino was originally supervised by Massimo Banzi and David Cuartielles of the Ivrea Institute in Italy.