Talk and workshop on the MINDtouch research project progress and findings

1. Camille Baker, MASc Interactive Arts
SMARTlab PhD Candidate
Goldsmiths Thursday Club
March 18, 2010
MindTouch:
biosensor-mobile
media collaborative
Vjing activities

2. • studied and performed modern dance as an undergraduate
and in high school, until 1992.
Camille dancing circa 1987
background/ bio

3. over 15 years in independent media production + art:
• owned and operated conventional contemporary art gallery in
Vancouver in 1998;
stills from gallery circa 1999
background/ bio

4. over 15 years in independent media production + art:
• documentary film development, online video and animation,
new media and web design;
stills from video pieces
background/ bio

5. over 15 years in independent media production + art:
• produced, wrote and edited several videos and web video
shorts/animation, online content, scripts and film projects
still from video piece
background/ bio

6. over 15 years in independent media production + art:
• founder, web editor-in-chief of own online magazine (2000-
2002), www.slackerbonding.com on GenX relationship and
pop culture issues;
screenshot from
slackerbonding.com
offline website
background/ bio

7. over 15 years in independent media production + art:
• studied technical applied skills in New Media, Web Design
and CD Rom Development Certificate Program at BC Institute
of Technology, Burnaby, BC, Canada, 2000-2001;
student animation piece
background/ bio

8. over 15 years in independent media production + art:
• singer/ songwriter, performer, and manager in electronic and
alt-rock/pop bands from 1993-2007: www.ultrapuss.com
(current) and www.spiritualheroine.com (former long-term
project);
band photos from Spiritual Heroine and ultrapuss
background/ bio

9. background/ bio
slideshow from gallery
exhibition NFF 2004
• Lead Curator and Event Producer for The New Forms
Festival producing + curating a gallery, conference and
performances

10. • Lead Curator and Event Producer for The New Forms
Festival, a new media arts festival in Vancouver from 2002-
2004 - 3 years;
background/ bio
slideshow from dance
performance NFF 2004

11. • Art Researcher/ Videographer / Installation coordinator for
the whisper project a wearable devices and biofeedback
research project for Prof. Thecla Schiphorst and Dr. Susan
Kozel from 2003-2006;
background/ bio
video I shot at
Siggraph 2005

12. SMARTlab PhD research: performance media
–has been to uncover any new understandings of the
sensations of ‘liveness’ and ‘presence’ that may emerge
when using mobile technologies and wearable devices in
performance contexts–

13. liveness in mobile video participatory performance
 liveness (mediated)/ virtual (real)
 presence (absence)
 experience (perception)
 locative mobile media / video
 embodied performance
 visceral / sensation
 intimacy / immediacy /mobility
 sensor technology and the body
 performers / audience

14. mobile devices as non-verbal expression
first video collection workshop in Vancouver June
2007

15. presence
presence
conscious perception
extended mind
telepathy
sensing from a distance
biochemistry
physical sensing
feltness
non-verbal communiction
virtual/ digital/
networked / mobile
voice / emotional
gestural / eye contact
spiritual / mystical
yoga / meditation
embodiment /
being "in" the body
situatedness
emotions
bodily experienced
emotions
sensors
focusing /
being "here" now
a presence/
sentient, ephemeral other
unconsciousness
dream body/mind
out-of body
experiences

16. liveness
liveness
embodied
physical interaction
feltness
non-verbal communiction
in body / in person
gestural / eye contact
performing in-person
nowness / this moment
time / duration
chemical reactions
interaction
instant emotional
reactions
this space
being "here" now
performance interaction:
sending + receiving energy
immersion
tactility, viscerality
+ physicality
being present
engagement
interaction
transitory / immediate
proximity / nearness
authenticity /realness
spontaneous / uncertain
original /
never the same
intimate

18. mobile embodied performance media

19. Image © 2007 C. Baker workshop still Dublin July 2007
video collection workshops : two - Dublin

20. video collection workshops : three - Vancouver
Images © 2007 C. Baker - stills from participants’ videos from Vancouver workshop August 2007

21. video collection workshops : fourth - Dublin
Images© 2007 C. Baker - stills from participants’ videos from Dublin workshop October 2007

22. mobile video – examples
videos © 2007-2009 C. Baker - participants’ videos from Vancouver and Dublin workshops 2007

23. sensors + software
Images © 2008 C. Baker – Michael Markert working on biosensing electronics for custom software

24. mobile video software – development
Images © 2008 C. Baker – custom software by Michael Markert

25. smart garments
Images © 2008 C. Baker – sketches of garments with embedded electronics by Michael Markert and Tara Mooney

26. - making the 1st prototype
Images © 2008 C. Baker – sketches of garments with embedded electronics by Tara Mooney with Rachel Lesebikan

27. the 1st prototype
Images © 2008 C. Baker – design stages of garments with embedded electronics by Tara Mooney with Rachel Lesebikan

28. finished prototype 1: garment and embedded sensors
(completed November 2008)
Images © 2008 C. Baker – alpha garment with embedded electronics by Tara Mooney with Rachel Lesebikan

29. prototype 2: garment design
– design process
Images © 2009 C. Baker – beta garment with embedded electronics by Tara Mooney with Rachel Lesebikan

30. prototype 2: garment design
(completed August 2009)
Images © 2009 C. Baker – beta garment with embedded electronics by Tara Mooney with Rachel Lesebikan

31. figure © 2009 C. Baker

32. mobile video software – vision
Images © 2007 C. Baker

33. mobile
video
software
development

34. mobile video software
– development

35. mobile video software – available options
Image from http://r3nder.net/mt/mt-search.cgi?IncludeBlogs=7&search=mobileImage from http://mobile.processing.org/learning/example.php?name=photoslider

36. video software – beta (not ready for phones)

37. V
large screen video mixes from mobile collaboration
large screen video mixes from mobile collaboration
body
data
display
from
phone
body
data
display
from
phone
group 1 -
performing w/
biofeedback sensors
Performance Set Up / Staging
group 2 -
mobile phone VJ's
group 3 -
internal video exploring

38. performance experiment – London July 2009

39. mobile video from performance for mixing

40. still of mixes for broadcast

41. streaming server

42. remote
mobile
streaming
interaction

43. remote
mobile
streaming
interaction

44. network
chain /
interaction

45. streamed output videos

46. performance 1 – video documentation

47. recent video mini-doc for Ars Electronica

48. related work – video + biosensing
Tina Gonsalves http://www.tinagonsalves.com/
Tina Gonsalves is one of those artists, whose work is most similar in intention, nature and
outcomes to mine (although I just discovered her).
FEEL:TRACE : responsive biofeedback installation (Australia 2006) :
“…is a psychophysiologically responsive video installation synthesizing art, neuroscience
and technology. The project explores new, more embodied languages of interactive and
emotional communication, investigating the inter-relationship of the internal body and the
external world. Using biosensors, the participant’s heart rate responses are monitored.

49. related work – mobile video
Mobile Video Art : mobile video paintings by Dean Terry
painting from a cameraphone & unedited. www.100lies.com
http://www.deanterry.com/blog/index.php/2005/10/
Speech Marks 2004 Steve Hawley
http://www.artdes.mmu.ac.uk/profile/shawley/image/10349

50. Blast Theory (2003) – Uncle Roy All Around You a participatory
performance as well as a mobile and internet game. It engaged users in
unique, theatrical activities using the participants / gamers in the city
using portable devices and phones rather than only avatars only on
computers.
related work – participatory performance

51. • the whisper project a wearable devices and biofeedback
research project for Prof. Thecla Schiphorst and Dr. Susan
Kozel 2003-2006;
Siggraph 2005
related work – video + biosensing
http://whisper.iat.sfu.ca/
“…technology and
communications metaphors that
enable networked wearable
devices to communicate affective
states in a continuous manner”

52. SKIN, which examines the future integration of
sensitive materials in the area of emotional
sensing – the shift from ‘ intelligent’ to
‘sensitive’ products and technologies.
As part of SKIN, we have developed two ‘Soft
Technology’ outfits to identify the future for
high tech materials and Electronic Textile
Development in the area’s of skin and
emotional sensing.
The dresses show emotive technology and
how the body and the near environment can
use pattern and color change to interact and
predict the emotional state.
Philips Skin Probes
related work – biosensing + wearables
http://www.design.philips.com/probes/projects/dresses/index.page

53. questions?