SlideShare una empresa de Scribd logo
1 de 25
A WEARABLE MUSICAL
INTERFACE USING A WIRELESS
SENSOR NETWORK
Francesco Corucci – – 2012
MSc in Computer Engineering – University of Pisa
Introduction
 Goal: implement a platform that allows the use of a
wireless sensor network as a wearable interface
that can be used for producing music
 Requirements:
 Modular
 Flexible
 MIDI-compliant
2
What is MIDI
 MIDI (Musical Instrument Digital Interface) is a
standard technology for the interaction between
musical devices
 It standardizes:
 A communication protocol
 An hardware interface
 Versatile technology, widely used since the 80’s and still
reliable after more than 30 years
3
Shimmer sensor nodes
 For setting up the WSN we choose Shimmer sensor
nodes :
 Suitable for wearable applications
 Integrate a 3-axis accelerometer
 Optional gyroscope shield
(not used in this project)
 Bluetooth or 802.15.4
communication
 Support TinyOS
4
Shimmer sensor nodes
5
Overview of the system
USB
802.15.4
• Two or more wireless sensors on the body
• The sensors sample accelerometric data
associated with movements
• Data are then collected from the sink and
forwarded to the computer
Sink
6
Overview of the system (cont’d)
Oscilloscope application
(extended with MIDI map module)
Virtual MIDI
device
Musical software
Data from sink
• Filtering
• Displaying
• Mapping in MIDI commands
7
Detailed architecture
Serial forwarder
Computer
USB
TCP/IP
Oscilloscope
Radio
MIDI map
MIDI forwarder
Virtual MIDI port
TCP/IP
Musical sw
(VST-host)
VST
Plugin
VST
Plugin
MIDI
MIDI
8
…
MIDI controller
This way the WSN become a standard MIDI controller,
that can interoperate with every musical software
9
How to choose the mapping
 The way to map sensed data into MIDI messages is
absolutely arbitrary, and can lead to a multitude of
expressive results
 Every data channel can be mapped in a different
 instrument
 effect
 note/chord
 etc…
 …and this can be done in a variety of ways!
10
Demonstration time
(http://youtu.be/jIrE5j2Mdtw)
11
Features
 Modular, extensible:
 Is it possible to use an arbitrary number of sensor nodes, and
choose which on-board sensors must be used as data sources
 Flexible:
 Many degrees of freedom in the mapping
 Many expressive possibilities
 Sound generation is left to specialized and mature softwares
 Standard:
 Since the WSN becomes a MIDI controller, it is able to
interoperate with a moltitude of software platforms
12
Possible applications
The platform can be used for several purposes:
 New expressive forms based on gestuality
 In combination with traditional instruments
(e.g. influence of on-stage movements on played music)
 Any application that asks for sonification of human movements:
 Sport training
 Rehabilitation
 …
13
References
 Shimmer sensor nodes
 http://www.shimmer-research.com/
 General information about MIDI
 http://en.wikipedia.org/wiki/MIDI
 VST
 http://en.wikipedia.org/wiki/Virtual_Studio_Technology
 rtMidi, a cross-platform C++ library for MIDI
 http://www.music.mcgill.ca/~gary/rtmidi/
 Sonification
 http://en.wikipedia.org/wiki/Sonification
14
Thank you for your attention :-)
15
Implementation details
16
OscilloscopeAppC - overview
 OscilloscopeAppC is the TinyOS application running
on sensor nodes
 Receives commands from the sink:
start/stop/change_freq
 Samples the on-board tri-axis accelerometer using
three ADC channels in DMA mode
 When a block of samples has been collected, the
sensor node sends it to the sink via radio
17
OscilloscopeAppC - architecture
18
OscilloscopeAppC - interactions
19
…
BasestationC - overview
 BasestationC is the TinyOS application running on
the sink node
 Acts as a bridge between the serial and the radio
link
 Implements queues in both direction to handle
traffic spikes
20
BasestationC - architecture
21
MIDI map module
 Offers a simple GUI: the user can choose how to map
the active data channels into MIDI messages from a
predefined set of mappings
22
 The module then catches
incoming data, maps it into
MIDI command according
to user-defined
preferences, and forwards
them via TCP/IP socket
toward the MIDI forwarder
MIDI forwarder
 Implements a virtual
MIDI device: the WSN is
seen as an input MIDI
peripheral plugged into
the computer
 It then acts as a bridge:
 Waits for incoming MIDI
messages from a TCP/IP
socket
 Forwards MIDI messages
to the virtual MIDI port
23
Screenshots: Oscilloscope + MidiMap
24
Screenshots
25

Más contenido relacionado

Similar a A wearable MIDI interface using a wireless sensor network

Passenger amenities (2)
Passenger amenities (2)Passenger amenities (2)
Passenger amenities (2)Suraj Soni
 
Education set for collecting and visualizing data using sensor system based o...
Education set for collecting and visualizing data using sensor system based o...Education set for collecting and visualizing data using sensor system based o...
Education set for collecting and visualizing data using sensor system based o...IJMER
 
Experimental Analysis of an Energy-Efficient WSN
Experimental Analysis of an Energy-Efficient WSNExperimental Analysis of an Energy-Efficient WSN
Experimental Analysis of an Energy-Efficient WSNidescitation
 
StadtLärm - A Distributed Urban Noise Monitoring System
StadtLärm - A Distributed Urban Noise Monitoring System StadtLärm - A Distributed Urban Noise Monitoring System
StadtLärm - A Distributed Urban Noise Monitoring System M2M Alliance e.V.
 
Education set for collecting and visualizing data using sensor system based ...
Education set for collecting and visualizing data using sensor  system based ...Education set for collecting and visualizing data using sensor  system based ...
Education set for collecting and visualizing data using sensor system based ...IJMER
 
Decibel meter using IoT with notice board
Decibel meter using IoT with notice boardDecibel meter using IoT with notice board
Decibel meter using IoT with notice boardIRJET Journal
 
Garbage Monitoring System using Arduino
Garbage Monitoring System using ArduinoGarbage Monitoring System using Arduino
Garbage Monitoring System using Arduinoijtsrd
 
A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...
A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...
A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...csandit
 
A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...
A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...
A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...cscpconf
 
A proposed model for traffic signal preemption using global positioning syste...
A proposed model for traffic signal preemption using global positioning syste...A proposed model for traffic signal preemption using global positioning syste...
A proposed model for traffic signal preemption using global positioning syste...csandit
 
A ZigBee Based Wireless Sensor Network for an Agricultural Environment
A ZigBee Based Wireless Sensor Network for an Agricultural EnvironmentA ZigBee Based Wireless Sensor Network for an Agricultural Environment
A ZigBee Based Wireless Sensor Network for an Agricultural Environmentijsrd.com
 
6 article azojete vol 8 53 66
6 article azojete vol 8 53 666 article azojete vol 8 53 66
6 article azojete vol 8 53 66Oyeniyi Samuel
 
Wireless multimedia sensor networking
Wireless multimedia sensor networkingWireless multimedia sensor networking
Wireless multimedia sensor networkingKedar Kumar
 
Wireless Open Weather (WOW)
Wireless Open Weather (WOW)Wireless Open Weather (WOW)
Wireless Open Weather (WOW)Mirko Mancin
 

Similar a A wearable MIDI interface using a wireless sensor network (20)

Passenger amenities (2)
Passenger amenities (2)Passenger amenities (2)
Passenger amenities (2)
 
WIRELESS SENSOR NETWORK
WIRELESS SENSOR NETWORKWIRELESS SENSOR NETWORK
WIRELESS SENSOR NETWORK
 
Education set for collecting and visualizing data using sensor system based o...
Education set for collecting and visualizing data using sensor system based o...Education set for collecting and visualizing data using sensor system based o...
Education set for collecting and visualizing data using sensor system based o...
 
Experimental Analysis of an Energy-Efficient WSN
Experimental Analysis of an Energy-Efficient WSNExperimental Analysis of an Energy-Efficient WSN
Experimental Analysis of an Energy-Efficient WSN
 
StadtLärm - A Distributed Urban Noise Monitoring System
StadtLärm - A Distributed Urban Noise Monitoring System StadtLärm - A Distributed Urban Noise Monitoring System
StadtLärm - A Distributed Urban Noise Monitoring System
 
Education set for collecting and visualizing data using sensor system based ...
Education set for collecting and visualizing data using sensor  system based ...Education set for collecting and visualizing data using sensor  system based ...
Education set for collecting and visualizing data using sensor system based ...
 
Aquarius_IJARCCE
Aquarius_IJARCCEAquarius_IJARCCE
Aquarius_IJARCCE
 
Decibel meter using IoT with notice board
Decibel meter using IoT with notice boardDecibel meter using IoT with notice board
Decibel meter using IoT with notice board
 
Garbage Monitoring System using Arduino
Garbage Monitoring System using ArduinoGarbage Monitoring System using Arduino
Garbage Monitoring System using Arduino
 
published paper
published paperpublished paper
published paper
 
A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...
A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...
A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...
 
A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...
A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...
A PROPOSED MODEL FOR TRAFFIC SIGNAL PREEMPTION USING GLOBAL POSITIONING SYSTE...
 
A proposed model for traffic signal preemption using global positioning syste...
A proposed model for traffic signal preemption using global positioning syste...A proposed model for traffic signal preemption using global positioning syste...
A proposed model for traffic signal preemption using global positioning syste...
 
CDMA
CDMACDMA
CDMA
 
A ZigBee Based Wireless Sensor Network for an Agricultural Environment
A ZigBee Based Wireless Sensor Network for an Agricultural EnvironmentA ZigBee Based Wireless Sensor Network for an Agricultural Environment
A ZigBee Based Wireless Sensor Network for an Agricultural Environment
 
6 article azojete vol 8 53 66
6 article azojete vol 8 53 666 article azojete vol 8 53 66
6 article azojete vol 8 53 66
 
Contiki wsn
Contiki wsnContiki wsn
Contiki wsn
 
Wireless multimedia sensor networking
Wireless multimedia sensor networkingWireless multimedia sensor networking
Wireless multimedia sensor networking
 
Wireless Open Weather (WOW)
Wireless Open Weather (WOW)Wireless Open Weather (WOW)
Wireless Open Weather (WOW)
 
Arduino Radar System
Arduino Radar SystemArduino Radar System
Arduino Radar System
 

Más de Francesco Corucci

Teleoperating a robotic arm through a gyroscopic helmet
Teleoperating a robotic arm through a gyroscopic helmetTeleoperating a robotic arm through a gyroscopic helmet
Teleoperating a robotic arm through a gyroscopic helmetFrancesco Corucci
 
Real-time PID control of an inverted pendulum
Real-time PID control of an inverted pendulumReal-time PID control of an inverted pendulum
Real-time PID control of an inverted pendulumFrancesco Corucci
 
Social Network Analysis Project
Social Network Analysis ProjectSocial Network Analysis Project
Social Network Analysis ProjectFrancesco Corucci
 
Implementation of a lane-tracking system for autonomous driving using Kalman ...
Implementation of a lane-tracking system for autonomous driving using Kalman ...Implementation of a lane-tracking system for autonomous driving using Kalman ...
Implementation of a lane-tracking system for autonomous driving using Kalman ...Francesco Corucci
 
An overview on Quantum Key Distribution
An overview on Quantum Key DistributionAn overview on Quantum Key Distribution
An overview on Quantum Key DistributionFrancesco Corucci
 
P-Systems for approximating NP-Complete optimization problems
P-Systems for approximating NP-Complete optimization problemsP-Systems for approximating NP-Complete optimization problems
P-Systems for approximating NP-Complete optimization problemsFrancesco Corucci
 

Más de Francesco Corucci (6)

Teleoperating a robotic arm through a gyroscopic helmet
Teleoperating a robotic arm through a gyroscopic helmetTeleoperating a robotic arm through a gyroscopic helmet
Teleoperating a robotic arm through a gyroscopic helmet
 
Real-time PID control of an inverted pendulum
Real-time PID control of an inverted pendulumReal-time PID control of an inverted pendulum
Real-time PID control of an inverted pendulum
 
Social Network Analysis Project
Social Network Analysis ProjectSocial Network Analysis Project
Social Network Analysis Project
 
Implementation of a lane-tracking system for autonomous driving using Kalman ...
Implementation of a lane-tracking system for autonomous driving using Kalman ...Implementation of a lane-tracking system for autonomous driving using Kalman ...
Implementation of a lane-tracking system for autonomous driving using Kalman ...
 
An overview on Quantum Key Distribution
An overview on Quantum Key DistributionAn overview on Quantum Key Distribution
An overview on Quantum Key Distribution
 
P-Systems for approximating NP-Complete optimization problems
P-Systems for approximating NP-Complete optimization problemsP-Systems for approximating NP-Complete optimization problems
P-Systems for approximating NP-Complete optimization problems
 

Último

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j
 
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAzure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAndikSusilo4
 
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphSIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphNeo4j
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...HostedbyConfluent
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Alan Dix
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 3652toLead Limited
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksSoftradix Technologies
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 

Último (20)

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
 
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAzure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & Application
 
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphSIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
 
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
Tech-Forward - Achieving Business Readiness For Copilot in Microsoft 365
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 

A wearable MIDI interface using a wireless sensor network

  • 1. A WEARABLE MUSICAL INTERFACE USING A WIRELESS SENSOR NETWORK Francesco Corucci – – 2012 MSc in Computer Engineering – University of Pisa
  • 2. Introduction  Goal: implement a platform that allows the use of a wireless sensor network as a wearable interface that can be used for producing music  Requirements:  Modular  Flexible  MIDI-compliant 2
  • 3. What is MIDI  MIDI (Musical Instrument Digital Interface) is a standard technology for the interaction between musical devices  It standardizes:  A communication protocol  An hardware interface  Versatile technology, widely used since the 80’s and still reliable after more than 30 years 3
  • 4. Shimmer sensor nodes  For setting up the WSN we choose Shimmer sensor nodes :  Suitable for wearable applications  Integrate a 3-axis accelerometer  Optional gyroscope shield (not used in this project)  Bluetooth or 802.15.4 communication  Support TinyOS 4
  • 6. Overview of the system USB 802.15.4 • Two or more wireless sensors on the body • The sensors sample accelerometric data associated with movements • Data are then collected from the sink and forwarded to the computer Sink 6
  • 7. Overview of the system (cont’d) Oscilloscope application (extended with MIDI map module) Virtual MIDI device Musical software Data from sink • Filtering • Displaying • Mapping in MIDI commands 7
  • 8. Detailed architecture Serial forwarder Computer USB TCP/IP Oscilloscope Radio MIDI map MIDI forwarder Virtual MIDI port TCP/IP Musical sw (VST-host) VST Plugin VST Plugin MIDI MIDI 8 …
  • 9. MIDI controller This way the WSN become a standard MIDI controller, that can interoperate with every musical software 9
  • 10. How to choose the mapping  The way to map sensed data into MIDI messages is absolutely arbitrary, and can lead to a multitude of expressive results  Every data channel can be mapped in a different  instrument  effect  note/chord  etc…  …and this can be done in a variety of ways! 10
  • 12. Features  Modular, extensible:  Is it possible to use an arbitrary number of sensor nodes, and choose which on-board sensors must be used as data sources  Flexible:  Many degrees of freedom in the mapping  Many expressive possibilities  Sound generation is left to specialized and mature softwares  Standard:  Since the WSN becomes a MIDI controller, it is able to interoperate with a moltitude of software platforms 12
  • 13. Possible applications The platform can be used for several purposes:  New expressive forms based on gestuality  In combination with traditional instruments (e.g. influence of on-stage movements on played music)  Any application that asks for sonification of human movements:  Sport training  Rehabilitation  … 13
  • 14. References  Shimmer sensor nodes  http://www.shimmer-research.com/  General information about MIDI  http://en.wikipedia.org/wiki/MIDI  VST  http://en.wikipedia.org/wiki/Virtual_Studio_Technology  rtMidi, a cross-platform C++ library for MIDI  http://www.music.mcgill.ca/~gary/rtmidi/  Sonification  http://en.wikipedia.org/wiki/Sonification 14
  • 15. Thank you for your attention :-) 15
  • 17. OscilloscopeAppC - overview  OscilloscopeAppC is the TinyOS application running on sensor nodes  Receives commands from the sink: start/stop/change_freq  Samples the on-board tri-axis accelerometer using three ADC channels in DMA mode  When a block of samples has been collected, the sensor node sends it to the sink via radio 17
  • 20. BasestationC - overview  BasestationC is the TinyOS application running on the sink node  Acts as a bridge between the serial and the radio link  Implements queues in both direction to handle traffic spikes 20
  • 22. MIDI map module  Offers a simple GUI: the user can choose how to map the active data channels into MIDI messages from a predefined set of mappings 22  The module then catches incoming data, maps it into MIDI command according to user-defined preferences, and forwards them via TCP/IP socket toward the MIDI forwarder
  • 23. MIDI forwarder  Implements a virtual MIDI device: the WSN is seen as an input MIDI peripheral plugged into the computer  It then acts as a bridge:  Waits for incoming MIDI messages from a TCP/IP socket  Forwards MIDI messages to the virtual MIDI port 23