One of my main interests is working with interfaces for sound synthesis. Over the years I’ve been experimenting with a few different techniques to see how different interfaces inspire different styles of performance and how the interface affects the sound produced. Without having a clear goal, I’ve delved into circuit bending/ hardware hacking, computer-vision, touch screens and web/text based systems. Here are some of my findings:
Circuit Bending
Elvis box – Circuit Bent DS-1 Distortion
Circuit Bending provides a great introduction to incorrect electronics, the idea is that you use wires to randomly create connections within existing circuits (using only battery power, for safety) and explore the effect these connections have on the sound (or visual). I think this wires the brain in a great way because you expect failure, instead of total control you have only curiosity and luck. This got me thinking about how I was going to control these sounds. Why had I decided to use buttons, patch cables and knobs?
Idea-
Create a data visualization of an everyday far away space phenomenon in physical reality.
Similar Projects that I found
LIGO ART
Cosmic Gravitational waves are detected(data is acquired) and showcased through LEDs. What I like about this project is an unseen or invisible phenomenon is given a physical form.
Moon Creeper: Sublime Gadgets by Auger-Loizeau(2012)
Apollo 11 mission left giant reflectors on the surface of moon. Reflectors can be illuminate by telescope laser beams from earth. The reflected beam is measured to calculate speed of receding moon from earth(3.8 cm per year). Moon Creeper is a visualization of receding moon. What I find interesting in this project is that a subtle motion is chosen and represented as is, and not accelerated in the visualization.
The device shows potentially hazardous astroids rotating around earth and shows warnings when their orbit matches with that of earth. Device Highlights that our lives our result of giant probabilistic equation. There are so many things that can go wrong and yet we go on living our lives as if we are immortal.
Academia
This paper by Stéphanie Fleck and Gilles Simon defines augmented reality platform for young children to learn astronomy. Every market corresponds a planet in the solar system. The augmented objects react based on their position and distance with respect to each other, real time.
For my final project, I am interested in working with the idea of serendipity and the kinds of interactions that can be built around this concept. Because of the sensors we have access to both in our cell-phones and in the prototyping tools we are using (RPi and Arduino) the opportunities for facilitating happy accidents opens up. We can use sensors to determine where people are, what they are doing, how active they are… We can connect with each other’s devices and exchange these data and based on it take actions. So I want to focus on these data, the interactions between devices and the way that serendipity can be prompted by them. Some examples I am looking at from different fields:
From Industry – Wakie, a social alarm clock: You set a time you want to wake up and when the time comes, some random person logged in the app who wants to wake someone up at the time you set will call you from the app. It works the other way too, anytime you sign into the app, it will tell you how many people would like to wake up around the World and you can just click on “wake someone up”
From Art – Serendipity is a project developed by Kyle McDonald while in residency in Spotify. The project detects when two strangers in the world click on play at the same exact time on a song on Spotify and then shows the locations on a map. About once a second the song being shown changes to show a new song simultaneously started in two places around the World.
From Academia – 20 Day Stranger: A project by the MIT Media Playful Systems Group, for a period of 20 days, it connected pairs of strangers in the world and had them share their day to day experience with each other.
For the direction I want to head in for the final project I’ve chosen wearables. Specifically I want to get into emotion and how we can communicate emotions, feelings, and sensations through devices that are meant to be worn. Below are several examples from academia, industry, and art that show some of these concepts.
Apple Watch (Industry):
The biggest news out of the commercial wearables market this Fall was Apple’s Watch. There are many cool aspects to the device, but I the ones I find most inspiring are the sharing features (mentioned towards the video). Via the watch you can send drawings or messages to another watch as well as your heartbeat. I think the latter is a really interesting concept since there is such a personal and affection connection to the concept of one’s heartbeat. Additionally the device uses a series of haptic sensors to give the user feedback which I think is critical for a wearable.
InTouch is a system that uses haptic feedback to give users a physical interaction with digital content. In this application, the system allows users to manipulate 3D surfaces and meshes with haptic feedback to improve the experience of the application. The main concept for this research that I like is the interaction between the digital and the physical where the developers are using haptic to bridge the gap and translate the digital interaction into a more relatable and manipulable physical sensation.
Synaptic Traces (Art):
Finally, in the realm of fashion and wearable technology comes the project called Synaptic Traces. The frock is made of a beautiful galaxy-like print that is curled into stripes and piece together. Inside of the garment are many LEDs that allow the garment to glow. When someone touches the garment, lights activate that represent the touch and hold for longer than a normal sensation would. In essence is holds the fleeting feelings and sensation that we get from another person or object and keeps it a bit longer in a digital representation. Other than the beautiful physical construction, I really liked the concept of preserving the intimate touches and sensations we feel when interacting with another person.
I am really interested in wearables and I am intrigued by the work of Anouk Wipprecht. Specifically her intimacy dress, which uses smart e-foils, which becomes transparent when you run a current through it.
I am also interested in projected images but I haven’t thoroughly thought through how to combine the electric foil with the projected image. Here is one of the projected images I found interesting:
I found this article to be helpful as I want to utilize computers as solutions into my final project. I am unsure of how this will occur, whether it be a developed part of the research how to utilize user interfaces or part of the final product. The research team designed software specifically for those who possess motor issues. In this study they dealt with children in Uruguay who have Cerebral palsey. In the academic article it states that “According to the teachers, the screen layout of this activity is too compact and the controls are not enough different one from the other.” (p.249). Feedback such as this will be important for a devices success to manipulate the prototype.
Industry: “Magic Arms” by Tariq Rahman, & Nemours/Alfred I. duPont Hospital for Children (2012)
Emma has Arthrogryposis Multiplex Congenita a disorder in which the joints are grown abnormally.The technology used to create the customized device was a Stratasys 3D printer. The Wilmington Robotic Exoskeleton was already created by the Nemours Biomedical Research, but not to an adequate size for a child. The researchers created casts and molds of the child to understand what her dimensions were and how devices could be adapted for a small child. I would like to utilize the idea of customization 3d printing to be able to create a device to help those with mobility issues. User centered research is another important aspect of the video which I would like to utilize, without patient feedback I wont be able to create a product with a purpose.
Stelarc created this performance piece as an extension of his body and performed with it from 1980 until 1998 traveling to Japan, the USA, Australia and Europe. According to the website of Stelarc (found above) “The Third Hand has come to stand for a body of work that explored intimate interface of technology and prosthetic augmentation- not as a replacement but rather as an addition to the body.” I found this piece to be intriguing as it utilizes mechanics to create a third arm. With those who are vision or motor impaired an extension of the body can allow for them to manipulate objects in a way they were unable to do previously. I think utilizing the ideas of addition as stated on Stelarcs website will allow for an integrative experience as you arent trying to fix what is already existing rather add to what you have.
I will be working towards developing a product that will again users with motor control issues. The most important aspect of this project is to find a participant to help me throughout the process. I dont want to just create a project with user-research. My goal is to have a working prototype to be shown off in December which outlines the journey of adapting the device to the users needs. I have already received the Frank-Ratchye Fund for Art @ the Frontier grant from the Studio for Creative Inquiry to hack a 3D printer to understand the benefits of rapid prototyping and how we can create customizable parts.
Industry: Time lapse photography is often used by high end developers and contractors to document a project. This can be shown to future or current clients.
Two interesting commercial venues for time lapse include construction time lapse camera’s such as Brinno’s and time lapse services such as The Time-Lapse Company’s. The former caters to mid range contractors while the later targets high end developers.
The scope of projects and the subject of the projects documented using construction cameras is fascinating, however, there is little or no interaction or relationship between the time lapse video and the construction project. As well, Brinno advertises “instant video,” however, instant here indicates that every day’s photographs are automatically compiled into a video. I am interested in real time updates and compilation. This offers the possibility for interaction with the time lapse video, perhaps on a scale of time that people don’t usually interact with video cameras on.
Art: The avant-garde theater troupe, The Wooster Group, plays with the kind of back and forth between recorded and live video that I am interested in. In this section from Hamlet the actors recreate a movie of Hamlet which the have memorized every movement of. Simultaneously a camera is trained at the actors. This causes the viewer to think of memory and time and their interaction through video. With the sort of live video feed I am interested in creating, these sorts of issues would be brought up, however, the viewer would get a chance not just to observe, but to interact with the video recording.
Academia: Another aspect of live video feed time-lapse would be the possibility to interact and see the results of one’s actions over a long period of time. This is something which The Long Now organization is trying to do; they are trying to shift our temporal focus from the short, to the long term. It’s an interesting problem when so much media today is focused on increasingly short time intervals; short cuts.
Open Controllers allows you to use your daily device as a controller of a different kind by placing it within specifically designed artifacts that embed the device and utilize its capabilities such as the gyroscope and the light detection. This project takes your physical movements and interprets them. I think this can be done with a range of patterns, which gets into the next video… More…
“Tilt Brush” by Skillman & Hackett (2014)
This is a lot more similar to what I want to to, except that I also am interested in the painting becoming three-dimensionalized as you are painting. Not that I intend on doing that exact thing…I just mean to draw a parallel. The gestures here looks swift and intuitive if it’s real, and the interface is quite honestly very confusing but maybe that’s because I’m not aware of the gestures it uses. Assuming this is true, I think this is the future of design through tangible interaction.
“3Doodler” by Pete and Max (2013)
This project does a lot of what I am hoping to accomplish with its ability to prototype quickly and efficiently without having to prepare ahead of time what will be made.
“3D Printed Blooming Flowers” by Mikaela Holmes (2014)
This kind of movement is exciting as a less mechanical way to actuate an artifact. I wonder how it could be applied at a more complex or responsive level.
I think these videos show the progression of my thoughts on the final project quite well. I want to be able to design something indirectly by using a tool, not unlike how surgeons now use robots to do the actual surgery. I am struggling to know the scope I am capable of, and coming up with a good project idea to make it happen. Something I had thought about is using my surface as a tool to communicate with a CNC robot to fabricate sketches drawn in real time on the surface.
Quotidian Record is a custom-made vinyl record, featuring Brian’s location-tracking data over a year. The data is mapped to a harmonic relationship from 365 days down to approximately 11 minutes. This project connects digital information to the physical world though music. I find it a very unique and interesting way to visualize the data that can be easily forgotten and make it tangible and memorable.
I am still deciding on the type of data I’d like to collect, and that would also determine the tool I use. Fitbit is definitely on top of the list. One idea is to track the amount of sleep I get each night, and how that affects my daily activities the following day.
Academia: “Understanding Quantified-Selfers’ Practices in Collecting and Exploring Personal Data” by Eun Kyoung Choe, Nicole B. Lee, Bongshin Lee, Wanda Pratt, and Julie A. Kientz (2014)
Even though everyone tracks something about themselves, this paper looks into the “extreme users” – the quanified-selfers, who diligently collect many different kinds of data about themselves. Through Meetup groups, blogging, and other outlets, they have shared their experiences of what they have found useful and the mistakes they recommend to avoid. This paper will help me narrow down the data I should be collecting, thinking about how the data would be beneficial or appropriate for this project.
Goal:
Since I am looking to work with my personal data that is collected over time, the most appropriate outlet would be to be published on a few notable design blogs. However, my ultimate goal for this project is to learn a little bit more about myself through making this data permanent and tangible.
