LAYERd

Assignment,Final Project,Hardware,Software — Tags: — John Mars @ 10:45 pm

LAYERd is a multi-layer display made from off-the shelf computer monitors. It allows for glasses-free 3D as well as a novel way to envision User Interfaces.

Every LCD screen on the planet is made of two main parts: a transparent assembly (made of laminated glass, liquid crystal, and polarizing filters) and a backlight. Without the backlight, an LCD monitor is actually completely transparent wherever a white pixel is drawn.

LAYERd uses three of these LCD assemblies illuminated by a single backlight to create a screen with real depth: objects drawn on the front display are physically in front of objects drawn on the rear ones.

My work is mainly focused on the potential UI uses for such a display: what can one do with discrete physical layers in space?

PROCESS

The process begins by disassembling the three monitors. After destroying two cheaper ones with static electricity before the project began in earnest, I was very careful to keep the delicate electronics grounded at all times, and I worked on top of an anti-static mat and used an anti-static wristband when possible.



With some careful prying, the whole thing came undone.



Here, you can see how the glass panel is transparent, and how the backlight illuminates it.

After disassembly came the design, laser cutting, and assembly of the frames and display.

Finally, the finished product.

It uses three networked Raspberry Pis to keep everything in sync, as well as the power supplies/drivers from the disassembled monitors.

LESSONS LEARNED

I learned a lot about polarization; mainly that about half of the films needed to be removed in order for light to pass through the assembly. Plus, this cool little trick with polarized light.

I also learned about safety/how monitors work: Alas! Disaster struck. I accidentally cut one of the ribbons while disassembling a monitor, which resulted in a single vertical stripe of dead pixels. Plus, my front display got smashed a little bit on the way to the gallery show, and made a black splotch.

RELATED WORK

One main body of research highly influenced my design and concept: the work being done at MIT Media Lab’s Camera Culture Group, notably their research in Compressive Light Field Photography, Polarization Fields, and Tensor Displays.

Their work uses a similar assembly of displays to mine, but is focused mainly on producing glasses-free 3D imagery by utilizing Light Fields and directional backlighting.

A few other groups of people have also done work in this field, namely Apple Inc., who has a few related patents — one for a Multilayer Display Device and one for a Multi-Dimensional Desktop environment.

On the industry side of things is PureDepth®, a company that produces MLDs® (Multi Layer Displays™). There isn’t much information in the popular media about them or their products, but it seems like they have a large number of patents and trademarks in the realm (over 90), and mainly produce their two-panel displays for slot and pachinko machines.

Another project from CMU is the Multi-Layered Display with Water Drops, that uses precisely synced water droplets and a projector to illuminate the “screens”.

REFERENCES

Chaudhri, I A, J O Louch, C Hynes, T W Bumgarner, and E S Peyton. 2014. “Multi-Dimensional Desktop.” Google Patents. www.google.com/patents/US8745535.

Lanman, Douglas, Gordon Wetzstein, Matthew Hirsch, Wolfgang Heidrich, and Ramesh Raskar. 2011. “Polarization Fields.” Proceedings of the 2011 SIGGRAPH Asia Conference on – SA ’11 30 (6). New York, New York, USA: ACM Press: 1. doi:10.1145/2024156.2024220.

Prema, Vijay, Gary Roberts, and BC Wuensche. 2006. “3D Visualisation Techniques for Multi-Layer DisplayTM Technology.” IVCNZ, 1–6. www.cs.auckland.ac.nz/~burkhard/Publications/IVCNZ06_PremaRobertsWuensche.pdf.

Wetzstein, Gordon, Douglas Lanman, Wolfgang Heidrich, and Ramesh Raskar. 2011. “Layered 3D.” ACM SIGGRAPH 2011 Papers on – SIGGRAPH ’11 1 (212). New York, New York, USA: ACM Press: 1. doi:10.1145/1964921.1964990.

Wetzstein, Gordon, Douglas Lanman, Matthew Hirsch, and Ramesh Raskar. 2012. “Tensor Displays: Compressive Light Field Synthesis Using Multilayer Displays with Directional Backlighting.” ACM Transactions on …. alumni.media.mit.edu/~dlanman/research/compressivedisplays/papers/Tensor_Displays.pdf.

Barnum, Peter C, and Srinivasa G Narasimhan. 2007. “A Multi-Layered Display with Water Drops.” www.cs.cmu.edu/~ILIM/projects/IL/waterDisplay2/papers/barnum10multi.pdf.

Lanman, Douglas, Matthew Hirsch, Yunhee Kim, and Ramesh Raskar. 2010. “Content-Adaptive Parallax Barriers.” ACM SIGGRAPH Asia 2010 Papers on – SIGGRAPH ASIA ’10 29 (6). New York, New York, USA: ACM Press: 1. doi:10.1145/1866158.1866164.

Mahowald, P H. 2011. “Multilayer Display Device.” Google Patents. www.google.com/patents/US20110175902.

Marwah, Kshitij, Gordon Wetzstein, Yosuke Bando, and Ramesh Raskar. 2013. “Compressive Light Field Photography Using Overcomplete Dictionaries and Optimized Projections.” ACM Transactions on Graphics 32 (4): 1. doi:10.1145/2461912.2461914.

This project was supported in part by funding from the Carnegie Mellon University Frank-Ratchye Fund For Art @ the Frontier.

Activate Yourself

Activate Yourself from Amy Friedman on Vimeo.

FullSizeRender

IMG_1584

IMG_1561

FullSizeRender (1)

 

CONCEPT/PROCESS:

Activate Yourself is a visualization aid, to understand muscle activity. Users follow on screen prompts to visually understand if their muscle of choice is being used during different motions they utilize. This is a being step to better understand our bodies and whether we “activate” ourselves during different activities the way we think we are.

My main interests involve body monitoring, and how information is conveyed to users. Many times we visualize data in a way that not everyone can understand, therefore our experience with data doesn’t add value to our everyday lives to change how we act or inform us about healthy activity. The notion of Muscle Activity can help understanding stroke victims ability to move during rehabilitation, trainers/athletes/kids to understand when they are using the muscles they want, and overall maximize training by understanding if your body is responding the way you believe it to be. Using Processing 2.0 I created the onscreen prompts and software. The software connected to the EMG Shield/Arduino through Firmata imports into Processing, and using the Firmata Code on the Arduino.

Using the Backyard Brains Electromyogram(EMG) Arduino Shield I was able to retrieve readable data that informed whether a muscle had been “activated” or used which someone was moving. The higher the analog read, the more the muscle was trying to be utilized through local electric muscle activity sent from the brain. I first began by testing out the different Backyard Brains experiments, such as Muscle Action Potentials (measuring the amount of activity), and Muscle Contraction and Fatigue. The latter is what inspired my original path to further understand our bodies.

We currently visualize signals through sin waves, but is there a better way to visualize this information. I then tried to utilize the EMG to detect muscle activity to determine if a muscle is fatigued, not active, active, and rested. This information could optimize working out and lifting. A wearable versatile device that could be worn on any muscle group with haptic or LED feedback would be the idealized version of this project.

I first began by reading how EMGs measure information, can muscle fatigue be recognized, how would I even do this? I read the following articles:

Sakurai, T. ; Toda, M. ; Sakurazawa, S. ; Akita, J. ; Kondo, K. ; Nakamura, Y. “Detection of Muscle Fatigue by the Surface Electromyogram and its Application.” 9th IEEE/ACIS International Conference on Computer and Information Science, 43-47 (2010).

Subasi, A., Kemal Kiymik, M. “Muscle Fatigue Detection in EMG Using Time-Frequency Methods, ICA and Neural Networks”  J Med Syst 34:777-85 (2010).

Reaz, M.B.I., Hussain, M.S., Mohd-Yasin, F. “Techniques of EMG signal analysis: detection, processing, classification and applications.” Biological Procedures Online 8: 11-35 (2006).

Saponas, T.S., Tan, D.S., Morris, D., Turner, J., Landay, J.A. “Making Muscle-Computer Interfaces More Practical.” CHI 2010, Atlanta, Georgia, USA.

I created my own analysis of my data using the procedures in the article:

Allison, G.T., Fujiwara, T. “The relationship between EMG median frequency and low frequency band amplitude changes at different levels of muscle capacity.” Clinical Biomechanics 17 (6):464-469 (July 2002).

I realized that in order to better understand the data I needed to filter it to get rid of external noise, and compare frequencies using Signal Processing, after filtering the data I could use Machine Learning or Neural Network tools to recognize patterns of fatigued, active, rested or not-active. With the help of Ali Momeni, CmuArtFab Machine Learning Patch and my resources above, I created a patch in Max MSP by filtering the signal from Arduino, but this wasn’t enough to be able to recognize the signal. The sample rate of my data is only 1024 while the samplerate for Audio is 44100Hz making my data very tiny when transformed using the Fast Fourier Transform(FFT) settings in Max MSP. It was recommended that I try to utilize Pd. I was able to filter the data, but as I am not rehearsed in Signal Processing methods it was unclear to me how to go about the next phases to utilize Neural Networks.

Screen Shot 2014-12-16 at 7.43.30 PM

Max Patch

 

Screen Shot 2014-11-17 at 6.32.19 AM

PureData Patch

At this point I refocused my project scope to help visualize muscle activity, or as the Backyard Brains experiment calls this “Muscle Action Potentials”. Using Processing 2.0, I created the “Activate Yourself” software which instructed users how to put on an EMG based muscle choice (the tricep, bicep or forearm), gave them on screen instructions and feedback for each timed activity, while showing their activity levels on a metered display. Creating the software took me time as it was hard to navigate between menus and I had trouble moving words while utilizing a timer. I spent too much time on making this work, that the end “AH HA” moment needed more attention. I spent time sketching out the interactions and how they should experience the timers.

IMG_1546

IMG_1547

IMG_1551 IMG_1552

For the Physical Visual Piece, I used Rhino to create a Shadow Box and 3d printed Lightbulbs using the Cube and FormLab printers. The FormLab printer was able to print the bulbs without any issues, while the Cube required supports and the Cubify software doesnt provide this type of additive support options as the PreForm software for Formlabs does. In order to print without supports on the Cube I made the sphere flat at the top, but there were issues with printing the neck of the bulb as there was so support for it to print over/connect to, making that area brittle.

IMG_1508IMG_1509

Tests to create design with Cube

I also learned that you can copy several of the same parts into a print to allow for it to print quicker using a Formlab printer, which sped up the process alot!

IMG_1517

Printing with FormLab Printer

IMG_1504

Printing with Cube

Connecting the NeoPixels to be insync with the Firmata code was hard and I am still figuring this part out better at the moment. I will post when I have this fixed!

LESSONS LEARNED:

1. I have low knowledge in Signal Processing, but it was a good start to my task of learning about wearable technology and has helped me focus on what I need to learn next semester

2. Creating the software took me time as it was hard to navigate between menus and I had trouble moving words while utilizing a timer. I spent too much time on making this work, that the end “AH HA” moment needed more attention.

3. I got to work with Max MSP and PureData which was a great opportunity, and although it was just beginning it was nice to work in both softwares and understand their basic setups better. I was previously overwhelmed by each.

4. Balancing between the physical components and software components was not easy as if one didnt work then you couldnt utilize the other.

RELATED WORK:

Athos – Athos is a wearable fitness shirt that utilizes a six-axis accelerometers, EMG to measure muscle effort, muscle target zones and muscle fatigue. Heart rate and breathing patterns are tracked to further enhance you overall performance, and this device determines your recovery rate to truly maximize your workout.

Mio Link- Heart Monitor with Bluetooth connectivity, acknowledges current training zone by color on the wristband. Images below is off of the Mio Website . It indicates the 5 heart rate zones that are tracked by the heart rate monitor.

Screen Shot 2014-10-26 at 6.14.54 PM

KibbleControl

Kibble Control! This is a pet bowl device that can connect to the internet! Sure, there are other bowls out there. Bowls that detect RFID. Bowls that schedule your cats feeding time. Bowls that connect to the internet and can update you on when your cat ate. Problem is, none of these bowls have the ability to accommodate multiple pets while being able to connect to the internet and allow for the pet owner to control exactly how much the cat should eat each meal and at what time. No other bowl understands whether one pet tends to bully another away from the food bowl and will update the owner over a connected web application. Other bowls that do connect over the internet do so via a phone app, which requires a smart phone to be used, and cannot be accessed via other devices.

Point is, we thought of [almost] everything! It’s a work in progress but we believe we are on to something here. We plan to continue to explore our options regarding this project because it was a lot of fun to work on, it benefits me personally to make this bowl the best it can be so at least I can use it in my home, and there does not seem to be any draw-backs to giving it a go when we have the time for it.

Horay for KibbleControl!

KibbleControl from Yeliz Karadayi on Vimeo.

 

 

IMAG0088

opened back

IMAG0089

closed back- locked in with magnets

IMAG0093

the mess inside

img3

img4

img9

img10

img11

img14

507 Mechanical Movements

Hardware,Reference — Tags: , , , , — epicjefferson @ 3:20 pm

507movements

I found out that the book is now in the public domain, so it’s free to download-

507 Mechanical Movements PDF

And here’s the site that has animated some of the movements.

507movements.com/

pd + OSC tutorial

Hardware,Software,Technique — Tags: , , , , , — epicjefferson @ 12:40 am

pdOSC

I made a quick tutorial on how to use OSC to communicate 2 devices running pd and use the [pduino] object to control each other’s leds and solenoids. yay!

github.com/epicjefferson/buttonOSC

“Dizzy, The Deceitful” by Patt Vira & Epic Jefferson

IMAG1084

Dizzy is a friendly looking teddy who actually watches your every move and makes everything it sees public.

Making use of the Raspberry Pi’s GPIO, we hooked up a PIR sensor to trigger a webcam capture event and publish the image to Tumblr.

patt-vira.tumblr.com/

 

Challenges

By far the most challenging part of this project was working with the API. Since it’s a lot easier to find examples of raspi projects written in python than in c++, we decided to use python. For example, the Tumblr API page has example code for python but not c++, adafruit has a great python tutorial for hooking up a PIR sensor to a Pi’s GPIO.

pytumblr is Tumblr’s official API client for python, but the instructions are unclear.

 

Python-to-tumblr Tutorial

epicjefferson.wordpress.com/2014/09/28/python-to-tumblr/

 

Source Code

github.com/epicjefferson/webcamToTumblr

 

spycam01

spycam

spycam03

 

Project 2 – Raspberio

Assignment,Hardware — Tags: — tdoyle @ 11:26 pm

For my second project, I created a controller reminiscent of the classic NES controller with three push buttons. This project aimed to expose us to aspects of hardware prototyping and the WiringPi library. In the end I created a demo “game” where the player could move Mario around the screen by moving left, right, and jumping.

(more…)

The Floor has a Voice

Assignment,Hardware,Project01,Software — Tags: — John Mars @ 12:58 am

I often find myself humming some made-up tune to the gentle whir of a room’s machinery in the background of my consciousness. What would happen if that whir became more pronounced, and the room started singing its own tune?

To accomplish this, I must do a few things:

1. Pick up the noise in a room with a microphone (the kind of which is undetermined)
2. Analyze the sound to determine the room’s base frequency. Continue analyzing that sound to determine if/when that frequency changes.
3. Create a never-ending tune from based upon the base frequency.
4. Send that tune into the room as unobtrusively as possible, to make it seem like the room itself is singing.

Mic

1. Pick up the noise in a room with a microphone (the kind of which is undetermined)

An [electret mic](https://en.wikipedia.org/wiki/Electret_microphone) is my microphone of choice in this case. The one I’m using from [Adafruit](https://www.adafruit.com/products/1063) is pretty good, and very easy to use. Sound has always been this mystical, mysterious thing, but over the past year or so, it’s all coming together – and it’s all a lot simpler than I was expecting.

2. Analyze the sound to determine the room’s base frequency. Continue analyzing that sound to determine if/when that frequency changes.

An FFT algorithm helps compute the amplitude of all frequencies of sound wave getting picked up by the microphone. The one I’m using splits the audible range into 64 bins of 75hz ranges each.

3. Create a never-ending tune from based upon the base frequency.

Via [OSC](https://en.wikipedia.org/wiki/Open_Sound_Control) I can send the FFT-derived base-frequency to a Raspberry Pi running [Pd-extended](http://puredata.info/downloads/pd-extended). With PD, tone generation is as simple as connecting a few nodes, and song generation is just a little bit more complicated than that.

A series of specific whole-number ratios multiplied by a frequency result in natural harmonies: for example, the base-frequency times five-fourths results in a Major Third above the base; fifteen-eights is a Major Seventh.

Using this knowledge in combination with a basic chord progression and a little randomness, I can create a never-ending song that perpetually realigns itself to the incoming frequency.

4. Send that tune into the room as unobtrusively as possible, to make it seem like the room itself is singing.

There isn’t much to show here, and that’s kind of the whole point. I’ve embedded my system into the ventilation vents in the floor below. A [surface transducer](https://www.adafruit.com/products/1784) (speaker without a cone) transfers the amplified music to highly reverberant metal air ducts.

Boards

Transducer

Smartphone EMF Detection

Assignment,Hardware,Project01,Software — Tags: , , — epicjefferson @ 10:36 pm

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
(c) 2017 Making Things Interactive | powered by WordPress with Barecity