Experiential Augmentation Visualisation in Augmented Reality

Project: Research Publication: Full paper CHI 2018 Publication: Work in progress TEI 2017 Thesis: CMU masters thesis

What does a floating USB drive mean?

This thesis looks at an alternative way to visualise information in augmented reality: By manipulating an object's physical characteristics, and using our learned understanding of these physical characteristics to create new meaning.

Video summary of concept

Visualisation on screens

On computer screens, much of the information we see are represented in symbols and icons.

Visualisation in augmented reality

Many visions of augmented reality would suggest we draw from 2d screen UI language to create interfaces for augmented reality.

But shouldn't the language of visualisation change when designing for 2d versus the real world? Does augmented reality have to look like a bunch of floating screens?

Floating screen AR future

Indexical visualisation

Indexical visualisation provides an alternative to the 2d UI vision of augmented reality.

Indexical visualizations are qualitative, and rely on our understanding of cause and effect to communicate information. For example the movement of tree leaves tells us how strong the wind is. Lights at night tell us about population distribution and where the borders of cities are.

Indexical visualisation of quantity

Another indexical visualisation is the way the pouch below sinks into the surface it sits on. The deformation of the surface tells us the pouch is heavy, probably filled with something.

The surface deformation is an indexical visualisation of quantity

Borrowing the meaning of indexical visualisations

What if we borrowed the understanding that "a sinking object is heavy/full" and applied it to USB drives?

Could we make a USB drive look like it is full, by creating the illusion it is sinking into the surface it sits on?

Sinking a USB drive into a surface to make it look full

Experiential Augmentation

Experiential augmentation is the borrowing of meaning from our understanding of indexical visualisations, and applying them to new objects, to visualise new information/states of that object.

For more information on experiments run, results and design recommendations please download the paper.

Following are other example tests from the research:

Fading out a phone show it is running out of battery

Floating a USB drive to show it is empty

Taking the color from an apple to show it is rotten

Early work in progress video

Send Availability
LinkedIn Messaging
Experiential Augmentation
AR Communication Design
Physical Code
Conversational UI
Tangible Interaction
Walled City
Voice/Sentiment Analysis
Speculative Design
Character Animation