Lens × Block

World as Construction Kit

 

by

Jay Saul Silver

B. Electrical Engineering, Georgia Tech (2002)

M. Phil. Internet Technology, Cambridge (2003)

M. Media Tech., M.I.T. (2008)

 

Submitted to the Program in Media Arts and Sciences

School of Architecture and Planning

In Partial Fulfillment of the Requirements for the Degree of

 

Doctor of Philosophy

 

at the

 

MASSACHUSETTS INSTITUTE OF TECHNOLOGY

 

September 2014

 

© 2014 Massachusetts Institute of Technology. All rights reserved.

 

 

 

 

 

 

Lens “By the means of Telescopes, there is nothing so far distant but may be represented to our view; and by the help of Microscopes, there is nothing so small, as to escape our inquiry; hence there is a new visible World discovered to the understanding.” ~Hooke

Block “The smooth shapely maple blocks with which to build, the sense of which never afterwards leaves the fingers: so form became feeling. And the box with a mast to set upon it, on which to hang with string the maple cubes and spheres and triangles, revolving them to discover subordinate forms." ~Frank Lloyd Wright

Discovery can be defined as the process of re-seeing the world we live in as Hooke proposes. Discovery can be considered as building something as Wright suggests. What if we do both simultaneously, taking a lens and combining it with a block?

 

Abstract

Purpose and meaning of the physical world can be re-assigned and re-made by individuals as they go rather than being pre-fixed by people who came before them. But this mindset is more rare than it should be if we want an empowered population full of creative powerful beings. So can we make special tools that by design help people to put into practice the mindset and actionable behavior that: The World is a Construction Kit? We can, and in fact people have already done so with some existing tools which I will present. Then, I will present several new digital construction kits with a focus on two, Drawdio and Makey Makey, that are designed to focus attention on the world as the construction kit. Rather than combining kit-parts that come in a box, participants take pieces of the world they live in and re-purpose and re-combine these everyday objects from their life.

 

I formalize this type of construction kit, explaining how it takes the constructive aspect of a traditional wooden block, and the world-transforming multiplicative aspect of the traditional looking glass lens, to make a block-and-lens-in-one, which I call a Constructive Lens. I consider traditional construction kits like LEGO, or kits that aren't necessarily thought of as "construction kits" per se, like Painting Kits: Brush/Paint/Canvas, and show how to transform these traditional construction kits, which offer their own pre-fixed components, to the realm where the world, that is the everyday objects in one’s life, is instead acting as the components of the kit.

 

The ultimate goal of the thesis is to show how we can we make tools and activities, “Constructive Lenses,” that, by design, catalyze:

 

re-seeing (lens) the everyday world as something we can re-make (block)

 

The thesis approaches this goal through a rich narrative with thick description of design studies and case studies, intended to experientially model the process of motivating, making, and deploying Constructive Lenses to hundreds of thousands of people.

 

 

 

 

 

                          

 

Constructive Lens

 

 

 

 

 

Thesis Reader

...................................................................................................................................................................

Mitchel Resnick

LEGO Papert Professor of Learning Research

Academic Head, Program in Media Arts and Sciences

 

 

Thesis Reader

...................................................................................................................................................................

Neri Oxman

Associate Professor of Media Arts and Sciences, MIT

 

 

Thesis Reader

...................................................................................................................................................................

Bakhtiar Mikhak

Lecturer and Research Advisor, Digital Media Arts and Instructional Design, Harvard University

 

 

Outline

Summary                                                                                                                                 

I. Introduction -----------------------------------------------------------------------------------------------------------------------------------------

1.     Motivation                                                                                                                     

a.     Re-Seeing (story about Nature Awareness)

b.    Re-Making (story about Staying with Natives)

c.     Coalescing of Thesis Ideas and Research Questions

2.     Background                                                                                                                   

a.     The success and failure of Camera for the Invisible

b.    Intellectual Traditions Drawn on and “World as Palette” Extension

c.     Looking Ahead (Drawdio and MaKey MaKey explained briefly)

3.     Contributions                                                                                                                 

a.     Lenses and Blocks

b.    Constructive Lenses

c.     World as Construction Kit

II. Design ----------------------------------------------------------------------------------------------------------------------------------------------

4.     Constructive Lenses in the World                                                                                    

a.     Open Lego

b.    Open Construction Kits

c.     Some Existing Constructive Lenses

5.     Drawdio Genesis                                                                                                            

a.     Nature Sensing Adventure

b.    Electronics Hacking in India

c.     The Making of an Invention Kit at MIT

6.     Makey Makey Genesis                                                                                                   

a.     Computer Crafting

b.    App Dumpster Diving

c.     Prototypes

III. Workshops and Discussion ------------------------------------------------------------------------------------------------------------------

7.     Drawdio Case Studies                                                                                                    

a.     Workshops in a Camp and a Museum

b.    Workshops at MIT

c.     Workshops in Silicon Valley 

8.     Makey Makey Case Studies                                                                                            

a.     Workshop at a Conference

b.    Workshops at Maker Faires

c.     “Workshops” in the Wild

9.     Discussion                                                                                                                    

a.     Tricks and Design Guidelines for Making Constructive Lenses

b.    Sample Project Spaces

c.     Conclusion

Appendix

Thanks and Gratitude

Bio

 

 

 

 

The mindset that the everyday world is malleable and I can take creative action on and with it is crucial to solving future unknown problems [Louv, 2008, Last Child in the Woods] and to being able to playfully reimagine one’s world and oneself [Silver, 2014, This Thesis Just Now Above this Line].

 

 

But what type of learning tools can be designed to directly catalyze such a world view?

 

 

 

 

As a teenager I was reading some literature reviews of Thoreau’s writing, and someone, maybe it was Walt Whitman, said (paraphrased, as even the internet can’t help me find this one): “Thoreau’s writing has the quality of a good country western, he rides in on a horse and dust is flying everywhere, the rider shoots bullets in all directions, some hit their target, some miss, and some deflect and hit the rider, but what a show!”

 

The direction to which I strive in my writing style in this thesis shall be exactly as described. I want to put on a fantastic show, and risk death! Here it goes...

 

 

 

Although the MIT Registrar’s office has rejected the concept formally, let it be known here that this thesis was written in collaboration with all beings, and specifically the co-author M. Nature, and for philosophical purposes I share credit and copyright and my PhD degree with M. Nature.

 

 

 

Summary

 

The Challenge

I am inspired by natives’ abilities to see the world they live in as a construction kit: leaves, rocks, plants, are building blocks. What types of tools can we design to catalyze this action in our modern largely human-made environment? Traditional lenses, like magnifying glasses, make it more likely that we can see things we couldn’t otherwise easily see. Traditional blocks, like wooden blocks, catalyze us to build structures, like castles, without which we would not as easily build. What is a metaphorical “lens” that helps us to see the pieces of the world we live in, pencils, houseplants, old cans, as if those things were metaphorical “building blocks” ready to be rearranged and remixed into new creations? What hardware or software, set of instructions, or other product or information or ritual, can we offer to human beings, that would catalyze a behavior of repurposing and remixing the everyday objects they live with to give those objects new meaning and new purpose, different from their original meaning and purpose? Just re-seeing is not enough for me, it is re-seeing the world as something that can be re-made that is the goal. This thesis addresses the challenge of:

 

designing tools that catalyze the direct action of re-making the everyday world

 

Example: Tatt-Blue

Tools that allow you to see your fingerprints or see the tiny molecules inside of a blueberry are just lenses; cool as those things may be, they don’t systematically lead to direct creative action. Tools that result in you discovering your fingerprints and then deciding to dip them in smushed up blueberries and make an artistic image by stamping your finger print out onto your friend’s face, are Constructive Lenses: you have seen your fingerprint as a stamping machine, you have seen blueberry as paint, and used them both to make the world a different place. You have taken fingerprints, you have taken blueberries, neither of whose explicit purpose is to create a tattoo-mask for your friend, and you have given them new purpose and made a tattoo-mask on your friend’s face.

 

Example: Trash Tower

To build a tower out of blocks is to take creative action on the construction kit which is included in the box. To build a tower out of soda cans in the recycling bin is to see trash and soda cans in a new way, as if they were building blocks, and to take action on them to make something new.

 

Constructive Lenses

The soda cans are not pre-designated as a construction kit, they are the everyday world being repurposed to become a construction kit. The blueberries and fingerprints are not a construction kit in a box, they are the everyday world we live in being repurposed to become a construction kit. To create this new type of seeing is to catalyze the seeing of the World as Construction Kit. In doing so, the world starts to look reconfigurable, less fixed, more possibilitized. Purpose becomes internalized and re-determinable, fluid and changeable, not something cast down upon us by some official third party. Locus of control migrates more toward the internal, creative confidence grows with direct experience of making the world different, first small, then bigger. This is the goal of the “Constructive Lens” which is the result of crossing the Block with the Lens.

 

The thesis will go into great detail analyzing the effects of releasing two digital Constructive Lenses that I made into the wild: Drawdio and Makey Makey. By analyzing seven workshops, and looking at some cases of use in the wild, we will see specifically what types of re-making and re-purposing of the everyday world can happen using these two tools. Then I will offer folk wisdom and design guidelines for creating other types of Constructive Lenses with a special analysis of the role of the Sample Project Space.

 

 

 

I. Introduction

 

1. Motivation

 

 

 

Re-seeing (story about Nature Awareness)

 

Ten years ago, I went on a nature walk at Kenburn Orchards in Western Massachusetts. It was led by a 19-year-old named Connor. We made fire with sticks. I thought why are we doing this when we already know how to make fire with technology. He showed us some bird calls. He really cared about talking to the birds, and some of them answered back. I did not understand what the big deal was. What’s the use in talking to birds? Then when we were walking on the trail he stopped me and pointed to the ground. Jay, you’re not getting this Nature awareness thing, I seemed to hear him say subverbally, as he said, “I’ll be back in 5 minutes. When I come back, tell me -- what do you see?” I crouched down low and looked at the mud on the ground that he was pointing to. I thought, well, I think it’s mud. But there must be something more to it. Maybe he’s playing a trick on me and it’s actually poop. Or maybe I should look really close ‘cause there are some bugs down there. I kept looking and looking and he came back and I felt confused and silly and I blurted out, “Is it mud?”

 

He put his finger to the ground and traced a shape on the ground... and then... I saw it! It was an animal foot print. A deer hoof he said. I spotted another, and soon another. I started to see lots of deer hooves. And then lots of animal prints of other sorts, birds and other mammals that I couldn’t identify. But until he opened my eyes -- the prints were totally invisible.

 

I started to wonder, “What else am I not seeing that is right in front of me?”

 

I could stop right here, because this simple question, “What am I not seeing in my everyday life?” Is enough to keep me fascinated forever.

 

So I left the Nature walk and went back to Boston, but I wanted to learn more about seeing the invisible. I wanted to apply what I had learned in nature to the reality of my urban life in Boston.

 

 

Re-Making (Story about Staying with Natives)

 

A year later, I found myself staying with some Ngäbe (known to me at the time as Guaymí) natives in Costa Rica. I watched and worked with them for many days and they seemed to have an ability to see right through things. They saw the waterproof-ness and shape of the leaves and chose them carefully to put together their roofs. They made medicine out of other leaves by smushing them up and combining them with something from the ground, not sure what.

 

They knew which fibers were in different leaves that they could take out of the leaves and roll into strings and then weave bags. Take this palm frond apart and peel little threads off of it and then she would roll the threads together and make little thicker threads like strings and she would weave the strings together. As the materiality of this exact very bag formed before my eyes over those 3 days, the materiality of the way the world works, of reality, started to unravel in my mind.

 

Figure 1.1. The author, Jay Silver, holding a Ngäbe made bag which he watched a Ngäbe woman make over 3 days from nothing but naturally occurring materials in her environment. (Image taken by TED videographer)

 

Because I realized that this bag, and your trampoline, and your clothes, and your pencil sharpener, everything you have is made out of either a tree or a rock or something you dug out of the ground and did some process to, maybe a more complicated one, but still everything was made that way. So I had to start studying who is making these decisions? who is making these things that our world consists of? how do they make them? what stops people from making them?

 

I realized that they weren’t just seeing the invisible, but they were making things out of the invisible. Making changes to the environment based on what they were seeing. They were re-making.

 

I returned to Boston again, and this time I started to try to see through the concrete in my environment just like the plants in the rain forest. What is inside of the sidewalk? Why do blades of grass grow through some cracks or are they causing the cracks? And, most importantly, how can I re-make my modern urban environment?

 

I started trying to see the invisible, and to turn it into something constructive. I joined a dumpster diving team. We called dumpsters “Urban Gardens” and we called dumpster diving “Urban harvesting.” We would go out and collect the night’s harvest, naked juice, bread, almonds, vegetables, etc. in a biodiesel farm truck that ran on reclaimed oil. Back in those days we got the used vegetable oil from a little restaurant called “the middle east” so the joke was that we got our oil from the middle east. We ran a community supported agriculture model of dumpster diving, which means we let people pay $3 for a box before we went out, then we’d split the harvest equally when we returned. We’d bring the food home and each person would get enough food for about one week.

 

I started to see harvestable food everywhere. I had done it! I had learned to re-see, and not only that, I could re-purpose “trash” to “food” and recreate my world.

 

 

 

Coalescing of Thesis and Research Questions

 

Starting 8 years ago, I began studying how people re-see the invisible in the modern urban environment and then re-make it. Over that time I introduced several tools into people’s hands and watched how it affected their ability to re-see and re-make the world they live in. The rest of the thesis tells the story with my ups and downs, with a narrative that is based primarily around the development of 3 of these technologies:

- Camera for the Invisible

- Drawdio

- MaKey MaKey

 

The thesis discusses how these 3 tool-based experiences influenced the actions people took and the way people looked at the world. In the next section I’ll introduce a name for the type of tool I’m aiming to create; it will be called a Constructive Lens. A Constructive Lens, as I’ll explain, is designed to help people re-see (lens) and re-make (constructive) the everyday world they live in, both at the same time.

 

No thesis is a straight path, and many things happened that influenced my thought patterns toward focusing in on this thesis topic. I will now present 3 windows into my thought patterns that are meant to be representative of how I was thinking about

1) re-seeing,

2) re-making, and

3) theories around learning

This will allow the reader to glimpse into various resonant thoughts I had over and over again throughout the last 8 years and understand better where I am coming from and where I am motivated to head towards. This is how the thesis actually evolved in my mind nonlinearly.

Re-Seeing

After learning about the power of Nature Awareness, I started to learn more about the “Beginner’s Mind” or how to see as if seeing for the first time. I started watching young and inexperienced people approach things, and I interviewed a number of Nature Awareness experts. Here is my favorite quote from the interviews, this quote comes from Erik Plakanis who is a professional Nature Awareness Guide at “A Walk in the Woods,” as well as some photo outtakes from a video of some of my favorite moments in trying to see with a beginner’s mind.

 

“You can read about it, you can see it on TV, you can hear about it, you can be lectured about it. But when you can take a stethoscope in the spring, and put it on the outer bark of a tree, and you can hear the pulse of that sap being pulled up, you know water being pulled up from the ground, that sap pumping up rhythmically up until the branches until the buds unfold.”   I would also simply observe beginners of all sorts looking at things for the first time, including adults, but more often babies and children.   Here is my son interacting with snow and then pollen for the first time ever:   It’s inspiring to me to both watch his reactions and try to imagine his mindstate: “What is this for? What does it feel like? What can I do with it? What happens when I do this with it? Why does it move and feel like that? What is it made of? Where did it come from? In the mind of a beginner, anything is possible.

Window Into My Thoughts 1.1. An excerpt from an interview I conducted with a Nature Awareness Guide and some photos of my son experiencing various phenomena for the first time ever.

 

 

Re-Making

 

Similarly to the above I would look out in my everyday life for examples of ways I could take re-seeing to a new level and turn it into re-making and re-purposing:

 

 

I was inspired by the artist Andy Goldsworthy:   (images reproduced from Andy Goldsworthy)     The image on the left is a photo Andy Goldsworthy took of a bunch of leaves. Of course they didn’t just happen that way. Andy goes into nature with nothing but his hands and clothes and a camera and rearranges only what he finds to make some new aesthetic statement. On the left, he sorted the leaves by hue very carefully to give the appearance of a perfect hue fade. That was a photo I took out of one of his books. On the right is a freeze frame from his movie “Rivers and Tides.” He comes onto the beach and breaks icicles from the rocks. He finds a stone that is bowl shaped and fills it with sea water. All this before the sun rises. Then he melts an icicle with the warmth of his breath, wets the edge of it with seawater, and then holds it onto his structure so it refreezes into the location he wants to (the air is well under 32 degrees). As he is building it’s a race against the clock because when the sun rises the whole sculpture starts to melt. Seen above is his third attempt on the third morning. Each time when he fails he is so disappointed it’s beautiful. So on this third attempt he gets the whole icicle sculpture modeling the shape of a river swirling through a stone built just as the sun is coming up. The icicles in this scene are backlit by the sun. He finishes, and then watches happily as the sun melts away the successful sculpture.   I wondered if you could take Andy’s ideas and have teens work with it. Can everyone walk into nature and repurpose what they find? I took a group of teens at a camp called “Not Back to School Camp” into the woods by a stream.     I showed them some Andy Goldsworthy art books, and set them off. First I said to take 5 minutes and form some simple geometry out of what you find. Here’s what they did:       On the left is an oak leaf shape made out of other oak leaves, a sort of fractal meta-oak-leaf. In the middle is a leaf tied to a stick with a blade of grass. On the right is a triangle in progress being made out of sticks underneath a flowing stream.   Then we launched into full project mode for one hour to make something beautiful. Here’s what happened after 1 hour:   On the left is a wavy river-like shape made out of the negative space of some leaves. Top left is a wreath with a hue fade made by sorting leaves. On the top right is some stark contrast provided by birch bark striped across decaying matter. Bottom right is a mushroom with sticks and leaves stuck into it. And bottom left is called “Fire” and is a bunch of sticks and leaves stuck in between two trees. Someone asked “how did you get the sticks to hold up?” The creator answered “I don’t know but I can show you after.” The reply represents a sort of bodily and material intuition that can’t yet be verbalized. To not know is to leave space for possibility. To do things the way you are supposed to is to suppose that other ways are not possible.   I tried in my own life to see if I could find little moments to repurpose my own “modern nature,” the stuff of my everyday surroundings from animals and plants all the way to even human-made artifacts and structures.   One day at Whole Foods in Cambridge I was going up an escalator and dropped an M&M:     On the left you can see the M&M it just rolled and rolled against the little forks at the top of the escalator as wave after wave of metallic stair pushed it along but the forks held it back from going over the bump. While this could be viewed as an accident and time to move along, I was thinking “what is this escalator really? Maybe it’s an M&M surf board. What else can I put down on this surfboard?” And I threw down what I had in my pockets, and that’s why you see the chapstick over on the right, a beautiful cylindrical body spinning to keep up with the speed of its new chapstick-treadmill.   Out in the yard one day there were some flies and we were running through them. They were the kind that hover in place and when you run through they move out of your way: My son was screaming and I noticed that the flies were moving whenever he would scream. We all started “voice controlling” the flies. The frame on the right is from a video I took of them responding to my voice. The frame on the left is after I decided this was a moment to see what those flies could be. I took out a portable speaker and played them a Black Eyed Peas song from my phone. The flies moved to the beat and became a form of graphics equaliser display as they would go up with rising pitches, down with lowering pitches and speed up or slow down in proportion to how high or low pitch the sounds were, and of course they would respond through time as the sounds came out.

Window Into My Thoughts 1.2. Examples of experimenting with re-making the world in my everyday life

 

 

Theories around Learning

 

These ideas about re-seeing and re-making were overlaid upon a landscape of reading about learning theorists. The theories of my favorite authors mixed with my own theories led me to have daydreaming sessions about human potential. Here’s one of the thought patterns influenced by learning theorist Ivan Illich, one of 8 or more important “learning saints” in my personal cosmology, with more of the “learning saints” listed at the bottom:

Thoughts on Malleability and Modernized Poverty Is the world malleable? If so, whose hands are bestowed with the magic of forming and reforming the world’s parts and their relationships? What process did those people go through in order to have the special power of creative action? How does the perceived malleability of the world affect one’s own concept of the malleability of mind, thoughts, and beliefs?   There have always been obstacles to taking creative action (Banaji, Cranmer, & Perrotta, 2010) on the world, reforming it, repurposing it, and bringing meaning and inspiration from inside the mind to manifestation outside. There are problems of skill, confidence, motivation, etc. These are timeless. The same industrialized modern situation that leads to the proliferation of available creation materials is also leading to an even larger proliferation of pre-created solutions, dulling the capacity for everyday innovation.   Ivan Illich has a name for the inability to act creatively in the industrialized world: Modernized Poverty (Illich, 1971, DeSchooling Society). He defines it explicitly   Modernized poverty appears when the intensity of market dependence reaches a certain threshold. Subjectively, it is the experience of frustrating affluence which occurs in persons mutilated by their overwhelming reliance on the riches of industrial productivity. Simply, it deprives those affected by it of their freedom and power to act autonomously, to live creatively; it confines them to survival through being plugged into market relations.   We need a way of simultaneously acting-and-seeing which bypasses our industrialized affluence and renews our sense of self-reliant unadulterated creative capacity.   Here are other Radical Learning Philosophers that have motivated my work:

Window Into My Thoughts 1.3. An example of my underlying motivations fueled by revolutionary learning philosophers. From left to right: Eleanor Duckworth, Rudolf Steiner, Ivan Illich, Maria Montessori, John Taylor Gatto, Paulo Freire, Grace Llewellyn, Larry Harvey.

 

 

Research Questions

 

Based on my 8 years of experimenting and daydreaming, I started asking these questions which are now the central questions posed in this thesis:

 

 

a) What are some examples of Constructive Lenses and their applications?

There is currently no wide analysis of Constructive Lenses. I will introduce two well-played out, solid examples of such a tool

- in many different situations

- over a long time

- with many people

- both in the wild and in facilitated workshops

I will discuss not just the technical features of the two tools, but also the historical genesis in thick description [Geertz, In the Interpretation of Cultures 1973, Thick Description: Toward an Interpretive Theory of Culture]. I will also discuss other existing tools, some made by myself and collaborators and some made by others, that could be called Constructive Lenses, and how to change existing tools to become Constructive Lenses.

 

b) What kind of re-purposing can people do when you put Constructive Lenses in their hands?

 

How do Constructive Lenses affect the workflow and re-purposing behaviors, as well as the thinking and learning, of people like Beginners? Experts? Academics? Professionals? What do these people do with Constructive Lenses out in the wild and in workshops? I will run and display the results of 9 workshops and show results of use in the wild including interviews and photos.

 

c) What are the design principles to guide the development of Constructive Lenses?

Are there principles that help tie together the design of all these different tools? What design approaches are the most successful historically at generating tools that are as close as possible to being Constructive Lenses? What intentions are likely to lead to good Constructive Lens designs? If others were to try to design Constructive Lenses, what set of wisdom should they keep in mind? I will offer dozens of tips and generative constructs to help others and my future self form Constructive Lenses. Hopefully the advice is applicable to design in general but especially creative tools and specifically Constructive Lenses.

 

 

 

 

 

 

2. Background

 

The Success and Failure of Camera for the Invisible

On my quest to learn how other people catalyze and teach re-seeing and re-making the world they live in, I found out about professor Anne Spirn, who was researching the hidden urban nature in modern society with her students. Here is the cover of her book:

 

                   

Figure.2.1. Left. Anne Spirn’s book cover “Granite Garden.” Right. My markup of Anne Spirn’s book cover to show one macro perspective on how the city we live in can be thought of as urban nature. (Image reproduced from book cover of Granite Garden by Anne Spirn)

 

Anne’s students’ use regular cameras as one of their main tools. I interviewed a number of students.

 

One of her students had drawn this diagram.

 

Figure 2.2. Map of wind near the intersection of Mass Ave. and Marlborough St. in Boston. (Image reproduced from project by Anne Sprin’s student)

 

This diagram was made by the student sticking a licked finger in the air and measuring the wind speed and direction at each point on the street intersection and inward into the courtyard. But what was at the circle with the “X” in it? This:

 

Figure 2.3. Picture of a pile of debris in the courtyard of the intersection in Figure 2.2 at the circle with the “X” in it. (image reproduced from project by Anne Spirn’s student)

 

Ahah! Nothing! And Everything! At the same time. A pile of debris. What a powerful map and photograph. I thought, what if we could take “pictures” of invisible phenomena (like wind) with a special “camera” and explore the hidden dimensions of the urban natural environment systematically?

 

I started to wonder how you could build a “camera for the invisible” for taking pictures of these types of invisible situations, to help people re-see.

 

That same day I did a quick mockup. I built 4 temperature-to-color sensors. These lights show a temperature gradient from hot to cold emanating from the laptop.

 

Figure.2.4. Four temperature to color converter modules spread out on a table next to a hot laptop in a cold room. Red represents the hottest temperature and blue the coldest.

 

Then I started a 6-month project to build a Camera for the Invisible. That is a camera that lets you sense non-visible aspects of the environment with sensors/lenses and transform those non-sensory phenomena into some form of phenomena that can be sensed by a human with actuators/viewfinders.

 

Figure 2.5. A diagram predicting the camera form factor of the Camera for the Invisible.

 

 

It let you see carbon dioxide as a color, or listen to temperature as a sound, or feel brightness as a vibration.

 

Figure 2.6. Camera for the Invisible prototype laid out with all its lenses and viewfinders, along with a peek to the back interface with two buttons.

 

Read in detail about the Camera for the Invisible in my Master’s Thesis [Silver, 2008, Camera for the Invisible]

 

 

The Camera for the Invisible had “lenses” (lens-shaped sensors) and “viewfinders” (viewfinder shaped actuators) that you could mix and match.

 

     

Figure 2.7. A lens (left) and viewfinder (right) example that work with Camera for the Invisible.

 

For example, you could have a temperature lens (temperature sensor on the input) and a color viewfinder (colored LED on the output) which would give you a temperature-to-color camera setup. Or you could have a carbon dioxide lens, and a sound viewfinder, which would let you hear carbon monoxide. Or an electrical resistance lens and a tactile viewfinder, which lets you feel the resistance of a material as a vibration on your fingers.

 

Because synesthesia is a condition where stimulation of one sense in a person automatically triggers the stimulation of another sense (for example hearing a sound makes someone think of a color), I sometimes refer to the lens and viewfinder combination as “Synesthetic Pairs.”

 

The camera was made of a natural cherry wood body with holes for routing wires from the hand-soldered motherboard

 

  

Figure 2.8. Left. Cherry Wood laser cut layers of the camera body splayed out. Design lead was UROP Melodia Kao. Right. The motherboard of the camera hand soldered and CnC milled guided “by hand” on fiber backed copper.

 

It could be used to explore any urban natural environment to re-see the invisible aspects, such as this sewer cover’s temperature in different spots, re-seen as color in the viewfinder.

 

Figure 2.9. Camera for the Invisible with a temperature lens and a color viewfinder used to explore a range of temperatures on a cold day at various parts of the cover of a sewer grating.

 

I tried it out with several people, one of them a 15-year-old teenager.

 

Figure 2.10. Site for Camera for the Invisible Study with Teen

 

She was able to make some discoveries (see master’s thesis for details) about shadows and clouds, her fingerprints, and the temperature and electrical conductivity of her body. But when I asked her what she would do with those discoveries in the future she said she didn’t know.

 

That was important. She had investigated electricity indoors, sound and light phenomena outside, resistance and skin-pattern concepts within herself, and noted seeing the world in a new way. But, in relation to action after the discovery events, there was no noticeable difference in her willingness to do something, to change the world, to make something new.

 

So I set out to build a new version of the Camera for the Invisible. A simpler one that encouraged people to look at things anew (re-see), AND build something out of what they discovered (re-purpose and/or re-create).

 

In fact, when I first started researching the Camera for the Invisible, it had consisted of simpler pairings that were more like blocks and less like a camera. In fact when I had introduced the camera concept above, I skipped some of the story that had come before. Let’s work backwards from the camera concept point in the story to see where the camera diverged from.

 

The camera concept came only after an initial exploration that consisted of simple A to B sensory transformative pairs. The design immediately preceding the camera looked like these “railroad blocks”:

 

Figure 2.11. An example of input/output pair temperature to light

 

Working backwards from the blocks simulation, I had a working Scratch software simulation of these sensor pairs (again more like blocks):

Figure 2.12. Scratch simulation of input output pairs with simulated proximity to various natural phenomena.

 

Again stepping backwards, this software was based on this abstract diagram, of an input/output pairing system that I drew:

Figure 2.13. Input/Output theoretical diagram

 

Today a diagram like this is even more familiar in the marketplace of products with examples such as Little Bits in existence:

 

Inputs                                                      Outputs

 

        

Figure 2.14. Little Bits set of Inputs and outputs. (Images reproduced from littlebits.cc)

 

Inputs are on the left in pink, outputs are on the right in green, and you can snap the two together with magnetic connectors and have input/output pairs.

 

Note it is no surprise that around this time there were many input output-sensory transformative kits, and there is also a history of such thinking before this. I was working in the Cube at the Media Lab, which was a repurposed theater space that held Lifelong Kindergarten and a class called How to Make Almost Anything, both of which were hotbeds of kit ideas that had sensors paired with actuators. Lifelong Kindergarten is the group that LEGO Mindstorms and PicoCrickets came out of, both of which were commercially available microcontroller-based kits that let you very easily pair inputs and outputs with computation in between. Flow Blocks was also created here, which allowed you to very easily and magnetically snap computation blocks together and then break them apart. So this was the location where many such input/output easy-access kits were created. Furthermore, the founder of Little Bits was also working in the Cube in a group unrelated to education prior to starting Little Bits, so she would have most likely been peripherally influenced by these types of projects.

 

What is interesting, however, is the breakthrough I had that was a bit counterintuitive.

 

Simplify and Constrain the Synesthetic Input/Output Pairs

 

I veered away from the mix-and-match inputs/outputs model. You see, the results of my study of Camera for the Invisible was that it was great as a lens but terrible at spurring creative/constructive action. Too much time was spent on the mixing and matching. While plugging blocks and components into each other might be great for “engineering education” we run the risk of using too much valuable energy/attention on the blocks part if the goal is: to focus the participant’s attention on the everyday objects and how those can be re-made.

 

In an attempt to focus and preserve the valuable user attention and cognition, I decided it was necessary to simplify the mode of interaction to the point where the tool itself is boring unless multiplied on the environment/world. This simultaneously also simplifies the cognitive load of understanding the tool so more time can be spent in flow with objects the user already has a long history with in the physical world.

 

My final categorization scheme of tools, as inspired by Camera for the Invisible, was a scheme that positioned Instruments, People, and Environments relative to each other in typical use cases. In this analysis the camera was positioned like this:

 

        

Figure 2.15. Model showing physical relationship of a scientific instrument when a person uses it to study and interact with the environment

Thus making it a “PIE” use scenario, with the instrument coming right in between the environment and the person. I then went on to consider a dozen or so other configurations, and I proposed that for putting people into direct contact with the environment the best configuration would look more like this:

 

Figure 2.16. The instrument in this diagram is represented by a box with an “x” on it.  Left: Dubbed “Puddle Jumper” this is an instrument which reacts when two people go jump in a puddle while holding the instrument. Right: Dubbed “Flower Petter”, this instrument reacts when two people touch elements of nature while holding the instrument.

 

In this configuration the instrument (represented in the above drawings by a box with an “x” inside) puts the person directly into contact with the environment by nature of its use. In the figure on the left, to use the instrument the feet must jump into and come into contact directly with the water in the puddle. In the figure on the right, the people must touch the plants in their environment with the skin on their fingers as a forcing function of the instrument. This makes the instrument in this case in a configuration of “Instrument, Person, Environment” or “IPE”

 

Figure 2.17. Diagram of person coming into contact with the environment with the instrument out of the way

 

Thus bringing the person into direct contact as much as possible with the environment, and the instrument is placed more out of the way. This was one step closer to the stated goal of my master’s thesis of instrument transcendance.

Figure 2.18. Diagram of instrument transcendance

 

In this diagram, the person is in direct contact with the world they live in without the need for the instrument.

 

The IPE configuration, which puts the person in direct contact with the environment, was the model that came before the idea that people should directly reconfigure the world they live in. That is to say, coming into contact with the environment was just one step before making changes with the environment in my mind as a designer. As explained earlier and to be explained more in Chapter 3, the Constructive Lens is the name I am giving to a tool that helps someone to see the world as something they can and want to reconfigure. The title of the thesis, “LENS X BLOCK,”  is the first half of a formula:

 

LENS X BLOCK = Constructive Lens

 

Figure 2.19. An illustrative but fictional depiction of a “Constructive Lens” that makes any everyday object look like it’s made of blocks that can be rearranged. In reality, a Constructive Lens must also motivate you to build something interesting not just turn objects into blocks, but visually this image is meant to impart the idea that, “When a Constructive Lens is in your hands, the world you live in starts to look like something that is ready to be re-seen and re-made, as if it were made of wooden blocks.”  (Image created by Sophie Diehl under guidance of Jay Silver)

 

The instrument transcendance depicted above, with the person directly reconfiguring and reconstructing the world they live in, could be seen as the ultimate goal of the tools evangelized in this thesis. When this viewpoint, that the world is something that can and should be reconfigured, is held over long periods of time, creative action becomes a mindset and a way of seeing. When a mindset of creative action becomes dominant, the world is seen as a construction kit, as depicted and spelled out in the subtitle of this thesis “World as Construction Kit”

 

Figure 2.20. World as Construction Kit depicted in a drawing. An adventurer overlooks a landscape after climbing the most recent peak. In her backpack are some tools, perhaps Constructive Lenses. She looks out and sees music notes in the tree bark, wires in the roads, paint in the plants, lenses in the lamp posts, wooden blocks in the houses, dice and keyboard keys in the rocks, pianos in the river, and electrical circuit components in village. (Image created by Sophie Diehl under guidance of Jay Silver)

 

Thus the question becomes: what is an example of an instrument (better named “tool” in this case), and more broadly what are the characteristics of tools in general, that by their nature directly catalyze constructive action on the world one lives in? This is effectively the topic of this thesis, and tools with such properties shall be named and explored in the following chapter.

 

 

Intellectual Traditions Drawn On and “World as Palette” Extension

 

At this point I was starting to merge many fields of study:

a) Inquiry-Based Learning

“Humans are born inquirers. You can see it from the moment of birth: Babies use all of their senses to make connections with their environment.”

~[National Science Foundation, 2000, FOUNDATIONS A monograph for professionals in science, mathematics, and technology education]

 

Inquiry-based learning is learning through asking questions. Exploring and questioning gives rise to knowledge through experiential interaction with the environment. Inquiry is often affiliated with a constructivist way of learning, and is often used in the context of science learning to gain a deeper scientific understanding. In “The Having of Wonderful Ideas” [Duckworth, 1996, The Having of Wonderful Ideas], Eleanor Duckworth describes watching the moon nightly while keeping a journal to start to piece together the moon’s patterns. This delightful activity is a good example of inquiry, and yet even the title of Eleanor’s book gives away the fact that inquiry is more on the side of “having” ideas and less on the side of “enacting” ideas in the physical world. Like a lens, inquiry lends itself to developing new ways of seeing the world. But also like a lens, there is not a direct call for creative action.

 

b) Constructionism

Learning Through Design Process

Constructionism [Papert +,  1991, Situating Constructionism], is in many ways an extension of constructivism [Piaget, 1967, Biologie et Connaissance], the idea that people learn by building their own knowledge when their current ideas interact with their experiences. Constructionism takes constructivism for granted as a way that people learn, and says that this happens especially effectively when people are involved in the process of construction in the context of a project that they find personally meaningful.

 

Like the “Block,” constructionism advocates learning through creative action. But it doesn’t focus on exploring the world or everyday objects in order to repurpose them as an explicit part of the creative action. Take the Logo programming language [Papert, Logo] for example. Logo focused on many powerful ideas, such as recursion, and recursion does exist in the everyday world. However, Logo provides no specific scaffolding for repurposing things from your everyday experience in general. Rather, all of the activity takes place “in the box” (within the sandbox, albeit an expansive sandbox, that is Logo) even though the thinking is in many ways outside of the box. Papert said that Logo was supposed to help people think about their own thinking, and in that way the learner starts to apply what they are doing in the kit to matters outside of the kit.

 

But there were some constructionist threads, such as the Beyond Black Boxes project [Eisenberg +, Journal of Learning Sciences 2000, Beyond Black Boxes: Bringing Transparency and Aesthetics Back to Scientific Investigation], that came closer to trying to explore the world outside of the kit. The projects Metacricket [Martin +, IBM Systems Journal 2000, MetaCricket: a designer's kit for making computational devices), the Playful Invention and Exploration Network [Petrich +, Proposal to the U.S. National Science Foudnation 2000, The PIE Network: Promoting science inquiry and engineering through playful invention and exploration with new digital technologies], and Sensory Design [Mikhak +, 1998, Sensory Design], took traditional constructionism in a more sense-oriented and environment-oriented direction. I pick up where these projects left off, pushing even further in the direction of less computation focus and more nature-based design and instant-feedback sensory-action.

 

c) Bricolage

The term “bricolage” is borrowed from the French verb “bricoler,” the core meaning in French being:

 

"fiddle, tinker" and, by extension, "to make creative and resourceful use of whatever materials are at hand (regardless of their original purpose).”

 

A person who is engaged in bricolage is called a bricoleur. A bricoleur is often known to curate a junk pile so that many interesting items are on hand, and some expert bricoleurs carefully investigate the properties of the items in the junk pile so that they have a mental catalog of the uses of the various items.

 

In any case, in order to make use of everyday objects one must both investigate the properties of those items and also fiddle and tinker with them in order to create a vision. This process engages both inquiry and constructionism, both investigation and creative action, nearly at the same time.

 

Take for example the genre of Earth Art, and specifically the work of Andy Goldsworthy [Goldsworthy, 2001, Rivers and Tides: Andy Goldsworthy Working with Time]. Goldsworthy arrives at a place in nature, and using only what he finds, creates a work of art: an ice sculpture made of found icicles, or a hue fade made by sorting Fall leaves and sticking them together with thorns and mud. Another more digital example is I/O Brush [Ryokai +, Computer Human Interaction 2004, I/O Brush: Drawing with Everyday Objects as Ink]. With I/O Brush, a bricoleur paints on the screen with swatches instantaneously photographed from all types of objects found in the real world. This is very similar to the practice of collage, except digitized so that the swatches can be smeared, replicated, and animated.

 

I/O Brush and “World as Palette” by Kimiko Ryokai

 

I/O Brush is the object of Kimiko Ryokai’s thesis, which is named “World as Palette: Painting with Attributes of the Environment”

 

Figure 2.21. Left: I/O Brush takes a “Photo” of a plate of M&M’s. Middle: I/O Brush smears the photo of the M&Ms across the screen to make a snake-like drawing with the texture and color of M&Ms. Right: a “Snail” is drawn from photos taken with I/O brush of the eyes of a girl, the peel of an orange and an apple, and the colorful stripes of a handbag. (Note: All photos are stills taken from the I/O Brush video made by Kimiko Ryokai).

 

The way I/O Brush works is that it allows you to take a “sample” consisting of a circular photo of any object by pressing the brush up against the object. The color and texture from that object now becomes like “digital paint” which you can smear on the screen. This is where the thesis title “World as Palette” comes from: each object in the world is a veritable swatch of digital paint just waiting to be used by I/O Brush. This simple theme captured my imagination during my time at Media Lab and subtly inspired many of my projects leading to this thesis: World as Construction Kit.

 

World as Construction Kit could be viewed as a direct extension of World as Palette. In the chapter on Constructive Lenses, I will introduce how Palette will be just one component of a construction kit, which can have the world take its place. In this way, this thesis can almost be seen as an attempt to generalize the idea of the World as a Palette.

 

Looking Ahead (Drawdio and MaKey MaKey)

In trying to manifest an instrument that would be similar to the puddle jumper or the flower petter referenced above, two tools which will be highlighted in this thesis were ultimately discovered, (as tools are) by a combination of accident and intention through a series of life events. They will be explained in more detail in the chapter “Drawdio and Makey Makey.” But as an overview let’s consider them in introductory form here.

 

Drawdio is a simple device that measures electrical conductivity (the inverse of electrical resistance) between two wire leads, and makes a pitched sound as a result of the measurement: high conductance leads to high pitch like a high soprano note, and low conductance leads to low pitch like a low bass note.

 

In light of the framework behind Camera for the Invisible, Drawdio could be considered as an input/output synesthetic pair with the input being electrical resistance and the output being pitched sound. The important difference is that there is no swappable input lens or swappable output viewfinder. You might call this a dumbed-down less flexible Camera for the Invisible. But what it inspired ended up being much more important. Another way to say “less flexible” is “more focused”. Additionally, due to the lack of complexity, the bar for understanding and using the tool was significantly lowered.

 

Once I had one of these in my hands, it was possible to explore with it just like a “lens” or “camera”. But it was also a catalyst for inventions because when it was hooked to certain devices those devices became musical instruments or user interfaces

- musical pencils

- musical paintbrushes

- kitchen sink theremins

- tree bass guitar

- macaroni slide whistle

- nose bump beeper

- etc.

 

 

  

Figure 2.22. Left: A cartoon depicting the flow of electricity when using a Drawdio Pencil: out one side of the circuit, across the outside of the skin of the body, through the graphite on the paper, into the pencil’s graphite core, and back into the other side of the circuit. RIght: A Drawdio circuit thumb-tacked onto a regular wooden pencil along with example of musical instruments drawn on the paper.

 

Although technically the camera could have performed the same result, the form factor (big and camera shaped) and the support materials (suggested usage as displayed in videos and instruction) prevented it from doing so despite its more advanced feature set. The forcing function of Drawdio is to connect things in the world together (it ships with 2 alligator clips), which is inherently an interrogative-action-based process by which in order to simply “see” the next thing, you have to change the way the world is connected up.

 

Throughout the years of working with Drawdio, the question arose, what if we could hook everyday objects up to a computer instead of just to a speaker that makes pitched sounds? This question eventually led to the creation of a circuit called Makey Makey which allows the connection of up to 18 objects to a computer. Each object, upon being touched, can trigger an action on the computer through an HID keyboard press, or through a mouse action such as a left-click. Note that in order to do this the circuit board “pretends” from the computer’s point of view to just be a keyboard and mouse, sending keyboard and mouse commands to the keyboard just as a USB Keyboard or USB Mouse would. However instead of sending these commands when a user pushes a key on the keyboard or clicks on the mouse, instead it sends these commands to the computer when two things (for example, a person and a banana) that are hooked up to the circuit by alligator clips, each come into contact with the other, thus closing a circuit. See the diagram for two such examples:

 

Figure 2.23.

Left: A person and four metallic pieces of jewelry are connected by alligator clips to a circuit board. When the person touches one of the pieces of jewelry a circuit is completed and the microcontroller (small computer on the circuit board) detects this event. It then sends a keypress command to the computer to make the video game character turn around.

Right: A person and a banana are each connected to a circuit board. When the person touches the banana, a circuit is completed. The microcontroller (small computer on the circuit board) then sends a keypress command to the computer. The computer just thinks a key was pressed on a regular keyboard and responds in just that same way, by either putting a “space bar” character into a word processor, or playing a sound on a digital piano, whichever app is currently in focus on the computer screen.

 

 

The Makey Makey can then be used to make:

 

- Banana Piano - Google for “web piano” and bring up a web app that plays piano noises when you push the keyboard keys. Then hook up a banana to each of the Makey Makey ports which corresponds to the key on the web app that triggers a piano noise. Now when touching the banana the piano noise corresponding to the key that banana is hooked up to is triggered. All the bananas make a different sound.

 

Figure 2.24. Banana Piano diagram (left) and photo of it in action (right)

 

- Play dough Video Game Controller - Google for your favorite video game, such as Super Mario. Load the web app and start playing the video game with the computer keyboard. Now hook up Makey Makey to some pieces of play dough corresponding to the keyboard keys that trigger your character, perhaps Mario, to run and jump. Now form the play dough into custom shapes, perhaps buttons or arrows, and orient them in a layout that is conducive to that particular game’s style.

 

Figure 2.25. Play dough Game Controller Photo-Diagram (left) and in action (right)

 

- Cat Selfie Cam - Google for a an online camera. Load up the app and give it permission to access your webcam. Then hover the mouse of the “take a picture” button on the app. Connect an alligator clip to the “click” part of the Makey Makey, then connect the other side to a cat bowl full of water. Ground a piece of aluminum foil next to the cat bowl. Then when the cat stands on the aluminum foil and licks the water in the bowl the cat closes the circuit and triggers a “mouse click” event. The computer then takes a picture of the cat drinking water.

 

Figure 2.26. Diagram of Cat Bowl Selfie (Left), and photo of it in action before cat arrives (middle), after cat arrives (right)

 

These three examples cover three of the main types of Makey Makey inventions: Musical instrument, Game interface, and “general invention”. Of course all 3 are inventions of sorts, but the instruments and game controllers are especially conducive and popular.

 

Whether making a musical pencil with Drawdio, or a cat selfie cam with Makey Makey, these tools allow you to explore and re-see the resistance of everyday objects, while repurposing them to do new things.

 

 

Re-seeing and Re-purposing a pencil

 

People don’t normally see a pencil as an object with a “conductive wire” of graphite running down the middle. Nor do they see the graphite markings a pencil makes on paper as little wires that could carry an electrical current. The wood in the pencil is an insulator that protects the conductive wire that runs down the middle from leaking current to objects that come into contact with the pencil. The body of the pencil is an electrical connection point for human hands to plug into the Drawdio circuit. Finally, a musical pencil is made, which is not the original intended purpose of the maker of the pencil, so new meaning and new function is given to the everyday world we live in.

 

 

Re-seeing and Re-purposing a cat water bowl

 

People don’t normally see a cat water bowl as a button for taking a photo. The bowl is an insulator which holds a conductive liquid-form “button” in the form of water. The cat is insulated by its fur, but the cat tongue is conductive. We wouldn’t normally look at the body of a cat as a jumper wire to close a circuit between a sheet of aluminum foil and a cat bowl. The cat’s pads on its feet are also conductive (as is the ears and nose, but not most other parts of its body). The cat is re-purposed as a photographer or at the very least a trigger for a photograph, and a jumper wire of sorts. The household kitchen material of aluminum foil is used as a grounding plate, and the cat water bowl as both an attractor of the cat, which is a typical use, as well as a button to trigger the camera.

 

This is the special feature of the Constructive Lens, it allows us to re-see and re-make the world we live in. It catalyzes new vision and new action.