What You See Is What You Touch? Double description of seeing and touching in ‘Display Acts’

What you see is what you touch?

View more presentations from Masanori MIzuno.

There is, in fact, almost no formal theory dealing with analogue communication and, in particular, no equivalent of Information Theory or Logical Type Theory. This gap in formal knowledge is inconvenient when we leave the rarified world of logic and mathematics and come face to face with the phenomena of natural history. In the natural world, communication is rarely either purely digital or purely analogic.1

This is Gregory Bateson’s words. I consider that user interface has the same problem. Computer is digital. Computer is controlled by formal theory. However, We, human, is not purely digital. We, are analogic rather than digital. Therefore, user interface should be the place where purely digital computer becomes analogic one, and analogic human becomes digital one in order to communicate each other.

Display Acts

When we use a computer, what do we do? Almost all of us look at some images on an electric display, grab and move a mouse, and type on a keyboard, then our right hand holds the mouse in order to point to an image called an icon on the display. This is very 'natural' for us. If our body makes some actions with a plastic object, then the images on the electric display change. However, this relationship between our body and the image did not exist until the computer, and especially until the Graphical User Interface (GUI), appeared. I call this phenomenon 'Display Acts': action which is formed by connecting our body action with the change of images on the electric display.2 However, we do not understand this action well because we have only separately studied body actions and images in man-computer communication.

In order to make clear the relationship between our body action and images in the man-computer world, this presentation focuses on the cursor in the electric display and the mouse in the real world. We see the cursor in the electric display: The cursor is "a small mark on a computer screen that can be moved and that shows the position on the screen, where, for example, text will be added."3 Moreover, the shape of cursor is almost arrow (→) which is "used to show direction or position."4 We just see the arrow cursor in order to know where it is and point some images on the display. Then, we also hold the mouse.

Exemplification

What does the cursor show us? It shows us the position and direction, however, the cursor often points nothing. When the arrow cursor points nothing, it does not work. We don't care this. Why? Because we not only see the images in the display, but also grabbing the mouse. What is a relationship between seeing and grabbing? In order to make clear this relationship between the cursor and mouse, I refer to Nelson Goodman’s idea; Exemplification. According to Goodman;

Exemplification, though one of the most frequent and important functions of works of art, is the least noticed and understood. Not only some troubles about style but many futile debates over the symbolic character of art can be blamed on ignoring the lessons, readily learned from everyday cases of the relation of being-a-sample-of, that mere possession of a property does not amount to exemplification, that exemplification involves reference by what possesses to the property possessed, and thus that exemplification though obviously different from denotation (or description or representation) is no less a species of reference.5

Possession means that some samples show us their own properties, and we interpret their properties for our interests. The user interface is not work of art, but I consider that the images in the electrical images with the mouse or other input devices show us almost same functions of work of art.

We see the shape of cursor as a sample in order to know functions of cursor. We get a pointing function from the arrow shape, therefore, we label this pointing function to the arrow cursor. However, the arrow cursor points nothing. As a result, we think this arrow possess another functions; Maybe it is moving for pointing because the arrow is shape for pointing. Only when we see the electric display, we can understand that the functions of the cursor is only pointing. However, user interface has the mouse. We touch and grab the mouse, then we see that the cursor moves and point the image. This touching is another function of the cursor because the cursor is designed for connecting with the mouse in order to work in the display. Only when we see the display, we can just know pointing and moving functions of the cursor (fig. 1). Only when we touch the mouse, we can understand the cursor is able to touch the image and change images in the display (fig. 2). Only pointing can not change anything in the display. Changing needs touching. In short, we not only see the images on the display, but also grabbing the mouse when we use the computer. Therefore, we have to consider what a relationship between seeing and touching is.

　　　　　　　　　　　　fig.1

　　　　　　　　　　　　fig. 2

Moreover, the arrow cursor sometimes suddenly changes into the spinning wait cursor or something else. When we see transformations of the cursor's shape, the mouse we grasp does not change. However, the moment the arrow cursor changes into the spinning wait cursor, our action with the mouse changes. Before transformation, we use the arrow cursor and the mouse in order to point to an image on the display, and therefore the cursor exemplifies the function of pointing and our body action with the mouse is formed for pointing. After transformation, the spinning wait cursor exemplifies the function of, not pointing, but showing the computer status. As a result, our action with the mouse is not formed for pointing. After a moment, we realize that what we can do with the mouse in hand is just moving the cursor in the electric display. The spinning wait cursor exemplifies that the computer is still working, but it does not leave us alone; the cursor and the mouse are connecting, please wait. Throughout the whole process, we grasp the same plastic object, even though the cursor's shape is changing.

Double Description

In user interface, the image we see is changing while the object we touch is not changing. Moreover, this transformation of image is sometimes out of our control. But we naturally accept this phenomena. In order to make it clear, I refer Bateson’s Double Description. He quotes Shakespeare’s Macbeth and says that:

This literary example will serve for all those cases of double description in which data from two or more different senses are combined. Macbeth “proves” that the dagger is only an hallucination by checking with his sense of touch, but even that is not enough. Perhaps his eyes are “worth all the rest.” It is only when “gouts of blood” appear on the hallucinated dagger that he can dismiss the whole matter: “There’s no such thing.” 6

We need Double Description of seeing and touching in order to consider and prove what the user interface is. However, user interface mainly focuses on seeing. For example, What You See Is What You Get; WYSIWYG. From Wikipedia, the main attraction of WYSIWYG is the ability of the user to be visualize what he or she is producing.7 Here, ‘to be visualize’ is emphasized. One more example. Ben Shneiderman proposed a very important idea for user interface: Direct Manipulation. From Shneiderman, Direct Manipulation’s central idea is to be visibility of object of interest.8 Here, only visibility is emphasized.

I want to focus on Direct Manipulation from anther point of view. According to George Lakoff and Mark Johnson. Direct Manipulation leads us not only seeing but also touching because we see and touch the object in order to create something. And they say that the Direct Manipulation makes the concept of CAUSATION.9 Furthermore, Søren Pold connects GUI with the concept of CAUSATION. He says that WYSIWYG interface has many buttons, and buttons are essential part of controls in GUI10 and also continues that:

When pushing a button in a interface ─ that is, by movement of the mouse, directing the representation of one’s hand onto the representation of a button in the interface and activating a script by clicking or double-clicking ─ we somehow know we are in fact manipulating several layers of symbolic representation and,…… It is a software simulation of a function, and this simulation aims to hide its mediated character and acts as if the function were natural or mechanical in a straight cause-and-effect relation.11

According to Pold, pushing button in the user interface makes a straight cause-and-effect relation in the electric display, although this cause-and effect appears by hiding manipulating several layers of symbolic representation.

Another view of the cursor and mouse. A media artist, Masaki Fujihata says computer does not have a question of materiality.12 This means that computer has different principle from us because we live in the material world. Fujihata considers that if the repeated experience are provided us, through interactive experience, Images are made into object.13 According to artist’s intuition, we go through new phenomena: Image are made into object.

What You See Is What You Touch?

However, this ‘object’ in the display is not real object. According to Pold, this ‘object’ in the display shows us cause-and-effect, therefore, we feel as if we directly manipulate it by our own hands. On the other side, Fujihata says that the ‘object’ does not have its own materiality. We, of course, see the ‘object’ but can we touch it? Maybe, yes. Maybe, no. I consider that this ‘object’ is something like “switch” as Bateson defined;

We do not notice that the concept “switch” is of quite a different order from the concepts “stone,” “table,” and the like. Closer examination shows that the switch, considered as a part of an electric circuit, does not exist when it is in the on position. From the point of view of the circuit, it is not different from the conducting wire which leads to it and the wire which leads away from it. It is merely “more conductor.” Conversely, but similarly when the switch is off, it does not exist from the point of view of the circuit. It is nothing, a gap between two conductors which, themselves exist only as conductors when the switch is on. In other words, the switch is not except at the moments of its change of setting, and the concept “switch” has thus a special relation to time. It is related to the notion “change” rather than to the notion “object.” 14

Like the concept ‘switch’, the ‘object’ is related to the notion “change” rather than to the notion “object”. The cursor is given a special relation to time by the mouse.

Why is this happen? Because the computer is purely logic machine which is made by analogic human in cause-and-effect world; “the if ... then of logic in the syllogism is very different from the if ... then of cause and effect.” 15 Bateson makes a question: Can logic simulate all sequences of cause-and-effect? His answer is maybe no.16 The reason is that “the if ... then of causality contains time, but the if ... then of logic is timeless. It follows that logic is an incomplete model of causality.”17 However, we want to re-made the computer as cause-and-effect machine via Double Description of seeing and touching. Introducing the special relation to time with the mouse, user interface makes new images with coupling cause-and-effect and logic in the electric display. We are making pseudo-cause-and-effect with logic in the user interface. Therefore, we touch the same plastic object, even though the ‘object’ in the display is changing.

                                             fig.3

Returning to the cursor and the mouse. The cursor looks like showing cause-and-effect, however, it stands for the logic in the computer: there is no time. But, we live in time: cause-and-effect world. Pold tells us that the user interface make us believe there is a straight cause-and-effect relation, even though this cause-and effect appears by hiding manipulating several layers of symbolic representation. On the other hand, Fujihata says that if we see cause-and-effect behind the image, then we can touch it. Almost all us do not understand what the logic in the computer is, but we use it with the display, the keyboard, the mouse, and something else. The reason is that we believe that the computer is not only logic, nor only cause-and-effect, but just ‘something’. Therefore, when we consider about user interface, we should not confuse ‘if ... then of logic’ with ‘if ... then of cause-and-effect’. We see the change of image in the electric display; it shows us the logic in the computer but we feel something like cause-and-effect, which is made from computer’s pure logic. We touch the same object while the image is changing: our grasping mouse introduces the special relation of time to the computer and makes pseudo-cause-and-effect in the computer (fig. 3). Therefore, only seeing is not enough and only touching is neither. In order to made clear ‘something’; the pseudo-cause-and-effect with logic, we have to consider the double description of seeing and touching in the user interface. In short, the computer has changed the relationship between our seeing and touching. It means that the computer changes our notion and experience for touching object: Touching not “object” but “change”.

References
1 Gregory Bateson, “Steps to an Ecology of Mind”, The University of Chicago Press, 2000, p.291.
2 Masanori Mizuno, “The formative process of “Display Acts” on the establishment of GUI”, Doctor Thesis,2009, p.3.
3 Oxford Advanced Learner’s Dictionary (electric version), Oxford University Press, 2000
4 Ibid.
5 Nelson Goodman, “Way of Worldmaking”, Hackett Publishing Company, 1978, p.32.
6 Gregory Bateson, “Mind and Nature”, Wildwood House, 1979, p.73.
7 Wikipedia, http://en.wikipedia.org/wiki/WYSIWYG (9, April 2009 access).
8 Ben Shneiderman, ‘Direct Manipulation: A Step Beyond Programming Languages’ in Noah Wardrip-Fruin
and Nick Montfort ed., “The New Media Reader”, MIT Press, 2003, p.486.
9 George Lakoff & Mark Johnson, “Metaphor We Live By”, The University of Chicago Press, 2003, pp.75-76.
10 Søren Pold, ‘Button’ in Matthew Fuller ed., “Software Studies: a lexicon”, MIT Press, 2008, p.31.
11 ibid., p.33.
12 Masaki Fujihata, “Masaki Fujihata: The Conquest of Imperfection ─ New Realities Created with Images and Media”, Center for Contemporary Graphic Art, 2007, p.178.
13 ibid., p.178.
14 Bateson (1979), pp.108-109.
15 ibid., p.58.
16 ibid., p.58.
17 ibid., p.59.

Origina text: Proceedings of 4th International Symposium on Multi-Sensory Design

The (pseudo) causal chain

   http://zig.media.mit.edu/Work/G-stalt

In last entry, I considered why a gesture interface like G-stalt is studied in Tangible Media Group. Prof. Ishii says that:

Interfaces that control the computer with a mouse are like a black box, and it's difficult to understand what's going on inside. If gestures are used, however, one can clearly see the causal chain, as with a violinist's response to the conductor's baton. Research into gesture interfaces is attempting to hammer out a new paradigm by seamlessly merging body, objects, and space."

MIT Media Lab Tangible Media Group's Gesture Interfaces: Dynamic Interaction in Hybrid Space in AXIS 2009.8 vol.140

Although G-stalt seamlessly connects body, objects, and space, why the image is not in this relationship? The thing that is moved by the gestures is not the object itself but the image on the display, isn't it? Are 'body, objects, space and images' seamlessly merged in G-stalt? However, Prof. Ishii does not say the image. Why? Maybe, the images have already become the objects in our life. If we think that, I can smoothly connect the gesture interfaces with 'Radical Atoms' which is next vision of TMG. Now, we rarely think that the image is the 'image'. The image is something like the object, so it is tangible. Therefore, we can feel that the image is the object and the objects is the objects at the same time now. This idea makes totally new paradigm. This paradigm may be related with plasticity, I think. 

I want to focus on Prof. Ishii' word " If gestures are used, one can clearly see the causal chain". I consider that the mouse and the cursor on the display make "pseudo-cause-and-effect" between our action which is ruled by physical cause and effect and computer logic which is not ruled by one. Then, The pseudo-cause-and effect changes the images on the display into the 'object [entity]' which is something tangible. Even though the image which we see on the display has already become the 'object', it is supported by the pseudo-cause-and-effect. However, the gesture may interfaces remove  'pseudo' from the pseudo-cause-and-effect because the gesture is closely connected our body. Therefore, Prof. Ishii is maybe right. Our action will be tightly connected with the image on the display because the gesture is very close to our body. In the mouse and cursor, 'to grab and move something' is transformed into  'to point and move something' and this transformation makes the pseudo-cause-and-effect on the display. 'Grab' and 'point' are very general actions for us, then we can do many things with the combinations of these two actions on the display. As result, there is the ambiguous relationship between the image on the display and our action. However, the gesture requires one to one relationship with the image on the display. This one-to-one relationship make us think why an action is connected with the change of image on the display. Connecting our gesture with the image on the display in one to one relationship, the image may be more close to the object. Then, may the ambiguity of our action be reduced in turn?  

Atom → Bit → Pixel → Atom

I feel something strange that Tangible Media Group(TMG) studies the gestural interface which is not like the tangible. And more, TMG leaded by professor Hiroshi Ishii shows a new vision 'Radical Atoms'; Study the interface based on the material which can freely change like the pixel on the display. There is a stream of idea; Atom → Bit → Pixel → Atom. We have touched Atom for a long time. Then, we have to touch Bit like touching Atom. We have to touch Pixel like touching Atom. In the mean time, we have to touch Atom like touching Bit and Pixel. The first Atom does not change. It is just solid material. However, the second Atom can freely change. It is like fluid material. Now, we can change Bit and Pixel at our will but Atom does not. Prof. Ishii feels strange this solid Atom, therefore he give us the vision of 'Radical Atoms' for study the interface based on the idea that we can freely change the Atoms. I think that the second Atom is the new entity based on the appearance of Bit and Pixel which is always changing. Bit and Pixel is freely changing therefore we have to make a new interface in order to correspond them. It is g-stalt. This gestural interface make us re-think our actions to Bit and Pixel. This thought leads us new actions for the new fluid Atom. I think that this is 'Radical Atoms'.  In this way,　I understand why TGM studies the gestural interface like g-stalt.

Seeing the ‘light-colour’ seduces a new kind of touching

When we use a computer, what do we do? Almost all of us look at some image on an electric display, grab and move a mouse, and type on a keyboard, then our right hand holds the mouse in order to point to an image called an icon on the display. This is very 'natural' for us; if our body makes some actions, then the images on the electric display change. However, this relationship between our body and the image did not exist until the computer, and especially until the Graphical User Interface, appeared. I call this phenomenon 'Display Acts': the action formed by connecting our body action with the change of images on the electric display. (Mizuno, 2009) Through living with the computer, we have acquired new actions in order to inhabit this new image world. In other words, 'Display Acts' is the first step for our new actions in the man-computer world. I have already discussed ‘Display Acts’ on the first computer graphic system, Sketchpad, concerning the action of drawing the image with light. (Mizuno & Motomaya, 2008) However, that study did not show why we touch the image on the electric display.

Now, the electric display, for example iPhone, seduces us to touch the image. Erkki Huhtamo writes that:

While the classical cinema and even television broadcasting still emphasize distanced and physically nonactive forms of spectatorship, video game consoles, mobile phones, laptops, iPod and other >handy< electronic devices have familiarized millions to the >>tactile dimension<< [emphasis in original]. (Huhtamo, 2008, p.130)

In other article, Huhtamo adds, “How those development will affect the realm of tactility as we know it remains to be seen.”(Huhtamo, 2007, p.94) This paper takes over Huhtamo’s suggestion and sees a new realm of tactility which the new technology opens to us.

What do we see in the electric display?

At first, we have to consider how our body reacts the image made from the electric light, because this artificial light has totally changed the world. Marshal Mcluhan wrote that;

In a word, the message of the electric light is total change. It is pure information without any content to restrict its transforming and informing power. (Mcluhan, 2003, p.77)

We should know how the electric light affect us when we see it. Most of us see the electric light from TV or computer display everyday. The electric display came into our life as TV in late 1930s, therefore we may say that the TV is the first electric display we are familiar with. Mcluhan gives TV one chapter in “Understanding Media” because he believes that it opens new world. He points out the nature of the TV as bellow;

The mode of the TV image has nothing in common with film or photo, except that it offers also a nonverbal gestalt or posture of forms. With TV, the viewer is the screen. He is bombarded with light impulses, that James Joyce called the “Charged of the Light Brigade” that imbues his “soulskin with subconscious inklings.” The TV image is visually low in data. The TV image is not a still shot. It is not photo in any sense, but a ceaselessly forming contour of things limned by the scanning-finger. The resulting plastic contour appears by light through, not light on, and the image so formed has the quality of sculpture and icon, rather than of picture. (Mcluhan, 2003, p.413)

Mcluhan’s statement is famous because he tells us that the TV image is made from ‘light-through’ and gives us not visual sensation but tactile sensation. Seeing the contour of ‘light through’ means that we directly see the light source beyond the image. As a result, we see the electric light itself in TV image.

Based on above idea, I will especially focus on this phrase; “A ceaselessly forming contour of things limned by the scanning-finger” in order to examine the nature of the relationship between our body and the electric display. This forming contour makes the apparent motion on the electric display. Nelson Goodman considers about the apparent motion as ‘A Puzzle about Perception’ in his “Ways of Worldmaking”. Goodman writes, “that virtually every clear case of visual motion perception depends upon abrupt shift in color.” (Goodman, 1978, p.88) Goodman shows us that the apparent motion happens due not object-shape but object-colour. Moreover, he continues that;

With visual system taking such leaps in stride, with their indispensability for motion-perception, with object-identity dependent not upon smooth color transition but upon contrast with the background at the contour, the color-jumps in the Kolers experiments seem so inevitable as to leave us wondering how we let a false analogy trick us into expecting anything different. (Goodman, 1978, p.89)

Even though we normally think that smoothly changing the colour of object causes the apparent motion, Goodman focuses on the colour-jumps at the contour between the object and the background. It means that we look at not just the object alone but also the relationship between the object and the background. The contour is the place that the background becomes the object, and the object becomes the background, therefore the contour is the place they are ceaselessly merged in each other of colour. This merge makes the apparent motion.

Now, I want to go back to Mcluhan. He gave important role to the electric light because it has the potential to merge figure and ground. (Mcluhan, 1988, p.194) Therefore, the electric light is best media for the apparent motion, “a ceaselessly forming contour of things” in Mcluhan words. However, Mcluhan misses the colour aspect for the apparent motion that Goodman mentioned while Goodman also misses the aspect of electric light for it that Mcluhan mentioned. Therefore, we have to consider not only the nature of electric light but also the nature of colour formed by the electric light.

‘Light-colour’

When we see the electric display, aesthetician Asao Komachiya says that we see ‘light-colour’. ‘Light-colour’ throws away material information and extract only colour information from the object. Komachiya writes:

“Light has no weight. This is our recognition from the experience of human history. Similarly, light-colour cannot express its weight. However, the object described does have a weight. Therefore, the description of the object conveys the weight feeling for us. Paintings have expressed this. .... However, the image made from light-colour does not essentially fit this principle.” (Komachiya, 1996, pp.95-96)

Furthermore, Komachiya affirms that ‘light colour’ opens new image field due to the nature of no contour. (Komachiya, 1996, p.305) ‘Light-colour’ is mainly made from the electric light which has the potential to merge figure and ground, therefore this new colour does not have its contour. Owing to above natures, ‘light-colour’ looks similar to David Katz’s ‘film colour’. (Katz, 1996, pp.7-17) However, unlike Katz, Komachiya mainly takes directly seeing the electric light into consideration, which is a similar point of view to Mcluhan. I would like to suppose that the human has an innate ability to sense no materiality in the colour like Katz’s ‘film colour’ and the electric light make this our ability go into next step: It is ‘light-colour’

According to Komachiya, the ‘light-colour’ image is, however, beyond control for our sensations because it does not fit our traditional principles. ‘Light-colour’ forms an image but it is no weight and contour. We try to merge this new principle to our familiar one, but this task may be beyond the capacity of our brain. Therefore the brain may ask the body to make a new reality for the ‘light-colour’ Image. This seduces our body to touch the ‘light-colour’ image in order to compensate us for its no weight and contour. However, we have not been able to touch it because we have not made the device for that until now; although we tried to touch the ‘light-colour’ image on TV like “Winky-Dink and you [1953-57]”, even though it was just pretend touching.

‘Light-colour’ with the computer

Now, we have computers in order to control the weightless image formed by the electric light. It may mean that the pure information meets the information machine. In the traditional sense, the light reflected from the material of the world makes the images. Although he realizes ‘TV image’ is made from ‘light-trough’, Mcluhan dare to say ‘TV image’ but it’s not an image in the traditional sense. Moreover, Komachiya shows that ‘light-colour’ cannot tell the nature of weight; therefore it does not show us its own materiality. In short, the image made from the electric light may be just colour information of thing in the traditional sense.

Ron Burnett shows us the unique point about the image in our age. He writes, “The distinction between images and information blurs into pixels, lines, and rates of compression.” (Burnett, 2004, p.47) His point of view about the image is not analogue and digital, which gives us many suggestions. Furthermore, Masaki Fujihata re-defines the colour as a concept because the computer releases the colour from its materiality. (Fujihata, 1997, p.7-11) Although Burnett and Fujihata recognize that the computer gives us the chance to control the information of ‘light colour’, they forget the electric display. In fact, ‘light-colour’ merges the image and the information into one entity on the electric display because of its nature; no weight, no contour. The computer must need the electric display to generate ‘light-colour’ image.

Touching ‘light-colour’ with the smooth materiality = Believing our body

Our seeing of ‘light-colour’ seduces our recent new kind of touching. We have always touched materials which had their own weight. Material like the plastic mouse or the glass of the display do not change by our touch. However, touching the material causes some changes in a weightless entity on the electric display. The human and computer are making a new circuit for dealing with weightless entity made from ‘light-colour’ via the material object which is our body and something like the mouse or trackpad.

Moreover, there is the smooth materiality like the plastic or glass when we pay attention to only contact surface of our fingers and object. For example, Apple writes “to stay smooth and pristine, the new Multi-Touch trackpad is made from wear-resistant etched glass. [Emphasis is added]” (Apple, 2008, online) Why is touching ‘light-colour’ connected to the smooth materiality? ‘Light-colour’ merges figure and ground; therefore there may be something flat, no contour in our vision field. ‘Light-colour’ does not show its own weight, therefore there may be no friction to grab and move in our tactile field. Consequently, the smooth materiality is very close to the reality of ‘light-colour’. Then, we touch the smooth materiality and see the colour-jumps in the ‘light-colour’ image at the same time. However, there is a paradox because we have our own body with some weights and its own contour. We paradoxically feel the heavy density of our body by giving weight and contour to the weightless image via our smooth touching, when we see and touch ‘light-colour‘ entity,

After seeing ‘light-colour’ for a long time, we just begin full-scale investigation for the new realm of tactility with the smooth materiality. David Katz writes, “What has been touched is the true reality that leads to perception. [Emphasis in original]” (Katz, 1989, p.240) We re-train our fingers with the smooth materiality in order to touch and generate new reality of ‘light-colour.’ This demand us to believe our body’s weight and density in the ‘light-colour’ world. And this belief creates new diverse bodily sensations in ‘Display Acts’.

Reference

Apple, 2008: http://www.apple.com/macbookpro/features.html (June 14, 2009 access)

Burnett, Ron, 2004. How Images Think, MIT press.

Fujihata, Masaki, 1997. Colour As a Concept, Bijiutsu-Shuppan-sha.

Goodman, Nelson, 1978. Ways of Worldmaking, Hackett publishing company.

Huhtamo, Erkki, 2007. Twin-Touch-Test-Redux in MediaArtHistories, Oliver Grau ed., MIT press, pp.71-101.

Huhtamo, Erkki, 2008. Tactile Temptation: About Contemporary Art, Exhibition, and Tactility in Interface Cultures, Chista Sommere, Laurent Mignonneau, Dorothee King ed., transcript, pp.129-139.

Katz, David, 1989. The World of Touch, translated by L.E. Krueger, Lawrence Erlbaum Associates.

Katz, David, 1996. The World of Colour, translated by R.B. MacLeod and C.W. Fox, Routledge.

Komachiya, Asao, 1996. Chi no me・Sora no me [the human history of vision], Keisou-Shobou.

Mcluhan, Eric & Marshall, 1988. Laws of Media: the new science, University Tronto press.

Mcluhan, Marshall, 2003. Understanding Media: the extensions of man, critical edition, W. Terrence Gordon ed., Gingko press.

Mizuno, Masanori & Motomaya, Kiyofumi, 2008. To see and Touch the Light Source in Proceedings of ISEA2008, pp.329-330.

Mizuno, Masanori, 2009. The formative process of “Display Acts” on the establishment of GUI, Doctor thesis, Nagoya University.

the new realm of tactility with the 'light-colour' entity and the smooth materiality

After seeing 'light-colour' for a long time, we just begin full-scale investigation for the new realm of tactility with the smooth materiality.

David Katz writes, ﾒWhat has been touched is the true reality that leads to perception. [Emphasis in original] (Katz, 1989, p.240)

We re-train our body with the smooth materiality in order to touch and generate new reality of 'light-colour'.

Before, we touch something physical in order to feel the weight (in the flow of cause and effect).

Now, we touch something smooth in order to give the weight to the 'light-colour' entity (for making its smoothness in the information flow).

This demand us to believe our bodyﾕs weight and density in the 'light-colour' world.

And this belief creates new diverse bodily sensations in ﾔDisplay Actsﾕ.