Digital flatness asserts a question of tangibility.
I recently had a meeting using the Zoom web conferencing software. This was a gathering of two people meeting for the first time to discuss working together on a research project. We began our meeting with our cameras turned on and immediately greeted each other with a hello and a smile directed at them, and thus in the direction of one another. As is customary, I was using the speaker view setting, which prioritizes the person who is speaking by detecting their voice and placing their video image into a larger view, so it dominates the screen. With only two participants in a conference, this automatically shrinks my video image, regardless of who is speaking, minimizing my own video image staring back at me while in conversation with someone else. This is a preferred setting for me because, by contrast, the gallery view creates a grid of video images and highlights the speaker’s box upon detecting their voice, which produces an awkward horizontal ping-pong effect when only two participants are in conversation, tempting the participants to alternate their gaze from left to right. Neither option is ideal, but the speaker view prioritizes the person with whom you’re meeting in a one-on-one exchange, and in this way, it more accurately resembles an in-person verbal interaction wherein the speaker doesn’t see themself participating. Like many of us, I’ve had a great deal of experience with this software over the past year and a half, and thus am aware of its limitations in terms of replicating the behaviors and interactions that characterize in-person conversations, no matter the selected settings.
But something different happened this time, something unexpected. While I spoke, my listener appeared undeniably preoccupied with another matter. She dutifully locked eyes with the camera intended to communicate her participation in the meeting, looking at the “me” just beyond her screen, superficially alleging engagement in the conversation that was unfolding with her gaze fixed on the speaker. But I could tell she wasn’t making eye contact with me or directing her attention to our conversation. Instead, she fussed with her hair, adjusted her glasses, then returned to her hair. She shifted her head from side to side, inspecting the wave she’d just created with her hand, then proceeded to adjust the other side. I grew increasingly distracted. I could feel my face demonstrating my disbelief, but I also felt suspiciously invisible as she continued without pause, failing to recognize my growing frustration and adjusting her behavior accordingly. I began tripping over my words, losing my train of thought, and blurted out that I was sorry, but I was distracted. My passive-aggressive outburst broke her trance, but only briefly. The disruption helped me get back on track, but so did she. She proceeded to fix her hair and purse her lips in slightly different directions, as if she were trying on new micro-expressions. I realized I was trapped in the collapsed netherspace of Zoom software and our interfacing screens. I was reminded of the layers you can adjust in PhotoShop or PowerPoint to help you prioritize and deemphasize objects by reordering them by preference in a flattened space. In this remote conversation, I had been relegated to the unimportant back layer, the corridor tucked behind other layers of digital perspectives, merely background noise for the primping taking place. I was locked behind her camera, which was also her mirror.
There are several factors at work here, not least generational differences, uncommunicated social expectations, and any nervousness induced by our first-time meeting. But another, perhaps overlooked issue concerns the question of touch and how we (dis)regard it within real-time digital interactions. How the interface assumes a pivoting position that connects public and private, visible and hidden, fixed and manipulatable, distant and immersive, remote and tangible. Tangibility here should be understood in terms of occupying the same physical space; breathing the same air; responding to the same external conditions of sights, sounds, and temperatures; exchanging glances without mediation. Mediation is never neutral. Never merely an impartial conduit between here and there, you and me. It occupies a space of its own and carries its own conditions, sometimes fixed and sometimes changing, and often uniquely composed and experienced by individual participants. It has its own tangibility, separate from that which we experience in physical contexts. Yet we rarely reflect on its robustness, complexity, and hidden layers. We rarely consider ourselves as remote bodies interacting in layered digital space, confined both to our seats, our homes, our physical environments, while also psychically catapulted into a hybrid, illusory space of interaction and a virtual-physical logic that governs our movements and tricks our perception.
McLuhan* urged us to reconsider the myth of Narcissus and dispense with its emphasis on vanity in favor of a more flexible interpretation of the desires and experiential conditions that shaped Narcissus’s fixation. This urgency feels especially possible given our familiarity and comfort with the idea of the self as an external, separate other in the form of the mirror or reflective image, and more recently in the form of photography and video. Self-absorption, in this case, supports an irrefutable fascination with one’s image and, I would add, the performance of one’s identity. In my recent Zoom exchange and elsewhere, there is evidence of toying with the self-image: literally rearranging our hair, examining multiple angles of our face, letting the light hit here and there to accentuate and deemphasize certain physical features. We frown in response to a sudden increase in graininess of the video picture, a warning that light is fading in the room and requires restoration. Another lamp immediately gets switched on to replenish this loss and restore the face’s image. There. That’s better. To be sure, the widespread accusation of narcissism as the root cause of activities related to self-voyeurism, as described above, and the taking of selfies, as is often cited, is too simplistic. “It tends to block further thought,” argues Paul Frosh, “and frequently ignores its own gendered assumption linking young women with fickle self-obsession.”†
The mirror provides a necessary interaction with an acknowledgment of the self in the form of the copy, which has become a common expectation of visual cultures. The copy’s disposability and replicability make it even more user-friendly. The magnetic pull of a mirror’s reflection is merely one layer in our present digital-virtual-spectacular environment, but it’s often treated as the most important. The possibilities to modify and shape it appear endless as our perceptual awareness of this surface is the most immediate, and therefore most easily engaging. Lighting, camera angles, digital backgrounds: all can be adjusted. Yet, the real-time engagement of the Zoom and other web conferencing interfaces prove more mirror-like than photographic. Unlike the photo, the mirror image isn’t fixed. Like Narcissus, we can stare into its surface but can’t possess it, and that’s OK with us. In some ways, it’s preferred. There’s a freedom to its instability that allows us to let down our guard, requiring a less structured set-up and allowing for real-time, ongoing adjustments. The aesthetic pressures brought on by social media are in some ways alleviated because of this impermanence. However, this observation merely reinforces the influence of the surface layer and behaviors that discount or altogether ignore the depth of these digital interactions and the entangled network of engagements that hide beneath. Unlike Narcissus, behind the active digital “mirror” is a human interlocutor whose expectations of a peer-to-peer conversation are forced into negotiation.
This brings us back to the question of tangibility, how we experience touch when it is physically absent, and it isn’t unique to the digital sphere.
We experience the robust flatness of literature, maps, paintings, photography, and film. In each of these media, flatness is supplemented by its host: the reader, the walker, the viewer. The host is a narrative agent who negotiates the perceptual complexity and heft of the virtual object in each of these arrangements and synthesizes each part to form a meaningful whole. In a return to McLuhan, this calls to mind his concept of hot and cold media‡. In the former, a content-rich medium does most of the communicative work and activates a user’s single sense. In the latter, the low-resolution format demands several senses to engage, forcing the user to work harder to understand the message. Though this dichotomy is vague and overly simplistic, it’s a reminder to us that sensory perception and engagement with and through a system are important and often overlooked factors in discussions around users’ experiences with media. We are multisensory, multidimensional, and sentient — walking crypts of layered information. We can never fully know how or in what ways our senses are engaged to help us make meaning.
Digital software like Zoom tests us as they toy with our perceptual understanding by enacting the behaviors of physical interactions. But these enactments are performative. They are simulations, as Baudrillard would have called them, perhaps encroaching on simulacra as they inch away from referencing an original. However, through this form of stretching of the senses, it is possible to redefine how we experience the world and the things that are out of reach. When you stretch out your hands in Virtualshamanism: Towards an alternative digital reality of consciousness, a VR-based work created by artist Matias Brunacci, you can feel yourself getting pulled between (at least) two worlds. In the artist’s words, you are operating between two: “the modern one, mental and scientific, and the ancestral one, intuitive and shamanic, generating an approach between both sides through its role as a nexus and peacemaker, placing itself as a powerful mediator between these two dual forces.” The work successfully places you at this uncanny center: between the womb and the techno-future. The feeling of suspension between these two poles also operates with an undergirding reckoning with absence. You are forced to reconcile the never-ending chain of signification produced to satisfy touch, always remaining in a state of deferral.
In virtual environments, you can virtually reach for objects, pick them up, and toss them. You can virtually rub their surfaces. You can imagine their weight and what it’s like to possess them and sometimes try to, either for safekeeping or simply to observe them later. Objects in virtual environments tempt our desire to touch, but they perpetually resist it. No amount of lighting or visible angles will fill the gap between seeing and touching. The object of tangibility is not exclusive to digital objects. Museums, by and large, do not permit us to touch objects on display. Objects are cordoned off, framed, boxed, encased, situated beyond our reach, just in case we are unable to resist touching them. This protects the objects on display from damage; it helps preserve their material integrity. Objects are intended for our gaze only: we can look, but we can’t touch. But doesn’t our gaze perform a variety of touch? What can the weight of our gaze do? And how is touch understood without making contact with our hands or with our bodies?
Touch is a kind of communication. It is a transfer of information between two things. New York City’s Metropolitan Transportation Authority (MTA) is currently advertising its new “touchless” tap-to-pay fare payment system, the OMNY. It’s curious to see the descriptors touchless and tap being used side by side, since the latter suggests a form of touch, or to hit lightly, while the former insists this isn’t the case. But herein lies the crux of tangibility in the digital age: touch can transpire between objects we cannot see. The communication of touch is possible, in this example, in the form of a tap between the passenger’s personal device and a wave emitted from the surface of the system, which reads the information being transmitted from the app. Like the theremin, Leon Theremin’s “strangest musical instrument ever conceived,” contact is made without physically touching the instrument. With the OMNY, tangibility, a kind of touchableness and tactility, is made possible and contact is registered and visualized in the form of verification: successful payment releases the turnstile.
With this in mind, it seems that tangibility is contingent not merely on physical contact, but also on feedback. Therefore, touch can be communicated through this same logic. What are some other ways that such feedback can transpire if not through invisible signal waves? Consider the tangible effects of textures. Snakes have a terrible reputation for being slimy when in fact they are not. The luster that hangs on their skin communicates a gooey surface that suggests a viscosity that does not actually exist. By contrast, a snake’s skin is rather smooth and dry, sometimes soft, which elicits an element of surprise at the point of contact. The visual feedback delivered by the snake’s visual appearance communicates a message all its own. Objects we are prohibited from touching in a museum do this, too. What did Meret Oppenheim have in mind in terms of touch when she created Object (The Luncheon in Fur) (1936)? As a Surrealist and a photographer, Oppenheim understood well the object of absence as a material force. The work is of course physical, but it uniquely engages the virtual as it encourages us to imagine the shape of the object and its referential utility without touching it. Despite this limitation, it’s possible to nearly gag at its conflicting combination of materials, the event of looking transforms into a visceral experience through the contact of sight. The work operates on many levels, forcing the viewer to imagine touch through a combination of imagination, memory, and fantasy. In short, it activates the subconscious tendency to gain understanding through the allusion to physical contact.
The pandemic forced many of us online and the business-as-usual model proved broken. Physical touch was relegated almost exclusively to the digital realm: education, business meetings, fitness, art, socializing all previously existed in an online form but were newly detached from their physical counterparts. For a simulation to work, it needs a reference. But what if we acknowledge the relationship between a simulation and its referent is imperfect? What if we admit that we already operate — freely, openly, and with abandon — as digital subjects? The conditions are suitable — even optimized — for the preservation and experience of new ideas and lost objects, the experience of absence. Artists see these possibilities and rather than force the physical into the digital and obscure their differences, they embrace the gaps, encourage new ones, and invite us to participate in their intersection.
Brooklyn-based digital artist and sculptor Sophie Kahn’s solo exhibition Dematerialized was originally scheduled for March 2020 at the CP Projects Space at the School of Visual Arts in New York City. It was canceled due to the first wave of pandemic-related shutdowns. Kahn recreated the gallery space, inch by inch, using Hubs by Mozilla, and digitally installed the work there, where visitors can view the work using VR headphones or a web browser. In some ways, this instantiation is a more appropriate medium for the work to be understood, as the sculptures in Dematerialized are about the body’s unavoidable absence in relation to digital technology. Interested in how digital scanning drops information when registering a moving body, Kahn worked with Butoh dancers who posed in mid-fall so she could scan their bodies and explore the gaps produced from the missing data through sculpture. In its critique of creative technology and its inability to properly read living, breathing, moving bodies, Dematerialized raises other important questions about materiality: its shape, its context, proof of existence, and the traces we leave behind. The work encourages visitors to confront absence in its many forms, as if absence is measured by degrees. In the Hubs environment, Kahn’s body scans register as avatars that don’t move, almost like NPCs with limited functions. They hang fragmentedly and weightlessly from the ceiling or rest in suspension on a pedestal. If you could see yourself wandering the exhibition, you could say the same about your own avatar body: it is fragmented — my avatar design comprises a head and an armless torso — and weightless, a representation of absence. Kahn’s bodies are an invitation to explore the idea of the body’s trace: you can make out the human figure, but its incompleteness is emphasized. Some of the bodies are larger than life, monumentally transfixing your attention, reminding you of the absence of the humans who helped generate them.
As digital sculptures, these incomplete bodies defy gravity. When experienced in VR, so do you. Fly mode feels different, more adventurous, than moving forward, backward, and from side to side. You still can’t touch the artwork in the gallery, but you can’t not touch it either. As you fly around and gesture toward a piece, you possess it in a new way: there isn’t an accurate way to describe it because it has no referent. In this use of Hubs, the walls are impenetrable, though impact between walls, objects, and an avatar body aren’t represented visually. In this way, the body engages a form of invincibility wherein touch not only avoids harm, but the experience also feels indifferent. Was contact made or not? It’s always unclear. It seems the point of the exhibition is at least in part to encourage visitors to experience the work in a manner that feels dematerializing, sharing the suspension and fragmentation of the objects on display. The sculptures would be impossible to reproduce physically without substantial support to hold them together and scaffold them in place. They visually communicate fragility, large and stone-like but delicate. Even if you could touch them, you worry you might break them. Seeing their stony, broken surfaces produces a feeling of familiarity with the object as a trace or residue as you float around them covertly. These objects have shadows, and you don’t. Hubs could take a cue from Zoom and include a reference to you as an active participant, assuring your groundedness and three-dimensionality in this environment with a shadow that corresponds to your movement as you work your way through the space. Not seeing your own representation has a spectral effect: your inability to touch rests on your non-existence. Kahn’s Dematerialized frames absence in a way that triggers the idea of death and the physical realities and limitations of the body and its preservation and accurate representation.
It calls to mind Char Davies’s pioneering VR work, which situates the body at the center of each virtual experience by transforming it into an active interface with the work and thus invites participants to consider their bodies as agents of action and a sense of being in the world. Osmose (1995) relies on breathing and the body’s movement through space to elicit a self-aware connection between the physical and virtual worlds, producing a unique feeling of groundedness somewhere between them. A kind of third space emerges: like Brunacci’s Virtualshamanism, the effect feels womb-like but not necessarily futuristic. Davies describes the environments as world-spaces — there are 12 — that connect with nature, including the abyss, and refers to participants as immersants, underlining the overwhelming feeling of being surrounded by an environment that doesn’t exist in the physical world but nonetheless feels palpable. The idea of touch here is administered in more bodily ways than we’re accustomed to in many VR experiences because breathing is involved. Physical touch can be intimate, but breathing is visceral in noticeably profound ways.
Like Kahn’s Dematerialized, Davies embraces ambiguity and the liminal space between forms: “In contrast to the hard-edged realism of most 3D-computer graphics, the visual aesthetic of Osmose is semi-representational/semi-abstract and translucent, consisting of semi-transparent textures and flowing particles. Figure/ground relationships are spatially ambiguous, and transitions between worlds are subtle and slow. This mode of representation serves to ‘evoke’ rather than illustrate and is derived from Davies’ previous work as a painter.”§ While these choices suggest a disengagement with touch — how does one fully grasp semi-abstract translucency? — the effect is tactile, relying on an internal feedback response, through breathing, rather than sight. The body is engaged as an actor within the system despite the visual feedback that may suggest otherwise.
In lacking the ability to properly read and understand the body, Kahn’s scanner is an admission not merely of a technical limitation but a failure to register bodies as they truly are: live, moving, and breathing, yes, as well as diverse, differently sized, interactive, and programmed, unapologetically heterogeneous. Thus, Kahn’s sculptural avatars invoke Lynn Hershman Leeson’s idea of the anti-body, as a kind of “fictional persona” that enacts “codes of gestures”, among other things to operate freely from the physical body¶. Moreover, a useful way to consider Kahn’s emphasis on this error is through Legacy Russell’s notion of the glitch as a progression of Leeson’s anti-body as a “tactical strategy.”
“This strategy”, writes Russell, “becomes operable in the face of the failure of the systematized networks and the frameworks within which we build our lives. Glitches gesture toward the artifice of the social and cultural systems, revealing the fissures in a reality we assume to be seamless. They reveal the fallibility of bodies as cultural and social signifiers, their failure to operate only as hegemonic normative formulations of capital weaponized by the state. The binary body confuses and disorients, pitting our interests against one another across modalities of otherness” (Russell, 2020)#.
Tangibility is a consequence, a response between two or more things. It is activated by perception, guided by the senses as they work together to make meaning and reconcile both the gaps and areas of overlap between different objects, including intangible and invisible objects. Time, air, echoes, shadows, energy, radio waves, sound waves, electromagnetic waves, microwaves, infrared light, ultraviolet, visible light, x-rays, gamma rays. Energy is an invisible trace with the strength to move, transform, combine, replace. Electromagnetic waves are different from mechanical waves. They don’t need a medium to propagate. They can travel through both air and solid material, as well as through the vacuum of space. In social VR environments, like Hubs by Mozilla, we move in similar ways, sometimes passing through objects and people, represented by avatars, as we travel through the environment. Though the environment is architecturally conceived and simulates physical space, it is wholly unique as our movement disfigures the space into porous portals, disrupting our perceptual framework. Distance collapses: nearness and farness are redefined. Though spatial audio simulates the way sound travels in physical environments, avatars can enable fly mode and travel through teleportation, turning distance into abstractions. In these environments, we experience the presence and absence of digital objects, movement, and space. Movement is the communication that confirms their materiality. Our own materiality is absent. We cannot see ourselves — no hands, no reflection, no feet when you fix your gaze downward. We float around absently, like ghosts, who trespass through objects and other avatars. We don’t feel the force field that surrounds us and other objects, preventing touch and physically stopping us from intersecting with them. Digital objects collide without consequence. In other virtual worlds, parts of our virtual bodies are visible: as in Virtualshamanism, the hand that reaches the virtual object can visualize contact with objects and their environments.
These observations offer a return to the Zoom interface and the inefficiencies it attempts to conceal. Its imposition of flatness on users — advancing the false notion that the surface layer is the only one of concern — distorts and compresses not only bodies but personalities, behaviors, and perceptual awareness. Participants have no choice but to confront their own image, minimized or not, while pretending to socialize in a manner that denies its interference with the socialscape the software aims to recreate. Zoom and its counterparts are, in this way, simulacra: seeking to pass themselves off as something else by creating social situations that have no referent. Through this software, our images represent our anti-bodies, as described by Leeson, “self-created alternate identities,” for which we choose to “negate the self-hood into which [they were] born.” Like Kahn’s fragmented avatars, we are trapped in the digital flatness of our screen interactions, suspended between a sense material of wholeness and its anti-body other.
You can view Sophie Kahn’s Dematerialized VR here.