share this article

In the 2016 short film, Hyperreality, designer and filmmaker Keiichi Matsuda crafts a disorienting, even oppressive, depiction of everyday life mediated by ubiquitous augmented reality. Set in the Colombian city of Medellín—and shot from a first-person perspective filtered through an advanced eyepiece digital interface—from the first frame, the protagonist’s senses are bombarded with a deluge of advertisements, games, music videos, animations, and even renderings of computer-generated architecture and “nature.”

Digital communications and social media platforms have become fully environmental in this vision of the future: 3-dimensional, audio-visual, and immersive. Something of the overwhelming spatial logic and hyper-stimulating aesthetics of New York’s Times Square or Tokyo’s Shibuya Station are translated into an embodied sensory experience; urbanism, in Matsuda’s future-world, becomes a kind of media in itself. In one scene, the film’s protagonist-user (“Juliana”), becomes confused, even anxious, by a technical glitch which forces a reboot of her device while shopping for food, showing us a brief glimpse of an un-augmented reality wherein a supermarket’s unadorned architecture is clearly designed for the express purpose of accommodating a digital overlay. Matsuda’s film ultimately suggests that augmented reality may become so commonplace as to be essential to making sense of one’s world.

Hyperreality also extrapolates from, and radically accelerates, expectations for augmented reality’s future; expectations which have been largely conditioned by the technology’s first real-world success story, Pokémon Go. Darkly refracting Go’s seemingly innocent and benign rescripting of urban space, the film portrays augmented reality as an inherently gamified, commercial, and homogenizing interface experience. Indeed, since the sharp rise (and precipitous fall) of Pokémon Go, a critical mass of popular and financial speculation has swirled around AR as a new form of mass entertainment. Despite the game’s failure to sustain its initial popularity, AR in general is projected to become a multi-billion dollar industry within the next ten years – utilized for leisure (and labor), migrating across interfaces (from smartphone to wearables) to become an ever-present, normalized filter of human perception in built environments.

City Skins: Scenes from an Augmented Urban Reality

Pokémon Go in Times Square.

This speculative orientation, however, must be complemented with a dose of history. Though the pace of technological change in the field has been swift, we should seek to challenge prevailing attitudes that would posit digital “innovation” as monopolized by California tech entrepreneurs. Shedding just a little historical light onto the relatively slow, collective, and complicated development of AR can help challenge the obfuscating notion of “disruption,” and open up space for greater public debate into how, and for whom, this technology is being used.

Over the past couple of decades, artists and designers have developed augmented realities that propose vastly different, and often more radical perspectives of what a digitally enhanced public realm could look like. Countering Matsuda’s unsettling vision of a AR as a layer of perception primed for capitalist exploitation, many actually existing AR projects instead ask critical questions about the implementation of this novel technology and its potential to shift both the everyday experiences and political economies of architecture and cities. What does it mean to filter public space through a highly personalized form of digital sensing? How does the collapse of virtual onto physical space reorder established ways of understanding landscapes? What are the potential pitfalls and possibilities of a technology that promises re-enchantment of space, yet relies on privately-owned data infrastructures and a military-operated ensemble of near orbit satellites?

Growing interest in location-based AR projects, starting in the late 1990s, can be in part attributed to the confluence of art and networking technologies which emerged out of the gradual popularization of the Internet and the influence of “net art.” Net art, according to critic Josephine Bosma, has often concerned itself with “the public domain as a virtual, mediated space consisting of both material and immaterial matter,” indicating a conceptual and ethical foundation for augmented reality’s radical leap from the space of the screen to a “hybrid space” mixing real and virtual elements.

City Skins: Scenes from an Augmented Urban Reality

Cover and sample page from the Headmap Manifesto.

 

Ben Russell

Near the tail end of the 20th century, pseudonymous author and technologist Ben Russell released The Headmap Manifesto — a utopian vision of augmented reality at once referencing Australian aboriginal songlines and the history of the occult, while pulling heavily from cybernetic theory and the Temporary Autonomous Zones of Hakim Bey. At turns both wildly hypothetical and eerily prescient, Headmap explores in-depth the implications of “location-aware” augmented reality as a kind of “parasitic architecture” affording everyday people the chance to annotate and re-interpret their environments.

At the same time, artist and scholar Teri Rueb began developing her pioneering, site-specific augmented “soundwalks,” some of earliest and most influential examples of GPS-based art practice, influenced by “land art” practitioners such as Robert Smithson and Richard Serra. Her work identified the critical potential of locative AR as a direct mediator of spatial experience, capable of revealing hidden layers of meaning within landscapes. Beyond “land art” a number of AR practitioners and theorists would explicitly identify the Situationist International (and even Archigram) as conceptual touchstones for the kind of digital enhancement, and potential subversion, of space made possible through augmenting reality.

City Skins: Scenes from an Augmented Urban Reality

A Yellow Arrow sticker found on a New York City rooftop.

Compared to the pastoral setting of Rueb’s work, other projects —such as the early interactive AR game Can You See Me Now?  from the UK-based multimedia arts group Blast Theory or The Yellow Arrow Project, which ran from 2004 to 2006 — took a decidedly urban turn. Yellow Arrow, for instance, utilized uniquely-coded stickers and SMS text messaging to “draw attention to different locations and objects—whether a back-alley mural, a favorite dive bar, or a new perspective on a classic landmark.” This project’s mixing of digital and physical media, as well as its creator’s explicit references to street art and psychogeography, seemed to hint at a more subversive agenda. However, Yellow Arrow, produced by Counts Media — an “arts-driven entertainment company” — ultimately embraced more entrepreneurial uses for its platform, including a collaboration with travel guide publisher Lonely Planet on a book called Experimental Travel. One of its founding members, Jesse Shapins, would eventually become the Director of Public Realm & Culture at Alphabet’s urban start-up Sidewalk Labs (now largely known for its “smart city” machinations along Toronto’s waterfront).

City Skins: Scenes from an Augmented Urban Reality

Pre-smartphone interface used for Urban Tapestries.

Proboscis

City Skins: Scenes from an Augmented Urban Reality

Project map for Urban Tapestries.

Proboscis

Meanwhile, an annotative “montage” project called Urban Tapestries (originally Social Tapestries) was launched in 2004 as “a research project and experimental software platform for knowledge mapping and sharing” oriented towards establishing a “public knowledge commons.” This platform, similar in concept to the not yet world-conquering notion of social media, encouraged a range of geographically-bound participants — the project was limited to central London — to share personal stories and artefacts from everyday life through an early wireless, networked application built for bygone “personal digital assistant” devices. The project lasted three years, resulting, beyond the platform itself, in the release of a number of essays, reports, videos, workshops and installations; all requiring the technical and financial support of a number of partners, including the London School of Economics, Birkbeck College, Orange Telecom, Hewlett-Packard Labs, France Telecom R&D UK, Ordnance Survey — a robust mix of educational and governmental institutions and private technology firms. The project’s goal of establishing a “public knowledge commons” was, from the beginning, diluted by an uncritical examination of such partnerships and their implications, predicting in part the ambivalent privately-owned “public” spaces of today’s prominent social media networks. Yet in employing anthropologists and explicitly citing the Mass Observation movement of the 1930s, Urban Tapestries hinted at a more thoughtfully-considered and collectively-generated vision of spatial augmentation through mobile digital technology.

City Skins: Scenes from an Augmented Urban Reality

Tiananmen Squared

4 Gentlemen

Following the release of the first iPhone and advancements in mobile phone cameras and processing power, AR began to move toward the more visually-dominant experiences we are familiar with today — in the process also opening up possibilities for more explicitly political projects. The group 4 Gentlemen, for instance, embraced AR as a tool for criticizing oppressive government policies in China. A collective of exiled Chinese artists and one American artist, 4 Gentlemen (taking their name from a group of intellectual dissidents central to the Tiananmen Student Protest in 1989) developed a series of works that digitally recreated in situ both the famous “Tank Man” image and the “Goddess of Democracy” statue — two symbols of the Tiananmen protest which have defined the struggle for democracy and human rights in China since. The group would eventually join a number of other digital artists in 2011 for AR Occupy Wall Street, an augmented tribute to the famed protests movement organized by the anti-capitalist Manifest.AR collective.

City Skins: Scenes from an Augmented Urban Reality

Frontera de los Muertos

John Craig Freeman

City Skins: Scenes from an Augmented Urban Reality

Garden of the Anthropocene

Tamiko Thiel

Other new media artists have created more contemporary AR works that creatively challenge dominant political narratives. John Craig Freeman’s Water wARs overlaid a fictional “pavilion for artists/squatters and water war refugees” in Venice’s Piazza San Marco. Frontera de los Muertos geo-tagged 3D-renderings of a skeleton effigy, based on a traditional form of Oaxacan wood-carving, to the precise coordinates of where the very real remains of migrants illegally crossing the U.S.-Mexico border have been recovered. Tamiko Thiel’s body of work, going back to the 1980s and intersecting virtual reality as well as performance art, includes site-specific AR “installations” such as Garden of the Anthropocene, which explored the effects of climate change on the future of plant life. Even AR gaming has been touched by this more critical turn. Born out of The New Normal program at Moscow’s Strelka Institute, Patternist fuses a mysterious science-fiction narrative to explorations of real-world urban typologies and informal economies.

City Skins: Scenes from an Augmented Urban Reality

Screenshot from a promotional trailer for Patternist.

As the augmented landscape becomes ever-more crowded with commercial titles like Pokémon Go — similar games wedded to other massive media franchises such as Harry Potter are reportedly in the works — it is heartening to see the survival of more thoughtful AR projects that resonate with the technology’s surprisingly rich history of experimentation. And as this landscape increasingly constitutes a public realm in and of itself, a collection of hybrid real-virtual public spaces, there are even glimmers of direct challenges to its creeping privatization.

City Skins: Scenes from an Augmented Urban Reality

Digital “vandalism” of a Jeff Koons AR sculpture.

Sebastian Errazuriz

In late 2017, Snapchat collaborated with Jeff Koons to develop a geo-tagged AR public sculpture in New York City’s Central Park. Yet within one day, the work had already been “vandalised.” Rather than hacking Snapchat, the culprit, artist Sebastian Errazuriz, built an alternative, graffiti-bombed version of the Koons sculpture assigned to the exact same coordinates as the original piece. Though the act may have ultimately been more symbolic than an effective challenge to Snapchat’s augmented public space intervention, Errazuriz’s stated motivations capture some of tenor of debates to come.

"For a company to have the freedom to GPS-tag whatever they want is an enormous luxury we should not be giving out for free. The virtual public space belongs to us; we should charge them rent."
Sebastian Errazuriz

What does the virtual space that “belongs to us” look like? Would could it look like? We might imagine a future as steeped in AR as Matsuda’s Hyperreality, but where instead of a hybrid landscape dominated by ads and obfuscating distractions, augmented overlays are used to highlight the hidden dimensions of place, or serve as a distinctly spatial platform for (perhaps illicit) communication and alternative forms of (perhaps politicized) culture. Inverting the vision of a deeply commodified hybrid landscape, the seemingly inevitable barrage of immersive, interfacial capitalism could be transmuted into something democratic, artful, and even beautiful: mixed-reality as a conduit to open discourse, to collectively owned and managed hybrid spaces, the “community” as a social body intersecting physical and digital worlds. But if this more hopeful image of an AR-saturated future is to come to fruition, it will require a deeply collaborative spirit between programmers and urbanists, artists and technologists — and most certainly architects as well.