Augmented Reality Is Transforming Museums
New York’s Museum of Modern Art is under siege. Well, a virtual siege, at least. A group of renegade artists has co-opted the brightly-lit Jackson Pollock gallery on the museum’s fifth floor, turning it into their personal augmented reality playground.
To the uninitiated, the gallery remains unchanged; Pollock’s distinctive drip paintings are as prominent and pristine as ever. But to those that have downloaded the MoMAR Gallery app on their smartphones, the impressionist's iconic paintings are merely markers—points of reference telling the app where to display the guerilla artists’ works. Viewed through the app, Pollock’s paintings are either remixed beyond recognition or entirely replaced. One artist has framed a Pollock painting in an interactive illustration of a smartphone running Instagram, allowing viewers to “heart” the work over and over again. Another has overwritten Pollack’s imagery with an artistic interpretation of the many conspiracy theories peddled by Q, a mainstay of the far-right on 4chan. Together, the eight works form a virtual exhibition dubbed “Hello, we’re from the internet,” which uses AR to challenge MoMA’s gatekeepers and museum curators at large.
Related Stories
“When you think that art defines our cultural values, you also have to accept that those values are defined by a certain part of society—call it the elite,” says Damjan Pita, who, along with David Lobser, is the brains behind MoMAR.
MoMA, for its part, has stayed quiet about the app, and did not respond to a request for comment on this story. But the movement is about to go global: Lobser and Pita have heard from artists in Los Angeles, China, Germany, and Serbia, all hoping to use MoMAR’s open-source software to enact virtual takeovers of major museums in their own cities. Meanwhile, in recent months, art enthusiasts in Boston have used AR to “return” stolen artworks to their frames without the holding institution’s cooperation, and, in a particularly meta twist, an artist virtually vandalized a virtual work of art. The potential AR has to shake up the art world is slowly taking shape—and right now, it’s a lawless free-for-all.
Museums have long dealt with unauthorized augmentations of their exhibitions, such as unofficial tours, but technology has opened up new possibilities for activists and art enthusiasts eager to have a part in shaping the museum-going experience. Back in 1991, a project called “Masterpieces Without the Director” distributed cassette tapes on the steps of the Metropolitan Museum of Art, offering an alternative audio guide to the one provided by the Met itself and, as one of its creators told the New York Times at the time, “democratiz[ing] the viewing process.” Even MoMA itself is no stranger to AR interlopers: In 2010, artists Sander Veenhof and Mark Skwarek took over multiple floors of the museum, scattering virtual works throughout its various galleries and inviting visitors to spot them through their then-clunky smartphones. But with tools like Apple’s AR kit and Google’s ARCore have made it easier than ever for developers to build and distribute AR apps, and that newfound accessibility is raising a host of new questions for the art world. Who owns virtual space, and what recourse does a museum have if an outside party “trespasses” on its virtual space? Moreover, is it even in a museum’s best interest to retaliate against unauthorized virtual augmentations—or should they be embraced as a new, if uninvited, tool for visitor engagement?
Some projects, like MoMAR, are explicitly antagonistic to the institutions whose works they’re augmenting. But others fall into more of a grey area that comes from a lack of any precedent for how museums should handle these sorts of virtual intrusions. The latter was the experience of Cuseum, a Boston-based startup that helps museums use technology to boost visitor engagement. Last month, Brendan Ciecko and Dan Sullivan, respectively the startup’s CEO and head of partnerships and growth, used AR Kit to enhance a museum that they had long loved: the Isabella Stewart Gardner museum, a staple of the Boston arts scene. That museum is renowned in part because of what isn’t on display: In 1990, thieves stole 13 works of art valued at $500 million, and to date, the orchestrators of the heist have not been caught. Cuseum had been experimenting with AR for a while, helping the Pérez Art Museum Miami launch its first-ever AR exhibition last winter with funding from the Knight Foundation. In early 2018 when Apple released an AR Kit update that made it easier to work with vertical surfaces, Ciecko and Sullivan were inspired. They could use AR, they thought, to “restore” the missing paintings to their frames.
It just so happened that AR Kit’s new vertical capabilities coincided nearly perfectly with the 28th anniversary of the infamous heist. And so Ciecko and Sullivan scrambled to put together a functional app that would virtually return the stolen works by March 18. They spent hours in the gallery, and, on the weekend of the heist’s anniversary, they published a website featuring previews of the app and detailing how they went about “hacking the heist.”
Local press picked up the story, and by all accounts, the experiment was a grand success. But soon after the anniversary, Cuseum received what Ciecko describes as a “a very surprised inquiry from an individual at the museum that was not very happy about this.” Cuseum had informed the Isabella Stewart Gardner about its plans, and had hoped to work on the project cooperatively; Ciecko and Sullivan had even been given a soft green light by a museum staffer who stopped them in the gallery one day to ask what they were doing, before telling them that they weren’t breaking any rules. But the museum’s less-than-enthusiastic response to the project stopped Ciecko and Sullivan in its tracks. They’d hoped to release Hacking the Heist as an app available for public download. But they didn’t want to burn any bridges. And so, for now, the project is on hold.
Ciecko says now he gets a dozen emails a day from people eager to use the app; one person emailed to say that he and his wife met at the Isabella Stewart Gardner museum and were flying to Boston to celebrate, and wanted to see the stolen works. “I had to write back, ‘I’m so sorry, it’s not available to the public, but congrats on your anniversary,” Ciecko says. “It’s a weird place to be, between people being really excited about something and folks on the other side not being as excited. What’s the diplomatic thing to do?” A spokesperson for the museum says that though the Gardner was not involved in Cuseum’s project “the concept of using AR to see something that you can't actually see while you are visiting the museum (like the stolen works) is something we have been discussing.”
Ciecko and Sullivan may have been crossing their own moral boundary by releasing Hacking the Heist to the public—but they wouldn’t have been breaking any laws, even though they didn’t have the museum’s cooperation. The works are in the public domain, and as long as the app didn’t purport to be sponsored by the museum, Cuseum would have been in the legal clear. MoMAR, too, doesn’t appear to be breaking any laws: As an explicit commentary on museums’ institutional power, it falls pretty squarely under fair use. But the law around AR and art is fuzzy, at best.
“At the moment, there’s no such thing as a recognized right to control the space or virtual augmentations of your work,” says Alexia Bedat, an attorney specializing in AR and VR; however, Bedat adds that existing laws, such as copyright or the Visual Artists Rights Act, may apply to certain augmentations.
“Virtual trespassing” is a new, ill-defined concept, though ongoing class action against Pokémon Go could begin to clarify the legal limits of augmentation—that is, whether it’s legal for someone to place a virtual object on private property. The litigation around Pokémon Go has also brought up the idea that, even if the AR itself doesn’t constitute trespassing, it could prompt users of the app to trespass and cause a nuisance to the unwitting hosts of AR Charmanders and Squirtles. So far, none of the AR intrusions in museums have summoned crowds that could be deemed a “nuisance,” though MoMAR’s gallery opening, hosted on a Friday afternoon (when MoMA offers free admission), did attract some 50 visitors to crowd inside a typically modestly occupied gallery.
Despite the current lack of of clear laws around what can and cannot be done to virtually augment art, museums aren’t entirely powerless. When visitors enter a museum, they agree to whatever rules that institution has set out—no photography, for instance, or no touching the paintings. Museums could begin to add “no AR apps” to their rules, or ban the use of phones outright—though doing so might seem like a step backwards, considering that many museums only recently began embracing smartphones as a way to engage their visitors. Artists, too, could begin negotiating more complex contracts with museums, spelling out what can and cannot be done to augment their works. The latter may become more common as museums follow in the Pérez Art Museum Miami’s footsteps, experimenting with their own AR exhibitions. “There are a lot of interesting IP questions we have to navigate,” says Christina Boomer Vazquez, deputy director of marketing and public engagement at PAMM. “There’s also the issue of respecting the artists that are on view and the impact that [augmentation] would have on that artist and that work. [Augmentation] can alter the whole context and conversation of that artist’s work.”
But so far, the Isabella Stewart Gardner and MoMA have remained quiet about their AR interlopers; neither has tried to take legal action against the unauthorized augmentations. It’s a smart approach. React too quickly, or too defensively, and they might wind up doing themselves a disservice in the long run. AR—no matter the source—could be a great thing for museums, bringing in new visitors eager to experiment with the new technology. It could also pique younger visitors’ interest in older works. But it all comes down to a question of authority. Traditionally, the museum experience was one-directional: Curators conceived of and executed an exhibit, which visitors then enjoyed. Now, that’s all starting to change.
“Museums are obviously striving for relevance, because the world is increasingly splintered and competing at offerings, and a static object finds itself competing for our attention more and more,” says Maxwell Anderson, an art historian and former director at the Whitney, Dallas Art Museum, and other instutions. Exhibitions like the Museum of Ice Cream and the Rain Room at the Los Angeles County Museum of Art rely on interactivity and Instagram-friendliess to draw crowds—and AR is yet another play for engagement. That quest for relevance, Anderson posits, is what’s leading museums to both adopt and be co-opted by AR—and even unauthorized AR intrusions like MoMAR and Hacking the Heist can be a boon for institutions eager to avoid obsolescence.
“From my perspective, it’s not really worth fighting against it, because gravity is not working on our favor,” says Loic Tallon, the Metropolitan Museum of Art’s digital chief. The Met doesn’t currently have any of its own AR projects underway; Tallon says that he doesn’t think most visitors feel that anything is missing from the museum as is, and he wants to be very purposeful in how the museum adopts new technology, lest it winds up doing so just for the sake of novelty. But the Met, too, has experienced AR invasions, such as one project that animated Van Gogh’s First Steps, after Millet, and Tallon welcomes those augmentations with open arms.
“The museum’s mission is to collect, preserve, and study works of art,” he says. “If someone is making an AR experience out of the collection, I see it as pure mission fulfillment.”
More WIRED Culture
Related Video
Business
Magic Leap's Next Move? Bringing C-3PO to Your Living Room
This video of Star Wars characters C-3PO and R2-D2 recorded through Magic Leap's technology isn't just cool—it also shows the nearly unlimited potential of a new partnership between Lucasfilm and the mixed-reality company.