FuturePerfect Podcast

About FuturePerfect Podcast

The FuturePerfect Podcast features interviews with compelling people breaking new ground in art, media, and entertainment. This podcast is produced by FuturePerfect Studio, an extended reality studio creating immersive experiences for global audiences. Episodes are released every two weeks, visit futureperfect.studio for more details. futureperfect.substack.com (https://futureperfect.substack.com?utm_medium=podcast)

Publisher

FuturePerfect Studio

Category

arts · technology

Language

en

Explicit

No

First episode

19 Apr 2022

Latest episode

28 Feb 2023

Latest episodes

6 episodes in the feed.

1. 

   28 Feb 2023

   #006 - Andrew Keller: Columbia Records, David Bowie NFTs, and Web3 Experiments

   Welcome to the FuturePerfect Podcast where we talk with compelling people breaking new ground in art, media, and entertainment. This podcast is produced by FuturePerfect Studio, an extended reality studio creating immersive experiences for global audiences. Visit our website futureperfect.studio (http://futureperfect.studio/) for more details. The text version of this interview has been edited for length and clarity. Find the full audio version above or in your favorite podcast app. For episode 006, Wayne Ashley interviews Andrew Keller (https://twitter.com/andrewdkeller), founder of We Few Group (https://www.wefewgroup.com/), a post-media company that manages singers and songwriters, and develops brand partnerships and ventures with internationally known visual and musical artists. Making innovative use of blockchain technologies and NFTs, Andrew has been building compelling projects with such noted organizations as the David Bowie Estate and singer-songwriter Stefan Storm. Early career at Columbia Records Let me start right off by saying how excited I am to be speaking to someone who’s been part of the music industry for over 20 years. As much as I love music, I’ve had very little access to the inside workings of the business. You’ve had these long relationships with labels like Columbia and Capitol Records. What are some of the most important insights you’ve gained over the last two decades? Andrew Keller: One of the biggest insights I’ve gained is that change is necessary and hard. When you’re dealing with global major labels it’s really hard to make noticeable changes. There’s a lot of people working at major labels who are innovative and want to do good. It’s just really hard to turn a gigantic ship around. You’re dealing with all sorts of policies and god knows how many different types of contracts that were written over the years. How did you get started at Columbia Records and what were your major focuses were when you were there? AK: I started at Columbia as a 17 year old and was a total music-head. I grew up in New York City so I had access to absolutely everything, and thankfully, I had parents who were simultaneously really supportive and also slightly oblivious. From the time I was 13 or 14 I was going out to places I probably shouldn’t have been and exploring all sorts of different scenes from nightclubs to hardcore and punk shows to mainstream pop shows. I was really giving myself a crazy education in different subcultures and figuring out little scenes and pockets that had their own worlds, archetypes, and systems in place. And they’re probably all relatively similar, it’s just people wearing different clothes and different hair. But the whole concept of having these scenes and worlds always excited me. It kind of all played into ideas of identity—sonic identity and visual identity. I was going to lots of shows around 2002 when there was this giant emergence of bands in the tri-state area who were coming out of the punk scene. And it became kind of this pop-punk, emo, screamo post-hardcore thing. I refer to it lovingly as the Warped Tour (https://www.billboard.com/music/rock/warped-tour-2005-oral-history-interview-8461060/)-scene, which was this festival that traveled around the US for years. There was a ton of attention on bands that I was friends with and bands that I had built relationships with as a fan. One thing kind of led to another and I ended up getting hired as a junior A&R scout / assistant at Columbia and got to work for some amazing people. One of whom is kind of this incredibly legendary small bald man named Matt Pinfield, who was a DJ and then VJ on MTV. A lot of the time we kind of ended up having a deal where he would get booked to DJ and then I would basically cover his set and he’d split the money with me if he had to leave. So I started DJing and it was just this awesome moment in time where I was really young and exposed to so many people. It became clear to me at that point that I just loved being around creative people. I was never in a band. I don’t sing. I don’t consider myself an artist in any way, shape or form. I do consider myself a creative person and a professional fan. I realized early on that for me it was gonna be about being the conduit and the kind of middle man between the artist and the rest of the world. That period of time was really incredible. I got to work on MGMT (https://www.youtube.com/watch?v=B9dSYgd5Elk)’s first two albums as a coordinator then started to sign bands on my own like Cults (https://www.youtube.com/watch?v=eAM9diyVRiM) and St. Lucia (https://www.youtube.com/watch?v=lDKJ29357FU) and doing stuff in the dance space with Krewella (https://www.youtube.com/watch?v=J-gYJBsln-w) and Dillon Francis (https://www.youtube.com/watch?v=12CeaxLiMgE). And then one of the last things I got to do at Columbia was Bring Me the Horizon (https://www.youtube.com/watch?v=QJJYpsA5tv8), which as a metal kid was unreal. And that was all under a guy named Ashley Newton, who is still one of my closest friends and mentors. He was responsible for signing Spice Girls, Massive Attack, Daft Punk, and Pharrell. Ultimately I went with him and Steve Barnett (https://www.musicbusinessworldwide.com/steve-barnett-on-running-capitol-your-faith-has-to-be-greater-than-your-fear/), who had been the chairman of Columbia, over to Capitol Records when Steve was relaunching Capitol. Streaming music, shifts in the industry, and joining Capitol Records I’m very interested in the sort of crises that upend one’s assumptions about the world and motivate people to do something radically different than what they’ve been doing. You mentioned inertia in the music industry when we talked earlier. What happened that caused you to leave Columbia, join Capitol and eventually start your own company? AK: There were a few things. I was 30 and had started at Columbia when I was 17 and I really wanted a change. I loved my artists and everyone I worked with. It really was a family, but I was too comfortable. If you do A&R at a label your real challenge at the end of the day is to sell records. Your goal is to find artists, help them make the best record possible and have as much success as you can. There’s a million ways to do it, and I’m not even saying I was great at it, but it’s a very linear goal. You go from point A to point B, and that’s fine, but it had just done so much and I needed a change. So Steve Barnett called me and said what would you want to do if you were going to stay at a label? And my answer was that streaming was starting to become really dominant. And the thing that streaming changed was access to global music. Before streaming if you were a British band you would sign to a label in the UK for the world. But unless something really took off, you might not even get a US release. When Spotify started putting out music, for the most part, nothing was geo-locked. Everything was coming out day and date, but labels were still working territorially. So I said to Steve, I think there’s gonna be a really big shift in the way music is consumed from an international standpoint and I wanted to create “international A&R 2.0” for streaming. So off I went to Capitol Records where I started figuring out this whole system. And also taking systems that were already in place and trying to break them because there were things that didn’t make sense to me and that I thought needed to be changed. Part of that was really just being the ambassador and being the person who could go and have some difficult conversations. But also go and represent Capitol around the world. It was a lot of time on airplanes. I had a ton of fun at Capitol. I got to partner with Lewis Capaldi (https://www.youtube.com/watch?v=zABLecsR5UE) who is on his arena tour right now and is a fantastic artist. I introduced Capitol to SM Entertainment (https://www.smentertainment.com/), one of the biggest K-pop labels in the world, and they have an amazing and fruitful partnership. When I reached the end of my deal at Capitol I was in my thirties and my brand had always been owned by a major corporation. There were things I wanted to do that I’d never be able to do in those situations. For example I wouldn’t be able to music supervise a movie, I wouldn’t be able to launch my own projects or do certain things. An open-ended post-media company From here you started your own company We Few Group (https://www.wefewgroup.com/). I like to call it an “open-ended post-media company”. You take on so many roles—an artist manager, a mixing engineer, an NFT project producer, an entertainment strategy consultant, you’re also working closely with visual artists to produce a graphic novel and even knitwear. This kind of post-media practice that you engage in is so exciting to me and it’s exactly what FuturePerfect is doing. How are these different worlds and practices connected for you? AK: When I went off to start We Few Group (https://www.wefewgroup.com/) everyone was like what is it, what are you going to do? And I just said I’m going to do things that excite me with people I like. And that was it, that’s the entire thing. For example, you mentioned the knitwear. A couple of years ago, there was a painter that I was obsessed with and I wanted to buy his paintings. I ended up getting on a Zoom call with him for over two hours talking about what he was doing. At the end of the call he asked—if I manage artists could I also manage a painter? He’s in the kind of traditional art world, but makes these 30-second horror films around each of his pieces on TikTok. It’s all kind of neo-gothic stuff. He has all of these kids as fans, everyone from artists and influencers and just regular teenagers and 20-somethings, the typical TikTok audience. And they all wanted merch. They weren’t necessarily buying fine art, but they wanted merch. And he and I started having conversations about it. He wasn’t really interested in making merch because he is an artist, but he was like, I’ll start a clothing company. Next thing you know, I’m learning about knitwear. This connects back to your earlier question of why I was leaving. The answer is I wasn’t learning stuff. There was no time in my life at a label that I would ever be learning about making knitwear, or consulting for Arizona Iced Tea and helping them with their entertainment strategy for two years. I now know more about consumer packaged goods and beverage production and can wrapping. Is that the world’s most useful thing to know [laughs]? No but I love learning that kind of stuff and being around it. Crypto and transforming artistic engagement This brings me now to your fascination with the crypto space. That’s another expansive world. What about crypto most excites you and your efforts to transform both artistic and business practices? How did you get into it? AK: I got into crypto early. I’ll preface this with saying it’s not because I bought millions of Bitcoin at $2 and am now sitting here just pounding money. I wish that were the case, but it’s not. I started buying little fragments of Bitcoin in like 2010. Growing up things like business and banking and the stock market were like the devil to me. For better or worse, I kind of vilified it in my mind. But with crypto it felt almost like punk rock banking. I was completely intrigued by it as a kind of rebellion. For years friends heard me talking about crypto, and when the pandemic started and touring was shut down there was suddenly a bull market mentality in the crypto space. And NFTs, which had been around for a few years prior, started to be something that artists, managers and agents were paying attention to. People started calling me saying you probably know about this stuff, right? I gave pretty much everyone the exact same conversation. I said please don’t do this, it is really early. This is a real world and kind of culture. You do not care about them. They do not really care about you. Your fans do not care. You know, this is bad for everyone. I guess not a lot of people were voicing that at the time. A lot of people were saying we should do this and people made a bunch of money, but I think a lot of them also looked a little dumb. A lot of the projects were pretty empty. Most of them don’t get talked about anymore. But as this world kept growing I started getting calls again. I realized that I could help be a middle man and tour guide. I could help bring the right people with the right intentions in and help shape what adoption to Web3 looked like and help introduce people to the real crypto-native world and to the people who really care about this. A lot of what I do now with Web3 is try to find interesting projects and people who I think will love and enjoy this space and help them either dip their toe in the water or jump in the right way surrounded by the right people. Working with the David Bowie Estate Talk about the David Bowie project, because clearly this is something that you have a lot of passion for, and through it we can better understand what you mean by NFT, crypto space or blockchain. AK: It was without a doubt, one of the most incredible and surreal things I have and probably will ever get to be a part of. Let’s put it this way, when my son came home from the hospital in his nursery at home over the changing table is a caricature of David Bowie. My dog’s name is Bowie. I’m not a casual Bowie Fan. I revere Daivd Bowie. I couldn’t write it better. I’m sitting on the computer one night buying something on the NFT marketplace OpenSea (https://opensea.io/) with my dog Bowie sitting next to me and I get a text that says “I can’t really say much, but can I introduce you to the Bowie estate, they want to talk about NFTs.” We end up starting this dialogue with the executor of the Bowie estate, who is just an incredible man. He had heard a lot about crypto and what was going on in this space and was very cautious, but had a kind of bullishness. As someone who knew Bowie well, he knew Bowie would’ve been really excited by this. For me, there had to be a very specific why. That question of why are we doing this? What is our north star for this? What was the vision that emerged for the project? AK: I basically spent the weekend thinking about Bowie and everything I knew about him and digging into his art collection because he was a huge art collector. I happened to have the catalog of the Sotheby’s auction when they auctioned off his collection. So I started going into it and asking what did his collection look like? What kind of art did he collect? Then you start thinking about him as a technologist and a lover of new technology. And you’re like okay you had BowieWorld (https://www.dailydot.com/debug/david-bowie-world-virtual-3d/) before Metaverse was even a thing we talked about. You had BowieNet (https://slate.com/technology/2016/01/bowienet-david-bowie-s-early-internet-service-provider.html), which was his own ISP and kind of fan club site. You had BowieArt, which ultimately became a showcase of art that he liked and art made by his community. And then you start thinking about Bowie Bonds (https://www.billboard.com/music/music-news/david-bowies-bowie-bonds-55-million-wall-street-prudential-6843009/) and the idea of him having commodified his work and well, that sounds a lot like a bunch of NFT projects. And I just went, okay, this is a guy who kind of had the ethos of the crypto artist before that was a thing. If the blockchain is the permanent immutable ledger, then let’s go put this on the blockchain. Let’s go put his legacy there. So what does that mean to put his collections on the blockchain? AK: Well, it wasn’t his collections, it was, let’s do a project that puts on-chain—something that is on the ledger and can never be deleted—that he was here. And let’s create something to honor his legacy. I kept going back to the fact that he was a huge supporter of new artists. He was a digital artist himself. I got access to the Bowie archives through my partner Joaquin and I would literally get screenshots on my phone of anything from the archive whether it was an outfit he wore or a ticket to one of his shows or art he had done or a photograph. Everything is meticulously databased. And so Joaquin and I basically started narrowing down this idea. The project became, let’s get a handful of the best artists in the crypto art space, from super established to new and up-and-coming and give them free rein to create anything they want to contribute to Bowie. They also had the added bonus of incorporating anything from the archives, which no one has ever been able to do. From there I started to ask who do we want, who makes sense here? I did things like going through Bowie’s personal art collection and basically tagging a bunch of stuff like “landscape”, “British artist”, “contemporary African artist”, “sculptor”. And then I did the same with crypto-native artists and tried to find what the connections were. It wasn’t about finding a one to one match with everyone, but there were certainly people where I could say okay, this guy’s work kind of reminds me of this guy’s work. Or I think what this guy does with his art is kind of interesting in relation to this part of Bowie’s archive. And also people who were fans of Bowie and people whose art represents certain things that I thought were key to Bowie’s legacy. So we came up with a dream list and started approaching them. Almost everyone said yes. What makes this specifically crypto art? How do you differentiate the art that emerges within the blockchain? What is unique about it? AK: What makes this special? Well it is a new medium for art, a new way to distribute art. Artists are able to do things they couldn’t do with prior art forms. For example, there are some works that are coded to literally change the time of day; there are pieces that morph over time; some of the artists built mechanisms into the work that would change the work automatically at a future time…at the end of the day it’s just art. There’s a community aspect to this art. All of the artists in the project interact directly with their community, their fans, and collectors. There is no middleman. There are no traditional rules. There needs to be collaboration, direct connection, dynamic movement, and that’s what makes it special to me. Being on the blockchain is a means for it to exist. And then there is the “smart contract.” That’s a huge part of what an NFT is, the actual code you are gaining access to in this token. This contract protects not only Bowie, but artists and the NFT community from being exploited. Everything was a 50/50 partnership between the artists and the estate, and 100% of the profits went to charity. Launching a new transmedia art project Tell me about Kids of the Apocalypse (https://www.kidsoftheapocalypse.com/landing.html), another far reaching transmedia project that spans across music, NFTs, music videos, a comic book, and film. Here is a quote from the Kids of the Apocalypse Discord channel that sums up the project really well: In the aftermath of a cataclysmic bio-explosion, a social movement of revolutionaries is born to break the chains of the tyrannical rule of Horizon Corp in the wake of the apocalypse. A multimedia art project born from the music, Kids of the Apocalypse (KOTA) aspires to highlight the themes represented on the journey of our - awakening, unity, and deposition of unjust power. Positioned as a multimedia IP with strong experience and connections in the music industry alongside a world-class design team, KOTA is a love letter to those who dare to speak out and be themselves - delivered in the form of a suspenseful, emotional, and immersive sci-fi adventure. AK: Kids of the Apocalypse was a concept and story that was conceived about 10 years ago. There was a Swedish production group called The Sound of Arrows (https://www.youtube.com/watch?v=wxmQPzVqk94), and Stefan Storm from the group had started this kind of side project idea Kids of the Apocalypse. Imagine a dark comedy version of—I’m gonna say X-Men because it involves mutants—that is alt and internet leaning and kind of self-referential which also has a music component. There’s this whole story and a really immaculately created lore and universe. And now imagine 10 years ago saying let’s make this happen. Where the do you begin? So this idea sat there for a long time. So cut to 10 years later and I’m reconnecting with my friend Derek Davies, who was the founder of Neon Gold (http://neon.gold/), which was a label that I had actually done a label deal with when I was at Columbia and who had signed Sound of Arrows and put out their first EP. He was still in touch with Stefan, and Derek is also doing a lot of stuff in the Web3 space. He has an incredible company called Medallion (https://www.medallion.fm/). I think we have put together a pretty incredible team—we’ve got Derek and the Medallion crew, we have Stefan and his crew, we’ve got an incredible creative agency who have been working on all of this alongside Stefan. And we’ve got the guys at Bench Mob who are some of the best digital architects out there from a social media strategy perspective. We’ve got an incredible community manager, we’ve got a advisory board of people who really are some of the best and brightest in the space from Cooper Turley on the music NFT side to John Roger, who’s one of the advisors and was marketing for Star Wars for Lucasfilm and then became the first head of franchise development at Disney. We have released a few songs and one video, which I think might be the number one (I hate data points because they get outdated quickly so I might be wrong here) most traded piece on Glass (https://glass.xyz/) which is a music video NFT protocol. We’ve also created something that I think has never been done before, which was creating a mechanism that integrates traditional streaming with NFT minting. Where in order to gain access to the allow list you actually had to interact with and log yourself into your Spotify or Apple account. This is a very blockchain-native project, but I also want everyone to be able to hear the music, even if you don’t know what an NFT is, because the music is fantastic. The first drop (https://opensea.io/collection/kota-kids-of-the-apocalypse) is a PFP (profile picture) project. It’s season one and includes the eight main characters, which you’re randomly assigned to and each of them has its own unique theme music and properties, and it’s all happening on the Blockchain. We’re doing all this on Solana (https://solana.com/), which is a chain that I think allows for very quick movement. And part of what we want to do is really involve our community in the storytelling and really let them into all of the bits and pieces of the lore. And to me, a lot of that will come from surprise and delight and from airdrops and from opening up your wallet one day and seeing things you didn’t know were going to be in there. For those of us who are still new to this space, what is a wallet? AK: The wallet is the digital space through which everything is integrated and enters into your possession. Your wallet is where you hold your cryptocurrency, where you hold your NFTs. So you may log into your wallet one day and if you’re a holder of Kids of the Apocalypse NFT, you may find something new that has been added to your wallet. We’ve also created an amazing community in Discord, and the fan art alone is pretty incredible. I encourage people to check it out because we really are trying to build something unique. I think the website (https://www.kidsoftheapocalypse.com/landing.html) is one of the coolest websites I have seen, it is fully immersive, and we’re going to play with it. So how will the comic book and full film emerge? How is that all going to be integrated? AK: The full-length film is a dream. It could be a short film too, we don’t know. The plan is for this to be a story told in a moving visual media. We’re still having discussions about how and what that will look like. Before we stop, were there any other current projects that we should be aware of? AK: Yeah there are two things. After the Bowie project I felt that it was important for me to put myself out there more. I felt that there was an opportunity to experience what artists and creators in this space go through, their step-by-step process. My father was a photo journalist, and I’ve always loved photography. I had this idea going back to the idea of the blockchain as a permanent immutable ledger; and thinking of photographs, particularly snapshots, as as memory objects; and what if you minted those memories to the blockchain, and then they’re there forever. Now what happens if you then renounce your ownership of them? If you’re using the snapshot as a representation of a memory, what if I minted it, and sent it to you and now it’s yours? What does it mean for someone else to have my memory? It was an idea that I kept coming back to and I finally felt the need to bite the bullet and just put this out there. It’s called Memory Loss. I spoke to a few artists in the crypto space to see if the idea offended anyone. I got their support and encouragement. I minted some of the snapshots and made them available on the Tezos (https://tezos.com/) blockchain, which is a space that is more experimental, low key, less pressure. It’s on the marketplace Objkt (https://objkt.com/). I do have ideas for future ways for this to evolve. The other project, equally experimental, gives support to visual artists who want to start playing in the music NFT space. There are some artists making really amazing music and building a fan base and building patrons and building their own worlds with their own rules in this space. I wanted to create a way for visual artists, who are curious, to safely and with integrity and respect, integrate themselves into that world. The short of it is I am launching an NFT-based singles label. The name is, W3 F3W like my company, except that the ees are 3s. It’s going to be a place for great new music and great visual art to collide and hopefully again, just bring people into a world that I’m pretty excited about. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit futureperfect.substack.com (https://futureperfect.substack.com?utm_medium=podcast&utm_campaign=CTA_1)

2. 

   7 Jul 2022

   #005 - Nick Fortugno: Storytelling, Design Strategies, and Interactive Theater

   Welcome to the FuturePerfect Podcast where we talk with compelling people breaking new ground in art, media, and entertainment. This podcast is produced by FuturePerfect Studio, an extended reality studio creating immersive experiences for global audiences. Episodes are released every two weeks, visit our website futureperfect.studio (http://futureperfect.studio/) for more details. The text version of this interview has been edited for length and clarity. Find the full audio version above or in your favorite podcast app. For episode 005, Wayne Ashley interviews Nick Fortugno, co-founder of the New York-based game studio Playmatics (https://playmatics.com/) and designer of numerous digital and non-digital projects, including board games, collectible card games, large-scale social games, and theater. INTRODUCTION AND ROLEPLAYING Hey Nick, thanks for joining us. I’m really excited to dig into some of your background, ideas, projects, and particularly your alternative vision for a future of theater. I see you as a catalyst, a kind of cultural interlocutor making links across different forms of knowledge and practice, and the work you've done really attests to this. You've designed video and board games as well as outdoor public games. You're the co-founder of Playmatics, a New York game studio and the lead designer on many theater works, including Frankenstein AI (https://frankenstein.ai/) and The Raven (https://inculture.microsoft.com/people-who-inspire/raven-immersive-iot-theater/). And of course, one of the lead creators of the blockbuster mobile game Diner Dash. But first I want to go back a bit. Your cousin introduced you to roleplaying when you were quite young and you ran your first game of Dungeons and Dragons at six years old. Is it too much to assume that roleplaying is one of the most critical activities for you, if not a central organizing practice leaking into everything you do? Give us a sense of how roleplaying has activated much of your thinking and practice. Nick Fortugno: I think a central organizing principle is like a good way of thinking about it. It doesn't inform all of my work in a literal sense, but it's the heart of how I think about aesthetics. In Dungeons and Dragons, essentially what you do is you tell stories with other people and you use a rule system to adjudicate disagreement. You have a lot of “I hit you”—“no you didn’t” stuff in roleplaying so you need rules to deal with that. When you’re storytelling in that system and you're the person responsible for making the story, you don't story-tell the way you do in other forms where you have an idea of the story in your head and you're figuring out how to implement it in a way that will affect the audience. Instead, the players or the protagonists are interacting with you and they're changing it constantly. And so you don't know where the story is going. You have ideas of where you could go, you have ideas of what you might want to happen, but you're really in this collaborative process. And so this idea of improvising and using systems to generate things and being responsive to the interactions of other people is very much at the heart of my work. It's how I teach, how I think about storytelling centrally, and it informs a lot of my aesthetics. So yeah I would not be the person I was today if my cousin Joey didn't teach me D&D (Dungeons and Dragons) in Glen Ellyn, Illinois. DESIGN THINKING You're also a prolific researcher, not only of games, but of literature, theme parks, new technologies, and performance. I'm thinking about a previous discussion we had where in one breath you mentioned cultural forms that most people would never bring together in the same conversation. The list is long, but indulge me here: the British theater company Gob Squad (https://www.gobsquad.com/), Galaxy’s Edge (https://disneyland.disney.go.com/destinations/disneyland/star-wars-galaxys-edge/) at Disneyland, Harry Potter hotel, the theater collective The Wooster Group (https://thewoostergroup.org/blog/), the blockbuster event Sleep No More (https://mckittrickhotel.com/sleep-no-more/#/), the novels of Joyce and Pynchon, Evermore Park (https://www.evermore.com/) in Utah, and the epic video game Elden Ring (https://www.youtube.com/watch?v=AKXiKBnzpBQ). This cluster excites me because it’s how we think as well, across these kinds of groupings. You also use this concept of affordances to enable you to think systematically across all these activities. Can you say more about that? NF: Affordance is a concept from design thinking, Donald Norman (https://www.youtube.com/watch?v=CkPQj7Y6-gc) really popularized it. It's the idea that a form has features about it that lead to certain kinds of use. There are things that are intuitive in a way, or natural in a way, that come from a form. If I put a handle in a certain place, you hold the handle and that changes your use of the device. That idea that the forms start speaking to certain kinds of use cases is very central to thinking about interactive design. Because when you're a designer in those spaces you make the affordances. You don't tell users what to do. You give them something and you have them do it. That's why it's interactive. It’s not like a roller coaster where I strap myself in and I just ride the rails that were put out in front of me. It's more like a theme park where there's just a bunch of stuff. But I don’t go wandering off into the most boring part of the theme park. I go towards the lights, I go towards the sound, I go towards the interactive things. The design of those things that attract me, the things that challenge me, the obstacles and the rewards, all of that stuff moves me around in those spaces. This is central to the way I think about my practice. LITERATURE, PLAY AND AMBIGUITY You have a BA in graduate study and literature. In our previous conversation, you noted an overlapping relationship between post-war American literature and the kinds of interactive narratives found in gaming. Do I have that right? In our other podcasts I've been really interested in what brings disparate people to these emerging hybrid media spaces. They come from film, dance, theater, visual art, and gaming. I think you're the first person in our podcast series making connections between Pynchon and James Joyce with interactive gaming structures. I'm curious about how you came to make these connections. NF: When I got interested in literature I was drawn to postwar postmodernist approaches to writing, like I’m thinking fifties, sixties, and seventies. But really you could stretch it from a Borgesian and Joycean and Steinean space up through the modern day. There's still authors like Ali Smith doing stuff like this. But when you look at like things like Pynchon and Nabokov in particular, their works start becoming a little bit obsessed with interpretation. Interpretation becomes the center of the novel. The novels become games about interpretations. There are other authors in that space who are really breaking down the sense of what you're supposed to consume from the story because they are, in a meta way, thinking about the fact that you're interpreting them. Whether it's Crying of Lot 49 asking you to think about what communication systems are and then challenging you on how we interpret conspiracies. And that's also all over Foucault's Pendulum. Or a book like Lolita, which is basically laughing in your face about your attempts to understand it. Or Pale Fire for that matter, which I think is an even deeper experiment. What you see over and over again is this idea that the novel is a game that the reader is playing with the novelist. It's not a puzzle. You're not going to get the answer out of it. That's not the point. And certainly postmodern poetry and people like Asbury would argue that if you got one meaning out of a poem, you didn't really read the poem anyway. The work becomes something that you as the audience have some ownership of because it is open to you and because it’s an ambiguous object that you have to work with. That’s what got me. I was already, just from roleplaying, very used to the idea that I participate in stories and that they come from this relationship with me and the text. So I don't like talking about interactive narrative. I think that's a bad phrase because I I'm always interacting with story. That's not new, what's new is the types of affordances of interaction that I get from stories, and what the possibilities for changing those stories are, and how much the story is a fixed thing that I encounter, and how much the story is flexible to my input. To me, the literature study was partly just giving me an outlet for stories and a place where stories can actually be quite experimental because when you just write it's cheap to make crazy worlds. It's the same amount of ink to write a crazy world as it is to write a realistic one. You can go very far with literature in a way that would be harder to do in film because you have to shoot all that stuff. The drive of novels from the modernist period on has been a drive towards more and more stylistic experimentation and that has been really engaging to me because you start seeing it as almost a formal thing. You can look at it like a structure and then you can see that the structure is doing something. Joyce’s Ulysses is an excellent example of that. Each chapter is written stylistically and formally different. There are chapters that are dialogues, there are chapters where the stream of consciousness changes radically, there are chapters that drift, and that's part of the narrative. If you go back to the Oulipo experimentation that Calvino and other French and Italian authors were doing, they were literally creating that whole idea of branching trees. You start to see that there are patterns of structures of story that we can start to establish. That's the approach I take to this question of rhetoric. Exploration is a set of tropes, and branching is a set of tropes. It's similar, whether you're branching in a YouTube video or branching in a choose your own adventure, or branching in a game like Until Dawn (https://www.youtube.com/watch?v=3NcF7EOnjow). The branching is similar, it has similar tropes. So we can look at it structurally and say, well, what does the structure do? How do the choices in the design of the structure change things independent of content. And then what is the intersection between the content and the structure? DYNAMIC STRUCTURES AND GAMES It's interesting to note how the strategies found in avant-garde and experimental literature have leaked into, or have become one of the dominant ways of constructing narrative within popular culture, video games, and even marketing. What was on the periphery has, in a sense, moved to the center and become part of the entertainment industry. NF: I think so because as you start moving into more dynamic and particularly digitally dynamic work it starts to have to be structural. Although that spills back into the analog, especially as internet of things (IOT) becomes very reduced in size and cost and technology starts coming back into the real world. You start seeing this there too. I'm riffing a lot on arguments in a book called Expressive Processing (https://mitpress.mit.edu/books/expressive-processing) by Noah Wardrip-Fruin. If I make a piece of work that changes with every user and produces a different outcome, then the output of that work is not really an analysis of that work. If the work has a hundred thousand possibilities, one possibility is such a small segment of what it could be. That it gives me information as a user, but I can't really critique the work from that perspective. I have to look at the structure because it's procedural, it's not predetermined. And I think as we start moving into works that are like that, and since computers enable us to do that, that's what computers are good at is that kind of dynamic procedural, then we start to see that structural analysis and system design become more and more important. As it does, and we see the affordances that has, we can start pulling those affordances into other forms where we see similar audience relationships. So I don't think: does theater need this? Does film need this? Does installation need this? No, It doesn't need it. You can make good art without it, and obviously we have made thousands of years of good art without it, but the possibilities of the art change when you start seeing those things. That's why I think it’s starting to permeate. Digital games are a very big industry and there's been a lot of really interesting storytelling in them. I don't think all people who study this stuff know that because it's locked a bit behind barriers of picking up a PlayStation 4 controller and trying to get through it. Shadow of Colossus (https://www.youtube.com/watch?v=pdZQ98mWeto), for example, is one of the most important digital works ever made. But not many people experience it because it's a really hard digital game. And it has to be hard. That's part of its aesthetic. But I think that the people who have bridged this are starting to see that you can inherit things from those forms into these other spaces. That's just changing the way we think and then you start to see work in the world that is just more procedural. Work that does just become more dynamic in its nature. Then you end up with stuff like LARP (Live action role-playing (http://Live action role-playing game)) where, you can't make LARP the way you make theater because I don't know what the players are gonna do. So my scripts in LARP can't be like a theater script, it doesn't make sense. I need a structure that will support 40 people running around doing random things. PARTICIPATORY EXPERIENCES DRIVEN BY TECHNOLOGY This brings me to theater, particularly two participatory theatrical installations that you co-created. First, Frankenstein AI: a monster made by many (http://frankenstein.ai/) which was an AI powered immersive experience that premiered at the 2018 Sundance Film Festival. And The Raven (https://www.filmlinc.org/nyff2019/films/the-raven/), which was performed as part of the Lincoln Center’s New York Film Festival in 2019. Tell us what audiences might have experienced when they participated in Frankenstein AI and what was the genesis of that work? NF: Frankenstein AI has had a couple of different forms. Its original form was a small audience immersive experience where you came into a room and you interacted with another audience member at a surface computer that was like built into a table. It was formulated as an artificial intelligence asking you questions about what it was like to be human and you're sort of marking values on the table using a physical computing device that looked like an ouija board. That information was sent to an actual AI that was in a cloud which was used as the seed to determine a mood that the AI had. And then when you finished that exercise, you were brought into a room that was mapped with projections and IOT procedurally played drums and you would have a chance to talk to the artificial intelligence. The artificial intelligence would generate a question and then it would be delivered in text to speech to the audience in the room. And then the audience in the room would direct the docent to type a question into a typewriter and that would be sent back to the AI. This was all formulated where there’s this AI that's been created, it has escaped into the internet and it is trying to understand what it is and what humanity is. And it’s using the narrative of Frankenstein as this thing that was created that doesn't understand its role as a seed to understand where it's going. The whole thing was essentially a meditation on two things. One is this question of what is AI and what should we be worried about AI? These were the conversations that I had with Lance Weiler and Rachel Eve Ginsburg who were the co-creators of that project. My big argument was that everyone worries about Terminator, but what we should really be worried about is Kafka. AI is not a monster that takes us over. AI is a thing that doesn't understand us and then just acts procedurally in ways we don't understand. This is around the time that Microsoft had released an AI that became wildly racist (https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-revealed-the-dangers-of-online-conversation#toggle-gdpr) and we were thinking about what it meant that we're teaching AI and how could we make a piece that gets people to reflect on the idea that we're engaged with artificial intelligence in the world? We are training it and we are going to teach the AI what it does. So if that's the case, what is our responsibility? The whole piece was kind of a meditation on that process. I did the creative technology design on that and some of the interactive narrative design of the sequencing of it. I'm very proud of that piece personally, because it was the first piece of creative technology that I ever actually showed in an exhibit. I worked on the technology that connected all of devices. So it meant that when the AI changed mood, the projections changed, and the drums changed and it pulled the AI's response and then fed that into the speech to text and delivered it into the room. So I basically did the technology that connected the surface tables to the AI, to the projectors, and to the drums. This was a topic of research I've had for a long time about how technology could be used to create these like kind of seamless connections between things. You didn't see anything happen, you just asked a question and suddenly the projections and drums changed. I call that seamless technology—technology that doesn't have clear lines where it connects. I think that could be a kind of magic and that was important to me. What did you learn from producing Frankenstein AI that changed your approaches when you then began to develop The Raven? How does The Raven work as an experience that grew from or built upon your previous work? NF: The Raven was an immersive performance where we allowed an audience into The American Irish Historical Society where they experienced a magically real story of Edgar Allan Poe’s The Raven. The center of the technology of the piece was that every user had a lantern that they carried around with them. The lantern was an IOT device that was reading beacons in the space and connected to a central system. The audience also had a set of headphones that were playing audio for them. So most of the audio that was present in the piece came from the headset that was being played based on where they were and based on a character they picked at the beginning of the piece. Everyone was sort of playing a performer in the piece. The performer Ava Lee Scott, who was playing Poe and co-wrote the piece, was moving through the space as Poe meditating with these characters. But you, as the audience, were one of the people that Poe knew from his life or his creations. What Lance Weiler and I carried from Frankenstein AI was this idea that we could create a central technology system that was guiding all these users without having to have actors on top of those users moving them around. And that the storytelling could really be based on their decisions, because it was in part based on where you went and what you encountered. The other thing that Frankenstein AI taught me, in a real sense, was that these technologies could be stable. The work had a server system, that's how it ran, it was a server that was running on a small piece of technology called the Raspberry Pi (https://www.raspberrypi.org/). We turned it on and on the first day when we were running it we just didn't turn it off. We wanted to see if it would stay up overnight. And then we didn't turn it off for two full weeks. It just ran nonstop for two weeks and it never broke. We never had to restart it. So that taught me these things can be made battle ready. We brought a similar kind of technology to The Raven. There were obviously different technical constraints to The Raven and there were different bugs we were facing, but we went through a similar process of creating a central system that guided the narrative. If we do that right and we have the right affordances to connect to the audience that can take the place of a bunch of docents, a bunch of rules, a bunch of structures, and people can just explore. Then through that exploration they can find story. I should say that we worked with pretty robust technologies on that project. We were in partnership with Microsoft and we were using pretty heavy Azure servers and things like that, but it was not for heavy lifting stuff. It was for reliability of the delivery of the material. And then we built this gigantic XML file that was the branching script of the entire piece so that we knew where people were. We could time lights and sound cues and things like that. THE LIMITS OF THEATER What I find compelling about both of these projects is their capacity to posit alternative models for theater's future. They either directly or implicitly suggest that theater needs to be remediated or fixed. For the purposes of this discussion, can I make that assertion? NF: Yeah, I will also defend traditional theater, but… [laughs] That’s good [laughs], but what is it about certain kinds of theater that need to be remediated and how are your explorations accomplishing this? I’m very careful to say alternative models and I’m not asking you to generalize. I think from our audience's perspective, people are going to ask: what’s wrong with the kind of theater that I do? And why do I need these other systems? Why do I need to even consider these technologies? All these kinds of questions are implied, for better or worse, in the kind of work that you're proposing and the kind of exciting research that you're carrying out. NF: First of all, there's just aesthetic possibilities that are very hard to create in a linear format like theater. Guilt is hard to create in an audience. Triumph is hard to create in an audience because they don't do anything. You can get to shame, but there's types of shame you can't get to. So there's aesthetics that become possible just when someone is culpable and when someone has the ability to achieve. That becomes kind of interesting. Games have lots of emotions attached to victory and failure that can be leveraged in all sorts of interesting and weird ways. There are pieces like The Privilege of Escape (https://www.risapuno.com/the-privilege-of-escape#:~:text=The%20Privilege%20of%20Escape%20was,room%20before%20time%20runs%20out.), which was an escape room that was a meditation on systemic bias. That's an interesting example of a piece where the designer was trying to use the affordances of games to demonstrate a problem in the world. And games typically do that. There's just pure emotions that are inaccessible to linear media. I think because there aren't affordances for the audience to access them, despite the diversity of emotions that these forms can create. The second possibility is, it’s a question of how you want to engage with your audience. As an artist, I don't really like telling people stories, that doesn't really engage me. You're the second person we've interviewed who has talked disparagingly about stories and storytelling (https://futureperfect.substack.com/p/001-krzysztof-garbaczewski-digital#details). Say more about that. NF: I don't mind being blunt about this. I'm not that interested in my biology. I'm not that interested in my history. I don't find those things that interesting. I don't think I have a vision of storytelling that's so powerful that some muse came to me uniquely and now the word of heaven is coming through my body or something. And this isn’t to knock people who do that, there are geniuses who make that work, but that's not how I create and that's not what I do. What I want is to play with you. I want to be able to engage with you and you know, catch the ball you throw and throw it back. And this isn't altruistic just to be really clear, I mean I like doing that with people, but it's also really fun to catch a bunch of balls coming at you in crazy directions and keep the whole thing on track. There's an artistry to that. That's what running an RPG is, it's like throwing track in front of a moving train. So I think that's really powerful and you get things that you would never get otherwise. Similarly, if you jam you get something that you would never get when you compose. The improvisation and the participation of other people leads you to create something new and you can do that with audiences. And you can do that with audiences in ways that don't make crappy, thin, gray, over-democratized work. Because I'm not saying that's not a problem, if you just let everybody come in and cook in the kitchen then you get no food or you get bland food or inedible stuff. Structures make it possible for people to participate in ways that are meaningful, but controlled, that fit within the aesthetic. So people understand what kinds of creations are possible in this space. And that is a whole set of techniques that then allows audiences to come in completely ignorant of what you're doing and then tell a story that they helped make that is still in the aesthetic you wanted. There's a magic to that that I think is really powerful. It opens up whole new kinds of forms and it's a different way of engaging with the world for the audience and I think that's powerful because we haven't really seen it before. There are some experiences like that, but they tend to be very high demand on the creativity or they tend to be gate-kept or they’re high skill-based. And what immersive theater can do that I think is unique and independent of digital games and LARPs, is that they can be approachable. I can show up and not really know much and still participate. And I think that's a space that's really powerful. And then the third beat that I just have to mention all the time is that tickets are very expensive to these things. They charge a lot of money to get people into those things. I think that there's opportunity, from a business perspective, if you can figure out the scaling. You're seeing pieces like Particle Ink (https://www.particleink.com/about) in Las Vegas which is a piece with projection mapping and dance where they're starting to figure out how to grow the audiences in ways that don't hurt the piece. You start looking at genuine business models for keeping those things up. What are other business models that can keep dancers, actors, and set designers involved? Because none of those people are going away in immersive theater, we need all of those people. We need them the same way we need them in other forms. It’s a parallel skill if not an identical skill right. So we're not telling actors they're out of work. We had actors in The Raven, the actor was the center of The Raven in a lot of ways, but the actor was supplemented by all of these other things to create a new form where people can explore and make choices and feel directly engaged. NEW FORMS OF PEDAGOGY Given this technologically seamless environment within which performance might take place, do you see the training of actors taking a different path? Or different ways for how writers produce scripts? Do we need new kinds of training for scenographers, sound and lighting designers that will accommodate and respond to these ideas and new approaches to performance? NF: Well acting, for example, in these kinds of cases, has a lot more improvisation in it. It's much more deeply based in that kind of improvisation, but it's also a lot about vulnerability. This is something that I'm just going to riff off of a writer and actor that I know Char Simpson would talk about. Char was part of the Blackout Haunted House (http://www.theblackoutexperience.com/)for many years and talks very much about how they created vulnerability and that the creation of vulnerability was really important. That becomes a different way of thinking about acting. But also the idea that an audience member might ask you your favorite color and you need an answer that seems natural. That's a more roleplaying kind of acting than I think some actors are trained in, of course some actors are good at that. You don't know what's going to happen so you can't write a script the way you would normally write a script. It has to have some variation in it. You have to think about it more like story, like world building. I think directing changes because I don't know when we're gonna hit a specific moment or I don't know what perspective I'm gonna be coming from in a specific moment. So I have to think differently about that too. And you see that in digital games which will sometimes have cut scenes that are very film-like, but they'll also have scenes where users can walk around and watch what's happening. Which is why when we talk about VR we talk more about immersive theater because the viewpoint is not singular, it is a multiple viewpoint environment. So I'm thinking about it more from that perspective. Theater in the round is also relevant here. Again, that's not a new form, but it solved this problem. So maybe VR should look at theater in the round and then learn some lessons for how you keep an audience's attention in a broad space. And in fact, we're getting that big, we could think about station-based theater where people are really just drifting over a whole plaza and engaged in an experience. Are these forms going to change acting, writing, directing and set design? Sure, of course they are because the affordances of the audience are going be different and that's going to lead to different outputs. But it's not like we made up all this stuff just because the technology came along. We had happenings, we had station-based theater, we had rituals. I'm thinking about the Ramlila which I participated in India many decades ago in Varanasi. This is a month-long event that is played out over the entire city in which the inhabitants take on all the various roles. The city performs and becomes an immersive ritual and religious space. So there are absolutely precedences that are centuries old that we can draw upon. I’m thinking about how the pedagogical needs of theater will continue to change in response to these new forms that are becoming more and more central to our lives. NF: Yeah I teach immersive and dynamic narrative and I teach it in the way that we’ve been talking about. I teach it in this very broad, cut-across-media way. Media does not matter for the purpose of the class, that's not what it's about. It's about the tropes that the media use and how those things relate. And then you see this in disciplines like narratology where people are really coming at narrative from lots of different directions and trying to figure out how stories get told. Another point that's just very important to me is in the intersection of these forms. Because you're not going to get immersive theater from theater alone. There's a bunch of pieces that theater doesn't really know about like interaction design and a sort of multiple viewpoint about the pacing for that kind of stuff. Games understand that, but games don't understand what theater's good at. Games don’t understand how you create scenes or understand how you create dramatic power, and games don't understand the value of liveness, frankly. Some of that we can get from LARPs, but LARPs aren't theater either. So it really is in the intersection of all of these fields. I think more of this is happening. You're seeing escape rooms get more theatrical. I think it's too slow, like way too slow. We could have gotten to where we are five years ago and we could be five years ahead of where we are right now. But you're starting to see some of that thinking happen. You're starting to see immersive pieces that are bringing some game elements into them. You can have conversations with people about VR where you talk about digital games and they don't scoff. This focuses again on the ideas of interaction and affordance and how those relate to storytelling that changes the orbit of everything. And then the skills that people have been learning, like the acting, writing, directing, set design, costuming, they all have a place. They're all going to be there, they're just going to circle around a different sun. And that sun is this audience member who can change what you do. That's different. Nick, thanks for all of the conversations we've had. I look forward to working with you. I think you're a really important thinker and maker, and your experiments and research bring a lot of insight into the future of performance. NF: Thank you, I appreciate that there are people like you that are thinking about these problems and working in these problems. Like with your own wonderful work and that podcasts like this exist to have these conversations. I look forward to a really bright future because there's other people like you in it. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit futureperfect.substack.com (https://futureperfect.substack.com?utm_medium=podcast&utm_campaign=CTA_1)

3. 

   2 Jun 2022

   #004 - Auriea Harvey: Early Web Art, Video Games, and XR

   This is the FuturePerfect Podcast where we talk with compelling people breaking new ground in art, media, and entertainment. This podcast is produced by FuturePerfect Studio, an extended reality studio creating immersive experiences for global audiences. Episodes are released every two weeks, visit our website futureperfect.studio (http://futureperfect.studio/) for more details. The text version of this interview has been edited for length and clarity. Find the full audio version above or in your favorite podcast app. For episode 004, Wayne Ashley interviews Auriea Harvey (https://auriea.art/), a prolific artist producing simulations and sculptures that bridge both physical and digital space. Over the past decades she has produced net.art, online performances, video games, and sculptures that blend digital and handmade production. Harvey's work can be found in the collections of the Walker Art Center, San Francisco Museum of Modern Art, Contemporary Art Museum of Luxembourg, and Rhizome's Net Art Anthology. Her video games (https://taleoftales.itch.io/) and VR projects have been exhibited in venues all over the world. You and I have such a long overlapping history going back to the mid nineties with the emergence of net art. I was extremely inspired by your work, which was so personal and tactile, specifically your online journals composed of these sumptuous collages, poetry, photography, drawing, painting, and 3D sculptural elements. Already, one can see how passionate you were about creating dialogues between analog and digital production, which I completely connect to. There was no hierarchy between these two. When I look at the extraordinary depth of work that you've created over the past two and a half decades, I can easily tease out a history of the internet and digital culture with all its promises of emancipation and boundless creativity, as well as its many discontents, which we'll get into shortly. But first I, I want to go back to the beginnings of your practice. You studied sculpture at Parson School of Design before learning web design, and then founded the game studio Tale of Tales (https://tale-of-tales.com/). How did you make that leap from sculpture to net art? Auriea Harvey: I would say that I not only studied sculpture, but I also studied design to a certain extent. Although I was a bit of an autodidact for a long time, meaning that my main skill was computers in addition to sculpture. So the fact that I was so passionate about computers really led me directly into this confrontation. Well, okay love of computers and lack of space, being a young person in New York City in the early nineties. And when I found the internet, it immediately struck me that everything I could do online was a sculpture. You know, it was a time of broadening these definitions of what a sculpture could be. There were people who were asking is video sculpture? Is installation sculpture? It seems obvious now, but at the time it was very much a question. And so I looked at the internet and said, is the internet sculpture? I began seeking the ways in which the internet was sculpture. And in some ways that sculpture was social. In some ways that sculpture was this multimedia and interactive landscape that was totally unexplored. And that was really interesting for me. Out of this you started doing online performance. Back in 1999, we brought you to Brooklyn Academy of Music to perform Wirefire (http://www.entropy8zuper.org/wirefire/i.html). 23 years ago you were already thinking about the internet as a place to do performance. Can you talk about this? AH: Wirefire was very much a realtime performance. Michaël [Samyn] and I met every week, we had started it before we even lived together when I was still in New York City and he was in Belgium, and we created it as a way to communicate with one another. If you rewind and think back, the only way you could really talk to someone back then through the internet was via text. There was no video and there was no audio really. We thought text was completely inadequate and both of us, being very adept at internet languages, decided to create this system where we could communicate with each other through anything. We could upload sounds, animations, and could have real time chat. We could also invite the audience and offer them a way to interact with the performance and everyone could see it at the same time. And this was something that was kind of unheard of in 1999 or very rare. We did this from 1999 until around 2003. The site is still there and it sort of has a documentation of all the places where we also took it live. After he and I started living together, we started doing these things live also because it creates a big spectacle. It’s something that we did that I'm really proud of. So with the emergence of Web 2.0, you stopped making this work. What happened? AH: Well, the beauty of the early web or Web 1.0 as people will say, was that it was this very big time of innocence. I would say it started with blogging, that was the beginning of the end. That was like the end of it all for me. It felt like it was taking away the power of computing from people. It did open up the web to a different subset of people, but I felt that those people should learn how to program HTML, I felt like this was very empowering. I could see that this was slowly eating away at people's ability to see the computer for what it was, which was an open a box of tricks. And you could pull out any of those tricks and use them in any number of ways. Slowly over the years, indeed, you've seen this closing down, you know, to the point where now websites have a hamburger menu on the side and there’s expected to be good navigation (I’m making air quotes). Whereas we were all about crashing the browser and making people think. Of course there were bad actors who took advantage of the freedom that the web had to offer, but it seems to me like there were better solutions than what Web 2.0 turned out to be. It became much more templatized as well. AH: Yeah it became more templatized, people are basically ignorant about what they can do with their computers now. It's not about computers, it's about phones. It's not about computers, it's about using services. It's about subscriptions. It's not about building anything, you know? Only a few people still build, and it’s made more and more complicated through the way corporations have controlled the computing environment and the internet and our interactions on it. Back then I could see that coming, let's say, and I was like, nope we're outta here, let’s do something else. I want to read something from a manifesto (https://tale-of-tales.com/tales/RAM.html) you wrote with your partner Michäel Samyn in 2006. I think this will form a kind of way for you discuss your whole new transformation into working with gaming. Realtime 3D is the most remarkable new creative technology since oil on canvas. It is much too important to be wasted on computer games alone. This manifesto is a call-to-arms for creative people (including, but not limited to, video game designers and fine artists) to embrace this new medium and start realizing its enormous potential. As well as a set of guidelines that express our own ideas and ideals about using the technology. It is much too important to remain in the hands of toy makers and propaganda machines. We need to rip the technology out of their greedy claws and put them to shame by producing the most stunning art to grace this planet so far. (And claim the name “game” for what we do even if it is inappropriate.) I love hearing that. What did you want to accomplish with this manifesto? And why did gaming suddenly become a compelling arena for you to explore and experiment? AH: Well, we really saw video games as an interactive art form. But this was 2002 or 2003, and so video games didn't know that yet. But we just looked at it and we’re like, this is interactive. People spend hours 20 hours playing a video game, and you can't get that with a painting [laughs]. We had been playing a lot of video games and we didn't understand them at first. We played them and questioned why they were doing this with the technology? It's as if we were visiting an alien planet and we could not compute why this was the only thing that was happening—RPGs with random battles, fighting games, driving games, adventure games. There were several genres that you had to fit into in order to sell a video game at that time. There were, of course, exceptions. But this was pretty much the world we were walking into. So when we gave that manifesto, which is called the Realtime Art Manifesto (https://tale-of-tales.com/tales/RAM.html), we really thought that the most remarkable thing here was that you were making something that, like the internet, allowed for realtime communication. People could be inside a world when you played a video game, you were completely lost in it. Now you would look at certain video games and you wouldn't understand what was so special about that world, but at the time when you played, and even now when you play video games, of course it's like being inside a book, but more real, it feels real. That was what was important to us. It was something I had experienced with early VR, for example, but more so in a certain way, because it was these works of imagination. Now our problem was the imagination that we saw within video games seemed extremely limited and we wanted to be able to use it for ourselves, but also encourage others to look at video games as something that was wide open. That was, again, that box of tricks that you could just do whatever you wanted with. Literally it's like, come on creators, you can do anything with this, you can make any world! And we were some of the first to really make a point of this. There was an undercurrent in game studies at that time in 2006 trying to point this out, but there were very few examples. So we really devoted ourselves to creating that example and encouraging other people to change their thinking around video games. And we threw in that last part “even if the word game is inappropriate,” because people were eager for us to name this change. There was a big discourse at the time around: what is a game? We thought that was the most boring question ever. We were like, we're not doing that, we're not talking semantics, this is real stuff here. So we said it's a video game and that’s it. So we started our little journey there, with the manifesto, and it was quite a controversy, at the time, to say these things. What was controversial about it? AH: Well, the thing we had to learn about gamers was that they quickly feel like you're coming for their stuff. There was a lot of discussion back then, and maybe there still is, about “do games cause violence?” Everyone was kind of down on gaming, saying it’s for kids, or it’s dangerous. And you had the US Army with their recruitment game. People were testing the limits, both psychologically and aesthetically, of what a game could be. And so gamers could be very touchy about this subject of “what is a game?” We never tried to take anything away, we were just trying to add something to that, but gamers often were feeling sensitive about [laughs] their Mario. They couldn’t stand it if you dared to disdain these types of games. And we disdained a lot. But out of that you produced several games and one of them, The Endless Forest (https://tale-of-tales.com/TheEndlessForest/), was, according to your words, one of the most successful games that you've created. It continues to circulate and you're currently developing a new version with Unreal Engine. Tell us a little bit about what it’s like to play or experience. AH: We released The Endless Forest back in 2006. It was a multiplayer game where everyone plays a deer in a forest. It was ultimately meant to be this very peaceful gesture at a time when everyone was playing World of Warcraft (https://www.youtube.com/watch?v=YEvW64bTeBk). We made this game as sort of the antidote to that. It was something we felt people who played World of Warcraft could dip into for five minutes. The thought of playing a game for a short amount of time in 2005 was rare. The fact that people would play games for hours and hours was what drew me to games. But at the same time, once I got there, it felt like people need experiences that they only play for 5 or 10 minutes. They can come to a world that’s always there for them, but they go there and it's not about killing. It's not about points. It's not about gaining anything. It's just about being there and feeling it. And so when you're in The Endless Forest, you're there and you feel like an animal, you feel like a deer running through a forest and it's very joyful and funny. You run into other animals in the forest and those are all people playing the game and you sort of have to make up a language because there's no chat in the game. That was the big innovation, that you couldn't talk to each other, you could only make noises or sort of dance, but you find things in the forest, you play together in the forest, and do whatever you want. That was the other thing, there were no rules. So people made up their own rules about the world. They made up their own stories. There were songs written about The Endless Forest, tons of stories, and artworks created about it. People made friends in the forest, people died and had memorials in the forest, like any other multiplayer game. Except for this one was a very specific fantasy about nature and utopia, I suppose. It was our hope that people would take all this beauty and feeling of joy and take that out into the real world and in their interactions with other people and understand that you don't really have to understand each other exactly. I think it was very successful because people are still playing it and it's always been free. We've kept it that way all through the ideas of monetization. We wanted to let people enjoy it. Which again, was kind of anti Web 2.0 thinking, you know? And you're going to relaunch this as a new updated version. When is that coming out? AH: The beta (https://tale-of-tales.com/TheEndlessForest/) is out already. We don't know when it'll be fixed because it's one of those done when it's done kind of projects since we're doing this as a side thing. We decided to remake it, and it wasn't exactly necessary. We were just just worried about the technology failing on us, so we wanted to make sure we had found a way to make The Endless Forest truly endless. With all this excitement, then, about video games and their artistic and social possibilities, you stopped making video games. Why? AH: I stopped making video games because I felt that we had said all that we had to say through the format of the commercial video game that is sold through an online store that people download and play on a PC. The part of the manifesto that I never let go of, that I felt was the key to that whole manifesto, is the realtime part. Even with the internet, when I was making websites, that was the important thing. It was realtime. People were there all connected to the same page. We made several works that were just visualizing this fact. Now we take it for granted that when you're on Twitter, everybody's sitting there on Twitter together, but back then it was very special to say, look, there's someone else here on this page. That real time aspect was something we took from games, this notion that things are being executed 60 frames a second. Even if someone else isn't there with you, you are there with the virtual creature and that virtual creature is reacting to you in realtime. So I stopped making video games because in some ways it felt like the world of gaming was getting in my head a little too much. In a very negative way, because it was a business we were running and I didn't feel like being a business person on the one hand, and on the other hand dealing with the audience part became a much bigger task. It was sometimes really unpleasant. Not so much for the players of our games, but just in general, the whole way in which games were created, sold and marketed became something that I couldn't agree with. We made our last video game in 2015, a game called Sunset (https://tale-of-tales.com/presskit/sheet.php?p=sunset), which had a very strong political message that was in some ways over people's heads. We knew it was going to be, but at the same time it was kind of like our final say, the last thing we could say about video games. We were going to try to make a game that's kind of normal, being that it was just a first person game played like other first person games, but at the same time use it to get across this political message about the time we live in by talking about the past. It’s a very complex game. And I think it ended up being so complex because we knew what we were doing at last, we knew exactly how to make a game in a year, how to stay under budget, how to put together a team, how to market it. But then everything we wanted to say came out in a rush. It was like trying to put together a very intricate puzzle and make it something that people could explore. And it just felt like maybe this doesn't need to be a game. For the first time, after all the years of people telling us, did that really need to be a game? We suddenly asked ourselves does this really have to be a game? It felt like it was time to do something else. In our previous conversations you talked about the work of Polish director Tadeusz Kantor and this residency you had in 2017. It sounded like a kind of really interesting re-engagement with the potential of computation and VR. What was it about Kantor’s work that moved you and re-inspired you? AH: So we had stopped, we had left what we felt was a toxic environment, but that toxic environment was still in our heads. We still didn't understand how to just be artists again and not be these people involved in this business and the games industry. We accepted a very strange offer that we got from the Polish Ministry of Culture to come to Poland and take part in this residency through the Tadeusz Kantor Foundation (http://kantorfoundation.pl/en/) where they were opening up his summer home to artists to come and live there and create work. They sent us videos of his performances and we watched them and I was completely shocked. I was open mouth, flabbergasted, because I was just like, what is this? It was a whole other world and it took a while for me to really parse it. And I loved that. I loved the fact that it did something to my brain, that my brain was not ready for. Even though these were performances that happened in the nineties or eighties, it was beautiful. The more we watched his work and the way that he was interacting with his performers on stage it became an inspirational moment. We felt, this is the same as what we’re interested in, he’s directing this realtime performance, very much moving his actors around the stage and emotionally involved in every single line, and we felt this could translate into VR. The thing that really did it for us was that for him, there was no difference between a doll and a human actor. And that reminded us, okay, what is that in 3D? When you make a world, what makes one thing alive and another thing not alive. And in a way it's just all programming. Out of this we made Cricoterie (https://taleoftales.itch.io/cricoterie), which we started at his summer home in the in the middle of the woods outside Kraków, Poland in this really creepy house. I mean it wasn’t just creepy, It was beautiful, but he's got this giant chair outside, one of his artworks. It was just really an interesting difference between what we were doing before and what we were hoping to do now. VR was something we knew we wanted to experiment with as a way of getting out of our normal computing mode. Also, we liked the fact that when you put on the VR headset, you are really there. It's not so much about imagination, but about tangible facts, for lack of better ways of saying it. We wanted to make performances. We saw VR as something that few people were ever going to have the equipment at home, so it's perfect for actually staging things. We wanted to create installations around our VR. We staged Cricoterie at the Palace of Science and Culture in Warsaw and also had a showing at a gallery in Kraków, which was really wonderful. And had a few other showings of it also in Basel at the Museum Tinguely. And now you're going to have two shows that are upcoming, one at The Momentary (https://themomentary.org/artist-in-residence-program/) in Bentonville, Arkansas, which is an old cheese factory turned to a contemporary art space for visual and performing arts. And then a second show inside the online space called Feral File (https://feralfile.com/). AH: Yeah, during the pandemic I leaned into the fact that I wanted to make sculpture. I had my first solo show at bitforms gallery (https://bitforms.art/) in New York City during lockdown. It was around that time where I realized that the digital work that I had been creating all along was suddenly also valued. Because outside of games, digital work was under-appreciated by the art world and really by everyone. I had been making all these digital objects all along and 3D modeling has been my life for the past 20 years. I wanted to create sculptures that were not only physical sculptures, but also digital, people could see these works as AR sculptures. And of course it was during this time when people started selling NFTs and so suddenly it became a moment where people wanted to collect these virtual sculptures as if they were real. The thing about sculpture is, and it touches on a conclusion that I came to when I had the burnout experience in 2011, was that sometimes things don't have to move, sometimes things can be simply what they are. This beauty that can transcend space time and that realtime that I always comes back to For Feral File (https://feralfile.com/profiles/Auriea%20Harvey) it’s going to be completely virtual pieces being that it’s an NFT show. It’s a well respected space run by Casey Reas, one of those old school digital artists and someone I respect greatly. He's managed to create a space that isn’t just about being marketplace, but it's about online exhibition, something we all kind of became used to during the pandemic. But now that it’s not that anymore, how can you even understand what an online show is? How can you create a venue for that for artists who take digital work seriously? It’s not just about, how do I put it, the ugly JPEGs, but about the actual work that a lot of us have been doing all along. Thank you Auriea, this is great. There’s of course a lot more to talk about. This conversation gives us a good overview, not only into your career, but also a history of the internet. I can completely relate to that and have gone through similar phases. AH: Thank you so much. It's been great to know you all these years and to think back on what we have seen, right? So thank you for having me on the podcast. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit futureperfect.substack.com (https://futureperfect.substack.com?utm_medium=podcast&utm_campaign=CTA_1)

4. 

   26 May 2022

   #003 - Liz Rosenthal: Film Industry, Venice Biennale, and New Social VR Experiences

   Welcome to episode #003 of the FuturePerfect Podcast where we talk with compelling people breaking new ground in art, media, and entertainment. This podcast is produced by FuturePerfect Studio, an extended reality studio creating immersive experiences for global audiences. Episodes are released every two weeks, visit our website futureperfect.studio (http://futureperfect.studio/) for more details. The text version of this interview has been edited for length and clarity. Find the full audio version above or in your favorite podcast app. This week Wayne Ashley interviews Liz Rosenthal, curator of Venice Biennale’s International Film Festival's Official Selection and Competition program Venice VR. We first met online back in 2020 when we participated as international jurors for the TAICCA (https://en.taicca.tw/) immersive grant. And since then I've been following you and impressed with how fully you participate in this expanding field of VR and immersive content production. You're a curator, executive producer, mentor, incubator, CEO, and an international speaker, but you actually started out in film. How did you arrive at VR from your experimental practice in filmmaking? Liz Rosenthal: I got involved in cinema in my late twenties. I started making short films with an ex-partner. Then I lived in Edinburgh, Scotland where the most important film festival in the UK used to be, the Edinburgh Film Festival. And I got a job running the project finance market for the festival and I had to learn everything about the film business. At that time I was watching screeners of British films to include in the marketplace I was running and I found a film by a director called Christopher Nolan that had been made on no budget, and it was brilliant. No one knew about it. I then met somebody who was working with ultra low budget filmmakers in the US as a part of the independent film channel. He'd just met Chris [Nolan], literally hadn't seen the film and was asking me what great films are you seeing? He was running a finishing fund helping first-time and second-time filmmakers to make their first features. His big thing was talking about using digital video instead of using super 16. And thinking about how we could use available tools like these new prosumer cameras and later editing systems, which people weren’t using at the time, like Final Cut Pro and Premiere. So I got involved in working with them and we hit it off obviously. We bonded over Chris' first film and we were very much at the forefront of using digital production tools. Then in the beginning of the 2000s, we got interested in what the internet and digital tools were going to do in terms of engagement, distribution, and new forms of creativity. And that's where I took off. When the company got shut down by the film channel we had done 13 feature films. I was excited about this area, because I thought, wow this was the place where it's going to get exciting. You and I talked about how important the internet was to both of our practices. The internet for me changed my whole trajectory. What effect did the emergence of the internet as a public media space have specifically on your practice? LR: Well, I think you'll probably get a clue from the name of my company, which I set up in 2006, called Power to the Pixel (https://www.powertothepixel.com/). I've always been someone who's interested in working with producers and artists and how they can get their work supported and out into the world and use the best tools, and the best ways to do that from a creative and a sustainability perspective. I was seeing the film business, like many businesses and sectors as they develop, get more and more fragmented and you get people siloed into different parts of these sectors doing different jobs. The relationship between the maker, the producer and the audience is completely distanced. And that's a real problem, both in terms of return-on-investment or impact for the producers, and also in terms of how the form develops. You've got to be in touch with the user and the user experience, or art forms kind of slow down or become irrelevant. So that really changed my practice. Suddenly you've got all these available tools, prosumer then, and now consumer tools. And I was thinking, how is this going to change the way that we make things and engage with audiences? Of course, being someone who's very much a producer and artist focused, I'm always looking at how new ecosystems have to be developed around these things. I take my hat off to people who start experimenting in this area and are curious. In the past, you've discussed VR as the result of a complex stream of influences, most specifically film, theater and video games. How do you see these influences come together in VR? LR: VR and immersive content or XR is the area that I'm now working on that completely consumes me and I don't have enough time to follow all the developments. VR has the cinematic qualities of a big screen in certain circumstances, and that can be both live-action shot 360 video, or it could be animation. The immensity of being in a whole space relates in some way to film. Filmmakers in the film community say: you've gotta see a film on a big screen. With VR it’s way beyond that. And then there's a participatory, spatial, and free roam aspect of immersive theater. When you’re in a virtual world and characters can interact with you in realtime. If you're designing realtime characters that are going to be interactive in terms of what you do as a player or a user then you're going to have to understand that realtime interactivity. There’s also the influence of the interactivity and agency of games, and of course there are people from the visual arts, sound, and all types of interactive designers and UX experts. I'm conflating VR and XR. And I know that VR is only one set of technologies and creative practices within a much larger field. Can you briefly lay out what XR is so that our audiences understand that this is actually quite a complex range of technologies and practices? LR: XR, which often gets mistaken for the environmental movement [Extinction Rebellion], stands for extended reality. I would break XR into three different types of realities. The first type is AR or augmented reality, which you experience on a flat screen like phone or a tablet. This is 2D digital information that's layered over the real world. The second type is MR or mixed reality, that’s 3D holographic information that has to be viewed on special mixed reality glasses. So for MR it’s really only B2B solutions or enterprise, there’s the HoloLens (https://www.microsoft.com/en-us/hololens) by Microsoft and the Magic Leap (https://www.magicleap.com/en-us/), both are very difficult and expensive devices. I've seen beautiful MR projects shown at festivals. With MR you have 3D information that you can move around and interact with, with your hands, and pick things up. It's very exciting, but you need a controlled environment to make it work. New smart glasses are starting to come out, the first big company that are a kind forerunner is Nreal (https://www.nreal.ai/) who are doing deals with telco companies. I don’t know if it’s in the US or Canada yet, but it’s in the UK and launched in Germany, Spain, Korea, and Japan. Those are consumer versions of the HoloLens, they're about $500 as opposed to about $2,500 or $3,000. And then the final type is VR or virtual reality, which completely dislocates you from the real world. Many people are going to assume that VR is confined only to a headset and involves a mostly private and singular experience, but you have really opened up quite an expansive range of ways that audiences can encounter this new medium. Layout for us the many forms and contexts that audiences might encounter VR at, for example, the Venice Biennale. LR: Amongst the other things I do, one of my main roles is curating Venice Immersive together with with a dear friend and colleague Michel Reilhac. We handle the immersive content competition section and official selection for the Venice International Film Festival, which is one of the four or five A-list film festivals. We are very lucky that the Venice Biennale, who also run the art and architecture events, but also the film festival, are very excited about embracing the medium. We have our own section and our own island where we can build up an exhibition and show all types of immersive experiences. I'll run through what that means, because I think a lot of people get a chance to see one VR project and they go, that's VR. We are talking about a set of many tools and technologies where you can create all kinds experiences and worlds. VR can be put into kind of two different categories, there’s three degrees of freedom (3DOF) and six degrees of freedom (6DOF). In 3DOF, you're basically contained in a sphere. So you put on a headset and you can look around, but the field of distance between your eye and that sphere doesn't change. You kind of have no agency. Sometimes there's a bit of gaze interactivity with eye tracking or head tracking, but let's say you have no agency to change the world. You can't really interact with the what's happening. It plays out in the same way a film would. You have a subsection of 3DOF experiences that are live action. The creator shoots with a 360 camera that's fixed and you stitch together the images in postproduction. We've had some amazing 360 documentaries shown in Venice. For example, one work that’s a big hit on the Meta Quest store is called Space Explorers: The ISS Experience (https://www.oculus.com/experiences/quest/3006696236087408/) by Felix and Paul (https://www.felixandpaul.com/). They make a beautiful live action documentary that's shot in the International Space Station. It’s the biggest media project ever in space, it's quite remarkable. There are four episodes and it is extraordinary. It's 360 video, amazing quality and a moving experience. Moving on to 6DOF because a lot of people say 3DOF is not real VR. With 6DOF you are in a space where you have agency, you can move around the world, interact with things in the world, and be with other people in that world. You feel immersed and have a sense of presence. This is the most powerful thing about this medium is the sense of embodiment and presence. There’s a wide range of different kinds of 6DOF experiences. You have single person experiences that have very little interactivity or agency, but you’re in a spatial world, like the project Gloomy Eyes (https://www.oculus.com/experiences/quest/2933720383334150/). This features beautiful animations and a kind of Tim Burton-esque type story between a zombie and a little girl in an incredible sort of diorama world. There's no agency, your eye is led around and you’re in a spatial environment. Then you have things that are very complex, long hybrid kind of narrative games that feature hours and hours of gameplay. One of my favorite projects is Down the Rabbit Hole (https://www.oculus.com/experiences/quest/2476104599150595/), which is made by a Swedish studio called Cortopia Studios (https://www.cortopia.com/). It's kind of a prequel to Alice in Wonderland and you’re in this unbelievable animated world where you are moving this girl through the environment and she's meeting characters and it's just so much fun. It's puzzle game and it’s so beautifully designed. Then the final category I’ll go into is multiplayer social VR. This is what I’ve been doing over the last couple years and for Venice we had to make a completely virtual edition. My favorite platform is VRChat (https://hello.vrchat.com/) because it's closest to where people imagine the metaverse or this next spatial version of the internet is going. There’s live theater performances and live music performances where you’re there as an avatar with other people. And there are games, extraordinary art worlds, and performance worlds that can exist on social platforms or you can make a multiplayer standalone app where you can have multiple people in an experience. That's where I'm really excited about VR. When people say, especially cinema people, but it's a singular experience, you’re trapped, and dislocated from the world and it's a solitary experience. I always find that weird. I think watching any piece of media that’s not interactive is a solitary experience because it’s you engaging with the medium. When I go to the cinema I'm not talking to someone asking what are you feeling? What are you doing? What are you feeling now? It's a solitary experience, I happen to be with other people. But in these virtual worlds you are with other people, exploring with them, collaborating, doing incredible things together in amazing avatars of your choice. Can you talk a bit about two other contexts: installation experiences and live motion capture for a virtual audience. LR: In Venice, the great thing is we’re showing works that can only be shown on location. They can't be shown through a remote virtual platform. We've shown a number of these types of works that involve building complex sets and having complex technology where people can go into experiences together on site. The way this works is you have a VR headset connected to a backpack which is its own computer. This referred to as free roam, meaning you’re free to move around the space and can see other people and interact with them including maybe actors, dancers, and performers. In Venice, I’ll give you some examples of the kind of thing I’m talking about. Our biggest production was called The Horrifically Real Virtuality (https://digitaldozen.io/projects/the-horrifically-real-virtuality/) by a French group called DVgroup. They make these incredible experiences which involve actors and haptics and sets. The first experience by DVgroup we showed was called Alice, The Virtual Reality Play. DVgroup invited me to see the work at Cannes Film Festival and didn’t tell me anything about. I went to this beautiful park and went through these red curtains into this beautiful room and put on a headset and few minutes into the experience I realized I wasn't talking to an animated character. I was talking to a real person who was represented by an avatar. What's happening is they are being realtime motion captured into a virtual environment that I'm in at the same time. And I'm realtime as well, you can full body track your participants. The second work was The Horrifically Real Virtuality where they led you into the world of American black and white B-movies where you met Ed Wood and Bela Lugosi shooting the last movie they did together. And it was a crazy descent that started in the real beautiful set and they captured images that you were capturing that you then went into through into a theater. We had three physical sets and we put on the VR headsets and then we went crazy down the rabbit hole into these different layers. There were six audience members and three to four actors. It was an hour long and was pretty astonishing. There's another experience I can tell you about called VR_I (https://www.labiennale.org/en/cinema/2018/lineup/venice-virtual-reality/vri), it's a dance experience by a Swiss choreographer called Cie Gilles Jobin and Artanim. It was actually the first multiplayer live motion capture experience I tried and he is exceptional about experimenting with this medium. He motion captures his dancers and then there are avatars for six audience members. So what happens is you go into the experience with with a backpack that's connected to a VR headset. You realize you have your own avatar and there are dancers that are not in the same space, but you're not quite sure. You’re in the same space with the other audience members and he's playing with scale and embodiment and it's incredible seeing what different audiences do. The dancers are in environments where they’re the same height as you then there's moments when they're giants. And they're kind of almost playing with you. And it's an incredible experience. Suddenly being embodied and seeing other people you can interact with. People start dancing together in the scene. He has even the avatar for wheelchair users. The best way to explain these experiences is having a split screen where you’ve got footage of the person with the headset on and footage of what they see in the headset. In our previous conversations you've mentioned at least two critical reasons why VR and in particular world building is so important to you. One, it's potential for turning taken-for-granted narratives upside down and disrupting commonly held perceptions of our world. And two, the ability to embrace a kind of in-betweenness, this magnet for drawing together practitioners who just don't fit into any single category or role or skillset. Can you say more about this? LR: I came kind of late to this business and it seemed kind of crazy and dysfunctional and was kind of locked in. I could see these [digital] changes happening and how it was gonna blow apart how we engage with each other and how we communicate. So my natural inclination was to say hey everyone, look, this is coming, how can we adapt what we're doing to this? I have always been somebody who gets excited about the new and how it's gonna change and some people aren't. It took me a long while to realize this, that I like things that change. And change is kind of terrifying for some people. I'm not like that. I love the idea that we can adapt as human beings and change. And that's what makes me curious. I feel knowledge is fragmented into silos. And in a way we have loads of things that we can find out now because they're supposedly all online. But in way I sometimes think we've become dumber as human beings because our knowledge is fragmented. When you study now, it tends to be very specific and narrow. You can see that in computer science and also in medicine. When you look at medicine, you go to a doctor and they look at one part of your body. We are a system and we are connected to the outside and everything we do connects. And depending on your philosophy, some people will think, and I very much think this, we are connected to the universe.With world building, we're starting to build spatial environments that we can exist in, and embody, and be present in. We need all kinds of skills to do that. We talked about cinema, immersive theater and games being the three pillars. But we also want to think about how does this affect us? What is it gonna do to us? There are neuroscientists, psychologists, and people thinking about quantum physics, and there's people in healthcare and therapeutics. When you step out of your sector, you get labeled as somebody that doesn't really fit anywhere. I've seen it in people. The digital person at a TV company, or the digital person an arts organization or cultural organization or a big media company or studio, they're always the person who nobody really understands what they do. And they generally aren’t given the IT and the operations to do anything creative. When there's a staffing cut or there's a strategy change, they are the first people to go. Or they get so fed up with having to explain things or find a reason to be in a company that they leave. Those in-between people are really needed. Because they help us adapt and view things in a bigger perspective. There are incredible disciplines that have been developed over decades and centuries that we need to respect and lines of thought and social and cultural commentary that is really important that needs to come into this environment. But it's about not seeing these people as peripheral people, but seeing that they are key to any practice or business. 20 years ago, I began working with a variety of cultural institutions in the US who were seeking to expand their audiences through selective integration of new technologies into their programs. I experienced a profound resistance, almost a resentment, toward having to contend with these new technologies and creative processes. Have things changed? What are the points of contention that continue to circulate after all this time? LR: I think there's always this fear, fear is one of the things that drives people to do unfortunately, desperate things in the world. Fear of change is always difficult. We don't put these in-between people and really deep and strategic people who have an overview on where things are going culturally, technologically and socially, in the forefront of organizations and support them. It's still not happening because people are still in their media silos. And often what happens is people ask “what’s the business model?” And you go, come on, this is the point! What you’re doing is going to be severely hampered by not having the foresight and the strategy to develop your organization and practice. They’re going to help you define that. I ran a big conference event that was a project finance market, a think tank for commissioners and financiers with leading artists and speakers, and people used to go, why should we fund that? What's the business model? I used to go, well, what's your business model? Because I came from film, I said, that's a destroyed business model. You cannot tell me that it’s not. And you know, I'm not going to tell you what the business model is or you’ll pay me loads of money, because people always expect you to just, in a networking event, suddenly explain the business model. What happens is obviously things crumble and people get scared. Then when there's money put into something, they generally rush towards it. There's always these hype curves of things and then suddenly traditional sectors go, oh god, we should be doing that. And then they may, but then it's usually a hype for a while and then they go, oh, well that was the load of rubbish, and let's forget about it. So that's what happens, they never give enough longevity or support. Are you experiencing these kind of challenges at the Venice Biennale? LR: The Biennale was amazing to support us in the first place. And that was the president who left a couple of years ago, Paolo Barata, he's a visionary in a way. And Alberto Barbera, who's the director of the film festival, who really listened to Michel and I about where this was going. And Paolo really understood philosophically what this meant and he wanted to be the person getting behind this new exciting art form. At the same point, the Biennale is an incredible organization, but divided into silos as well. I think in every organization it's really hard to do the new thing that kind of fits in-between so many things. We're in the film festival, so a lot of it rotates around the cinema industry. I've worked with a lot of film institutions, and it’s always been hard. Outside of the US, in countries that have subsidy (government or soft subsidy for culture and media), it tends to be the film institutes who hold the purse strings. The government gives them their ministry of culture, gives the money to them, and some of them are further along than others, but most of them are thinking in film still, so it's quite hard. Can you discuss any projects, ideas or development that our audiences need to know. Can you tell me about any projects on the horizon that will be critical to this field? LR: It's incredible looking at these studios and artists who are making work, they're using new technologies, which are evolving the whole time. There's no standard business model for them to get financed or distributed or exhibited. And they’re working so hard on all these different fronts and it's new creative practice they're developing. You know, it's hard enough making a film. And getting money for it. It's tough because you are putting together support from different sectors. So I’ll talk about some areas where there have been some interesting things. I'm mad about social VR and VRChat (https://hello.vrchat.com/). And I think it's an incredibly exciting space for people to get involved in. I know this is a very weird analogy because it sounds a bit basic, it's kind of like the YouTube of VR in a way, or the social of VR I should say, because you’re on a platform where you can upload a world that you've designed in Unity (https://unity.com/). Or you can start building your avatars in a selection of different software and there's so much information in groups and the community. You can start by just visiting and getting into it because if you have PC you can go on desktop, but if you have a headset, obviously it's better. And a headset connect to a PC is even better. The world builders are kind of people who wouldn't consider themselves artists, but are doing these spectacular adventures. So now, pioneering artists are watching or starting to work with these mediums. So I'd say the live performance projects in the club scene in VRChat are amazing. Can you tell me about some of the live performances that we should look at or be aware of? LR: There was a work that won one of our main awards, our Lions back in 2020, it was called Finding Pandora X (https://www.labiennale.org/en/cinema/2020/venice-vr-expanded/finding-pandora-x) by Kiira Benzing, an amazing pioneering director based out of New York. This was pandemic led, so she'd done some amazing experiments and the year before we had something called Loveseat (https://www.labiennale.org/en/cinema/2019/venice-virtual-reality/loveseat), it was a live performance for 50 people watching actors who were motion captured into a virtual world that was screened around us. So it was already combining live performance and virtual words. She went a step further with Finding Pandora X, it was a reworking of the myth of Pandora in VRChat in worlds they created. I think there’s about 15 audience members and 3 actors who play the gods, so there's Zeus, Hera, and Pandora, and it was just exceptional. I tried it at the beginning of the pandemic and you learn to fly, they have a mechanism for flying. Because there's all kinds of things you can do in VRChat and they have all these plugins that people design and release into the community. So I was stuck in my flat, it was the first awful months in April 2020, and I was flying in VRChat above this amazing world. So we showed that and it was brilliant and won an award. Then something you may be able to see again because it’s replaying, is one of the actresses who was in Finding Pandora X, Deirdre Lyons and her partner Stephen Butchko, did this project called Welcome to Respite (https://www.labiennale.org/en/cinema/2021/lineup/venice-vr-expanded/severance-theory-welcome-respite). It’s another performance set in VRChat that may have different runs still. We showed it in Venice last year, it’s another beautifully designed world in VRChat. I want to talk about another thing that I love that we had in our events program last year, it was called Mycelia (https://www.labiennale.org/en/cinema/2021/venice-vr-expanded/mycelia), and it is an incredible performance by a Canadian artist Nanotopia (https://www.nanotopia.net/) who makes music with mycelia or mushrooms in her studio in Canada. She was performing live into a VRChat world that was designed by an amazing community, it was five people from that community, the Meta Crew South Africa, who designed this incredible crystal light mushroom cave world. And she's performing in this incredible stage in the middle of this cave. They give you mycelia avatars and when you come in the she's performing live in an avatar with her mushrooms and mycelia through a very special MIDI system. And you’re there in your mycelia bathing in this incredible world and it’s one of my favorite things I've ever seen in VRChat. We talked about world building and I didn't answer that question about how this is really important. What kind of worlds are we going to build? And obviously a lot of what happens in these spaces is driven by dystopic visions from science fiction books and movies. And so many people who are in VR are, you know, huge fans of Snow Crash or Ready Player One. Or there’s many dystopic visions we've seen of what's happening in the media, and they're so negative, the views of who we are as humanity. And I sometimes wonder, is our dystopia, the way we are now, have they been designed by artists [laughs]? Have people created them because they've seen them in books and movies? And in a way the architecture that's built, what's happening in the world, the pandemics, all these things, we've seen all these things. I think, why can't you have it a different way? The importance of what kind of worlds you build is so essential, that we think about what kind of virtual spaces we’re going to be in. I’m not a technologist at all, my main reason for going into this area is because I was thinking, wow we're going to engage and communicate in such different ways. And it's really important to think ethically, environmentally, aesthetically and therapeutically how that's going to affect us as humans. These spaces are going to be really important. And there's spaces where they have all kinds of uses that help us think out ideas in spatial settings, solve problems, and embody to a certain extent, what it feels like to be in a situation, and give you some kind of insight, or they have therapeutic benefits that are really powerful. And so this is a really exciting space, the space between art and technology, our minds and bodies and the wellbeing of our minds and bodies and the world is really important. I think this is a great place to stop, with some very hopeful words. And Liz, thank you so much for meeting with us and I look forward to a lot more conversation with you. LR: It's a total pleasure. Thank you so much for the great conversation and looking forward to more, this is the beginning of many. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit futureperfect.substack.com (https://futureperfect.substack.com?utm_medium=podcast&utm_campaign=CTA_1)

5. 

   6 May 2022

   #002 - Team Rolfes: 3D Art, Digital Communities, and Platform Capitalism

   This is episode #002 of the FuturePerfect Podcast where we talk with compelling people breaking new ground in art, media, and entertainment. This podcast is produced by FuturePerfect Studio, an extended reality studio creating immersive experiences for global audiences. Episodes are released every two weeks, visit our website futureperfect.studio (http://futureperfect.studio/) for more details. The text version of this interview has been edited for length and clarity. Find the full audio version above or in your favorite podcast app. This week we interview Team Rolfes, a digital performance and image studio led by Sam and Andy Rolfes. The studio specializes in figurative animation, VR puppetry, and mixed reality collage. They create works across multiple formats, including livestream improvisational comedy, live motion capture animation on large festival stages and in underground rave bunkers, print design for fashion collections, album covers and music videos. They have collaborated with Lady Gaga, Danny Elfman, Danny L Harle, Nike, Netflix, Adult Swim and performed at music festivals across the world. On June 4th, 2022, they will premiere their live 3D musical 3-2-1 RULE at Carriageworks (https://carriageworks.com.au/events/soft-centre-2022/) in Sydney, Australia. The work is being developed together with writer and net artist Jacob Bakkila and artist songwriter Lil Mariko. I first encountered your work as an online video in 2020 as a part of the Lunchmeat Festival of electronic music and art based in Prague. I think it was called Sam Rolfes 360° AV experience. I watched it on my Oculus headset and the work was so exhilarating, but also disconcerting and humorous at the same time. It was like a fever dream complete with moving walls, objects melting, spaces constantly changing sizes, and yet was extremely beautiful. For me, the work exemplified this intriguing in-betweenness that you embrace: part puppet show, theme park ride, sculpture, live performance, gaming, and installation. And this makes absolute sense because you've been making experiences across media and genres for a very long time. You were both originally trained in painting and fine art. How did you get from there to the work that you're doing now? Sam Rolfes: Yes, Andy and I both come from a painting background. Our mom was a painter. She ran a little 3D studio when we were kids. She had these big huge books on Blender and 3ds Max laying around. Andy Rolfes: It was a long path back to 3D. We played around with 2D a lot more. We read about musculature systems in the 3D books and wondered how in the world people can even set this stuff up. SR: There was also a lot about wireframes. When we were kids 3D was just kind of boring. It felt like math and I didn’t want to do math I just wanted to make a cool race car. AR: Yeah a lot of math. I remember making a sword in Blender when I was 12. It’s a pretty linear shape, but it was the most taxing process. So I went back to 2D. I could just play with a plane and an abstraction and it was more fun. These 3D tools, along with game engines and other design software, have become some of the most significant toolsets for conceptualizing and building your work. What happened in terms of your training where you suddenly realized you needed to leave painting and watercolor and shift into 3D? SR: I don't remember how I came across it, but I came across ZBrush, a 3D sculpting program where you can mash things around like digital clay. That was the big aha moment for me. A lot times it hides (honestly oftentimes to its detriment) the mathy elements and we found that it was actually in keeping with our painting background where it allows for semi-improvisation, but with an impressionistic sculptural object. Andy started playing more with Maya and Blender as well. And we both slowly got into it just because it was fun. AR: I went through the whole watercolor track and was doing semi-pro photography and developing an interest in photogrammetry. As I was seeing Sam play around with ZBrush, I got into it and jumped back into 3D. I actually went back to 3ds Max. I was putting photogrammetry scans in there and throwing grass around and rendering that out and realized it had gotten way better. And I started bringing in my 2D stuff and playing with ways to collage that in. I played around with that and Cinema 4D before I ended up going back to ZBrush. SR: This was in tandem with the 2012 to 2016 era of internet art and post internet art. There were a lot of people doing 3D art. They would kind of kludge something together in Maya and make it shiny and spin around. And that stuff still exists to some extent these days, but was increasingly present in Chicago where I was living at the time. I had just moved back from Austin after being there for a year after graduating art school. I was starting to do more show flyers and stuff like that and I was trying to find whatever scene existed in Chicago. You wouldn't know it because none of the people would actually hang out in person, but a lot of interesting things in the glitch scene and post internet scene were coming out of Chicago. I was trying to engage with this new community and was finding our perspective within that. I realized we could take a different approach because of our painting background. All these other people were coming more from a digital art or computer science background. They had an art game program at SAIC where I went to school, but I was so turned off by it because everybody was making these white box gallery experiences and they were all the same. That was one reason why it took me a while to get into Unreal Engine. I was still traumatized by having to virtually walk through all these terribly designed spaces. And then I started doing music videos. Our first one was for this group Amnesia Scanner. And I started using ZBrush as a live visual performance tool and did visuals for shows. I would make characters for every musician performing. There’s no real rigging in ZBrush, but I managed to make the characters bounce around like marionettes. From there I got a bit of an understanding of realtime performance. And then Amnesia Scanner kind of blew up on the internet. We don't reach out to musicians like this, but I just like sent them an email. They're very mysterious and I didn't know where they were based. I sent them an email that was in four different languages that was like, please let's work together. And they responded to me. So I spent two months with an initial dev trying out both Unity and Unreal. And Unreal ended up being better. I got in contact through a friend of a friend with this guy Eric Anderson, who was running a three-story punk venue in Chicago called The Keep. We met and he had a prototype Oculus Rift. This was back in 2015 or something like that. And I went to this DIY spot and then stayed there for a week and we just banged out this crazy video. I just palmed the prototype Oculus headset to do the camera. There was no sequencer and there was nothing rendered in Unreal. This was all recorded. I exported it all and took it to my painting mentor's place and uploaded it to his 12 year old daughter's gaming computer. And it took like 24 hours for it to load on that computer and then we performed it there and just recorded it straight from the screen. It felt good enough that we kind of just kept running with it for everything after that. So in terms of music, your past works have a long dialogue with rave culture, hyperpop, and new forms of media that circulate on the internet. Tell us more about that dialogue and how it informs some of your current work. AR: I was kind of plugged into, or at least aware of, both vaporwave and glitch and everything in between that, like the acerbic visuals and everyone realizing 3D is a lot more approachable. The communities I’ve engaged with have definitely been varied and scattered. It’s a lot of pulling things together and trying to figure out what works. Up until recently not many friends or people I’ve know have directly engaged with 3D. But I show them what I’m working on and try to connect different communities together and see how we can work together. SR: And more recently you’ve been more active in the visual artist communities than I have. I've been more interested in those rave cultures. I have a long career of DJing and producing. I've been in the turntable scene, the glitch hop scene, the witch house scene, and now it’s hyperpop. It all ends up being the same. The through line is just experimentalism basically. It's just like a certain amount of interest in a new sound. Hyperpop is an interesting illustration of this to talk about because it's this weird thing where underground culture was made mainstream and at the same time, at least initially, was not diluted upon becoming mainstream. I guess this has happened all the time, but it's the most recent occurrence that I participated in. Hyperpop is this weird sound that somehow a ton of people know about and it became a meme and a joke because of course it was gonna be. But watching that dynamic was very interesting. We've had a long history with different music scenes. Both me performing as a DJ, but also us doing stage performances with musicians on big festival stages with mocap (motion capture) VR performances that are kind of accompaniments to their music. We've got an opera and a kind of a 3D musical in the works right now. But where it all started was album covers and then music videos. It was about participating in those communities and finding a way to, as visual artists, be a part if it more than just fans, but actually help shape the ideas and shape where everything is going. What are the ideas you're shaping? What's the content and the substance of what you’re trying to shape right now? SR: Generally we try and get in and maybe expand the visual dynamic range. With a lot of experimental approaches, especially in the music scenes, it ends up being a lot about vibe or the nerdy tech or kind of esoteric stuff. For us, we can use all these esoteric tech tools, but use them hopefully for a compelling overarching narrative. And I’m sure we’ll talk more about the performative aspects of our work with using digital tools. But in these electronic scenes it ends up losing a certain humanity. A lot of it for us has been trying to reconnect to this live, in-the-moment feeling. Our work is trying to hit the same subconscious feeling of being in the moment and having all these things happening rather than have some kind of contrived tech demo construction or something. AR: Especially nowadays where people are like—oh yeah I need to touch grass. We want to somehow bring that back to the digital and think how can we make this more physical? We’re combining that with strong motivations and guiding lights in theater, performance, athletics, heavy physicality. And we’re thinking what can we really do with having our bodies fling around, often literally, and have that cascade and become a deeper narrative that also has its own motivations of speaking to the community or wherever our eyes are fixated at the moment. Performance in front of a live audience is super central to you guys. Give us a sense of the infrastructure you need to build in order to create one of your dynamic realtime performances. How does it work compositionally, dramaturgically and technically? What does it take to put together and create a realtime dynamic performance in front of a live audience. SR: Right now, one of our projects is this stage adaptation for this short film, this bigger thing 3-2-1 RULE that’s going to debut in Australia in a month. That one is going to be significantly more structured and quality controlled beforehand rather than being a crazy thing where it’s incredibly improvisational. Often times each show is purpose built to a certain extent. Most of our projects inherit worlds and characters and assets from previous projects, but they they build on each other. We’ll have a collection of scenes that are modular and existing in the same world. Each one is setup for a specific type of camera shot and a specific type of motion capture or VR mechanic. AR: Before we get into designing the motion, we also have to figure what the arc of the performance is. What's the energy, what modes want to fit where? Is this going to be a soft moment or is it going to be more excitable? We chart the long arc and mini arcs of the scene. SR: Oftentimes we're not able to meet with the musicians until we get to whatever country we're going to. Prior to meeting them we set up these modular scenes, each with their arc in terms of mechanics and scene dynamic. We have a whole collection of things and plug them together to an extent. Because the performances are so improvisational, it’s kind of like acting the part of a good DJ who’s watching the audience, watching the musicians, listening, and deciding what’s right in the moment. We work this way when we're making music videos as well. Where we build the environment in VR and then kind of feel out where the choreography of a scene is supposed to go. This big Australian debut of 3-2-1 RULE is going to be pretty regimented. We’re going to have everything planned, but there’s still going to be a fair amount of improvisation since it’s all realtime. I would never want to cut out the potential for those kind of magical moments to happen. It sounds like 3-2-1 RULE is a very important transitional project for you where you're in control of the narrative and you're not in service of some other musicians. Tell me where the title 3-2-1 RULE comes from and give me a sense of what you're producing. SR: The name comes from this backup strategy in tech where you’re supposed to have three backups. I'm gonna get this wrong, but one is local, one is on the cloud, and one is offsite. The staged work is an adaptation of a short film and will eventually be either a feature film or a playable game. It’s one of the major projects for us this year. It’s kind of a parody of both the metaverse stuff and the contemporary moment. But also a way to talk about memory and people’s relationships and history together on the internet and what happens when you use the cloud platforms as a prosthetic brain or a prosthetic memory where you’re offloading moments together. The work follows these gig economy workers who respond to listings posted on an app that gathers memories for people in a metaverse space. If someone wants to remember the best day they ever had or the way their dad danced around when he made breakfast they would use this app and the gig economy workers dive in and play these genre parody games to unlock the memory for them. The conceit is that AI can obviously go in and scan your brain or scan the internet and grab this stuff, but it could never recreate the senses that really make up the core of what the memory is. So you have these gig economy workers who kind of chemically collage and assemble these things together for their clients. The stage adaptation served the dual function of giving us an excuse to start building out everything for this broader narrative project really fast. And to start developing this format that's closer to a musical. The debut in Australia will be with the musician Lil Mariko, but the idea is that we would put this on all over the world, and it could be any musician friend that would star in this role. It might be customized for each musician a bit. There are moments where there’s narrative and there are moments were they could just perform their songs. This is kind of our pitch for a new performance format that could be replicated elsewhere and could really bring variety to the music performance world. Because I mean I love music shows. I love venues. I love playing them. I love going to them. I'm at them all the time. But I'm sick of music shows and the format has hardly changed. There exists this potential to unite all these different formats including visuals, sound, music, and narrative. And it takes a little more work. But I think we might be good people to try it out. You're working with writer and social network artist Jacob Bakkila. What is he bringing to the work? SR: We initially brought Jacob in on our now defunct Netflix project we were developing. He has a whole career of performing as bots on the internet or doing genre parody things and all these satirical things that are really brilliant. The project was going really well, but there was too much red tape and it got canceled. But we were talking afterward about working together and we had a kernel of the idea for 3-2-1 RULE. He said, okay I think I can do this and went away for a few days and came back with the base concept for 3-2-1 RULE. And it just threaded the needle between stuff that our team had already been working on for our game and other projects. I work directly with Jacob on the broader concepts and the story and where it goes, but he can churn out hilarious writing very quickly. It’s a mishmash of different online references from every generation and he’s so conversant in that kind of dialogue that he can make it feel genuinely realistic. He's able to sit in this incredibly online space that I feel is very essential to this story. He just generally knows how to fit everything together in a very nice way and was able to bring the emotion to the project. Do you have a sense of what you want the audience to experience? What do you want them to come away with? What kind of impact do you want to have on them? SR: Maybe it varies a bit between the live show, the eventual short, and then whatever the final big project is. I want it to be jarring, but funny. I want it to reflect upon our online relationships and what we've given up in terms of community, interpersonal dialogue, memory and moments together. How much are we sacrificing for platforms? Would it be safe to say that you obviously have a fraught relationship with these platforms? You've experimented in these spaces, you draw inspiration from these spaces, you post in these spaces, and simultaneously, you're frustrated and critical of these spaces. SR: We’re participating in them because there really is no alternative. I have friends who are making their own distributed web3 based platforms like people doing Channel (https://www.channel.xyz/) and people doing other projects, like more horizontal lefty things here and there. But they still have to promote it on platforms because that is just where all this stuff exists. So much of our stuff, especially if it has any narrative, does have a platform critical element to it because I can't think of anything else to comment on. It feels so absurd to be forced to fit this art that we do, that could take so many different forms, into a box that’s 1080 by 1080 pixels and lasts a minute. There's always been constraints to art, but with platforms it's not a meritocracy, and the best stuff does not rise to the surface. The platforms themselves do not promote things that are in keeping with the value system of anybody within their right minds. It promotes things that will do well on the platform for its own good. I don't think that’s a healthy thing for an artistic community, or for an artist, or for anything. I think most people recognize this to an extent. In a sense, critiquing it and putting it in my little skits is just coping. It’s like acknowledging it, but I only have so much ability to actually do anything about it. It’s also just generally frustrating with the moment we’re in. The trick is speaking to that moment and then not getting too trapped in the Twitter style riffing on the discourse of the day. That stuff will do better, it is incentivized because you will get better metrics and the platforms want that kind of momentary ephemeral thing. But then if you go back a week later, it doesn't hit the same. So that's also a trap. Having things be somehow engaging with the contemporary moment, acknowledging where we are right now, and what our relation is to these platforms and to the economy and to how they have basically become the air we breath. Doing that and then also figuring out how you have it be something that lasts longer than 10 minutes is always a struggle artistically. In all of our discussion we haven't touched on the literal politics of the day. I mean, we haven't talked about Ukraine, we haven't talked about Russia. We haven't talked about the elections. We haven't talked about any of that. What's your relationship to these events and the work you’re doing? Is it something you avoid, something you engage with, or something you don't wanna participate in? SR: All the political discourse, at least between the conservative and liberal sphere, I don't give a s**t about. My interest is in the working class relation to their power, and collective bargaining and what we can do about it. I have opinions about imperialism and being against it and what the US should be doing abroad. But a much more tangible thing to engage with is union and platform issues. AR: It feels more actionable. Stuff that doesn’t feel like beating the same drum. We’re not trying to be Beeple where we just do modern day political cartoons. SR: That’s that momentary discourse thing I'm talking about where it’s like oh, I'm going to make an Elon thing. Like who cares? AR: It feels far too ephemeral. And there's a time in place for that, the political art. SR: And I have done some stuff like that, I mean I've thrown Zuckerberg into some s**t, but I don't know. AR: But that's also trying to keep things contemporary and keeping with a sense of immediacy. I feel like we usually try to tie things down to more. Not really universal, well sort of universal because working class issues are fairly universal outside of maybe the top 1%. But try to speak to the broader issues, and try to speak more to the individual themselves rather than trying to talk to political issues that will come and go all the time. Even if they don't seem like they ever go away. SR: Talking about the news of the day and making art about the news of the day is both a symptom of a broader issue that is very much not the discourse in the mainstream media or however you wanna phrase it. Not to sound like too much like a post-left guy, but it's a liberal trap to make your art about an issue that is being discussed by the media that you have no control over. It's a liberal trap in that it is a culture war fabrication that art can change the world. Like if we make the most moral Disney movie, then everybody will be good. It ignores people’s relation to their labor and all these other things. It’s like, if we have no more bad villains who do problematic things on TV, then everybody's gonna be okay. And I think a lot of artists end up in that trap, feeling the push to have to make work about things like this. Both because it's incentivized by the platform, and because again it's the churn of the daily discourse you're supposed to plug into. And just morally they feel like oh, I have to be saying something. And I'm not saying that my stuff is not cope because there's a left version of this that is just cope too. But it’s just like posting on Twitter. It's not doing anything. We've all been trained to be cultural commentators. All we are doing is quote tweeting people endlessly while the same structural system continues. And I just have no interest in participating in that. It's entertainment at the end of the day and it's entertainment for some people and my stuff is entertainment for lefty types and I'm not necessarily accomplishing anything more, but I at least think that the topics that I'm interested in maybe are more realistically accomplished. AR: I usually just look to the actual items. I just made an artwork for the Queer Museum of Digital Art (https://qmoda.art/), which is part of the whole web3 sphere. They’re trying to fundraise. SR: Just to clarify, I was not talking about that kind of stuff. I'm not saying that fundraising's bad or anything like that. AR: I know. I know. For Ukraine or other huge issues, I’m just going to donate or help however I can. If sharing something might help connect one or two other people, I’m aware of my presence as a node within this whole network. If i’m one of a thousand other people sharing this, but there are three other people in my network who didn't see this it’s cool if it’s actionable. Not if it’s just hot takes. SR: That community building is also way more important than making art about it. Communities can make art and have that steer people in a certain direction. Just to self roast a little bit, if I made the most perfectly leftist take down of whatever, that doesn't accomplish anything either. So making these alternative structures, not to get into dual power talk, but building community structures that exist outside of these platform capital dependent things, I think is the most important thing. What communities are you working with specifically? SR: I have yet to start helping them really in a way that I can give myself credit for, but Jaded (https://jaded.site/) is a new organization. It’s some people from the Black Socialists in America (https://blacksocialists.us/), Zack Fox, and a bunch of comedians have started this artist co-op and community. They're building a venue, they’re going to be funding scripts, they just debuted this podcast they're doing. Black Socialists in America also have all these other projects like The Dual Power App (https://dualpower.app/), which helps give people tools for building co-ops and horizontal things and community structures that don't rely on basic finance capital. They are a great example. And then Channel (https://www.channel.xyz/), I did some work for them. They’re a web3 venture. I don't want to over explain their thing because I will probably do a bad job. They've done a lot of platform critical work, podcasts, and they're a bunch of lefty artists. But from time to time they would get shadowbanned. And they are still, regardless of how critical they are, dependent on these platforms to a certain extent. They’re working to untether that. In the same way that people are tethered to their jobs because they can't get universal healthcare, they have to stay at the job for healthcare. To give themselves a life raft or a way to untether from that toxic situation, the idea is that basically their followers are on the chain so that they can move to whatever platform. You don’t lose followers when you jump somewhere else. It's a first step towards an alternative platform structure or an alternative community structure that does not rely on passing through AWS and Google and relying on this huge stack from just a couple companies. Both of them, Channel and Jaded are awesome examples, and we help where we can. That's great. This really helps fill in a whole other part of your practice that I’m learning more and more about all the time. So I’m super excited to hear you talk about that. We have so many things in common and we have some really interesting overlapping happening between Team Rolfes and FuturePerfect Studio. It’s very exciting and I can’t wait to see more of your work and have more conversations with both of you. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit futureperfect.substack.com (https://futureperfect.substack.com?utm_medium=podcast&utm_campaign=CTA_1)

6. 

   19 Apr 2022

   #001 - Krzysztof Garbaczewski: Digital Theater, VR, and Philosophy

   We’re excited to launch the FuturePerfect Podcast where we talk with compelling people breaking new ground in art, media, and entertainment. The podcast is produced by FuturePerfect Studio, an extended reality studio creating immersive experiences for global audiences. Episodes are released every two weeks, visit our website futureperfect.studio (http://futureperfect.studio) for more details. The text version of this interview has been lightly edited for clarity. Find the audio version above or in your favorite podcast app. This week Wayne Ashley interviews Krzysztof Garbaczewski, a theater director from Poland and founder of the Dream Adoption Society. He is a graduate of the Faculty of Theatre Directing and Dramaturgy at the Ludwik Solski State Theatre School in Kraków. He uses the medium of theater to touch upon existential issues and search for the limits of human experience. Since 2017, Krzysztof has been developing a new theatrical language using 3D virtual environments, avatars, physical scenography, and live actors. He combines these into carefully crafted mixed reality experiences for both online platforms and theatrical audiences. What was the impetus for your shift from classical physical theater training into a focus on the virtual? Krzysztof Garbaczewski: Theater has this ability of being a very experimental field for human interactions, and from the beginning, that’s been important for me. From this approach I became very interested in playing with the audience’s expectations and I began to use live streaming techniques. For example, when the audience comes to the theater they suddenly see a screen in front of them completely covering the stage and see that there are people behind the screen. Almost like in Plato's cave with the performers moving like shadows. It’s also important for me that all the footage we are using is made live. Even if it is happening in virtual reality, my work is still live. I think there's some amazing energy around being with the audience in the same room in this process. Theater has come to an end for you in some way. Theater has failed or it's no longer a site for experimentation and innovation. Something about that is thrilling to me. KG: There is something about reaching this end especially in the work of Grotowski. He reduced theater to the basic elements of the actor and the spectator. At some point he crossed this division into some kind of form that starts to be ritual. Grotowski was looking for ritual in theater. I'm looking for it in this digital form of theater. Ritual is something that connects us. Grotowski had this term art as vehicle. For me it became really interesting to interpret that as digital art as vehicle. He was trying to show the boundary between the spectator and the actor. The actor is in the process of reaching some higher energies through gathering the energy of the spectator and performing some very codified movements and songs to transcend his condition. Somehow I saw that that is exactly what is happening for us when we are using VR headsets. But through technology we can reach this state in a very special way and practice much more quickly. All of Grotowski’s writings are very inspiring for me. At some point he was unsatisfied with conventional theater and was in search of a form of theater that was alive. I think this aliveness is maybe not found in the conventional stage of the theater. (Laughing) Suddenly I am recognizing myself as someone who is making similar choices, but maybe in the opposite direction. But this direction is so opposite that I still feel very close to Grotowski. If he was alive today maybe he also would have also made VR. There's this spiritual and philosophical dimension that underlies your use and understanding of technology. You speak of transcendence and mysticism. You've talked about Plato and about the world as a simulation. You've brought up all kinds of transcendent ideas about technology. Tell me about those mystical ideas that you see in technology. How does that effect that way you work? KG: Since I studied philosophy years ago, it was always very tempting for me to stage The Symposium by Plato. It’s one of the most significant philosophical pieces in the whole of human history and it’s also a dialogue. It has parts, people are talking to each other, and in a kind of naive way that seemed to be perfect for theater. We did Symposium in Warsaw a few years ago. It was very successful and is still playing today, seven years later, to a full house. I find it kind of funny, you know, that people come and listen to Plato for two hours. All the Plato categories are also very inspiring for VR. With Plato’s cave there’s this feeling that there’s something behind the reality. Something behind our matrix that we live in. I have this intuition that technology is not something that is getting us farther from this essence, but actually can get us closer to it. It is similar with nature, which is a problematic term because all of the nature in the world has become culture or agriculture. There are no white spots on the maps anymore. Technology reveals to us new fields of research that we can apply to human consciousness. Tools like virtual reality or artificial intelligence allow us to recognize ourselves more. For me, this process of working with the virtual is like an alchemic process—where we bind some elements that are surprising and new—like new creations, creatures, avatars and all other digital entities that are starting to populate our world and we can interact with them. There's a whole new set of tools for theatrical composition, dramaturgy, and audience experience. You not only have the theater, but you also have the VR space, live film, and the possibility for another remote audience to connect through VR to the work. Give us a sense of what it's like to enter a theater and experience one of your works. KG: It’s something that works on my consciousness and the consciousness of the audience. We use all those tools to discover some new field for this consciousness to come together. Actors perform on the stage in VR headsets and then their avatar is transmitted to a screen onstage and we can see the actor performing live on stage but also performing in the virtual world to a virtual audience. I somehow feel that all these things are there on the stage, not always physically, but in a mediated form. This opens up philosophical questions like what is this reality? What is time? How do we experience this time when, suddenly, an actor is performing somewhere far away, but also very close? And this effects us and gives us very different sensations, so I'm looking for those sensations and trying to make poetry out of it. Here, Faust is important for me as a poetic language. All these things we are using are like words and poetry. They combine into very paradoxical meanings and sometimes work against our common sense and understanding of reality. For the audience, I think it's sometimes pretty shocking to experience this, but it's like a good shock, you know. It takes us out of this normal state of just participating in this reality without questioning it. We need to question this reality to somehow make this world a bit different. In Goethe, this is very special for me right now. A lot of freaky stuff is happening during the rehearsals. Goethe somehow becomes this spiritual experience of gathering different ghosts. Faust is calling those ghosts in the beginning of the drama and we are in a similar process of calling those ghosts. Sometimes I feel that Goethe is somehow present with us and guiding us through this process that is also a very hallucinogenic experience. There’s this part where Faust is drinking this potion—the witches potion that is making him younger—and there’s a guy who made the potion himself in the 1950s. He was drinking this potion with his friend and they both had a very intense experience. And I feel it’s like a trip somehow, this poetry, but described in a very mystical and philosophical language. You talk about the importance of theater as a space to explore boundaries. What kinds of boundaries do you think technologies enable us to cross? KG: Boundaries of time and space and boundaries of our understanding of being together, these are the main boundaries that we are crossing. For example you mentioned earlier that you watched my piece Exegesis (https://www.culturehub.org/exegesis) that we did together with La MaMa in New York and CultureHub. It’s inspired by the novel by Philip K. Dick, this amazing diary of transcendent experiences he had. He then describes that he spent the rest of his life trying to understand what happened to him. We were meeting for our rehearsals and recordings in virtual reality. We were spending all our time together in VR together with Jim Fletcher, Danusia Trevino, 3D artists like Anastasia Vorobiova, composers, and actors. And it was amazing to create this utopian digital community that is making theater and suddenly crossing boundaries. We were crossing this fact that at the time we couldn't just meet together and it was opening some new possibilities. I like that you are sharing the very positive and productive aspects of virtual production, because some of my colleagues would say you can't make live performance unless the bodies are in the same space and time. So I'm very excited to hear you talk and actually promote a different way of working with new results when you can't meet in the same space and time. I think here there is no opposition between those two fields. Of course, it’s very funny to go to theater and it’s also very funny to go to digital theater. I think both forms have their reason for existence. But for me it was, just recently, more mind blowing to experience all these virtual pieces because of the possibilities that it creates. With all the possibilities of creating things that are really not possible to experience in reality. This is in a sense hallucinogenic, but it’s like hallucinating without drugs, just through technology you can reach those levels. I feel that presence is still present, but maybe you need to spend a bit more time for your consciousness to really adapt to it. In theater, after finishing a performance, we hug each other when we do it live. We were doing the same after playing digital performances. It was maybe a bit awkward to hug someone and suddenly experience that you had just touched a bunch of scattered polygons, but the feeling was still with us. With the development of technology this will become something really powerful for performance. In this piece The Artist is (all but) Present (https://dreamadoptionsociety.com/the-artist-present) (the title is a joke on the Marina Abramović performance, but it also was maybe the most Grotowski piece that I made) the theatrical situation was really reduced. The spectator was wearing HoloLens glasses and was seeing the hologram of the actor on the empty stage. It was really amazing visually. The performer and spectator were creating poems together and everything the spectator was saying was appearing in the room, flying as words in space. It’s really something you cannot experience any other way. You’re participating in this poetic conversation and this creation is somehow automatically appearing. These are the kinds of moments that makes this work really new. I'm remembering this moment in our first encounter on Zoom, where you said that you have a distaste for the idea of story and that we should stop using the word storytelling. I’m very attracted to that. In the past few years the whole discourse, from marketing to theater and performance venues, has been that everyone is going to be telling their story and that storytelling is the most important thing that human beings do. So I was taken by your distaste for that. Do you remember what you were communicating to me when you said that? I would maybe make the supposition that experience is something more significant than storytelling and that not everything can be put into storytelling. Storytelling is something that is happening in time and experience is something you can have immediately like enlightenment. Enlightenment is of course a very big thing, but it doesn’t always have to be this big thing. It’s also those moments where you understand something in one second and you grasp all the meaning. When you try to tell a story out of that or try to describe that meaning it is suddenly all lost. You have to spend another week and write and essay and a lot of sweat to bring it back. I feel we are losing something by storytelling. We are losing some potential of the art by just putting everything into a story. We are closing ourselves on this much deeper experience. Like the fact that an art piece can leave you speechless. What then about storytelling? Of course if you expand the meaning of storytelling into some other realities or meanings, maybe then both make sense again. But I feel some possibility of art, metaphors, and abstract art can just go directly to your mind and your soul and change you. Change for us is some kind of mystery. I don’t want to say that I’m a mystic here, and some psychologist could put it in better words, but this change is very rapid it’s just happening. One day you wake up and you’re someone else. It’s the same with art, you experience something and you don’t exactly know why it’s so important. But then it comes back to you and becomes this very significant moment in life. It’s like falling in love, it’s more like a process than storytelling. This last image of you before we stop—this image that you sent to me of you in your boat. The boat is a very powerful symbol. It's not stable, it’s not on land, it’s in the process of going somewhere else. There’s a journey and potential threats and crises. It's also about leaving somewhere to go somewhere else. Tell me where you are with your boat right now. I call it liquid modernity. It's like surfing on liquid modernity, a term by Zygmunt Bauman, a philosopher based in England. (Laughing) Even today I was just thinking, my god, I chose the most unstable home I could imagine. We started by talking about the war that is happening right now. And we see that all these things that we think are stable are shifting and becoming at times dangerous. But of course this idea appeared to me before that. I’m not yet a refugee escaping on a boat. I’m more like Odysseus going for a moment to another island and then turning it into a trip for 10 years now. We’ll see. I also wonder how to make digital art on a boat using, as much as I can, sustainable energy from the sun and wind and traveling by those means of the wind and water. I like the knowledge that is necessary. The weather and the navigation and the stars and how you can use it for setting the point where you are like in those old times. Somehow for me it’s a very modern experience in a way, of trying to see the changing points, and not being in a stable position, and always in some kind of movement. So when is your next trip? I just have to finish this Faust piece and then I’m going on the sea. Where are you going? I have the boat in Poland right now. So I will probably go to Norway or Denmark or Germany. Or somewhere farther, hopefully reaching the Mediterranean or Iceland. Iceland is cold, but still very fascinating. I think this is a good place to stop to imagine you on this boat, leaving Poland to Iceland. And your journey with all of its stabilities and instabilities. and tools for locating directions and points of references. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit futureperfect.substack.com (https://futureperfect.substack.com?utm_medium=podcast&utm_campaign=CTA_1)

Reach and audience

Public platform figures. Ratings count people who left a rating, not total listeners.

Apple Podcasts (US)

5.0 / 5

4 ratings

Contact FuturePerfect Podcast

Guest appearances

Books guests

Based on episode analysis; this does not confirm that the show is currently accepting guests.