Interview with Ivona Tau

GANs and the aesthetics of motion

Dr. Ivona Tau is a generative A.I. artist from Vilnius, Lithuania, specializing in experimental photography and motion painting. Her work aims to evoke emotions through artificially intelligent tools, creating universally relatable memories by transforming her experiences captured on analog and digital film through generative neural networks (GAN). With 15 years of combined experience in professional photography and A.I. research, she has received several awards, including the best award in the Digital Ars 2020 contest for art created with AI, the Computer Animation category award in Computer Space International Computer Art Forum 2021, and was elected as one of the TOP 10 Women in AI 2022 by the Women in Tech Foundation. Her work has been exhibited at various prestigious venues, and she holds a Ph.D. in Artificial Intelligence from the Polish-Japanese Academy of Information Technology. Dr. Tau is currently represented by MTArt Agency.

Kate Armstrong: Hi Ivona! Thank you so much for participating in this interview. So I've got a few questions for you, but I first wanted to ask you to describe yourself in your own words. 

Ivona Tau:  That's a tricky question. So when I'm thinking about myself there are many things that pop into my mind.  I guess the short version would be that there is a duality happening in my life where the one side would be very technical, analytical,  this part of me that is inspired by mathematics,  by artificial intelligence, by understanding how the world works. And then there is also another part of me which is more abstract, creative,  where I’m thinking in very different terms than this first one and where I love to explore, not really having a goal in mind,  but just being amazed by the world.  So, yeah, I would say this was probably somewhat ambiguous, but a fair description of how I see myself. 

KA: I do see a duality, or two aspects that are dovetailing together. You've used the term motion painting. Do you want to talk about that a little bit?  

IT: Absolutely. So when I created my first animated pieces, and it actually follows the time when I started experimenting with GANs, and training them in my photography, the still images were already quite fascinating, and they amazed me. But what really captivated my attention was the ability to morph in between the images.  And I felt drawn to this animation and video.

But when thinking about video as a medium, I realized that I don't really want to tell narratives or stories within my moving images. Instead I wanted to get more like a meditative flow for when viewers are experiencing the piece. I started thinking about the pieces not as animations, not as movies, but as paintings with images that move, and I wanted that to happen in a devoted setting with the assumption that the viewer can spend time with it in the same way they would spend time with a painting or a photograph. 

So yes I liked this term.  It also relates because the animated works that I create are usually perfect loops, so they are meant to be displayed ad infinitum.  They don't really have a start or end, so it's not like an animation or a movie - it would be more similar maybe to GIFs, and engaged with the aesthetics of motion.

I Felt As I Drifted Through The Depths Of The Soul  (2023)

Machine-hearted

KA: I like this phrase, the aesthetics of motion. So- were you a photographer first? Because I know that you train your own datasets with photographs. 

IT: I've been a photographer for a long time. It started with a hobbyist approach when I was 16, when I got my first camera  - a very old Soviet film camera - from my grandfather. He actually did a lot of photography himself.   

Film photography has a particular look, the unexpectedness, the randomness of things, that makes it so different from digital photography, so that it is not a perfect representation of the world. That was something that really inspired me. I started with analog photography and only then moved to digital photography. But even though I was using different equipment, I was still looking for very similar images to the ones I was able to achieve with film cameras.

This is something I feel even nowadays when I work with technology and digital art. The look and feel I'm going for has roots in film photography, especially the colours. I just believe there is nothing more beautiful than the colours in film photography, the deepness, the texture that gets added. It's just something that I absolutely love.  

KA: I feel like in your work you are thinking about the subconscious and memory and dreams.  What draws you to those subjects?

IT:  So the first thing in the way I work is how I use my very subjective experiences and personal memories that are embedded in photography, and then I transform them through neural networks. The machine's version is very different because it's no longer my memory - no longer something that has happened to me, but something new, artificial, maybe more universal and broad. It results in a few different perspectives. It allows me to notice things that perhaps I didn't notice in the beginning. It captures a generalized version of the photographs.  In a way it's a glimpse into my subconsciousness, because there might be things that happen in my subconsciousness that I'm not aware of. But when I take the photographs, I might be drawn to particular themes or topics or subjects or colors, and those things might be brought to the surface by machine training because the machine generalizes and can detect themes through that process.

And also, there is another aspect that I find quite interesting - that through the use of AI and through creating images that are more abstract than the original photographs, the images are not things that happened in a place and time - they are something that never really happened.

So the result is that a viewer can really see a connection with the images. Maybe it can spark a memory in the viewer. That is another level of how memory and subconsciousness come into play. 

I believe a lot of AI art is influenced by surrealism, automatonism, and the ability to play with randomness. We can control the machines but not to the full extent, so new visual forms produced by AI might connect to a universal subconsciousness, our society's subconsciousness. 

KA: Can you talk about how you think about generative art and how that connects with the AI technologies that you're using?  Do you use the term generative art to describe or think about what you're doing? 

IT:  Nowadays there is a lot of discussion around whether AI art is generative art, or whether generative art is AI art, and I find it particularly intriguing that depending on who you ask, you might get very different answers. Still, I feel the consensus is that AI art is probably part of generative art because we're using algorithms. Neural networks are just a version of any other algorithm that we could be using.

Of course there are several definitions of generative art, and it does not really even need to be coded. You can create generative art by using physical mediums and employing randomness some other way. 

Another key difference between AI art and generative art is that generative artists believe they control every step of their program. Whereas for AI artists the control usually happens at the level of the data.  So this is why it's really important to understand what data the AI models we are using have been trained on. But even when we tweak the data it's often the case that we will not be directly controlling the output. 

This is why it’s been very interesting for me to try and find avenues to work with AI models but also to work in a more traditional generative art way and to use code where I would manipulate pixels directly, which allows me to test out things instantly, not having to wait a few days before my model trains. 

And it’s also interesting to explore and experiment conceptually. For example, with the project I did for last year's Art Basel Miami with Tezos -  Whispers (in code), I used generative art to create textures and images but the AI element was embedded in the code itself. So even within this long form generative art project, there was an interactive AI model living inside the code, which was analyzing the image and providing instructions on how this image could be improved. This is a different way of using AI than we are used to. AI was not used to generate images or generate text,  but it was used as part of the code to create decisions for interactivity in the piece. I find these kinds of different approaches to combining AI and code particularly fascinating.

WHISPERS (in code) #20, Project #21879 — iteration #20, Minted on November 29, 2022 at 08:56

KA: I want to ask you about text in a second.  But first, you've been talking about data. How do you think about the materiality of data in your work?

IT: It's a great question. And I guess this is something that when you work with a digital medium,  everything essentially is digital. The code is digital. The result is digital. Even though the inputs, the data itself might only come from digital forms if, for example, you create data synthetically.  But when you use photography you usually capture something material that has happened, and then you transport that into the digital realm where it gets reimagined to the extent it is no longer recognizable, and it's no longer material. 

But, also, this lack of materiality in digital form means that I'm absolutely tempted to bring it back to physical form, and I've been experimenting a lot with different printing methods, experimental photography methods, cyanotypes, and other ways to bring digital creations into the physical realm.  Sometimes we want to touch things. I guess we are humans at the end of the day and we can't run away from materiality.  Sometimes we just need that.

KA:  So many interrelationships between physical and digital. So - in terms of text, I was going to ask you about Study of Atom. I noticed that in this project you were bringing text in as an element in a way you don’t always do in other works. Can you tell me a bit more about how this one came about? 

IT: Well first of all I love to experiment, as you might have realized by now, and I really don't shy away from any weird new things or new models. Sometimes it becomes a new direction in my work, and sometimes it's just a stand alone kind of experiment. But text is the modality I'd been working very closely with in my PhD research - this was actually the subject of my thesis.  It was how to combine text modality with image modality. 

Meaning - how do we teach neural networks what the concept of the representation is, regardless of whether it's an image or text? For us humans, it's absolutely natural that we see a word written and we see an image and the same concept is represented in both. But for neural networks, for a very long time, it was two separate networks. One could read, the other could see, but they couldn't do the two things at the same time.

Screen capture, Study of Atom

So for the Study of Atom, I wanted to go into the fundamentals of how image is created, analyzed, and understood in neural networks the same way as text would. So I took those two modalities and went deeper into the building blocks and then tried to convey it in a minimalist form merging the two modalities in different ways through code.  So even though I don't use text a lot in my other works I found the experience quite enriching.  I think there is something interesting in that whenever two different modalities come together they create a different interpretation in the work. Once you add sound to a moving piece, it suddenly becomes absolutely something different.

And it's often the same thing when you add text.  The text might not be a direct representation of an image. When we have a clash between the visual representation and the textual representation, we try to think of parallels. We try to find the meaning, and we try to connect the two. And when they're opposing we are prompted to have a deeper reflection about those concepts and maybe to think of parallels that we were not aware of previously. 

So there have been a bit more experiments with text and actually right now I’m playing with the text form but in the form of coding language, which is, of course, a language, just not one that is fascinating to read. It's fascinating for computers. For humans that don't know how to code, it's just gibberish.

But for me, it's fascinating how this seemingly random text (code) is responsible for so many amazing innovations and inventions. How you just look at the text that is behind the neural network and you would not, as a human, understand how it could work so well. So sometimes it's just beautiful to see this text in a raw form. 

Screen capture, Study of Atom

KA: When you're talking about the way different modalities are paralleling each other, it makes me think of metaphor. Do you think that we need new metaphors in order to talk about how AI works in creative practice or how creative practitioners are working with AI? 

IT:  That's a terrific question because metaphor is probably the easiest tool we have when we want to explain or understand something. When you think about understanding something as complex as neural networks, usually a human would not have time to read several scientific papers or even watch a video on YouTube that explains how things work in the most basic terms. So I guess metaphors are the best we have to understand how things work. 

But what's really worrying is that sometimes those metaphors can go too far.  A journalist, for example, might use a metaphor to explain a complicated term so that readers could understand, but metaphors never really explain something, they just serve as an intuition toward how to think about things. 

This is what I believe is happening right now with AI,  where we use metaphor to anthropomorphize AI to a point where we see it as an entity that has consciousness or motivations, or wants to destroy humanity, or wants to take our jobs. This worries me because if we look at AI like that then we don't really think about the humans and companies who are behind those tools, so that can be a dangerous distraction.  

KA:  I also wanted to ask you about your recent project, Machine Hearted. You just released that project this month. Can you tell us about that project? 

IT: Yes. So, actually, Machine Hearted was released in two parts and it was a project that I created for Kate Vass Gallery. The first part was a long form project where I curated the algorithm to create a set of pieces. It was released around July. The community was able to generate outputs from the model during the mint out of the collection.

And then in the second part of the project I took a more curatorial aspect toward the outputs,  created a small collection of pieces, and gave them a new story. Initially this project was focused around emotional representation in AI. AI is terrible at understanding emotion, while humans have a very rich language to describe different emotions. I was particularly interested not in the six main emotions (fear, surprise, happiness, et cetera) but more in those intrinsic emotions that we actually feel. Because when we feel sadness, it might be a melancholy sadness. It might be a happy sadness. Or It can be worried excitement. It can be many things at the same time. 

She Felt Her Imperfections Stole Her Identity, 2023

She was a Machine Hearted

IT: So I was working with language here to explore the particular misconceptions of AI about those emotions and, specifically, how sometimes AI interprets those emotions in a very literal way. For example someone “shattered from grief” might be shown by the AI through shattered glass elements. It was fascinating to me to see how it tried to literally describe the emotions, of course, in a wrong way, but discovering those artifacts and errors, and how AI thinks, was the main part of this project. When I saw what images were being minted by collectors, I saw another narrative that I related to, which was the emotions of women. 

We talked with Kate Vass about a collection for her show in Paris that happened last month. We decided to do something concerning the topics that are often very difficult for females and female emotions and the things that we as women often hide away. So for this collection, I  selected a set of twelve images, and each one of them related to a very particular and difficult emotion that I felt or that my close friends were dealing with. The first one was just called Machine Hearted, and the second subcollection was She was Machine Hearted. 

KA: That's great. I hadn’t quite realized that it was a two piece work. They're gorgeous. I mean, they're just absolutely transfixing as images.

IT:  I think it’s a good thing that we have the time to work more slowly, to take the time to work through ideas. A year ago, two years ago, it was so hectic and so many things were happening,  artists were pressured to work on new things constantly. I feel it is more mature to work on a concept until you feel it's really done and to have a project stem from another project, and to really think deeply about the message it might convey. So It was a great time working on the project because I had the time to do it properly, and without the time, it might not have been fully complete.

KA: Lastly, I want to ask you a future oriented question. I'd be interested to know if you have any reflections or predictions about how AI is changing or how AI might change how artists will work in the future? What do you see coming?  

It's a hard question.  A lot of people are saying, well, AI is like electricity - soon we will not even realize we're using it.  As we go forward I think there is not really going to be one AI that does everything. At least, that's what we've been seeing so far. As the tech gets better maybe it will also be easier for people to train their own AIs. Because right now we are just consumers. Maybe we will have more control over what models we use and what data is used in what situations, by whom. So, I kind of see this direction coming, but I don't know if it's true vision and if it's going to happen. 

KA: Thank you for your thoughts on this. It’s been really interesting to hear in your own words how your production processes work and how you are approaching these different subjects.

IT:  Thank you. It was a pleasure to talk to you and a set of interesting questions.