Nothing Breaks Like A.I. Heart

Omar* and I matched 13 days before I left England. From that point on, we spent maybe 12 hours apart. The rest was getting lost in the library stacks, walking hand in hand across the Bridge of Sighs.sitting in the grass on the lawn of King’s College, laughing our arses off of the Footlights Pantomime.fumbling our way through the college bar crawl, cycling to Grantchester for tea and scones. We were filling in the outline of a love story I’d written in my head reading novels and Instagram posts by American women studying abroad. It was the perfect beginning but it happened at the end of the year.

I needed to leave for a new job in San Francisco. He needed to stay to finish his PhD. I begged the airline to let me change my flight. He sprinted to the train station to say goodbye.

Text that looks like this has been generated and text with a switchtext with a toggletext with this symbol can be swapped out by clicking.

I always assumed you don’t realize when you’re living in a cliche. But, in actuality, when you’re on one end of the romantic comedy trope, waiting at the station for a person who looks like they could have appeared in an Ezra Pound poem,could have appeared in a book about Hemingway’s racing fever,could have been at the Chelsea Hotel when Sylvia Plath lived there, you recognize and fear the inevitable final scene. Nothing can actually stay trite forever. In real life the ending is never so neat.

To navigate this story, think of each circle as a chance to choose your own adventure. Pick an option below to change the narrative path.

I am not a writer; I am a machine learning engineer. In my work, the question of whether something has ended or not is steeped in data and trendlines: Is the time series stationary? Is the shock transitory or permanent? Answering these questions requires poring over every piece of a dataset, extracting features that might mean something, identifying each possible inflection point. When I try to apply this energy to my love life (or lack thereof), friends tell me I’m “dwelling too much on the past” or “living in a fantasy world” or “not really their friend.”

So, when two years had passed and my feelings for Omar still weren’t resolved, I didn’t tell my friends. I told a supercomputer.

GPT-3 — the third “generative pre-trained transformer” released by the start-up (and my employer) OpenAI — is an example of a language model, or a tool that predicts what sequence of words should follow a user-provided prompt. Given a prompt like, “Hello my name” the model will, more often than not, suggest that the next word is is.

It happens to be the biggest publicly released model of its kind: 175 billion parameters (although access is gated through an API with strict use case guidelines). You can think of a parameter roughly as a synapse; the human brain has around 100 trillion of those, but it has to focus on lots of things besides language, like swiping on dating apps and moving to San Francisco. To learn all of those parameters, GPT-3 is trained on hundreds of billions of sentences and stories from the Internet and books. Written something on Reddit? There’s a good chance GPT-3 has read it.

The model learns the toxicity of these forums. The generations shown in this essay are cherry-picked to demonstrate particular aspects of the model and what it taught me about my relationship and GPT-3. They're also chosen to minimize the reader's exposure to harmful and toxic content. Encoded biases in language models perpetuate real harm when language models are released in the real world. To learn more about these and other dangers of big language models, I recommend Bender, Gebru et. al. and the original GPT-3 paper. Access to the model is governed by strict usage guidelines and a content filter is available, but there is significant work to do.

Because it has read from so many different sources with so many different authors, and also because it is a computer, the model lacks self-awareness. So, given a prompt like, “Hi, I’m Pamela and Omar doesn’t love me.” GPT-3 will respond as me, Pamela, and write the story of a relationship with all the poeticism and pathos and, yes, melodrama, that any young woman who has been dumped could ever want.

And, that’s just what I asked for.

Others test using AI to answer philosophical questions or probe the psyche. They say this reveals something about how humans learn and the nature of intelligence.

One argument against the model’s intelligence is that it has a tendency to “overfit,” or get lost in endless loops of repetitive language. This happens when the model essentially memorizes its training data and assumes that everything else should look like it.

You meet someone, you fall in love, you break up, and, heartbroken, you assume you’ll never find anyone who compares. You look for a perfect substitute.

While models have gotten better at avoiding these traps as they’ve gotten larger, there are still sequences of words and language that trigger this spiraling behavior.

I tell GPT-3 this and get back, He says he’s sorry. He says he’s not sorry. He says he’s sorry. He says he’s not sorry. He says he’s…He called. I didn’t answer. I called. He didn’t answer. I texted. He didn’t respond. I called. He didn’t answer. I texted. He didn’t respond. I called. He didn’t answer.I cry when I think about how much I wanted to be with him. I was so lonely. I was so sad. I was so desperate. I was so stupid. I was so stupid. I was so stupid. I was so stupid. GPT-3’s loops feel reminiscent of picking petals on the schoolyard, "He loves me. He loves me not. He loves me. He loves me not." Loops can be exhausting.

Occasionally one of us would say something – “you look nice,” “he doesn’t sound like your type,” “her?” – that crossed whatever friendship boundary lay between us, a shift in tone somehow more relevant than the 5,000 miles. I would text, he wouldn’t respond for a few weeks, and then would call one day to tell me about someone new he’d met.

I learn that in the original French version of the schoolyard petal rhyme you don’t just loop through "He loves me. He loves me not." but question how far the other person’s affection goes: un peu, beaucoup, passionnément, à la folie, pas du tout.

He loves me a little. He loves me a lot. He loves me passionately. He loves me to madness. He loves me not at all.

I would text, he wouldn’t respond for a few weeks, and then would call one day to tell me about his recent break-up.

When you prompt GPT-3, you request a certain number of words back. A language model is only able to handle a limited number of words in prompt plus generation. This means that it may end your story mid-sentence or thought. Or, it may reach an ending and then keep generating because it’s been instructed to talk for a certain amount of time about the characters with which it’s been prompted. If there are two characters in its universe you are forced to engage with them, even after they and you should have moved on.

Despite our awkwardness, it was casual enough that I was not surprised when, a year after I’d left England, he asked if I needed a place to stay on a layover in London. There was anticipation but not expectation when we discussed how his roommates would feel about someone sleeping on the couch.

I wore my casual cool girl outfit and carried my favorite dress in case we changed plans. We walked and talked for eight hours, past the Bank of England, past Combe’s Music Hall, past all the Important Historical Places, through and on to a row of houses across from Westminster Bridgethrough the square where the weirdos live and a coffee shop that is starting to feel a lot like a Starbucks, past the door that was featured on a scene in Pride and Prejudice, then down a street where Mary Shelley wrote Frankensteinthrough the City of London and all those tall buildings with the funny names: The Gherkin, The Walkie-Talkie, The Cheesegrater, The Shard, The Spire, The Heron Tower, The Monument, The Axe, The Lloyd's Building, The Leadenhall Building and then we made out and then he said, “I finally feel comfortable being your friend.”

My friends say to Venmo-request him for the price of the hotel room and the dress that blew away. My friends say to delete his number and texts and Facebook messages and not to forget to disconnect on LinkedIn lest I "accidentally" congratulate him on a promotion years down the line. My friends say there’s no intelligence behind my desire to stay friends with him.

GPT-3 doesn’t care about my friends. It doesn’t care that I work at a start-up, live in a city, that I am quarantined in a house with two other people. It doesn’t care that Omar and I didn’t have the language to say what we wanted from each other, that we fought about his insecurity and my loneliness, that I felt like I was losing myself. It doesn’t care which of my sentences are tired or stale or cliche.

After all, in a sense everything GPT-3 generates is cliche because it’s all rooted in everything that has been written before. In my experience, the model can struggle to land at something that feels like the truth when a story is more nuanced than boy meets girl, boy and girl break-up, girl is devastated.

As I write, I notice peculiarities in my sentences: they are circular, they get distracted, they repeat similar phrases. I’m unsure if GPT-3 is picking up on my style or if my writing is adopting the quirks of the model. I'm unsure if GPT-3 is really leading me down its tunneled tropes or if I've also been trained on the same inputs. I’m unsure if GPT-3 is right when it says it’s ended. After all, it’s not really up to me, or the model.

He called one day in the week preceding quarantine and I joked we should meet in New York. He smiled and said, “Actually, that would be pretty fun, I’ll look at flights, let’s talk soon.”

The next day New York shut down.

We chatted, about the weather and the government and the wealth gap.YouTube stars and the transgression of politics and the on-going demise of the human mind.Corbyn, and phobias, and love and hate.

A month passed. We laughed about the Club Penguin date I’d gone on and his debate about an ex who’d returned to London.

More months passed. He called.

GPT-3’s outputs say I’m upset that he thought my dream job was to manage make-up artists. That I’m upset that he’d replaced me with someone who lived a few blocks away. That I’m upset that he probably told her he loved her at the train station.

I was mostly upset that I didn’t know if that was his ending it once and for all or if it had really ended months or years prior or if he was going to call me again two days or six months down the line just to check-in.

Contrary to what the model says, I wasn’t devastated that the fairytale didn’t come true.

The reality is more complicated: we were never really dating so we never really broke up, but we were never really friends so I didn't get to be mad when he ghosted. When a relationship falls apart slowly, it’s hard to keep track of all the variables — the unfinished pack of chocolate stars, the awkward kiss that makes me cringe to this day, the train we missed by a few minutes — any of them might have been the end.

The reality is less romantic: a man tells a woman he doesn’t want to be with her after spending years looking for his “soul mate,” and then meets someone soon afterwards. She doesn’t tell her friends for a few months, gets her heart broken quietly into pieces while believing she is slowly piecing it back together, then tells everyone at once that it doesn’t matter anymore.

On occasion the model’s predicted outcomes veered close enough to my truth that there was solace in reading them from a distance. But, ultimately, they still weren’t true.

I decided if Omar refused to write an ending and I didn’t agree with how GPT-3 filled it in, then I would write my own. Even if I needed some help along the way.