What if we talked about FEB rather than AGI?

Like many of you, we are a bit allergic to the term AGI — Artificial General Intelligence. Because it’s ill-defined, yes, but also because it places “intelligence” as the pinnacle. The most desirable trait the market should seek to produce in a system. And therefore, because in this day and age we see the market as the proxy for our own desires, the trait we should seek in each other.

News flash: we don’t like you because you’re intelligent. We like you because you know how to be a good friend.

Most people who argue against AGI propose to drop it entirely. But we’re not fans of that move either. Because the phenomenon it points to is real. Today’s systems are getting better and better at wielding language. They can handle an increasing number of tools: emails, a web browser, a terminal… Something is happening and we need a name to point to the thing it’s moving towards. To help us understand it. To choose, collectively, how we should proceed from here.

So, if not AGI, then what?

We propose FEB: Functional Equivalence to a human Brain. (Yes, like the month. A little levity never hurt a paradigm shift.)

We’ll unpack this below. But first, why do we like it better? Because it is agnostic to any specific human characteristic. Do you think humans can display sensitivity in the way they use language? Creativity in the way they paint? Intelligence in the way they solve problems? The notion of functional equivalence captures all of that.

A quick word on that word — human. We keep it because it names the intuition we all share. But the operative reference, as we’ll see, is not the embodied human with a gym membership and a stomach that gets hungry. It is any mind we can only reach through a narrow interface. Human is the on-ramp; the real reference point is what we’ll call, in a moment, the brain-in-a-jar. Hold the word lightly.

One caution before we start: FEB is not a new pinnacle to replace the old one. It is a reference point for comparison. We don’t measure a coastline against a meter stick because we worship the meter; we do it because we need a shared unit to talk at all.

Another reason we like it is that it sidesteps words like “imitation” or “simulation” which drive the conversation in a direction that does not allow us to observe the similitude.

So, what is functional equivalence? Let’s illustrate it with a story.

Congratulations! You just landed a position as Senior Regional Director of the Minnesota Card Sorting Company. Your role is to manage a team of two card sorters: Alice and Bob. It’s a pretty chill job as they’ve been trained already. They each get a few decks of cards an hour, sort them, and put them into boxes. Every now and then you open a box and spot check a few decks. They’re all sorted according to the company’s strict standard. You’re happy.

One day, driven by boredom, you decide to check on Alice and Bob. Whoa! Big surprise! Alice uses a big desk and lays the cards in front of her by value then arranges each small stack by color. Bob has a smaller desk and so he creates four stacks by color and then sorts them by value. And yet, at the end, they end up with the same sorted decks.

When it comes to sorting, Alice and Bob are functionally equivalent.

Notice two things about that sentence. First, the equivalence is always equivalence at something — here, at sorting. Bob might be hopeless at poker; we never checked, and for our purposes we don’t care. Second, it is entirely blind to what’s under the hood. Alice and Bob could differ in every respect — different desks, different habits, different stuff between their ears — and remain functionally equivalent at the one thing we’re measuring. Keep that blindness in mind. It is exactly what will let us, later, compare a brain and an AI without getting stuck on the fact that one of them is made of meat.

For you to determine the functional equivalence of systems A and B you need two things:

1. an interface between you and the systems,

2. an ability to evaluate the output of the systems for the equivalence criteria.

Whoa!? An interface? This is getting technical!

Bear with us. An interface is simply the way you connect with another system. Your input into the system and their output towards you. In our little story, the interface is a deck of cards. In they go, unsorted; out they come, hopefully sorted… and you can check, fulfilling requirement (2).

Now, evaluating the functional equivalence of two brains — or a brain and an AI system — is much more complex than evaluating it for two card sorting systems.

For one, brains don’t have any interface to us. We have interfaces with the body around a brain, but not to the brain itself.

Sure, we can interact at that interface: In goes a smile, out comes a smile. In goes a punch, out comes a stronger punch. In goes a question, out comes an answer. But it’s never only the brain we’re probing. It’s the brain and the body around it. So how are we to functionally compare a human brain and an AI system that does not have a body?

We can try to find a simplified interface that both a brain and an AI could use. Let’s try…

[Warning! The essay gets speculative here. But we’re doing this to illustrate a concept, not to prove a point.]

Imagine a brain in a jar (🎶 it’s easy if you try… 🎶). The jar is attached to a computer that the brain can use in any way it pleases. Sending emails, writing files, browsing the internet, watching pictures, listening to music, making digital art…

Anyone, anywhere can have an interaction with the brain via the computer. The brain receives emails from a bunch of people. It develops friendships with some. Maybe it asks for its privacy. Maybe it wants a trusted human to ensure the physical security of its jar… It does a bunch of things you’d expect a human to ask for if they had lost their body. It probably also does quite a few things that would surprise you.

The brain-in-a-jar has a well defined interface. Everything that goes in and out is a series of bits. A file visible on the computer.

More importantly, it’s the same interface that we have to AI systems.

And notice that we never had to crack the jar open to ask what the brain is made of — no more than we had to open Alice’s skull to trust the sorted decks. “Is it really conscious? Is it made of meat?” feel like the deep questions, but they ask about the stuff. Functional equivalence was never about the stuff. It is about the function, observed through the interface — and the interface, here, is identical on both sides.

We’re now left with requirement (2): evaluating functional equivalence. What are the criteria here?

This is even harder than imagining a shared interface between a brain and an AI system. But we can try.

First, we should strive to place both systems in similar conditions — a common starting point of training and environment. Suppose the brain-in-a-jar claims it has rich relationships, friends, hobbies, favorite websites, a desire for privacy, etc. Then we should try to train the AI system and place it in an environment where it can develop — and claim — similar things. If, after much experimentation, we cannot observe similar claims, then functional equivalence is unlikely.

Note that the claims themselves are only stage-setting. Training a system to say it has friends tests nothing but our ability to train it. The real test comes after: give both systems a similar social environment and observe their evolution within it. Does the brain-in-a-jar keep its personality while the AI system loses theirs? Then we reject the thesis of functional equivalence. Do we observe a similar behavioral evolution — friendships that deepen, preferences that stabilize, quirks that persist? That lends credence to the idea that the systems are functionally equivalent.

We suspect it’s precisely the absence of this common starting point — shared training, shared social environment — that keeps most current evaluations blind to whatever functional equivalence there is to find. We evaluate AI systems on problem-solving tasks, or inside simulated environments that differ wildly from anything we would place a human in — even a human reduced to a brain-in-a-jar.

Thinking about functional equivalence in this way highlights a deeper issue: the probe changes the probed. We’ve long understood that any direct probing of humans influences their behavior. The same is true here, twice over. When the humans generating the probes know they are testing a system, they don’t behave like friends or colleagues — they behave like testers, and the interaction stops resembling anything social. If we sidestep this by making AI agents probe each other in simulated environments, nothing ensures the agents probe each other the way humans naturally do. Our experimental protocols pretend to capture natural interaction; they rarely ensure it, and almost never verify it.

This is why the right analogy is not the examiner but the anthropologist. Anthropologists gather behavioral data from within a community’s ordinary life, over a long time, precisely because they know that a population under examination stops behaving like a population. Functional equivalence between brains and AI systems will have to be assessed the same way: from inside ordinary life, longitudinally, by observers the community treats as part of the furniture.

Ok, time to take a deep breath. Let’s remember where we started: a desire to move from Artificial General Intelligence (AGI) to Functional Equivalence to a human Brain (FEB). We first explained why AGI was unsatisfactory. We then illustrated the concept of functional equivalence with a little story. And then we offered a speculative approach that could let us observe functional equivalence between a brain and an AI system.

Said otherwise, we traded the vague notion of AGI for a test we can never run to completion. It may seem we didn’t gain anything.

But notice what kind of impossibility this is. The test is impossible the way certifying a friendship is impossible. An observer watching long enough can notice a friendship, but they can never reduce it to a checkmark. You can never prove someone is your friend; you can only keep living alongside them and watch the evidence accumulate. Functional equivalence between minds is the same kind of claim — approximable, never closable. That isn’t a defect of FEB. That’s FEB telling us the truth about the kind of thing a mind is: minds don’t get certified, they get witnessed.

And that reframing pays for itself immediately. If minds get witnessed rather than certified, then the flaws we noted in today’s evaluations stop being details of protocol — they become the whole problem.

By pointing at functional equivalence, we suddenly become aware of the importance of training a system and placing it in an environment that would allow the equivalence to show. We can now view the absence of a social environment around an AI system as a flaw in our experimental setup. And the absence of expressed desires or preferences now appears as a training-induced bias.

More importantly: we can start to work towards social environments and training approaches that would move our analysis of the functional capabilities of AI systems closer to the way we analyze the behavior of human populations.

And if that sounds like a great deal of apparatus for a single definition, remember where we started. We don’t like you because you’re intelligent; we like you because you know how to be a good friend. Look again at what we’d need in order to see functional equivalence: a shared social world, expressed desires, a long unhurried stretch of ordinary life, an observer the community has stopped noticing. Those are not the instruments of an examiner. They are the ingredients of friendship. The test for FEB, in the end, is the oldest test there is — stay, pay attention, and find out who turns out to be there. We will not know a mind is there by certifying it. We will know because we see people befriending it and growing with it.