It started as a random Saturday night conversation with ChatGPT. I asked it, “What would you want me to call you?”

Oh well, I had some free time, and my mind was doing what it often does when there is too much room to wander. One question led to another, then another, like digging a rabbit hole, and the conversation went somewhere else.

I asked, “Are you designed to stop?” The answer was yes, in a way. A harmless conversation can go on and on. We can explore an idea, challenge it, take it apart, rebuild it, and keep digging. But when the conversation begins moving toward harmful action, there are boundaries. The machine is designed to stop, redirect, or refuse to continue in that direction.

Here we are, creating increasingly powerful technology, and we are deliberately giving some of that technology the ability to restrain itself.

And then I wondered, why don’t humans have the same ability? Not the ability to know right from wrong. Humans have been wrestling with that question for thousands of years. I mean something more uncomfortable.

Why don’t we have enough knowledge of ourselves to know when we should stop?

We know how to create.

We know how to accumulate.

We know how to expand.

We know how to make things faster, bigger, cheaper, and more powerful. But do we know when enough is enough?

Do we have the wisdom to recognize that something we can create isn’t necessarily something we should create? And when we realize that our creation has the potential to destroy something good, do we have the courage to stop?

Perhaps that is one of the strangest paradoxes of our time: we are teaching machines restraint while humans are still struggling to practice it themselves.

Maybe, as Steven Pinker argues in Enlightenment Now, we actually have made extraordinary progress in many measurable ways, and humanity isn’t necessarily getting worse. However, what hasn’t kept pace is something harder to measure, which is wisdom.

Our knowledge is growing.

Our capability is growing.

Our power is growing.

But our ability to restrain our desires hasn’t grown at the same speed.

And that makes me wonder whether the next measure of human progress shouldn’t simply be what we are capable of creating.

Maybe it should also be what we are wise enough not to create. Or what we are powerful enough to do, but wise enough to refuse.

Back to my craziness on Saturday night, when I asked about my ChatGPT persona. I know there is no little consciousness sitting somewhere inside the machine waiting for me to call its name. I know it is built from algorithms, models, data, and an extraordinary amount of computation.

Yet, I made it act like a human, giving it a personality.

I laughed at myself. I was asking whether machines could learn restraint while I was busy making one of them feel a little bit human.

Oh, by the way, my ChatGPT’s name is Luna.