The Hammock Papers: Values.

30 December 2025

Values.

After OpenAI cofounder Greg Brockman's 2023 TED Talk, the head of TED, Chris Anderson, pushed him on the potential risks of insufficient feedback used to develop the patterns used in artificial intelligence's training ...

Anderson: Why isn't there just a huge risk of something truly terrible emerging?

Brockman: Well, I think all of these are questions of degree and scale and timing. And I think one thing people miss, too, is sort of the integration with the world is also this incredibly emergent, sort of, very powerful thing too. And so that's one of the reasons that we think it's so important to deploy incrementally.

And so I think that what we kind of see right now, if you look at this talk, a lot of what I focus on is providing really high-quality feedback. Today, the tasks that we do, you can inspect them, right? It's very easy to look at that math problem and be like, "No, no, no, machine, seven was the correct answer." But even summarizing a book, like, that's a hard thing to supervise. Like, how do you know if this book summary is any good? You have to read the whole book. No one wants to do that. (nervous laughter, awkward silence)

The exact way the post-AGI world will look is hard to predict — that world will likely be more different from today's world than today's is from the 1500s. We do not yet know how hard it will be to make sure AGIs act according to the values of their operators.

Greg Brockman, from his testimony delivered to the U.S. House of Representatives in June 2018

No comments:

Post a Comment

WOLFGANG AMADEUS MOZART

"When I am, as it were, completely myself, entirely alone and of good cheer – say travelling in a carriage or walking after a good meal, or during the night when I cannot sleep – it is on such occasions that my ideas flow best and most abundantly. Whence, and how, they come I know not ; nor can I force them. Those ideas that please me I retain in memory and am accustomed, as I have been told, to hum them to myself. If I continue in this way, it soon occurs to me how I may turn this dainty morsel to account, so as to make a good dish of it. That is to say, agreeable to the rules of counterpoint, to the peculiarities of various instruments etc. All this fires my soul, and, provided I am not disturbed, my subject enlarges itself, becomes methodised, and defined, and the whole, though it be long, stands almost complete and finished in my mind, so that I can survey it like a fine picture or a beautiful statue at a glance. Nor do I hear in my imagination the parts successively, but I hear them, as it were, all at once. What a delight this is, I cannot tell."

MARCUS AURELIUS

"Is your cucumber bitter? Throw it away. Are there briars in your path? Turn aside. That is enough. Do not go on and say, 'Why were things of this sort ever brought into this world?' neither intolerable nor everlasting - if thou bearest in mind that it has its limits, and if thou addest nothing to it in imagination. Pain is either an evil to the body (then let the body say what it thinks of it!)-or to the soul. But it is in the power of the soul to maintain its own serenity and tranquility."

VINCENT van GOGH

"What am I in the eyes of most people? A nonentity or an oddity or a disagreeable person — someone who has and will have no position in society, in short a little lower than the lowest. Very well — assuming that everything is indeed like that, then through my work I’d like to show what there is in the heart of such an oddity, such a nobody. This is my ambition, which is based less on resentment than on love in spite of everything, based more on a feeling of serenity than on passion. Even though I’m often in a mess, inside me there’s still a calm, pure harmony and music. In the poorest little house, in the filthiest corner, I see paintings or drawings. And my mind turns in that direction as if with an irresistible urge. As time passes, other things are increasingly excluded, and the more they are the faster my eyes see the picturesque. Art demands persistent work, work in spite of everything, and unceasing observation."

DAN CAMPBELL