How to Fix OpenAI |

Governance

Moses Ma/Midjourney

OpenAI’s recent corporate blowup is now over, providing us with thrill-o-rama entertainment that was better than TV’s Succession. But the cliffhanger twist is that their non-profit governance board—which was supposed to save us from an A.I. apocalypse—failed. The company charter charged its board with “creating AI that benefits all of humanity” but it seems that its efforts were stymied when the employees flash unionized and threatened to follow the CEO to Microsoft. The governance board realized that it had been checkmated, and conceded the game.

In hindsight, the idea of putting in a governance system based on a regulatory model now looks short-sighted. What it needed instead was a leadership model, that encouraged the employees to be in alignment with the prime directive of not creating an AI that would someday end humanity. Instead, when the board exercised its regulatory function, the employees all roared back, “You’re not the boss of me!”

The good news is that what seems to have precipitated Altman’s firing wasn’t the imminent threat of Skynet emerging, but rather, the immaturity of the management team. OpenAI’s Board seemed to be seething with rage before firing the CEO. What happened was that a board member published an academic paper that badmouthed the company, and Altman complained, “You can’t do that if you’re on the board.” It does seem like a pretty terrible thing for a director to do, just as the Federal Trade Commission initiated an investigation into the company. The problem that Tezos and OpenAI both suffered was that they did not accommodate for human nature as part of the governance solution. What OpenAI needed more than a CTO was a chief psychology officer, to help raise the team’s EQ to match its IQ.

This leads us to the following question, “How can you successfully build a governance system that prevents a massively successful AI company from destroying the human race?”

Designing Governance Systems

There are many possible strategies to ensure fairness in organizations, like the UN model with a “security council” that can veto things, installing a poison pill into the charter, or establishing a “separation of powers” system like the U.S. government. However, the model I like the best is based on the work of the Nobel economist Elinor Ostrom. Her work focused on the “equitable management of the commons” and self-emergent collective resource management, which could help to address the very difficult issue of governance.

Her best example is the Swiss alpine cheese makers. These farmers lived high in the Swiss Alps, and they shared a grazing commons for their cattle. To avoid the tragedy of the commons, they had one simple rule: If you’ve got three cows, you can pasture those three cows in the commons if you carried them over from last winter. But you can’t bring new cows in just for the summer. It’s very costly to carry cows over to the winter—they need to be in barns and be heated, they have to be fed.

This solution was established in 1200 A.D. and has lasted 900 years.

According to Ostrom’s theory, communities can avoid the tragedy of the commons without requiring top-down regulation by addressing certain coordination challenges. She summarized the conditions for optimizing the process of managing common pool resources in the form of eight core design principles:

• Clearly defined boundaries
• Proportional equivalence between benefits and costs
• Collective choice arrangements
• Monitoring
• Graduated sanctions
• Fast and fair conflict resolution
• Local autonomy
• Appropriate relations with other tiers of rule-making authority

Ostrom’s approach is especially pertinent to the concept of major evolutionary transitions, whereby members of groups become so cooperative that the group becomes a higher-level organism in its own right. The cell biologist Lynn Margulis explained how nucleated cells evolved from symbiotic associations of bacteria. It was then generalized during the 1990s to explain other major transitions, such as the rise of the first bacterial cells, the emergence of multicellular organisms through coadunation, social insect colonies, and even human evolution.

Ostrom’s approach may work here too, for OpenAI. The key is to transcend thinking of ways to program the system like it was a software bug to fix in a top-down control system, and assuming a regulatory model. But at the same time, you have to transcend thinking the other way as well, based on rational choice theory. Perhaps the solution is based on finding a new way—figuring out how to transform the entire OpenAI organization into a “coadunated organization,” where everyone wants the same thing: to be successful economically, while also ensuring the emergence of safe AGI.

A New Way to Manage a 21st-Century Organization

This may be possible by adding tweaks to the capitalization model, borrowing concepts from DAOs—decentralized autonomous organizations—and using game theory, which director Larry Summers is eminently qualified to do. For example, requiring that the board and Altman not receive any equity was short-sighted. Instead, everyone should get equity, but there should be graduated sanctions for failing to keep AI safe and bonuses for coming up with brilliant ideas for making it happen. Then, everyone at the company will pull oars in the same direction.

At the same time, it still would be good to accelerate the development of greater EQ within the team, as it is a bit on the green side, and has had similar problems before. Maybe the idea of hiring a chief psychological officer isn’t too far off the mark.

By leveraging an innovative approach to the employee stock compensation model, incentivizing innovative thinking for safe AGI, with a way to increase group cohesion for the team, perhaps by training non-violent communication (NVC) skills, and increasing the diversity and EQ of the board, OpenAI could not only find a way to stay true to its original vision but also, could become a proof of concept for an exciting new way to manage a 21st-century organization.