Tag Archives: real world science

Untold AI: The Manifestos

by

4

So far along the course of the Untold AI series we’ve been down some fun, interesting, but admittedlydigressivepaths, so let’s reset context. The larger question that’s driving this series is, “What AI stories aren’t we telling ourselves (that we should)?” We’ve spent some time looking at the sci-fi side of things, and now it’s time to turn and take a look at the real-world side of AI. What do the learned people of computer science urge us to do about AI?

That answer would be easier if there was a single Global Bureau of AI in charge of the thing. But there’s not. So what I’ve done is look around the web and in books for manifestos published by groups dedicated to big picture AI thinking to understand has been said. Here is the short list of those manifestos, with links.

Careful readers may be wondering why the Juvet Agenda is missing. After all, it was there that I originally ran the workshop that led to these posts. Well, since I was one of the primary contributors to that document, I thought it would seem as inserting my own thoughts here, and I’d rather have the primary output of this analysis be more objective. But don’t worry, the Juvet Agenda will play into the summary of this series.
Anyway, if there are others that I should be looking at, let me know.

FOLI-letter.png
Add your name to the document at the Open Letter site, if you’re so inclined.

Now, the trouble with connecting these manifestos to sci-fi stories and their takeaways is that researchers don’t think in stories. They’re a pragmatic people. Stories may be interesting or inspiring, but they are not science. So to connect them to the takeaways, we must undertake an act of lossy compression and consolidate their multiple manifestos into a single list of imperatives. Similarly, this act is not scientific. It’s just me and my interpretive skills, open to debate. But here we are.


For each imperative I identified, I tagged the manifesto in which I found it, and then cross-referenced the others and tagged them if they had a similar imperative. Doing this, I was able to synthesize them into three big categories. The first is a set of general imperatives, which they hope to foster in regards to AI as long as we have AI. (Or, I guess, it has us.) Then—thanks largely to the Asilomar Conference—we see an explicit distinction between short-term and long-term imperatives, although for the long-term we only wind up with a handful that are mostly relevant once we have General AI.

marvin.jpg
Life? Don’t talk to me about life.

Describing them individually would, you know, result in another manifesto. So I don’t want to belabor these with explication. I don’t want to skip them either, because they’re important, and it’s quite possible they need some cleanup with suggestions from readers: joining two that are too similar, or breaking one apart. So I’ll give them a light gloss here, and in later posts detail the ones most important to the diff.

CompSci Imperatives for AI

General imperatives

  • We must take care to only create beneficial intelligence
  • We must prioritize prevention of malicious AI
  • We should adopt dual-use patterns from other mature domains
  • We should avoid overhyping AI so we don’t suffer another “AI Winter,” where funding and interest falls off
  • We must fund AI research
  • We need effective design tools for new AIs
  • We need methods to evaluate risk
  • AGI’s goals must be aligned with ours
  • AI reasoning must be explainable/understandable rationale, especially for judicial cases and system failures
  • AI must be accountable (human recourse and provenance)
  • AI must be free from bias
  • We must foster research cooperation, discussion
  • We should develop golden-mean world-model precision
  • We must develop inductive goals and models
  • Increase Broad AI literacy
    • Specifically for legislators (good legislation is separate, see below)
  • We should partner researchers with legislators
  • AI must be verified: Make sure it does what we want it to do
  • AI must be valid: Make sure it does not do what we don’t want it to do
  • AI must be secure: Inaccessible to malefactors
  • AI must be controllable: That we can we correct or unplug an AI if needed without retaliation
  • We must set up a watch for malicious AI (and instrumental convergence)
  • We must study Human-AI psychology

Specifically short term imperatives

  • We should augment, not replace humans
  • We should foster AI that works alongside humans in teams
  • AI must provide clear confidences in its decisions
  • We must manage labor markets upended by AI
  • We should ensure equitable benefits for everyone
    • Specifically rein-in ultracapitalist AI
  • We must prevent intelligence monopolies by any one group
  • We should encourage innovation (not stifle)
  • We must create effective public policy
    • Specifically liability law
    • Specifically banning autonomous weapons
    • Specifically humanitarian law
    • Specifically respectful privacy laws (no chilling effects)
    • Specifically fair criminal justice
  • We must find new metrics for measuring AI effects, capabilties
  • We must develop broad machine ethics dialogue
  • We should expand range of stakeholders & domain experts

Long term imperatives

  • We must ensure human welfare
  • AI should help humanity solve problems humanity cannot alone
  • We should enable a human-like learning capability
  • The AI must be reliable
  • We must specifically manage the risk and reward of AI
  • We must avoid mind crimes
  • We must prevent economic control of people
  • We must research and build ASIs that balance

So, yeah. Some work to do, individually and as a species, but dive into those manifestos. The reasons seem sound.

Connecting imperatives to takeaways

To map the imperatives in the above list to the takeaways, I first gave two imperatives a “pass,” meaning we don’t quite care if they appear in sci-fi. Each follows along with the reason I gave it a pass.

  1. We must take care to only create beneficial intelligence
    PASS: Again, sci-fi can serve to illustrate the dangers and risks
  2. We have effective design tools for new AIs
    PASS: With the barely-qualifying exception of Tony Stark in the MCU, design, development, and research is just not cinegenic.
mis-ch05-040.jpg
And even this doesn’t really illustrate design.

Then I took a similar look at takeaways. First, I dismissed the “myths” that just aren’t true. How did I define which of these are a myth? I didn’t. The Future of Life Institute did it for me: https://futureoflife.org/background/aimyths/.
I also gave two takeaways a pass. The first, “AI will be useful servants” is entailed in the overall goals of the manifestos. The second, “AI will be replicable, amplifying any of its problems” which is kind of a given, I think. And such an embarrassment.
With these exceptions removed, I tagged each takeaway for any imperative to which it was related. For instance, the takeaway “AI will seek to subjugate us” is related to both “Ensure that AI is valid: That is does not do what we do not want it to do” and “Ensure any AGI’s goals are aligned with ours.” Once that was done for all them, voilà, we had a map. See below a sankey diagram of how the scifi takeaways connect to the consolidated compsci imperatives.

sankey
Click to see a full-size image

So as fun as that is, you’ll remember it’s not the core question of the series. To get to that, I added dynamic formatting to the Google Sheet such that it reveals those computer science imperatives and sci-fi takeaways that mapped to…nothing. That gives us two lists.

  1. The first list is the takeaways that appear in sci-fi but that computer science just doesn’t think is important. These are covered in the next post, Untold AI: Pure Fiction.
  2. The second list is a set of imperatives that sci-fi doesn’t yet seem to care about, but that computer science says is very important. That list is covered in the next next post, with the eponymously titled Untold AI: Untold AI.