As of this posting, the Untold AI analysis stands at 11 posts and around 17,000 words. (And there are as yet a few more to come. Probably.) That’s a lot to try and keep in your head. To help you see and reflect on the big picture, I present…a big picture.
This data visualization has five main parts. And while I tried to design them to be understandable from the graphic alone, it’s worth giving a little tour anyway.
- On the left are two sci-fi columns connected by Sankey-ish lines. The first lists the sci-fi movies and TV shows in the survey. The first ten are those that adhere to the science. Otherwise, they are not in a particular order. The second column shows the list of takeaways. The takeaways are color-coded and ordered for their severity. The type size reflects how many times that takeaway appears in the survey. The topmost takeaways are those that connect to imperatives. The bottommost are those takeaways that do not. The lines inherit the takeaway color, which enables a close inspection of a show’s node to see whether its takeaways are largely positive or negative.
- On the right are two manifesto columns connected by Sankey-ish lines. The right column shows the manifestos included in the analysis. The left column lists the imperatives found in the manifestos. The manifestos are in alphabetical order. Their node sizes reflect the number of imperatives they contain. The imperatives are color-coded and clustered according to five supercategories, as shown just below the middle of the poster. The topmost imperatives are those that connect to takeaways. The bottommost are those that do not. The lines inherit the color of the imperative, which enables a close inspection of a manifesto’s node to see to which supercategory of imperatives it suggests. The lines connected to each manifesto are divided into two groups, the topmost being those that are connected and the bottommost those that are not. This enables an additional reading of how much a given manifesto’s suggestions are represented in the survey.
- The area between the takeaways and imperatives contains connecting lines, showing the mapping between them. These lines fade from the color of the takeaway to the color of the imperative. This area also labels the three kinds of connections. The first are those connections between takeaways and imperatives. The second are those takeaways unconnected to imperatives, which are the “Pure Fiction” takeaways that aren’t of concern to the manifestos. The last are those imperatives unconnected to takeaways, the collection of 29 Untold AI imperatives that are the answer to the question posed at the top of the poster.
- Just below the big Sankey columns are the five supercategories of Untold AI. Each has a title, a broad description, and a pie chart. The pie chart highlights the portion of imperatives in that supercategory that aren’t seen in the survey, and the caption for the pie chart posits a reason why sci-fi plays out the way it does against the AI science.
- At the very bottom of the poster are four tidbits of information that fall out of the larger analysis: Thumbnails of the top 10 shows with AI that stick to the science, the number of shows with AI over time, the production country data, and the aggregate tone over time.
You’ve seen all of this in the posts, but seeing it all together like this encourages a different kind of reflection about it.
Note that it is possible but quite hard to trace the threads leading from, say, a movie to its takeaways to its imperatives to its manifesto, unless you are looking at a very high resolution version of it. One solution to that would be to make the visualization interactive, such that rolling over one node in the diagram would fade away all non-connected nodes and graphs in the visualization, and data brush any related bits below.
A second solution is to print the thing out very large so you can trace these threads with your finger. I’m a big enough nerd that I enjoy poring over this thing in print, so for those who are like me, I’ve made it available via redbubble. I’d recommend the 22×33 if you have good eyesight and can handle small print, or the 31×46 max size otherwise.
Maybe if I find funds or somehow more time and programming expertise I can make that interactive version possible myself.
Some new bits
Sharp-eyed readers may note that there are some new nodes in there from the prior posts! These come from late-breaking entries, late-breaking realizations, and my finally including the manifesto I was party to.
- Sundar Pichai published the Google AI Principles just last month, so I worked it in.
- I finally worked the Juvet Agenda in as a manifesto. (Repeating disclosure: I was one of its authors.) It was hard work, but I’m glad I did it, because it turns out it’s the most-connected manifesto of the lot. (Go, team!)
- The Juvet Agenda also made me realize that I needed new, related nodes for both takeaways and imperatives: AI will enable or require new models of governance. (It had a fair number of movies, too.) See the detailed graph for the movies and how everything connects.
A colophon of sorts
- The data of course was housed in Google Sheets
- The original Sankey SVG was produced in Flourish
- I modified the Flourish SVG, added the rest of the data, and did final layout in Adobe Illustrator
- The poster’s type is mostly Sentinel, a font from Hoefler & Co., because I think it’s lovely, highly readable, and I liked that Sentinels are also a sci-fi AI.
Some readers reported being unable to read the prior post because of its script formatting. Here is the same post without that formatting…
INTERIOR. Sci-fi auditorium. Maybe the Plavalaguna Opera House. A heavy red velvet curtain rises, lifted by anti-gravity pods that sound like tiny TIE fighters. The HOST stands on a floating podium that rises from the orchestra pit. The HOST wears a velour suit with piping, which glows with sliding, overlapping bacterial shapes.
HOST: Hello and welcome to The Fritzes: AI Edition, where we give out awards for awesome movies and television shows about AI that stick to the science.
FX: Applause, beeping, booping, and the sound of an old modem from the audience.
HOST: For those wondering how we picked these winners, it was based on the Untold AI analysis from scifiinterfaces.com. That analysis compared what sci-fi shows suggest about AI (called “takeaways”) to what real world manifestos suggest about AI (called “imperatives”). If a movie had a takeaway that matched an imperative, it got a point. But if it perpetuated a pointless and distracting myth, it lost five points.
The Demon Seed metal-skinned podling thing stands up in the back row of the audience and shouts: Booooooo!
HOST: Thank you, thank you. But just sticking to the science is not enough. We also want to reward shows that investigate these ideas with quality stories, acting, effects, and marketing departments. So the sums were multiplied by that show’s Tomatometer rating. This way to top films didn’t just tell the right stories (according to the science), but it told them well.
Totals were tallied by the firm of Google Sheets algorithms. Ok, ok. Now, to give away awards 009 through 006 are those loveable blockheads from Interstellar, TARS and CASE.
TARS and CASE crutch-walk onto the stage and reassemble as solid blocks before the lectern.
HEADS UP: Because of SCRIPT FORMATTING, this post is best viewed on desktop rather than smaller devices or RSS. An non-script-formatted copy is available.
- INT. SCI-FI AUDITORIUM. MAYBE THE PLAVALAGUNA OPERA HOUSE. A HEAVY RED VELVET CURTAIN RISES, LIFTED BY ANTI-GRAVITY PODS THAT SOUND LIKE TINY TIE FIGHTERS. THE HOST STANDS ON A FLOATING PODIUM THAT RISES FROM THE ORCHESTRA PIT. THE HOST WEARS A VELOUR SUIT WITH PIPING, WHICH GLOWS WITH SLIDING, OVERLAPPING BACTERIAL SHAPES.
- Hello and welcome to The Fritzes: AI Edition, where we give out awards for awesome movies and television shows about AI that stick to the science.
- Applause, beeping, booping, and the sound of an old modem from the audience.
- For those wondering how we picked these winners, it was based on the Untold AI analysis from scifiinterfaces.com. That analysis compared what sci-fi shows suggest about AI (called “takeaways”) to what real world manifestos suggest about AI (called “imperatives”). If a movie had a takeaway that matched an imperative, it got a point. But if it perpetuated a pointless and distracting myth, it lost five points.
- The Demon Seed metal-skinned podling thing stands up in the back row of the audience and shouts: Booooooo!
- Thank you, thank you. But just sticking to the science is not enough. We also want to reward shows that investigate these ideas with quality stories, acting, effects, and marketing departments. So the sums were multiplied by that show’s Tomatometer rating*. This way the top shows didn’t just tell the right stories (according to the science), but it told them right.
- Totals were tallied by the firm of Google Sheets. Ok, ok. Now, to give away awards 009 through 006 are those lovable blockheads from Interstellar, TARS and CASE.
- TARS and CASE crutch-walk onto the stage and reassemble as solid blocks before the lectern.
And here we are at the eponymous answer to the question that I first asked at Juvet around 7 months ago: What stories aren’t we telling ourselves about AI?
In case this post is your entry to the series, to get to this point I have…
- Compiled a survey of AI movies and TV shows
- Tagged the shows for their shared takeaways (and done some fun, sometimes snarky, extra analysis on them)
- Compiled a handful of AI manifestos (tagging the manifestos for their shared imperatives)
- Noted which takeaways map to which imperatives
- Looked at the sci-fi takeaways that don’t relate to imperatives
In this post we look at the imperatives that don’t have matches in AI. Everything is built on a live analysis document, such that new shows and new manifestos can be added later. At the time of publishing, there are 27 of these Untold AI imperatives that sit alongside the 22 imperatives seen in the survey.
What stories about AI aren’t we telling ourselves?
To make these more digestible, I’ve synthesized the imperatives into five groups.
- We should build the right AI
- We should build the AI right
- We must manage the risks involved
- We must monitor AIs
- We must encourage an accurate cultural narrative
For each group…
- I summarize it (as I interpreted things across the manifestos).
- I list the imperatives that were seen in the survey and then those absent from the survey
- I take a stab at why it might not have gotten any play in screen sci-fi and hopefully some ideas about ways that can be overcome.
- Since I suspect this will be of practical interest to writers interested in AI, I’ve provided story ideas using those imperatives.
- Where to learn more about the topic.
Let’s unfold Untold AI. Continue reading
Now that we’ve compared sci-fi’s takeaways to compsci’s imperatives, we can see that there are some movies and TV shows featuring AI that just don’t have any connection to the concerns of AI professionals. It might be that they’re narratively expedient or misinformed, but whatever the reason, if we want audiences to think of AI rationally, we should stop telling these kinds of stories. Or, at the very least, we should try and educate audiences that these are to be understood for what they are.
The list of 12 pure fiction takeaways fall into four main Reasons They Might Not Be of Interest to Scientists.
1. AGI is still a long way off
The first two takeaways concern the legal personhood of AI. Are they people, or machines? Do we have a moral obligation to them? What status should they hold in our societies? These are good questions, somewhat entailed in the calls to develop a robust ethics around AI. They are even important questions for the clarity they help provide moral reasoning about the world around us now. But current consensus is that general artificial intelligence is yet a long way off, and these issues won’t be of concrete relevance until we are close.
- AI will be regular citizens: In these shows, AI is largely just another character. They might be part of the crew, or elected to government. But society treats them like people with some slight difference.
- AI will be “special” citizens: By special, I mean that they are categorically a different class of citizen, either explicitly as a servant class, legally constrained from personhood, or with artificially constrained capabilities.
Now science fiction isn’t constrained to the near future, nor should it be. Sometimes its power comes from illustrating modern problems with futuristic metaphors. But pragmatically we’re a long way from concerns about whether an AI can legally run for office. Continue reading
So far along the course of the Untold AI series we’ve been down some fun, interesting, but admittedlydigressivepaths, so let’s reset context. The larger question that’s driving this series is, “What AI stories aren’t we telling ourselves (that we should)?” We’ve spent some time looking at the sci-fi side of things, and now it’s time to turn and take a look at the real-world side of AI. What do the learned people of computer science urge us to do about AI?
That answer would be easier if there was a single Global Bureau of AI in charge of the thing. But there’s not. So what I’ve done is look around the web and in books for manifestos published by groups dedicated to big picture AI thinking to understand has been said. Here is the short list of those manifestos, with links.
- AAAI Presidential Panel on long-term AI futures: 2008-2009 study
- Asilomar AI Principles
- Future of life institute (FoLI) Benefits of AI
- FoLI Open letter & related research priorities (This is the one that you’ve heard about being signed by celebrity thinkers Elon Musk, Steven Hawking, Peter Norvig, etc.)
- The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
- MIRI Mission Statement
- Partnership on AI, tenets
- Nick Bostrum, Superintelligence
- Stuart Armstrong,Smarter than Us
Careful readers may be wondering why the Juvet Agenda is missing. After all, it was there that I originally ran the workshop that led to these posts. Well, since I was one of the primary contributors to that document, I thought it would seem as inserting my own thoughts here, and I’d rather have the primary output of this analysis be more objective. But don’t worry, the Juvet Agenda will play into the summary of this series.
Anyway, if there are others that I should be looking at, let me know.
Now, the trouble with connecting these manifestos to sci-fi stories and their takeaways is that researchers don’t think in stories. They’re a pragmatic people. Stories may be interesting or inspiring, but they are not science. So to connect them to the takeaways, we must undertake an act of lossy compression and consolidate their multiple manifestos into a single list of imperatives. Similarly, this act is not scientific. It’s just me and my interpretive skills, open to debate. But here we are.
This quickie goes out to writers, directors, and producers. On a lark I decided to run an analysis of AI show takeaways by rating. To do this, I referenced the Tomatometer ratings from rottentomatoes.com to the shows. Then I processed the average rating of the properties that were tagged with each takeaway, and ranked the results.
For instance, looking at the takeaway “AI will spontaneously emerge sentience or emotions,” we find the following shows and their ratings.
- Star Trek: The Motion Picture, 44%
- Superman III, 26%
- Hide and Seek, none
- Electric Dreams, 47%
- Short Circuit, 57%
- Short Circuit 2, 48%
- Bicentennial Man, 36%
- Stealth, 13%
- Terminator: Salvation, 33%
- Tron: Legacy, 51%
- Enthiran, none
- Avengers: Age of Ultron, 75%
I dismissed those shows that had no rating, rather than counting them as zero. The average, then, for this takeaway is 42%. (And it can thank the MCU for doing all the heavy lifting for this one.) There are of course data caveats, like that Black Mirror is given a single tomatometer rating (and one that is quite high) rather than one per episode, but I did not claim this was a clean science. Continue reading
So as interesting as the big donut of takeaways is, it is just a snapshot of everything, all at once. And of course neither people nor cinema play out that way. Like the tone of shows about AI, we see a few different things when we look at individual takeaways over time.
So you understand what you’re seeing: These charts are for the top 7 takeaways from sci-fi AI as described the takeaways post. The colors of each chart correspond to its takeaway in the big donut diagram.
Each chart shows, for each year between Metropolis in 1927 and the many films of 2017, what percentage of shows contained that takeaway. The increasing frequency of sci-fi has some effect on the charts. Up until 1977 there was at most one show per year, so it’s more likely during that early period to see any of the charts max out at 100%. And from 2007 until the time of publication, there have been multiple shows each year, so you would expect to see much lower peaks on the chart as many shows differentiate themselves from their competition, rather than cluster around similar themes. In between those dates it’s a bit of a crapshoot. Continue reading
Looking at the the many-to-many relationships of those takeaways, I wondered if some of them appeared together more commonly than others. For instance, do we tell “AI will be inherently evil” and “AI will fool us with fake media or pretending to be human” frequently? I’m at the upper boundary of my statistical analysis skills here (and the sample size is, admittedly small), but I ran some Pearson functions across the set for all two-part combinations. The results look like this.
What’s a Pearson function? It helps you find out how often things appear together in a set. For instance, if you wanted to know which letters in the English alphabet appear together in words most frequently, you could run a Pearson function against all the words in the dictionary, starting with AB, then looking for AC, then for AD, continuing all the way to YZ. Each pair would get a correlation coefficient as a result. The highest number would tell you that if you find the first letter in the pair then the second letter is very likely to be there, too. (Q & U, if you’re wondering, according to this.) The lowest number would tell you letters that appear very uncommonly together. (Q & W. More than you think, but fewer than any other pair.)
In the screen shot way above, you can see I put these in a Google Sheet and formatted the cells from solid black to solid yellow, according to their coefficient. The idea is that darker yellows would signal a high degree of correlation, lowering the contrast with the black text and “hide” the things that have been frequently paired, while simultaneously letting the things that aren’t frequently paired shine through as yellow.
The takeaways make up both the Y and X axes, so that descending line of black is when a takeaway is compared to itself, and by definition, those correlations are perfect. Every time Evil will use AI for Evil appears, you can totally count on Evil will use AI for Evil also appearing in those same stories. Hopefully that’s no surprise. Look at rest of the cells and you can see there are a few dark spots and a lot of yellow.
If you want to see the exact ranked list, see the live doc, in a sheet named “correlations_list,” but since there are 630 combinations, I won’t paste the actual values or a screen grab of the whole thing, it wouldn’t make any sense. The three highest and four lowest pairings are discussed below. Continue reading