So this is going to take a few posts. You see, the next interface that appears in The Avengers is a video conference between Tony Stark in his Iron Man supersuit and his partner in romance and business, Pepper Potts, about switching Stark Tower from the electrical grid to their independent power source. Here’s what a still from the scene looks like.
So on the surface of this scene, it’s a communications interface.
But that chat exists inside of an interface with a conceptual and interaction framework that has been laid down since the original Iron Man movie in 2008, and built upon with each sequel, one in 2010 and one in 2013. (With rumors aplenty for a fourth one…sometime.)
So to review the video chat, I first have to talk about the whole interface, and that has about 6 hours of prologue occurring across 4 years of cinema informing it. So let’s start, as I do with almost every interface, simply by describing it and its components. Continue reading →
Yes I did…I did not, however, invite you to sit, Lieutenant.
Are you aware that we have just lost contact with the Rodger Young?
Everyone’s talking about it, sir.
Well, I have the video feed from the bridge here. I understand you are the designer of the emergency evasion panel, and the footage raises some fundamental questions about that design. Watch with me now, Lieutenant.
ORTEGA PRESSES A BUTTON ON A CONSOLE ON HIS DESK. F/X: VIDEO WALL
As you can see, immediately after Captain Deladier issues her order, your panel slides up from a recess in the dash.
(He pauses the video)
(After a silence)
Is there a question, sir?
Why is this panel recessed?
To prevent accidental activation, sir.
But it’s an emergency panel. For crisis situations. It takes two incredibly valuable seconds for this thing to dramatically rise up. What else do you imagine that pilot might have done with those extra two seconds?
Don’t answer that. It’s rhetorical. Next I need you to not explain this layout. Why aren’t the buttons labeled? What does that second one do, and why does it look exactly the same as the emergency evasion button? Are you deliberately trying to confuse our pilots?
OK, now I actually do want you to explain something.
(Resuming the video)
Why did you cover the panel in glass? Ibanez—and I can’t believe I’m saying this—punches it.
The glass is there also to prevent accidental activation, sir.
But you already covered that with the time-wasting recession. You know she’s likely to have tendon, nerve, and arterial damage now, right? And she’s a pilot, Lieutenant. Without her hands, she’s almost useless to us. And now, in addition to having a giant, peanut-shaped boulder in their face, they’ve got a bridge full of loose glass shards scattered about. Let’s hope the artificial gravity lasts long enough for them to get a broom, or they’re going to be in for some floating laceration ballet.
That would be unfortunate, sir.
Damn right. Now honestly I might be of a mind to simply court martial you and treat you to some good old Federation-approved public flogging for Failure to Design. But today may be your lucky day. I believe your elegant design decisions were exacerbated by the pilot’s being something of a drama queen.
The glass was designed to be lifted off, sir.
(Resuming the video)
Fair enough. My last question…
Did I see correctly that all of the lights underneath the engine boost light up all at once? The ones labeled POWER ON? AUTO HOME? NOSE RAM? The ones that don’t have anything to do with the engine boost?
And…and the adjacent green LED, sir.
All at once.
Well, as you might not be able to imagine, we’re moving you. After you collect your belongings you are to report to the Reassignment Office.
(He scrubs back and forth over the drone video of the communication tower ripping off.)
Out of curiosity, WOODS, what was the last thing you designed as part of my department?
After Ibanez explains that the new course she plotted for the Rodger Young (without oversight, explicit approval, or notification to superiors) is “more efficient this way,” Barcalow walks to the navigator’s chair, presses a few buttons, and the computer responds with a blinking-red Big Text Label reading “COURSE OPTIMAL” and a spinning graphic of two intersecting grids.
Yep, that’s enough for a screed, one addressed first to sci-fi writers.
A plea to sci-fi screenwriters: Change your mental model
Think about this for a minute. In the Starship Troopers universe, Barcalow can press a button to ask the computer to run some function to determine if a course is good (I’ll discuss “good” vs. “optimal” below). But if it could do that, why would it wait for the navigator to ask it after each and every possible course? Computers are built for this kind of repetition. It should not wait to be asked. It should just do it. This interaction raises the difference between two mental models of interacting with a computer: the Stoic Guru and the Active Academy.
Stoic Guru vs. Active Academy
This movie was written when computation cycles may have seemed to be a scarce resource. (Around 1997 only IBM could afford a computer and program combination to outthink Kasparov.) Even if computation cycles were scarce, navigating the ship safely would be the second most important non-combat function it could possibly do, losing out only to safekeeping its inhabitants. So I can’t see an excuse for the stoic-guru-on-the-hill model of interaction here. In this model, the guru speaks great truth, but only when asked a direct question. Otherwise it sits silently, contemplating whatever it is gurus contemplate, stoically. Computers might have started that way in the early part of the last century, but there’s no reason they should work that way today, much less by the time we’re battling space bugs between galaxies.
A better model for thinking about interaction with these kinds of problems is as an active academy, where a group of learned professors is continually working on difficult questions. For a new problem—like “which of the infinite number of possible courses from point A to point B is optimal?”—they would first discuss it among themselves and provide an educated guess with caveats, and continue to work on the problem afterward, continuously, contacting the querant when they found a better answer or when new information came in that changed the answer. (As a metaphor for agentive technologies, the active academy has some conceptual problems, but it’s good enough for purposes of this article.)
Consider this model as you write scenes. Nowadays computation is rarely a scarce resource in your audience’s lives. Most processors are bored, sitting idly and not living up to their full potential. Pretending computation is scarce breaks believability. If ebay can continuously keep looking on my behalf for a great deal on a Ted Baker shirt, the ship’s computer can keep looking for optimal courses on the mission’s behalf.
In this particular scene, the stoic guru has for some reason neglected up to this point to provide a crucial piece of information, and that is the optimal path. Why was it holding this information back if it knew it? How does it know that now? “Well,” I imagine Barcalow saying as he slaps the side of the monitor, “Why didn’t you tell me that the first time I asked you to navigate?” I suspect that, if it had been written with the active academy in mind, it would not end up in the stupid COURSE OPTIMAL zone.
Optimal vs. more optimal than
Part of the believability problem of this particular case may come from the word “optimal,” since that word implies the best out of all possible choices.
But if it’s a stoic guru, it wouldn’t know from optimal. It would just know what you’d asked it or provided it in the past. It would only know relative optimalness amongst the set of courses it had access to. If this system worked that way, the screen text should read something like “34% more optimal than previous course” or “Most optimal of supplied courses.” Either text could show some fuigetry that conveys a comparison of compared parameters below the Big Text Label. But of course the text conveys how embarrassingly limited this would be for a computer. It shouldn’t wait for supplied courses.
If it’s an active academy model, this scene would work differently. It would have either shown him optimal long ago, or show him that it’s still working on the problem and that Ibanez’ is the “Most optimal found.” Neither is entirely satisfying for purposes of the story.
How could this scene gone?
We need a quick beat here to show that in fact, Ibanez is not just some cocky upstart. She really knows what’s up. An appeal to authority is a quick way to do it, but then you have to provide some reason the authority—in this case the computer—hasn’t provided that answer already.
A bigger problem than Starship Troopers
This is a perennial problem for sci-fi, and one that’s becoming more pressing as technology gets more and more powerful. Heroes need to be heroic. But how can they be heroic if computers can and do heroic things for them? What’s the hero doing? Being a heroic babysitter to a vastly powerful force? This will ultimately culminate once we get to the questions raised in Her about actual artificial intelligence.
Fortunately the navigator is not a full-blown artificial intelligence. It’s something less than A.I., and that’s an agentive interface, which gives us our answer. Agentive algorithms can only process what they know, and Ibanez could have been working with an algorithm that the computer didn’t know about. She’s just wrapped up school, so maybe it’s something she developed or co-developed there:
Barcalow turns to the nav computer and sees a label: “Custom Course: 34% more efficient than models.”
Um…OK…How did you find a better course than the computer could?
My grad project nailed the formula for gravity assist through trinary star systems. It hasn’t been published yet.
BAM. She sounds like a badass and the computer doesn’t sound like a character in a cheap sitcom.
So, writers, hopefully that model will help you not make the mistake of penning your computers to be stoic gurus. Next up, we’ll discuss this same short scene with more of a focus on interaction designers.
No I am not randomly typing on the screen. I’m taking a pause from the Starship Troopers review to establish some much-needed vocabulary. Oftimes in science fiction, details are added to things for the sake of feeling more real, but that don’t actually do anything and, more importantly to our interests in scifinterfaces, aren’t even guided by a design philosophy. They’re the equivalent of “bullshit” in the H.G. Frankfurt sense. They don’t care about the diegetic truth of themselves, they only care about their effect.
Collectively, I call these things dunsels. But don’t thank me. Thank the midshipmen in the Star Trek TOS universe.
Dunsels appear in three major places in sci-fi.
The surface of objects: Nurnies and greebles
When they appear on spacecraft or futuristic architecture, they’re called greebles or, interchangably, nurnies. These terms come to us from the folks at ILM, who coined the term while developing the style for Star Wars.
I think I’d also apply these terms to props as well, that get covered by details that may not do anything or have much design logic behind them. That means weapons and gadgets, too.
The walls: Gundans
When this suface detailing is applied to sets, it’s called gundans. This after the Star Trek TOS pipes that got labeled “GNDN,” for “goes nowhere, does nothing.” Hat tip to Berm Lee for pointing me to this term.
Not surprisingly, we need to have a word for the same sort of thing in screen interfaces, and I’ve never heard a word to describe them. (If a competitor’s already out there, speak up in the comments.) So after some nerdy social media talk amongst my Chief Nerds and Word People, my friend Magnus Torstensson of Unsworn Industries (and long time supporter of the scifiinterfaces project) suggested combining Mark Coleran‘s acronym “FUI” for “fictional user interfaces” and “widgets” to produce fuidgets, which is pronounced FWIDG-its. I love it. I’ll high-five you when I get to Malmö in November for Oredev, Magnus.
This neologism appropriately sounds as awkward as “nurnies,” “greebles,” and “gundans,” and simultaneously conveys their abstract, fantasy, digital nature. It’s a tough thing to wrap into a single word and I’m in awe that my Swedish friend beat me to it. 🙂
The spirit of apologetics (which is, perhaps, the core of this project) asks that you don’t dismiss details as H.G.Bullshit. You try as hard as you can to find sense in them. That way we don’t get caught up in a spiral of second-guessing an author’s intent, and moreover, that’s where some of the niftiest insights of this sort of analysis come from. But try though we might, sometimes there is just no explaining odd details that litter sci-fi displays, surfaces, and gadgets, other than to admit that they mean nothing and are there only to give a sense of truthiness. So, now we have that word. Fuidgets. You saw it in Monday’s posts, and I’m sure you’re going to see it again.
Sure, Samantha can sort thousands of emails instantly and select the funny ones for you. Her actual operating system functions are kind of a given. But she did two things that seriously undermined her function as an actual product, and interaction designers as well as artificial intelligence designers (AID? Do we need that acronym now?) should pay close attention. She fell in love with and ultimately abandoned Theodore.
There’s a pre-Samantha scene where Theodore is having anonymous phone sex with a girl, and things get weird when she suddenly imposes some weird fantasy where he chokes her with a dead cat. (Pro Tip: This is the sort of thing one should be upfront about.) I suspect the scene is there to illustrate one major advantage that OSAIs have over us mere real humans: humans have unpredictable idiosyncrasies, whereas with four questions the OSAI can be made to be the perfect fit for you. No dead cat unless that’s your thing. (This makes me a think a great conversation should be had about how the OSAI would deal with psychopathic users.) But ultimately, the fit was too good, and Theodore and Samantha fell in love. Continue reading →
Depending on how you slice things, the OS1 interface consists of five components and three (and a half) capabilities.
1. An Earpiece
The earpiece is small and wireless, just large enough to fit snugly in the ear and provide an easy handle for pulling out again. It has two modes. When the earpiece is in Theodore’s ear, it’s in private mode, hearable only by him. When the earpiece is out, the speaker is as loud as a human speaking at room volume. It can produce both voice and other sounds, offering a few beeps and boops to signal needing attention and changes in the mode.
2. Cameo phone
I think I have to make up a name for this device, and “cameo phone” seems to fit. This small, hand-sized, bi-fold device has one camera on the outside an one on the inside of the recto, and a display screen on the inside of the verso. It folds along its long edge, unlike the old clamshell phones. The has smartphone capabilities. It wirelessly communicates with the internet. Theodore occasionally slides his finger left to right across the wood, so it has some touch-gesture sensitivity. A stripe around the outside-edge of the cameo can glow red to act as a visual signal to get its user’s attention. This is quite useful when the cameo is folded up and sitting on a nightstand, for instance. Continue reading →
Me: Well…I like to think of myself as a design critic looking though the lens of–
The computer: “In your voice, I sense hesitance, would you agree with that?”
Me: Maybe, but I would frame it as a careful consider–
The computer: “How would you describe your relationship with Darth Vader?”
Me: It kind of depends. Do you mean in the first three films, or are we including those ridiculous–
The computer: Thank you, please wait as your individualized operating system is initialized to provide a review of OS1 in Spike Jonze’s Her.
A review of OS1 in Spike Jonze’s Her
Ordinarily I wait for a movie to make it to DVD before I review it, so I can watch it carefully, make screen caps of its interfaces, and pause to think about things and cross reference other scenes within the same film, or look something up on the internet.
There’s one wearable technology that, for sheer amount of time on screen and number of uses, eclipses all others, so let’s start with that. Star Trek: The Next Generation introduced a technology called a combadge. This communication device is a badge designed with the Starfleet insignia, roughly 10cm wide and tall, that affixes to the left breast of Starfleet uniforms. It grants its wearer a voice communication channel to other personnel as well as the ship’s computer. (And as Memory Alpha details, the device can also do so much more.)
Chapter 10 of Make It So: Interaction Design Lessons from Science Fiction covers the combadge as a communication device. But in this writeup we’ll consider it as a wearable technology.
The Hover Chair is a ubiquitous, utilitarian, all-purpose assisting device. Each passenger aboard the Axiom has one. It is a mix of a beach-side deck chair, fashion accessory, and central connective device for the passenger’s social life. It hovers about knee height above the deck, providing a low surface to climb into, and a stable platform for travel, which the chair does a lot of.
A Universal Wheelchair
We see that these chairs are used by everyone by the time that Wall-E arrives on the Axiom. From BNL’s advertising though, this does not appear to be the original. One of the billboards on Earth advertising the Axiom-class ships shows an elderly family member using the chair, allowing them to interact with the rest of the family on the ship without issue. In other scenes, the chairs are used by a small number of people relaxing around other more active passengers.
At some point between the initial advertising campaign and the current day, use went from the elderly and physically challenged, to a device used 24/7 by all humans on-board the Axiom. This extends all the way down to the youngest children seen in the nursery, though they are given modified versions to more suited to their age and disposition. BNL shows here that their technology is excellent at providing comfort as an easy choice, but that it is extremely difficult to undo that choice and regain personal control.
Logan is out and about doing his (admittedly horrible) Sandman job. While riding in a transport across the city, his attention drifts to a young lady waiting with a friend on a platform. He thinks she’s lovely and smiles. She catches his eye and smiles, too, before looking away. In the transport, he looks up at a glowing blue point on the ceiling near the windshield. It pulses in response.