Cut to the bottom of the Hudson River where some electrical “transmission lines” rest. Tony in his Iron Man supersuit has his palm-mounted repulsor rays configured such that they create a focused beam, capable of cutting through an iron pipe to reveal power cables within. Once the pipe casing is removed, he slides a circular device onto the cabling. The cuff automatically closes, screws itself tight, and expands to replace the section of casing. Dim white lights burn brighter as hospital-green rings glow brightly around the cable’s circumference. His task done, he underwater-flies away, flying up the southern tip of Manhattan to Stark Tower.
It’s quick scene that sets up the fact that they’re using Tony’s arc reactor technology to liberate Stark Tower from the electrical grid (incidentally implying that the Avengers will never locate a satellite headquarters anywhere in Florida. Sorry, Jeb.) So, since it’s a quick scene, we can just skip the details and interaction design issues, right?
Hidden behind a bookshelf console is the family’s other comm device. When they first use it in the show, Malla and Itchy have a quick discussion and approach the console and slide two panels aside. The device is small and rectangular, like an oscilloscope, sitting on a shelf about eye level. It has a small, palm sized color cathode ray tube on the left. On the right is an LED display strip and an array of red buttons over an array of yellow buttons. Along the bottom are two dials.
Without any other interaction, the screen goes from static to a direct connection to a hangar where Luke Skywalker is working with R2-D2 to repair some mechanical part. He simply looks up to the camera, sees Malla and Itchy, and starts talking. He does nothing to accept the call or end it. Neither do they. Continue reading →
The groomer is a device for sale at the Wookie Planet Trading Post C by local proprietor Saun Dann. It looks like a dust brush with an OXO designed, black, easy-grip handle, with a handful of small silver pushbuttons on one side (maybe…three?), and a handful of black buttons on the other (again, maybe three). It’s kind of hard to call it exactly, since this is lower-res than a recompressed I Can Haz Cheezburger jpg.
Let’s hear Saun describe it to the vaguely menacing Imperial shopper in his store.
Besides shaving and hair trimming, it’s guaranteed to lift stains off clothing, faces, and hands. Cleans teeth, fingers and toenails, washes eyes, pierces ears, calculates, modulates, syncopates life rhythms, and can repeat the Imperial Penal Code—all 17 volumes— in half the time of the old XP-21. Just the thing to keep you squeaky clean.
There are so many, many problems with this thing. On every level it’s wretched. Continue reading →
I am pleased to report that with this post, we are over 50% of the way through this wretched, wretched Holiday Special.
After Lumpy tries to stop stormtroopers from going upstairs, an Imperial Officer commands Malla to keep him quiet. To do so, she does what any self-respecting mother of a pre-teen in the age of technology does, and sits him down to watch cartoons. The player is a small, yellow device that sits flat on an angled tabletop, like a writing desk.
Two small silver buttons stack vertically on the left, and an upside down plug hole strainer on the right. A video screen sits above these controls. Since no one in the rest of his family wants to hear the cartoon introduction of Boba Fett, he dons a pair of headphones, which are actually kind of stylish in that the earpieces are square and perforated, but not beveled. There are some pointless animations that start up, but then the cartoon starts and Lumpy is, in fact, quiet for the duration. So, OK, point one Malla.
On board the R.S. Revenge, the purple-skinned communications officer announces he’s picked up something. (Genders are a goofy thing to ascribe to alien physiology, but the voice actor speaks in a masculine register, so I’m going with it.)
He attends a monitor, below which are several dials and controls in a panel. On the right of the monitor screen there are five physical controls.
A stay-state toggle switch
A stay-state rocker switch
The lower two dials have rings under them on the panel that accentuate their color.
The screen is a dark purple overhead map of the impossibly dense asteroid field in which the Revenge sits. A light purple grid divides the space into 48 squares. This screen has text all over it, but written in a constructed orthography unmentioned in the Wookieepedia. In the upper center and upper right are unchanging labels. Some triangular label sits in the lower-left. In the lower right corner, text appears and disappears too fast for (human) reading. The middle right side of the screen is labeled in large characters, but they also change too rapidly to make much sense of it.
While recording a podcast with the guys at DecipherSciFi about the twee(n) love story The Space Between Us, we spent some time kvetching about how silly it was that many of the scenes involved Gardner, on Mars, in a real-time text chat with a girl named Tulsa, on Earth. It’s partly bothersome because throughout the rest of the the movie, the story tries for a Mohs sci-fi hardness of, like, 1.5, somewhere between Real Life and Speculative Science, so it can’t really excuse itself through the Applied Phlebotinum that, say, Star Wars might use. The rest of the film feels like it’s trying to have believable science, but during these scenes it just whistles, looks the other way, and hopes you don’t notice that the two lovebirds are breaking the laws of physics as they swap flirt emoji.
Hopefully unnecessary science brief: Mars and Earth are far away from each other. Even if the communications transmissions are sent at light speed between them, it takes much longer than the 1 second of response time required to feel “instant.” How much longer? It depends. The planets orbit the sun at different speeds, so aren’t a constant distance apart. At their closest, it takes light 3 minutes to travel between Mars and Earth, and at their farthest—while not being blocked by the sun—it takes about 21 minutes. A round-trip is double that. So nothing akin to real-time chat is going to happen.
But I’m a designer, a sci-fi apologist, and a fairly talented backworlder. I want to make it work. And perhaps because of my recent dive into narrow AI, I began to realize that, well, in a way, maybe it could. It just requires rethinking what’s happening in the chat.
Let’s first acknowledge that we’ve solved long distance communications a long time ago. Gardner and Tulsa could just, you know, swap letters or, like the characters in 2001: A Space Odyssey, recorded video messages. There. Problem solved. It’s not real-time interaction, but it gets the job done. But kids aren’t so much into pen pals anymore, and we have to acknowledge that Gardner doesn’t want to tip his hand that he’s on Mars (it’s a grave NASA secret, for plot reasons). So the question is how could we make it work so it feels like a real time chat to her. Let’s first solve it for the case where he’s trying to disguise his location, and then how it might work when both participants are in the know.
Since 1984 (ping me, as always, if you can think of an earlier reference) sci-fi has had the notion of a digitally-replicated personality. Here I’m thinking of Gibson’s Neuromancer and the RAM boards on which Dixie Flatline “lives.” These RAM boards house an interactive digital personality of a person, built out of a lifetime of digital traces left behind: social media, emails, photos, video clips, connections, expressed interests, etc. Anyone in that story could hook the RAM board up to a computer, and have conversations with the personality housed there that would closely approximate how that person would (or would have) respond in real life.
Is this likely to actually happen? Well it kind of already is. Here in the real world, we’re seeing early, crude “me bots” populate the net which are taking baby steps toward the same thing. (See MessinaBot, https://bottr.me/, https://sensay.it/, the forthcoming http://bot.me/) By the time we actually get a colony to Mars (plus the 16 years for Gardner to mature), mebot technology should should be able to stand in for him convincingly enough in basic online conversations.
Training the bot
So in the story, he would look through cached social media feeds to find a young lady he wanted to strike up a conversation with, and then ask his bot-maker engine to look at her public social media to build a herBot with whom he could chat, to train it for conversations. During this training, the TulsaBot would chat about topics of interest gathered from her social media. He could pause the conversation to look up references or prepare convincing answers to the trickier questions TulsaBot asks. He could also add some topics to the conversation they might have in common, and questions he might want to ask her. By doing this, his GardnerBot isn’t just some generic thing he sends out to troll any young woman with. It’s a more genuine, interactive first “letter” sent directly to her. He sends this GardnerBot to servers on Earth.
Launching the bot
GardnerBot would wait until it saw Tulsa online and strike up the conversation with her. It would send a signal back to Gardner that the chat has begun so he can sit on his end and read a space-delayed transcript of the chat. GardnerBot would try its best to manage the chat based on what it knows about awkward teen conversation, Turing test best practices, what it knows about Gardner, and how it has been trained specifically for Tulsa. Gardner would assuage some of his guilt by having it dodge and carefully frame the truth, but not outright lie.
If during the conversation she raised a topic or asked a question for which GardnerBot was not trained, it could promise an answer later, and then deflect, knowing that it should pad the conversation in the meantime:
Ask her to answer the same question first, probing into details to understand rationale and buy more time
Dive down into a related subtopic in which the bot has confidence, and which promises to answer the initial question
Deflect conversation to another topic in which it has a high degree of confidence and lots of detail to share
Text a story that Gardner likes to tell that is known to take about as long as the current round-trip signal
OK, here’s one: If you had to live anywhere on Earth where they don’t speak English, where would you live?
GardnerBot has a low confidence that it knows Gardner’s answer. It could respond…
(you first) “Oh wow. That is a tough one. Can I have a couple of minutes to think about it? I promise I’ll answer, but you tell me yours first.”
(related subtopic) “I’m thinking about this foreign movie that I saw one time. There were a lot of animals in it and a waterfall. Does that sound familiar?”
(new topic) “What? How am I supposed to answer that one? 🙂 Umm…While I think about it, tell me…what kind of animal would you want to be reincarnated as. And you have to say why.”
(story delay) “Ha. Sure, but can I tell a story first? When I was a little kid, I used to be obsessed with this music that I would hear drifting into my room from somewhere around my house…”
Each of those responses is a delay tactic that allows the chat transcript to travel to Mars for Gardner to do some bot training on the topic. He would be watching the time-delayed transcript of the chat, keeping an eye on an adjacent track of data containing the meta information about what the bot is doing, conversationally speaking. When he saw it hit low-confidence or high-stakes topic and deflect, it would provide a chat window for him to tell the GardnerBot what it should do or say.
To the stalling GARDNERBOT…
For now, I’m going to pick India, because it’s warm and I bet I would really like the spicy food and the rain. Whatever that colored powder festival is called. I’m also interested in their culture, Bollywood, and Hinduism.
As he types, the message travels back to Earth where GardnerBot begins to incorporate his answers to the chat…
At a natural break in the conversation…
OK. I think I finally have an answer to your earlier question. How about…India?
Think about it! Running around in warm rain. Or trying some of the street food under an umbrella. Have you seen youTube videos from that festival with the colored powder everywhere? It looks so cool. Do you know what it’s called?
Note that the bot could easily look it up and replace “that festival with the colored powder everywhere” with “Holi Festival of Color” but it shouldn’t. Gardner doesn’t know that fact, so the bot shouldn’t pretend it knows it. A Cyrano-de-Bergerac software—where it makes him sound more eloquent, intelligent, or charming than he really is to woo her—would be a worse kind of deception. Gardner wants to hide where he is, not who he is.
That said, Gardner should be able to direct the bot, to change its tactics. “OMG. GardnerBot! You’re getting too personal! Back off!” It might not be enough to cover a flub made 42 minutes ago, but of course the bot should know how to apologize on Gardner’s behalf and ask conversational forgiveness.
If the signal to Mars got interrupted or the bot got into too much trouble with pressure to talk about low confidence or high stakes topics, it could use a believable, pre-rolled excuse to end the conversation.
Oh crap. Will you be online later? I’ve got chores I have to do.
Then, Gardner could chat with TulsaBot on his end without time pressure to refine GardnerBot per their most recent topics, which would be sent back to Earth servers to be ready for the next chat.
In this way he could have “chats” with Tulsa that are run by a bot but quite custom to the two of them. It’s really Gardner’s questions, topics, jokes, and interest, but a bot-managed delivery of these things.
So it could work, does it fit the movie? I think so. It would be believable because he’s a nerd raised by scientists. He made his own robot, why not his own bot?
From the audience’s perspective, it might look like they’re chatting in real time, but subtle cues on Gardner’s interface reward the diligent with hints that he’s watching a time delay. Maybe the chat we see in the film is even just cleverly edited to remove the bots.
How he manages to hide this data stream from NASA to avoid detection is another question better handled by someone else.
An honest version: bot envoy
So that solves the logic from the movie’s perspective but of course it’s still squickish. He is ultimately deceiving her. Once he returns to Mars and she is back on Earth, could they still use the same system, but with full knowledge of its botness? Would real world astronauts use it?
Would it be too fake?
I don’t think it would be too fake. Sure, the bot is not the real person, but neither are the pictures, videos, and letters we fondly keep with us as we travel far from home. We know they’re just simulacra, souvenir likenesses of someone we love. We don’t throw these away in disgust for being fakes. They are precious because they are reminders of the real thing. So would the themBot.
Hey, TulsaBot. Remember when we were knee deep in the Pacific Ocean? I was thinking about that today.
I do. It’s weird how it messes with your sense of balance, right? Did you end up dreaming about it later? I sometimes do after being in waves a long time.
I can’t remember, but someday I hope to come back to Earth and feel it again. OK. I have to go, but let me know how training is going. Have you been on the G machine yet?
Nicely, you wouldn’t need stall tactics in the honest version. Or maybe it uses them, but can be called out.
GardnerBot, you don’t have to stall. Just tell Gardner to watch Mission to Mars and update you. Because it’s hilarious and we have to go check out the face when I’m there.
Sending your loved one the transcript will turn it into a kind of love letter. The transcript could even be appended with a letter that jokes about the bot. The example above was too short for any semi-realtime insertions in the text, but maybe that would encourage longer chats. Then the bot serves as charming filler, covering the delays between real contact.
Ultimately, yes, I think we can backworld what looks physics-breaking into something that makes sense, and might even be a new kind of interactive memento between interplanetary sweethearts, family, and friends.
Given its wealth of capabilities, the main complaint might be its lack of language.
A mute sidekick
It has a working theory of mind, a grasp of abstract concepts, and intention, so why does it not use language as part of a toolkit to fulfill its duties? Let’s first admit that mute sidekicks are kind of a trope at this point. Think R2D2, Silent Bob, BB8, Aladdin’s Magic Carpet (Disney), Teller, Harpo, Bernardo / Paco (admittedly obscure), Mini-me. They’re a thing.
Yes, I know she could talk to other fairies, but not to Peter.
Despite being a trope, its muteness in a combat partner is a significant impediment. Imagine its being able to say, “Hey Steve, he’s immune to the halberd. But throw that ribcage-looking thing on the wall at him, and you’ll be good.” Strange finds himself in life-or-death situations pretty much constantly, so having to disambiguate vague gestures wastes precious time that might make the difference between life and death. For, like, everyone on Earth.Continue reading →
Since folks are asking (and it warms my robotic heart that you do), here’s my take on this issue. Boulet, this is for you.
Sci-fi serves different masters
Interaction and interface design answers to one set of masters: User feedback sessions, long-term user loyalty, competition, procurement channels, app reviews, security, regulation, product management tradeoffs of custom-built vs. off-the-shelf, and, ideally, how well it helps the user achieve their goals.
But technology in movies and television shows don’t have to answer to any of these things. The cause-and-effect is scripted. It could be the most unusable piece of junk tech in that universe and it will still do exactly what it is supposed to do. Hell, it’s entirely likely that the actor was “interacting” with a blank screen on set and the interface painted on afterward (in “post”). Sci-fi interfaces answer to the masters of story, worldbuilding, and often, spectacle.
I have even interviewed one of the darlings of the FUI world about their artistic motivations, and was told explicitly that they got into the business because they hated having to deal with the pesky constraints of usability. (Don’t bother looking for it, I have not published that interview because I could not see how to do so without lambasting it.) Most of these things are pointedly baroque where usability is a luxury priority.
So for goodness’ sake, get rid of the notion that the interfaces in sci-fi are a model for usability. They are not.
They are technology in narrative
We can understand how they became a trope by looking at things from the makers’ perspective. (In this case “maker” means the people who make the sci-fi.)
Not this Maker.
Transparent screens provide two major benefits to screen sci-fi makers.
First, they quickly inform the audience that this is a high-tech world, simply because we don’t have transparent screens in our everyday lives. Sci-fi makers have to choose very carefully how many new things they want to introduce and explain to the audience over the course of a show. (A pattern that, in the past, I have called What You Know +1.) No one wants to sit through lengthy exposition about how the world works. We want to get to the action.
With some notable exceptions.
So what mostly gets budgeted-for-reimagining and budgeted-for-explanation in a script are technologies that are a) important to the diegesis or b) pivotal to the plot. The display hardware is rarely, if ever, either. Everything else usually falls to trope, because tropes don’t require pausing the action to explain.
Secondly (and moreover) transparent screens allow a cinematographer to show the on-screen action and the actor’s face simultaneously, giving us both the emotional frame of the shot as well as an advancement of plot. The technology is speculative anyway, why would the cinematographer focus on it? Why cut back and forth from opaque screen to an actor’s face? Better to give audiences a single combined shot that subordinates the interface to the actors’ faces.
We should not get any more bent out of shape for this narrative convention than any of these others.
My god, these beings, who, though they lived a long time ago and in a galaxy far, far away look identical to humans! What frozen evolution or panspermia resulted in this?
They’re speaking languages that are identical to some on modern Earth! How?
Hasn’t anyone noticed the insane coincidence that these characters from the future happen to look exactly like certain modern actors?
How are there cameras everywhere that capture these events as they unfold? Who is controlling them? Why aren’t the villains smashing them?
Where the hell is that orchestra music coming from?
This happens in the future, how are we learning about it here in their past?
The Matter of Believability
It could be, that what we are actually complaining about is not usability, but believability. It may be that the problems of eye strain, privacy, and orientation are so obvious that it takes us out of the story. Breaking immersion is a cardinal sin in narrative. But it’s pretty easy (and fun) to write some simple apologetics to explain away these particular concerns.
Why is eye strain not a problem? Maybe the screens actually do go opaque when seen from a human eye, we just never see them that way because we see them from the POV of the camera.
Why is privacy not a problem? Maybe the loss of privacy is a feature, not a bug, for the fascist society being depicted; a way to keep citizens in line. Or maybe there is an opaque mode, we just don’t see any scenes where characters send dick pics, or browse porn, and would thereby need it. Or maybe characters have other, opaque devices at home specifically designed for the private stuff.
Why isn’t orientation a problem? Tech would only require face recognition for such an object to automatically orient itself correctly no matter how it is being picked up or held. The Appel Maman would only present itself downwards to the table if it was broken.
So it’s not a given that transparent screens just won’t work. Admittedly, this is some pretty heavy backworlding. But they could work.
But let’s address the other side of believability. Sci-fi makers are in a continual second-guess dance with their audience’s evolving technological literacy. It may be that Boulet’s cartoon is a bellwether, a signal that non-technological audiences are becoming so familiar with the real-world challenges of this trope that is it time for either some replacement, or some palliative hints as to why the issues he illustrates aren’t actually issues. As audience members—instead of makers—we just have to wait and see.
Sci-fi is not a usability manual.
It never was. If you look to sci-fi for what is “good” design for the real-world, you will cause frustration, maybe suffering, maybe the end of all good in the ’verse. Please see the talk I gave at the Reaktor conference a few years ago for examples, presented in increasing degrees of catastrophe.
I would say—to pointedly use the French—that the “raison d’être” of this site is exactly this. Sci-fi is so pervasive, so spectacular, so “cool,” that designers must build up a skeptical immunity to prevent its undue influence on their work.
I hope you join me on that journey. There’s sci-fi and popcorn in it for everyone.
After Joe goes through triage, he is directed to the “diagnosis area to the right.” He waits in a short queue, and then enters the diagnosis bay.
The attendant wears a SMARTSPEEK that says, “Your illness is very important to us. Welcome to the Healthmaster Inferno.”
The attendant, DR. JAGGER, holds three small metal probes, and hands each one to him in turn saying, “Uh, this one goes in your mouth. This one goes in your ear. And this one goes up your butt.” (Dark side observation about the St. God’s: Apparently what it takes to become a doctor in Idiocracy is an ability to actually speak to patients and not just let the SMARTSPEEK do all the talking.)
Joe puts one in his mouth and is getting ready to insert the rest, when a quiet beeping causes the attendant to pause and correct himself. “Shit. Hang on a second.” He takes the mouth one back and hands him another one. “This one…No.” He gathers them together, and unable to tell them apart, he shuffles them trying to figure it out, saying “This one. This one goes in your mouth.” Joe reluctantly puts the offered probe into his mouth and continues.
The diagnosis is instant (and almost certainly UNKNOWN). SMARTSPEEK says, “Thank you for waiting. Dr. Lexus will be with you shortly.”
The probes are rounded, metal cylinders, maybe a decimeter in length. They look like 3.6mm audio plugs with the tips ground off. The interface-slash-body-horror joke is that we in the audience know that you shouldn’t cross-contaminate between those orifices in a single person, much less between multiple people, and the probes look identical. (Not only that, but they aren’t cleaned or used with a sterile disposable sheath, etc.) So Joe’s not sure what he’s about to have to put in his mouth, and DR. JAGGER is too dumb to know or care.
Modeled on car wash aethetics, the bay is a molded-plastic arch, about 4 meters to a side. The inside has a bunch of janky and unsanitary looking medical probes and tools. Around the entrance of the bay are an array of backlit signs, clockwise from 7 o’ clock:
Form one line | Do not push
(Two right-facing arrows, one blue, one orange)
(A stop sign)
(A hepatitis readout, from Hepatitis A to Hepatitis F, which does not exist.)
Tumor | E-Coli | Just gas | Tapeworm | Unknown
Gout | Lice | Leprosy | Malaria
(Three left-facing arrows, orange, blue, and magenta)
(The comp created for the movie tells…) Be probe ready | Thank you!
Theoretically, the lights help patients understand what to do and what their diagnosis is. But the instruction panels don’t seem to change, and once the patient is inside the bay, they can no longer see the diagnosis panels. The people in the queue and the lobby, however, can. So not only does it rob the patients of any bodily privacy (as they’re having to ram a probe up their rears), but it also robs them of any privacy about their diagnosis. HIPPA and GDPR are rolling around in their then-500 year old graves.
A better solution would of course focus on hygiene first, offering a disposable sheath for the probes. They should still be sterilized between patients.
Second would be changing the design of the probes such that they were easy to distinguish between them. Color, shape, and labeling are initial ideas.
Third would be to constrain the probes so that…
The butt probe can’t reach up beyond the butt (maybe tying the cable to the floor? Though that means it’s likely to drop to the ground, which is clearly not sterile in this place, so maybe tying it the wall and having it klaxon loudly if it’s above butt height.)
The mouth probe can’t reach below the head (maybe tying the cable to the ceiling)
The ear probe should be smaller and ear-shaped rather than some huge eardrum-piercing thing.
And while modesty is clearly not an issue for people of Idiocracy, convention, modesty, and the law require us in our day to make this a LOT more private.
Prevention > remedy
Note that there is an error beep when Joe puts the wrong probe in his butt. Like many errors, by that time it is too late. It makes engineering sense for the machine to complain when there is a problem. It makes people sense to constrain so that errors are not possible, or at the very least, put the alarm where it will dissuade from error.
Also, can we turn the volume up on those quiet beeps to, say, 80 decibels? I think everyone’s interested in more of an alarm than a whisper for this.
A hidden, eviscerating joke
In addition to the base comedy—of treating diagnosis like a carwash, the interaction design of the missing affordances and constraints, and the poop humor of sticking a butt probe in your mouth—there is yet another layer of stupid evident here. Many of the diseases listed on the “proscenium” of the bay are ones that can be caused by, yep, ingesting feces. (Hepatitis A, Hepatitis E, tapeworm, E. “boli.”) Enjoy the full, appetizing list on Wikipedia. It’s a whole other layer of funny, and hearkens back to stories of when late-1800s doctors took umbrage at Ignaz Semmelweis’ suggestions that they wash their hands. (*huffgrumble* But we’re gentlemen!” *monocle pop*) This is that special kind of stupid when people are the cause of their own problems, and refuse to believe it because they are either proud…or idiots.
But of course, we’re so much wiser today. People are never, say, duped into voting for some sense of tribal identity despite mountains of evidence that they are voting against their community, or even their own self-interest.
Fighting the unsanitary butt plugs of the Idiocracy
“Action by action, day by day, group by group, Indivisibles are remaking our democracy. They make calls. They show up. They speak with their neighbors. They organize. And through that work, they’ve built hundreds of mini-movements in support of their local values. And now, after practice, training, and repetition, they’ve built lasting power on their home turf and a massive, collective political muscle ready to be exercised each and every day in every corner of the country.”
Donate or join the phone bankers at Indivisible to talk people into voting, and perhaps some sanity into Idiocrats. Indivisible’s mission is “to cultivate and lift up a grassroots movement of local groups to defeat the Trump agenda, elect progressive leaders, and realize bold progressive policies.”
Once a victim is wearing a Trivium Bracelet, any of Orlak’s henchmen can control the wearer’s actions. The victim’s expression is blank, suggesting that their consciousness is either comatose, twilit, or in some sort of locked in state. Their actions are controlled via a handheld remote control.
We see the remote control in use in four places in Las Luchadoras vs El Robot Asesino.
One gets clapped on Dr. Chavez to test it.
One goes on Gemma to demonstrate it.
One is removed from the robot.
One goes on Berthe to transform her to Black Electra.