Disclosure (1994)

4 Dec 2023 by scifihughf

Our next 3D file browsing system is from the 1994 film Disclosure. Thanks to site reader Patrick H Lauke for the suggestion.

Like Jurassic Park, Disclosure is based on a Michael Crichton novel, although this time without any dinosaurs. (Would-be scriptwriters should compare the relative success of these two films when planning a study program.) The plot of the film is corporate infighting within Digicom, manufacturer of high tech CD-ROM drives—it was the 1990s—and also virtual reality systems. Tom Sanders, executive in charge of the CD-ROM production line, is being set up to take the blame for manufacturing failures that are really the fault of cost-cutting measures by rival executive Meredith Johnson.

The Corridor: Hardware Interface

The virtual reality system is introduced at about 40 minutes, using the narrative device of a product demonstration within the company to explain to the attendees what it does. The scene is nicely done, conveying all the important points we need to know in two minutes. (To be clear, some of the images used here come from a later scene in the film, but it’s the same system in both.)

The process of entangling yourself with the necessary hardware and software is quite distinct from interacting with the VR itself, so let’s discuss these separately, starting with the physical interface.

Tom wearing VR headset and one glove, being scanned. Disclosure (1994)

In Disclosure the virtual reality user wears a headset and one glove, all connected by cables to the computer system. Like most virtual reality systems, the headset is responsible for visual display, audio, and head movement tracking; the glove for hand movement and gesture tracking.

There are two “laser scanners” on the walls. These are the planar blue lights, which scan the user’s body at startup. After that they track body motion, although since the user still has to wear a glove, the scanners presumably just track approximate body movement and orientation without fine detail.

Lastly, the user stands on a concave hexagonal plate covered in embedded white balls, which allows the user to “walk” on the spot.

Closeup of user standing on curved surface of white balls. Disclosure (1994)

Searching for Evidence

The scene we’re most interested in takes place later in the film, the evening before a vital presentation which will determine Tom’s future. He needs to search the company computer files for evidence against Meredith, but discovers that his normal account has been blocked from access. He knows though that the virtual reality demonstrator is on display in a nearby hotel suite, and also knows about the demonstrator having unlimited access. He sneaks into the hotel suite to use The Corridor. Tom is under a certain amount of time pressure because a couple of company VIPs and their guests are downstairs in the hotel and might return at any time.

The first step for Tom is to launch the virtual reality system. This is done from an Indy workstation, using the regular Unix command line.

The command line to start the virtual reality system. Disclosure (1994)

Next he moves over to the VR space itself. He puts on the glove but not the headset, presses a key on the keyboard (of the VR computer, not the workstation), and stands still for a moment while he is scanned from top to bottom.

Real world Tom, wearing one VR glove, waits while the scanners map his body. Disclosure (1994)

On the left is the Indy workstation used to start the VR system. In the middle is the external monitor which will, in a moment, show the third person view of the VR user as seen earlier during the product demonstration.

Now that Tom has been scanned into the system, he puts on the headset and enters the virtual space.

The Corridor: Virtual Interface

“The Corridor,” as you’ve no doubt guessed, is a three dimensional file browsing program. It is so named because the user will walk down a corridor in a virtual building, the walls lined with “file cabinets” containing the actual computer files.

Three important aspects of The Corridor were mentioned during the product demonstration earlier in the film. They’ll help structure our tour of this interface, so let’s review them now, as they all come up in our discussion of the interfaces.

There is a voice-activated help system, which will summon a virtual “Angel” assistant.
Since the computers themselves are part of a multi-user network with shared storage, there can be more than one user “inside” The Corridor at a time.
Users who do not have access to the virtual reality system will appear as wireframe body shapes with a 2D photo where the head should be.
There are no access controls and so the virtual reality user, despite being a guest or demo account, has unlimited access to all the company files. This is spectacularly bad design, but necessary for the plot.

With those bits of system exposition complete, now we can switch to Tom’s own first person view of the virtual reality environment.

Virtual world Tom watches his hands rezzing up, right hand with glove. Disclosure (1994)

There isn’t a real background yet, just abstract streaks. The avatar hands are rezzing up, and note that the right hand wearing the glove has a different appearance to the left. This mimics the real world, so eases the transition for the user.

Overlaid on the virtual reality view is a Digicom label at the bottom and four corner brackets which are never explained, although they do resemble those used in cameras to indicate the preferred viewing area.

To the left is a small axis indicator, the three green lines labeled X, Y, and Z. These show up in many 3D applications because, silly though it sounds, it is easy in a 3D computer environment to lose track of directions or even which way is up. A common fix for the user being unable to see anything is just to turn 180 degrees around.

We then switch to a third person view of Tom’s avatar in the virtual world.

Tom is fully rezzed up, within cloud of visual static. Disclosure (1994)

This is an almost photographic-quality image. To remind the viewers that this is in the virtual world rather than real, the avatar follows the visual convention described in chapter 4 of Make It So for volumetric projections, with scan lines and occasional flickers. An interesting choice is that the avatar also wears a “headset”, but it is translucent so we can see the face.

Now that he’s in the virtual reality, Tom has one more action needed to enter The Corridor. He pushes a big button floating before him in space.

Tom presses one button on a floating control panel. Disclosure (1994)

This seems unnecessary, but we can assume that in the future of this platform, there will be more programs to choose from.

The Corridor rezzes up, the streaks assembling into wireframe components which then slide together as the surfaces are shaded. Tom doesn’t have to wait for the process to complete before he starts walking, which suggests that this is a Level Of Detail (LOD) implementation where parts of the building are not rendered in detail until the user is close enough for it to be worth doing.

Tom enters The Corridor. Nearby floor and walls are fully rendered, the more distant section is not complete. Disclosure (1994)

The architecture is classical, rendered with the slightly artificial-looking computer shading that is common in 3D computer environments because it needs much less computation than trying for full photorealism.

Instead of a corridor this is an entire multistory building. It is large and empty, and as Tom is walking bits of architecture reshape themselves, rather like the interior of Hogwarts in Harry Potter.

Although there are paintings on some of the walls, there aren’t any signs, labels, or even room numbers. Tom has to wander around looking for the files, at one point nearly “falling” off the edge of the floor down an internal air well. Finally he steps into one archway room entrance and file cabinets appear in the walls.

Tom enters a room full of cabinets. Disclosure (1994)

Unlike the classical architecture around him, these cabinets are very modern looking with glowing blue light lines. Tom has found what he is looking for, so now begins to manipulate files rather than browsing.

Virtual Filing Cabinets

The four nearest cabinets according to the titles above are

Communications
Operations
System Control
Research Data.

There are ten file drawers in each. The drawers are unmarked, but labels only appear when the user looks directly at it, so Tom has to move his head to centre each drawer in turn to find the one he wants.

Tom looks at one particular drawer to make the title appear. Disclosure (1994)

The fourth drawer Tom looks at is labeled “Malaysia”. He touches it with the gloved hand and it slides out from the wall.

Tom withdraws his hand as the drawer slides open. Disclosure (1994)

Inside are five “folders” which, again, are opened by touching. The folder slides up, and then three sheets, each looking like a printed document, slide up and fan out.

Axis indicator on left, pointing down. One document sliding up from a folder. Disclosure (1994)

Note the tilted axis indicator at the left. The Y axis, representing a line extending upwards from the top of Tom’s head, is now leaning towards the horizontal because Tom is looking down at the file drawer. In the shot below, both the folder and then the individual documents are moving up so Tom’s gaze is now back to more or less level.

Close up of three “pages” within a virtual document. Disclosure (1994)

At this point the film cuts away from Tom. Rival executive Meredith, having been foiled in her first attempt at discrediting Tom, has decided to cover her tracks by deleting all the incriminating files. Meredith enters her office and logs on to her Indy workstation. She is using a Command Line Interface (CLI) shell, not the standard SGI Unix shell but a custom Digicom program that also has a graphical menu. (Since it isn’t three dimensional it isn’t interesting enough to show here.)

Tom uses the gloved hand to push the sheets one by one to the side after scanning the content.

Tom scrolling through the pages of one folder by swiping with two fingers. Disclosure (1994)

Quick note: This is harder than it looks in virtual reality. In a 2D GUI moving the mouse over an interface element is obvious. In three dimensions the user also has to move their hand forwards or backwards to get their hand (or finger) in the right place, and unless there is some kind of haptic feedback it isn’t obvious to the user that they’ve made contact.

Tom now receives a nasty surprise.

The shot below shows Tom’s photorealistic avatar at the left, standing in front of the open file cabinet. The green shape on the right is the avatar of Meredith who is logged in to a regular workstation. Without the laser scanners and cameras her avatar is a generic wireframe female humanoid with a face photograph stuck on top. This is excellent design, making The Corridor usable across a range of different hardware capabilities.

Tom sees the Meredith avatar appear. Disclosure (1994)

Why does The Corridor system place her avatar here? A multiuser computer system, or even just a networked file server, obviously has to know who is logged on. Unix systems in general and command line shells also track which directory the user is “in”, the current working directory. Meredith is using her CLI interface to delete files in a particular directory so The Corridor can position her avatar in the corresponding virtual reality location. Or rather, the avatar glides into position rather than suddenly popping into existence: Tom is only surprised because the documents blocked his virtual view.

Quick note: While this is plausible, there are technical complications. Command line users often open more than one shell at a time in different directories. In such a case, what would The Corridor do? Duplicate the wireframe avatar in each location? In the real world we can’t be in more than one place at a time, would doing so contradict the virtual reality metaphor?

There is an asymmetry here in that Tom knows Meredith is “in the system” but not vice versa. Meredith could in theory use CLI commands to find out who else is logged on and whether anyone was running The Corridor, but she would need to actively seek out that information and has no reason to do so. It didn’t occur to Tom either, but he doesn’t need to think about it, the virtual reality environment conveys more information about the system by default.

We briefly cut away to Meredith confirming her CLI delete command. Tom sees this as the file drawer lid emitting beams of light which rotate down. These beams first erase the floating sheets, then the folders in the drawer. The drawer itself now has a red “DELETED” label and slides back into the wall.

Tom watches Meredith deleting the files in an open drawer. Disclosure (1994)

Tom steps further into the room. The same red labels appear on the other file drawers even though they are currently closed.

Tom watches Meredith deleting other, unopened, drawers. Disclosure (1994)

Talking to an Angel

Tom now switches to using the system voice interface, saying “Angel I need help” to bring up the virtual reality assistant. Like everything else we’ve seen in this VR system the “angel” rezzes up from a point cloud, although much more quickly than the architecture: people who need help tend to be more impatient and less interested in pausing to admire special effects.

The voice assistant as it appears within VR. Disclosure (1994)

Just in case the user is now looking in the wrong direction the angel also announces “Help is here” in a very natural sounding voice.

The angel is rendered with white robe, halo, harp, and rapidly beating wings. This is horribly clichéd, but a help system needs to be reassuring in appearance as well as function. An angel appearing as a winged flying serpent or wheel of fire would be more original and authentic (yes, really: Biblically Accurate Angels) but users fleeing in terror would seriously impact the customer satisfaction scores.

Now Tom has a short but interesting conversation with the angel, beginning with a question:

Tom
Is there any way to stop these files from being deleted?
Angel
I’m sorry, you are not level five.
Tom
Angel, you’re supposed to protect the files!
Angel
Access control is restricted to level five.

Tom has made the mistake, as described in chapter 9 Anthropomorphism of the book, of ascribing more agency to this software program than it actually has. He thinks he is engaged in a conversational interface (chapter 6 Sonic Interfaces) with a fully autonomous system, which should therefore be interested in and care about the wellbeing of the entire system. Which it doesn’t, because this is just a limited-command voice interface to a guide.

Even though this is obviously scripted, rather than a genuine error I think this raises an interesting question for real world interface designers: do users expect that an interface with higher visual quality/fidelity will be more realistic in other aspects as well? If a voice interface assistant has a simple polyhedron with no attempt at photorealism (say, like Bit in Tron) or with zoomorphism (say, like the search bear in Until the End of the World) will users adjust their expectations for speech recognition downwards? I’m not aware of any research that might answer this question. Readers?

Despite Tom’s frustration, the angel has given an excellent answer – for a guide. A very simple help program would have recited the command(s) that could be used to protect files against deletion. Which would have frustrated Tom even more when he tried to use one and got some kind of permission denied error. This program has checked whether the user can actually use commands before responding.

This does contradict the earlier VR demonstration where we were told that the user had unlimited access. I would explain this as being “unlimited read access, not write”, but the presenter didn’t think it worthwhile to go into such detail for the mostly non-technical audience.

Tom is now aware that he is under even more time pressure as the Meredith avatar is still moving around the room. Realising his mistake, he uses the voice interface as a query language.

“Show me all communications with Malaysia.”
“Telephone or video?”
“Video.”

This brings up a more conventional looking GUI window because not everything in virtual reality needs to be three-dimensional. It’s always tempting for a 3D programmer to re-implement everything, but it’s also possible to embed 2D GUI applications into a virtual world.

Tom looks at a conventional 2D display of file icons inside VR. Disclosure (1994)

The window shows a thumbnail icon for each recorded video conference call. This isn’t very helpful, so Tom again decides that a voice query will be much faster than looking at each one in turn.

“Show me, uh, the last transmission involving Meredith.”

There’s a short 2D transition effect swapping the thumbnail icon display for the video call itself, which starts playing at just the right point for plot purposes.

Tom watches a previously recorded video call made by Meredith (right). Disclosure (1994)

While Tom is watching and listening, Meredith is still typing commands. The camera orbits around behind the video conference call window so we can see the Meredith avatar approach, which also shows us that this window is slightly three dimensional, the content floating a short distance in front of the frame. The film then cuts away briefly to show Meredith confirming her “kill all” command. The video conference recordings are deleted, including the one Tom is watching.

Tom is informed that Meredith (seen here in the background as a wireframe avatar) is deleting the video call. Disclosure (1994)

This is also the moment when the downstairs VIPs return to the hotel suite, so the scene ends with Tom managing to sneak out without being detected.

Virtual reality has saved the day for Tom. The documents and video conference calls have been deleted by Meredith, but he knows that they once existed and has a colleague retrieve the files he needs from the backup tapes. (Which is good writing: the majority of companies shown in film and TV never seem to have backups for files, no matter how vital.) Meredith doesn’t know that he knows, so he has the upper hand to expose her plot.

Analysis

How believable is the interface?

I won’t spend much time on the hardware, since our focus is on file browsing in three dimensions. From top to bottom, the virtual reality system starts as believable and becomes less so.

Hardware

The headset and glove look like real VR equipment, believable in 1994 and still so today. Having only one glove is unusual, and makes impossible some of the common gesture actions described in chapter 5 of Make It So, which require both hands.

The “laser scanners” that create the 3D geometry and texture maps for the 3D avatar and perform real time body tracking would more likely be cameras, but that would not sound as cool.

And lastly the walking platform apparently requires our user to stand on large marbles or ball bearings and stay balanced while wearing a headset. Uh…maybe…no. Apologetics fails me. To me it looks like it would be uncomfortable to walk on, almost like deterrent paving.

Software

The Corridor, unlike the 3D file browser used in Jurassic Park, is a special effect created for the film. It was a mostly-plausible, near future system in 1994, except for the photorealistic avatar. Usually this site doesn’t discuss historical context (the “new criticism” stance), but I think in this case it helps to explain how this interface would have appeared to audiences almost two decades ago.

I’ll start with the 3D graphics of the virtual building. My initial impression was that The Corridor could have been created as an interactive program in 1994, but that was my memory compressing the decade. During the 1990s 3D computer graphics, both interactive and CGI, improved at a phenomenal rate. The virtual building would not have been interactive in 1994, was possible on the most powerful systems six years later in 2000, and looks rather old-fashioned compared to what the game consoles of the 21st C can achieve.

For the voice interface I made the opposite mistake. Voice interfaces on phones and home computing appliances have become common in the second decade of the 21st C, but in reality are much older. Apple Macintosh computers in 1994 had text-to-speech synthesis with natural sounding voices and limited vocabulary voice command recognition. (And without needing an Internet connection!) So the voice interface in the scene is believable.

The multi-user aspects of The Corridor were possible in 1994. The wireframe avatars for users not in virtual reality are unflattering or perhaps creepy, but not technically difficult. As a first iteration of a prototype system it’s a good attempt to span a range of hardware capabilities.

The virtual reality avatar, though, is not believable for the 1990s and would be difficult today. Photographs of the body, made during the startup scan, could be used as a texture map for the VR avatar. But live video of the face would be much more difficult, especially when the face is partly obscured by a headset.

How well does the interface inform the narrative of the story?

The virtual reality system in itself is useful to the overall narrative because it makes the Digicom company seem high tech. Even in 1994 CD-ROM drives weren’t very interesting.

The Corridor is essential to the tension of the scene where Tom uses it to find the files, because otherwise the scene would be much shorter and really boring. If we ignore the virtual reality these are the interface actions:

Tom reads an email.
Meredith deletes the folder containing those emails.
Tom finds a folder full of recorded video calls.
Tom watches one recorded video call.
Meredith deletes the folder containing the video calls.

Imagine how this would have looked if both were using a conventional 2D GUI, such as the Macintosh Finder or MS Windows Explorer. Double click, press and drag, double click…done.

The Corridor slows down Tom’s actions and makes them far more visible and understandable. Thanks to the virtual reality avatar we don’t have to watch an actor push a mouse around. We see him moving and swiping, be surprised and react; and the voice interface adds extra emotion and some useful exposition. It also helps with the plot, giving Tom awareness of what Meredith is doing without having to actively spy on her, or look at some kind of logs or recordings later on.

Meredith, though, can’t use the VR system because then she’d be aware of Tom as well. Using a conventional workstation visually distinguishes and separates Meredith from Tom in the scene.

So overall, though the “action” is pretty mundane, it’s crucial to the plot, and the VR interface helps make this interesting and more engaging.

How well does the interface equip the character to achieve their goals?

As described in the film itself, The Corridor is a prototype for demonstrating virtual reality. As a file browser it’s awful, but since Tom has lost all his normal privileges this is the only system available, and he does manage to eventually find the files he needs.

At the start of the scene, Tom spends quite some time wandering around a vast multi-storey building without a map, room numbers, or even coordinates overlaid on his virtual view. Which seems rather pointless because all the files are in one room anyway. As previously discussed for Johnny Mnemonic, walking or flying everywhere in your file system seems like a good idea at first, but often becomes tedious over time. Many actual and some fictional 3D worlds give users the ability to teleport directly to any desired location.

Then the file drawers in each cabinet have no labels either, so Tom has to look carefully at each one in turn. There is so much more the interface could be doing to help him with his task, and even help the users of the VR demo learn and explore its technology as well.

Contrast this with Meredith, who uses her command line interface and 2D GUI to go through files like a chainsaw.

Tom becomes much more efficient with the voice interface. Which is just as well, because if he hadn’t, Meredith would have deleted the video conference recordings while he was still staring at virtual filing cabinets. However neither the voice interface nor the corresponding file display need three dimensional graphics.

There is hope for version 2.0 of The Corridor, even restricting ourselves to 1994 capabilities. The first and most obvious is to copy 2D GUI file browsers, or the 3D file browser from Jurassic Park, and show the corresponding text name next to each graphical file or folder object. The voice interface is so good that it should be turned on by default without requiring the angel. And finally add some kind of map overlay with a you are here moving dot, like the maps that players in 3D games such as Doom could display with a keystroke.

Film making challenge: VR on screen

Virtual reality (or augmented reality systems such as Hololens) provide a better viewing experience for 3D graphics by creating the illusion of real three dimensional space rather than a 2D monitor. But it is always a first person view and unlike conventional 2D monitors nobody else can usually see what the VR user is seeing without a deliberate mirroring/debugging display. This is an important difference from other advanced or speculative technologies that film makers might choose to include. Showing a character wielding a laser pistol instead of a revolver or driving a hover car instead of a wheeled car hardly changes how to stage a scene, but VR does.

So, how can we show virtual reality in film?

There’s the first-person view corresponding to what the virtual reality user is seeing themselves. (Well, half of what they see since it’s not stereographic, but it’s cinema VR, so close enough.) This is like watching a screencast of someone else playing a first person computer game, the original active experience of the user becoming passive viewing by the audience. Most people can imagine themselves in the driving seat of a car and thus make sense of the turns and changes of speed in a first person car chase, but the film audience probably won’t be familiar with the VR system depicted and will therefore have trouble understanding what is happening. There’s also the problem that viewing someone else’s first-person view, shifting and changing in response to their movements rather than your own, can make people disoriented or nauseated.

A third-person view is better for showing the audience the character and the context in which they act. But not the diegetic real-world third-person view, which would be the character wearing a geeky headset and poking at invisible objects. As seen in Disclosure, the third person view should be within the virtual reality.

But in doing that, now there is a new problem: the avatar in virtual reality representing the real character. If the avatar is too simple the audience may not identify it with the real world character and it will be difficult to show body language and emotion. More realistic CGI avatars are increasingly expensive and risk falling into the Uncanny Valley. Since these films are science fiction rather than factual, the easy solution is to declare that virtual reality has achieved the goal of being entirely photorealistic and just film real actors and sets. Adding the occasional ripple or blur to the real world footage to remind the audience that it’s meant to be virtual reality, again as seen in Disclosure, is relatively cheap and quick.
So, solving all these problems results in the cinematic trope we can call Extradiegetic Avatars, which are third-person, highly-lifelike “renderings” of characters, with a telltale Hologram Projection Imperfection for audience readability, that may or may not be possible within the world of the film itself.

IMDB: https://www.imdb.com/title/tt0109635/Currently streaming on:

Identity Processing Program of America

2 Nov 2018 by Christopher Noessel

After his initial arrest, Joe is led by a noose stick (and a police officer speaking some devolved version of copspeak) to a machine to get an identity tattoo. Joe sits in the chair and a synthesized voice says, “Welcome to the Identity Processing Program of America. Please insert your forearm into the forearm receptacle.” Joe does as instructed and it locks his arm into place. A screen in front of him shows the legend “Identity Processing Program of America” superimposed over an USA pattern made up of company names and Carls Junior amputated star logos. Five rectangles across the top are labeled: System, Identity, Verify, Imprint, and Done.

It prompts him to “…speak your name as it appears on your current federal identity card, document number G24L8.” Joe says, “I’m not sure if—“ The machine interprets this as input and blinks the name as it says, “You have entered the name ‘Not Sure.’ Is this correct, Not Sure?”

Joe tries to correct it, saying, “No…it’s not correct.” On the word “correct” it dings and continues, “Thank you. ‘Not’ is correct.” “Not” stops blinking in the interface.

“…Is ‘Sure’ correct?” Joe has patience and tries to correct it. “No, it’s not. My name is Joe.” It blurts out some error beeps. “You’ve already confirmed your first name is Not. Please confirm your last name, ‘Sure.’”

Joe: My…My last name is not Sure.
Kiosk: Thank you, Not Sure.
Joe: No. What I mean is my name is Joe.
Kiosk: Confirmation is complete. Please wait while I tattoo your new identity on your arm.
The machine begins to shake and make noises. Joe says, “Wait a second. Can we start over? Can I cancel this?” He sees a progress bar, labeled “Tattoo In Progress…”

“Can we cancel this and just go back to the beginning? They’re gonna tat—Ow. Could I speak to your supervisor? Ow!” While he’s trying to wrench his arm free, the machine instructs him, “Please hold still for your photograph.” It flashes an unflattering picture of him and the clamp on his arm releases. He removes his arm to see the new tattoo. The screen shows him his identity card.

In exasperation from the whole ordeal he mumbles, “Oh, that’s fuckin’ great.”

This scene is played for the Vaudevillian yuks, but it does illustrate some problems with conversational design. And note that, if you’re interested in this topic, let me make an early shout out to the book Conversational Design by Erika Hall, published earlier this year.

Understanding intent

When it hears Joe say “I’m not sure…” it takes it as a literal answer. It does not recognize that Joe isn’t answering the question. It is one meta-level up. He has a question about the question being asked. Humans are pretty good at recognizing when another human is breaking the usual logic of adjacency pairs and not providing an answer to the question. (This was discussed in Make It So in Chapter 5, “Gestural Interfaces” in relation to Minority Report) Computers have a harder time of it. If this kiosk understood it, it would be know that he’s not answering the question, and resolve what conversational analysis calls “the expansion” before returning to the question. (Disclosure: I work there and know the guy who wrote that.)

Aside: Douglas Hofstadter in his mind-expanding book Godel, Escher, and Bach, writes about the trick question “Have you stopped beating your spouse?” for which neither “yes” nor “no” are good answers, but the only “correct” answers according to the binary frame of the question. In that text he introduces the eastern answer of “mu” (or “wu” in Chinese languages) that means roughly “the answer does not fit the question.” So it can be said that computers have a hard time understanding mu.

Designers of digital assistants have to wrangle with this, but it’s rarely a problem that the individual designer must wrestle with. Language and naming are informal, slippery notions as far as computers are concerned, so it’s understandably a hard problem. It’s entirely possible that someone has chosen “Not Sure” as a name, but it’s highly unlikely. And that’s another problem.

Understanding likelihood

Understanding intent might be a little easier if the computer could recognize that “Not” and “Sure” are unlikely values for a name. (Even in Idiocracy where names tend to be brands like Lexus, and Frito, and Biggiez. More on this later.) If it knew that, it would have a low confidence that it “heard” correctly, and shift into a repair or at least clarification mode. “‘Not’ would be very uncommon name. Let me be extra careful, here…” It could even shift into a more deliberate mode of input, like a keyboard, or asking him to spell his name out (or, you know, cancel the whole thing.)

Catching not

When the kiosk is asking for confirmation, it hears Joe say, “No it’s not correct” and registers the keyword “correct” but misses the function word “not” which completely flips the meaning.

Again, avoiding this speech-to-text error would be a developer’s task, but dealing with the back-and-forth would definitely fall to a designer. When clarifying low-confidence input, users should be able to provide discrete high-confidence feedback.

Joe could, for instance, be shown the kiosk’s (stupid) understanding of his input and—since this has a pretty permanent consequence—wait on his confirmation and providing the simple option to redo it so he can try some other tactic to getting “Joe Bauers” in there until he gets it right.
But, of course, this is Idiocracy, and Joe is stuck with it.

But we’re not

I mean, Republicans have done all they can to suppress votes that don’t favor them. They don’t care about Democracy or the will of the American people as they do staying in power to serve their 1% overlords. But research shows that people who have a plan to vote are more likely to actually do it, and if we all do it, we can overwhelm them with sheer numbers.
There are lots of tools to help you make a plan, but let’s send some traffic to our friends at Planned Parenthood. They’ve been under a lot of pressure during this administration. Maybe throw them a few sheckles while you’re there. Not for the election, but because you’re a good person. And vote all of them out.

The Court of Idiots

31 Oct 2018 by Christopher Noessel

It’s Halloween, as if the news of the past week were not scary enough. Pipe bombs to Democratic leaders. The largest massacre of Jewish people in on American soil in history. The murder of two black senior citizens by a white supremacist in Kentucky. Now let’s add to it with this nightmare scene from Idiocracy. Full disclosure: We’re covering technology as old as civilization here, so there won’t be any screen interfaces.

Joe is wheeled into the courtroom in a cage. There is a large gallery there, all of whom are booing him. One throws his milkshake at the accused. Others throw trash. The narrator says, “Joe was arrested for not paying his hospital bill and not having his IPP tattoo. He would soon discover that in the future, Justice was not only blind, but had become rather retarded as well.”

Joe is let out of his cage. The judge, identified by his name plate as The Honorable Hector “The Hangman,” stands at his bench in a spotlight in front of a wall of logos, grinning in anticipation at a new victim. He slams a massive gavel and shouts at the booing crowd, “Listen up! Now. I’m fixin’ to commensurate this trial here. [All of this is sic.] We gon’ see if we can’t come up with a verdict up in here. Now. Since y’all say y’ain’t got no money, we have proprietarily obtained you one of them court-appointed lawyers. So, put your hands together to give it up for Frito Pendejo!”

Frito, wearing a long-sleeve t-shirt with “ATTORNEY AT LAW” running down the sleeve, sits and looks at a paper on the counsel table, saying “Says here you, uh, robbed a hospital?! Why’d you do that?” Joe says, “But I’m not guilty!” Frito replies, “That’s not what the other lawyer said.”

When the trial officially starts, the judge says, “Shut up. Shut up! Now, prosecutor. Why you think he done it?” The prosecutor stands up and says, “K. Number one, your honor. Just look at him.” Most everyone in the courtroom, including, Frito, laughs at this. Frito stands up and says, “He talks like a fag, too!” When more laughter dies down, the prospector continues, saying, “We’ve got all this, like, evidence, of how, like, this guy didn’t even pay at the hospital and I heard that he doesn’t even have his tattoo.” There are collective gasps. He continues, “I know! And I’m all, ‘You gotta be shitting me.’ But check this out, man, judge should be like, ‘Guilty!’ Peace.” The gallery erupts with applause and cheers. There is one guy wearing a helmet with a camera mounted to the top and he takes it all in dumbly.

When Frito stands to raise an objection, he says, “Your honor, I object…that this guy, also broke my apartment and shit.” There are gasps from the gallery, and Frito feels emboldened. “Yeah! And you know what else, I object that he’s not going to have any money to pay me after he pays back all the money he stole from the hospital!” This last bit is addressed to the gallery. They shout in anger. Frito finishes, saying, “And I object. I OBJECT that he interrupted me while I was watching, OW, MY BALLS! THAT IS NOT OK! I REST MY CASE.”

Joe stands and says, “Your Honor, I’m pretty sure we have a mistrial, here, sir.” Hector looks confused at this statement. Frito gets hostile and says, “I’m going to mistrial my foot up your ass if you don’t shut up!”

Joe ignores him and says, “Please listen!” The prosecuting attorney simply mocks him, “Please listen!” Hector laughs. The lawyers high five each other.

The narrator says, “Joe stated his case, logically and passionately. But his perceived effeminate voice only drew big gales of stupid laughter. Without adequate legal representation, Joe was given a stiff sentence.”

So…this nightmare

For most Americans, the drafting and adjudication of laws feels like something that happens “out there.” But upholding the law is something that, through jury duty or being part of a court case, is something most citizens directly experience some time in their lives. So it may seem like familiar, everyday stuff.

WitnessForTheProsecution575-600x365 — Wait. I know this.

But take that less trodden path to ask why it is the way it is, and you’ll eventually find yourself at the foundations of civilization and in the throes of philosophy. A key premise and promise of civilized society is having a fair and rational place for citizens to air grievances, and institutions that make and enforce just laws. In practice it’s far from perfect, but I’d bet most folks agree it’s better than the alternatives of lawlessness, personal violence, revenge, tribal feuds, and vigilantism.

Laws and courts are institutions that are so old and foundational, it’s hard to remember that:

They were, in fact, invented. There was a time before them.
There were reasons for the way they were designed.
Their design, like all designs, aren’t perfect, and involve some trade offs.
They evolve over time.
There are alternate designs to consider.

Idiocracy illustrates how important its norms are, and how tragic it can be when those norms are lost, and everyone involved is a fucking idiot. It’s not exactly monster rampage, body horror, or torture porn, but this scene is as scary a thing as you could ask for Halloween.

Violence as a means

Frito threatens violence a lot. So do a lot of the people in Idiocracy. I’m not even sure if they mean it, but they think that threatening or beating or shooting someone into silence is a fine way to disagree. One of the key promises of civilization is to undo that might-makes-right bullshit that kept generations of peasants suffering while an aristocracy lived the high life peeing in hallways, flubbing courtly love, and stuffng birds into each other like meaty nesting-dolls. We’ve moved on.

Skepticism

It’s funny-terrifying that the case made against Joe is so stupid. There are crass appeals to emotion. Prosecution says that there’s “all this, like, evidence” but does not actually share any, like, evidence. Frito riles the Gallery up with insults “he talks like a fag” and personal grievances that aren’t germane to the case. (Ow My Balls! should not be evidence in, well, anything.) It’s emotional theater and the audience eats it up, because what they’re there for is hits to the amygdala bong: The emotional highs and lows that you might get from a sports show. People screaming in cheers when their confirmation bias is rewarded, or fury when there’s a bit of cognitive dissonance. There’s probably nothing like a Fair Witness left on the planet, and if it weren’t for Joe’s intelligence that gets him out of jail, he would wind up suffering in prison for no reason at all. The lack of skepticism, of applying doubt to all things—especially the things that thrill you emotionally—causes unnecessary suffering. Sure, it’s an unjust universe, but civilization was invented to try and hold a little light against that darkness. The citizenry of Idiocracy are gullible and blithely self-serving to a cruel fault.

Impartiality

The system that pits prosecution against defense and presumes is called “adversarial,” since each counsel will present opposing cases to an impartial judge or jury, who is presumed competent to sort through the competing claims to come to the truth. It’s contrasted with an inquisitorial (or nonadversarial) system where the judges run the line of inquiry. Congressional hearings, for example, are much more of an inquisitorial system. Congresspeople ask the questions, rather than the lawyers.

The adversarial system depends on the impartiality of the judge and jury. If they presume guilt or innocence from the start, why have the trial at all? (This is sometimes called a kangaroo court, where procedure and ethics are ignored, and the outcome is a foregone conclusion.) Only dispassionate, neutral participants can look past their feelers to find a fair and impartial truth. You need people capable of impartial, critical thinking to look past the tricks and their own cognitive biases. That’s out the window in Idiocracy.

Joe is presumed guilty by literally everyone involved. It’s not wholly their fault. Joe is brought in wearing prison fatigues. He is wheeled in a cage. On the surface, he looks guilty. Humans have an availability bias, wherein easily recalled information is believed to be true and representative, and for very stupid people, that can be strictly what they see. And Joe looks guilty. Everyone around me is shouting that he’s guilty. So, the idiot thinks, he must be guilty.

Posers, surface evidence, and meaning

One of the worst things about the scene—and this plays out in the movie generally—is how people’s status is based on stupid, surface markers. For instance: Why do they accept that the judge is smart? Well, he uses fancy, “faggy” words like “commensurate” (It’s a malaprop. He means commence. But the Idiocrats are too stupid to know this.) and “proprietarily.” (I think he means “properly,” but it’s marvelously hard to tell.) Investigating this moves us very quickly into the semiosis treadmill, whereby a sign comes slowly to become the signified, even when it stupidly contradicts the original thing that was signified.

How do courts protect against idiocy?

It doesn’t always. But generally, there are lots of checks against incompetence. Anyone can request a mistrial if it’s shown that counsel, a judge, or a jury is incompetent, prejudiced, or doesn’t follow the rules. Lawyers can be reviewed and disbarred. Remedy for judges varies by state, but judges can be impeached, or undergo judicial review and be removed from their office. Their cases can be overturned by higher courts. (Yes, even that one.)

But notably all these presume that the incompetence is abnormal, and that others are competent to review and judge their competence. When everyone either gets so stupid or so corrupt that they have no interest in checking each other for fairness, the system just collapses.

Can design do anything?

This blog is primarily targeted at people who work in technology and love sci-fi. (Like I do.) A natural question is: Can design do anything to help inoculate courts against Idiocracy? And normally, I try to offer some hopeful answer to these questions, but this one has me stumped. What design interventions could we impose to increase skepticism? Impartiality? Critical thinking? Understanding? Seriously, if you have something, I’d love to hear it, because as of right now, I’ve got nothing.

How’s that for a Halloween scare?

Trick or Treat

Any Trump acolyte who has gleefully shouted “Lock her up! Lock her up!” at one of their farrowing-crate rallies, despite Clinton’s exoneration by the FBI, despite Trump’s own ongoing litany of scandals (including that ongoing issue of security), is guilty of this same kind of mob mentality as these morons in Idiocracy. It is all raw hate without reason. Hits from the tribalism bong. It’s not just that the mob hasn’t heard reason, it’s not interested in reason, only in justification. Only in thinking of political conversations as hobo fights. Did our guy land a hit? Insult? Hur dur, yarp.

It’s pretty soul crushing. So let’s have some spooky, escapist fun tonight for self-care. But as of tomorrow we’ll have 6 days and we should hit it hard.

The Cloak of Levitation, Part 4: Improvements

18 Jul 2017 by Christopher Noessel

In prior posts we looked at an overview of the cloak, pondered whether it could ever work in reality (Mostly, in the far future), and whether or not the cloak could be considered agentive. (Mostly, yes.) In this last post I want to look at what improvements we might make if I was designing something akin to this for the real world.

Given its wealth of capabilities, the main complaint might be its lack of language.

A mute sidekick

It has a working theory of mind, a grasp of abstract concepts, and intention, so why does it not use language as part of a toolkit to fulfill its duties? Let’s first admit that mute sidekicks are kind of a trope at this point. Think R2D2, Silent Bob, BB8, Aladdin’s Magic Carpet (Disney), Teller, Harpo, Bernardo / Paco (admittedly obscure), Mini-me. They’re a thing.

Yes, I know she could talk to other fairies, but not to *Peter*.

Despite being a trope, its muteness in a combat partner is a significant impediment. Imagine its being able to say, “Hey Steve, he’s immune to the halberd. But throw that ribcage-looking thing on the wall at him, and you’ll be good.” Strange finds himself in life-or-death situations pretty much constantly, so having to disambiguate vague gestures wastes precious time that might make the difference between life and death. For, like, everyone on Earth.

Additionally, its muteness makes it very difficult for Strange to ever understand the reasoning behind any but its most obvious actions. That would be very important if it ever did anything ethically questionable, because intent would be difficult to gauge without the abstract representation of language.

So between the speed and clarity gains of language, I’d want to equip the Cloak with language: spoken, sign, direct-to-brain, or maybe even a mystical weave which, like the Marauder’s Map in Harry Potter, could mystically transform in real-time to convey linguistic information.

Using apologetics, though, we can see that the muteness isn’t a bug, it’s an important feature. Read on.

But, the hero…

One of the things writers “buy” with a mute sidekick is it lets the hero be the hero. If the Cloak could speak, suddenly we have a “sidekick” who knows much more and can do a few more things—i.e. fly—than our hero can. This is not good for optics. Note that the Cloak’s last intention seems to be “Keep him looking sorcererly,” which implies a service status, and which it could not do if it upstaged him. So from a narrative perspective, the muteness means they are less like a crime fighting duo, and more of a hero-with-a-sidekick, or dare I note it: a familiar.

Also if you contrast the cape with JARVIS and the Iron HUD, note that this has language, but that conversation is wholly private with Tony. To an outsider, it looks like Tony is just the Iron Man. They may have no clue that it’s probably all the suit. So JARVIS’ use of language doesn’t get in the way of keeping Tony looking hero-ey. I’d even go so far as to say that the muteness is the thing that lets the Cloak be primarily agentive, as language would encourage its use as an assistant.

Other improvements

Language is the big one, but there are a few other improvements I could imagine.

Cloaklet

If this were for a soldier or a police officer or anyone else who could expect combat as part of their job, the Cloak might make them look cartoonish, like a cosplayer rather than a person with authority. But, hey, if it let them fly, and can supernatural protection, maybe that impression wouldn’t last long. Maybe they could take advantage of being underestimated. But I’d still want to look at how little fabric we could get away with and still maintain its benefits.

Remote control / telecommunication

Another constraint seems to be that it has some proximity limits. To imprint on Tony, it makes some social sense that it would want to be within “handshake distance.” But when Strange is astrally overseeing the emergency operation of his own body (a later post), the Cloak sits back at the Sanctum and doesn’t rush to help when things get fighty. Extending its connectivity would help it fulfill its duties. At the very least tape a cell phone to a pocket and let Strange shout orders at it.

Whew. So that’s my take on the Cloak of Levitation. It’s a marvelous piece of speculative “tech” that fits Tony, his future-role as Sorcerer Supreme, and is nicely unique in the MCU.

It’s a great sophisticated example of agentive tech the constraints of which make complete narrative sense. I hope we get to learn much more of this marvelous, agentive familiar in the sequel.

R. S. Revenge Comms

13 Dec 2016 by Christopher Noessel

Note: In honor of the season, Rogue One opening this week, and the reviews of Battlestar Galactica: The Mini-Series behind us, I’m reopening the Star Wars Holiday Special reviews, starting with the show-within-a-show, The Faithful Wookie. Refresh yourself of the plot if it’s been a while.

On board the R.S. Revenge, the purple-skinned communications officer announces he’s picked up something. (Genders are a goofy thing to ascribe to alien physiology, but the voice actor speaks in a masculine register, so I’m going with it.)

He attends a monitor, below which are several dials and controls in a panel. On the right of the monitor screen there are five physical controls.

A stay-state toggle switch
A stay-state rocker switch
Three dials

The lower two dials have rings under them on the panel that accentuate their color.

Map View

The screen is a dark purple overhead map of the impossibly dense asteroid field in which the Revenge sits. A light purple grid divides the space into 48 squares. This screen has text all over it, but written in a constructed orthography unmentioned in the Wookieepedia. In the upper center and upper right are unchanging labels. Some triangular label sits in the lower-left. In the lower right corner, text appears and disappears too fast for (human) reading. The middle right side of the screen is labeled in large characters, but they also change too rapidly to make much sense of it.

Luke, looking over the shoulder of the comms officer at the same monitor, exclaims, “It’s the Millennium Falcon!”

faithful-wookiee-12 — Seriously, Luke, how can you tell this?

Watching the glowing dot and crosshairs blink and change position several times, the comms officer says, “They’re coming out of light speed. I can’t make contact.” An off-screen voice tells him to “Try a lower channel.” Something causes the channel to change (the comms officer’s hands do not touch anything that we can see), and then the monitor shows a video feed from the Falcon.

Video Feed

The video feed has an overlay to the upper left hand side, consisting of lines of text which appear from top to bottom in a palimpsest formation, even though the copy is left-aligned. At the top is a label with changing characters, looking something like a time stamp.

Analysis of the Map View

Since we can’t read the video overlay in the video feed, and it doesn’t interfere with the image, there’s not much to say about it. Instead I’ll focus on the map view.

Hand-drawn Inconsistency

In the side angle shot, which we see first, we see the dial colors go from top to bottom, as beige, red, yellow. In the facing shot of this interface, which immediately follows the side shot, the dials go beige, yellow, red. The red and yellow are transposed. Itʼs of course possible that the dials have a variable hue, and changed at exactly the same time the camera switches. But then we have to explain where his hand went, and why we don’t see any of the other elements changing color, and so on…

A purple character with a space suit interacts with a technological device, while a group of characters, including one dressed in yellow and one in a white gown, observe the scene in a futuristic setting.

A futuristic monitor displaying a purple grid with various cosmic elements and symbols, featuring graphical data and navigation controls on the right side.

This illustrates one of the problems with reviewing hand-drawn animation (and why scifiinterfaces generally frowns upon it.) It takes a hand-drawing animator extra work to keep things consistent from screen to screen. She must have a reference when drawing the interface from any new angle, and this extra work is on top of all the other things she has to manage like color and timing. Fewer people will notice transposed dial colors than, say, the comms officer turning orange instead of purple, so the interface is low on that priority stack.

Contrast that with live-action and computer-animated interfaces. In these modes of working, it takes extra work to change interfaces from shot to shot, so you run into consistency problems much less frequently.

I’ve written about this before in the abstract, but it’s nice to have a simple and easily shown example in the blog to point to.

2Dness

Another problem with the interface is that it is 2-D, but space is 3-D.

When picking a projection to display, we have to keep in mind that it is more immediate to understand an impending collision when presented as 2-D information: Constant bearing, decreasing range = Trouble. So, perhaps the view has automatically aligned itself to be perpendicular to the Falcon’s approach, which makes it easier to monitor the decreasing distance.

If so, he would need to see that automatically-aligned status reflected somewhere in the interface, and have access to controls that let him change the view and snap back to this Most Useful View. Admittedly, this is a lot of apologetics to apply, when really, it’s most likely the old trope 2-D Space.

Attention and memory

There are some nicely designed attention cues. The crosshairs, glowing dot, and motion graphics makes it so that—even though we can’t read the language—we can tell what’s of interest on the screen. One dot moving towards another, stationary dot. We’re set up for the Falcon’s buzzing the base.

That’s probably the best thing that can be said for it.

The text is terrible, changing too fast for a human reader. (Yes, yes, put down that emerging comment. Purple-face isnʼt human, but we must evaluate interfaces considering what is useful to us, and right now that means us humans.) The text changes so much faster than the blinking, in fact, that it’s pulling attention away from it. Narratively, the rapid-fire text helps convey a sense of urgency, but it greatly costs readability. It’s not a good model for real world design.

The blinking crosshair might most accurately reflect the actual position of the detected object within the radar sweep. But it could help the officer more. As with medical signals, data points are not as interesting as information trends. As it is, it relies on his memory to piece together the information, which means he has to constantly monitor the screen to make sense. If instead the view featured an evaporating trail of data points, not only could he look away without missing too much information, but he would also notice that the speed and direction are slightly erratic, which would prove quite interesting to anyone trying to ascertain the status of the ship. One glance shows things are not as they should be. The Falcon is clearly careening.

tracking_assistance — Actual points from the animation.

Mysterious Control

When we first see the comms officer, he has his unmoving hand on one of the dials. But when we see the map switch to the video feed, none of the controls we can see are touched. This raises a possibility and a question.

The possibility is that there is control by some other mechanism. My best guess is that it is voice control, since the Rebel General says “try a lower channel” just before it switches. Maybe he was not speaking to the comms officer, but to the machine itself. And given C3PO, they clearly have the technology to recognize and act on natural language, though it’s usually associated with a full general artificial intelligence. A Rebel Siri (33 years before it came out in Apple’s iOS) makes sense from an apologetics sense.

If so, there are some aspects of the UI missing to signal to an operator that the machine is listening, and hearing, and understanding what is being said, as well as whether the speaker is authorized to control. After all, the comms officer is wearing the headset, but it was the red-bearded general who issued the command. I imagine it’s not OK for anyone on the bridge to just shout out controls.

faithful-wookiee-13 — Just General Burnside, here.

The question then, is if the channel is controlled by voice, what are the physical controls for? They’re lacking labels of any kind. Perhaps they’re there as a backup, should voice control fail. Perhaps they are vestigial, left over from before voice control was installed. Maybe only the general has a voice override and the comms officer must use the physical controls. Any of these would be fine backworlding explanations, but my favorite idea is that the dials are for controlling nuanced variables in very fluid ways with instant feedback.

It’s easier to twiddle a dial to change the frequency of a radio to find a low-power signal than to keep saying “back…forward…no, back just a bit.” That would help explain what the comms officer was doing with his hands on the dials when he got something but not when the general voice-controls the channel.

In general

The interface shows some sophistication in styling and visual hierarchy, and if we give it lots of benefit of the doubt, might even be handling some presentation variables for the user in sophisticated ways. But the distractions of the rapid-fire text, the lack of trend lines, the lack of labels for the physical controls, and the missing affordances for projection control and voice control feedback make it a poor model for any real world design.

Iron Man HUD: A Breakdown

1 Jul 2015 by Christopher Noessel

So this is going to take a few posts. You see, the next interface that appears in The Avengers is a video conference between Tony Stark in his Iron Man supersuit and his partner in romance and business, Pepper Potts, about switching Stark Tower from the electrical grid to their independent power source. Here’s what a still from the scene looks like.

So on the surface of this scene, it’s a communications interface.

But that chat exists inside of an interface with a conceptual and interaction framework that has been laid down since the original Iron Man movie in 2008, and built upon with each sequel, one in 2010 and one in 2013. (With rumors aplenty for a fourth one…sometime.)

So to review the video chat, I first have to talk about the whole interface, and that has about 6 hours of prologue occurring across 4 years of cinema informing it. So let’s start, as I do with almost every interface, simply by describing it and its components.

Exosuit

The Iron Man is the name of the series of superpowered exosuits designed by Tony Stark. They range from the Mark I, a comparatively crude suit of armor to escape imprisonment by terrorists, through the Mark XLVI, the armor seen in The Avengers: Age of Ultron. The suit acts as defense against nearly every type of weapon known. It has repulsor beams built into the palms and in later models the arc reactor mounted in the chest that can be used to deliver concussive force. It allows the wearer to fly. Offensive weaponry varies between models, but has included a high powered laser system, and auto-targeting minigun pod and missiles. The suit can act semi-autonomously or via remote control. One of the models in The Avengers has parts that are seen to self-propel to Tony, targeting a beacon bracelet he wears, and self-assemble around him very quickly.

Immersive display

Though Tony’s head is completely covered, he has a virtual reality display within his helmet. It is a full-field-of-vision, very high-resolution, full-color display that provides stereoscopic imaging. It allows Tony to see the world around him as if he were not wearing the helmet, augment the view with goal-, person-, location-, and object-sensitive awareness.

The display varies a great deal, changing to the needs of the situation. But five icons persistently in the lower part of the display seem to be: suit status, targeting and optics, radar, artificial horizon, and map.

An interpretive view of Tony’s experience, from Iron Man (2008). — An interpretive view of Tony’s experience, from *Iron Man* (2008).

An first-person view from within the HUD, Iron Man (2008). — An first-person view from within the HUD, *Iron Man* (2008).

There is much to critique about the readability of the complex layering and translucency, the limits of human perception, and the necessarily- (and strictly-) interpretive nature of what we as audience see, but let me save those three points for a later post. For now it’s enough to log the features as aspects of the system.

Head NUI

Though Tony could use his hands to interact with an interface projected into the augmented reality view around him, his hands are often occupied in controlling flight or in combat. For this reason the means of input are head gesture, eye gesture, and voice input. A bit more on each follows.

Elements within the HUD such as reticles around his eyes follow and track his head gestures. Other elements stay locked in place. The HUD can track his gaze perfectly, allowing him to designate targets for his weapons with a fixation. Using this perfect eye tracking, Tony can also speak about something he is looking at, either in the real world or in the interface, and the system understands exactly what he’s talking about.

In fact, Tony is able to speak fully natural language commands, and indeed, carry out full-Turing conversations with the suit because of the presence of…

Strong artificial intelligence: JARVIS

An on-board artificial intelligence known as JARVIS handles any information task Tony asks of it, and monitors the surroundings and anticipates informational needs. There is strong evidence that most of the functions of the suit are handled by JARVIS behind the scenes. The crucialness of the artificial intelligence to the function of the suit cannot be overstated. It’s difficult to imagine how most of the suit could function as it does without an artificial intelligence behind the scenes facilitating results and even guiding Tony. With this in mind it is instructive to reframe the AI as the thing being named the Iron Man, with Tony Stark being an onboard manager, or, more charitably, a command-and-control center. Who quips.

Next up in the Iron HUD series: Lets review the functions of the suit.

MedPod

28 Dec 2012 by Christopher Noessel

Early in the film, when Shaw sees the MedPod for the first time, she comments to Vickers that, “They only made a dozen of these.” As she caresses its interface in awe, a panel extends as the pod instructs her to “Please verbally state the nature of your injury.”

The MedPod is a device for automated, generalized surgical procedures, operable by the patient him- (or her-, kinda, see below) self.

When in the film Shaw realizes that she’s carrying an alien organism in her womb, she breaks free from crewmembers who want to contain her, and makes a staggering beeline for the MedPod.

Once there, she reaches for the extended touchscreen and presses the red EMERGENCY button. Audio output from the pod confirms her selection, “Emergency procedure initiated. Please verbally state the nature of your injury.” Shaw shouts, “I need cesarean!” The machine informs her verbally that, “Error. This MedPod is calibrated for male patients only. It does not offer the procedure you have requested. Please seek medical assistance else–”

I’ll pause the action here to address this. What sensors and actuators are this gender-specific? Why can’t it offer gender-neutral alternatives? Sure, some procedures might need anatomical knowledge of particularly gendered organs (say…emergency circumcision?), but given…

the massive amounts of biological similarity between the sexes
the needs for any medical device to deal with a high degree of biological variability in its subjects anyway
most procedures are gender neutral

…this is a ridiculous interface plot device. If Dr. Shaw can issue a few simple system commands that work around this limitation (as she does in this very scene), then the machine could have just done without the stupid error message. (Yes, we get that it’s a mystery why Vickers would have her MedPod calibrated to a man, but really, that’s a throwaway clue.) Gender-specific procedures can’t take up so much room in memory that it was simpler to cut the potential lives it could save in half. You know, rather than outfit it with another hard drive.

Aside from the pointless “tension-building” wrong-gender plot point, there are still interface issues with this step. Why does she need to press the emergency button in the first place? The pod has a voice interface. Why can’t she just shout “Emergency!” or even better, “Help me!” Isn’t that more suited to an emergency situation? Why is a menu of procedures the default main screen? Shouldn’t it be a prompt to speak, and have the menu there for mute people or if silence is called for? And shouldn’t it provide a type-ahead control rather than a multi-facet selection list? OK, back to the action.

Desperate, Shaw presses a button that grants her manual control. She states “Surgery abdominal, penetrating injuries. Foreign body. Initiate.” The screen confirms these selections amongst options on screen. (They read “DIAGNOS, THERAP, SURGICAL, MED REC, SYS/MECH, and EMERGENCY”)

The pod then swings open saying, “Surgical procedure begins,” and tilting itself for easy access. Shaw injects herself with anesthetic and steps into the pod, which seals around her and returns to a horizontal position.

Why does Shaw need to speak in this stilted speech? In a panicked or medical emergency situation, proper computer syntax should be the last thing on a user’s mind. Let the patient shout the information however they need to, like “I’ve got an alien in my abdomen! I need it to be surgically removed now!” We know from the Sonic chapter that the use of natural language triggers an anthropomorphic sense in the user, which imposes some other design constraints to convey the system’s limitations, but in this case, the emergency trumps the needs of affordance subtleties.

Once inside the pod, a transparent display on the inside states that, “EMERGENCY PROC INITIATED.” Shaw makes some touch selections, which runs a diagnostic scan along the length of her body. The terrifying results display for her to see, with the alien body differentiated in magenta to contrast her own tissue, displayed in cyan.

Shaw shouts, “Get it out!!” It says, “Initiating anesthetics” before spraying her abdomen with a bile-yellow local anesthetic. It then says, “Commence surgical procedure.” (A note for the grammar nerds here: Wouldn’t you expect a machine to maintain a single part of speech for consistency? The first, “Initiating…” is a gerund, while the second, “Commence,” is an imperative.) Then, using lasers, the MedPod cuts through tissue until it reaches the foreign body. Given that the lasers can cut organic matter, and that the xenomorph has acid for blood, you have to hand it to the precision of this device. One slip could have burned a hole right through her spine. Fortunately it has a feather-light touch. Reaching in with a speculum-like device, it removes the squid-like alien in its amniotic sac.

OK. Here I have to return to the whole “ManPod” thing. Wouldn’t a scan have shown that this was, in fact, a woman? Why wouldn’t it stop the procedure if it really couldn’t handle working on the fairer sex? Should it have paused to have her sign away insurance rights? Could it really mistake her womb for a stomach? Wouldn’t it, believing her to be a man, presume the whole womb to be a foreign body and try to perform a hysterectomy rather than a delicate caesarian? ManPod, indeed.

After removing the alien, it waits around 10 seconds, showing it to her and letting her yank its umbilical cord, before she presses a few controls. The MedPod seals her up again with staples and opens the cover to let her sit up.

She gets off the table, rushes to the side of the MedPod, and places all five fingertips of her right hand on it, quickly twisting her hand clockwise. The interface changes to a red warning screen labeled “DECONTAMINATE.” She taps this to confirm and shouts, “Come on!” (Her vocal instruction does not feel like a formal part of the procedure and the machine does not respond differently.) To decontaminate, the pod seals up and a white mist fills the space.

OK. Since this is a MedPod, and it has something called a decontamination procedure, shouldn’t it actually test to see whether the decontamination worked? The user here has enacted emergency decontamination procedures, so it’s safe to say that this is a plague-level contagion. That’s doesn’t say to me: Spray it with a can of Raid and hope for the best. It says, “Kill it with fire.” We just saw, 10 seconds ago, that the MedPod can do a detailed, alien-detecting scan of its contents, so why on LV-223 would it not check to see if the kill-it-now-for-God’s-sake procedure had actually worked, and warn everyone within earshot that it hadn’t? Because someone needs to take additional measures to protect the ship, and take them, stat. But no, MedPod tucks the contamination under a white misty blanket, smiles, waves, and says, “OK, that’s taken care of! Thank you! Good day! Move along!”

For all of the goofiness that is this device, I’ll commend it for two things. The first is for pushing the notion forward of automated medicine. Yes, in this day and age, it’s kind of terrifying to imagine devices handling something as vital as life-saving surgery, but people in the future will likely find it terrifying that today we’d rather trust an error prone, bull-in-a-china-shop human to the task. And, after all, the characters have entrusted their lives to an android while they were in hypersleep for two years, so clearly that’s a thing they do.

Second, the gestural control to access the decontamination is well considered. It is a large gesture, requiring no great finesse on the part of the operator to find and press a sequence of keys, and one that is easy to execute quickly and in a panic. I’m absolutely not sure what percentage of procedures need the back-up safety of a kill-everything-inside mode, but presuming one is ever needed, this is a fine gesture to initiate that procedure. In fact, it could have been used in other interfaces around the ship, as we’ll see later with the escape pod interface.

I have the sense that in the original script, Shaw had to do what only a few very bad-ass people have been willing to do: perform life-saving surgery on themselves in the direst circumstances. Yes, it’s a bit of a stretch since she’s primarily an anthropologist and astronomer in the story, but give a girl a scalpel, hardcore anesthetics, and an alien embryo, and I’m sure she’ll figure out what to do. But pushing this bad-assery off to an automated device, loaded with constraints, ruins the moment and changes the scene from potentially awesome to just awful.

Given the inexplicable man-only settings, requiring a desperate patient to recall FORTRAN-esque syntax for spoken instructions, and the failure to provide any feedback about the destruction of an extinction-level pathogen, we must admit that the MedPod belongs squarely in the realm of goofy narrative technology and nowhere near the real world as a model of good interaction design.

Sci-fi interfaces

Stop watching sci-fi. Start using it.

Tag Archives: natural language