t’s Halloween, as if the news of the past week were not scary enough. Pipe bombs to Democratic leaders. The largest massacre of Jewish people in on American soil in history. The murder of two black senior citizens by a white supremacist in Kentucky. Now let’s add to it with this nightmare scene from Idiocracy. Full disclosure: We’re covering technology as old as civilization here, so there won’t be any screen interfaces.
Joe is wheeled into the courtroom in a cage. There is a large gallery there, all of whom are booing him. One throws his milkshake at the accused. Others throw trash. The narrator says, “Joe was arrested for not paying his hospital bill and not having his IPP tattoo. He would soon discover that in the future, Justice was not only blind, but had become rather retarded as well.”
Joe is let out of his cage. The judge, identified by his name plate as The Honorable Hector “The Hangman,” stands at his bench in a spotlight in front of a wall of logos, grinning in anticipation at a new victim. He slams a massive gavel and shouts at the booing crowd, “Listen up! Now. I’m fixin’ to commensurate this trial here. [All of this is sic.] We gon’ see if we can’t come up with a verdict up in here. Now. Since y’all say y’ain’t got no money, we have proprietarily obtained you one of them court-appointed lawyers. So, put your hands together to give it up for Frito Pendejo!”Continue reading →
If you’re reading these chronologically, let me note here that I had to skip Bea Arthur’s marvelous turn as Ackmena, as she tends the bar and rebuffs the amorous petitions of the lovelorn, hole-in-the-head Krelman, before singing her frustrated patrons out of the bar when a curfew is announced. To find the next interface of note, we have to forward to when…
Han and Chewie arrive, only to find a Stormtrooper menacing Lumpy. Han knocks the blaster out of his hand, and when the Stormtrooper dives to retrieve it, he falls through the bannister of the tree house and to his death.
Why aren’t these in any way affiiiiixxxxxxeeeeeeddddddd?
Han enters the home and wishes everyone a Happy Life Day. Then he bugs out.
But I still have to return for the insane closing number. Hold me.
In the priorthreeposts, I’ve discussed the goods-and-bads of the Eye of Agamotto in the Tibet mode. (I thought I could squeeze the Hong Kong and the Dark Dimension modes into one post, but turns out this one was just too long. keep reading. You’ll see.) In this post we examine a mode that looks like the Tibet mode, but is actually quite different.
Hong Kong mode
Near the film’s climax, Strange uses the Eye to reverse Kaecilius’ destruction of the Hong Kong Sanctum Sanctorum (and much of the surrounding cityscape). In this scene, Kaecilius leaps at Strange, and Strange “freezes” Kaecilius in midair with the saucer. It’s done more quickly, but similarly to how he “freezes” the apple into a controlled-time mode in Tibet.
But then we see something different, and it complicates everything. As Strange twists the saucer counterclockwise, the cityscape around him—not just Kaecilius—begins to reverse slowly. (And unlike in Tibet, the saucer keeps spinning clockwise underneath his hand.) Then the rate of reversal accelerates, and even continues in its reversal after Strange drops his gesture and engages in a fight with Kaecilius, who somehow escapes the reversing time stream to join Strange and Mordo in the “observer” time stream.
So in this mode, the saucer is working much more like a shuttle wheel with no snap-back feature.
A shuttle wheel, as you’ll recall from the first post, doesn’t specify an absolute value along a range like a jog dial does. A shuttle wheel indicates a direction and rate of change. A little to the left is slow reverse. Far to the left is fast reverse. Nearly all of the shuttle wheels we use in the real world have snap-back features, because if you were just going to leave it reversing and pay attention to something else, you might as well use another control to get to the absolute beginning, like a jog dial. But, since Strange is scrubbing an endless “video stream,” (that is, time), and he can pull people and things out of the manipulated-stream and into the observer-stream and do stuff, not having a snap-back makes sense.
For the Tibet mode I argued for a chapter ring to provide some context and information about the range of values he’s scrubbing. So for shuttling along the past in the Hong Kong mode, I don’t think a chapter ring or content overview makes sense, but it would help to know the following.
The rate of change
Direction of change
Timedate difference from when he started
In the scene that information is kind of obvious from the environment, so I can see the argument for not having it. But if he was in some largely-unchanging environment, like a panic room or an underground cave or a Sanctum Sanctorum, knowing that information would save him from letting the shuttle go too far and finding himself in the Ordovician. A “home” button might also help to quickly recover from mistakes. Adding these signals would also help distinguish the two modes. They work differently, so they should look different. As it stands, they look identical.
He still (probably) needs future branches
Can Strange scrub the future this way? We don’t see it in the movie. But if so, we have many of the same questions as the Tibet mode future scrubber: Which timeline are we viewing & how probable is it? What other probabilities exist and how does he compare them? This argues for the addition of the future branches from that design.
Selecting the mode
So how does Strange specify the jog dial or shuttle wheel mode?
One cop-out answer is a mental command from Strange. It’s a cop-out because if the Eye responds to mental commands, this whole design exercise is moot, and we’re here to critique, practice, and learn. Not only that, but physical interfaces are more cinemagenic, so better to make a concrete interaction for the film.
You might think we could modify the opening finger-tut (see the animated gif, below). But it turns out we need that for another reason: specifying the center and radius-of-effect.
Center and radius-of-effect
In Tibet, the Eye appears to affect just an apple and a tome. But since we see it affecting a whole area in Hong Kong, let’s presume the Eye affects time in a sphere. For the apple and tome, it was affecting a small sphere that included the table, too, it’s just that table didn’t change in the spans of time we see. So if it works in spheres, how is the center and the radius of the sphere set?
Let’s say the Eye does some simple gaze monitoring to find the salient object at his locus of attention. Then it can center the effect on the thing and automatically set the radius of effect to the thing’s size across likely-to-be scrubbed extents. In Tibet, it’s easy. Apple? Check. Tome? Check. In Hong Kong, he’s focusing on the Sanctum, and its image recognition is smart enough to understand the concept of “this building.”
But the Hong Kong radius stretches out beyond his line of sight, affecting something with a very vague visual and even conceptual definition, that is, “the wrecked neighborhood.” So auto-setting these variables wouldn’t work without reconceiving the Eye as a general artificial intelligence. That would have some massive repercussions throughout the diegesis, so let’s avoid that.
If it’s a manual control, how does he do it? Watch the animated gif above carefully and see he’s got two steps to the “turn Eye on” tut: opening the eye by making an eye shape, and after the aperture opens, spreading his hands apart, or kind of expanding the Eye. In Tibet that spreading motion is slow and close. In Hong it’s faster and farther. That’s enough evidence to say the spread*speed determines the radius. We run into the scales problem of apple-versus-neighborhood that we had in determining the time extents, but make it logarithmic and add some visual feedback and he should be able to pick arbitrary sizes with precision.
So…back to mode selection
So if we’re committing the “turn on” gesture to specifying the center-and-radius, the only other gesture left is the saucer creation. For a quick reminder, here’s how it works in Tibet.
Since the circle works pretty well for a jog dial, let’s leave this for Tibet mode. A contrasting but related gesture would be to have Strange hold his right hand flat, in a sagittal plane, with the palm facing to his left. (See an illustration, below.) Then he can tilt his hand inside the saucer to reverse or fast forward time, and withdraw his hand from the saucer graphic to leave time moving at the adjusted rate. Let the speed of the saucer indicate speed of change. To map to a clock, tilting to the left would reverse time, and tilting to the right would advance it.
The yank out
There’s one more function we see twice in the Hong Kong scene. Strange is able to pull Mordo and Wong from the reversing time stream by thrusting the saucer toward them. This is a goofy choice of a gesture that makes no semantic sense. It would make much more sense for Strange to keep his saucer hand extended, and use his left hand to pull them from the reversing stream.
So one of the nice things about this movie interface, is that while it doesn’t hold up under the close scrutiny of this blog, the interface to the Eye of Agamotto works while watching the film. Audience sees the apple happen, and gets that gestures + glowing green circle = adjusting time. For that, it works.
That said, we can see improvements that would not affect the script, would not require much more of the actors, and not add too much to post. It could be more consistent and believable.
But we’re not done yet. There’s one other function shown by the Eye of Agamotto when Strange takes it into the Dark Dimension, which is the final mode of the Eye, up next.
A major problem with the use of the Eye is that it treats the past and the future similarly. But they’re not the same. The past is a long chain of arguably-knowable causes and effects. So, sure, we can imagine that as a movie to be scrubbed.
But the future? Not so much. Which brings us, briefly, to this dude.
If we knew everything, Pierre-Simon Laplace argued in 1814, down to the state of every molecule, and we had a processor capable, we would be able to predict with perfect precision the events of the future. (You might think he’s talking about a computer or an AI, but in 1814 they used demons for their thought experiments.) In the two centuries since, there have been several major repudiations of Laplace’s demon. So let’s stick to the near-term, where there’s not one known future waiting to happen, but a set of probabilities. That means we have to rethink what the Eye shows when it lets Strange scrub the future.
Note that in the film, the “future” of the apple shown to Strange was just a likelihood, not a fact. The Eye shows it being eaten. In the actual events of the film, after the apple is set aside:
Strange repairs the tome
Mordo and Wong interrupt Strange
They take him into the next room for some exposition
The Hong Kong sanctum portal swings open
Kaecilius murders a redshirt
Kaecilius explodes Strange into the New York sanctum
Then for the next 50 minutes, The Masters of Mysticism are scrambling to save the world. I doubt any of them have time to while away in a library, there to discover an abandoned apple with a bite taken out of it, and decide—staphylococcus aureus be damned—a snack’s a snack. No, it’s safe to say the apple does not get eaten.
So the Eye gets the apple wrong, but it showed Strange that future as if it were a certainty. That’s a problem. Sure, when asked about the future, it ought to show something, but better would be to…
Indicate somewhere that what is being displayed is one of a set of possibilities
Provide options to understand the probability distribution among the set
Explore the alternates
Be notified when new data shifts the probability distribution or inserts new important possibilities
So how to display probabilities? There are lots of ways, but I am most fond of probability tree diagrams. In nerd parlance, this is a unidirectional graph where the nodes are states and the lines are labeled for probabilities. In regular language they look like sideways two-dimensional trees. See an example below from mathisfun.com. These diagrams seem to me a quick way to understand branching possibilities. (I couldn’t find any studies giving me more to work on than “seem to me”.)
In addition to being easy to understand, they afford visual manipulation. You can work branching lines around an existing design.
Now if we were actually working out a future-probabilities gestural scrubber attached to the Eye of Agamotto saucer, we’d have a whole host of things to get into next, like designing…
A compact but informative display that signals the relative probabilities of each timeline
The mechanism for opening that display so probabilities can be seen rather than read
Labels so Strange wouldn’t have to hunt through all of them for the thing of interest (or some means of search)
A selection process for picking the new timeline
A comparison mode
A means of collapsing the display to return to scrub mode
A you-are-here signal in the display to indicate the current timeline
Which is a big set of design tasks for a hobbyist website. Fortunately for us, Strange only deals with a simple, probable (but wrong) scenario of the apple’s future as an illustration for the audience of what the Eye can do; and he only deals with the past of the tome. So while we could get into all of the above, it’s most expedient just to resolve the first one for the scene and tidy up the interface as it helps illustrate a well-thought-out and usable world.
Below I’ve drafted up an extension of my earlier conceptual diagram. I’ve added a tree to the future part of the chapter ring, using some dots to indicate the comparative likelihood of each branch. This could be made more compact, and might be good to put on a second z-axis layer to distinguish it from the saucer, but again: conceptual diagram.
If this were implemented in the film, we would want to make sure that the probability tree begins to flicker right before Wong and Mordo shut him down, as a nod to the events happening off screen with Kaecilius that are changing those futures. This would give a clue that the Eye is smartly keeping track of real-world events and adjusting its predictions appropriately.
These changes would make the Eye more usable for Strange and smart as a model for us.
Twist ending: This is a real problem we will have to solve
I skipped those design tasks for this comp, but we may not be able to avoid those problems forever. As it turns out, this is not (just) an idle, sci-fi problem. One of the promises of assistive AI is that it will be giving its humans advice, based on predictive algorithms, which will be a set of probabilistic scenarios. There may be an overwhelmingly likely next scenario, but there may also be several alternatives that users will need to explore and understand before deciding the best strategy. So, yeah, an exercise for the reader.
Wrapping up the Tibet Mode
So three posts is not the longest analysis I’ve done, bit it was a lot. In recap: Gestural time scrubbing seems like a natural interaction mapped well to analog clocks. The Eye’s saucer display is cool, but insufficient. We can help Strange much more by adding an events-based chapter ring detailing the facts of the past and the probabilities of the future.
Alas. We’re not done yet. As you’ll recall from the intro post, there are two other modes: The Hong Kong and Dark Dimension modes. Let’s next talk the Hong Kong mode, which is like the Tibet mode, but different.
Without a display, the Eye asks Strange to do all the work of exploring the range of values available through it to discover what is of interest. (I am constantly surprised at how many interfaces in the real world repeat this mistake.) We can help by doing a bit of “pre-processing” of the information and provide Strange a key to what he will find, and where, and ways to recover exactly where interesting things happen.
To do this, we’ll add a ring outside the saucer that will stay fixed relative to the saucer’s rotation and contain this display. Since we need to call this ring something, and we’re in the domain of time, let’s crib some vocabulary from clocks. The fixed ring of a clock that contains the numbers and minute graduations is called a chapter ring. So we’ll use that for our ring, too.
What chapter ring content would most help Strange?
Good: A time-focused chapter ring
Both the controlled-extents and the auto-extents shown in the prior post presume a smooth display of time. But the tome and the speculative meteorite simply don’t change much over the course of their existence. I mean, of course they do, with the book being pulled on and off shelves and pages flipped, and the meteorite arcing around the sun in the cold vacuum of space for countless millennia, but the Eye only displays the material changes to an object, not position. So as far as the Eye is concerned, the meteoroid formed, then it stays the same for most of its existence, then it has a lot of activity as it hits Earth’s atmosphere and slams into the planet.
A continuous display of the book shows little of interest for most of its existence, with a few key moments of change interspersed. To illustrate this, lets make up some change events for the tome.
Now let’s place those along an imaginary timeline. Given the Doctor Strange storyline, Page Torn would more likely be right next to Now, but making this change helps us explore a common boredom problem, see below. OK. Placing those events along a timeline…
And then, wrapping that timeline around the saucer. Much more art direction would have to happen to make this look thematically like the rest of the MCU magic geometries, but following is a conceptual diagram of how it might look.
On the outside of the saucer is the chapter ring with the salient moments of change called out with icons (and labels). At a glance Strange would know where the fruitful moments of change occur. He can see he only has to turn his hand about 5° to the left to get to the spot where the page was ripped out.
Already easier on him, right? Some things to note.
The chapter ring must stay fixed relative to the saucer to work as a reference. Imagine how useless a clock would be if its chapter ring spun in concert with any of its hands. The center can still move with his palm as the saucer does.
The graduations to the left and right of “now” are of a different density, helping Strange to understand that past and future are mapped differently to accommodate the limits of his wrist and the differing time frames described.
When several events occur close together in time, they could be stacked.
Having the graduations evenly spaced across the range helps answer roughly when each change happened relative to the whole.
The tome in front of him should automatically flip to spreads where scrubbed changes occur, so Strange doesn’t have to hunt for them. Without this feature, if Strange was trying to figure out what changed, he would have to flip through the whole book with each degree of twist to see if anything unknown had changed.
Better: A changes-focused chapter ring
If, as in this scene, the primary task of using the Eye is to look for changes, a smooth display of time on the chapter ring is less optimal than a smooth display of change. (Strange doesn’t really care when the pages were torn. He just wants to see the state of the tome before that moment.) Distribute the changes evenly around the chapter ring, and you get something like the following.
This display optimizes for easy access to the major states of the book. The now point is problematic since the even distribution puts it at the three o’clock point rather than the noon, but what we buy in exchange is that the exact same precision is required to access any of the changes and compare them. There’s no extra precision needed to scrub between the book made and the first stuff added moments. The act of comparison is made simpler. Additionally, the logarithmic time graduations help him scrub detail near known changes and quickly bypass the great stretches of time when nothing happens. By orienting our display around the changes, the interesting bits are made more easy to explore, and the boring bits are more easy to bypass.
In my comp, more white areas equal more time. Unfortunately, this visual design kind of draws attention to the empty stretches of time rather than the moments of change, so would need more attention; see the note above about needing a visual designer involved.
So…the smooth time and the distributed events display each has its advantages over the other, but for the Tibet scene, in which he’s looking to restore the lost pages of the tome, the events-focused chapter ring gets Strange to the interesting parts more confidently.
One computer in the control room is dedicated to showing the status of the Jeeps out on tour, and where they currently are on the island.
Next to the vehicle outline, we see the words “Vehicle Type: Ford Explorer” (thank you, product placement) along with “EXP” 4–7. EXP 4 & 5 look unselected, but have green dots next to them, while EXP 6 & 7 look selected with red dots next to them. No characters interact with this screen. Mr. Arnold does tap on it with a pen (to make a point though, not to interact with it).
On the right hand side of the screen also see a top-down view of the car with the electric track shown underneath, and little red arrows pointing forward. Below the graphic are the words “13 mph”. The most visible and obvious indicator on the screen is the headlights. A large “Headlights On” indicator is at the top of the screen, with highlighted cones coming out of the Jeep where the headlights are on the car. Continue reading →
As Jack searches early in the film for Drone 172, he parks his bike next to a sinkhole in the desert and cautiously peers into it. As he does so, he is being observed from afar by a sinister looking Scav through a set of asymmetrical…well, it’s not exactly right to call them binoculars.
They look kind of binocular, but that term technically refers to a machine that displays two slighty-offset images shown independently to each eye such that the user perceives stereopsis, or a single field in 3D. But a quick shot from the Scav’s perspective shows that this is not what is shown at all. Continue reading →
The TETVision display is the only display Vika is shown interacting with directly—using gestures and controls—whereas the other screens on the desktop seem to be informational only. This screen is broken up into three main sections:
The left side panel
The main map area
The right side panel
The left side panel
The communications status is at the top of the left side panel and shows Vika the status of whether the desktop is online or offline with the TET as it orbits the Earth. Directly underneath this is the video communications feed for Sally.
Beneath Sally’s video feed is the map legend section, which serves the dual purposes of providing data transfer to the TET and to the Bubbleship as well as a simple legend for the icons used on the map.
The communications controls, which are at the bottom of the left side panel, allow Vika to toggle the audio communications with Jack and with Sally. Continue reading →
As Jack begins his preflight check in the Bubbleship, Vika touches the center of the glass surface to power up the desktop that keeps her in contact with Sally on the TET and allows her to assist and monitor Jack as he repairs the drones on the ground.
Forgive me, as I am but a humble interaction designer (i.e., neither a professional visual designer nor video editor) but here’s my shot at a redesigned DuoMento, taking into account everything I’d noted in the review.
There’s only one click for Carl to initiate this test.
To decrease the risk of a false positive, this interface draws from a large category of concrete, visual and visceral concepts to be sent telepathically, and displays them visually.
It contrasts Carl’s brainwave frequencies (smooth and controlled) with Johnny’s (spiky and chaotic).
It reads both the brain of the sender and the receiver for some crude images from their visual cortex. (It would be better at this stage to have the actors wear some glowing attachment near a crown to show how this information was being read.)
These changes are the sort that even in passing would help tell a more convincing narrative by being more believable, and even illustrating how not-psychic Johnny really is.