Sci-fi Spacesuits: Moving around

20 Jul 2021 by Christopher Noessel

Whatever it is, it ain’t going to construct, observe, or repair itself. In addition to protection and provision, suits must facilitate the reason the wearer has dared to go out into space in the first place.

One of the most basic tasks of extravehicular activity (EVA) is controlling where the wearer is positioned in space. The survey shows several types of mechanisms for this. First, if your EVA never needs you to leave the surface of the spaceship, you can go with mountaineering gear or sticky feet. (Or sticky hands.) We can think of maneuvering through space as similar to piloting a craft, but the outputs and interfaces have to be made wearable, like wearable control panels. We might also expect to see some tunnel in the sky displays to help with navigation. We’d also want to see some AI safeguard features, to return the spacewalker to safety when things go awry. (Narrator: We don’t.)

Mountaineering gear

In Stowaway (2021) astronauts undertake unplanned EVAs with carabiners and gear akin to mountaineers use. This makes some sense, though even this equipment needs to be modified for use by astronauts’ thick gloves.

Stowaway (2021) Drs Kim and Levinson prepare to scale to the propellant tank.

Sticky feet (and hands)

Though it’s not extravehicular, I have to give a shout out to 2001: A Space Odyssey (1969), where we see a flight attendant manage their position in the microgravity with special shoes that adhere to the floor. It’s a lovely example of a competent Hand Wave. We don’t need to know how it works because it says, right there, “Grip shoes.” Done. Though props to the actress Heather Downham, who had to make up a funny walk to illustrate that it still isn’t like walking on earth.

*2001: A Space Odyssey* (1969)
Pan Am: “Thank god we invented the…you know, whatever shoes.”

With magnetic boots, seen in Destination Moon, the wearer simply walks around and manages the slight awkwardness of having to pull a foot up with extra force, and have it snap back down on its own.

*Destination Moon* (1950): Jim Barnes tests his magnetic boots.

Battlestar Galactica added magnetic handgrips to augment the control provided by magnetized boots. With them, Sergeant Mathias is able to crawl around the outside of an enemy vessel, inspecting it. While crawling, she holds grip bars mounted to circles that contain the magnets. A mechanism for turning the magnet off is not seen, but like these portable electric grabbers, it could be as simple as a thumb button.

*Battlestar Galactica* (2008, Season 4, Episode 5, “The Road Less Traveled”): Sergeant Erin Mathias inspects the damaged cylon raider for tracking devices.

Iron Man also had his Mark 50 suit form stabilizing suction cups before cutting a hole in the hull of the Q-Ship.

In the electromagnetic version of boots, seen in Star Trek: First Contact, the wearer turns the magnets on with a control strapped to their thigh. Once on, the magnetization seems to be sensitive to the wearer’s walk, automatically lessening when the boot is lifted off. This gives the wearer something of a natural gait. The magnetism can be turned off again to be able to make microgravity maneuvers, such as dramatically leaping away from Borg minions.

Star Trek: Discovery also included this technology, but with what appears to be a gestural activation and a cool glowing red dots on the sides and back of the heel. The back of each heel has a stack of red lights that count down to when they turn off, as, I guess, a warning to anyone around them that they’re about to be “air” borne.

*Star Trek: Discovery* (2017–) S01E01, “The Vulcan Hello”
Ensign Burnham takes a step out onto the hull before lifting off toward the mysterious beacon.

Quick “gotcha” aside: neither Destination Moon nor Star Trek: First Contact bothers to explain how characters are meant to be able to kneel while wearing magnetized boots. Yet this very thing happens in both films.

*Destination Moon* (1950): Kneeling on the surface of the spaceship.

*Star Trek: First Contact* (1996): Worf rises from operating the maglock to defend himself.

Controlled Propellant

If your extravehicular task has you leaving the surface of the ship and moving around space, you likely need a controlled propellant. This is seen only a few times in the survey.

In the film Mission to Mars, the manned mobility unit, or MMU, seen in the film is based loosely on NASA’s MMU. A nice thing about the device is that unlike the other controlled propellant interfaces, we can actually see some of the interaction and not just the effect. The interfaces are subtly different in that the Mission to Mars spacewalkers travel forward and backward by angling the handgrips forward and backward rather than with a joystick on an armrest. This seems like a closer mapping, but also seems more prone to error by accidental touching or bumping into something.

*Mission to Mars* (2000): Woody inspects the craft for the leak.

The plus side is an interface that is much more cinegenic, where the audience is more clearly able to see the cause and effect of the spacewalker’s interactions with the device.

*Mission to Mars* (2000): The crew performs a daring EVA.

If you have propellent in a Moh’s 4 or 5 film, you might need to acknowledge that propellant is a limited resource. Over the course of the same (heartbreaking) scene shown above, we see an interface where one spacewalker monitors his fuel, and another where a spacewalker realizes that she has traveled as far as she can with her MMU and still return to safety.

*Mission to Mars* (2000): Woody sees that he’s out of fuel.

*Mission to Mars* (2000): Terry sees that she cannot continue her rescue of Woody.

For those wondering, Michael Burnham’s flight to the mysterious signal in that pilot uses propellant, but is managed and monitored by controllers on Discovery, so it makes sense that we don’t see any maneuvering interfaces for her. We could dive in and review the interfaces the bridge crew uses (and try to map that onto a spacesuit), but we only get snippets of these screens and see no controls.

*Star Trek: Discovery* (2017–) S01E01, “The Vulcan Hello”

Iron Man’s suits employ some Phlebotinum propellant that lasts for ever, can fit inside his tailored suit, and are powerful enough to achieve escape velocity.

All-in-all, though sci-fi seems to understand the need for characters to move around in spacesuits, very little attention is given to the interfaces that enable it. The Mission to Mars MMU is the only one with explicit attention paid to it, and that’s quite derived from NASA models. It’s an opportunity for film makers should the needs of the plot allow, to give this topic some attention.

Sci-fi Spacesuits: Interface Locations

6 Jul 2021 by Christopher Noessel

A major concern of the design of spacesuits is basic usability and ergonomics. Given the heavy material needed in the suit for protection and the fact that the user is wearing a helmet, where does a designer put an interface so that it is usable?

Chest panels

Chest panels are those that require that the wearer only look down to manipulate. These are in easy range of motion for the wearer’s hands. The main problem with this location is that there is a hard trade off between visibility and bulkiness.

Arm panels

Arm panels are those that are—brace yourself—mounted to the forearm. This placement is within easy reach, but does mean that the arm on which the panel sits cannot be otherwise engaged, and it seems like it would be prone to accidental activation. This is a greater technological challenge than a chest panel to keep components small and thin enough to be unobtrusive. It also provides some interface challenges to squeeze information and controls into a very small, horizontal format. The survey shows only three arm panels.

The first is the numerical panel seen in 2001: A Space Odyssey (thanks for the catch, Josh!). It provides discrete and easy input, but no feedback. There are inter-button ridges to kind of prevent accidental activation, but they’re quite subtle and I’m not sure how effective they’d be.

The second is an oversimplified control panel seen in Star Trek: First Contact, where the output is simply the unlabeled lights underneath the buttons indicating system status.

The third is the mission computers seen on the forearms of the astronauts in Mission to Mars. These full color and nonrectangular displays feature rich, graphic mission information in real time, with textual information on the left and graphic information on the right. Input happens via hard buttons located around the periphery.

*Mission to Mars* (2000): Phil sees that Woody is likely to overshoot

Side note: One nifty analog interface is the forearm mirror. This isn’t an invention of sci-fi, as it is actually on real world EVAs. It costs a lot of propellant or energy to turn a body around in space, but spacewalkers occasionally need to see what’s behind them and the interface on the chest. So spacesuits have mirrors on the forearm to enable a quick view with just arm movement. This was showcased twice in the movie Mission to Mars.

*Mission to Mars* (2000): Phil looks for Earth with his “rear view mirror.” It’s that tiny white dot.

HUDs

The easiest place to see something is directly in front of your eyes, i.e. in a heads-up display, or HUD. HUDs are seen frequently in sci-fi, and increasingly in sc-fi spacesuits as well. One is Sunshine. This HUD provides a real-time view of each other individual to whom the wearer is talking while out on an EVA, and a real-time visualization of dangerous solar winds.

These particular spacesuits are optimized for protection very close to the sun, and the visor is limited to a transparent band set near eye level. These spacewalkers couldn’t look down to see the top of a any interfaces on the suit itself, so the HUD makes a great deal of sense here.

Star Trek: Discovery’s pilot episode included a sequence that found Michael Burnham flying 2000 meters away from the U.S.S. Discovery to investigate a mysterious Macguffin. The HUD helped her with wayfinding, navigating, tracking time before lethal radiation exposure (a biological concern, see the prior post), and even doing a scan of things in her surroundings, most notably a Klingon warrior who appears wearing unfamiliar armor. Reference information sits on the periphery of Michael’s vision, but the augmentations occur mapped to her view. (Noting this raises the same issues of binocular parallax seen in the Iron HUD.)

Iron Man’s Mark L armor was able to fly in space, and the Iron HUD came right along with it. Though not designed/built for space, it’s a general AI HUD assisting its spacewalker, so worth including in the sample.

Aside from HUDs, what we see in the survey is similar to what exists in existing real-world extravehicular mobility units (EMUs), i.e. chest panels and arm panels.

Inputs illustrate paradigms

Physical controls range from the provincial switches and dials on the cigarette-girl foldout control panels of Destination Moon to the simple and restrained numerical button panel of 2001, to strangely unlabeled buttons of Star Trek: First Contact’s arm panels (above), and the ham-handed touch screens of Mission to Mars.

*Mission to Mars* (2000): Jim tests the atmosphere and learns it is Earth-like.

As the pictures above reveal, the input panels reflect the familiar technology of the time of the creation of the movie or television show. The 1950s were still rooted in mechanistic paradigms, the late 1960s interfaces were electronic pushbutton, the 2000s had touch screens and miniaturized displays.

Real world interfaces

For comparison and reference, the controls for NASA’s EMU has a control panel on the front, called the Display and Control Module, where most of the controls for the EMU sit.

The image shows that inputs are very different than what we see as inputs in film and television. The controls are large for easy manipulation even with thick gloves, distinct in type and location for confident identification, analog to allow for a minimum of failure points and in-field debugging and maintenance, and well-protected from accidental actuation with guards and deep recesses. The digital display faces up for the convenience of the spacewalker. The interface text is printed backwards so it can be read with the wrist mirror.

The outputs are fairly minimal. They consist of the pressure suit gauge, audio warnings, and the 12-character alphanumeric LCD panel at the top of the DCM. No HUD.

The gauge is mechanical and standard for its type. The audio warnings are a simple warbling tone when something’s awry. The LCD panel provides information about 16 different values that the spacewalker might need, including estimated time of oxygen remaining, actual volume of oxygen remaining, pressure (redundant to the gauge), battery voltage or amperage, and water temperature. To cycle up and down the list, she presses the Mode Selector Switch forward and backward. She can adjust the contrast using the Display Intensity Control potentiometer on the front of the DCM.

The DCMs referenced in the post are from older NASA documents. In more recent images on NASA’s social media, it looks like there have been significant redesigns to the DCM, but so far I haven’t seen details about the new suit’s controls. (Or about how that tiny thing can house all the displays and controls it needs to.)

Report Card: Doctor Strange

2 Oct 2017 by Christopher Noessel and scoutlandish

Read all Doctor Strange reviews in chronological order.

Chris: I really enjoyed Doctor Strange. Sure, it’s blockbuster squarely in origin story formula, but the trippiness, action, special effects, and performances made it fun. And the introduction of the new overlapping rulespace of magic makes it a great addition to the Marvel Cinematic Universe. And hey, another Infinity Stone! It’s well-connected to the other films.

Scout: Doctor Strange is another delightful film that further rounds out the Marvel universe. It remained faithful (enough) to the comics that I loved growing up and the casting of Benedict Cumberbatch was spot-on perfect, much as Robert Downey Jr. was for Tony Stark. It is a joyful and at times psychedelic ride that I’m eager to take again. “The Infinity Wars” will be very interesting indeed.

But, as usual, this site is not about the movie but the interfaces, and for that we turn to the three criteria for evaluating movies here on scifiinterfaces.com.

How believable are the interfaces? (To keep you immersed.)
How well do the interfaces inform the narrative of the story? (To tell a good story.)
How well do the interfaces equip the characters to achieve their goals? (To be a good model for real-world design?)

Sci: B- (3 of 4) How believable are the interfaces?

Magic might be a tricky question for narrative believability, as by definition it is a breaking of some set of rules. It’s tempting laziness to patch every hole we find by proclaiming “it’s magic!” and move on. But in most modern stories, magic does have narrative rules; what it’s breaking is known laws of physics or the capabilities of known technology, but still consistent within the world. Oh, hey, kind of like a regular sci-fi story.

The artifacts mostly score quite well for believability. The Boots, the Staff, and the Bands are constrained in what they do, so no surprise there. Even the Cloak is a believable intelligent agent acting for Strange. Its flight-granting and ability to pull in any spatial direction arbitrarily don’t quite jive, but they don’t contradict each other, just raise questions that aren’t answered in the movie itself.

But, the Sling Rings are a trainwreck in terms of usability and believability. With that and the Eye missing some key variables that simply must be specified for it to do what we see it doing, it breaks the diegesis, taking us out of the movie.

Fi: A (4 of 4) How well do the interfaces inform the narrative of the story?

None of these are tacked-on gee-whiz.

Since Strange is occupying an office (Master) that is part of a venerated and peacekeeping secret organization (the Masters of Mysticism) we would expect it to have some tools in place to help the infantry and the boss.
That the powerful artifacts choose their masters helps establish Strange as unique and worthy.
The Eye is core to the plot, and the film uses it to convey how much of a talent and rulebreaking maverick Strange is.
The Staff helps us see Mordo’s militancy, threat, and lawful neutral-ness.
The laugh-out-loud comedy of the Cloak comes from its earnestly trying to help, its constraints, and how Strange is really, really new to this job.
Even the dumb Sling Ring helps show Strange’s learning and confidence, and set up how Strange gets stabbed and yadda yadda yadda begins his reconciliation with Dr. Palmer.

Cloak-of-Levitation-pulling — Once more, because it was so damned funny.

All great narrative uses of the “tech” in the film.

Interfaces: C+ (2 of 4)
How well do the interfaces equip the characters to achieve their goals?

The Boots do. The Cloak totally does. The “AR” surgical assistant does. (And it’s not even an artifact.) If we ever get to technologies that would enable such things, these would be fine models for real world equivalents. (With the long note about general intelligence needing language for strategic discussions with humans.)

That aside, the Sling Ring services a damned useful purpose, but its design is a serious impediment to its utility, and all the Masters of the Mystic Arts uses it. The Staff kind of helps its user, i.e. Mordo, but you have to credit it with a great deal of contextual intelligence or some super-subtle control mechanism. The Bands are so clunky that they’re only useful in the exact context in which they are used. And the Eye, with its missing controls, missing displays, and dangerously ambiguous modes, is a universe-crashing temporal crisis just waiting to happen. This is where the artifacts suffer the most. For that, it gets the biggest hit.

Final Grade B- (9 of 12), Must-see.

Definitely see it. It’s got some obvious misses, but a lot of inventive, interesting stuff, and some that are truly cutting edge concepts. In a hat tip to Arthur C. Clarke’s famous third law, I suppose this is “sufficiently advanced technology.”

Dr. Strange’s augmented reality surgical assistant

26 Sep 2017 by Christopher Noessel

We’re actually done with all of the artifacts from Doctor Strange. But there’s one last kind-of interface that’s worth talking about, and that’s when Strange assists with surgery on his own body.

After being shot with a soul-arrow by the zealot, Strange is in bad shape. He needs medical attention. He recovers his sling ring and creates a portal to the emergency room where he once worked. Stumbling with the pain, he manages to find Dr. Palmer and tell her he has a cardiac tamponade. They head to the operating theater and get Strange on the table.

When Strange passes out, his “spirit” is ejected from his body as an astral projection. Once he realizes what’s happened, he gathers his wits and turns to observe the procedure.

When Dr. Palmer approaches his body with a pericardiocentesis needle, Strange manifests so she can sense him and recommends that she aim “just a little higher.” At first she is understandably scared, but once he explains what’s happening, she gets back to business, and he acts as a virtual coach.

Continue reading →

The Dark Dimension mode (5 of 5)

19 Sep 2017 by Christopher Noessel

We see a completely new mode for the Eye in the Dark Dimension. With a flourish of his right hand over his left forearm, a band of green lines begin orbiting his forearm just below his wrist. (Another orbits just below his elbow, just off-camera in the animated gif.) The band signals that Strange has set this point in time as a “save point,” like in a video game. From that point forward, when he dies, time resets and he is returned here, alive and well, though he and anyone else in the loop is aware that it happened.

In the scene he’s confronting a hostile god-like creature on its own mystical turf, so he dies a lot.

An interesting moment happens when Strange is hopping from the blue-ringed planetoid to the one close to the giant Dormammu face. He glances down at his wrist, making sure that his savepoint was set. It’s a nice tell, letting us know that Strange is a nervous about facing the giant, Galactus-sized primordial evil that is Dormammu. This nervousness ties right into the analysis of this display. If we changed the design, we could put him more at ease when using this life-critical interface.

Initiating gesture

The initiating gesture doesn’t read as “set a savepoint.” This doesn’t show itself as a problem in this scene, but if the gesture did have some sort of semantic meaning, it would make it easier for Strange to recall and perform correctly. Maybe if his wrist twist transitioned from moving splayed fingers to his pointing with his index finger to his wrist…ok, that’s a little too on the nose, so maybe…toward the ground, it would help symbolize the here & now that is the savepoint. It would be easier for Strange to recall and feel assured that he’d done the right thing.

I have questions about the extents of the time loop effect. Is it the whole Dark Dimension? Is it also Earth? Is it the Universe? Is it just a sphere, like the other modes of the Eye? How does he set these? There’s not enough information in the movie to backworld this, but unless the answer is “it affects everything” there seems to be some variables missing in the initiating gesture.

Setpoint-active signal

But where the initiating gesture doesn’t appear to be a problem in the scene, the wrist-glance indicates that the display is. Note that, other than being on the left forearm instead of the right, the bands look identical to the ones in the Tibet and Hong Kong modes. (Compare the Tibet screenshot below.) If Strange is relying on the display to ensure that his savepoint was set, having it look identical is not as helpful as it would be if the visual was unique. “Wait,” he might think, “Am I in the right mode, here?”

In a redesign, I would select an animated display that was not a loop, but an indication that time was passing. It can’t be as literal as a clock of course. But something that used animation to suggest time was progressing linearly from a point. Maybe something like the binary clock from Mission to Mars (see below), rendered in the graphic language of the Eye. Maybe make it base-3 to seem not so technological.

Seeing a display that is still, on invocation—that becomes animated upon initialization—would mean that all he has to do is glance to confirm the unique display is in motion. “Yes, it’s working. I’m in the Groundhog Day mode, and the savepoint is set.”

The Hong Kong Mode (4 of 5)

12 Sep 2017 by Christopher Noessel

In the prior three posts, I’ve discussed the goods-and-bads of the Eye of Agamotto in the Tibet mode. (I thought I could squeeze the Hong Kong and the Dark Dimension modes into one post, but turns out this one was just too long. keep reading. You’ll see.) In this post we examine a mode that looks like the Tibet mode, but is actually quite different.

Hong Kong mode

Near the film’s climax, Strange uses the Eye to reverse Kaecilius’ destruction of the Hong Kong Sanctum Sanctorum (and much of the surrounding cityscape). In this scene, Kaecilius leaps at Strange, and Strange “freezes” Kaecilius in midair with the saucer. It’s done more quickly, but similarly to how he “freezes” the apple into a controlled-time mode in Tibet.

But then we see something different, and it complicates everything. As Strange twists the saucer counterclockwise, the cityscape around him—not just Kaecilius—begins to reverse slowly. (And unlike in Tibet, the saucer keeps spinning clockwise underneath his hand.) Then the rate of reversal accelerates, and even continues in its reversal after Strange drops his gesture and engages in a fight with Kaecilius, who somehow escapes the reversing time stream to join Strange and Mordo in the “observer” time stream.

So in this mode, the saucer is working much more like a shuttle wheel with no snap-back feature.

A shuttle wheel, as you’ll recall from the first post, doesn’t specify an absolute value along a range like a jog dial does. A shuttle wheel indicates a direction and rate of change. A little to the left is slow reverse. Far to the left is fast reverse. Nearly all of the shuttle wheels we use in the real world have snap-back features, because if you were just going to leave it reversing and pay attention to something else, you might as well use another control to get to the absolute beginning, like a jog dial. But, since Strange is scrubbing an endless “video stream,” (that is, time), and he can pull people and things out of the manipulated-stream and into the observer-stream and do stuff, not having a snap-back makes sense.

For the Tibet mode I argued for a chapter ring to provide some context and information about the range of values he’s scrubbing. So for shuttling along the past in the Hong Kong mode, I don’t think a chapter ring or content overview makes sense, but it would help to know the following.

The rate of change
Direction of change
Shifted datetime
Timedate difference from when he started

In the scene that information is kind of obvious from the environment, so I can see the argument for not having it. But if he was in some largely-unchanging environment, like a panic room or an underground cave or a Sanctum Sanctorum, knowing that information would save him from letting the shuttle go too far and finding himself in the Ordovician. A “home” button might also help to quickly recover from mistakes. Adding these signals would also help distinguish the two modes. They work differently, so they should look different. As it stands, they look identical.

He still (probably) needs future branches

Can Strange scrub the future this way? We don’t see it in the movie. But if so, we have many of the same questions as the Tibet mode future scrubber: Which timeline are we viewing & how probable is it? What other probabilities exist and how does he compare them? This argues for the addition of the future branches from that design.

Selecting the mode

So how does Strange specify the jog dial or shuttle wheel mode?

One cop-out answer is a mental command from Strange. It’s a cop-out because if the Eye responds to mental commands, this whole design exercise is moot, and we’re here to critique, practice, and learn. Not only that, but physical interfaces are more cinegenic, so better to make a concrete interaction for the film.

You might think we could modify the opening finger-tut (see the animated gif, below). But it turns out we need that for another reason: specifying the center and radius-of-effect.

Center and radius-of-effect

In Tibet, the Eye appears to affect just an apple and a tome. But since we see it affecting a whole area in Hong Kong, let’s presume the Eye affects time in a sphere. For the apple and tome, it was affecting a small sphere that included the table, too, it’s just that table didn’t change in the spans of time we see. So if it works in spheres, how is the center and the radius of the sphere set?

Center

Let’s say the Eye does some simple gaze monitoring to find the salient object at his locus of attention. Then it can center the effect on the thing and automatically set the radius of effect to the thing’s size across likely-to-be scrubbed extents. In Tibet, it’s easy. Apple? Check. Tome? Check. In Hong Kong, he’s focusing on the Sanctum, and its image recognition is smart enough to understand the concept of “this building.”

Radius

But the Hong Kong radius stretches out beyond his line of sight, affecting something with a very vague visual and even conceptual definition, that is, “the wrecked neighborhood.” So auto-setting these variables wouldn’t work without reconceiving the Eye as a general artificial intelligence. That would have some massive repercussions throughout the diegesis, so let’s avoid that.

If it’s a manual control, how does he do it? Watch the animated gif above carefully and see he’s got two steps to the “turn Eye on” tut: opening the eye by making an eye shape, and after the aperture opens, spreading his hands apart, or kind of expanding the Eye. In Tibet that spreading motion is slow and close. In Hong it’s faster and farther. That’s enough evidence to say the spread*speed determines the radius. We run into the scales problem of apple-versus-neighborhood that we had in determining the time extents, but make it logarithmic and add some visual feedback and he should be able to pick arbitrary sizes with precision.

So…back to mode selection

So if we’re committing the “turn on” gesture to specifying the center-and-radius, the only other gesture left is the saucer creation. For a quick reminder, here’s how it works in Tibet.

Since the circle works pretty well for a jog dial, let’s leave this for Tibet mode. A contrasting but related gesture would be to have Strange hold his right hand flat, in a sagittal plane, with the palm facing to his left. (See an illustration, below.) Then he can tilt his hand inside the saucer to reverse or fast forward time, and withdraw his hand from the saucer graphic to leave time moving at the adjusted rate. Let the speed of the saucer indicate speed of change. To map to a clock, tilting to the left would reverse time, and tilting to the right would advance it.

How the datetime could be shown is an exercise for the reader.

The yank out

There’s one more function we see twice in the Hong Kong scene. Strange is able to pull Mordo and Wong from the reversing time stream by thrusting the saucer toward them. This is a goofy choice of a gesture that makes no semantic sense. It would make much more sense for Strange to keep his saucer hand extended, and use his left hand to pull them from the reversing stream.

Whew.

So one of the nice things about this movie interface, is that while it doesn’t hold up under the close scrutiny of this blog, the interface to the Eye of Agamotto works while watching the film. Audience sees the apple happen, and gets that gestures + glowing green circle = adjusting time. For that, it works.

That said, we can see improvements that would not affect the script, would not require much more of the actors, and not add too much to post. It could be more consistent and believable.

But we’re not done yet. There’s one other function shown by the Eye of Agamotto when Strange takes it into the Dark Dimension, which is the final mode of the Eye, up next.

Tibet Mode Analysis: Representing the future (3 of 5)

5 Sep 2017 by Christopher Noessel

A major problem with the use of the Eye is that it treats the past and the future similarly. But they’re not the same. The past is a long chain of arguably-knowable causes and effects. So, sure, we can imagine that as a movie to be scrubbed.

But the future? Not so much. Which brings us, briefly, to this dude.

If we knew everything, Pierre-Simon Laplace argued in 1814, down to the state of every molecule, and we had a processor capable, we would be able to predict with perfect precision the events of the future. (You might think he’s talking about a computer or an AI, but in 1814 they used demons for their thought experiments.) In the two centuries since, there have been several major repudiations of Laplace’s demon. So let’s stick to the near-term, where there’s not one known future waiting to happen, but a set of probabilities. That means we have to rethink what the Eye shows when it lets Strange scrub the future.

Note that in the film, the “future” of the apple shown to Strange was just a likelihood, not a fact. The Eye shows it being eaten. In the actual events of the film, after the apple is set aside:

Strange repairs the tome
Mordo and Wong interrupt Strange
They take him into the next room for some exposition
The Hong Kong sanctum portal swings open
Kaecilius murders a redshirt
Kaecilius explodes Strange into the New York sanctum

Then for the next 50 minutes, The Masters of Mysticism are scrambling to save the world. I doubt any of them have time to while away in a library, there to discover an abandoned apple with a bite taken out of it, and decide—staphylococcus aureus be damned—a snack’s a snack. No, it’s safe to say the apple does not get eaten.

So the Eye gets the apple wrong, but it showed Strange that future as if it were a certainty. That’s a problem. Sure, when asked about the future, it ought to show something, but better would be to…

Indicate somewhere that what is being displayed is one of a set of possibilities
Provide options to understand the probability distribution among the set
Explore the alternates
Be notified when new data shifts the probability distribution or inserts new important possibilities

So how to display probabilities? There are lots of ways, but I am most fond of probability tree diagrams. In nerd parlance, this is a unidirectional graph where the nodes are states and the lines are labeled for probabilities. In regular language they look like sideways two-dimensional trees. See an example below from mathisfun.com. These diagrams seem to me a quick way to understand branching possibilities. (I couldn’t find any studies giving me more to work on than “seem to me”.)

In addition to being easy to understand, they afford visual manipulation. You can work branching lines around an existing design.

Now if we were actually working out a future-probabilities gestural scrubber attached to the Eye of Agamotto saucer, we’d have a whole host of things to get into next, like designing…

A compact but informative display that signals the relative probabilities of each timeline
The mechanism for opening that display so probabilities can be seen rather than read
Labels so Strange wouldn’t have to hunt through all of them for the thing of interest (or some means of search)
A selection process for picking the new timeline
A comparison mode
A means of collapsing the display to return to scrub mode
A you-are-here signal in the display to indicate the current timeline

Which is a big set of design tasks for a hobbyist website. Fortunately for us, Strange only deals with a simple, probable (but wrong) scenario of the apple’s future as an illustration for the audience of what the Eye can do; and he only deals with the past of the tome. So while we could get into all of the above, it’s most expedient just to resolve the first one for the scene and tidy up the interface as it helps illustrate a well-thought-out and usable world.

Below I’ve drafted up an extension of my earlier conceptual diagram. I’ve added a tree to the future part of the chapter ring, using some dots to indicate the comparative likelihood of each branch. This could be made more compact, and might be good to put on a second z-axis layer to distinguish it from the saucer, but again: conceptual diagram.

If this were implemented in the film, we would want to make sure that the probability tree begins to flicker right before Wong and Mordo shut him down, as a nod to the events happening off screen with Kaecilius that are changing those futures. This would give a clue that the Eye is smartly keeping track of real-world events and adjusting its predictions appropriately.

These changes would make the Eye more usable for Strange and smart as a model for us.

Twist ending: This is a real problem we will have to solve

I skipped those design tasks for this comp, but we may not be able to avoid those problems forever. As it turns out, this is not (just) an idle, sci-fi problem. One of the promises of assistive AI is that it will be giving its humans advice, based on predictive algorithms, which will be a set of probabilistic scenarios. There may be an overwhelmingly likely next scenario, but there may also be several alternatives that users will need to explore and understand before deciding the best strategy. So, yeah, an exercise for the reader.

Wrapping up the Tibet Mode

So three posts is not the longest analysis I’ve done, bit it was a lot. In recap: Gestural time scrubbing seems like a natural interaction mapped well to analog clocks. The Eye’s saucer display is cool, but insufficient. We can help Strange much more by adding an events-based chapter ring detailing the facts of the past and the probabilities of the future.

Alas. We’re not done yet. As you’ll recall from the intro post, there are two other modes: The Hong Kong and Dark Dimension modes. Let’s next talk the Hong Kong mode, which is like the Tibet mode, but different.

Tibet mode: Display for interestingness (2 of 5)

29 Aug 2017 by Christopher Noessel

Without a display, the Eye asks Strange to do all the work of exploring the range of values available through it to discover what is of interest. (I am constantly surprised at how many interfaces in the real world repeat this mistake.) We can help by doing a bit of “pre-processing” of the information and provide Strange a key to what he will find, and where, and ways to recover exactly where interesting things happen.

The watch from the film, for reasons that will shortly become clear.

To do this, we’ll add a ring outside the saucer that will stay fixed relative to the saucer’s rotation and contain this display. Since we need to call this ring something, and we’re in the domain of time, let’s crib some vocabulary from clocks. The fixed ring of a clock that contains the numbers and minute graduations is called a chapter ring. So we’ll use that for our ring, too.

What chapter ring content would most help Strange?

Good: A time-focused chapter ring

Both the controlled-extents and the auto-extents shown in the prior post presume a smooth display of time. But the tome and the speculative meteorite simply don’t change much over the course of their existence. I mean, of course they do, with the book being pulled on and off shelves and pages flipped, and the meteorite arcing around the sun in the cold vacuum of space for countless millennia, but the Eye only displays the material changes to an object, not position. So as far as the Eye is concerned, the meteoroid formed, then it stays the same for most of its existence, then it has a lot of activity as it hits Earth’s atmosphere and slams into the planet.

A continuous display of the book shows little of interest for most of its existence, with a few key moments of change interspersed. To illustrate this, lets make up some change events for the tome.

Now let’s place those along an imaginary timeline. Given the Doctor Strange storyline, Page Torn would more likely be right next to Now, but making this change helps us explore a common boredom problem, see below. OK. Placing those events along a timeline…

And then, wrapping that timeline around the saucer. Much more art direction would have to happen to make this look thematically like the rest of the MCU magic geometries, but following is a conceptual diagram of how it might look.

With time flowing smoothly, though at different speeds for the past and the future.

On the outside of the saucer is the chapter ring with the salient moments of change called out with icons (and labels). At a glance Strange would know where the fruitful moments of change occur. He can see he only has to turn his hand about 5° to the left to get to the spot where the page was ripped out.

Already easier on him, right? Some things to note.

The chapter ring must stay fixed relative to the saucer to work as a reference. Imagine how useless a clock would be if its chapter ring spun in concert with any of its hands. The center can still move with his palm as the saucer does.
The graduations to the left and right of “now” are of a different density, helping Strange to understand that past and future are mapped differently to accommodate the limits of his wrist and the differing time frames described.
When several events occur close together in time, they could be stacked.
Having the graduations evenly spaced across the range helps answer roughly when each change happened relative to the whole.
The tome in front of him should automatically flip to spreads where scrubbed changes occur, so Strange doesn’t have to hunt for them. Without this feature, if Strange was trying to figure out what changed, he would have to flip through the whole book with each degree of twist to see if anything unknown had changed.

Better: A changes-focused chapter ring

If, as in this scene, the primary task of using the Eye is to look for changes, a smooth display of time on the chapter ring is less optimal than a smooth display of change. (Strange doesn’t really care when the pages were torn. He just wants to see the state of the tome before that moment.) Distribute the changes evenly around the chapter ring, and you get something like the following.

This display optimizes for easy access to the major states of the book. The now point is problematic since the even distribution puts it at the three o’clock point rather than the noon, but what we buy in exchange is that the exact same precision is required to access any of the changes and compare them. There’s no extra precision needed to scrub between the book made and the first stuff added moments. The act of comparison is made simpler. Additionally, the logarithmic time graduations help him scrub detail near known changes and quickly bypass the great stretches of time when nothing happens. By orienting our display around the changes, the interesting bits are made more easy to explore, and the boring bits are more easy to bypass.

In my comp, more white areas equal more time. Unfortunately, this visual design kind of draws attention to the empty stretches of time rather than the moments of change, so would need more attention; see the note above about needing a visual designer involved.

So…the smooth time and the distributed events display each has its advantages over the other, but for the Tibet scene, in which he’s looking to restore the lost pages of the tome, the events-focused chapter ring gets Strange to the interesting parts more confidently.

Note that all the events Strange might be scrubbing through are in the past, but that’s not all the Eye can do in the Tibet mode. So next up, let’s talk a little about the future.

Eye of Agamotto (1 of 5)

22 Aug 2017 by Christopher Noessel

This is one of those sci-fi interactions that seems simple when you view it, but then on analysis it turns out to be anything but. So set aside some time, this analysis will be one of the longer ones even broken into four parts.

The Eye of Agamotto is a medallion that (spoiler) contains the emerald Time Infinity Stone, held on by a braided leather strap. It is made of brass, about a hand’s breadth across, in the shape of a stylized eye that is covered by the same mystical sigils seen on the rose window of the New York Sanctum, and the portal door from Kamar-Taj to the same.

World builders may rightly ask why this universe-altering artifact bears a sigil belonging to just one of the Sanctums.

We see the Eye used in three different places in the film, and in each place it works a little differently.

The Tibet Mode
The Hong Kong Modes
The Dark Dimension Mode

The Tibet Mode

When the film begins, the Eye is under the protection of the Masters of the Mystic Arts in Kamar-Taj, where there’s even a user manual. Unfortunately it’s in mysticalese (or is it Tibetan? See comments) so we can’t read it to understand what it says. But we do get a couple of full-screen shots. Are there any cryptanalysists in the readership who can decipher the text?

They really should put the warnings before the spells.

The power button

Strange opens the old tome and reads “First, open the eye of Agamotto.” The instructions show him how to finger-tut a diamond shape with both hands and spread them apart. In response the lid of the eye opens, revealing a bright green glow within. At the same time the components of the sigil rotate around the eye until they become an upper and lower lid. The green glow of this “on state” persists as long as Strange is in time manipulation mode.

Once it’s turned on, he puts the heels of his palms together, fingers splayed out, and turns them clockwise to create a mystical green circle in the air before him. At the same time two other, softer green bands spin around his forearm and elbow. Thrusting his right hand toward the circle while withdrawing his left hand behind the other, he transfers control of the circle to just his right hand, where it follows the position of his palm and the rotation of his wrist as if it was a saucer mystically glued there.

Then he can twist his wrist clockwise while letting his fingers close to a fist, and the object on which he focuses ages. When he does this to an apple, we see it with progressively more chomps out of it until it is a core that dries and shrivels. Twisting his wrist counter clockwise, the focused object reverses aging, becoming younger in staggered increments. With his middle finger upright, the object reverts to its “natural” age.

Pausing and playing

At one point he wants to stop practicing with the apple and try it on the tome whose pages were ripped out. He relaxes his right hand and the green saucer disappears, allowing him to manipulate it and a tome without changing their ages. To reinstate the saucer, he extends his fingers out and gives his hand a shake, and it fades back into place.

Tibet Mode Analysis: The best control type

The Eye has a lot of goodness to it. Time has long been mapped to circles in sun dials and clock faces, so the circle controls fit thematically quite well. The gestural components make similar sense. The direction of wrist twist coincides with the movement of clock hands, so it feels familiar. Also we naturally look at and point at objects of focus, so using the extended arm gesture combined with gaze monitoring fits the sense of control. Lastly, those bands and saucers look really cool, both mystical in pattern and vaguely technological with the screen-green glow.

Readers of the blog know that it rarely just ends after compliments. To discuss the more challenging aspects of this interaction with the Eye, it’s useful to think of it as a gestural video scrubber for security footage, with the hand twist working like a jog wheel. Not familiar with that type of control? It’s a specialized dial, often used by video editors to scroll back and forth over video footage, to find particular sequences or frames. Here’s a quick show-and-tell by YouTube user BrainEatingZombie.

Is this the right kind of control?

There are other options to consider for the dial types of the Eye. What we see in the movie is a jog dial with hard stops, like you might use for an analogue volume control. The absolute position of the control maps to a point in a range of values. The wheel stops at the extents of the values: for volume controls, complete silence on one end and max volume at the other.

But another type is a shuttle wheel. This kind of dial has a resting position. You can turn it clockwise or counterclockwise, and when you let go, it will spring back to the resting position. While it is being turned, it enacts a change. The greater the turn, the faster the change. Like a variable fast-forward/reverse control. If we used this for a volume control: a small turn to the left means, “Keep lowering the volume a little bit as long as I hold the dial here.” A larger turn to the left means, “Get quieter faster.” In the case of the Eye, Strange could turn his hand a little to go back in time slowly, and fully to reverse quickly. This solves some mapping problems (discussed below) but raises new issues when the object just doesn’t change that much across time, like the tome. Rewinding the tome, Strange would start slow, see no change, then gradually increase speed (with no feedback from the tome to know how fast he was going) and suddenly he’d fly way past a point of interest. If he was looking for just the state change, then we’ve wasted his time by requiring him to scroll to find it. If he’s looking for details in the moment of change, the shuttle won’t help him zoom in on that detail, either.

There are also free-spin jog wheels, which can specify absolute or relative values, but since Strange’s wrist is not free-spinning, this is a nonstarter to consider. So I’ll make the call and say what we see in the film, the jog dial, is the right kind of control.

So if a jog dial is the right type of dial, and you start thinking of the Eye in terms of it being a video scrubber, it’s tackling a common enough problem: Scouring a variable range of data for things of interest. In fact, you can imagine that something like this is possible with sophisticated object recognition analyzing security footage.

The investigator scrubs the video back in time to when the Mona Lisa, which since has gone missing, reappears on the wall.
INVESTIGATOR
Show me what happened—across all cameras in Paris—to that priceless object…
She points at the painting in the video.
…there.

So, sure, we’re not going to be manipulating time any…uh…time soon, but this pattern can extend beyond magic items a movie.

The scrubber metaphor brings us nearly all the issues we have to consider.

What are the extents of the time frame?
How are they mapped to gestures?
What is the right display?
What about the probabilistic nature of the future?

What are the extents of the time frame?

Think about the mapping issues here. Time goes forever in each direction. But the human wrist can only twist about 270 degrees: 90° pronation (thumb down) and 180° supination (thumb away from the body, or palm up). So how do you map the limited degrees of twist to unlimited time, especially considering that the “upright” hand is anchored to now?

The conceptually simplest mapping would be something like minutes-to-degree, where full pronation of the right hand would go back 90 minutes and full supination 2 hours into the future. (Noting the weirdness that the left hand would be more past-oriented and the right hand more future-oriented.) Let’s call this controlled extents to distinguish it from auto-extents, discussed later.

What if -90/+180 minutes is not enough time to entail the object at hand? Or what if that’s way too much time? The scale of those extents could be modified by a second gesture, such as the distance of the left hand from the right. So when the left hand was very far back, the extents might be -90/+180 years. When the left hand was touching the right, the extents might be -90/+180 milliseconds to find detail in very fast moving events. This kind-of backworlds the gestures seen in the film.

That’s simple and quite powerful, but doesn’t wholly fit the content for a couple of reasons. The first is that the time scales can vary so much between objects. Even -90/+180 years might be insufficient. What if Strange was scrubbing the timeline of a Yareta plant (which can live to be 3,000 years old) or a meteorite? Things exist in greatly differing time scales. To solve that you might just say OK, let’s set the scale to accommodate geologic or astronomic time spans. But now to select meaningfully between the apple and the tome his hand must move mere nanometers and hard for Strange to get right. A logarithmic time scale to that slider control might help, but still only provides precision at the now end of the spectrum.

If you design a thing with arbitrary time mapping you also have to decide what to do when the object no longer exists prior to the time request. If Strange tried to turn the apple back 50 years, what would be shown? How would you help him elegantly focus on the beginning point of the apple and at the same time understand that the apple didn’t exist 50 years ago?

So letting Strange control the extents arbitrarily is either very constrained or quite a bit more complicated than the movie shows.

Could the extents be automatically set per the focus?

Could the extents be set automatically at the beginning and end of the object in question? Those can be fuzzy concepts, but for the apple there are certainly points in time at which we say “definitely a bud and not a fruit” and “definitely inedible decayed biomass.” So those could be its extents.

The extents for the tome are fuzzier. Its beginning might be when its blank vellum pages were bound and its cover decorated. But the future doesn’t have as clean an endpoint. Pages can be torn out. The cover and binding could be removed for a while and the pages scattered, but then mostly brought together with other pages added and rebound. When does it stop being itself? What’s its endpoint? Suddenly the Eye has to have a powerful and philosophically advanced AI just to reconcile Theseus’ paradox for any object it was pointed at, to the satisfaction of the sorcerer using it and in the context in which it was being examined. Not simple and not in evidence.

Auto-extents could also get into very weird mapping. If an object were created last week, each single degree of right-hand-pronation would reverse time by about 2 hours; but if was fated to last a millennium, each single degree of right-hand-supination would advance time by about 5 years. And for the overwhelming bulk of that display, the book wouldn’t change much at all, so the differences in the time mapping between the two would not be apparent to the user and could cause great confusion.

So setting extents automatically is not a simple answer either. But between the two, starting with the extents automatically saves him the work of finding the interesting bits. (Presuming we can solve that tricky end-point problem. Ideas?) Which takes us to the question of the best display, which I’ll cover in the next post.

Sling Ring

8 Aug 2017 by scoutlandish

A sling ring opens magical portals of varying sizes between two locations. A sorcerer imagines the destination, concentrates, holds the hand wearing the ring upright and with the other gesticulates in a circle, and the portal opens with a burst of yellow sparks around the edges of the portal.

How might this function as technology

It can’t.

Teleportation, even given cutting-edge concepts of quantum entanglement, is limited to bits of information. All the writing on this topic that I can find online says that physical portals require too much energy. So we have to write the totality of this device off as a narrative conceit.

We can imagine the input working, though, as a reading-from-the-brain interface that matches a sorcerer’s mental image of a location to a physical location in the world. As if you were able to upload an image and have a search engine identify its location. That said, reading-from-the-brain has edge cases to consider.

What if the envisioned place is only imaginary?
What if the sorcerer only has the vaguest memory of it? Or just a name?
What if the picture is clear but the place no longer exists? (Like, say, Sokovia.)

Perhaps of course the portal just never opens, but how does the sorcerer know that’s the cause of the malfunction? Perhaps a glowing 404 would help the more modern sorcerers understand.

@scifiinterfaces has you covered, Steven.

The gestural component

The circular gesture is the mechanism for initiating the portal, an active meditation that likely makes concentrating on the location easier. If we had to compliment one thing, it’s that the gesture is well mapped to the shape of the portal, and having a gesture-concentration requirement ensures that portals aren’t just popping up at whim around Kamar-taj anytime someone wearing a ring remembers a place.

OK. That done, we’re at the end of the compliments. Because otherwise, it’s just dumb.

No, really. Dumb.

The physical design of the Sling Ring is dumb. Like Dumb and Dumber dumb. There are plenty of examples of objects or interfaces in movies that only exist because a writer was lazy, but the SlingRing™ deserves a special award category unto itself. Continue reading →