Years ago in grad school I heard a speaker tell of a possibly-imaginary museum called the Museum of One Painting. In that telling, the museum was a long hall. The current One Painting (they were occasionally switched out) was hung at the far end from the entrance. As you walked the length of it, to your left you would see paintings and exhibits of the things that had influenced the One Painting. Then at the end you would spend time with the One Painting. On your way out, to your left you could see paintings and other artworks that were influenced by the One Painting.
Ah yes, I believe this was made during Henry Hillinick’s Robbie the Robot phase. Hat tip to the awesome Matte Shot.
I loved this museum concept. It was about depth of understanding. It provided context. It focused visits on building a shared understanding that you could discuss with other visitors, even if they’d gone a week before you. My kind of art museum. I fell in love with this concept pretty hard and began to believe in the intervening decades that it was just a fable, constructed by wishful museum theorists.
Nope. Today I searched for it, and found it. It’s real. It’s housed in a small building in Penza, Russia. The reality is a little different than how I had it described (or, rather, how I wrote it to memory), and I think it’s only in Russian (mine is a pittance), and given recent politics I’m not sure I’d be welcome there; but its beautiful core concept is intact. A deep dive into a single painting at a time.
Just a quick 9-hour drive from Moscow.
The whole reason I bring this up on the blog is because the awesome American cinema chain Alamo Drafthouse is doing something like this, but for film. And not just any film, but for the upcoming Blade Runner 2049. Their Road to Nowhere series examines dystopian films that influenced or were influenced by Blade Runner. Some you probably know and love. (Metropolis!Logan’s Run!The Fifth Element!) Some I’d never heard of but now want to. (1990: The Bronx Warriors! Hardware! Prayer of the Rollerboys!)
I try not to be a gushing fanboy for anything on this blog, but I gotta hand it to the Drafthouse for this. This is my kind of film nerdery. If I was to run a film series, it would be just like this, only with some sci-fi interface analysis and redesign meetups thrown in for good measure. Just thrilled that it’s happening, and there’s a Drafthouse near me in San Francisco. (Sorry if there’s not one near you, but maybe there’s something similar?)
Anyway, I was not paid by anyone to write this. Just…just happy and nerding out. Hope to see you there.
In the priorthreeposts, I’ve discussed the goods-and-bads of the Eye of Agamotto in the Tibet mode. (I thought I could squeeze the Hong Kong and the Dark Dimension modes into one post, but turns out this one was just too long. keep reading. You’ll see.) In this post we examine a mode that looks like the Tibet mode, but is actually quite different.
Hong Kong mode
Near the film’s climax, Strange uses the Eye to reverse Kaecilius’ destruction of the Hong Kong Sanctum Sanctorum (and much of the surrounding cityscape). In this scene, Kaecilius leaps at Strange, and Strange “freezes” Kaecilius in midair with the saucer. It’s done more quickly, but similarly to how he “freezes” the apple into a controlled-time mode in Tibet.
But then we see something different, and it complicates everything. As Strange twists the saucer counterclockwise, the cityscape around him—not just Kaecilius—begins to reverse slowly. (And unlike in Tibet, the saucer keeps spinning clockwise underneath his hand.) Then the rate of reversal accelerates, and even continues in its reversal after Strange drops his gesture and engages in a fight with Kaecilius, who somehow escapes the reversing time stream to join Strange and Mordo in the “observer” time stream.
So in this mode, the saucer is working much more like a shuttle wheel with no snap-back feature.
A shuttle wheel, as you’ll recall from the first post, doesn’t specify an absolute value along a range like a jog dial does. A shuttle wheel indicates a direction and rate of change. A little to the left is slow reverse. Far to the left is fast reverse. Nearly all of the shuttle wheels we use in the real world have snap-back features, because if you were just going to leave it reversing and pay attention to something else, you might as well use another control to get to the absolute beginning, like a jog dial. But, since Strange is scrubbing an endless “video stream,” (that is, time), and he can pull people and things out of the manipulated-stream and into the observer-stream and do stuff, not having a snap-back makes sense.
For the Tibet mode I argued for a chapter ring to provide some context and information about the range of values he’s scrubbing. So for shuttling along the past in the Hong Kong mode, I don’t think a chapter ring or content overview makes sense, but it would help to know the following.
The rate of change
Direction of change
Shifted datetime
Timedate difference from when he started
In the scene that information is kind of obvious from the environment, so I can see the argument for not having it. But if he was in some largely-unchanging environment, like a panic room or an underground cave or a Sanctum Sanctorum, knowing that information would save him from letting the shuttle go too far and finding himself in the Ordovician. A “home” button might also help to quickly recover from mistakes. Adding these signals would also help distinguish the two modes. They work differently, so they should look different. As it stands, they look identical.
He still (probably) needs future branches
Can Strange scrub the future this way? We don’t see it in the movie. But if so, we have many of the same questions as the Tibet mode future scrubber: Which timeline are we viewing & how probable is it? What other probabilities exist and how does he compare them? This argues for the addition of the future branches from that design.
Selecting the mode
So how does Strange specify the jog dial or shuttle wheel mode?
One cop-out answer is a mental command from Strange. It’s a cop-out because if the Eye responds to mental commands, this whole design exercise is moot, and we’re here to critique, practice, and learn. Not only that, but physical interfaces are more cinegenic, so better to make a concrete interaction for the film.
You might think we could modify the opening finger-tut (see the animated gif, below). But it turns out we need that for another reason: specifying the center and radius-of-effect.
Center and radius-of-effect
In Tibet, the Eye appears to affect just an apple and a tome. But since we see it affecting a whole area in Hong Kong, let’s presume the Eye affects time in a sphere. For the apple and tome, it was affecting a small sphere that included the table, too, it’s just that table didn’t change in the spans of time we see. So if it works in spheres, how is the center and the radius of the sphere set?
Center
Let’s say the Eye does some simple gaze monitoring to find the salient object at his locus of attention. Then it can center the effect on the thing and automatically set the radius of effect to the thing’s size across likely-to-be scrubbed extents. In Tibet, it’s easy. Apple? Check. Tome? Check. In Hong Kong, he’s focusing on the Sanctum, and its image recognition is smart enough to understand the concept of “this building.”
Radius
But the Hong Kong radius stretches out beyond his line of sight, affecting something with a very vague visual and even conceptual definition, that is, “the wrecked neighborhood.” So auto-setting these variables wouldn’t work without reconceiving the Eye as a general artificial intelligence. That would have some massive repercussions throughout the diegesis, so let’s avoid that.
If it’s a manual control, how does he do it? Watch the animated gif above carefully and see he’s got two steps to the “turn Eye on” tut: opening the eye by making an eye shape, and after the aperture opens, spreading his hands apart, or kind of expanding the Eye. In Tibet that spreading motion is slow and close. In Hong it’s faster and farther. That’s enough evidence to say the spread*speed determines the radius. We run into the scales problem of apple-versus-neighborhood that we had in determining the time extents, but make it logarithmic and add some visual feedback and he should be able to pick arbitrary sizes with precision.
So…back to mode selection
So if we’re committing the “turn on” gesture to specifying the center-and-radius, the only other gesture left is the saucer creation. For a quick reminder, here’s how it works in Tibet.
Since the circle works pretty well for a jog dial, let’s leave this for Tibet mode. A contrasting but related gesture would be to have Strange hold his right hand flat, in a sagittal plane, with the palm facing to his left. (See an illustration, below.) Then he can tilt his hand inside the saucer to reverse or fast forward time, and withdraw his hand from the saucer graphic to leave time moving at the adjusted rate. Let the speed of the saucer indicate speed of change. To map to a clock, tilting to the left would reverse time, and tilting to the right would advance it.
How the datetime could be shown is an exercise for the reader.
The yank out
There’s one more function we see twice in the Hong Kong scene. Strange is able to pull Mordo and Wong from the reversing time stream by thrusting the saucer toward them. This is a goofy choice of a gesture that makes no semantic sense. It would make much more sense for Strange to keep his saucer hand extended, and use his left hand to pull them from the reversing stream.
Whew.
So one of the nice things about this movie interface, is that while it doesn’t hold up under the close scrutiny of this blog, the interface to the Eye of Agamotto works while watching the film. Audience sees the apple happen, and gets that gestures + glowing green circle = adjusting time. For that, it works.
That said, we can see improvements that would not affect the script, would not require much more of the actors, and not add too much to post. It could be more consistent and believable.
But we’re not done yet. There’s one other function shown by the Eye of Agamotto when Strange takes it into the Dark Dimension, which is the final mode of the Eye, up next.
A major problem with the use of the Eye is that it treats the past and the future similarly. But they’re not the same. The past is a long chain of arguably-knowable causes and effects. So, sure, we can imagine that as a movie to be scrubbed.
But the future? Not so much. Which brings us, briefly, to this dude.
If we knew everything, Pierre-Simon Laplace argued in 1814, down to the state of every molecule, and we had a processor capable, we would be able to predict with perfect precision the events of the future. (You might think he’s talking about a computer or an AI, but in 1814 they used demons for their thought experiments.) In the two centuries since, there have been several major repudiations of Laplace’s demon. So let’s stick to the near-term, where there’s not one known future waiting to happen, but a set of probabilities. That means we have to rethink what the Eye shows when it lets Strange scrub the future.
Note that in the film, the “future” of the apple shown to Strange was just a likelihood, not a fact. The Eye shows it being eaten. In the actual events of the film, after the apple is set aside:
Strange repairs the tome
Mordo and Wong interrupt Strange
They take him into the next room for some exposition
The Hong Kong sanctum portal swings open
Kaecilius murders a redshirt
Kaecilius explodes Strange into the New York sanctum
Then for the next 50 minutes, The Masters of Mysticism are scrambling to save the world. I doubt any of them have time to while away in a library, there to discover an abandoned apple with a bite taken out of it, and decide—staphylococcus aureus be damned—a snack’s a snack. No, it’s safe to say the apple does not get eaten.
So the Eye gets the apple wrong, but it showed Strange that future as if it were a certainty. That’s a problem. Sure, when asked about the future, it ought to show something, but better would be to…
Indicate somewhere that what is being displayed is one of a set of possibilities
Provide options to understand the probability distribution among the set
Explore the alternates
Be notified when new data shifts the probability distribution or inserts new important possibilities
So how to display probabilities? There are lots of ways, but I am most fond of probability tree diagrams. In nerd parlance, this is a unidirectional graph where the nodes are states and the lines are labeled for probabilities. In regular language they look like sideways two-dimensional trees. See an example below from mathisfun.com. These diagrams seem to me a quick way to understand branching possibilities. (I couldn’t find any studies giving me more to work on than “seem to me”.)
In addition to being easy to understand, they afford visual manipulation. You can work branching lines around an existing design.
Now if we were actually working out a future-probabilities gestural scrubber attached to the Eye of Agamotto saucer, we’d have a whole host of things to get into next, like designing…
A compact but informative display that signals the relative probabilities of each timeline
The mechanism for opening that display so probabilities can be seen rather than read
Labels so Strange wouldn’t have to hunt through all of them for the thing of interest (or some means of search)
A selection process for picking the new timeline
A comparison mode
A means of collapsing the display to return to scrub mode
A you-are-here signal in the display to indicate the current timeline
Which is a big set of design tasks for a hobbyist website. Fortunately for us, Strange only deals with a simple, probable (but wrong) scenario of the apple’s future as an illustration for the audience of what the Eye can do; and he only deals with the past of the tome. So while we could get into all of the above, it’s most expedient just to resolve the first one for the scene and tidy up the interface as it helps illustrate a well-thought-out and usable world.
Below I’ve drafted up an extension of my earlier conceptual diagram. Let’s call it a future-inclusive timeline display. I’ve added a tree to the future part of the chapter ring, using some dots to indicate the comparative likelihood of each branch. This could be made more compact, and might be good to put on a second z-axis layer to distinguish it from the saucer, but again: conceptual diagram.
If this were implemented in the film, we would want to make sure that the probability tree begins to flicker right before Wong and Mordo shut him down, as a nod to the events happening off screen with Kaecilius that are changing those futures. This would give a clue that the Eye is smartly keeping track of real-world events and adjusting its predictions appropriately.
These changes would make the Eye more usable for Strange and smart as a model for us.
Twist ending: This is a real problem we will have to solve
I skipped those design tasks for this comp, but we may not be able to avoid those problems forever. As it turns out, this is not (just) an idle, sci-fi problem. One of the promises of assistive AI is that it will be giving its humans advice, based on predictive algorithms, which will be a set of probabilistic scenarios. There may be an overwhelmingly likely next scenario, but there may also be several alternatives that users will need to explore and understand before deciding the best strategy. So, yeah, an exercise for the reader.
Wrapping up the Tibet Mode
So three posts is not the longest analysis I’ve done, bit it was a lot. In recap: Gestural time scrubbing seems like a natural interaction mapped well to analog clocks. The Eye’s saucer display is cool, but insufficient. We can help Strange much more by adding an events-based chapter ring detailing the facts of the past and the probabilities of the future.
Alas. We’re not done yet. As you’ll recall from the intro post, there are two other modes: The Hong Kong and Dark Dimension modes. Let’s next talk the Hong Kong mode, which is like the Tibet mode, but different.
Without a display, the Eye asks Strange to do all the work of exploring the range of values available through it to discover what is of interest. (I am constantly surprised at how many interfaces in the real world repeat this mistake.) We can help by doing a bit of “pre-processing” of the information and provide Strange a key to what he will find, and where, and ways to recover exactly where interesting things happen.
The watch from the film, for reasons that will shortly become clear.
To do this, we’ll add a ring outside the saucer that will stay fixed relative to the saucer’s rotation and contain this display. Since we need to call this ring something, and we’re in the domain of time, let’s crib some vocabulary from clocks. The fixed ring of a clock that contains the numbers and minute graduations is called a chapter ring. So we’ll use that for our ring, too.
What chapter ring content would most help Strange?
Good: A time-focused chapter ring
Both the controlled-extents and the auto-extents shown in the prior post presume a smooth display of time. But the tome and the speculative meteorite simply don’t change much over the course of their existence. I mean, of course they do, with the book being pulled on and off shelves and pages flipped, and the meteorite arcing around the sun in the cold vacuum of space for countless millennia, but the Eye only displays the material changes to an object, not position. So as far as the Eye is concerned, the meteoroid formed, then it stays the same for most of its existence, then it has a lot of activity as it hits Earth’s atmosphere and slams into the planet.
A continuous display of the book shows little of interest for most of its existence, with a few key moments of change interspersed. To illustrate this, lets make up some change events for the tome.
Now let’s place those along an imaginary timeline. Given the Doctor Strange storyline, Page Torn would more likely be right next to Now, but making this change helps us explore a common boredom problem, see below. OK. Placing those events along a timeline…
And then, wrapping that timeline around the saucer. Much more art direction would have to happen to make this look thematically like the rest of the MCU magic geometries, but following is a conceptual diagram of how it might look.
With time flowing smoothly, though at different speeds for the past and the future.
On the outside of the saucer is the chapter ring with the salient moments of change called out with icons (and labels). At a glance Strange would know where the fruitful moments of change occur. He can see he only has to turn his hand about 5° to the left to get to the spot where the page was ripped out.
Already easier on him, right? Some things to note.
The chapter ring must stay fixed relative to the saucer to work as a reference. Imagine how useless a clock would be if its chapter ring spun in concert with any of its hands. The center can still move with his palm as the saucer does.
The graduations to the left and right of “now” are of a different density, helping Strange to understand that past and future are mapped differently to accommodate the limits of his wrist and the differing time frames described.
When several events occur close together in time, they could be stacked.
Having the graduations evenly spaced across the range helps answer roughly when each change happened relative to the whole.
The tome in front of him should automatically flip to spreads where scrubbed changes occur, so Strange doesn’t have to hunt for them. Without this feature, if Strange was trying to figure out what changed, he would have to flip through the whole book with each degree of twist to see if anything unknown had changed.
Better: A changes-focused chapter ring
If, as in this scene, the primary task of using the Eye is to look for changes, a smooth display of time on the chapter ring is less optimal than a smooth display of change. (Strange doesn’t really care when the pages were torn. He just wants to see the state of the tome before that moment.) Distribute the changes evenly around the chapter ring, and you get something like the following.
This display optimizes for easy access to the major states of the book. The now point is problematic since the even distribution puts it at the three o’clock point rather than the noon, but what we buy in exchange is that the exact same precision is required to access any of the changes and compare them. There’s no extra precision needed to scrub between the book made and the first stuff added moments. The act of comparison is made simpler. Additionally, the logarithmic time graduations help him scrub detail near known changes and quickly bypass the great stretches of time when nothing happens. By orienting our display around the changes, the interesting bits are made more easy to explore, and the boring bits are more easy to bypass.
In my comp, more white areas equal more time. Unfortunately, this visual design kind of draws attention to the empty stretches of time rather than the moments of change, so would need more attention; see the note above about needing a visual designer involved.
So…the smooth time and the distributed events display each has its advantages over the other, but for the Tibet scene, in which he’s looking to restore the lost pages of the tome, the events-focused chapter ring gets Strange to the interesting parts more confidently.
Note that all the events Strange might be scrubbing through are in the past, but that’s not all the Eye can do in the Tibet mode. So next up, let’s talk a little about the future.
This is one of those sci-fi interactions that seems simple when you view it, but then on analysis it turns out to be anything but. So set aside some time, this analysis will be one of the longer ones even broken into four parts.
The Eye of Agamotto is a medallion that (spoiler) contains the emerald Time Infinity Stone, held on by a braided leather strap. It is made of brass, about a hand’s breadth across, in the shape of a stylized eye that is covered by the same mystical sigils seen on the rose window of the New York Sanctum, and the portal door from Kamar-Taj to the same.
World builders may rightly ask why this universe-altering artifact bears a sigil belonging to just one of the Sanctums.
We see the Eye used in three different places in the film, and in each place it works a little differently.
The Tibet Mode
The Hong Kong Modes
The Dark Dimension Mode
The Tibet Mode
When the film begins, the Eye is under the protection of the Masters of the Mystic Arts in Kamar-Taj, where there’s even a user manual. Unfortunately it’s in mysticalese (or is it Tibetan? See comments) so we can’t read it to understand what it says. But we do get a couple of full-screen shots. Are there any cryptanalysists in the readership who can decipher the text?
They really should put the warnings before the spells.
The power button
Strange opens the old tome and reads “First, open the eye of Agamotto.” The instructions show him how to finger-tut a diamond shape with both hands and spread them apart. In response the lid of the eye opens, revealing a bright green glow within. At the same time the components of the sigil rotate around the eye until they become an upper and lower lid. The green glow of this “on state” persists as long as Strange is in time manipulation mode.
Once it’s turned on, he puts the heels of his palms together, fingers splayed out, and turns them clockwise to create a mystical green circle in the air before him. At the same time two other, softer green bands spin around his forearm and elbow. Thrusting his right hand toward the circle while withdrawing his left hand behind the other, he transfers control of the circle to just his right hand, where it follows the position of his palm and the rotation of his wrist as if it was a saucer mystically glued there.
Then he can twist his wrist clockwise while letting his fingers close to a fist, and the object on which he focuses ages. When he does this to an apple, we see it with progressively more chomps out of it until it is a core that dries and shrivels. Twisting his wrist counter clockwise, the focused object reverses aging, becoming younger in staggered increments. With his middle finger upright, the object reverts to its “natural” age.
Pausing and playing
At one point he wants to stop practicing with the apple and try it on the tome whose pages were ripped out. He relaxes his right hand and the green saucer disappears, allowing him to manipulate it and a tome without changing their ages. To reinstate the saucer, he extends his fingers out and gives his hand a shake, and it fades back into place.
Tibet Mode Analysis: The best control type
The Eye has a lot of goodness to it. Time has long been mapped to circles in sun dials and clock faces, so the circle controls fit thematically quite well. The gestural components make similar sense. The direction of wrist twist coincides with the movement of clock hands, so it feels familiar. Also we naturally look at and point at objects of focus, so using the extended arm gesture combined with gaze monitoring fits the sense of control. Lastly, those bands and saucers look really cool, both mystical in pattern and vaguely technological with the screen-green glow.
Readers of the blog know that it rarely just ends after compliments. To discuss the more challenging aspects of this interaction with the Eye, it’s useful to think of it as a gestural video scrubber for security footage, with the hand twist working like a jog wheel. Not familiar with that type of control? It’s a specialized dial, often used by video editors to scroll back and forth over video footage, to find particular sequences or frames. Here’s a quick show-and-tell by YouTube user BrainEatingZombie.
Is this the right kind of control?
There are other options to consider for the dial types of the Eye. What we see in the movie is a jog dial with hard stops, like you might use for an analogue volume control. The absolute position of the control maps to a point in a range of values. The wheel stops at the extents of the values: for volume controls, complete silence on one end and max volume at the other.
But another type is a shuttle wheel. This kind of dial has a resting position. You can turn it clockwise or counterclockwise, and when you let go, it will spring back to the resting position. While it is being turned, it enacts a change. The greater the turn, the faster the change. Like a variable fast-forward/reverse control. If we used this for a volume control: a small turn to the left means, “Keep lowering the volume a little bit as long as I hold the dial here.” A larger turn to the left means, “Get quieter faster.” In the case of the Eye, Strange could turn his hand a little to go back in time slowly, and fully to reverse quickly. This solves some mapping problems (discussed below) but raises new issues when the object just doesn’t change that much across time, like the tome. Rewinding the tome, Strange would start slow, see no change, then gradually increase speed (with no feedback from the tome to know how fast he was going) and suddenly he’d fly way past a point of interest. If he was looking for just the state change, then we’ve wasted his time by requiring him to scroll to find it. If he’s looking for details in the moment of change, the shuttle won’t help him zoom in on that detail, either.
There are also free-spin jog wheels, which can specify absolute or relative values, but since Strange’s wrist is not free-spinning, this is a nonstarter to consider. So I’ll make the call and say what we see in the film, the jog dial, is the right kind of control.
So if a jog dial is the right type of dial, and you start thinking of the Eye in terms of it being a video scrubber, it’s tackling a common enough problem: Scouring a variable range of data for things of interest. In fact, you can imagine that something like this is possible with sophisticated object recognition analyzing security footage.
The investigator scrubs the video back in time to when the Mona Lisa, which since has gone missing, reappears on the wall.
INVESTIGATOR
Show me what happened—across all cameras in Paris—to that priceless object…
She points at the painting in the video.
…there.
So, sure, we’re not going to be manipulating time any…uh…time soon, but this pattern can extend beyond magic items a movie.
The scrubber metaphor brings us nearly all the issues we have to consider.
What are the extents of the time frame?
How are they mapped to gestures?
What is the right display?
What about the probabilistic nature of the future?
What are the extents of the time frame?
Think about the mapping issues here. Time goes forever in each direction. But the human wrist can only twist about 270 degrees: 90° pronation (thumb down) and 180° supination (thumb away from the body, or palm up). So how do you map the limited degrees of twist to unlimited time, especially considering that the “upright” hand is anchored to now?
The conceptually simplest mapping would be something like minutes-to-degree, where full pronation of the right hand would go back 90 minutes and full supination 2 hours into the future. (Noting the weirdness that the left hand would be more past-oriented and the right hand more future-oriented.) Let’s call this controlled extents to distinguish it from auto-extents, discussed later.
What if -90/+180 minutes is not enough time to entail the object at hand? Or what if that’s way too much time? The scale of those extents could be modified by a second gesture, such as the distance of the left hand from the right. So when the left hand was very far back, the extents might be -90/+180 years. When the left hand was touching the right, the extents might be -90/+180 milliseconds to find detail in very fast moving events. This kind-of backworlds the gestures seen in the film.
That’s simple and quite powerful, but doesn’t wholly fit the content for a couple of reasons. The first is that the time scales can vary so much between objects. Even -90/+180 years might be insufficient. What if Strange was scrubbing the timeline of a Yareta plant (which can live to be 3,000 years old) or a meteorite? Things exist in greatly differing time scales. To solve that you might just say OK, let’s set the scale to accommodate geologic or astronomic time spans. But now to select meaningfully between the apple and the tome his hand must move mere nanometers and hard for Strange to get right. A logarithmic time scale to that slider control might help, but still only provides precision at the now end of the spectrum.
If you design a thing with arbitrary time mapping you also have to decide what to do when the object no longer exists prior to the time request. If Strange tried to turn the apple back 50 years, what would be shown? How would you help him elegantly focus on the beginning point of the apple and at the same time understand that the apple didn’t exist 50 years ago?
So letting Strange control the extents arbitrarily is either very constrained or quite a bit more complicated than the movie shows.
Could the extents be automatically set per the focus?
Could the extents be set automatically at the beginning and end of the object in question? Those can be fuzzy concepts, but for the apple there are certainly points in time at which we say “definitely a bud and not a fruit” and “definitely inedible decayed biomass.” So those could be its extents.
The extents for the tome are fuzzier. Its beginning might be when its blank vellum pages were bound and its cover decorated. But the future doesn’t have as clean an endpoint. Pages can be torn out. The cover and binding could be removed for a while and the pages scattered, but then mostly brought together with other pages added and rebound. When does it stop being itself? What’s its endpoint? Suddenly the Eye has to have a powerful and philosophically advanced AI just to reconcile Theseus’ paradox for any object it was pointed at, to the satisfaction of the sorcerer using it and in the context in which it was being examined. Not simple and not in evidence.
Auto-extents could also get into very weird mapping. If an object were created last week, each single degree of right-hand-pronation would reverse time by about 2 hours; but if was fated to last a millennium, each single degree of right-hand-supination would advance time by about 5 years. And for the overwhelming bulk of that display, the book wouldn’t change much at all, so the differences in the time mapping between the two would not be apparent to the user and could cause great confusion.
So setting extents automatically is not a simple answer either. But between the two, starting with the extents automatically saves him the work of finding the interesting bits. (Presuming we can solve that tricky end-point problem. Ideas?) Which takes us to the question of the best display, which I’ll cover in the next post.
A sling ring opens magical portals of varying sizes between two locations. A sorcerer imagines the destination, concentrates, holds the hand wearing the ring upright and with the other gesticulates in a circle, and the portal opens with a burst of yellow sparks around the edges of the portal.
How might this function as technology
It can’t.
Teleportation, even given cutting-edge concepts of quantum entanglement, is limited to bits of information. All the writing on this topic that I can find online says that physical portals require too much energy. So we have to write the totality of this device off as a narrative conceit.
We can imagine the input working, though, as a reading-from-the-brain interface that matches a sorcerer’s mental image of a location to a physical location in the world. As if you were able to upload an image and have a search engine identify its location. That said, reading-from-the-brain has edge cases to consider.
What if the envisioned place is only imaginary?
What if the sorcerer only has the vaguest memory of it? Or just a name?
What if the picture is clear but the place no longer exists? (Like, say, Sokovia.)
Perhaps of course the portal just never opens, but how does the sorcerer know that’s the cause of the malfunction? Perhaps a glowing 404 would help the more modern sorcerers understand.
@scifiinterfaces has you covered, Steven.
The gestural component
The circular gesture is the mechanism for initiating the portal, an active meditation that likely makes concentrating on the location easier. If we had to compliment one thing, it’s that the gesture is well mapped to the shape of the portal, and having a gesture-concentration requirement ensures that portals aren’t just popping up at whim around Kamar-taj anytime someone wearing a ring remembers a place.
OK. That done, we’re at the end of the compliments. Because otherwise, it’s just dumb.
No, really. Dumb.
The physical design of the Sling Ring is dumb. Like Dumb and Dumber dumb. There are plenty of examples of objects or interfaces in movies that only exist because a writer was lazy, but the SlingRing™ deserves a special award category unto itself.
First off they aren’t rings, they are more like a set of brass knuckles as designed by a 10 year old who’s never seen a brass knuckle.
It restricts the movement of two fingers on the user’s hand which seem to be critical to casting all other spells in the film.
It makes any kind of physical combat with that hand significantly more difficult which, curiously, is something sorcerers seem to do a lot of. Punching someone while wearing a sling ring would likely break a finger or three.
It’s interesting to note that the sentience that makes other relics special may also be their security. However, any sorcerer can apparently pick up any sling ring and use it. Have you ever been bowling? The first 5 minutes of any trip to a bowling alley is trying to find the right sized ball. So unless there’s someone making bespoke sling rings for new magicians, there’s going to be a lot of poorly sized sling rings out there. This might explain why Strange’s ring is loose enough that it can easily fall off or be pulled off, which is inconvenient at best when fighting another sorcerer.
Redesigning this device as a single ring, or a glove or a wristband, or a necklace, or a tiara or a codpiece…frankly just about anything would be better than this tragically flawed artifact and would likely solve all of the above problems.
Hang on, let me remove my not-sling-not-ring first.
And what the heck does sling even mean in this context?
Dr. Strange uses the Crimson Bands of Cyttorak to immobilize Kaecilius while they are fighting in the New York Sanctum.
The bands are a flexible torso shaped device, that look like a bunch of metal ribs attached to a spine. We do not actually know whether this relic has “chosen” Strange or if it simply functions for anyone who wields it correctly. But given its immense power, it definitely qualifies as a relic and opens up the conversation about whether some relics are simply masterless.
On the name
Discussing the bands is made semantically difficult for two reasons. The first is that “they” are multiple bands joined together by a single “spine” and handled in combat like a single thing. So it needn’t be plural “Bands.” That’s like calling a shoe the Running Laces of Reebok. It is an it not a they. Also it is not Crimson (even in the comic books, most folks would call them pink.) They are not actually named in the film, but authoritative source material indicates that is what these are. So forgive the weirdness, but this post will discuss the bands as a single thing. An it.
So where did it get its plural name? Comic book fans have already noted: In the books, the Crimson Bands of Cyttorak are actually a spell for binding. They are—no surprise—glowing crimson bands of energy, and used by many spellcasters, not just Strange. Here they are in The Uncanny X-Men, cast by the Scarlet Witch and subsequently smashed by Magik.
Since these are in fact, multiples, its plural there makes sense. Its reimagining as a single thing for the Marvel Cinematic Universe is what has us speaking of it in the singular.
Operation
The operation of the device seems to be very simple. Upon impact with a person, the bands quickly twist and clamp to immobilize the limbs, arms, head, and torso of the person. It also covers their mouth so no spells can be cast (or instructions given to minions).
How might this work as technology?
The Crimson Bands are robotic device composed of a self-powered collection of electromagnetic metal hinges and locks, with bands that are interwoven with muscle wire to adjust to a person’s body.
En route to a collision with a humanoid body, the microcomputer scans the surface area of the person using LIDAR to create a surface mapping and body model. This allows the Bands to adjust itself midair to achieve the proper orientation. Then the primary mechanical controller snaps each band shut in sequence beginning with the torso and expanding to the extremities. As each section finds purchase, it tightens, adjusting and locking in place. The sequence takes only a few seconds and leaves the victim in a rigid kneeling position where they are immobilized to prevent any kind of action or spellcasting. The bands that cover the mouth can be removed independently to allow interrogation.
What works well
It’s non-lethal, but still neutralizes a very powerful enemy.
It does not require a great deal of precision. Toss it roughly toward the enemy, and it will do its best to compensate for your imprecision. “Close enough” counts in horseshoes, hand grenades, earthquakes, and the Crimson Bands.
It probably works on a variety of humanoid body shapes and sizes.
The ability to remove the mouth band is useful for interrogation.
Improvements: Provide a keyhole
We never see an obvious release mechanism, but there is one, as Strange discovers when Kaecilius has been freed when Strange returns to the Sanctum. We can’t exactly critique what we don’t see, but we can discuss the affordance of the function. Should it be obvious? Probably but not too obvious, less a zealot run up and quickly figure out how to free their boss. But there should be some sort of “keyhole” that will help Strange know that he needs some sort of “key” to release his prisoner and not leave him in a stress position indefinitely. So there should be the equivalent of a keyhole on the bands. And hopefully one the Cloak can explain, or Strange can figure out.
Improvements: Miniaturize
While highly effective, this artifact is not particularly useful unless you happen to be in a fight while standing next to it. It’s simply too large and bulky to carry around. So, if it could be miniaturized, it would prove more generally useful. Or maybe, since this occurs in a world with magical flight and teleportation, they could be summonable.
Improvements: Authorization
It seems that the Cloak isn’t authorized to use this artifact. (Why else waste all that time trying to pull Strange to it?) Authorizing the Cloak would mean additional agentive controls for dealing with, you know, groups of zealots. We also don’t know if it has a “whitelist” of people it should not immobilize, but that would be useful to prevent it being turned against its owner.
Improvements: Nonhumanoid
The overall shape of the bands are humanoid, indicating that they would not likely function on anything other than a being with the standard humanoid legs, arms, head, mouth, and torso. This fortunately covers most of the characters in the Marvel Universe, but there are a handful of exceptions. If it had a more micro-component based design, it might be able to reconfigure itself to help with non-humanoid malefactors as well.
Then we’d have to come up with some other mechanism to account for scaling to Rhunian, Celestial, or Faltine-sized things.
Mordo wears the Vaulting Boots of Valtor throughout the movie and first demonstrates their use to Dr. Strange when they are sparring. The Boots allow the user to walk, run, or jump on air as if it were solid ground.
When activated, the sole of each boot creates a circular field of force in anticipation of a footfall in midair, as if creating free-floating stepping stones.
How might this work as tech?
The main interaction design challenge is how the wearer indicates where he wants a stepping-stone to appear. The best solution is to let Mordo’s footfall location and motion inform the boots when and where he expects there to be a solid surface. (Anyone who has stumbled while misjudging the height or location of a step on a stairway knows how differently you treat a step where you expect there to be solid footing.)
If this were a technological device, sensors within the boots would retain a detailed history of the wearer’s stride for all possible speeds and distances of movement. The boots would detect muscle tension and flexion combined with the owner’s direction and velocity to accurately predict the placement of each step and then insert an appropriately elevated and angled stepping stone. The boots would know the difference between each of these styles of movement, walking, running, and sprinting and behave accordingly.
As a result, Mordo could always remain upright and stable regardless of his intended direction or how high he had climbed. And while Mordo may be a sorcerer with exceptional physical training, he isn’t superhuman. With the power of the boots he is only able to run and step as high as he could normally if for example he was taking a set of stairs two or three at a time.
As a magical device, the intelligence imbued in the boots is limited to the awareness of the intent of the sorcerer and knows where to place each force-field stepping-stone.
The glowing bits
As each step lands, the placement of the boot results in a brief energy discharge in the shape of a brilliant glowing gold circle. Is this a bug in combat, or a feature? The blog has before called out how glowing bits on a warrior make them an easier target, but it’s worth noting that Mordo’s feet are actually on individual stepping stones for less than a half a second. He leaves them behind as he goes. If someone targeted the circles themselves, they’d mostly be targeting where he was rather than where he is, so I’d count it as a distracting feature. As long as he wasn’t being targeted with a long-distance area-of-effect weapon.
Activation?
When describing them to Strange, Mordo demonstrates the effect with a subtle kick. It’s not clear if he’s activating the boots or just demonstrating that they have inherent magical powers.
These boots are awesome. They would require a lot of practice to get used to, but after some tumbles a user could always acquire the high ground on an opponent and they would never need a ladder to change a light bulb. What’s not known is what would happen if the user tried to do parkour style moves where a step would be perpendicular to the ground. Could Mordo walk on walls or the ceiling of a room?
More!
It would be cool to know more about these boots. Could Mordo climb to a given height and then just stand there or is each step is a limited duration effect? Could the boots be used offensively as a kind of boot sized force field? In a fight, Mordor could lash out with a sidekick/step that stops an onrushing attacker not unlike hitting a brick wall.
Since he’s heavily set up to the Big Bad in the sequel, we’ll likely see more of these relics, and get some more of the questions answered.
Given its wealth of capabilities, the main complaint might be its lack of language.
A mute sidekick
It has a working theory of mind, a grasp of abstract concepts, and intention, so why does it not use language as part of a toolkit to fulfill its duties? Let’s first admit that mute sidekicks are kind of a trope at this point. Think R2D2, Silent Bob, BB8, Aladdin’s Magic Carpet (Disney), Teller, Harpo, Bernardo / Paco (admittedly obscure), Mini-me. They’re a thing.
Yes, I know she could talk to other fairies, but not to Peter.
Despite being a trope, its muteness in a combat partner is a significant impediment. Imagine its being able to say, “Hey Steve, he’s immune to the halberd. But throw that ribcage-looking thing on the wall at him, and you’ll be good.” Strange finds himself in life-or-death situations pretty much constantly, so having to disambiguate vague gestures wastes precious time that might make the difference between life and death. For, like, everyone on Earth.
Additionally, its muteness makes it very difficult for Strange to ever understand the reasoning behind any but its most obvious actions. That would be very important if it ever did anything ethically questionable, because intent would be difficult to gauge without the abstract representation of language.
So between the speed and clarity gains of language, I’d want to equip the Cloak with language: spoken, sign, direct-to-brain, or maybe even a mystical weave which, like the Marauder’s Map in Harry Potter, could mystically transform in real-time to convey linguistic information.
Like this but woven.
Using apologetics, though, we can see that the muteness isn’t a bug, it’s an important feature. Read on.
But, the hero…
One of the things writers “buy” with a mute sidekick is it lets the hero be the hero. If the Cloak could speak, suddenly we have a “sidekick” who knows much more and can do a few more things—i.e. fly—than our hero can. This is not good for optics. Note that the Cloak’s last intention seems to be “Keep him looking sorcererly,” which implies a service status, and which it could not do if it upstaged him. So from a narrative perspective, the muteness means they are less like a crime fighting duo, and more of a hero-with-a-sidekick, or dare I note it: a familiar.
Also if you contrast the cape with JARVIS and the Iron HUD, note that this has language, but that conversation is wholly private with Tony. To an outsider, it looks like Tony is just the Iron Man. They may have no clue that it’s probably all the suit. So JARVIS’ use of language doesn’t get in the way of keeping Tony looking hero-ey. I’d even go so far as to say that the muteness is the thing that lets the Cloak be primarily agentive, as language would encourage its use as an assistant.
Other improvements
Language is the big one, but there are a few other improvements I could imagine.
Cloaklet
If this were for a soldier or a police officer or anyone else who could expect combat as part of their job, the Cloak might make them look cartoonish, like a cosplayer rather than a person with authority. But, hey, if it let them fly, and can supernatural protection, maybe that impression wouldn’t last long. Maybe they could take advantage of being underestimated. But I’d still want to look at how little fabric we could get away with and still maintain its benefits.
Remote control / telecommunication
Another constraint seems to be that it has some proximity limits. To imprint on Tony, it makes some social sense that it would want to be within “handshake distance.” But when Strange is astrally overseeing the emergency operation of his own body (a later post), the Cloak sits back at the Sanctum and doesn’t rush to help when things get fighty. Extending its connectivity would help it fulfill its duties. At the very least tape a cell phone to a pocket and let Strange shout orders at it.
Whew. So that’s my take on the Cloak of Levitation. It’s a marvelous piece of speculative “tech” that fits Tony, his future-role as Sorcerer Supreme, and is nicely unique in the MCU.
It’s a great sophisticated example of agentive tech the constraints of which make complete narrative sense. I hope we get to learn much more of this marvelous, agentive familiar in the sequel.
So I mentioned in the intro to this review that I was drawn to review Doctor Strange (with my buddy and co-reviewer Scout Addis) because the Cloak displays some interesting qualities in relation to the book I just published. Buy it, read it, review it on amazon.com, it’s awesome.
That sales pitch done, I can quickly cover the key concepts here.
A tool, like a hammer, is a familiar but comparatively-dumb category of thing that only responds to a user’s input. A hammer is an example. Tool has been the model of the thing we’re designing in interaction design for, oh, 60 years, but it is being mostly obviated by narrow artificial intelligence, which can be understood as automatic, assistive, or agentive.
Assistant technology helps its user with the task she is focused on: Drawing her attention, providing information, making suggestions, maybe helping augment her precision or force. If we think of a hammer again, an assistive might draw her attention to the best angle to strike the nail, or use an internal gyroscope to gently correct her off-angle strike.
Agentive technology does the task for its user. Again with the hammer, she could tell hammerbot (a physical agent, but there are virtual ones, too) what she wants hammered and how. Her instructions might be something like: Hammer a hapenny nail every decimeter along the length of this plinth. As it begins to pound away, she can then turn her attention to mixing paint or whatever.
When I first introduce people to these distinctions, I step one rung up on Wittgenstein’s Ladder and talk about products that are purely agentive or purely assistive, as if agency was a quality of the technology. (Thabks to TU prof P.J. Stappers for distinguishing these as ontological and epistemological approaches.) The Roomba, for example, is almost wholly agentive as a vacuum. It has no handle for you to grab, because it does the steering and pushing and vacuuming.
Once you get these basic ideas in your head, we can take another step up the Ladder together and clarify that agency is not necessarily a quality of the thing in the world. It’s subtler than that. It’s a mode of relationship between user and agent, one which can change over time. Sophisticated products should be able to shift their agency mode (between tool, assistant, agent, and automation) according to the intentions and wishes of their user. Hammerbot is useful, but still kind of dumb compared to its human. If there’s a particularly tricky or delicate nail to be driven, our carpenter might ask hammerbot’s assistance, but really, she’ll want to handle that delicate hammering herself.
Which brings us back to the Cloak.
I wish I knew how to quit you.
Watch the movie carefully and you’ll see that the only time it acts purely on command is when Strange uses it to fly to the Dark Dimension. So it has all of one tool-like function. It initiates the rest of its actions on its own.
So it is assistive or agentive? Well, again, that’s tricky, because it depends on which function we’re talking about. Here is my backworlded list of those functions, in order of importance.
Obey commands (subject to some mystical and unmentioned 4 Laws of Relics?)
Prevent harm to Strange
Halt the thing threatening him
If that’s not possible, get Strange out of harm’s way (pull him to safety)
Catch him, if he’s falling
Critical case: if Strange is disabled and threatened, take care of the threat (the head-wrap scene)
Guide him toward the best tactic for his current situation
Keep him looking sorcererly
Don yourself when appropriate
Try and do your work while being worn (don’t jump off Strange’s shoulders to do something, if you can avoid it)
Groom him if he becomes untidy
Let’s look at each of these.
1. Obeying commands could be any category. Strange can gesturally command it to fly, as he does, treating it as a tool. He can command it to assist him in some task, or assign it a task to do on its own. So that one is all over the board.
2. Preventing harm is mostly agentive. After all, if it’s preventing harm from happening, it should just do it, and not ask, right? The question would just be noise because the answer would almost always be, “of course.” Like a mystical Spider-Sense, this helps Strange avoid a threat he didn’t even know was there. This allows Strange to focus on the most critical thing in the situation, because the Cloak has his back for the minor stuff. (Which, admittedly, isn’t quite how Spider-Man’s Spider-Sense works.)
There’s a bit of conceptual hairsplitting to do here. When Strange is in combat, you might wonder, isn’t his attention on the combat, so it’s assisting him with it? Sure, but that’s the category of thing he’s trying to do, more appropriate to our description of it rather than what he’s thinking. His attention is not on the category of thing he’s doing but on the thing itself. Not combat, but bringing down Kaecelius.
So I’d argue what we see is agentive. Note that it’s easy to imagine an assistive prevention of harm, too. If Strange knows there is a big rock heading his way, he’s going to dodge. The Cloak can predict how well his dodge will work, and add a little oomph if it’s not going to be enough. But I don’t think we see that in the movie.
3. Guiding him toward the best situational tactic is an interesting case. It’s agentive, but in the one case we really see it in action, it looks assistive. Strange is reaching for the halberd on the wall, and the Cloak, knowing that would be ineffective at best, is dragging him toward the thing that will work, which are the Crimson Bands of Cyttorak. Is it assisting him to grab the right thing?
Yes, but again, it’s a question of attention. Strange’s attention is on getting the halberd. The Cloak has run through the scenarios, and knows that probabilistically, the halberd is the wrong choice. So it is having to intervene and draw Strange’s attention to the Bands. In monitoring and correcting Strange’s tactics, it is operating outside his attention and acting agentively.
4. When the Cloak works to keep Strange looking sorcererly it is similarly monitoring his appearance and making adjustments as needed. Strange’s attention does not need to go there, so it’s agentive.
Hang on, you cry, what about that time Strange tells it to stop wiping his face? That’s a command. His attention is on it. Yes, but Strange isn’t commanding it to begin some new task that he wanted it to accomplish. The Cloak found a trigger (blood on face) and initiated a behavior to correct the problem. Strange is correcting the Cloak that he finds this level of grooming to be too much. In the second part of the book I refer to this as tuning a behavior, and it’s one of the interactions that distinguish agentive tech from automation, which does not afford this kind of personalization.
So, in short, the Cloak demonstrates lots and lots of agentive properties, and provides a rich example that helps illuminate differences in the core concepts. If you want to know more about these thoughts as they apply to real world tech as opposed to sci-fi interfaces, there’s that book I mentioned.
Next up, we’ll get to some critiques of the Cloak and suggestions for how it might be improved if it were a real world thing.
