Staff of the Living Tribunal

This staff appears to be made of wood and is approximately a meter long when in its normal form. When activated by Mordo it has several powers. With a strong pull on both ends, the staff expands into a jointed energy nunchaku. It can also extend to an even greater length like a bullwhip. When it impacts a solid object such as a floor, it seems to release a crack of loud energy. Too bad we only ever see it in demo mode.

How might this work as technology?

The staff is composed of concentric rings within rings of material similar to a collapsing travel cup. This allows the device to expand and contract in length. The handle would likely contain the artificial intelligence and a power source that activates when Mordo gives it a gestural command, or if we’re thinking far future, a mental one. There might also be an additional control for energy discharge.

In the movie, sadly, Mordo does not use the Staff to its best effect, especially when Kaecilius returns to the New York sanctum. Mordo could easily disrupt the spell being cast by the disciples using the staff like a whip, but instead he leaps off the balcony to physically attack them. Dude, you’re the franchise’s next Big Bad? But let’s put down the character’s missteps to look at the interface.

Mode switching and inline meta-signals

Any time you design a thing with modes, you have to design the state changes between those modes. Let’s look at how Mordo moves between staff, nunchaku, and whip in this short demonstration scene.

To go from staff to nunchaku, Mordo pulls it apart. It’s now in a dangerous state, so is there any authentication or safety switch here? It could be there, but all passive via contact sensors, which would be the best so it could be activated in a hurry. The film doesn’t give us any clue, really, so that’s an open question.

How does it know to go from nunchaku to whip? It sure would be crappy to bet on a disabling thwack against your opponent only to find it lazily draping over a shoulder instead. (Pere Perez might have advanced ideas, given his ideas on light saber tactics.) Again, this state change could be passive, detecting in real time the subtle gestural differences in a distal snap, which a bullwhip would need, and lateral force, which sets the nunchaku spinning, and adjust between the two accordingly. Gestural and predictive technologies are not cinemagenic, so let’s give it the benefit of the doubt and say that’s what’s happening.

A last mode is After Mordo cracks it against the ground, it retracts back to Staff form. This is the hardest one to buy. Certainly it’s a most dramatic ending for Mordo’s demonstration. But does it snap back automatically after it strikes a surface? Automation is not always the answer. Deliberate control would mean Mordo doesn’t have to waste time undoing unwanted automatic actions.

Critical systems must be extremely confident in their interpretations before automation is the right choice.

It might be that this particular gesture is a retraction signal, but how the Staff distinguishes this from a mid-combat strike is tricky. It would have to have sophisticated situational awareness to know the difference, and it doesn’t display this. Better backworlding would point at some subtle gestural signal from Mordo. A double-tightening of his grip, maybe. Or even a double-slight-release of his grip, since that’s something he’s quite unlikely to do in combat.

This is a broad pattern for designers to remember. Inline control signals should be simple-to-provide, but unlikely to occur in literal use. Imagine if the Winter Soldier’s Trigger Phrase wasn’t “Longing, rusted, 17, daybreak, furnace, 9, benign, homecoming, 1, freight car” but instead was the word “the.” He’d be berserking every few seconds. Unworkable. So, if you were designing the Staff’s retraction command gesture, you’d have to pick something he could remember and perform easily, and that would be difficult to accidentally provide.

If Mordo has the staff in the next film, I hope the control modes are clearer and of course well-designed.

Cyberspace: Newark Copyshop

The transition from Beijing to the Newark copyshop is more involved. After he travels around a bit, he realizes he needs to be looking back in Newark. He “rewinds” using a pull gesture and sees the copyshop’s pyramid. First there is a predominantly blue window that unfolds as if it were paper.

jm-35-copyshop-a-animated

And then the copyshop initial window expands. Like the Beijing hotel, this is a floor plan view, but unlike the hotel it stays two dimensional. It appears that cyberspace works like the current world wide web, with individual servers for each location that can choose what appearance to present to visitors.

Johnny again selects data records, but not with a voice command. The first transition is a window that not only expands but spins as it does so, and makes a strange jump at the end from the centre to the upper left.

jm-35-copyshop-c-animated

Once again Johnny uses the two-handed expansion gesture to see the table view of the records. Continue reading

Talking to a Puppet

As mentioned, Johnny in the last phone conversation in the van is not talking to the person he thinks he is. The film reveals Takahashi at his desk, using his hand as if he were a sock puppeteer—but there is no puppet. His desk is emitting a grid of green light to track the movement of his hand and arm.

jm-22-puppet-call-c

The Make It So chapter on gestural interfaces suggests Takahashi is using his hand to control the mouth movements of the avatar. I’d clarify this a bit. Lip synching by human animators is difficult even when not done in real time, and while it might be possible to control the upper lip with four fingers, one thumb is not enough to provide realistic motion of the lower lip. Continue reading

Jefferson Projection

SWHS-musicVP-01

When Imperial troopers intrude to search the house, one of the bullying officers takes interest in a device sitting on the dining table. It’s the size of a sewing machine, with a long handle along the top. It has a set of thumb toggles along the top, like old cassette tape recorder buttons.

Saun convinces the officer to sit down, stretches the thin script with a bunch of pointless fiddling of a volume slider and pantomimed delays, and at last fumbles the front of the device open. Hinged at the bottom like a drawbridge, it exposes a small black velvet display space. Understandably exasperated, the officer stands up to shout, “Will you get on with it?” Saun presses a button on the opened panel, and the searing chord of an electric guitar can be heard at once.

SWHS-musicVP-04

Inside the drawbridge-space a spot of pink light begins to glow, and mesmerized officer who, moments ago was bent on brute intimidation, but now spends the next five minutes and 23 seconds grinning dopily at the volumetric performance by Jefferson Starship.

During the performance, 6 lights link in a pattern in the upper right hand corner of the display. When the song finishes, the device goes silent. No other interactions are seen with it.

tappa

Many questions. Why is there a whole set of buttons to open the thing? Is this the only thing it can play? If not, how do you select another performance?Is it those unused buttons on the top? Why are the buttons unlabeled? Is Jefferson Starship immortal? How is it that they have only aged in the long, long time since this was recorded? Or was this volumetric recording somehow sent back in time?  Where is the button that Saun pressed to start the playback? If there was no button, and it was the entire front panel, why doesn’t it turn on and off while the officer taps (see above)? What do the little lights do other than distract? Why is the glow pink rather than Star-Wars-standard blue? Since volumetric projections are most often free-floating, why does this appear in a lunchbox? Since there already exists ubiquitous display screens, why would anyone haul this thing around? How does this officer keep his job?

Perhaps it’s best that these questions remain unanswered. For if anything were substantially different, we would risk losing this image, of the silhouette of the lead singer and his microphone. Humanity would be the poorer for it.

SWHS-musicVP-09

Avengers, assembly!

Avengers-lookatthis.png

When Coulson hands Tony a case file, it turns out to be an exciting kind of file. For carrying, it’s a large black slab. After Tony grabs it, he grabs the long edges and pulls in opposite directions. One part is a thin translucent screen that fits into an angled slot in the other part, in a laptop-like configuration, right down to a built-in keyboard.

The grip edge

The grip edge of the screen is thicker than the display, so it has a clear, physical affordance as to what part is meant to be gripped and how to pull it free from its casing, and simultaneously what end goes into the base. It’s simple and obvious. The ribbing on the grip unfortunately runs parallel to the direction of pull. It would make for a better grip and a better affordance if the grip was perpendicular to the direction of pull. Minor quibble.

I’d be worried about the ergonomics of an unadjustable display. I’d be worried about the display being easily unseated or dislodged. I’d also be worried about the strength of the join. Since there’s no give, enough force on the display might snap it clean off. But then again this is a world where “vibrium steel” exists, so material critiques may not be diegetically meaningful.

Login

Once he pulls the display from the base, the screen boops and animated amber arcs spin around the screen, signalling him to login via a rectangular panel on the right hand side of the screen. Tony puts his four fingers in the spot and drags down. A small white graphic confirms his biometrics. As a result, a WIMP display appears in grays and amber colors.

Avengers-asset-browser05

Briefing materials

One window on the left hand side shows a keypad, and he enters 1-8-5-4. The keypad disappears and a series of thumbnail images—portraits of members of the Avengers initiative—appear in its place. Pepper asks Tony, “What is all this?” Tony replies, saying, “This is, uh…” and in a quick gesture, places his ten fingertips on the screen at the portraits, and then throws his hands outward, off the display.

The portraits slide offscreen to become ceiling-height volumetric windows filled with rich media dossiers on Thor, Steve Rogers, and David Banner. There are videos, portraits, schematics, tables of data, cellular graphics, and maps. There’s a smaller display near the desktop where the “file” rests about the tesseract. (More on this bit in the next post.)

Briefing.gif

Insert standard complaint here about the eye strain that a translucent display causes, and the apology that yes, I understand it’s an effective and seemingly high-tech way to show actors and screens simultaneously. But I’d be remiss if I didn’t mention it.

The two-part login shows an understanding of multifactor authentication—a first in the survey, so props for that. Tony must provide something he “is”, i.e. his fingerprints, and something he knows, i.e. the passcode. Only then does the top secret information become available.

I have another standard grouse about the screen providing no affordances that content has an alternate view available, and that a secret gesture summons that view. I’d also ordinarily critique the displays for having nearly no visual hierarchy, i.e. no way for your eyes to begin making sense of it, and a lot of pointless-motion noise that pulls your attention in every which way.

But, this beat is about the wonder of the technology, the breadth of information SHIELD in its arsenal, and the surprise of familiar tech becoming epic, so I’m giving it a narrative pass.

Also, OK, Tony’s a universe-class hacker, so maybe he’s just knowledgeable/cocky enough to not need the affordances and turned them off. All that said, in my due diligence: Affordances still matter, people.

Abidjan Operation

Avengers_BartonCompromised

After Hawkeye is enthralled by Loki, agent Coulson has to call agent Romanoff in from the field, mid-mission. While he awaits her to extract herself from a situation, he idly glances at case file 242-56 which consists of a large video of Barton and Romanoff mid-combat, and overview profiles of the two agents. A legend in the upper right identifies this as STRIKE TEAM: DELTA, and a label at the top reads ABIDJAN OPERATION. There is some animated fuigetry on the periphery of the video, and some other fuigetry in windows that are occluded by the case file. bartoncompromised

Continue reading

Carrier Control

The second instantiation of videochat with the World Security Council that we see is  when Fury receives their order to bomb the site of the Chitauri portal. (Here’s the first.) He takes this call on the bridge, and rather than a custom hardware setup, this is a series of windows that overlay an ominous-red map of the world in an app called CARRIER CONTROL. These windows represent a built-in chat feature for discussing this very topic. There is some fuigetry on the periphery, but our focus is on these windows and the conversation happening through them.

Avengers-fury-secure-transmission01

In this version of the chat, we are assured that it is a SECURE TRANSMISSION by a legend across the top of each, but there is not the same level of assurance as in the videoconference room. If it’s still HOTP, Fury isn’t notified of it. There’s a tiny 01_AZ in the upper right of every screen, but it never changes and is the same for each participant. (An homage to Arizona? Lighter Andrew Zink? Cameraman Arthur Zajac?) Though this is a more desperate situation, you imagine that the need for security is no less dire. Having that same cypher key would be comforting if it is in fact a policy.

Different sizes of windows in the app seem to indicate a hierarchy, since the largest window is the fellow who does most of the talking in both conferences, and it does not change as others speak. Such an automated layout would spare Fury the hassle of having to manage multiple windows, though visually these look more like individual objects he’s meant to manipulate. Poor affordances.

dismiss

The only control we see is when Fury dismisses them, and to do this he just taps at the middle of the screen. The teleconference window is “push wiped” by a satellite view of New York City. Fine, he feels like punching them. But…

a) How does he actually select something in that interface without a tap?

b) A swipe would have been more meaningful, and in line with the gestural pidgin I identified in the gestural chapter of the book.

And of course, if this was the real world, you’d hope for better affordances for what can be done on this window across the board.

So though mostly effective, narratively, could use some polish.