Agent Ross’ remote piloting

Remote operation appears twice during Black Panther. This post describes the second, in which CIA Agent Ross remote-pilots the Talon in order to chase down cargo airships carrying Killmonger’s war supplies. The prior post describes the first, in which Shuri remotely drives an automobile.

In this sequence, Shuri equips Ross with kimoyo beads and a bone-conducting communication chip, and tells him that he must shoot down the cargo ships down before they cross beyond the Wakandan border. As soon as she tosses a remote-control kimoyo bead onto the Talon, Griot announces to Ross in the lab “Remote piloting system activated” and creates a piloting seat out of vibranium dust for him. Savvy watchers may wonder at this, since Okoye pilots the thing by meditation and Ross would have no meditation-pilot training, but Shuri explains to him, “I made it American style for you. Get in!” He does, grabs the sparkly black controls, and gets to business.

The most remarkable thing to me about the interface is how seamlessly the Talon can be piloted by vastly different controls. Meditation brain control? Can do. Joystick-and-throttle? Just as can do.

Now, generally, I have a beef with the notion of hyperindividualized UI tailoring—it prevents vital communication across a community of practice (read more about my critique of this goal here)—but in this case, there is zero time for Ross to learn a new interface. So sure, give him a control system with which he feels comfortable to handle this emergency. It makes him feel more at ease.

The mutable nature of the controls tells us that there is a robust interface layer that is interpreting whatever inputs the pilot supplies and applying them to the actuators in the Talon. More on this below. Spoiler: it’s Griot.

Too sparse HUD

The HUD presents a simple circle-in-a-triangle reticle that lights up red when a target is in sights. Otherwise it’s notably empty of augmentation. There’s no tunnel in the sky display to describe the ideal path, or proximity warnings about skyscrapers, or airspeed indicator, or altimeter, or…anything. This seems a glaring omission since we can be certain other “American-style” airships have such things. More on why this might be below, but spoiler: It’s Griot.

What do these controls do, exactly?

I take no joy in gotchas. That said…

  • When Ross launches the Talon, he does so by pulling the right joystick backward.
  • When he shoots down the first cargo ship over Birnin Zana, he pushes the same joystick forward as he pulls the trigger, firing energy weapons.

Why would the same control do both? It’s hard to believe it’s modal. Extradiegetically, this is probably an artifact of actor Martin Freeman’s just doing what feels dramatic, but for a real-world equivalent I would advise against having physical controls have wholly different modes on the same grip, lest we risk confusing pilots on mission-critical tasks. But spoiler…oh, you know where this is going.

It’s Griot

Diegetically, Shuri is flat-out wrong that Ross is an experienced pilot. But she also knew that it didn’t matter, because her lab has him covered anyway. Griot is an AI with a brain interface, and can read Ross’ intentions, handling all the difficult execution itself.

This would also explain the lack of better HUD augmentation. That absence seems especially egregious considering that the first cargo ship was flying over a crowded city at the time it was being targeted. If Ross had fired in the wrong place, the cargo ship might have crashed into a building, or down to the bustling city street, killing people. But, instead, Griot quietly, precisely targets the ship for him, to insure that it would crash safely in nearby water.

This would also explain how wildly different interfaces can control the Talon with similar efficacy.

An stained-glass image of William of Ockham. A modern blackletter caption reads, “It was always Griot.”

So, Occams-apology says, yep, it’s Griot.

An AI-wizard did it?

In the post about Shuri’s remote driving, I suggested that Griot was also helping her execute driving behind the scenes. This hearkens back to both the Iron HUD and Doctor Strange’s Cloak of Levitation. It could be that the MCU isn’t really worrying about the details of its enabling technologies, or that this is a brilliant model for our future relationship with technology. Let us feel like heroes, and let the AI manage all the details. I worry that I’m building myself into a wizard-did-it pattern, inserting AI for wizard. Maybe that’s worth another post all its own.

But there is one other thing about Ross’ interface worth noting.

The sonic overload

When the last of the cargo ships is nearly at the border, Ross reports to Shuri that he can’t chase it, because Killmonger-loyal dragon flyers have “got me trapped with some kind of cables.” She instructs him to, “Make an X with your arms!” He does. A wing-like display appears around him, confirming its readiness.

Then she shouts, “Now break it!” he does, and the Talon goes boom shaking off the enemy ships, allowing Ross to continue his pursuit.

First, what a great gesture for this function. Very ordinarily, Wakandans are piloting the Talon, and each of them would be deeply familiar with this gesture, and even prone to think of it when executing a hail Mary move like this.

Second, when an outsider needed to perform the action, why didn’t she just tell Griot to just do it? If there’s an interpretation layer in the system, why not just speak directly to that controller? It might be so the human knows how to do it themselves next time, but this is the last cargo ship he’s been tasked with chasing, and there’s little chance of his officially joining the Wakandan air force. The emergency will be over after this instance. Maybe Wakandans have a principle that they are first supposed to engage the humans before bringing in the machines, but that’s heavy conjecture.

Third, I have a beef about gestures—there’s often zero affordances to tell users what gestures they can do, and what effects those gestures will have. If Shuri was not there to answer Ross’ urgent question, would the mission have just…failed? Seems like a bad design.

How else could have known he could do this? If Griot is on board, Griot could have mentioned it. But avoiding the wizard-did-it solutions, some sort of context-aware display could detect that the ship is tethered to something, and display the gesture on the HUD for him. This violates the principle of letting the humans be the heroes, but would be a critical inclusion in any similar real-world system.

Any time we are faced with “intuitive” controls that don’t map 1:1 to the thing being controlled, we’re faced with similar problems. (We’ve seen the same problems in Sleep Dealer and Lost in Space (1998). Maybe that’s worth its own write-up.) Some controls won’t map to anything. More problematic is that there will be functions which don’t have controls. Designers can’t rely on having a human cavalry like Shuri there to save the day, and should take steps to find ways that the system can inform users of how to activate those functions.

Fit to purpose?

I’ve had to presume a lot about this interface. But if those things are correct, then, sure, this mostly makes it possible for Ross, a novice to piloting, to contribute something to the team mission, while upholding the directive that AI Cannot Be Heroes.

If Griot is not secretly driving, and that directive not really a thing, then the HUD needs more work, I can’t diegetically explain the controls, and they need to develop just-in-time suggestions to patch the gap of the mismatched interface. 


Black Georgia Matters

Each post in the Black Panther review is followed by actions that you can take to support black lives. As this critical special election is still coming up, this is a repeat of the last one, modified to reflect passed deadlines.

The state flag of Georgia, whose motto clearly violates the doctrine of separation of church and state.
Always on my mind, or at least until July 06.

Despite outrageous, anti-democratic voter suppression by the GOP, for the first time in 28 years, Georgia went blue for the presidential election, verified with two hand recounts. Credit to Stacey Abrams and her team’s years of effort to get out the Georgian—and particularly the powerful black Georgian—vote.

But the story doesn’t end there. Though the Biden/Harris ticket won the election, if the Senate stays majority red, Moscow Mitch McConnell will continue the infuriating obstructionism with which he held back Obama’s efforts in office for eight years. The Republicans will, as they have done before, ensure that nothing gets done.

To start to undo the damage the fascist and racist Trump administration has done, and maybe make some actual progress in the US, we need the Senate majority blue. Georgia is providing that opportunity. Neither of the wretched Republican incumbents got 50% of the vote, resulting in a special runoff election January 5, 2021. If these two seats go to the Democratic challengers, Warnock and Ossof, it will flip the Senate blue, and the nation can begin to seriously right the sinking ship that is America.

Photograph: Erik S Lesser/EPA

What can you do?

If you live in Georgia, vote blue, of course. You can check your registration status online. You can also help others vote. Important dates to remember, according to the Georgia website

  • 14 DEC Early voting begins
  • 05 JAN 2021 Final day of voting

Residents can also volunteer to become a canvasser for either of the campaigns, though it’s a tough thing to ask in the middle of the raging pandemic.

The rest of us (yes, even non-American readers) can contribute either to the campaigns directly using the links above, or to Stacey AbramsFair Fight campaign. From the campaign’s web site:

We promote fair elections in Georgia and around the country, encourage voter participation in elections, and educate voters about elections and their voting rights. Fair Fight brings awareness to the public on election reform, advocates for election reform at all levels, and engages in other voter education programs and communications.

We will continue moving the country into the anti-racist future regardless of the runoff, but we can make much, much more progress if we win this election. Please join the efforts as best you can even as you take care of yourself and your loved ones over the holidays. So very much depends on it.

Black Reparations Matter

This is timely, so I’m adding this on as well rather than waiting for the next post: A bill is in the house to set up a commission to examine the institution of slavery and its impact and make recommendations for reparations to Congress. If you are an American citizen, please consider sending a message to your congresspeople asking them to support the bill.

Image, uncredited, from the ACLU site. Please contact me if you know the artist.

On this ACLU site you will find a form and suggested wording to help you along.

Luke’s predictive HUD

When Luke is driving Kee and Theo to a boat on the coast, the car’s heads-up-display shows him the car’s speed with a translucent red number and speed gauge. There are also two broken, blurry gauges showing unknown information.

Suddenly the road becomes blocked by a flaming car rolled onto the road by a then unknown gang. In response, an IMPACT warning triangle zooms in several times to warn the driver of the danger, accompanied by a persistent dinging sound.

childrenofmen-impact-08

It commands attention effectively

Continue reading

Iron Man HUD: 2nd-person view

In the prior post we looked at the HUD display from Tony’s point of view. In this post we dive deeper into the 2nd-person view, which turns out to be not what it seems.

The HUD itself displays a number of core capabilities across the Iron Man movies prior to its appearance in The Avengers. Cataloguing these capabilities lets us understand (or backworld) how he interacts with the HUD, equipping us to look for its common patterns and possible conflicts. In the first-person view, we saw it looked almost entirely like a rich agentive display, but with little interaction. But then there’s this gorgeous 2nd-person view.

When in the first film Tony first puts the faceplate on and says to JARVIS, “Engage heads-up display”… IronMan1_HUD00 …we see things from a narrative-conceit, 2nd-person perspective, as if the helmet were huge and we are inside the cavernous space with him, seeing only Tony’s face and the augmented reality interface elements. IronMan1_HUD07 You might be thinking, “Of course it’s a narrative conceit. It’s not real. It’s in a movie.” But what I mean by that is that even in the diegesis, the Marvel Cinematic World, this is not something that could be seen. Let’s move through the reasons why. Continue reading

The Bubbleship Cockpit

image01 Jack’s main vehicle in the post-war Earth is the Bubbleship craft. It is a two seat combination of helicopter and light jet. The center joystick controls most flight controls, while a left-hand throttle takes the place of a helicopter’s thrust selector. A series of switches above Jack’s seat provide basic power and start-up commands to the Bubbleship’s systems. image05 Jack first provides voice authentication to the Bubbleship (the same code used to confirm his identity to the Drones), then he moves to activate the switches above his head. Continue reading

Prometheus’ Flight instrument panels

There are a great many interfaces seen on the bridge of the Prometheus, and like most flight instrument panels in sci-fi, they are largely about storytelling and less about use.

Prometheus-100

The captain of the Prometheus is also a pilot, and has a captain’s chair with a heads-up display. This HUD has with real-time wireframe displays of the spaceship in plan view, presumably for glanceable damage feedback.

Prometheus-028

He also can stand afore at a waist-high panel that overlooks the ship’s view ports. This panel has a main screen in the center, grouped arrays of backlit keys to either side, a few blinking components, and an array of red and blue lit buttons above. We only see Captain Janek touch this panel once, and do not see the effects.

Prometheus-097

Navigator Chance’s instrument panel below consists of four 4:3 displays with inscrutable moving graphs and charts, one very wide display showing a topographic scan of terrain, one dim panel, two backlit reticles, and a handful of lit switches and buttons. Yellow lines surround most dials and group clusters of controls. When Chance “switches to manual”, he flips the lit switches from right to left (nicely accomplishable with a single wave of the hand) and the switches lights light up to confirm the change of state. This state would also be visible from a distance, useful for all crew within line of sight. Presumably, this is a dangerous state for the ship to be in, though, so some greater emphasis might be warranted: either a blinking warning, or a audio feedback, or possibly both.

Prometheus-094Prometheus-102

Captain Janek has a joystick control for manual landing control. It has a line of light at the top rear-facing part, but its purpose is not apparent. The degree of differentiation in the controls is great, and they seem to be clustered well.

Prometheus-104

A few contextless flight screens are shown. One for the scientist known only as Ford features 3D charts, views of spinning spaceships, and other inscrutable graphs, all of which are moving.

Prometheus-095

A contextless view shows the points of metal detected overlaid on a live view from the ship.

Prometheus-101

There is a weather screen as well that shows air density. Nearby there’s a push control, which Chance presses and keeps held down when he says, “Boss, we’ve got an incoming storm front. Silica and lots of static. This is not good.” Thought we never see the control, it’s curious how such a thing could work. Would it be an entire-ship intercom, or did Chance somehow specify Janek as a recipient with a single button?

Prometheus-078

Later we see Chance press a single button that illuminates red, after which the screens nearby change to read “COLLISION IMMINENT,” and an all-ship prerecorded announcement begins to repeat its evacuation countdown.

Prometheus-309

This is single button is perhaps the most egregious of the flight controls. As Janek says to Shaw late in the film, “This is not a warship.” If that’s the case, why would Chance have a single control that automatically knows to turn all screens red with the Big Label and provide a countdown? And why should the crew ever have to turn this switch on? Isn’t a collision one of the most serious things that could happen to the ship? Shouldn’t it be hard to, you know, turn off?