3D interfaces: Observations and Reflections

1 Jan 2024 by scifihughf

So what is a 3D interface?

These examples, although fictional, demonstrate that “3D” can be used in different ways.

In Jurassic Park and Hackers, 3D graphics are used to create a richer display with more information density, though it is not photorealistic. The Jurassic Park file browser is primarily a symbolic 2D representation of the file system hierarchy, projected onto a perspective ground plane to make more elements visible at once. The third dimension is used to indicate the number of sub elements or their size. In Hackers, the City of Text towers most likely represent the actual contents of each physical disk drive in the corresponding real world location, and the pulses and colors indicate levels of activity or threat.

The Corridor in Disclosure, and its VirtuGood 6500 close copy in Community, instead create a more photorealistic virtual world. The file system becomes a building or landscape, and the users are embodied within the virtual world as an avatar. Like the pre-computer memory palace, this should take advantage of the human ability to remember and navigate our way around. But The Corridor blows it by putting all the files within one room, and representing them as sheets of paper within identical filing cabinets. Walking through the 3D architecture becomes a pretty but time wasting diversion.

I’m personally disappointed not to find any true computer memory palaces, whether fictional or real. As mentioned in the introduction, an essential characteristic of the memory palace is that each item be stored in a unique location, visually distinct from any other. None of the 3D file systems I’ve been able to find do this, instead using generic icons throughout. Computers are actually quite good at creating almost infinite variations in appearance, e.g. fractals in 2D and various CGI landscapes and underwater environments in 3D. A computer memory palace would at least be more interesting to look at.

Where are they today?

Since the 1990s the 3D file browser has seemingly faded away, both in reality and in film/TV. Let’s (briefly) think about why.

The SGI 3D file browser shown in Jurassic Park was not the only one to be released as a real piece of software. Although personal computers could easily run such a 3D file browser by the year 2000, and mobile phones a few years later, the systems we actually use have remained two dimensional. The only widespread use of 3D spatial organisation that I’m aware of is the Apple Time Machine backup software, which uses distance from the viewer to represent increasing age. It’s a linear sequence of 2D desktops rather than allowing true three dimensional movement in any direction. Even native 3D systems like the Oculus Quest present the user a 2D GUI wrapped around the user in a cylinder.

We don’t have our files arranged into 3D buildings or worlds, but there have been other developments since the first 2D file browsers. Keyword search is now built into most GUI desktops. Photo collections can be viewed by timeline, or by geographical location; and music collections arranged by genre, artist, or album. So one likely reason why we don’t have real world 3D file browsers is that in themselves they don’t provide enough of an advantage over the existing 2D GUIs to make changing worthwhile.

User interfaces in film and TV are not constrained by reality or practicality so their absence must be due to other reasons. Sometimes real world interface trends affect what we see on the screen, for instance the replacement of command line interfaces by graphical, but for file browsing we’re still using the 2D GUI browsers from the 1990s. And it’s not because of technical difficulty or expense, because we’ve seen that 1990 feature-film 3D effects can now be created in the budget of a sitcom episode.
An example is the 2008 film Iron Man, already mentioned for using a 3D trashcan within Tony Stark’s CAD software system. Later in the film, Pepper needs to copy some files from the corporate PC of evil executive Obadiah Stane. As in the earlier films covered in this review, Stark Industries is portrayed as an advanced technology company so this PC also has a custom GUI created for the film. Here though there is only a very slight use of 3D to arrange flat file icons in order, otherwise it closely resembles existing 2D desktops. The filmmakers could have inserted a 3D file browser with perhaps volumetric projection to match Tony’s 3D CAD system but chose not to.

Pepper selects a folder in the text list at left and it is also highlighted in the graphical list of overlaid translucent icons at right. *Iron Man* (2008)

Copying computer files (or more dramatically “the data”) still happens in science fiction or near future film settings, but also has become more common in everyday life with the spread of personal computers and now smartphones worldwide. In my opinion, this is the most likely reason why we don’t see 3D and VR file browsers any more: we the audience know how to copy files and search for them, and won’t be impressed by attempts to make it “high tech” with fanciful user interfaces. File systems and browsers have become, well, boring. So we can look back on these cinematic dalliances with 3D file management fondly, but recognize it as a thing we tried for a while, and learned from, but eventually put down.

Vibranium-based Cape Shields

16 Mar 2021 by Christopher Noessel

Editor’s Note: Today’s guest post is penned by Lonny Brooks. Be sure and read his introduction post if you missed it when it was published.

The Black Panther film represents one of the most ubiquitous statements of Afrofuturist fashion and fashionable digital wearables to celebrate the Africana and Black imagination. The wearable criteria under Director Ryan Cooglar’s lead and that of the formidable talent of costume designer Ruth E. Carter took into account African tribal symbolism. The adinkra symbol, for “cooperation,” emblazoned across W’Kabi’s (played by Daniel Kaluuya) blanket embodies the role of the Border tribe where they lived in a small village tucked into the mountainous borderlands of Wakanda, disguised as farmers and hunters.

Beautiful interaction

Chris’ blog looks at the interactions with speculative technology, and here the interactions are marvelously subtle. They do not have buttons or levers, which might give away their true nature. To activate them, a user does what would come naturally, which is to hold the fabric before them, like a shield. (There might be a mental command as well, but of course we can’t perceive that.) The shield-like gesture activates the shield technology. It’s quick. It fits the material of the technology. You barely even have to be trained to use it. We never see the use case for when a wearer is incapacitated and can’t lift the cape into position, but there’s enough evidence in the rest of the film to expect it might act like Dr. Strange’s cape and activate its shield automatically.

But, for me, the Capes are more powerful not as models of interaction, but for what they symbolize.

The Dual Role of the Capes

The role of the Border Tribe is to create the illusion of agrarian ruggedness as a deception for outsiders that only tells of a placid, developing nation rather than the secret technologically advanced splendor of Wakanda’s lands. The Border Tribe is the keeper of Wakanda’s cloaking technology that hides the vast utopian advancement of Wakandan advantage.

The Border Tribe’s role is built into the fabric of their illustrious and enviably fashionable capes. The adinkra symbol of cooperation embedded into the cape reveals, by the final scenes of the Black Panther film, how the Border Tribe defenders wield their capes into a force field wall of energy to repel enemies.

Ironically we only see them at their most effective when Wakanda is undergoing a civil war between those loyal to Kilmonger who is determined to avenge his father’s murder and his own erasure from Wakandan collective memory, and those supporting King T’Challa. Whereas each Black Panther King has selected to keep Wakanda’s presence hidden literally under the cooperative shields of the Border Tribe, Kilmonger—an Oakland native and a potential heir to Wakandan monarchy—was orphaned and left in the U.S.

If this sounds familiar, consider the film as a grand allusion to the millions of Africans kidnapped and ripped from their tribal lineages and taken across the Atlantic as slaves. Their cultural heritage was purposefully erased, languages and tribal customs, memories lost to the colonial thirst for their unpaid and forced labor.

Kilmonger represents the Black Diaspora, former descendants of African homelands similarly deprived of their birthrights. Kilmonger wants the Black Diaspora to rise up in global rebellion with the assistance of Wakandan technical superiority. In opposition, King T’Challa aspires for a less vengeful solution. He wantsWakanda to come out to the world, and lead by example. We can empathize with both. T’Challa’s plan is fueled by virtue. Kilmonger’s is fueled by justice—redeploy these shields to protect Black people against the onslaught of ongoing police and state violence.

W. E. B. Du Bois in 1918
(image in the public domain)

Double Consciousness and the Big Metaphor

The cape shields powered by the precious secret meteorite called Vibranium embodies what the scholar W.E.B. Dubois referred to as a double consciousness, where members of the Black Diaspora inhabit two selves.

Their own identity as individuals
The external perception of themselves as a members of an oppressed people incessantly facing potential erasure and brutality.

The cape shields and their cloaking technology cover the secret utopic algorithms that power Wakanda, while playing on the petty stereotypes of African nations as less-advanced collectives.

The final battle scene symbolizes this grand debate—between Kilmonger’s claims on Wakanda and assertion of Africana power, and King T’Challa’s more cooperative and, indeed, compliant approach working with the CIA. Recall that in its subterfuge and cloaking tactics, the CIA has undermined and toppled numerous freely-elected African and Latin American governments for decades. In this final showdown, we see W’Kabi’s cloaked soldiers run down the hill towards King T’Challa and stop to raise their shields cooperatively into defensive formation to prevent King T’Challa’s advance. King T’Challa jumps over the shields and the force of his movement causes the soldier’s shields to bounce away while simultaneously revealing their potent energy.

The flowing blue capes of the Border Tribe are deceptively enticing, while holding the key to Wakanda’s survival as metaphors for cloaking their entire civilization from being attacked, plundered, and erased. Wakanda and these capes represent an alternative history: What if African peoples had not experienced colonization or undergone the brutal Middle Passage to the Americas? What if the prosperous Black Greenwood neighborhood of Tulsa, Oklahoma had developed cape shield technology to defend themselves against a genocidal white mob in 1921? Or if the Black Panther Party had harnessed the power of invisible cloaking technology as part of their black beret ensemble?

Gallery Images: World Building with the Afrofuturist Podcast—Afro-Rithms From The Future game, Neuehouse, Hollywood, May 22, 2019 [Co-Game Designers, Eli Kosminsky and Lonny Avi Brooks, Afro-Rithms Librarian; Co-Game Designer and Seer Ahmed Best]

In the forecasting imagination game, Afro-Rithms From The Future, and the game event we played in 2019 in Los Angeles based on the future universe we created, we generated the question:

What would be an article of fashion that would give you more Black Feminist leadership and more social justice?

One participant responded with: “I was thinking of the notion of the invisibility cloak but also to have it be reversed. It could make you invisible and also more visible, amplifying what you normally” have as strengths and recognizing their value. Or as another player, states “what about a bodysuit that protects you from any kind of harm” or as the game facilitator adds “how about a bodysuit that repels emotional damage?!” In our final analysis, the cape shields have steadfastly protected Wakanda against the emotional trauma of colonization and partial erasure.

In this way the cape shields guard against emotional damage as well. Imagine how it might feel to wear a fashionable cloak that displays images of your ancestral, ethnic, and gender memories reminding you of your inherent lovability as multi-dimensional human being—and that can technologically protect you and those you love as well.

Black Lives Matter

Chris: Each post in the Black Panther review is followed by actions that support black lives.

To thank Lonny for his guest post, I offered to donate money in his name to the charity of his choice. He has selected Museum of Children’s Arts in Oakland. The mission of MOCHA is to ensure that the arts are a fundamental part of our community and to create opportunities for all children to experience the arts to develop creativity, promote a sense of belonging, and to realize their potential.

And, since it’s important to show the receipts, the receipt:

Thank you, Lonny, for helping to celebrate Black Panther and your continued excellent work in speculative futures and Afrofuturism. Wakanda forever!

Vibranium sand tables

21 Sep 2020 by Christopher Noessel

There are a number of vibranium sand tables seen in Black Panther.

The horseshoe-shaped shelf in which Okoye sits as she pilots the Royal Talon. (We never see it activated.)
The small sand table in the center of the Royal Talon.
The big sand tables in Shuri’s lab.

You can see the Royal Talon one in the post about piloting that craft. The other two are described below.

All of these build on the given that vibranium is a very powerful substance and that Wakanda’s scientists have managed to gain a very, very sophisticated control over it.

In the Talon

This table is about a meter square, and raised off the floor around knee-height. As Okoye and T’Challa approach the traffickers in the Sambisa Forest, T’Challa approaches the table and it springs to life, showing him real-time model of the traffickers’ vehicle train. T’Challa picks up the model of the small transport truck and with a finger, wipes off its roof, revealing that there are over a dozen people huddled within. One of the figures glows amber. (It’s Nakia.) He places the truck back into the display, and the display collapses back to inert sand.

A quick critique of this interaction. The sand highlights Nakia for T’Challa, but why did it wait for him to find her truck and wipe off the top of it to look inside? It knew his goals (find Nakia), can clearly conduct a scan into the vehicle, and understood the context (she’s in one of those trucks), it should not wait for him to pick up each car and scrape off its roof to check and see which one she was in. The interface should have drawn his attention to the truck it knew she was in. This is a “stoic guru” mistake that I’ve critiqued before. You know, the computer knows all, but only tells you when you ask it. It is much more sensible for the transport truck to be glowing from the moment the table goes live, as in the comp below.

Designers: Don’t wait for users to ask just the the right thing at the right time.

Otherwise, this is a good high-tech use of the sand table for the more common meaning of “sand table,” which is a 3-dimensional surface for understanding a theatre of conflict. It doesn’t really help him run through scenarios, testing various tactics, but T’Challa is a warrior king, he can do all that in his head.

The interaction also nicely blurs the line between display and gestural interactive tool, in the same way that the Prometheus astrometrics display did. Like that other example, it would be useful for the display to distinguish when it is representing reality, and when the display is being interrupted or modified. Also, T’Challa is nice enough to put the truck back where it “belongs,” but a design would also need to handle how to respond when T’Challa put the truck back in the wrong place, or, say, crushed the truck model with his hand in fury.

In *Prometheus* it was an Earth, not a truck, but still focused on Africa.

Shuri’s lab

The largest table we see in the movie is in Shuri’s lab. After Black Panther challenges Killmonger and engages in battle outside the capital city, Shuri, Nakia, and Agent Ross rush down to the lab. As they approach an edge-lit hexagonal table, the vibranium sand lowers to reveal 3D-printed armor and weaponry for Shuri and Nakia to join the fight. (Though it’s not like modern 3D printing, these are powered weapons and kimoyo beads, items with very sophisticated functionality.)

Shuri outfits Ross with kimoyo beads from the print and takes off to join the fight. In the lab, the table creates a seat for Ross to remote-pilot the Royal Talon. Up on the flight deck, Shuri throws a control bead onto the Talon, and an AI in the lab named Griot announces to Agent Ross, “Remote piloting system activated.” (Hey, Trevor Noah, we hear you there!)

Around the seat, a volumetric projection of the Talon appears around him, including a 360° display just beyond the windshield that gives him a very immersive remote flying experience. We hear Shuri’s voice explain to Ross “I made it American Style for you. Get in!”

Ross sits down, grabs joystick controls, and begins remote-chasing down the cargo ships that are carrying munitions to Killmonger’s War Dogs around the world. (The piloting controls and HUD for Ross are a separate issue, and will be handled in their own post.)

The moment that Ross pilots the Talon through the last cargo ship, the volumetric projection disappears and the piloting seat returns to sand, ungraciously plopping Ross down the floor level of the lab.

It is in this shot that we realize that the dark tiles of the lab’s floor are all recessed vibranium sand tables. I can count seven in the shot. So the lab is full of them.

Display material

Let’s talk for a bit about the display choices. Vibranium can change to display any color and a shape down to a fine level of detail. See the screen cap below for an example of perfectly lifelike (if scaled) representation.

This is a vibranium-powered volumetric display.
It raises the gaze matching issues we’ve seen before.

So why would it be designed so that in most cases, the display is sparkly and black like black tourmaline? Wouldn’t the truck that T’Challa picks up be most useful if it was photographically rendered? Wouldn’t the remote piloting chair be more comfortable if it had pleather- and silicone-like surfaces?

Extradiegetically, I understand the reason is because art direction. We want Wakandan tech to be visibly different than other tech in the MCU, and having it look like vibranium dust ties it back to that key plot element.

But, per the stance of this blog, I try to look for a diegetic reason. It might be a deliberate reminder of the resource on which their technological fortunes are built. And as the Okoye VP above shows, they aren’t purists about it. When detail is needed, it’s included. So perhaps this is it. That implies a great deal of sophistication on the part of the displays to know when photorealism is needed and when it is not, but the presence of Griot there tells us that they have something approaching general AI.

Missing interactions

So, just like I had to do for the Royal Talon, I have to throw my hands up about reviewing the interactions with the sand tables, because we don’t see the interactions that would give these results.

How were the mission goals communicated to the Royal Talon table? Is it programmed to activate when someone approaches it, or did T’Challa issue a mental command? How did Shuri specify those weapons and that armor? What did she do to make the ship “American style” for Ross? Is that a template? Was it Griot’s interpretation of her intention? Why did the remote piloting seat vanish the moment the mission was complete? Was this something Shuri set up in advance, or Griot’s way of telling Agent Ross to GTFO for his own safety? How does someone in the lab instruct a floor tile to leap up and become a table and do stuff? It’s almost certainly via mental commands through the kimoyo beads, but that’s conjecture. The film really provides little evidence.

On the one hand, this is appropriate for us mere non-Wakandans observing the most technologically advanced society on earth. Much of it would feel like inexplicable magic to us.

On the other, sci-fi routinely introduces us to advanced technologies, and doesn’t always eschew the explanatory interactions, so the absence is notable here. It’s magic.

Black Lives Matter

Each post in the Black Panther review is followed by actions that you can take to support black lives.

In the last post we grieved Chadwick Boseman’s passing. This week we’re grieving the loss of Ruth Bader Ginsburg. May her memory be a blessing. With her loss, the GOP is ratcheting up its outrageous hypocrisy by reversing a precedent that they themselves established when Obama was president. The ~~“Moscow Mitch Rule”~~ (oh, oops, sorry) “McConnell Rule” was that new Justices should not be appointed within a year of a general election, so the people’s voice can be taken into account. Of course, the bastards are just ignoring that now and trying to ram through one of their own before election day. This Justice will certainly be a conservative, and we know with this administration that means reactionary, loyal to tiny-hand Twittler, and racist as a Jim Crow law.

There are a few arrows in citizen’s quivers to stop this. One is to convince at least 4 Republican Senators to reject this outright hypocrisy, put country over party, and adhere to the McConnell rule.

To help put pressure where it might work, you can leave voicemails with Republican Senators who may be mulling whether to put country over party. Those 6 Senators’ names and numbers are below. Here’s a script for your message:

Hello, my name is ______. In 2016, Mitch McConnell created the principle of not confirming a Supreme Court Justice in an election year until after the next inauguration. For the legitimacy of the Court in the eyes of the people, I’m asking Senator ________ to uphold that principle by refusing to confirm a new Justice until after a new President is installed. Thank you.
—You, hopefully

Lisa Murkowski, Alaska; (202) 224-6665
Mitt Romney, Utah: (202) 224-5251
Susan Collins, Maine: (202) 224-2523
Martha McSally, Arizona: (202) 224-2235
Cory Gardner, Colorado: (202) 224-5941
Chuck Grassley, Iowa: (202) 224-3744

I’ve made my calls and left my messages. Can you do the same to stop the hypocritical Trumpian power grab that would tip the Supreme Court for generations?

UPDATE: Nevermind. Romney caved.

Panther Suit 2.0

24 Aug 2020 by Christopher Noessel

The suit that the Black Panther wears is critical to success. At the beginning of the movie, this is “just” a skintight bulletproof suit with homages to its namesake. But, after T’Challa is enthroned, Shuri takes him to her lab and outfits him with a new one with some nifty new features. This write-up is about Shuri’s 2.0 Panther Suit.

Authorizing

At the demonstration of the new suit, Shuri first takes a moment to hold up a bracelet of black Kimoyo beads (more on these in a later post) to his neck. With a bubbly computer sound, the glyphs on the beads begin to glow vibranium-purple, projecting two particular symbols on his neck. (The one that looks kind of like a reflective A, and the other that looks like a ligature of a T and a U.)

This is done without explanation, so we have to make some assumptions here, which is always shaky ground for critique.

I think she’s authorizing him to use the suit. At first I thought the interaction was her “pairing” him with the suit, but I can’t imagine that the bead would need to project something onto his skin to read his identity or DNA. So my updated guess is this is a dermal mark that, like the Wakandan tattoos, the suit will check for with a “intra-skin scan,” like the HAN/BAN concepts from the early aughts. This would enable her to authorize many people, which is, perhaps, not as secure.

This interpretation is complicated by Killmonger’s wearing one of the other Black Panther suits when he usurps T’Challa. Shuri had fled with Queen Romonda to the Jibari stronghold, so Shuri couldn’t have authorized him. Maybe some lab tech who stayed behind? If there was some hint of what’s supposed to be happening here we would have more grounds to evaluate this interaction.

There might be some hint if there was an online reference to these particular symbols, but they are not part of the Wakandan typeface, or the Andinkra symbols, or the Nsibidi symbols that are seen elsewhere in the film. (I have emails out to the creator of the above image to see if I can learn more there. Will update if I get a response.)

Activation

When she finishes whatever the bead did, she says, “Now tell it to go on.” T’Challa looks at it intensely, and the suit spreads from the “teeth” in the necklace with an insectoid computer sound, over the course of about 6 seconds.

We see him activate the suit several more times over the course of the movie, but learn nothing new about activation beyond this. How does he mentally tell it to turn it on? I presume it’s the same mental skill he’s built up across his lifetime with kimoyo beads, but it’s not made explicit in the movie.

A fun detail is that while the suit activates in 6 seconds in the lab—far too slow for action in the field considering Shuri’s sardonic critique of the old suit (“People are shooting at me! Wait! Let me put on my helmet!”)—when T’Challa uses it in Korea, it happens in under 3. Shuri must have slowed it down to be more intelligible and impressive in the lab.

Another nifty detail that is seen but not discussed is that the nanites will also shred any clothes being worn at the time of transformation, as seen at the beginning of the chase sequence outside the casino and when Killmonger is threatened by the Dora Milaje.

Hopefully they weren’t royal…oh. Oh well?

Deactivation

T’Challa thinks the helmet off a lot over the course of the movie, even in some circumstances where I am not sure it was wise. We don’t see the mechanism. I expect it’s akin to kimoyo communication, again. He thinks it, and it’s done. (n.b. “It’s mental” is about as satisfying from a designer’s critique as “a wizard did it”, because it’s almost like a free pass, but *sigh* perfectly justifiable given precedent in the movie.)

Kinetic storage & release

At the demonstration in her lab, Shuri tells T’Challa to, “Strike it.” He performs a turning kick to the mannequin’s ribcage and it goes flying. When she fetches it from across the lab, he marvels at the purple light emanating from Nsibidi symbols that fill channels in the suit where his strike made contact. She explains “The nanites have absorbed the kinetic energy. They hold it in place for redistribution.”

He then strikes it again in the same spot, and the nanites release the energy, knocking him back across the lab, like all those nanites had become a million microscopic bigclaw snapping shrimp all acting in explosive concert. Cool as it is, this is my main critique of the suit.

First, the good. As a point of illustration of how cool their mastery of tech is, and how it works, this is pretty sweet. Even the choice of purple is smart because it is a hard color to match in older chemical film processes, and can only happen well in a modern, digital film. So extradiegetically, the color is new and showing off a bit.

Tactically though, I have to note that it broadcasts his threat level to his adversaries. Learning this might take a couple of beatings, but word would get around. Faithful readers will know we’ve looked at aposematic signaling before, but those kinds of markings are permanent. The suit changes as he gets technologically beefier. Wouldn’t people just avoid him when he was more glowy, or throw something heavy at him to force him to expend it, and then attack when he was weaker? More tactical I think to hold those cards close to the chest, and hide the glow.

Now it is quite useful for him to know the level of charge. Maybe some tactile feedback like a warmth or or a vibration at the medial edge of his wrists. Cinegenics win for actual movie-making of course, but designers take note. What looks cool is not always smart design.

Not really a question for me: Can he control how much he releases? If he’s trying to just knock someone out, it would be crappy if he accidentally killed them, or expected to knock out the big bad with a punch, only to find it just tickled him like a joy buzzer. But if he already knows how to mentally activate the suit, I’m sure he has the skill down to mentally clench a bit to control the output. Wizards.

Regarding Shuri’s description, I think she’s dumbing things down for her brother. If the suit actually absorbed the kinetic energy, the suit would not have moved when he kicked it. (Right?) But let’s presume if she were talking to someone with more science background, she would have been more specific to say, “absorbed some of the kinetic energy.”

Explosive release

When the suit has absorbed enough kinetic energy, T’Challa can release it all at once as a concussive blast. He punches the ground to trigger it, but it’s not clear how he signals to the suit that he wants to blast everyone around him back rather than, say, create a crater, but again, I think we can assume it’s another mental command. Wizards.

Claws

To activate the suit’s claws, T’Challa quickly extends curved fingers and holds them there, and they pop out.

This gesture is awesome, and completely fit for purpose. Shaping the fingers like claws make claws. It’s also when fingers are best positioned to withstand the raking motion. The second of hold ensures it’s not accidental activation. Easy to convey, easy to remember, easy to intuit. Kids playing Black Panther on the sidewalk would probably do the same without even seeing the movie.

We have an unanswered question about how those claws retract. Certainly the suit is smart enough to retract automatically so he doesn’t damage himself. Probably more mental commands, but whatever. I wouldn’t change a thing here.

Black Lives Matter

Each post in the Black Panther review is followed by actions that you can take to support black lives. I had something else planned for this post, but just before publication another infuriating incident has happened.

While the GOP rallies to the cause of the racist-in-chief in Charlotte, right thinking people are taking to the streets in Kenosha, Wisconsin, to protest the unjust shooting of a black man, Jacob Blake. The video is hard to watch. Watch it. It’s especially tragic, especially infuriating, because Kenosha had gone through “police reform” initiatives in 2014 meant to prevent exactly this sort of thing. It didn’t prevent this sort of thing. As a friend of mine says, it’s almost enough to make you an abolitionist.

Information is still coming in as to what happened, but here’s the narrative we understand right now: It seems that Blake had pulled over his car to stop a fight in progress. When the police arrived, he figured they had control of the situation, and he walked back to his car to leave. That’s when officers shot him in the back multiple times, while his family—who were still waiting for him in the car—watched. He’s out of surgery and stable, but rather than some big-picture to-do tonight, please donate to support his family. They have witnessed unconscionable trauma.

Several fundraisers posted to support Blake’s family have been taken down by GoFundMe for being fake, but “Justice for Jacob Blake” remains active as of Monday evening. Please donate.

The Dark Dimension mode (5 of 5)

19 Sep 2017 by Christopher Noessel

We see a completely new mode for the Eye in the Dark Dimension. With a flourish of his right hand over his left forearm, a band of green lines begin orbiting his forearm just below his wrist. (Another orbits just below his elbow, just off-camera in the animated gif.) The band signals that Strange has set this point in time as a “save point,” like in a video game. From that point forward, when he dies, time resets and he is returned here, alive and well, though he and anyone else in the loop is aware that it happened.

In the scene he’s confronting a hostile god-like creature on its own mystical turf, so he dies a lot.

An interesting moment happens when Strange is hopping from the blue-ringed planetoid to the one close to the giant Dormammu face. He glances down at his wrist, making sure that his savepoint was set. It’s a nice tell, letting us know that Strange is a nervous about facing the giant, Galactus-sized primordial evil that is Dormammu. This nervousness ties right into the analysis of this display. If we changed the design, we could put him more at ease when using this life-critical interface.

Initiating gesture

The initiating gesture doesn’t read as “set a savepoint.” This doesn’t show itself as a problem in this scene, but if the gesture did have some sort of semantic meaning, it would make it easier for Strange to recall and perform correctly. Maybe if his wrist twist transitioned from moving splayed fingers to his pointing with his index finger to his wrist…ok, that’s a little too on the nose, so maybe…toward the ground, it would help symbolize the here & now that is the savepoint. It would be easier for Strange to recall and feel assured that he’d done the right thing.

I have questions about the extents of the time loop effect. Is it the whole Dark Dimension? Is it also Earth? Is it the Universe? Is it just a sphere, like the other modes of the Eye? How does he set these? There’s not enough information in the movie to backworld this, but unless the answer is “it affects everything” there seems to be some variables missing in the initiating gesture.

Setpoint-active signal

But where the initiating gesture doesn’t appear to be a problem in the scene, the wrist-glance indicates that the display is. Note that, other than being on the left forearm instead of the right, the bands look identical to the ones in the Tibet and Hong Kong modes. (Compare the Tibet screenshot below.) If Strange is relying on the display to ensure that his savepoint was set, having it look identical is not as helpful as it would be if the visual was unique. “Wait,” he might think, “Am I in the right mode, here?”

In a redesign, I would select an animated display that was not a loop, but an indication that time was passing. It can’t be as literal as a clock of course. But something that used animation to suggest time was progressing linearly from a point. Maybe something like the binary clock from Mission to Mars (see below), rendered in the graphic language of the Eye. Maybe make it base-3 to seem not so technological.

Seeing a display that is still, on invocation—that becomes animated upon initialization—would mean that all he has to do is glance to confirm the unique display is in motion. “Yes, it’s working. I’m in the Groundhog Day mode, and the savepoint is set.”

The Hong Kong Mode (4 of 5)

12 Sep 2017 by Christopher Noessel

In the prior three posts, I’ve discussed the goods-and-bads of the Eye of Agamotto in the Tibet mode. (I thought I could squeeze the Hong Kong and the Dark Dimension modes into one post, but turns out this one was just too long. keep reading. You’ll see.) In this post we examine a mode that looks like the Tibet mode, but is actually quite different.

Hong Kong mode

Near the film’s climax, Strange uses the Eye to reverse Kaecilius’ destruction of the Hong Kong Sanctum Sanctorum (and much of the surrounding cityscape). In this scene, Kaecilius leaps at Strange, and Strange “freezes” Kaecilius in midair with the saucer. It’s done more quickly, but similarly to how he “freezes” the apple into a controlled-time mode in Tibet.

But then we see something different, and it complicates everything. As Strange twists the saucer counterclockwise, the cityscape around him—not just Kaecilius—begins to reverse slowly. (And unlike in Tibet, the saucer keeps spinning clockwise underneath his hand.) Then the rate of reversal accelerates, and even continues in its reversal after Strange drops his gesture and engages in a fight with Kaecilius, who somehow escapes the reversing time stream to join Strange and Mordo in the “observer” time stream.

So in this mode, the saucer is working much more like a shuttle wheel with no snap-back feature.

A shuttle wheel, as you’ll recall from the first post, doesn’t specify an absolute value along a range like a jog dial does. A shuttle wheel indicates a direction and rate of change. A little to the left is slow reverse. Far to the left is fast reverse. Nearly all of the shuttle wheels we use in the real world have snap-back features, because if you were just going to leave it reversing and pay attention to something else, you might as well use another control to get to the absolute beginning, like a jog dial. But, since Strange is scrubbing an endless “video stream,” (that is, time), and he can pull people and things out of the manipulated-stream and into the observer-stream and do stuff, not having a snap-back makes sense.

For the Tibet mode I argued for a chapter ring to provide some context and information about the range of values he’s scrubbing. So for shuttling along the past in the Hong Kong mode, I don’t think a chapter ring or content overview makes sense, but it would help to know the following.

The rate of change
Direction of change
Shifted datetime
Timedate difference from when he started

In the scene that information is kind of obvious from the environment, so I can see the argument for not having it. But if he was in some largely-unchanging environment, like a panic room or an underground cave or a Sanctum Sanctorum, knowing that information would save him from letting the shuttle go too far and finding himself in the Ordovician. A “home” button might also help to quickly recover from mistakes. Adding these signals would also help distinguish the two modes. They work differently, so they should look different. As it stands, they look identical.

He still (probably) needs future branches

Can Strange scrub the future this way? We don’t see it in the movie. But if so, we have many of the same questions as the Tibet mode future scrubber: Which timeline are we viewing & how probable is it? What other probabilities exist and how does he compare them? This argues for the addition of the future branches from that design.

Selecting the mode

So how does Strange specify the jog dial or shuttle wheel mode?

One cop-out answer is a mental command from Strange. It’s a cop-out because if the Eye responds to mental commands, this whole design exercise is moot, and we’re here to critique, practice, and learn. Not only that, but physical interfaces are more cinegenic, so better to make a concrete interaction for the film.

You might think we could modify the opening finger-tut (see the animated gif, below). But it turns out we need that for another reason: specifying the center and radius-of-effect.

Center and radius-of-effect

In Tibet, the Eye appears to affect just an apple and a tome. But since we see it affecting a whole area in Hong Kong, let’s presume the Eye affects time in a sphere. For the apple and tome, it was affecting a small sphere that included the table, too, it’s just that table didn’t change in the spans of time we see. So if it works in spheres, how is the center and the radius of the sphere set?

Center

Let’s say the Eye does some simple gaze monitoring to find the salient object at his locus of attention. Then it can center the effect on the thing and automatically set the radius of effect to the thing’s size across likely-to-be scrubbed extents. In Tibet, it’s easy. Apple? Check. Tome? Check. In Hong Kong, he’s focusing on the Sanctum, and its image recognition is smart enough to understand the concept of “this building.”

Radius

But the Hong Kong radius stretches out beyond his line of sight, affecting something with a very vague visual and even conceptual definition, that is, “the wrecked neighborhood.” So auto-setting these variables wouldn’t work without reconceiving the Eye as a general artificial intelligence. That would have some massive repercussions throughout the diegesis, so let’s avoid that.

If it’s a manual control, how does he do it? Watch the animated gif above carefully and see he’s got two steps to the “turn Eye on” tut: opening the eye by making an eye shape, and after the aperture opens, spreading his hands apart, or kind of expanding the Eye. In Tibet that spreading motion is slow and close. In Hong it’s faster and farther. That’s enough evidence to say the spread*speed determines the radius. We run into the scales problem of apple-versus-neighborhood that we had in determining the time extents, but make it logarithmic and add some visual feedback and he should be able to pick arbitrary sizes with precision.

So…back to mode selection

So if we’re committing the “turn on” gesture to specifying the center-and-radius, the only other gesture left is the saucer creation. For a quick reminder, here’s how it works in Tibet.

Since the circle works pretty well for a jog dial, let’s leave this for Tibet mode. A contrasting but related gesture would be to have Strange hold his right hand flat, in a sagittal plane, with the palm facing to his left. (See an illustration, below.) Then he can tilt his hand inside the saucer to reverse or fast forward time, and withdraw his hand from the saucer graphic to leave time moving at the adjusted rate. Let the speed of the saucer indicate speed of change. To map to a clock, tilting to the left would reverse time, and tilting to the right would advance it.

How the datetime could be shown is an exercise for the reader.

The yank out

There’s one more function we see twice in the Hong Kong scene. Strange is able to pull Mordo and Wong from the reversing time stream by thrusting the saucer toward them. This is a goofy choice of a gesture that makes no semantic sense. It would make much more sense for Strange to keep his saucer hand extended, and use his left hand to pull them from the reversing stream.

Whew.

So one of the nice things about this movie interface, is that while it doesn’t hold up under the close scrutiny of this blog, the interface to the Eye of Agamotto works while watching the film. Audience sees the apple happen, and gets that gestures + glowing green circle = adjusting time. For that, it works.

That said, we can see improvements that would not affect the script, would not require much more of the actors, and not add too much to post. It could be more consistent and believable.

But we’re not done yet. There’s one other function shown by the Eye of Agamotto when Strange takes it into the Dark Dimension, which is the final mode of the Eye, up next.

Sling Ring

8 Aug 2017 by scoutlandish

A sling ring opens magical portals of varying sizes between two locations. A sorcerer imagines the destination, concentrates, holds the hand wearing the ring upright and with the other gesticulates in a circle, and the portal opens with a burst of yellow sparks around the edges of the portal.

How might this function as technology

It can’t.

Teleportation, even given cutting-edge concepts of quantum entanglement, is limited to bits of information. All the writing on this topic that I can find online says that physical portals require too much energy. So we have to write the totality of this device off as a narrative conceit.

We can imagine the input working, though, as a reading-from-the-brain interface that matches a sorcerer’s mental image of a location to a physical location in the world. As if you were able to upload an image and have a search engine identify its location. That said, reading-from-the-brain has edge cases to consider.

What if the envisioned place is only imaginary?
What if the sorcerer only has the vaguest memory of it? Or just a name?
What if the picture is clear but the place no longer exists? (Like, say, Sokovia.)

Perhaps of course the portal just never opens, but how does the sorcerer know that’s the cause of the malfunction? Perhaps a glowing 404 would help the more modern sorcerers understand.

@scifiinterfaces has you covered, Steven.

The gestural component

The circular gesture is the mechanism for initiating the portal, an active meditation that likely makes concentrating on the location easier. If we had to compliment one thing, it’s that the gesture is well mapped to the shape of the portal, and having a gesture-concentration requirement ensures that portals aren’t just popping up at whim around Kamar-taj anytime someone wearing a ring remembers a place.

OK. That done, we’re at the end of the compliments. Because otherwise, it’s just dumb.

No, really. Dumb.

The physical design of the Sling Ring is dumb. Like Dumb and Dumber dumb. There are plenty of examples of objects or interfaces in movies that only exist because a writer was lazy, but the SlingRing™ deserves a special award category unto itself.

First off they aren’t rings, they are more like a set of brass knuckles as designed by a 10 year old who’s never seen a brass knuckle.
It restricts the movement of two fingers on the user’s hand which seem to be critical to casting all other spells in the film.
It makes any kind of physical combat with that hand significantly more difficult which, curiously, is something sorcerers seem to do a lot of. Punching someone while wearing a sling ring would likely break a finger or three.
It’s interesting to note that the sentience that makes other relics special may also be their security. However, any sorcerer can apparently pick up any sling ring and use it. Have you ever been bowling? The first 5 minutes of any trip to a bowling alley is trying to find the right sized ball. So unless there’s someone making bespoke sling rings for new magicians, there’s going to be a lot of poorly sized sling rings out there. This might explain why Strange’s ring is loose enough that it can easily fall off or be pulled off, which is inconvenient at best when fighting another sorcerer.

Redesigning this device as a single ring, or a glove or a wristband, or a necklace, or a tiara or a codpiece…frankly just about anything would be better than this tragically flawed artifact and would likely solve all of the above problems.

Hang on, let me remove my not-sling-not-ring first.

And what the heck does sling even mean in this context?

Staff of the Living Tribunal

20 Jun 2017 by Christopher Noessel and scoutlandish

This staff appears to be made of wood and is approximately a meter long when in its normal form. When activated by Mordo it has several powers. With a strong pull on both ends, the staff expands into a jointed energy nunchaku. It can also extend to an even greater length like a bullwhip. When it impacts a solid object such as a floor, it seems to release a crack of loud energy. Too bad we only ever see it in demo mode.

How might this work as technology?

The staff is composed of concentric rings within rings of material similar to a collapsing travel cup. This allows the device to expand and contract in length. The handle would likely contain the artificial intelligence and a power source that activates when Mordo gives it a gestural command, or if we’re thinking far future, a mental one. There might also be an additional control for energy discharge.

In the movie, sadly, Mordo does not use the Staff to its best effect, especially when Kaecilius returns to the New York sanctum. Mordo could easily disrupt the spell being cast by the disciples using the staff like a whip, but instead he leaps off the balcony to physically attack them. Dude, you’re the franchise’s next Big Bad? But let’s put down the character’s missteps to look at the interface.

Mode switching and inline meta-signals

Any time you design a thing with modes, you have to design the state changes between those modes. Let’s look at how Mordo moves between staff, nunchaku, and whip in this short demonstration scene.

To go from staff to nunchaku, Mordo pulls it apart. It’s now in a dangerous state, so is there any authentication or safety switch here? It could be there, but all passive via contact sensors, which would be the best so it could be activated in a hurry. The film doesn’t give us any clue, really, so that’s an open question.

How does it know to go from nunchaku to whip? It sure would be crappy to bet on a disabling thwack against your opponent only to find it lazily draping over a shoulder instead. (Pere Perez might have advanced ideas, given his ideas on light saber tactics.) Again, this state change could be passive, detecting in real time the subtle gestural differences in a distal snap, which a bullwhip would need, and lateral force, which sets the nunchaku spinning, and adjust between the two accordingly. Gestural and predictive technologies are not particularly cinegenic, so let’s give it the benefit of the doubt and say that’s what’s happening.

A last mode is After Mordo cracks it against the ground, it retracts back to Staff form. This is the hardest one to buy. Certainly it’s a most dramatic ending for Mordo’s demonstration. But does it snap back automatically after it strikes a surface? Automation is not always the answer. Deliberate control would mean Mordo doesn’t have to waste time undoing unwanted automatic actions.

Critical systems must be extremely confident in their interpretations before automation is the right choice.

It might be that this particular gesture is a retraction signal, but how the Staff distinguishes this from a mid-combat strike is tricky. It would have to have sophisticated situational awareness to know the difference, and it doesn’t display this. Better backworlding would point at some subtle gestural signal from Mordo. A double-tightening of his grip, maybe. Or even a double-slight-release of his grip, since that’s something he’s quite unlikely to do in combat.

This is a broad pattern for designers to remember. Inline control signals should be simple-to-provide, but unlikely to occur in literal use. Imagine if the Winter Soldier’s Trigger Phrase wasn’t “Longing, rusted, 17, daybreak, furnace, 9, benign, homecoming, 1, freight car” but instead was the word “the.” He’d be berserking every few seconds. Unworkable. So, if you were designing the Staff’s retraction command gesture, you’d have to pick something he could remember and perform easily, and that would be difficult to accidentally provide.

If Mordo has the staff in the next film, I hope the control modes are clearer and of course well-designed.

Kubris

8 Mar 2017 by Christopher Noessel

Perhaps the most unusual interface in the film is a game seen when Theo visits his cousin Nigel for a meal and to ask for a favor. Nigel’s son Alex sits at the table silent and distant, his attention on a strange game that it’s designer, Mark Coleran, tells me is called “Kubris,” a 3D hybrid of Tetris and Rubik’s Cube.

Alex operates the game by twitching and sliding his fingers in the air. With each twitch a small twang is heard. He suspends his hand a bit above the table to have room. His finger movements are tracked by thin black wires that extend from small plastic discs at his fingertips back to a device worn on his wrist. This device looks like a streamlined digital watch, but where the face of a clock would be are a set of multicolored LEDs arranged in rows. These LEDs flicker on and off in inscrutable patterns, but clearly showing some state of the game. There is an inset LED block that also displays an increasing score.

The game also features a small, transparent, flat screen that rests on the table in front of him. It displays a computer-generated cube, similar to a 5×5 Rubik’s Cube, made up of smaller transparent cubes that share colors with the LEDs on his wrist. As Alex plays, he changes the orientation of the cube, and positions smaller cubes along the surface of the larger.

Alex plays this game continually during the course of the scene. He is so engrossed in it that when Nigel asks him twice to take his pills, he doesnt even register the instruction. Nigel must yell at him to get Alex to comply.

Though the exact workings of the game are a mystery, it serves to illustrate in a technological way how some of the younger people in 2027 disengage from the horror of the world through games that have been designed for addiction and obsession.

Gestural Spheres

8 Feb 2016 by Christopher Noessel

While working on some other material this weekend, I just noticed two unusual, but similar gestures from different movies in 2015, which are gestures on the outside of spheres.

First, the Something control sphere from Tomorrowland.

sphere_gesture_tomorrowland_tight

And, the core memories in Inside Out.

sphere_gesture_insideout_tight

The gestures are subtly different (Tomorrowland is full palm, Inside Out is two fingers) and their meanings are different (Tomorrowland is to shift direction of travel of the time camera, Inside Out is to scrub the time itself) but they are a nice gestural rhyme of each other.

The Inside Out image reminds me that I really, really need to do a full retrospective of interfaces in Pixar movies, because they are quite extraordinary in the aggregate.