Loki’s glaive

When Loki materializes on the dais, he is holding one the key objects to The Avengers and indeed the whole Marvel Cinematic Universe multi-franchise Infinity Stones plot. What is it?

Avengers-Glaive-02

NIck Fury calls the thing a spear. Others call it a staff. The official Disney wiki calls it the Chitauri Sceptre, but this thing is very much a tool. Over this and the next several posts, I’ll talk about how it is used alternately as the following.

  • A melée weapon
  • A projectile weapon
  • A bad-mojo radiator
  • A teleconferencing device
  • An enthrallment knife

Notably, in no scene does he carry it on a ceremonial occasion as a symbol of sovereignty, so scepter really doesn’t fit our purposes. What does? Well, any RPG fan worth their Deck of Many Things knows that the blades-on-a-stick category of weapons are many and nuanced. Finding a perfect term is tough since historians and medievalists have categorized other pole arms according to their construction and function, and none of them are quite like this one.

Avengers-Glaive

So though it hurts to let go of possibilities like falx, svärdstav, or bohemian earspoon—and also because I apparently hate the SEO that would earn me all the millionsI think the thing fits most readily into the category of glaive, since glaives are defined as a single-edged (I know, but it’s not quite double-edged either) slicing pole arm with a piercing tip. Like this one. So debate the choice in the comments if you must, but you’ll have to be pretty convincing since I’ve already written and scheduled the other posts and I have a lot to do in the UK at UX London over the next weeks.

And of course recognizing it as a glaive also gives us an opportunity for this joke.

toglaive

J.D.E.M. LEVEL 5

The first computer interface we see in the film occurs at 3:55. It’s an interface for housing and monitoring the tesseract, a cube that is described in the film as “an energy source” that S.H.I.E.L.D. plans to use to “harness energy from space.” We join the cube after it has unexpectedly and erratically begun to throw off low levels of gamma radiation.

The harnessing interface consists of a housing, a dais at the end of a runway, and a monitoring screen.

Avengers-cubemonitoring-07
Fury walks past the dais they erected just because.

The housing & dais

The harness consists of a large circular housing that holds the cube and exposes one face of it towards a long runway that ends in a dais. Diegetically this is meant to be read more as engineering than interface, but it does raise questions. For instance, if they didn’t already know it was going to teleport someone here, why was there a dais there at all, at that exact distance, with stairs leading up to it? How’s that harnessing energy? Wouldn’t you expect a battery at the far end? If they did expect a person as it seems they did, then the whole destroying swaths of New York City thing might have been avoided if the runway had ended instead in the Hulk-holding cage that we see later in the film. So…you know…a considerable flaw in their unknown-passenger teleportation landing strip design. Anyhoo, the housing is also notable for keeping part of the cube visible to users near it, and holding it at a particular orientation, which plays into the other component of the harness—the monitor.

Avengers-cubemonitoring-03

The monitor

In the underground laboratory, an (unnamed?) technician warns lead scientist Selvig that, “it’s spiking again,” and the camera pans down to this monitoring interface.

JDEM

Header

The header is a static barcode followed by the initialism J.D.E.M. along with its full name, the Joint Dark Energy Mission. (Sounds super cool and sci-fi, right? Turns out it is a real program between NASA and the US DOE.) Another label across the top identifies the screen as LEVEL 5 and that it belongs to PROJECT PEGASUS and NASA.

3D map

A main display shows a 3D wireframe of the tesseract, with color-coded nebula-like shapes within the cube. The wireframe (and most of the text on screen) are a bright cyan, with internal features progressing in color from the cyan through white to a blood red, all the way to lens flares near the most active areas in the cube. The color choices make for a quick read of what is “cool” and what is “hot,” so are effective for being immediate, but if the lens flares are designed into the system to indicate peakness, it’s a bad choice for obscuring other data in the display.

Note that the wireframe of the cube is also rotating slightly, which is  very helpful for a user to more fully understand 3D information from a 2D screen. It might be even better mapping with less cognitive load if the display was a volumetric projection. (VPs exist within the Marvel Cinematic Universe (MCU), but so far I believe we’ve only ever seen them in Tony Stark’s possession so perhaps he has not released it to the outside world.) Hopefully in its rotation on this monitor it does not rotate in 360°, as the regularness of the cube would make it difficult to understand where an internal anomaly might exist in the real thing. Hopefully the wireframe only wavers back and forth within a few degrees, and is oriented in roughly the same way an observer glancing at the real thing would see it in the housing, to allow for instant mapping of problem areas.

Avengers-cubemonitoring-01

Warning

Just to the left of the 3D map is a data monitoring panel. Its top label blinks a red WARNING CRITICAL ENERGY LEVELS and a percentage readout. The panel also features a key whose colors match those of the map. (As it should.) Hopefully a microinteraction allows a user to touch any part of the map, freeze the rotation, and get the percentage details of the touched point. A detail box wavers its vertical position along the key to provide a user a quick assessment of its value, and also contains a percentage readout for precision. Judging by the position of the box and the readout, it looks like the 100% mark is about halfway up the screen. Hopefully the upper part of the scale is logarithmic to accommodate massive surges in values.

Additional elements of the display include several scrolling waveforms and text boxes with inscrutable data and labels. It’s easy to imagine these as useful (say total energy values for specific electromagnetic frequencies) but they’re difficult to read, so difficult to formally evaluate.

All told, a nice display (per some assumptions) for monitoring what’s happening with the cube.

Now if only they had applied that solid design thinking to that dais vs. cage problem.

Avengers-cubemonitoring-04

The Avengers (2013): Overview

Release date: 4 May 2013, United States

Avengers-title

As with all overviews, ALL THE SPOILERS ahead.

A mysterious alien artifact called the Tesseract summons the Asgardian god Loki to the Earth, where he uses a powerful staff to either kill or enthrall several S.H.I.E.L.D. operatives before stealing the Tesseract and making his escape with them. The head of S.H.I.E.L.D., Nick Fury, gains permission from a shadowy council to assemble a team of superheroes (Iron Man, Bruce Banner, Black Widow, and Captain America) code named The Avengers Initiative to help capture Loki and recover the Tesseract. They find and capture him in Berlin but his operatives get away with a cache of rare metals. Loki’s brother Thor shows up to claim him but after fighting Iron Man and Captain America, Thor agrees to let Loki remain captured in S.H.I.E.L.D.’s helicarrier.

Loki’s operatives trace him and sabotage the helicarrier to free him as Banner becomes the Hulk and goes on a rampage through the vessel. Through fierce combat and resourcefulness, the helicarrier is saved from crashing, but Loki escapes with his staff.

In New York City Loki’s operatives use the metals they stole and the Tesseract to create an interdimensional gate through which he summons an alien army. Though the Avengers mount a strong defense of the city, the shadowy council orders a nuclear strike on the city to destroy the alien army. Iron Man intercepts the missile, flies it through the portal into the alien mothership, disabling the invaders em masse before falling back through the portal to Earth.

At the resolution of the film, Thor returns to Asgard with Loki and the Tesseract and the staff remains on Earth. Also the team enjoys some shawarma.

Avengers-shawarma

IMDB: https://www.imdb.com/title/tt0848228/

Sci-fi University Episode 2: Synecdoche & The Ghost in the Shell

How can direct manipulation work on objects that are too large to be directly manipulated?

Sci-fi University critically examines interfaces in sci-fi that illustrate core design concepts. In this 3:30 minute episode, Christopher discusses how the interfaces of Ghost in the Shell introduces synecdoche to our gestural language.

If you know someone who likes anime, and is interested in natural user interfaces—especially gesture—please share this video with them.

Special ありがとう to Tom Parker for his editing.

Watch a supercut of every user interface from The Avengers (2013)

With the reviews of Oblivion behind us, and The Avengers: Age of Ultron upon us in a matter of days, I thought it would be good to review the movie that canonized Joss Whedon into Hollywood sainthood so hard they had to retcon Catholicism into the timeline so this joke could happen.

Here it is, a supercut of every user interface in The Avengers (2013).

Boy there are some amazing interfaces in there. Got any favorites?

Report Card: Oblivion

Read all the Oblivion reviews in chronological order.

According to the director, Oblivion is “a daylight science fiction film with a kind of Twilight Zone story,” a callback to pre-Star Wars, 1970’s lonely man sci-fi set against a huge backdrop. (Read the full interview by Germain Lussier on /Film for more.) Certainly, it’s more visually-satisfying thing than intellectually-satisfying thing, but fortunately that same thing does not play out in the interfaces.

Oblivion-Report-Card

Sci: B (3 of 4) How believable are the interfaces?

One of the great strengths of the interfaces are their deep ties to the diegesis. There’s little fuidgetry, little that could be generically lifted and placed in another film. It’s what we used to call site-specific in design school and that’s a good thing for believability.

See how in Vika’s desktop the sections of interface contain things she has to monitor: Land, hydrorigs, drones, the Tet’s orbital position. Most of the interfaces in the film are this considered.

image00

On the flip side, there are communication systems that suffer more downtime than modern systems. There’s a flight control interface that omits the weather. The Scav binoculars just don’t make sense. And the Odyssey has a bunch of problems given that’s meant to be a near-future-ish extension of what we know today.

And then…then…then there’s the narrative-shortcut trope of the oh-by-the-way faster-than-light communication system that would have meant a much more advanced (and more defended?) world for the Tet to encounter in the first place.

Desktop_2014_07_15_19_32_16_746

So, some dings.

Fi: A (4 of 4) How well do the interfaces inform the narrative of the story?

This is where Oblivion’s interfaces really shine. They’re gorgeously realized with a rich stylistic and motion language. But moreso IMHO some of the apparent “problems” with the interfaces actually tell of the deep deception by the Tet. It’s core to telling that central story, and partly told through the interfaces.

Home 49 disconnects its inhabitants from the land they’re tasked to protect. Tet’s thinking: Perfect.

image02

Jack’s bike doesn’t make a lot of sense in the diegesis except that it is a perfect outlet for his sense of “freedom.” Tet’s thinking: Whew. Glad he has that outlet.

image00

Other narrative aspects of the interfaces like the drone programmer help underscore the drones as aggressive, suspect, and alien, rather than defensive human measures.

I’d add a + to that A if the drones hadn’t been designed to look evil and menacing. Had they been more Hello Kitty and less Galactic Empire, Jack might have been less suspicious.

hello-drone
If it needs to be said: Not actually from Oblivion. Maybe the reboot.

Interfaces: B (3 of 4)
How well do the interfaces equip the characters to achieve their goals?

The centerpiece of the film is Vika’s desktop. It’s her command and control center workstation that enables her to manage the strategy to Jack’s tactics, and even rest her teacup as she works. The most commonly accessed bits are in easy reach, and the display-only information is turned vertically for her like a clock on the wall.

Oblivion-Desktop-Overview-003

It has a few ergonomic problems, like angling its displays away from her observational sphere (just for a teacup?) It doesn’t equip her for crisis conversations like it should. Some of its interactions are inconsistent. It sometimes makes her hunt for information rather than leading her there. But, all in all, a nice dashboard for her task.

There are other interesting bits, like the situationally-shaped reticle, the breakfast table that allows for sitrep breakfasts, and well-mapped Odyssey controls that imply a bit of agentive support.

There are some usability problems throughout, or it would have fared better, but overall a good show.

Final Grade B+ (10 of 12), MUST-SEE

All told, these interfaces are rich and powerful and embody solid modern thinking about visual styling, motion design, gestural interaction, and heads-up-displays. Big props to that pro gmunk for his work (and keep an eye out for an interview with him about his work on the film soon.)

And may I send out a special shout-out to the guest bloggers for their excellent insights and write-ups: Clayton, Aleatha, Heath, and Maximilion. They did great and I’m very glad that at least four other people in the world know how much effort goes into providing these in-depth interface analyses. Let’s hope we hear from more about them on this blog in the future.

Pitch time: Learn more lessons about gestural interfaces, heads-up-displays, and other interface concepts from a vast survey of science fiction movies and television programs in the book I co-authored with Nathan Shedroff, Make It So: Interaction Design Lessons from Science Fiction.

IMDB: https://www.imdb.com/title/tt1483013/Currently streaming on:

A definitive list of sci-fi drones (in progress)

In chats with friends and followers of the blog about sci-fi Drone Week, folks seem interested in coming up with a definitive list of sci-fi drones in movies and TV shows. While we might get there eventually by reviewing the movies and TV shows in which they appear, let’s beat that to the punch by creating the list FROM OUR MINDS.

Criteria:

  • In a sci-fi movie or television show
  • Is not simply a representation of a real-world drone
  • Is mobile (in the air, on the ground, or in space)
  • Appears (or defined as) technolgoical, not biological
  • Does not have a sentient controller/pilot aboard
  • Does not possess strong/general artificial intelligence
  • Either automous or remotely-controlled

Here’s what I’ve collected so far. Add more in the comments if you think of them.

Aerial (UAVs)

  • Star Wars: Episode IV – A New Hope (1977)
    • The lightsaber training orb
    • The mouse-bot that Chewbacca scares
  • Flash Gordon (1980) the bot in Ming’s chamber
  • Viper Probe Droid seen on Hoth in Star Wars: Episode V – The Empire Strikes Back (1980)
  • Terminator diegesis (1984–)
    • Aerostats
    • Moto-Terminators (Salvation)
    • Aerostats (Salvation)
    • Hydrobots (Salvation)
  • Star Trek: The Next Generation (1987)
  • They Live (1988) but only when you wore the glasses
  • Back to the Future II (1989) had USA Today drones (unclear if they’re AI, but benefit of the doubt?)
  • Babylon 5’s (1994) camera drones
  • Stargate diegesis (1997–)
    • (early versions of the) Replicators
    • Kinos
    • World-testing UAVs
    • S4E2 The Other Side was all about drone warfare
  • Star Wars: Episode I – The Phantom Menace (1999) had “holocameras” following the podraces
  • Farscape (1999) has adorable little DRDs
  • Dark Angel (2000) had Police Hover Drones that the titular character got to surf.
  • The Incredibles (2004) Syndrome controls a few drones to do his bidding
  • Stealth (2005) (prior to the lightning strike that gives it strong general intelligence
  • Sleep Dealer (2008)
  • Wall•E (2008) SO many, though the level of their AI might disclude some
  • Skyline (2010) (has both alien drones and real world human drones)
  • The topography “pups” of Prometheus (2012)
  • Robocop (2013) (has both aerial and ground)
  • Agents of S.H.I.E.L.D. (2013–) (Seriously, this show has a thing for them)
  • Star Trek: Insurrection’s (1998) transporter/transponder drones
  • Battleship (2012) battle bots
  • Hunger Games (2012) delivery droids
  • Elysium (2013)
  • Drone (2013)
  • Oblivion’s numbered, Tet-tech drones (2014)
  • Captain America: The Winter Soldier’s city-sized drones (2014)
  • Chappie (2015) (aerial-capable, but mostly ground)

Ground

  • The floor-sweeping robots from The Fifth Element (1995)
  • The Robot from the Lost in Space movie (1998), which Will could remotely control
  • The Spyders of Minority Report (2002)
  • The remotely-controlled robots in Surrogates (2009)
  • Microbots from Big Hero 6 (2014) (a swarm of drones)

Thanks to the following suggestors for the initial list: Kelley Strang, John Danuil, Devin Hartnett, Derek Eclavea, Lane Bourn, Wally Pfingsten & kedamono.

Almost but not quite

Some suggestions seem like they would be perfect candidates, but for some reason skim the definition.

  • Bit from Tron (1982) may have been limited to yes or no answers, but was an AI
  • Machine gun drones from the deleted scenes of Aliens (1986) can only swivel, not move
  • Dreadnought from Star Trek: Voyager (1995)(VOY, Dreadnought) it’s AI
  • Jarvis from the cinematic Iron Man/Avengers diegesis, also strong AI

Keep them coming in the comments. What did we forget?

Scav Reticle

The last Scav tech (and the last review of tech in the nerdsourced reviews of Oblivion) is a short one. During the drone assault on the Scav compound, we get a glimpse of the reticle used by the rebel Sykes as he tries to target a weak spot in a drone’s backside.
Scav reticle

The reticle has a lot of problems, given Sykes’ task. The data on the periphery is too small to be readable. There are some distracting lines from the augmentation boxes which, if they’re just pointing to static points along the hairline, should be removed. The grid doesn’t seem to serve much purpose. There aren’t good differentiations among the ticks to be able to quickly subitize subtensions. (Read: tell how wide a thing is compared to the tick marks.) (You know, like with a ruler.)

ruler

The reticle certainly looks sci-fi, but real-world utility seems low.

The nicest and most surprising thing though is that the bullseye is the right shape and size of the thing he’s targeting. Whatever that circle thing is on the drone (a thermal exhaust port, which seem to be ubiquitously weak in spherical tech) this reticle seems to be custom-shaped to help target it. This may be giving it a lot of credit, but in a bit of apologetics, what if it had a lot of goal awareness, and adjusted the bullseye to match the thing he was targeting? Could it take on a tire shape to disable a car? Or a patella shape to help incapacitate a human attacker? That would be a very useful reticle feature.

Scav dual-monoculars

As Jack searches early in the film for Drone 172, he parks his bike next to a sinkhole in the desert and cautiously peers into it. As he does so, he is being observed from afar by a sinister looking Scav through a set of asymmetrical…well, it’s not exactly right to call them binoculars.

scav_oculars_04

They look kind of binocular, but that term technically refers to a machine that displays two slighty-offset images shown independently to each eye such that the user perceives stereopsis, or a single field in 3D. But a quick shot from the Scav’s perspective shows that this is not what is shown at all.

scavnolculars

This device’s two lenses take in different spectrums of light and displays them side by side, with a little (albeit inscrutable) augmentation at the periphery. The larger display on the left appears to be visible light and the smaller on the right appears—based on the strong highlight around the bike’s engine and Jack’s body—to be infrared, or heat.

At this point in the story, the audience is meant to believe that the scavs are still the evil alien race, and this interface helps to convey that. It seems foreign, mysterious. All of its typographic elements (letters, numbers, symbols) are squeezed into little more than 4×4 grids of pixels, so we’re not even sure if this is a human language. So, fine, this interface serves its narrative purpose here. “Oh my,” we must think, “…he is being watched. But by what? And why?”

But after we find out [again, spoilers] that the scavs are the Terran survivors after the Tet attack, we can look at this again to understand that this interface is for humans, and with that in mind it does not fare well.

Yes, the periphery is augmented, so that’s good, but the information is unusably small, and forces the user to glance back and forth between the two images to put the disparate information together.

Two views reduce the amount of information

It almost goes without saying, but let’s say it—by dividing the available display into two halves, the amount of visual information provided to the Scav is roughly a quarter of what it would be with a single view. And since the purpose of the device is to magnify, this is a significant loss.

Two views add work

In this scene, which is quite barren, it’s very easy to tell that the objects that are warm in the right are the only two objects on the left, but if you imagine looking at a cityscape, where the bomb (hot) looks very much like every other thing around it, you can see where piecing those two disparate views together in your head can become problematic.

This is made worse when the views aren’t even positionally synchronized. In the gif below you’ll see that when you superimpose them, they drift away from each other, making the comparison between the two even more difficult. There are diegetic reasons why this might have happened, but rather than reverse engineering why, let’s just leave it that it makes using it more difficult.

scavnolculars_overlaid

The blur and low-contrast don’t help

Note that the thermal view is blurrier and lower-contrast. That might be an artifact of the diegetic tech, but it would confound quick mapping in a complex image. Even if it’s just a lower-res image, at least the device should perform some auto-leveling and sharpening functions on the live image to help make it easy to use.

Having one scaled makes it worse

The scaling makes the mapping of items from one screen to the other more difficult. Again, in the Oblivion example, there are two objects on the left and two objects on the right, and the “horizons” on which they walk are roughly aligned, so it’s trivial to track one to the other. But if the image is highly repetitive—say for example, a building—the scaling would make it difficult to map the useful point-of-interest on the right to the best-resolution image on the left. Quick…in which window is the sniper?

scav_oculars_buildings

A more direct solution

Better would be a live augmentation of a single, visual-light image. The visual light is the best anchor to the real world, with augmentation helping to convey specialness to the objects in the scene. In the comp below, you’ll see a single image where the “hot spots” have been augmented with a soft red and some trend lines in white. That red color is not arbitrary, by the way. It builds on the human experience with black body radiation associations of red == hot. This saves the (quite human) user both the physical work of glancing back and forth and the extra cognitive processing to recall that green/yellow == heat.

scav_oculars_comp

Ghost in the Shell: Home viewing

San Francisco Bay Area folks may have been wondering what was up with the Ghost in the Shell 20th anniversary movie night. Well, some bad news.

GitS-Aramaki-11

There weren’t enough pre-sales to rent the cinema. We might have just run it as a public showing, but the cinema could not find a way to secure the rights for a public showing despite best efforts and the use of Google Translate on promising Japanese sites. You might think in that case that you could just show it anyway, but the owners cited a story in which independent filmmakers once had to fork over a cool $8000 for an unauthorized showing of a film, even when the normal licensing was only $200. So, without licensing, no public showing. But that doesn’t have to stop us. We have technology.

GitS-Hands-06

A synchronozed home viewing of Ghost in the Shell

I’m watching Ghost in the Shell on Saturday, 28 March 2015, starting at 20:30 PDT. I may have a few friends over. Want to join? Well, my couch will likely be full, but get a copy of the movie yourself on Blu-Ray, or Amazon instant video, or Netflix DVD (not available at this time streaming through Netflix), and we can live tweet the event. I’ve just launched the twitter handle @SFImovienight, where

  • I will announce upcoming movie nights
  • I will track movie night requests
  • I will live tweet movies as we’re watching
  • Anyone else watching along can join in

The hashtag for this viewing will be #gits20.

Yes a contest

Since this won’t be a live event, let’s shake the contest up a bit. No trivia. Whatever tweet that:

  • Includes #gits20
  • Tags @SFImovienight
  • Gets the most retweets

…between now and 28 March 2015 23:00 PDT will win an Adobe Creative Cloud license for 1 year, a $600 value, as an offer by in-kind sponsor Adobe.

Has anyone tried this before? Have suggestions?