The holocircus

21 Dec 2015 by Christopher Noessel

To distract Lumpy while she tends to dinner, Malla sits him down at a holotable to watch a circus program. She leans down to one of the four control panels inset around the table’s edge, presses a few buttons, and the program begins.

In the program small volumetric projections of human (not Wookie) performers appear on the surface of the table and begin a dance and acrobatic performance to a soundtrack that is, frankly, ear-curdling.

A furry creature with long, greenish hair stands next to a patterned circular table, looking down with a curious expression.

A person observing a small dancer on a patterned floor, with a plant and furniture visible in the background.

Partway through the performance, Lumpy presses the second-from-the-left button, and the main dancer (who has just got to be a Radical Faerie, a full year before that august organization formed) disappears from the table only to become a life-sized projection just off the table, as if the figure had invisible legs but stood on the floor of the house.

Close-up view of a vintage computer keyboard integrated into a modern device, set against a blurred colorful background.

A close-up of a yellow, fluffy object being placed into a machine, with a blurred background.

When Lumpy is done with the performance, he presses a button, and RF guy disappears from the room and reappears in small size on the table to join the other performers on the table for a bow and curtain call before they all disappear and that ghastly, inhumane music stops.

A close-up of a furry, humanoid creature with a serious expression and large eyes, resembling a character from a science fiction universe.

A green ghostly figure levitates and dances above a table, while a furry creature observes in a room filled with shelves and plants.

So…yeah. This interface.

The terrible usability of tape recorders

It’s pretty clear the control panels are cassette recorders. I know that young readers won’t know what the pfassk that is, so for due diligence, let me explain. Those devices recorded and played back audio stored on magnetic tapes wound in cartridges. The controls for it were six stay-state toggles. These types of buttons have two states. Pushing one locks it down and sets the device to a mode: Play, Rewind, or Fast Forward. One special button, Record (often colored red) had to be chorded with the Play button as a safety precaution against accidentally destroying an existing recording. Users had to press a special Stop button to release any pressed button and stop the mode. Fast forward and Rewind could be pressed while audio was playing, but in such cases they would act as a momentary button. When any of the modes reached the end of a tape, higher-end recorders would also automatically release that button.

So let’s all admit that these are pretty terrible controls. From the fact that they almost all look the same to the fact that toggles make much more sense. (Press one button to play, press that same button again to make it stop.) But toggles weren’t the consumer model of the time. And this holocircus interface, hastily put together from things that the prop department could purchase off the shelf in the one day they were given to hack it together, inherits all that unusability and piles some more confusion onto it.

For instance

When we see Lumpy press the “Make the Dancing Man Bigger” mode, no other buttons are depressed. That means Malla had to set the circus mode, start the program, and press something like a stop button that released those. In turn that means if there was nothing on the table, it could either be because this was a blank spot in the media or that it wasn’t actually playing. You couldn’t tell, though, because the neither the controls nor the media didn’t reflect the state of the thing. This is probably not a common problem because we see that the garish programs and their terrible soundtracks are pretty effing hard to miss, but from a pure usability point of view, it sucks and should be avoided.

Now, there’s probably a thought exercise that could be done to figure out what keys mapped to which functions (and how), as well as second-guess the capabilities of the media (could he have picked any performer to enlarge, or was there just this option?) but honestly, it would be a Sisyphean multithreaded process of single-function vs. combined-letter arguments made by myself in a wall of text with little to no payoff for either of us. Knock yourself out in the comments if you want, but I’d rather focus on another aspect of the interface that I think does pay off, and that’s the visual language of the VP.

You know, for kids

This volumetric projection does not have most of the visual signals common to sci-fi hologram trope:

Edge-lighting
Blue cast
Scan lines
Projection rays

It does have translucency and an unusual scale, but that’s it. This visual language was actually established in Star Wars: Episode IV – A New Hope. So its absence here is conspicuous. As I noted in Make It So, these signals make it very clear to the observer that what they are observing is media—i.e. not real. Of course the real reason these VPs lack those signals is because this show had the budget of this blog post, but can we find a diegetic reason?

Surprisingly, I think yes. Kid’s media is different than regular media. Just like we pretend that Santa Claus is real for young children until they’re old enough to start to think critically about it, perhaps special effort is taken on these VPs to make it really immersive for the hairy kiddo. He doesn’t get the it’s-not-real signals because they want him to believe that it is.

Which is some fine holiday apologetics, if I do say so myself.

Stark Tower monitoring

6 Oct 2015 by Christopher Noessel

Since Tony disconnected the power transmission lines, Pepper has been monitoring Stark Tower in its new, off-the-power-grid state. To do this she studies a volumetric dashboard display that floats above glowing shelves on a desktop.

Volumetric elements

The display features some volumetric elements, all rendered as wireframes in the familiar Pepper’s Ghost (I know, I know) visual style: translucent, edge-lit planes. A large component to her right shows Stark Tower, with red lines highlighting the power traveling from the large arc reactor in the basement through the core of the building.

The center of the screen has a similarly-rendered close up of the arc reactor. A cutaway shows a pulsing ring of red-tinged energy flowing through its main torus.

This component makes a good deal of sense, showing her the physical thing she’s meant to be monitoring but not in a photographic way, but a way that helps her quickly locate any problems in space. The torus cutaway is a little strange, since if she’s meant to be monitoring it, she should monitor the whole thing, not just a quarter of it that has been cut away.

Flat elements

The remaining elements in the display appear on a flat plane.

To her upper left a spectrum analysis shows bars all near the middle, and the status is confirmed with a label as REACTOR OUTPUT NORMAL. Notably the bars are bright cyan at their top and fade to darker near their bases, drawing attention to the total arc. There few bands that are lower or higher than others are immediately visible from the contrast. That’s good visual design for attention management.

In the top center is a ring chart with three bands of colors: slate gray, cyan, and dark red. A large percentage read out in the center hovers around 90%. To her upper right are three other ring charts with the same color scheme. Above the three rings is a live trend line chart in red with a separate line in white (Target? Trend line?). A tiny side view of Stark Tower stands to the left of the three rings and trend line. The purpose of these are unclear as the labels are too small to read, but it seems strange to have the physical representation here. If the rings or trend lines are related to the physical tower, how they are related is unclear. If they are unrelated, having the rendering present there is misleading.

In a band across the middle are some tiny text blocks spitting out lines of data and auto scrolling. There are more of these, so for ease of reference, let’s call them teletype boxes. They look too small and move too fast to convey much meaning, as such things often do.

The lower left of the screen has a dim white section labeled MAGNETIC CONTAINMENT. It contains a 3D cuboid with a wildly undulating surface. As this shape doesn’t match anything else in the display, nor is it visually connected, its utility is unclear. If she is meant to track some variable other than undulationness, I can’t see how this would help. Also in this panel are some sliders wildly sliding back and forth, and some more teletype boxes.

In the lower center part of the screen is an overhead-view wireframe of the reactor that does not change in this segment, as well as a dim white box that appears to contain a static formula.

This display is not fit to Pepper’s task

Sadly, Gwen Paltrow looks a little wall-eyed here, so it’s hard to tell where exactly where she’s looking to determine as she says that “Levels are holding steady.” Her gaze appears roughly above the large ring chart, so let’s presume she’s looking there. If she is actually meant to be monitoring a set of levels to determine if they’re fluctuating or not, the ring chart and percentage are the wrong display types. These are valuable for showing a state that doesn’t change very often, i.e. where change over time is not of concern.

Sure, she might be able to suss out the steadiness of the variables by focusing on the percentage and the ring chart elements to see if they’re currently wobbling, but that requires her to keep her focus there to observe what’s happening with those pixels. That puts an unnecessary burden on her attention and short term memory. What if she has to glance away for a few seconds, and in those seconds, levels begin to wobble unsteadily? Sure, an alert could sound, but when she looks back she’ll have no idea what has happened while her attention was elsewhere. Humane interfaces remove unnecessary burdens from their users, and this display should do the same.

Use line charts and sparklines to show change over time

For any variable that is meant to be monitored for changes over time—like the scripted steadiness of levels—the line chart or sparkline is much more fit to task. Each plots variables over time, so even if a user glances away for a bit, it’s not a problem, she can look back and quickly suss out what’s been happening. The thing she’s looking at should look more like a ECG than a page out of an annual report.

The Iron Man HUD is an impossible thing

29 Jul 2015 by Christopher Noessel

In the prior post we looked at the HUD display from Tony’s point of view. In this post we dive deeper into the 2nd-person view, which turns out to be not what it seems.

The HUD itself displays a number of core capabilities across the Iron Man movies prior to its appearance in The Avengers. Cataloguing these capabilities lets us understand (or backworld) how he interacts with the HUD, equipping us to look for its common patterns and possible conflicts. In the first-person view, we saw it looked almost entirely like a rich agentive display, but with little interaction. But then there’s this gorgeous 2nd-person view.

When in the first film Tony first puts the faceplate on and says to JARVIS, “Engage heads-up display”… …we see things from a narrative-conceit, 2nd-person perspective, as if the helmet were huge and we are inside the cavernous space with him, seeing only Tony’s face and the augmented reality interface elements. You might be thinking, “Of course it’s a narrative conceit. It’s not real. It’s in a movie.” But what I mean by that is that even in the diegesis, the Marvel Cinematic World, this is not something that could be seen. Let’s move through the reasons why.

Not a mini-TARDIS

First, it looks like we’re in some TARDIS-like space where the helmet extends so far we can fit in it, or a camera can, about a meter from his face. But of course the helmet isn’t huge on the inside. Tony hasn’t broken those laws of physics. The helmet is helmet-sized on the inside.

Not a volumetric projection

Then there’s the issue of the huge display. It looks like a volumetric projection, like what R2-D2 can project, but that can’t be true, either. The projection would extend way beyond the boundaries of the helmet-sized helmet. Which as you can see below, is a non-starter. So it’s not a volumetric projection.

So, retinal projection

Then what is the display technology? Given the size constraints, retinal projection makes the most sense, but if we could make the helmet go invisible, it would look like Tony was having diffuse LASIK, or maybe playing The Game from Star Trek: The Next Generation.

STTNG The Game-02 — Let’s face it, this is not the worst thing you’ve caught me doing.

Representation of the projections?

So, OK, fine. Maybe what we see is what’s being projected, the separate stereoscopic images onto individual retinas. Nope. Then we would see two similar, slightly offset images, like in older anaglyph stereoscopy, but more confusing, because there wouldn’t be a color difference, just double vision.

i_am_iron_man____in_3d_by_homerjk85-d57gs7u — Let’s pray that poor Tony doesn’t have to wear anaglyph glasses in there.
(Props to Deviantartist homerjk85 for the awesome conversion.)

Nope.

So what we are left with is that we are not seeing anything in the real world of the diegesis. This 2° view is strictly a narrative conceit: A projection of what Tony’s brain puts together from the split views of the stereographic projection into a cohesive whole, i.e. retinally-projected augmentation of his eyesight. It’s a testament to the talent of the filmmakers that this HUD, as narratively constructed as it is, just works. We think it’s something real. We instantly get it. But…

The damned multilayering

But even that notion—that this HUD is what Tony experiences, perceptually—is troubled by the multilayering in the HUD. Information in the HUD is typically displayed across multiple layers. See the three squares in the left side of this screen shot for an example. So many problems with this. If this is meant to be what he perceives, then we immediately have trouble with parallax. Parallax is the way that objects shift against background objects when seen from two different viewpoints, like, say, Tony’s two eyes. If Tony perceives these layers through both eyes, i.e. stereoscopically, as an actual set of three layers floating in front of his face, then those graphics shift around depending on which eye JARVIS is optimizing for. One eye might see it beautifully, but then the other eye is wholly confounded. In the worst possible situation, neither eye is really satisfied. See the Wikipedia article on parallax as parallaxed for a meta-example. If on the other hand it’s just one eye that’s seeing these layers, then the layering is utterly pointless, because a single eye has no depth perception and therefore these would just appear as a single layer. It would have no benefit for Tony and only be there for our gee-whizification.

Our choices are: Terrible or Pointless

So, it’s either a terrible, confusing display for Tony (which I can’t imagine, given how genius of a technologist he is meant to be), or this view is not even a representation of what Tony sees, but a strictly narrative construction. And we can’t say for sure which it is because this multilayering is never seen in the first-person views. In those screens it’s been reasonably cleaned up to be intelligible. Note the difference between the car views below in the first- and second-person shots.

IronMan1_HUD11 — Layers include end views and a side view.

IronMan1_HUD10 — Only the side view is shown, the end views are absent.

Then, the damned head movement

Note also that in the 2nd-person view, Tony is very expressive, moving his head around a lot in response to the HUD. But looking at him from the outside, Iron Man’s head doesn’t swivel around except to look at things in the real world. Is the interface requiring him to move his head or is he just a drama queen? If it requires him, that’s terrible. That would move his head away from important things in the real world to focus on something in this virtual world? If he’s a drama queen, fine, nothing to do about that and glad that JARVIS can accomodate. In any case, when we see the him in the helmet outside the TARDIS-HUD, he is not swiveling his head apropos of nothing, which reinforces the notion that this is strictly a cinematic conceit. (Hat tip to Jonathan Korman for sharing this observation with me.)

So…

So ultimately what I’m saying here is this is an impossible thing, and for being impossible, we should not just freak out about how cool it is and declare it the necessary and good future. It has major problems, even as gorgeous and exciting as it is. Hey, no surprise, nobody has forgotten that it’s a movie, but recognize that what you thought was just maybe exaggerated was in fact a bold-faced impossibility.

Next up in the Iron HUD series: Iron Man forces us to get clear about some terms.

Sci-fi interfaces

Stop watching sci-fi. Start using it.

Tag Archives: pepper’s ghost