The holocircus

21 Dec 2015 by Christopher Noessel

To distract Lumpy while she tends to dinner, Malla sits him down at a holotable to watch a circus program. She leans down to one of the four control panels inset around the table’s edge, presses a few buttons, and the program begins.

In the program small volumetric projections of human (not Wookie) performers appear on the surface of the table and begin a dance and acrobatic performance to a soundtrack that is, frankly, ear-curdling.

A furry creature with long, greenish hair stands next to a patterned circular table, looking down with a curious expression.

A person observing a small dancer on a patterned floor, with a plant and furniture visible in the background.

Partway through the performance, Lumpy presses the second-from-the-left button, and the main dancer (who has just got to be a Radical Faerie, a full year before that august organization formed) disappears from the table only to become a life-sized projection just off the table, as if the figure had invisible legs but stood on the floor of the house.

Close-up view of a vintage computer keyboard integrated into a modern device, set against a blurred colorful background.

A close-up of a yellow, fluffy object being placed into a machine, with a blurred background.

When Lumpy is done with the performance, he presses a button, and RF guy disappears from the room and reappears in small size on the table to join the other performers on the table for a bow and curtain call before they all disappear and that ghastly, inhumane music stops.

A close-up of a furry, humanoid creature with a serious expression and large eyes, resembling a character from a science fiction universe.

A green ghostly figure levitates and dances above a table, while a furry creature observes in a room filled with shelves and plants.

So…yeah. This interface.

The terrible usability of tape recorders

It’s pretty clear the control panels are cassette recorders. I know that young readers won’t know what the pfassk that is, so for due diligence, let me explain. Those devices recorded and played back audio stored on magnetic tapes wound in cartridges. The controls for it were six stay-state toggles. These types of buttons have two states. Pushing one locks it down and sets the device to a mode: Play, Rewind, or Fast Forward. One special button, Record (often colored red) had to be chorded with the Play button as a safety precaution against accidentally destroying an existing recording. Users had to press a special Stop button to release any pressed button and stop the mode. Fast forward and Rewind could be pressed while audio was playing, but in such cases they would act as a momentary button. When any of the modes reached the end of a tape, higher-end recorders would also automatically release that button.

So let’s all admit that these are pretty terrible controls. From the fact that they almost all look the same to the fact that toggles make much more sense. (Press one button to play, press that same button again to make it stop.) But toggles weren’t the consumer model of the time. And this holocircus interface, hastily put together from things that the prop department could purchase off the shelf in the one day they were given to hack it together, inherits all that unusability and piles some more confusion onto it.

For instance

When we see Lumpy press the “Make the Dancing Man Bigger” mode, no other buttons are depressed. That means Malla had to set the circus mode, start the program, and press something like a stop button that released those. In turn that means if there was nothing on the table, it could either be because this was a blank spot in the media or that it wasn’t actually playing. You couldn’t tell, though, because the neither the controls nor the media didn’t reflect the state of the thing. This is probably not a common problem because we see that the garish programs and their terrible soundtracks are pretty effing hard to miss, but from a pure usability point of view, it sucks and should be avoided.

Now, there’s probably a thought exercise that could be done to figure out what keys mapped to which functions (and how), as well as second-guess the capabilities of the media (could he have picked any performer to enlarge, or was there just this option?) but honestly, it would be a Sisyphean multithreaded process of single-function vs. combined-letter arguments made by myself in a wall of text with little to no payoff for either of us. Knock yourself out in the comments if you want, but I’d rather focus on another aspect of the interface that I think does pay off, and that’s the visual language of the VP.

You know, for kids

This volumetric projection does not have most of the visual signals common to sci-fi hologram trope:

Edge-lighting
Blue cast
Scan lines
Projection rays

It does have translucency and an unusual scale, but that’s it. This visual language was actually established in Star Wars: Episode IV – A New Hope. So its absence here is conspicuous. As I noted in Make It So, these signals make it very clear to the observer that what they are observing is media—i.e. not real. Of course the real reason these VPs lack those signals is because this show had the budget of this blog post, but can we find a diegetic reason?

Surprisingly, I think yes. Kid’s media is different than regular media. Just like we pretend that Santa Claus is real for young children until they’re old enough to start to think critically about it, perhaps special effort is taken on these VPs to make it really immersive for the hairy kiddo. He doesn’t get the it’s-not-real signals because they want him to believe that it is.

Which is some fine holiday apologetics, if I do say so myself.

Grabby hologram

1 Dec 2015 by Christopher Noessel

After Pepper tosses off the sexy bon mot “Work hard!” and leaves Tony to his Avengers initiative homework, Tony stands before the wall-high translucent displays projected around his room.

Amongst the videos, diagrams, metadata, and charts of the Tesseract panel, one item catches his attention. It’s the 3D depiction of the object, the tesseract itself, one of the Infinity Stones from the MCU. It is a cube rendered in a white wireframe, glowing cyan amidst the flat objects otherwise filling the display. It has an intense, cold-blue glow at its center. Small facing circles surround the eight corners, from which thin cyan rule lines extend a couple of decimeters and connect to small, facing, inscrutable floating-point numbers and glyphs.

Wanting to look closer at it, he reaches up and places fingers along the edge as if it were a material object, and swipes it away from the display. It rests in his hand as if it was a real thing. He studies it for a minute and flicks his thumb forward to quickly switch the orientation 90° around the Y axis.

Then he has an Important Thought and the camera cuts to Agent Coulson and Steve Rogers flying to the helicarrier.

So regular readers of this blog (or you know, fans of blockbuster sci-fi movies in general) may have a Spidey-sense that this feels somehow familiar as an interface. Where else do we see a character grabbing an object from a volumetric projection to study it? That’s right, that seminal insult-to-scientists-and-audiences alike, Prometheus. When David encounters the Alien Astrometrics VP, he grabs the wee earth from that display to nuzzle it for a little bit. Follow the link if you want that full backstory. Or you can just look and imagine it, because the interaction is largely the same: See display, grab glowing component of the VP and manipulate it.

Two anecdotes are not yet a pattern, but I’m glad to see this particular interaction again. I’m going to call it grabby holograms (capitulating a bit on adherence to the more academic term volumetric projection.) We grow up having bodies and moving about in a 3D world, so the desire to grab and turn objects to understand them is quite natural. It does require that we stop thinking of displays as untouchable, uninterruptable movies and more like toy boxes, and it seems like more and more writers are catching on to this idea.

More graphics or more information?

Additionally, the fact that this object is the one 3D object in its display is a nice affordance that it can be grabbed. I’m not sure whether he can pull the frame containing the JOINT DARK ENERGY MISSION video to study it on the couch, but I’m fairly certain I knew that the tesseract was grabbable before Tony reached out.

On the other hand, I do wonder what Tony could have learned by looking at the VP cube so intently. There’s no information there. It’s just a pattern on the sides. The glow doesn’t change. The little glyph sticks attached to the edges are fuigets. He might be remembering something he once saw or read, but he didn’t need to flick it like he did for any new information. Maybe he has flicked a VP tesseract in the past?

Augmented “reality”

Rather, I would have liked to have seen those glyph sticks display some useful information, perhaps acting as leaders that connected the VP to related data in the main display. One corner’s line could lead to the Zero Point Extraction chart. Another to the lovely orange waveform display. This way Tony could hold the cube and glance at its related information. These are all augmented reality additions.

Augmented VP

Or, even better, could he do some things that are possible with VPs that aren’t possible with AR. He should be able to scale it to be quite large or small. Create arbitrary sections, or plan views. Maybe fan out depictions of all objects in the SHIELD database that are similarly glowy, stone-like, or that remind him of infinity. Maybe…there’s…a…connection…there! Or better yet, have a copy of JARVIS study the data to find correlations and likely connections to consider. We’ve seen these genuine VP interactions plenty of places (including Tony’s own workshop), so they’re part of the diegesis.

In any case, this simple setup works nicely, in which interaction with a cool media helps underscore the gravity of the situation, the height of the stakes. Note to selves: The imperturbable Tony Stark is perturbed. Shit is going to get real.

Stark Tower monitoring

6 Oct 2015 by Christopher Noessel

Since Tony disconnected the power transmission lines, Pepper has been monitoring Stark Tower in its new, off-the-power-grid state. To do this she studies a volumetric dashboard display that floats above glowing shelves on a desktop.

Volumetric elements

The display features some volumetric elements, all rendered as wireframes in the familiar Pepper’s Ghost (I know, I know) visual style: translucent, edge-lit planes. A large component to her right shows Stark Tower, with red lines highlighting the power traveling from the large arc reactor in the basement through the core of the building.

The center of the screen has a similarly-rendered close up of the arc reactor. A cutaway shows a pulsing ring of red-tinged energy flowing through its main torus.

This component makes a good deal of sense, showing her the physical thing she’s meant to be monitoring but not in a photographic way, but a way that helps her quickly locate any problems in space. The torus cutaway is a little strange, since if she’s meant to be monitoring it, she should monitor the whole thing, not just a quarter of it that has been cut away.

Flat elements

The remaining elements in the display appear on a flat plane.

To her upper left a spectrum analysis shows bars all near the middle, and the status is confirmed with a label as REACTOR OUTPUT NORMAL. Notably the bars are bright cyan at their top and fade to darker near their bases, drawing attention to the total arc. There few bands that are lower or higher than others are immediately visible from the contrast. That’s good visual design for attention management.

In the top center is a ring chart with three bands of colors: slate gray, cyan, and dark red. A large percentage read out in the center hovers around 90%. To her upper right are three other ring charts with the same color scheme. Above the three rings is a live trend line chart in red with a separate line in white (Target? Trend line?). A tiny side view of Stark Tower stands to the left of the three rings and trend line. The purpose of these are unclear as the labels are too small to read, but it seems strange to have the physical representation here. If the rings or trend lines are related to the physical tower, how they are related is unclear. If they are unrelated, having the rendering present there is misleading.

In a band across the middle are some tiny text blocks spitting out lines of data and auto scrolling. There are more of these, so for ease of reference, let’s call them teletype boxes. They look too small and move too fast to convey much meaning, as such things often do.

The lower left of the screen has a dim white section labeled MAGNETIC CONTAINMENT. It contains a 3D cuboid with a wildly undulating surface. As this shape doesn’t match anything else in the display, nor is it visually connected, its utility is unclear. If she is meant to track some variable other than undulationness, I can’t see how this would help. Also in this panel are some sliders wildly sliding back and forth, and some more teletype boxes.

In the lower center part of the screen is an overhead-view wireframe of the reactor that does not change in this segment, as well as a dim white box that appears to contain a static formula.

This display is not fit to Pepper’s task

Sadly, Gwen Paltrow looks a little wall-eyed here, so it’s hard to tell where exactly where she’s looking to determine as she says that “Levels are holding steady.” Her gaze appears roughly above the large ring chart, so let’s presume she’s looking there. If she is actually meant to be monitoring a set of levels to determine if they’re fluctuating or not, the ring chart and percentage are the wrong display types. These are valuable for showing a state that doesn’t change very often, i.e. where change over time is not of concern.

Sure, she might be able to suss out the steadiness of the variables by focusing on the percentage and the ring chart elements to see if they’re currently wobbling, but that requires her to keep her focus there to observe what’s happening with those pixels. That puts an unnecessary burden on her attention and short term memory. What if she has to glance away for a few seconds, and in those seconds, levels begin to wobble unsteadily? Sure, an alert could sound, but when she looks back she’ll have no idea what has happened while her attention was elsewhere. Humane interfaces remove unnecessary burdens from their users, and this display should do the same.

Use line charts and sparklines to show change over time

For any variable that is meant to be monitored for changes over time—like the scripted steadiness of levels—the line chart or sparkline is much more fit to task. Each plots variables over time, so even if a user glances away for a bit, it’s not a problem, she can look back and quickly suss out what’s been happening. The thing she’s looking at should look more like a ECG than a page out of an annual report.

The Iron Man HUD is an impossible thing

29 Jul 2015 by Christopher Noessel

In the prior post we looked at the HUD display from Tony’s point of view. In this post we dive deeper into the 2nd-person view, which turns out to be not what it seems.

The HUD itself displays a number of core capabilities across the Iron Man movies prior to its appearance in The Avengers. Cataloguing these capabilities lets us understand (or backworld) how he interacts with the HUD, equipping us to look for its common patterns and possible conflicts. In the first-person view, we saw it looked almost entirely like a rich agentive display, but with little interaction. But then there’s this gorgeous 2nd-person view.

When in the first film Tony first puts the faceplate on and says to JARVIS, “Engage heads-up display”… …we see things from a narrative-conceit, 2nd-person perspective, as if the helmet were huge and we are inside the cavernous space with him, seeing only Tony’s face and the augmented reality interface elements. You might be thinking, “Of course it’s a narrative conceit. It’s not real. It’s in a movie.” But what I mean by that is that even in the diegesis, the Marvel Cinematic World, this is not something that could be seen. Let’s move through the reasons why.

Not a mini-TARDIS

First, it looks like we’re in some TARDIS-like space where the helmet extends so far we can fit in it, or a camera can, about a meter from his face. But of course the helmet isn’t huge on the inside. Tony hasn’t broken those laws of physics. The helmet is helmet-sized on the inside.

Not a volumetric projection

Then there’s the issue of the huge display. It looks like a volumetric projection, like what R2-D2 can project, but that can’t be true, either. The projection would extend way beyond the boundaries of the helmet-sized helmet. Which as you can see below, is a non-starter. So it’s not a volumetric projection.

So, retinal projection

Then what is the display technology? Given the size constraints, retinal projection makes the most sense, but if we could make the helmet go invisible, it would look like Tony was having diffuse LASIK, or maybe playing The Game from Star Trek: The Next Generation.

STTNG The Game-02 — Let’s face it, this is not the worst thing you’ve caught me doing.

Representation of the projections?

So, OK, fine. Maybe what we see is what’s being projected, the separate stereoscopic images onto individual retinas. Nope. Then we would see two similar, slightly offset images, like in older anaglyph stereoscopy, but more confusing, because there wouldn’t be a color difference, just double vision.

i_am_iron_man____in_3d_by_homerjk85-d57gs7u — Let’s pray that poor Tony doesn’t have to wear anaglyph glasses in there.
(Props to Deviantartist homerjk85 for the awesome conversion.)

Nope.

So what we are left with is that we are not seeing anything in the real world of the diegesis. This 2° view is strictly a narrative conceit: A projection of what Tony’s brain puts together from the split views of the stereographic projection into a cohesive whole, i.e. retinally-projected augmentation of his eyesight. It’s a testament to the talent of the filmmakers that this HUD, as narratively constructed as it is, just works. We think it’s something real. We instantly get it. But…

The damned multilayering

But even that notion—that this HUD is what Tony experiences, perceptually—is troubled by the multilayering in the HUD. Information in the HUD is typically displayed across multiple layers. See the three squares in the left side of this screen shot for an example. So many problems with this. If this is meant to be what he perceives, then we immediately have trouble with parallax. Parallax is the way that objects shift against background objects when seen from two different viewpoints, like, say, Tony’s two eyes. If Tony perceives these layers through both eyes, i.e. stereoscopically, as an actual set of three layers floating in front of his face, then those graphics shift around depending on which eye JARVIS is optimizing for. One eye might see it beautifully, but then the other eye is wholly confounded. In the worst possible situation, neither eye is really satisfied. See the Wikipedia article on parallax as parallaxed for a meta-example. If on the other hand it’s just one eye that’s seeing these layers, then the layering is utterly pointless, because a single eye has no depth perception and therefore these would just appear as a single layer. It would have no benefit for Tony and only be there for our gee-whizification.

Our choices are: Terrible or Pointless

So, it’s either a terrible, confusing display for Tony (which I can’t imagine, given how genius of a technologist he is meant to be), or this view is not even a representation of what Tony sees, but a strictly narrative construction. And we can’t say for sure which it is because this multilayering is never seen in the first-person views. In those screens it’s been reasonably cleaned up to be intelligible. Note the difference between the car views below in the first- and second-person shots.

IronMan1_HUD11 — Layers include end views and a side view.

IronMan1_HUD10 — Only the side view is shown, the end views are absent.

Then, the damned head movement

Note also that in the 2nd-person view, Tony is very expressive, moving his head around a lot in response to the HUD. But looking at him from the outside, Iron Man’s head doesn’t swivel around except to look at things in the real world. Is the interface requiring him to move his head or is he just a drama queen? If it requires him, that’s terrible. That would move his head away from important things in the real world to focus on something in this virtual world? If he’s a drama queen, fine, nothing to do about that and glad that JARVIS can accomodate. In any case, when we see the him in the helmet outside the TARDIS-HUD, he is not swiveling his head apropos of nothing, which reinforces the notion that this is strictly a cinematic conceit. (Hat tip to Jonathan Korman for sharing this observation with me.)

So…

So ultimately what I’m saying here is this is an impossible thing, and for being impossible, we should not just freak out about how cool it is and declare it the necessary and good future. It has major problems, even as gorgeous and exciting as it is. Hey, no surprise, nobody has forgotten that it’s a movie, but recognize that what you thought was just maybe exaggerated was in fact a bold-faced impossibility.

Next up in the Iron HUD series: Iron Man forces us to get clear about some terms.

The bug VP

9 Apr 2014 by Christopher Noessel

In biology class, the (unnamed) professor points her walking stick (she’s blind) at a volumetric projector. The tip flashes for a second, and a volumetric display comes to life. It illustrates for the class what one of the bugs looks like. The projection device is a cylinder with a large lens atop a rolling base. A large black plug connects it to the wall.

The display of the arachnid appears floating in midair, a highly saturated screen-green wireframe that spins. It has very slight projection rays at the cylinder and a "waver" of a scan line that slowly rises up the display. When it initially illuminates, the channels are offset and only unify after a second.

STARSHIP_TROOPERS_vdisplay

The top and bottom of the projection are ringed with tick lines, and several tick lines runs vertically along the height of the bug for scale. A large, lavender label at the bottom identifies this as an ARACHNID WARRIOR CLASS. There is another lavendar key too small for us to read.The arachnid in the display is still, though the display slowly rotates around its y-axis clockwise from above. The instructor uses this as a backdrop for discussing arachnid evolution and "virtues."

After the display continues for 14 seconds, it shuts down automatically.

STARSHIP_TROOPERS_vdisplay2

Interaction

It’s nice that it can be activated with her walking stick, an item we can presume isn’t common, since she’s the only apparently blind character in the movie. It’s essentially gestural, though what a blind user needs with a flash for feedback is questionable. Maybe that signal is somehow for the students? What happens for sighted teachers? Do they need a walking stick? Or would a hand do? What’s the point of the flash then?

That it ends automatically seems pointlessly limited. Why wouldn’t it continue to spin until it’s dismissed? Maybe the way she activated it indicated it should only play for a short while, but it didn’t seem like that precise a gesture.

Of course it’s only one example of interaction, but there are so many other questions to answer. Are there different models that can be displayed? How would she select a different one? How would she zoom in and out? Can it display aimations? How would she control playback? There are quite a lot of unaddressed details for an imaginative designer to ponder.

Display

The display itself is more questionable.

Scale is tough to tell on it. How big is that thing? Students would have seen video of it for years, so maybe it’s not such an issue. But a human for scale in the display would have been more immediately recognizable. Or better yet, no scale: Show the thing at 1:1 in the space so its scale is immediately apparent to all the students. And more appropriately, terrifying.

And why the green wireframe? The bugs don’t look like that. If it was showing some important detail, like carapice density, maybe, but this looks pretty even. How about some realistic color instead? Do they think it would scare kids? (More than the “gee-whiz!” girl already is?)

And lastly there’s the title. Yes, having it rotate accomodates viewers in 360 degrees, but it only reads right for half the time. Copy it, flip it 180º on the y-axis, and stack it, and you’ve got the most important textual information readable at most any time from the display.

Better of course is more personal interaction, individual displays or augmented reality where a student can turn it to examine the arachnid themselves, control the zoom, or follow up on more information. (Wnat to know more?) But the school budget in the world of Starship Troopers was undoubtedly stripped to increase military budget (what a crappy world that would be amirite?), and this single mass display might be more cost effective.

Pilot seat

25 Jan 2013 by Christopher Noessel

The reawakened alien places his hand in the green display and holds it there for a few seconds. This summons a massive pilot seat. If the small green sphere is meant to be a map to the large cyan astrometric sphere, the mapping is questionable. Better perhaps would be to touch where the seat would appear and lift upwards through the sphere.

He climbs into the seat and presses some of the “egg buttons” arrayed on the armrests and on an oval panel above his head. The buttons illuminate in response, blinking individually from within. The blink pattern for each is regular, so it’s difficult to understand what information this visual noise conveys. A few more egg presses re-illuminate the cyan astrometric display.

A few more presses on the overhead panel revs up the spaceship’s engines and seals him in an organic spacesuit. The overhead panel slowly advances towards his face. The purpose for this seems inexplicable. If it was meant to hold the alien in place, why would it do so with controls? Even if they’re just navigation controls that no longer matter since he is on autopilot, he wouldn’t be able to take back sudden navigation control in a crisis. If the armrest panels also let him navigate, why are the controls split between the two parts?

On automatic at this point, the VP traces a thin green arc from the chair to the VP earth and adds highlight graphics around it. Then the ceiling opens and the spaceships lifts up into the air.

Alien Astrometrics

22 Jan 2013 by Christopher Noessel

When David is exploring the ancient alien navigation interfaces, he surveys a panel, and presses three buttons whose bulbous tops have the appearance of soft-boiled eggs. As he presses them in order, electronic clucks echo in in the cavern. After a beat, one of the eggs flickers, and glows from an internal light. He presses this one, and a seat glides out for a user to sit in. He does so, and a glowing pollen volumetric projection of several aliens appears. The one before David takes a seat in the chair, which repositions itself in the semicircular indentation of the large circular table.

The material selection of the egg buttons could not be a better example of affordance. The part that’s meant to be touched looks soft and pliable, smooth and cool to the touch. The part that’s not meant to be touched looks rough, like immovable stone. At a glance, it’s clear what is interactive and what isn’t. Among the egg buttons there are some variations in orientation, size, and even surface texture. It is the bumpy-surfaced one that draws David’s attention to touch first that ultimately activates the seat.

The VP alien picks up and blows a few notes on a simple flute, which brings that seat’s interface fully to life. The eggs glow green and emit green glowing plasma arcs between certain of them. David is able to place his hand in the path of one of the arcs and change its shape as the plasma steers around him, but it does not appear to affect the display. The arcs themselves appear to be a status display, but not a control.

After the alien manipulates these controls for a bit, a massive, cyan volumetric projection appears and fills the chamber. It depicts a fluid node network mapped to the outside of a sphere. Other node network clouds appear floating everywhere in the room along with objects that look like old Bohr models of atoms, but with galaxies at their center. Within the sphere three-dimensional astronomical charts appear. Additionally huge rings appear and surround the main sphere, rotating slowly. After a few inputs from the VP alien at the interface, the whole display reconfigures, putting one of the small orbiting Bohr models at the center, illuminating emerald green lines that point to it and a faint sphere of emerald green lines that surround it. The total effect of this display is beautiful and spectacular, even for David, who is an unfeeling replicant cyborg.

At the center of the display, David observes that the green-highlighted sphere is the planet Earth. He reaches out towards it, and it falls to his hand. When it is within reach, he plucks it from its orbit, at which point the green highlights disappear with an electronic glitch sound. He marvels at it for a bit, turning it in his hands, looking at Africa. Then after he opens his hands, the VP Earth gently returns to its rightful position in the display, where it is once again highlighted with emerald, volumetric graphics.

Finally, in a blinding flash, the display suddenly quits, leaving David back in the darkness of the abandoned room, with the exception of the small Earth display, which is floating over a small pyramid-shaped protrusion before flickering away.

After the Earth fades, david notices the stasis chambers around the outside of the room. He realizes that what he has just seen (and interacted with) is a memory from one of the aliens still present.

Hilarious and insightful Youtube poster CinemaSins asks in the video “Everything Wrong with Prometheus in 4 minutes or Less,” “How the f*ck is he holding the memory of a hologram?” Fair question, but not unanswerable. The critique only stands if you presume that the display must be passive and must play uninterrupted like a television show or movie. But it certainly doesn’t have to be that way.

Imagine if this is less like a YouTube video, and more like a playback through a game engine like a holodeck StarCraft. Of course it’s entirely possible to pause the action in the middle of playback and investigate parts of the display, before pressing play again and letting it resume its course. But that playback is a live system. It would be possible to run it afresh from the paused point with changed parameters as well. This sort of interrupt-and-play model would be a fantastic learning tool for sensemaking of 4D information. Want to pause playback of the signing of the Magna Carta and pick up the document to read it? That’s a “learning moment” and one that a system should take advantage of. I’d be surprised if—once such a display were possible—it wouldn’t be the norm.

The only thing I see that’s missing in the scene is a clear signal about the different state of the playback:

As it happened
Paused for investigation
Playing with new parameters (if it was actually available)

David moves from 1 to 2, but the only change of state is the appearance and disappearance of the green highlight VP graphics around the Earth. This is a signal that could easily be missed, and wasn’t present at the start of the display. Better would be some global change, like a global shift in color to indicate the different state. A separate signal might compare As it Happened with the results of Playing with new parameters, but that’s a speculative requirement of a speculative technology. Best to put it down for now and return to what this interface is: One of the most rich, lovely, and promising examples of sensemaking interactions seen on screen. (See what I did there?)

For more about how VP might be more than a passive playback, see the lesson in Chapter 4 of Make It So, page 84, VP Systems Should Interpret, Not Just Report.

Mission Briefing

20 Dec 2012 by Christopher Noessel

Once the Prometheus crew has been fully revived from their hypersleep, they gather in a large gymnasium to learn the details of their mission from a prerecorded volumetric projection. To initiate the display, David taps the surface of a small tablet-sized handheld device six times, and looks up. A prerecorded VP of Peter Weyland appears and introduces the scientists Shaw and Holloway.

This display does not appear to be interactive. Weyland does mention and gesture toward Shaw and Holloway in the audience, but they could have easily been in assigned seats.

Cue Rubik’s Space Cube

After his introduction, Holloway places an object on the floor that looks like a silver Rubik’s Cube with a depressed black button in the center-top square.

Prometheus-055

He presses a middle-edge button on the top, and the cube glows and sings a note. Then a glowing-yellow “person” icon appears, glowing, at the place he touched, confirming his identity and that it’s ready to go.

He then presses an adjacent corner button. Another glowing-yellow icon appears underneath his thumb, this one a triangle-within-a-triangle, and a small projection grows from the side. Finally, by pressing the black button, all of the squares on top open by hinged lids, and the portable projection begins. A row of 7 (or 8?) “blue-box” style volumetric projections appear, showing their 3D contents with continuous, slight rotations.

Gestural control of the display

After describing the contents of each of the boxes, he taps the air towards either end of the row (there is a sparkle-sound to confirm the gesture) and he brings his middle fingers together like a prayer position. In response, the boxes slide to a center as a stack.

He then twists his hands in opposite directions, keeping the fingerpads of his middle fingers in contact. As he does this, the stack merges.

Prometheus-070

Then a forefinger tap summons an overlay that highlights a star pattern on the first plate. A middle finger swipe to the left moves the plate and its overlay off to the left. The next plate automatically highlights its star pattern, and he swipes it away. Next, with no apparent interaction, the plate dissolves in a top-down disintegration-wind effect, leaving only the VP spheres that illustrate the star pattern. These grow larger.

Halloway taps the topmost of these spheres, and the VP zooms through intersteller space to reveal an indistinct celestial sphere. He then taps the air again (nothing in particular is beneath his finger) and the display zooms to a star. Another tap zooms to a VP of LV-223.

After a beat of about 9 seconds, the presentation ends, and the VP of LV-223 collapses back into its floor cube.

Evaluating the gestures

In Chapter 5 of Make It So we list the seven pidgin gestures that Hollywood has evolved. The gestures seen in the Mission Briefing confirm two of these: Push to Move and Point to Select, but otherwise they seem idiosyncratic, not matching other gestures seen in the survey.

That said, the gestures seem sensible. On tapping the “bookends” of the blue boxes, Holloway’s finger pads come to represent the extents of the selection, so bringing them together is a reasonable gesture to indicate stacking. The twist gesture seems to lock the boxes in place, to break the connection between them and his fingertips. This twist gesture turns his hand like a key in a lock, so has a physical analogue.

It’s confusing that a tap would perform four different actions (highlight star patterns in the blue boxes, zoom to the celestial sphere, zoom to star, zoom to LV-223) but there is no indication that this is a platform for manipulating VPs as much as it is a presentation software. With this in mind he could arbitrarily assign any gesture to simply “advance the slide.”

Alien VPs

18 Dec 2012 by Christopher Noessel

In the volumetric projection chapter of Make It So, we note that sci-fi makers take pains to distinguish the virtual from the real most often with a set of visual treatments derived from the “Pepper’s Ghost” parlor trick, augmented with additional technology cues: translucency, a blue tint, glowing whites, supersaturated colors for wireframed objects, clear pixels and/or flicker, with optional projection rays.

Prometheus has four types of VPs that adhere to this style in varying degrees. Individual displays (with their interactions) are discussed in other posts. This collection of posts compares their styles. This particular post describes the alien VPs.

The two alien VPs are quite different from the human VPs in appearance and behavior. The first thing to note is that they adhere to the Pepper’s Ghost style more readily, with glowing blue-tinted whites and transparency. Beyond that they differ in precision and implied technology.

Precision VPs

The first style of alien VP appears in the bridge of the alien vessel, where projection technology can be built into the architecture. The resolution is quite precise. When the grapefruit-sized Earth gets close to the camera in one scene, it appears to have infinite resolution, even though this is some teeny tiny percentage of the whole display.

Glowing Pollen

The other alien VP tech is made up of small, blue-white voxels that float, move in space, obey some laws of physics, and provide a crude level of resolution. These appear in the caves of the alien complex where display tech is not present in the walls, and again as “security footage” in the bridge of the alien ship. Because the voxels obey some laws of physics, it’s easier to think of them as glowing bits of pollen.

Pollen behavior

These voxels appear to not be projections of light in space, but actual motes that float through the air. When David activates the “security footage” in the alien complex, a wave of this pollen appears and flows past him. It does not pass through him, but collides with him, each collided mote taking a moment to move around him and regain its roughly-correct position in the display. (How it avoids getting in his mouth is another question entirely.) The motes even produce a gust of wind that disturb David’s bleached coif.

Pollen inaccuracy

The individual lines of pollen follow smooth arcs through the air, but lines appear to be slightly off from one another.

This style is beautiful and unique, and conveys a 3D display technology that can move to places even where there’s not a projector in line of sight. The sci-fi makers of this speculative technology use this inaccuracy to distinguish it from other displays. But if a precise understanding of the shapes being described is useful to its viewers, of course it would be better if the voxels were more precisely positioned in space. That’s a minor critique. The main critique of this display is when it gets fed back into the human displays as an arbitrary style, as I’ll discuss in the next post about the human-tech, floating-pixel displays.

Human VPs

17 Dec 2012 by Christopher Noessel

Blue-box displays

One type of human-technology VPs are the blue-box displays:

David’s language program
Halloway and Shaw’s mission briefing
The display in Shaw’s quarters

These adhere more closely to the Pepper’s Ghost style, being contained in a translucent blue cuboid with saturated surface graphics and a grid pattern on the sides.

Weyland-Yutani VP

The other type of human displays are the Weyland-Yutani VPs. These have translucency and supersaturated wireframes, but they do not have any of the other conventional Pepper’s Ghost cues. Instead they add two new visual cues to signal to the audience their virtualness: scaffolded transitions and edge embers.

When a Weyland-Yutani VP is turned on, it does not simply blink into view. It builds. First, shapes are described in space as a tessellated surface, made of yellow-green lines describing large triangles that roughly describe the forthcoming object or its extents. These triangles have a faint smoky-yellow pattern on their surface. Some of the lines have yellow clouds and bright red segments along their lengths. Additionally, a few new triangles extend to a point space where another piece of the projection is about to appear. Then the triangles disappear, replaced with a fully refined image of the 3D object. The refined image may flicker once or twice before settling into persistence. The whole scaffolding effect is staggered across time, providing an additional sense of flicker to the transition.

Motion in resolved parts of the VP begins immediately, even as other aspects of the VP are still transitioning on.

When a VP is turned off, this scaffolding happens in reverse, as elements decay into tessellated yellow wireframes before flickering out of existence.

Edge embers

A line of glowing, flickering, sliding, yellow-green points illustrates the extents of the VP area, where a continuous surface like flooring is clipped at the limits of the display. These continue across the duration of the playback.

A growing confidence in audiences

This slightly different strategy to distinguishing VPs from the real world indicates the filmmaker’s confidence that audiences are growing familiar enough with this trope that fewer cues are needed during the display. In this case the translucency and subtle edge embers are the only persistent cues, pushing the major signals of the scaffolding and surface flicker to the transitions.

If this trend continues and sci-fi makers become overconfident, it may confuse some audiences, but at the same time give the designers of the first real-world VPs more freedom with their appearance. They wouldn’t have to look like Star Wars’.

Something new: Projected Reflectance

One interesting detail is that when we see Vickers standing in the projection of Weyland’s office, she casts a slight reflection in the volumetric surface. It implies a technology capable of projecting not just luminance, but reflectivity as well. The ability to project volumetric mirrors hasn’t appeared before in the survey.

Lesson: Transition by importance

Another interesting detail is that when the introduction to the Mission briefing ends, the environment flickers out first, then the 2D background, then Weyland’s dog, then finally Weyland.

This order isn’t by position, brightness, motion, or even surface area (the dog confounds that.) It is by narrative importance: Foreground, background, tertiary character, primary character. The fact that the surrounding elements fade first keep your eyes glued onto the last motion (kind of like watching the last bit of sun at a sunset), which in this order is the most important thing in the feed, i.e. the human in view. If a staggered-element fade-out becomes a norm in the real world for video conferencing (or eventually VP conferencing), this cinematic order is worth remembering.