Glossary: Facing, Off-facing, Lengthwise, and Edgewise

As part of the ongoing review of the Iron Man HUD, I noticed a small feature in the Iron Man 3 UI 2nd-person UI that—in order to critique—I have to discuss some new concepts and introduce some new terms. The feature itself is genuinely small and almost not worth posting about, but the terms are interesting, so bear with me.

Most of the time JARVIS animates the HUD, the UI elements sit on an invisible sphere that surrounds his head. (And in the case of stacked elements, on concentric invisible spheres.) The window of Pepper in the following screenshot illustrates this pretty clearly. It is a rectangular video feed, but appears slightly bowed to us, being on this sphere near the periphery of this 2nd-person view.

IronMan3_HUD68

…And Pepper Potts is up next with her op-ed about the Civil Mommy Wars. Stay tuned.

Having elements slide around on the surface of this perceptual sphere is usable for Tony, since it means the elements are always facing him and thereby optimally viewable. “PEPPER POTTS,” for example, is as readable as if it was printed on a book perpendicular to his line of sight. (This notion is a bit confounded by the problems of parallax I wrote about in an earlier post, but since that seems unresolvable until Wim Wouters implements this exact HUD on Oculus Rift, let’s bypass it to focus on the new thing.)

So if it’s visually optimal to have 2D UI elements plastered to the surface of this perceptual sphere, how do we describe that suboptimal state where these same elements are not perpendicular to the line of sight, but angled away? I’m partly asking for a friend named Tony Stark because that’s some of what we see in Iron Man 3, both in 1st- and 2nd-person views. These examples aren’t egregious.

IronMan3_HUD44

The Iron Patriot debut album cover graphic is only slightly angled and so easy to read. Similarly, the altimeter thingy on the left is still wholly readable.

Continue reading

Advertisements

The Ultimate Weapon

TheFifthElement-UWeapon-002

The most interesting interface in the film belongs to the Ultimate Weapon, because it raises such unusual challenges to interface design.

The Design Challenge

According to the movie, the Ultimate Evil arrives every 5000 years, and this is the only time the weapon needs to be fired. (Its prior firing would be around 2737 B.C.E., and if it was on Earth before then, in prehistory.) Its designers must ensure that it will be usable to users separated by around 250 (human) generations. Given such an expanse of time, how can a designer ensure that any necessary inputs will be available between potential uses? What materials will survive that long to ensure structural and functional viability? What written instructions can survive the vast changes in language and cultural contexts? How can you ensure that spoken instructions or principles will be passed down accurately from generation to generation? Presuming some lossy transmission, what clues can you give in the interface itself as to the intended use?

Mondoshawan physiology

Fortunately, the Mondoshawan physiology is not a substantial problem. In their suits they are still similarly-sized, bilateral, upright bipeds with a head where sensory organs are clustered at the top, and, emerging from the tops of their torsos, prehensile arms at the end of which are manipulator digits. This solves a great deal of what could be difficult interspecies issues. Imagine, for contrast, trying to design an interface usable by intelligent versions of both butterflies and cephalopods. Not easy. But an interface for two humanoid species: Much less difficult.

How to ensure the interface material lasts?

Certainly, the system must maintain some physical integrity over time. Passing over the creative license of “advanced alien technologies,” we see that the material for the weapon is quite-long lasting, i.e. stone and in the case of the key, metal. Additionally, the weapon is kept in a temple in the desert, a non-volatile environment suited to preserving such materials.

TheFifthElement-temple

There are materials for the stones that could last longer and be more resistant to damage, like metal or industrial ceramics, but we do not know anything about the provenance of the weapon, and whether such materials were available.

How to hide the weapon from malefactors?

In the words of Cornelius, an evil person could stand on the platform and activate the weapon to “turn light to dark.” No one wants that to happen. The Mondoshawans hide the weapon in the Egyptian temple, and take pains to carefully conceal the presence of the door to the weapon room and its keyhole. Ordinarily Mondoshawans keep the key to the door of the room which houses the weapon to themselves offworld, but when they take the stones for protection, they leave the key with a member of a sect that worships the weapon, ensuring that the key is passed down through the generations along with the weapon’s instructions.

How to ensure the instructions persist?

Even with durable materials, if the use of the weapon isn’’t so completely intuitive as to be automatic, the instructions on how to activate it must endure transmission through time, across the lives of generations of people (and Mondoshawans). In this case, the instruction set is fairly simple; one must have access to “the” five elements for the weapon to work. Four are the familiar classical alchemical elements of earth, air, fire, and water. These are represented in the movie by four patterns of lines. The lines have subtle variations that reflect physical properties of that element. Earth was flat horizontal lines. Water was wavy horizontal lines at the base. Air was wavy horizontal lines at the top. Fire was vertical wavy lines.

The simplicity, replicability, and memetic nature of this part of the instruction set is demonstrated as we see the symbol repeated in a number of places: on the walls of the pyramid, on the sides of the stones, on the pedestals to which each stone fits, on Cornelius’’ belt buckle, and as a mark on Leeloo’’s skin. Had these symbols been more complex in nature, there would have been more risk that they would have shifted and evolved, like language does, beyond recognition and therefore use as a clue to the weapon’s function.

The instructions are also kept alive through the ages via myth and religious fervor. The characters Cornelius and David belong to a sect devoted to the Ultimate Weapon. This is clever cultural design. Humans have historically demonstrated a desire to worship, and the Mondoshawans have taken advantage of this, providing the Ultimate Weapon a group of people wholly dedicated to its preservation regardless of whether or not their generation is the one to see it fire. The rites, rituals, and artifacts of this religion that act as a backup for the instructions on firing the Ultimate Weapon, as we see when Cornelius tries to explain it all to the President.

The transmission media of memes and religious fervor are not—as we see in the film—perfect. Language and culture are lossy media. But they do get the characters close enough so that they can figure out the rest on their own.

How to make sure it can be figured out?

The weapon is initialized by placing the sacred stones on the proper pedestals. But which stones go on which pedestal? Fortunately, anyone with a visual or tactile sense can match the right stone to the right pedestal by matching the pattern. Furthermore, since the stones and their fitting are almost triangular, it is easy to tell how they should be seated. See the pilot of Sci-Fi University for more about these affordances and constraints.

SciFiU-5E-title

The main challenge within this part of the bigger challenge is the spans of time involved. Given 5,000 years between firings, entire cultures, countries, technologies, and languages come and go in that time. How many people alive now are fluent in languages from 5 millennia past? You have to use mechanisms that don’t depend on culture, technology, or language. Physical affordances and constraints are a fine tool for these reasons.

How to let users know they’re on the right track?

When a little bit of the required element is provided to the placed stones, there is immediate feedback as small rectangles open just a bit near the tops. It is this partway state that indicates to the protagonists that, even though they haven’t completely supplied enough material, they are on the right track. This clue gives them enough of a signal that they continue trying to deduce control of the interface.

TheFifthElement-progress-01

What activation materials to use?

The stones require some small amount of each element to be supplied to their topmost surface to become active. For three of the four (earth, air, water), these elements are in abundance here on Earth.

To consider the fourth, fire, takes us to strategic questions about the design.

Why this design?

It’s possible that the design of the weapon is constrained by some unknown cosmogenic power source in the stones. <handwaving>It’s mystical physics that requires that there be 4 stones and 4 pillars and smooches in the center.</handwaving> But it is of course of more use to us to imagine that it wasn’t, but some deliberate design. Which leads me to ask why wasn’t it a single big button? Well, I can see five effects this particular design has.

1. It allows you to disable the weapon

A major part of the plot involves the fact that the stones—keys to operating the machine—have been removed from Earth to keep them safe. This proves to be a major complication and a minor mystery to the protagonists, but is in fact one of main features of the weapon. Much of it is architectural and would be very difficult to move. By adding activation keys, the Mondoshawans ensured that they could disable it if necessary.

2. It tests for environmental stewardship

If three of the activation elements were not available: earth, air, and water, it would raise serious issues about the human caretakers of the planet. Do they stand on a scorched earth? Is their air ruined? Have they let the water of Earth, like what happened on Mars, evaporate into space? Any of these scenarios raise serious doubts about whether life on the planet is worth saving. Or is there to save.

3. It tests for cultural stewardship

Unlike the other elements, fire isn’t as abundant. In pre-cultural Earth, it was an accident of geothermal activity and lightning. To be able to control it to a level that it can be applied to the stone speaks of a fundamental level of cultural and technological advancement. If humans have not kept stewardship of their culture well enough to be able to control fire, it again raises the question of whether they are worth saving.

The key to the weapon room similarly tests for cultural stewardship. It looks like a fragile thing, made of thin perforated metal. Having a reverant group treat it as a holy artifact ensures that it will not get crushed or rusted, and in the process lose access to the room that contains the weapon.

4. It tests for basic intelligence

The affordances and constraints that help the characters position the stones correctly require a level of basic, intelligence as individuals. Can they do pattern matching? Do they understand simple physics? This isn’t the strongest of tests, but I’m pretty had humans devolved to primates by this point or distracted by constant war, they’d have been screwed.

5. It tests for a capacity for love

The “fifth element” (ignoring wu xing and similar actual 5-element philosophies) in this case is love. In the film Korben must overcome his reticence to confess his love for Leeloo. When he does, she realizes that humanity—including its capacity for war—are worth saving, and the weapon fires. Love is a big word of course, so it’s not clear whether familial, friendly, platonic, or even purely sexual love would suffice, but perhaps it doesn’t matter. The designers wanted to make sure that humanity still has some capacity for feeling intense care toward another. If not, why bother saving them?

It’s made a bit dubious because it’s specifically for the love of an “ultimate warrior,” a “perfect being.” Leeloo looks very much like a very fit, pretty example of one of a human, who has shown very human capacities for joy, pain, fear, delight, &c. It’s not that hard of a test for Korben to love her, except, to overcome his own sense of awkwardness and humility and openness to rejection (in front of a small crowd, no less.) If she had looked like a Mangalore it would have been a more difficult—and more telling—test of the capacity for altruistic love, but perhaps that’s not the point.

These five effects seem like pretty good reasons to design the interface to this weapon in this particular way. In total, they test to make sure there’s a humanity there worth saving. And fortunately for humanity in 2263, Korben (and the culture that produced him) prove just enough of a match.

TheFifthElement-UWeapon-008

As if that wasn’t enough, bear with me for just two more bits of nerdery about the weapon. These are a bit extraneous to the interface, but derived from study of the interface, and so may be of note to readers.

1. We can’t ignore the fact that the Ultimate Evil plummets toward the Earth in a straight line. A straight line, that is, that puts it directly in the path of the ultimate weapon, which fires a perfectly straight line. And recall that the weapon is on a planet that’s orbiting around a star, and precessing its rotational axis. This is too slim a chance to be coincidence. It stands to reason that this is not, as Cornelius says, a sentient evil bent on ending “all life” (which would just veer a few degrees out of the way to safety), but part of the same system as the weapon, designed to identify and tempt the worst of people, i.e. Zorg, and try and thwart these aspects of humanity that are ultimately tested. If that’s the case, and the Mondoshawans installed the weapon, did they, by extension, install the Ultimate Evil as well? Is this some sort of “invisible fence” meant to keep humanity in check, and destroy it if it ever evolves for the worse?

2. Many of these same issues have been addressed in the real world by the designers of containers of radioactive waste (the danger of which persists between 10,000 and 1,000,000 years) and, more positively, the the Long Now Foundation working on its main project, the Millennium Clock. For those unfamiliar with this project, it is a prospective, large-scale clock that once built, will chime every thousand years. The clock mechanism and function is intended to last for 10,000 years. The Long Now foundation is faced with similar long-term design challenges and have come to some similar conclusions as the designers for the film. The clock will be made of Bronze Age materials and technology, and it will be situated in the desert. The clock will largely be self-maintaining, but the Foundation is also developing a Rosetta Wheel containing many, many examples of existing human language, useful for decoding written instructions. The idea itself has many elements that ensure its persistence as a meme, being simple, distinct, and a powerful embodiment of an important message about the value of long-term thinking. The Long Now Foundation was begun in 01996, the year prior to the release of The Fifth Element. I am a huge fan of the Foundation and its initiatives, and I encourage readers to read further to learn more.

Who did it better? Fingernail-o-matic edition

The Fifth Element

fifthelement-088

When in The Fifth Element the Mangalore Aknot calls Zorg to report that the “mission is accomplished,” we get a few seconds of screen time with Zorg’s secretary who receives the call. During this moment, she’s a bit bored, and idly shoves a finger into a small, lipstick-case sized device. When she removes it, the device has colored her fingernail a lovely shade of #81002c.

fifthelement-100

The small device is finger-sized, the industrial design feels very much like cosmetics, and its simple design clearly affords inserting a finger. There’s also a little icon on the side that indicates its color. This one device speaks well of what the entire line of products might look like. All told, a simple and lovely interaction in a domain, i.e. cosmetics, that typically doesn’t get a lot of attention in sci-fi.

But what is even more remarkable is that this isn’t the only fingernail interface in the Make It So survey. There is one other, 7 years earlier, and it happens to be used by someone with the exact same job. This other interface comes from the 1990 movie Total Recall.

Total Recall (1990)

TotalRecall-032

As you can see, this receptionist has an interface for coloring her nails as well, but the interaction is entirely different. This device has something like a a tablet with a connected stylus. It displays 16 color options in a full screen grid. She selects a particular color with the tap of the stylus. Then when she taps the stylus to a nail, the nail wipe-transitions to the new color from the tip to the cuticle.

TotalRecall-025

This device is cumbersome. It’s not something that could fit into a purse. Does she just leave it on her desk? Doesn’t her supervisor have opinions about that? My sense is that this is something better suited to a salon than an office space.

As a selection and application mechanism, the stylus is a bad choice. It requires quite a bit of precision to tap the tip of the nail. Our old friend Paul Fitts certainly would use something different for his nails. Since the secretary has to have to have some kind of high-tech coating, perhaps similar to electrophoretic ink, why is the stylus necessary at all? Can’t she just tap her fingernails to the color square of her choice? That would disintermediate the interaction and save her the hassle of targeting her nails with that stylus, especially when she has to switch to her off-hand.

The color display poses some other interesting problems as well. It needs to show colors, but why just 16? We don’t see any means of selecting others. Are these just this season’s most popular? Why not offer her any color she likes? Or some means of capturing her current outfit and suggesting colors based on that? Even the layout is problematic. Because of the effect of simultaneous contrast, the perception of a color alters when seen directly adjacent to other colors. These squares should have some sort of neutral border around them to make perception of them more “true.” But why should we burden her with having to imagine what the color will look like? Show her an image of her hand and let her see in advance what the new color will look like on her fingers. Any sort of low-level augmented reality would help her feel less like she’s picking paint for her living room wall.

And the winner is…

Comparing the two, I’d say that The Fifth Element fingernail-o-matic wins out. It’s more personal, more ergonomic, fits into the user’s lifestyle more, feels more fashionable than techy (which that receptionist clearly cares about). Yes, it’s more restricted in choices, but I’d much rather figure out how to augment that little device with a color selector than try to make a stylus and tablet fingernail-o-matic actually work.

Chat follow-up: Ultimate Weapon Against Evil & Constraints

The live chat of the O’Reilly webinar that Christopher delivered on 18 April 2013 had some great questions, but not all of them made it out of the chat room and onto the air. I’m taking a short break from the release of the sci-fi survey to answer some of those questions.

Q: Dennis Ward asks: There’s a gaff in The Fifth Element scene referenced—Corbin Dallas places one of the stones upsidedown relative to the other three. Is that a constraint issue?

5E-up

Yes and no.

Yes, if the stones could be placed on their pedestals in the wrong way. You wouldn’t want to design the weapon such that that were possible. As we saw, seconds count, and the stakes are pretty high. (c.f. Ultimate Evil.) Constraints, as Don Norman defined them in The Design of Everyday Things, would be one way to fix that. For example, you could widen one end of the stones so they were too large to fit in the pedestal the wrong way.

But (and here’s the “no”) it turns out that in The Ultimate Weapon Against Evil, the stones work whichever way they’re inserted. Take a look at the scene and though the stones aren’t all oriented the same way, i.e. pattern- or smooth-side up, they all work. (There is a third possibility, that orientation does matter, but they just got lucky in orienting them correctly. The odds of getting this right the first time is just over 6%, so we can discount this.)

5E-down

This is a superior design solution since it eliminates the need for the user to worry about orientation. Let’s call the pattern Make Orientation a Non-Issue.

But wait, we’re not done. We shouldn’t disregard the fact that you perceived it as a gaff. The design of the object signaled to you that there was an orientation that mattered. So yes, let’s keep the stones the same basic shape such that orientation is a non-issue, but one small improvement would be to have the visual design match the interaction design: The patterns should be symmetrical, perhaps completely covering each long side of the stones. That way, anyone wondering how they fit the pedestals wouldn’t falsely perceive that there is an orientation issue when there really isn’t one.

Thanks for this question, by the way. The Fifth Element is one of the first I reviewed for the Make It So survey since it’s one of my favorite sci-fi movies of all time. It makes me want to post that one next. I’ve got other plans, though, so perhaps after that. 🙂

UPDATE ————————————–

Since writing this post, I’ve done deeper analysis on this topic. See the Pilot episode of Sci Fi University for an even better and more thorough answer to this question.