As part of the ongoing review of the Iron Man HUD, I noticed a small feature in the Iron Man 3 UI 2nd-person UI that—in order to critique—I have to discuss some new concepts and introduce some new terms. The feature itself is genuinely small and almost not worth posting about, but the terms are interesting, so bear with me.
Most of the time JARVIS animates the HUD, the UI elements sit on an invisible sphere that surrounds his head. (And in the case of stacked elements, on concentric invisible spheres.) The window of Pepper in the following screenshot illustrates this pretty clearly. It is a rectangular video feed, but appears slightly bowed to us, being on this sphere near the periphery of this 2nd-person view.
Having elements slide around on the surface of this perceptual sphere is usable for Tony, since it means the elements are always facing him and thereby optimally viewable. “PEPPER POTTS,” for example, is as readable as if it was printed on a book perpendicular to his line of sight. (This notion is a bit confounded by the problems of parallax I wrote about in an earlier post, but since that seems unresolvable until Wim Wouters implements this exact HUD on Oculus Rift, let’s bypass it to focus on the new thing.)
So if it’s visually optimal to have 2D UI elements plastered to the surface of this perceptual sphere, how do we describe that suboptimal state where these same elements are not perpendicular to the line of sight, but angled away? I’m partly asking for a friend named Tony Stark because that’s some of what we see in Iron Man 3, both in 1st- and 2nd-person views. These examples aren’t egregious.
As I mentioned in the opening paragraph, these things aren’t terrible in and of themselves, but as a UI pattern could get bad as people misunderstand and overuse it, so we need a way to talk about it. To be precise, we need a way to talk about the degree of tilt away from a plane perpendicular to the line of sight. except “degree of tilt away from a plane perpendicular to the line of sight” is waaay too long.
To find this term, I did some asking around on social media. At first, lots of folks jumped to anatomical terms of location like sagittal or caudal, but should you be similarly tempted, note that these terms are fixed per the body. A UI element that is coronal in front of the face, and perfectly readable there, is utterly unreadable near the ear. A facing element would be readable in both places, and a whatever-the-antonym-is element similarly unreadable as it slid from the nose around the side.
Eventually I got some nice adjectives that describe the particular tilt away from the line of sight. I was most happy with industrial designer Abhinav Dapke’s suggestion of “lengthwise” for a tilt away from line-of-sight, since it’s a word we have already and very descriptive. It also implies another existing word for yawed-against line-of-sight, and that’s “edgewise.” (Roll along line-of-sight can be handled simply as rotation, for you completionists.)
But for the single variable that we can discuss as an antonym to facing, my crowdsourcing turned up nothing, and so I’m going to coin the ungainly adjectives off-facing and off-faced. Each is short, decryptable, not currently defined as something else, and obviously connected to its source concept, so works for many reasons.
With these we now we can speak of those elements that are off-faced in Iron Man and similar bubble HUDs, and do a Invasion of the Body Snatchers-esque pointing and screeching when it’s too extreme.
Note that this only applies to 2D UI elements that are meant to be read. The overwhelming majority of things we see in the physical world are not oriented to our line of sight and that poses little problem. Even in the Iron Man HUD we see plenty of objects that are off-faced but rightly so, since as augmentations they bear orientation to the world, not the viewer.
One of the main reasons I went to such trouble to come up with these terms is that I think the Iron Man HUD is one of the most forward-provoking sci-fi interfaces in the survey. It ought to be the Minority Report Precrime Scrubber of it’s day. I suspect it will become more and more influential, and so having these new terms are likely to become more useful and necessary as sci-fi keeps on keepin’ on.
Next up in the Iron HUD series: We discuss how JARVIS is straight-up lying to Tony Stark.
There’s some value to posing 3D elements in off-facing ways. If the elements are wider than they are deep (e.g., Iron Patriot example above) then you can rotate them to make them optically narrower (i.e. foreshortened). When they are narrower, more can be squeezed into the view. If you’re not looking directly at something then its OK if its posed suboptimally but it should rotate into an optimal pose when you look directly at it. If you’re not looking directly at something then its visually suboptimal anyway. It might be tricky to implement so that the virtual objects aren’t too wiggly.
If you’re interacting with virtual elements with your hands then it may be good to slant them to make them move comfortable to interact with. I made some virtual button panels and eventually slanted them because it was more comfortable to drum than to punch.
Foreshortening can be accomplished as well with type/font manipulations, yes? I agree that off-faced elements are useful to signal what optional content is available (see https://scifiinterfaces.wordpress.com/2012/11/29/vp-language-instructor/), but 3D manipulations are not necessary for managing real estate. The virtual tilt is nice virtual ergonomics, though that’s not at play here.
The term “off-axis” from computer graphics describes a projection onto a plane that is not perpendicular to the line of sight, so it might be reasonable to use that term for this as well.
BTW, not to toot my own horn, but if you look at the paper “VEWL: A Framework for Building a Windowing Interface in a Virtual Environment” by Dan Larimer and myself, you’ll see our implementation of 2D content attached to an invisible virtual sphere surrounding the user (the content was kept 2D and the “windows” were placed tangent to the sphere so they remained flat). We also looked at how to control a 2D cursor with a 3D device to interact with this content.
Toot away, Doug! I thought about “off-axis” but it only implies the line of sight axis, and given that “axis” in 3D software usually refers to X, Y, or Z I thought it might have carried some ambiguity. When you were writing your paper, did you come across anyone who had done a readability study of degrees of off-faced text?
Thanks for referring to the Kinect-ironman demo I created. Also for the challenge on Twitter 😉 Would really love to accept. But having some second thoughts. Recreate/improve something that EXISTS (ironman) or create something UNIQUE (tbd).
The ironman demo was made in less than 1 week (good research timeframe for something I can not monitize).
Pingback: Grabby hologram | Sci-fi interfaces
Pingback: Tony Stark is being lied to (by his own creation) | Sci-fi interfaces