When Luke is driving Kee and Theo to a boat on the coast, the car’s heads-up-display shows him the car’s speed with a translucent red number and speed gauge. There are also two broken, blurry gauges showing unknown information.
Suddenly the road becomes blocked by a flaming car rolled onto the road by a then unknown gang. In response, an IMPACT warning triangle zooms in several times to warn the driver of the danger, accompanied by a persistent dinging sound.
As part of the ongoing review of the Iron Man HUD, I noticed a small feature in the Iron Man 3 UI that—in order to critique—I have to discuss some new concepts and introduce some new terms. The feature itself is genuinely small and almost not worth posting about, but the terms are interesting, so bear with me.
Most of the time JARVIS animates the HUD, the UI elements sit on an invisible sphere that surrounds his head. (And in the case of stacked elements, on concentric invisible spheres.) The window of Pepper in the following screenshot illustrates this pretty clearly. It is a rectangular video feed, but appears slightly bowed to us, being on this sphere near the periphery of this 2nd-person view.
…And Pepper Potts is up next with her op-ed about the Civil Mommy Wars. Stay tuned.
Having elements slide around on the surface of this perceptual sphere is usable for Tony, since it means the elements are always facing him and thereby optimally viewable. “PEPPER POTTS,” for example, is as readable as if it was printed on a book perpendicular to his line of sight. (This notion is a bit confounded by the problems of parallax I wrote about in an earlier post, but since that seems unresolvable until Wim Wouters implements this exact HUD on Oculus Rift, let’s bypass it to focus on the new thing.)
So if it’s visually optimal to have 2D UI elements plastered to the surface of this perceptual sphere, how do we describe that suboptimal state where these same elements are not perpendicular to the line of sight, but angled away? I’m partly asking for a friend named Tony Stark because that’s some of what we see in Iron Man 3, both in 1st- and 2nd-person views. These examples aren’t egregious.
The Iron Patriot debut album cover graphic is only slightly angled and so easy to read. Similarly, the altimeter thingy on the left is still wholly readable.
The HUD itself displays a number of core capabilities across the Iron Man movies prior to its appearance in The Avengers. Cataloguing these capabilities lets us understand (or backworld) how he interacts with the HUD, equipping us to look for its common patterns and possible conflicts. In the first-person view, we saw it looked almost entirely like a rich agentive display, but with little interaction. Now, let’s look at that gorgeous 2nd-person view.
When in the first film Tony first puts the faceplate on and says to JARVIS, “Engage heads-up display”… …we see things from a narrative-conceit, 2nd-person perspective, as if the helmet were huge and we are inside the cavernous space with him, seeing only Tony’s face and the augmented reality interface elements. You might be thinking, “Of course it’s a narrative conceit. It’s not real. It’s in a movie.” But what I mean by that is that even in the diegesis, the Marvel Cinematic World, this is not something that could be seen. Let’s move through the reasons why. Continue reading →
When we first see the HUD, Tony is donning the Iron Man mask. Tony asks, JARVIS, “You there?” To which JARVIS replies, “At your service sir.” Tony tells him to Engage the heads-up display, and we see the HUD initialize. It is a dizzying mixture of blue wireframe motion graphics. Some imply system functions, such as the reticle that pinpoints Tonys eye. Most are small dashboard-like gauges that remain small and in Tonys peripheral vision while the information is not needed, and become larger and more central when needed. These features are catalogued in another post, but we learn about them through two points-of-view: a first-person view, which shows us what Tony’s sees as if we were there, donning the mask in his stead, and second-person view, which shows us Tony’s face overlaid against a dark background with floating graphics.
This post is about that first-person view. Specifically it’s about the visual design and the four awarenesses it displays.
In the Augmented Reality chapter of Make It So, I identified four types of awareness seen in the survey for Augmented Reality displays:
The Iron Man HUD illustrates all four and is a useful framework for describing and critiquing the 1st-person view. Continue reading →
There is a great deal to say about the interactions and interface, but let’s just take a moment to recount everything that the HUD does over the Iron Man movies and The Avengers. Keep in mind that just as there are many iterations of the suit, there can be many iterations of the HUD, but since it’s largely display software controlled by JARVIS, the functions can very easily move between exosuits.
Along the bottom of the HUD are some small gauges, which, though they change iconography across the properties, are consistently present.
For the most part they persist as tiny icons and thereby hard to read, but when the suit reboots in a high-altitude freefall, we get to see giant versions of them, and can read that they are:
The last Scav tech (and the last review of tech in the nerdsourced reviews of Oblivion) is a short one. During the drone assault on the Scav compound, we get a glimpse of the reticle used by the rebel Sykes as he tries to target a weak spot in a drone’s backside.
The reticle has a lot of problems, given Sykes’ task. The data on the periphery is too small to be readable. There are some distracting lines from the augmentation boxes which, if they’re just pointing to static points along the hairline, should be removed. The grid doesn’t seem to serve much purpose. There aren’t good differentiations among the ticks to be able to quickly subitize subtensions. (Read: tell how wide a thing is compared to the tick marks.) (You know, like with a ruler.)
The reticle certainly looks sci-fi, but real-world utility seems low.
The nicest and most surprising thing though is that the bullseye is the right shape and size of the thing he’s targeting. Whatever that circle thing is on the drone (a thermal exhaust port, which seem to be ubiquitously weak in spherical tech) this reticle seems to be custom-shaped to help target it. This may be giving it a lot of credit, but in a bit of apologetics, what if it had a lot of goal awareness, and adjusted the bullseye to match the thing he was targeting? Could it take on a tire shape to disable a car? Or a patella shape to help incapacitate a human attacker? That would be a very useful reticle feature.
Each drone is a semi-autonomous flying robot armed with large cannons, heavy armor, and a wide array of sensor systems. When in flight mode, the weapon arms retract. The arms extend when the drone senses a threat.
Each drone is identical in make and temperament, distinguishable only by large white numbers on its “face”. The armored shell is about a meter in diameter (just smaller than Jack). Internal power is supplied by a small battery-like device that contains enough energy to start a nuclear explosion inside of a sky-scraper-sized hydrogen distiller. It is not obvious whether the weapons are energy or projectile-based.
The Drone Interface is a HUD that shows the drone’s vision and secondary information about its decision making process. The HUD appears on all video from the Drone’s primary camera. Labels appear in legible human English.
Video feeds from the drone can be in one of several modes that vary according to what kind of searching the drone is doing. We never see the drone use more than one mode at once. These modes include visual spectrum, thermal imaging, and a special ‘tracking’ mode used to follow Jack’s bio signature.
Occasionally, we also see the Drone’s primary objective on the HUD. These include an overlay on the main view that says “TERMINATE” or “CLEAR”.