High Tech Binoculars

In Johnny Mnemonic we see two different types of binoculars with augmented reality overlays and other enhancements: Yakuz-oculars, and LoTek-oculars.

Yakuz-oculars

The Yakuza are the last to be seen but also the simpler of the two. They look just like a pair of current day binoculars, but this is the view when the leader surveys the LoTek bridge.

jm-25-yakuza-binocs-adjusted

I assume that the characters here are Japanese? Anyone?

In the centre is a fixed-size green reticule. At the bottom right is what looks like the magnification factor. At the top left and bottom left are numbers, using Western digits, that change as the binoculars move. Without knowing what the labels are I can only guess that they could be azimuth and elevation angles, or distance and height to the centre of the reticule. (The latter implies some sort of rangefinder.)

So far, this is a simple uncluttered display. But why is there a brightly glowing Pharmakom logo at the top right? It blocks part of the view, and probably doesn’t help anyone trying to keep their eyes adapted for night vision.

LoTek-oculars

The LoTeks, despite their name, have more impressive binoculars. They’re first used when Johnny gets out of his airport taxi.

jm-11-lotek-binocs-a-adjusted

There’s a third tube above the optics, a rectangular inlet, and an antenna.

In these binoculars, the augmented reality overlay is much more dynamic. Instead of a fixed circle, green lines converge in a bounding box around the image of Johnny. Text slides onto the display from left to right, the last line turning yellow.

jm-11-loteks-animated

Zoomrect

The animated transition of the bounding box resembles what Classic MacOS programmers of the 1990s called “zoomrects” used for showing windows opening or closing. It’s a very effective technique to draw attention to a particular area of an image.

Animated text

Text appearing character by character is ubiquitous in film interfaces. In the 1960s and 1970s mainframe and minicomputer terminals really did display incrementally, as the characters arrived one by one over slow serial port links. On any more recent computer it actually takes extra programming to achieve this effect, as the normal display of text is so fast that we would perceive it as instantaneous. But people like to see incremental text, or have been conditioned by film to expect it, so why not?

Bioscanning

The binoculars detect Johnny’s implant. It might just be possible to detect this passively from infrared or electronic signals, but more likely the binoculars include a high resolution microwave radar as well. If there had been more than one person in view, the bounding box would indicate which one the text refers to. And note that the last line of text is a different color. What that means is unclear here, but it becomes clear (and I’ll discuss it) later.

The second time we see the LoTek binoculars is when a lookout spots Street Preacher, a very bad guy and another who wants to remove Johnny’s head. Once again the binoculars have performed more than just a visual scan.

jm-17-lotek-binocs-a-adjusted

The binocular view and overlay are being relayed to another character, the LoTek leader J-Bone who can watch on a monitor. Here the film anticipates the WiFi webcam.

jm-17-lotek-binocs-b-adjusted

The overlay text now changes.

jm-17-lotek-binocs-c-adjusted

Narrow AI?

This is interesting, because the binoculars can not only detect implants and other cyborg modifications, but are apparently able to evaluate and offer advice. It appears that the green text is used for the factual (more or less) information about what has been detected, while yellow text is uncertain or or speculative.

Does this imply a general artificial intelligence? Not necessarily. This warning could be based solely on the detected signature, in the same way that current day military passive sonars and radar warning receivers can identify threats based on identifying characteristics of a received signal. In the world of Johnny Mnemonic it would make sense to assume that anyone with full custom biomechanics is extremely dangerous. Or, since Street Preacher is a resident rather than a stranger and already feared by others, his appearance and the warning could have been entered into a LoTek facial recognition database that the binocular system uses as a reference.

These textual overlays are an excellent interface, not interfering with normal vision and providing a fast and easy-to-understand analysis. But, the user must have faith that the computer analysis is accurate. There’s no reason given as to why any of the text is displayed. If Johnny was carrying an implant in his pocket instead of his brain, would the computer know the difference?

An alternative approach would be some kind of sensor fusion or false spectrum display, with the raw infrared or radar image overlaid over the visuals and the viewer responsible for interpreting the data. The problem with such systems is that our visual system didn’t evolve to interpret such imagery, so a lot of training and practice is required to be both fast and accurate. And the overlay itself interferes with our normal visual recognition and processing. If the computer can do a better job of deciphering the meaning of non-visual data, it should do so and summarise for the human viewer.

Further advantages of this interface are that even a novice sentry will benefit from the built-in scanning and threat analysis, and the wireless transmission ensures that the information is shared rather than being limited to the person on watch. 

Grabby hologram

After Pepper tosses off the sexy bon mot “Work hard!” and leaves Tony to his Avengers initiative homework, Tony stands before the wall-high translucent displays projected around his room.

Amongst the videos, diagrams, metadata, and charts of the Tesseract panel, one item catches his attention. It’s the 3D depiction of the object, the tesseract itself, one of the Infinity Stones from the MCU. It is a cube rendered in a white wireframe, glowing cyan amidst the flat objects otherwise filling the display. It has an intense, cold-blue glow at its center.  Small facing circles surround the eight corners, from which thin cyan rule lines extend a couple of decimeters and connect to small, facing, inscrutable floating-point numbers and glyphs.

Avengers_PullVP-02.png

Wanting to look closer at it, he reaches up and places fingers along the edge as if it were a material object, and swipes it away from the display. It rests in his hand as if it was a real thing. He studies it for a minute and flicks his thumb forward to quickly switch the orientation 90° around the Y axis.

Then he has an Important Thought and the camera cuts to Agent Coulson and Steve Rogers flying to the helicarrier.

So regular readers of this blog (or you know, fans of blockbuster sci-fi movies in general) may have a Spidey-sense that this feels somehow familiar as an interface. Where else do we see a character grabbing an object from a volumetric projection to study it? That’s right, that seminal insult-to-scientists-and-audiences alike, Prometheus. When David encounters the Alien Astrometrics VP, he grabs the wee earth from that display to nuzzle it for a little bit. Follow the link if you want that full backstory. Or you can just look and imagine it, because the interaction is largely the same: See display, grab glowing component of the VP and manipulate it.

Prometheus-229 Two anecdotes are not yet a pattern, but I’m glad to see this particular interaction again. I’m going to call it grabby holograms (capitulating a bit on adherence to the more academic term volumetric projection.) We grow up having bodies and moving about in a 3D world, so the desire to grab and turn objects to understand them is quite natural. It does require that we stop thinking of displays as untouchable, uninterruptable movies and more like toy boxes, and it seems like more and more writers are catching on to this idea.

More graphics or more information?

Additionally,  the fact that this object is the one 3D object in its display is a nice affordance that it can be grabbed. I’m not sure whether he can pull the frame containing the JOINT DARK ENERGY MISSION video to study it on the couch, but I’m fairly certain I knew that the tesseract was grabbable before Tony reached out.

On the other hand, I do wonder what Tony could have learned by looking at the VP cube so intently. There’s no information there. It’s just a pattern on the sides. The glow doesn’t change. The little glyph sticks attached to the edges are fuigets. He might be remembering something he once saw or read, but he didn’t need to flick it like he did for any new information. Maybe he has flicked a VP tesseract in the past?

Augmented “reality”

Rather, I would have liked to have seen those glyph sticks display some useful information, perhaps acting as leaders that connected the VP to related data in the main display. One corner’s line could lead to the Zero Point Extraction chart. Another to the lovely orange waveform display. This way Tony could hold the cube and glance at its related information. These are all augmented reality additions.

Augmented VP

Or, even better, could he do some things that are possible with VPs that aren’t possible with AR. He should be able to scale it to be quite large or small. Create arbitrary sections, or plan views. Maybe fan out depictions of all objects in the SHIELD database that are similarly glowy, stone-like, or that remind him of infinity. Maybe…there’s…a…connection…there! Or better yet, have a copy of JARVIS study the data to find correlations and likely connections to consider. We’ve seen these genuine VP interactions plenty of places (including Tony’s own workshop), so they’re part of the diegesis.

Avengers_PullVP-05.pngIn any case, this simple setup works nicely, in which interaction with a cool media helps underscore the gravity of the situation, the height of the stakes. Note to selves: The imperturbable Tony Stark is perturbed. Shit is going to get real.