Unity Vision

22 Oct 2019 by Christopher Noessel

One of my favorite challenges in sci-fi is showing how alien an AI mind is. (It’s part of what makes Ex Machina so compelling, and the end of Her, and why Data from Star Trek: The Next Generation always read like a dopey, Pinnochio-esque narrative tool. But a full comparison is for another post.) Given that screen sci-fi is a medium of light, sound, and language, I really enjoy when filmmakers try to show how they see, hear, and process this information differently.

In Colossus: The Forbin Project, when Unity begins issuing demands, one of its first instructions is to outfit the Computer Programming Office (CPO) with wall-mounted video cameras that it can access and control. Once this network of cameras is installed, Forbin gives Unity a tour of the space, introducing it visually and spatially to a place it has only known as an abstract node network. During this tour, the audience is also introduced to Unity’s point-of-view, which includes an overlay consisting of several parts.

The first part is a white overlay of rule lines and MICR characters that cluster around the edge of the frame. These graphics do not change throughout the film, whether Unity is looking at Forbin in the CPO, carefully watching for signs of betrayal in a missile silo, or creepily keeping an “eye” on Forbin and Markham’s date for signs of deception.

In these last two screen grabs, you see the second part of the Unity POV, which is a focus indicator. This overlay appears behind the white bits; it’s a blue translucent overlay with a circular hole revealing true color. The hole shows where Unity is focusing. This indicator appears, occasionally, and can change size and position. It operates independently of the optical zoom of the camera, as we see in the below shots of Forbin’s tour.

A first augmented computer PoV? 🥇

When writing about computer PoVs before, I have cited Westworld as the first augmented one, since we see things from The Gunslinger’s infrared-vision eyes in the persistence-hunting sequences. (2001: A Space Odyssey came out the year prior to Colossus, but its computer PoV shots are not augmented.) And Westworld came out three years after Colossus, so until it is unseated, I’m going to regard this as the first augmented computer PoV in cinema. (Even the usually-encyclopedic TVtropes doesn’t list this one at the time of publishing.) It probably blew audiences’ minds as it was.

And as such, we should cut it a little slack for not meeting our more literate modern standards. It was forging new territory. Even for that, it’s still pretty bad.

Real world computer vision

Though computer vision is always advancing, it’s safe to say that AI would be looking at the flat images and seeking to understand the salient bits per its goals. In the case of self-driving cars, that means finding the road, reading signs and road makers, identifying objects and plotting their trajectories in relation to the vehicle’s own trajectory in order to avoid collisions, and wayfinding to the destination, all compared against known models of signs, conveyances, laws, maps, and databases. Any of these are good fodder for sci-fi visualization.

Source: Medium article about the state of computer vision in Russia, 2017.

Unity’s concerns would be its goal of ending war, derived subgoals and plans to achieve those goals, constant scenario testing, how it is regarded by humans, identification of individuals, and the trustworthiness of those humans. There are plenty of things that could be augmented, but that would require more than we see here.

Unity Vision looks nothing like this

I don’t consider it worth detailing the specific characters in the white overlay, or backworlding some meaning in the rule lines, because the rule overlay does not change over the course of the movie. In the book Make It So: Interaction Design Lessons from Sci-fi, Chapter 8, Augmented Reality, I identified the types of awareness such overlays could show: sensor output, location awareness, context awareness, and goal awareness, but each of these requires change over time to be useful, so this static overlay seems not just pointless, but it risks covering up important details that the AI might need.

Compare the computer vision of *The Terminator*.

Many times you can excuse computer-PoV shots as technical legacy, that is, a debugging tool that developers built for themselves while developing the AI, and which the AI now uses for itself. In this case, it’s heavily implied that Unity provided the specifications for this system itself, so that doesn’t make sense.

The focus indicator does change over time, but it indicates focus in a way that, again, obscures other information in the visual feed and so is not in Unity’s interest. Color spaces are part of the way computers understand what they’re seeing, and there is no reason it should make it harder on itself, even if it is a super AI.

Largely extradiegetic

So, since a diegetic reading comes up empty, we have to look at this extradiegetically. That means as a tool for the audience to understand when they’re seeing through Unity’s eyes—rather than the movie’s—and via the focus indicator, what the AI is inspecting.

As such, it was probably pretty successful in the 1970s to instantly indicate computer-ness.

One reason is the typeface. The characters are derived from MICR, which stands for magnetic ink character recognition. It was established in the 1950s as a way to computerize check processing. Notably, the original font had only numerals and four control characters, no alphabetic ones.

Note also that these characters bear a style resemblance to the ones seen in the film but are not the same. Compare the 0 character here with the one in the screenshots, where that character gets a blob in the lower right stroke.

I want to give a shout-out to the film makers for not having this creeper scene focus on lascivious details, like butts or breasts. It’s a machine looking for signs of deception, and things like hands, microexpressions, and, so the song goes, kisses are more telling.

Still, MICR was a genuinely high-tech typeface of the time. The adult members of the audience would certainly have encountered the “weird” font in their personal lives while looking at checks, and likely understood its purpose, so was a good choice for 1970, even if the details were off.

Another is the inscrutability of the lines. Why are they there, in just that way? Their inscrutability is the point. Most people in audiences regard technology and computers as having arcane reasons for the way they are, and these rectilinear lines with odd greebles and nurnies invoke that same sensibility. All the whirring gizmos and bouncing bar charts of modern sci-fi interfaces exhibit the same kind of FUIgetry.

So for these reasons, while it had little to do with the substance of computer vision, its heart was in the right place to invoke computer-y-ness.

Dat Ending

At the very end of the film, though, after Unity asserts that in time humans will come to love it, Forbin staunchly says, “Never.” Then the film passes into a sequence that is hard to tell whether it’s meant to be diegetic or not.

In the first beat, the screen breaks into four different camera angles of Forbin at once. (The overlay is still there, as if this was from a single camera.)

This says more about computer vision than even the FUIgetry.

This sense of multiples continues in the second beat, as multiple shots repeat in a grid. The grid is clipped to a big circle that shrinks to a point and ends the film in a moment of blackness before credits roll.

Since it happens right before the credits, and it has no precedent in the film, I read it as not part of the movie, but a title sequence. And that sucks. I wish wish wish this had been the standard Unity-view from the start. It illustrates that Unity is not gathering its information from a single stereoscopic image, like humans and most vertebrates do, but from multiple feeds simultaneously. That’s alien. Not even insectoid, but part of how this AI senses the world.

Captain’s Board

20 Sep 2016 by claytonbeese

The Captain’s Board is a double hexagon table at the very center of the CIC. This board serves as a combination of podium and status dashboard for the ship’s Captain. Often, the ship’s XO or other senior officers will move forward and use a grease pen or replacement transparency sheet to update information on the board.

For example, after jumping from their initial position to the fleet supply base in the nebula, Colonel Tigh replaces the map on the ‘left’ side of the board with a new map of the location that the Galactica had just jumped to. This implies that the Galactica has a cache of maps in the CIC of various parts of the galaxy, or can quickly print them on the fly.

After getting hit by a Cylon fighter’s nuclear missile, Tigh focuses on a central section of the board with a grease pen to mark the parts of the Galactica suffering damage or decompression. The center section of the board has a schematic, top-down view of the Galactica.

During the initial fighting, Lt. Gaeda is called forward to plot the location of Galactica’s combat squadrons on the board. This hand-drawn method is explicitly used, even when the Dradis system is shown to be functioning.

The transparency sheets are labeled with both a region and a sector: in this case, “Caprica Region, SECT OEL”. More text fills the bottom of the label: “Battlestar Galactica Starchart…”

Several panels of physical keys and low-resolution displays ring the board, but we never see any characters interacting with them. They do not appear to change during major events or during shifts in the ship status.

The best use of these small displays would be to access reference data with a quick search or wikipedia-style database. Given what we see in the show, it is likely that it was just intended as fuigetry.

Old School

Charts and maps are an old interface that has been well developed over the course of human history. Modern ships still use paper charts and maps to track their current location as a backup to GPS.

Given the Galactica’s mission to stay active even in the face of complete technological superiority of the opponent, a map-based backup to the Dradis makes sense in spite of the lack of detailed information it might need to provide. It is best as, and should be, a worst-case backup.

Here, the issue becomes the 3-dimensional space that the Galactica inhabits. The maps do an excellent job of showing relationships in a two dimensional plane, but don’t represent the ‘above’ and ‘below’ at all.

In those situations, perhaps something like a large fish tank metaphor might work better, but wouldn’t allow for quick plotting of distance and measurements by hand. Instead, perhaps something more like the Pin Table from the 2000 X-Men movie that could be operated by hand:

It would provide a shake-resistant, physical, no-electricity needed 3-D map of the surrounding area. Markups could be easily accomplished with a sticky-note-like flag that could attach to the pins.

Stark Tower monitoring

6 Oct 2015 by Christopher Noessel

Since Tony disconnected the power transmission lines, Pepper has been monitoring Stark Tower in its new, off-the-power-grid state. To do this she studies a volumetric dashboard display that floats above glowing shelves on a desktop.

Volumetric elements

The display features some volumetric elements, all rendered as wireframes in the familiar Pepper’s Ghost (I know, I know) visual style: translucent, edge-lit planes. A large component to her right shows Stark Tower, with red lines highlighting the power traveling from the large arc reactor in the basement through the core of the building.

The center of the screen has a similarly-rendered close up of the arc reactor. A cutaway shows a pulsing ring of red-tinged energy flowing through its main torus.

This component makes a good deal of sense, showing her the physical thing she’s meant to be monitoring but not in a photographic way, but a way that helps her quickly locate any problems in space. The torus cutaway is a little strange, since if she’s meant to be monitoring it, she should monitor the whole thing, not just a quarter of it that has been cut away.

Flat elements

The remaining elements in the display appear on a flat plane. Continue reading →

Abidjan Operation

25 Jun 2015 by Christopher Noessel

After Hawkeye is enthralled by Loki, agent Coulson has to call agent Romanoff in from the field, mid-mission. While he awaits her to extract herself from a situation, he idly glances at case file 242-56 which consists of a large video of Barton and Romanoff mid-combat, and overview profiles of the two agents. A legend in the upper right identifies this as STRIKE TEAM: DELTA, and a label at the top reads ABIDJAN OPERATION. There is some animated fuigetry on the periphery of the video, and some other fuigetry in windows that are occluded by the case file.

Continue reading →

Portal Monitor

4 Jun 2015 by Christopher Noessel

After Loki has enthralled Selvig, enthralled-Hawkeye lets Loki know that, “This place is about to blow and drop a hundred feet of rock on us.” Selvig looks to the following screen and confirms, “He’s right. The portal is collapsing in on itself.”

This is perhaps one of the most throwaway screens in the film, given the low-rez twisty graphics that could be out of Lawnmower Man, its only-vague-resemblance to the portal itself…

c.f.

…the text box of wildly scrolling and impossible to read pink code with what looks like a layer of white code hastily slapped over it, and—notably—no trendline of data that would help Selvig quickly identify this Very Important Fact. Maybe he’s such a portal whisperer that he can just see it, but why show the screens rather than show him looking up to the blue thing itself?

There might be some other data on the left of this bank of screens seen a few seconds later in the background…

…but it has more red text overlays, so I’m disinclined to give it the blurry benefit of the doubt.

Fair enough, this is there merely to establish Selvig’s enthrallment, and the scientific certainty of the stakes for the next beat. But, we see his eyes, and the certainty is evidenced by everything collapsing. We don’t need scientific assurance. If the designers were not given time to make it passable, I wish that the beat had been handled without a view of the screens rather than shaky-cam.

Sleep Pod—Wake Up Countdown

5 Mar 2015 by Maximiliano Pena

On each of the sleep pods in which the Odyssey crew sleep, there is a display for monitoring the health of the sleeper. It includes some biometric charts, measurements, a body location indicator, and a countdown timer. This post focuses on that timer.

To show the remaining time of until waking Julia, the pod’s display prompts a countdown that shows hours, minutes and seconds. It shows in red the final seconds while also beeping for every second. It pops-up over the monitoring interface.

Julia’s timer reaches 0:00:01.

The thing with pop-ups

We all know how it goes with pop-ups—pop-ups are bad and you should feel bad for using them. Well, in this case it could actually be not that bad.

The viewer

Although the sleep pod display’s main function is to show biometric data of the sleeper, the system prompts a popup to show the remaining time until the sleeper wakes up. And while the display has some degree of redundancy to show the data—i.e. heart rate in graphics and numbers— the design of the countdown brings two downsides for the viewer.

Position: it’s placed right in the middle of the screen.
Size: it’s roughly a quarter of the whole size of the display

Between the two, it partially covers both the pulse graphics and the numbers, which can be vital, i.e. life threatening—information of use to the viewer. Continue reading →

Planet P Uplink

8 Dec 2014 by Christopher Noessel

When the Roughnecks respond to a distress call from an outpost on Planet P, they quickly learn that it is a trap. Rasczak tells Dizzy to immediately summon an evacuation. To do so she uses this “uplink” interface. It has five components. Continue reading →

P.K.E. Meter

31 Oct 2014 by Christopher Noessel

Ghostbusters have a handheld device that detects “psycho-kinetic energy” (PKE), called, appropriately, the P.K.E. Meter.

Early in the library scene, Spengler is holding the device up in front of his eyes when he explains to Stentz that, “It’s moving.” So theoretically, there is some way by which it helps locate the source of PKE concentration. When we see the front of the device, it’s a bunch of small indictor lights, and impossible to make any sense of what we see there. (In fact, the later scene seems to have the orientation of the device horizontally flipped.) The inscrutability of the interface is fine, diegetically, since it’s a custom device built by scientists for themselves, but as they try and hire new ghostbusters and train them, they’re going to want to think about ease-of-training.

You might think that that green display we see in this first scene is pointing in a cardinal direction, i.e. just behind Spengler, but when we see the same device later, right next to the Keymaster, those indicator lights are spinning around a center that’s off the edge of the screen, so, again, the meaning of these lights is inscrutable.

Its most salient feature are the two arms that rotate out from either side, which are topped with a line of amber lights. When in the presence of high levels of PKE, the arms raise up and the lights blink more rapidly. At max PKE, the arms extend horizontally and the lights blink quite rapidly (see above). Unlike the user interface, these little guys are awesome. Let me explain why.

Both their lights and their extension help convey immediately the main point of the device: more PKE nearby. Notably they do so using two hard-to-miss signals that build on universal mental maps: up equals more, and faster blink is more urgent. Even if you can’t parse the UI, you get this. The immediacy means a lower cognitive load for the ghostbuster who’s attention needs to be on the environment around them primarily and on this device secondarily. Having physical motion and blinking even means it could be well in their peripheral vision and still convey the information. But it gets better. Even if it was out of sight, the motors that move the arms would provide a bit of sound and even haptic information that something has changed.

And lastly, we should keep in mind that ghostbusting is a service, and the customers are a vital part of that equation. Even if they don’t use it directly, they see it being used, and what it conveys to them as a touchpoint is important. If they are doubting, for example, the existence of ghosts, the little arms and lights would provide an immediately-understandable sense of whoa, and there’s a direct-feedback loop to verify that it’s not random. It’s detecting something in the environment. That would give the customers and onlookers a confidence that is important to new businesses operating within dubious domains.

So, seriously, those PKE arms are awesome and now I wish I had gone as a P.K.E. Meter for Halloween this year. Well, there’s always next year maybe at a sci-fi interfaces all-costume Halloween ball.

Rescue Shuttle

7 Oct 2014 by Christopher Noessel

After the ambush on Planet P, Ibanez pilots the shuttle that rescues survivors and…and Diz. We have a shot of the display that appears on the dashboard between the pilot and copilot. Tiny blue columns of text too small to read that spill onto the left. One big column of tiny green text that wipes on and flashes. Seizure-inducing yellow dots spazzing around on red grids. A blue circle on the right is probably Planet P or a radar, but the graphic…spinning about its center so quick you cannot follow. There’s not…I can’t…how is this supposed to…I’m just going to call it: fuigetry.

Rodger Young combat interfaces

23 Sep 2014 by Christopher Noessel

The interfaces aboard the Rodger Young in combat are hard to take seriously. The captain’s interface, for instance, features arrays of wireframe spheres that zoom from the bottom of the screen across horizontal lines to become blinking green squares. The shapes bear only the vaguest resemblance to the plasma bolts, but don’t match what we see out the viewscreen or the general behavior of the bolts at all. But the ridiculousness doesn’t end there.