About the critical stance of this blog

I knew I was going to piss of some fans of Blade Runner when I called the Voight-Kampff machine shit. I stand by it, but some of the discussion led me to realize I should make some of the implicit guiding principles of my approach to critique (and which lead me to call it shit) explicit. This is that.

OMG I’ve always wanted to read art critique theory on a sci-fi blog!

I’m going to drop this into the “About this site” page on the blog, but because a majority of my readers subscribe via RSS (Hi, you) and would not see that link, I’ll also copy the content to a post.

Short answer, I’m here for constructive criticism. That is, to ask of interfaces in sci-fi movies and television shows, “Is this the best form for its purpose?” followed by “If not, what is a better form, and why?” All for the 8 reasons I’ve outlined before:

  1. Build skepticism.
  2. Farm for good ideas.
  3. Use their bad ideas.
  4. Avoid their mistakes.
  5. Practice design critique.
  6. Build literacy.
  7. Mine its blind spots.
  8. Think big.

It’s made quite complicated because…

  • These are speculative technologies depicted in fictious tales.
  • The people making the art being reviewed may not have studied or even be interested in design for the real world.
  • The concepts of users and goals in this domain are complex: diegetic users with goals, actors using props, extradiegetic users of the film as engaging entertainment, sci-fi interface designers trying to balance believability with spectacle and narrative function, other sci-fi interface designers looking at other work in their field, writers of sci-fi trying to make a point, and designers of real-world technologies examining the design. All of these are “users” with different goals of use.
  • Narratively, an interface can serve multiple purposes: conveying plot points, telling us something about the character using it or the organization that made it, set dressing to convey science-y-ness, or even comedy.
  • Makers of sci-fi interfaces are often constrained with limited resources, paradigms of their time, conflicting directives, and intense pressures.
  • It is a fait accompli that these speculative technologies “work” in the way the script needs them to. They are not subject to ordinary forces of usability.
  • Often times actors are “interacting” with blank technologies on set, and interfaces painted on afterward in post to fit the actor’s motions.
  • The semiotics are multilayered, increasingly self-referential, and span the whole supergenre.
  • Most of the time the audience “reads” the interfaces in real-time for their narrative purpose rather than “seeing” them and contemplating their intricacies, the way I do on this blog. (Humans are surprisingly adept at knowing when to read a thing as “for the audience” and “for the characters” and rolling with the appropriate interpretation. We just “get” Tony’s Iron HUD, even though it is an impossible thing.)
  • Many of the films I review were made in a time when “pausing” was not even possible, so the artists could reasonably expect for details to zip by unnoticed.

But honestly, that complexity is why I find it an engaging place to be working. If it was simple, it might be boring.

Those few paragraphs might be enough of an explanation, except I have invoked “New Criticism” several times on the blog and in conversations, which bears some additional detail.

So, a longer-form answer follows.


When I was studying art history in undergraduate, we talked not just about art, but about how we talk about art, and as you might imagine, formal critique is a rich and high-minded topic. Turns out there are entire competing schools of thought, most of which end in “ism,” about what are good and useful ways to critique. Now, I wouldn’t consider myself exactly literate in critical theory. At most I have exposure to its key concepts as part of an undergrad minor, and then nearly a decade of putting it into practice here. It’s possible I’m only half-remembering what it is, and I’m missing out on key trends in modern critical theories. But the principles I’m going to attribute to it are sound, and I’ll stand by them even if I’m misattributing the source.

Here goes.

For hundreds of years prior to the mid-20th century, critique of art and literature was largely about examining the artists’ intent, circumstance, technique, and position within the artist’s body of work, as well as its contribution to the school of expression with which it was associated. It was largely about examining things from the artist’s perspective.

Historical critique: Where does this work sit in relation to Picasso’s blue period, and how did that inspire his later work or other artists?

But New Criticism rejects nearly all of this, instead looking at a thing from the perspective of a receiver [reader|watcher|hearer] of it. The work is the thing that is closely examined, not the artist nor their context. New Criticism seeks to ask what does the thing mean to us, as an audience, now?

Picasso’s famous “Weeping woman” paintng, showing woman, weeping, as a highly-deconstructed face.
New Criticism: What on earth could this deconstructed face mean?

Let’s look at these two approaches with an example from sci-fi.

In the literally-explosive last act of Alien, Ellen Ripley wants to set the Nostromo to self-destruct to destroy the titular xenomorph, while she and Jonesy the cat take the escape pod to cuddly safety. To start the self-destruct sequence, Ripley must use a labyrinthine interface, part of which is a push button interface with some arcane labels, including “lingha,” “yoni,” and “agaric fly.” How should we critique this?

The historical critique would examine the person who made it, how they made it, and what constraints they were working under. In this case, we would find (via the Alien Explorations blog) that the designer, Simon Deering, was reading the philosopher/occultist Helena Blavatsky’s book The Secret Doctrine, and in a pinch decided to use weird phrases from her book to fill out the “extra,” non-plot-critical buttons on the panel.

Do I need to mention this was Photoshopped? This was totally Photoshopped.

Great. We now have an answer to who made it and why. And…so what? It’s an interesting bit for trivia night or to trot out at your next cocktail party, but almost worthless for practitioners hoping to improve their craft.

(Almost worthless: The story could help sharpen one’s sense of skepticism, i.e. that just because these interfaces are cool doesn’t mean they are good models for real world design, but that’s the only takeaway I can think of.)

(And maybe a fine, semi-random way to discover new works of Russian occultism, but again, useless for design practice.)

(See, it’s complicated.)

The tag line for this very blog

Historicism is not a bad approach. Dave Addey has made quite an entertaining book and blog out of just this sort of examination. It’s even where I learned the Deering trivia in the first place. It’s just that while it’s entertaining, I don’t find this approach useful.

Better, the New Criticism argument goes, is to disregard the maker and their circumstance. Better is to try and find a diegetic reason the thing might be the way it is. Prioritize critique of the art over the artist. Sometimes this is easy. Other times you have to add a rule or idea to the diegesis to have it make sense, an act I call backworlding. Sometimes you can push through the surface of what appears broken to realize it might actually be brilliant, an act of apologetics (borrowing the term from religious philosophy). Now, staying in the diegesis, even with these techniques, isn’t always possible. Trying to connect “LINGHA” to self-destruct on the Nostromo would be a credibility-breaking stretch. But this deliberate first rejection of artist’s context and focus on the internal consistency is the “close reading” for which New Criticism is known.

The next step I take after a close reading is to make the critique useful to the reader. Sometimes that’s suggesting a design that better achieves the purposes of the work. (I did this with the Voight-Kampff machine. I did it with the Logan’s Run Circuit. How well I did is open to…critique.) Other usefulnesses are contrasting the design to known best practices or dark patterns in the field. Sometimes it’s acknowledging that the sci-fi interface is a great idea, and formalizing a new best practice—or Alexandrian pattern—around it. Sometimes it’s connecting the thing to other real world designs that share similar issues.

I consider real-world designers my primary audience, partly because having done this kind of design (and hey, even won some awards for it) for decades, I can claim some authority in the space.

I do know that another component of my readership are the writers and makers of sci-fi interfaces. I count some of them as friends (Hi, you). So ideally, when I suggest a new design, I try to have it both work as a real-world model and meeting the needs of the narrative. That’s not always possible, but I try. Sometimes that’s even a script rewrite. I have much less authority here, since I‘ve not yet done it professionally. <hangs shingle/> But yes, I try to consider those needs, too.

What this all means is that I will reject some common complaints about my reviews:

  1. But, it’s art. (Yes, I know. That does not exempt it from critique.)
  2. The [original book|novelization|toy|wiki|expanded universe] says differently. (Don’t care. I’m not reviewing those.)
  3. The designer didn’t have this [technology|paradigm|editing capability] at the time of creation. (Not useful to us today.)
  4. The designer couldn’t have imagined you’d be looking at it this closely. (Not my concern.)
  5. Maybe the aliens have some extra sense or capability that we don’t, and that is why it works for them. (Not useful for real-world design, at least, because we don’t live in a world with these actual, sentient aliens. We almost always design for people, and so frame the critique in light of that.)
  6. But there might be a diegetic reason it’s bad. Often times there are plausible backworlding reasons why a thing might be bad (no time, no expertise, no resources on hand) but we can’t learn anything from that, so if there’s another interpretation that helps us learn, I’ll tend toward that.
  7. This is mean. (I never attack the designer, and just address the design. If you’re looking for pure fawning, I’m just not your guy.)
  8. But this was pretty good, for its time. (That’s a historical read.)
  9. But it’s soooooo cool. (I offer skeptical critique and entertainment. There are plenty of other places to go to soak in spectacle and just be inspired.)
  10. Shhh. Just let people enjoy things. (This blog is opt-in. If you just want to enjoy these things without critical evaluation, press command-K and go elsewhere. I understand there are some charming cat videos hereabouts.)
With due respect: No. Not in this case.

Lastly, note that all of this is a general stance I take, not a set of laws to which I pedantically adhere. Sometimes an interface or speculative technology is so broken or so unusual that this approach just doesn’t work, and I have to take another tack. I’m OK with that.

I know this is long and screedy, but should help explain where I’m coming from, and whether this blog is for you. I hope it helps.

Advertisements

8 Reasons The Voight-Kampff Machine is shit (and a redesign to fix it)

Distinguishing replicants from humans is a tricky business. Since they are indistinguishable biologically, it requires an empathy test, during which the subject hears empathy-eliciting scenarios and watched carefully for telltale signs such as, “capillary dilation—the so-called blush response…fluctuation of the pupil…involuntary dilation of the iris.” To aid the blade runner in this examination, they use a portable machine called the Voight-Kampff machine, named, presumably, for its inventors.

The device is the size of a thick laptop computer, and rests flat on the table between the blade runner and subject. When the blade runner prepares the machine for the test, they turn it on, and a small adjustable armature rises from the machine, the end of which is an intricate piece of hardware, housing a powerful camera, glowing red.

The blade runner trains this camera on one of the subject’s eyes. Then, while reading from the playbook book of scenarios, they keep watch on a large monitor, which shows an magnified image of the subject’s eye. (Ostensibly, anyway. More on this below.) A small bellows on the subject’s side of the machine raises and lowers. On the blade runner’s side of the machine, a row of lights reflect the volume of the subject’s speech. Three square, white buttons sit to the right of the main monitor. In Leon’s test we see Holden press the leftmost of the three, and the iris in the monitor becomes brighter, illuminated from some unseen light source. The purpose of the other two square buttons is unknown. Two smaller monochrome monitors sit to the left of the main monitor, showing moving but otherwise inscrutable forms of information.

In theory, the system allows the blade runner to more easily watch for the minute telltale changes in the eye and blush response, while keeping a comfortable social distance from the subject. Substandard responses reveal a lack of empathy and thereby a high probability that the subject is a replicant. Simple! But on review, it’s shit. I know this is going to upset fans, so let me enumerate the reasons, and then propose a better solution.

-2. Wouldn’t a genetic test make more sense?

If the replicants are genetically engineered for short lives, wouldn’t a genetic test make more sense? Take a drop of blood and look for markers of incredibly short telomeres or something.

-1. Wouldn’t an fMRI make more sense?

An fMRI would reveal empathic responses in the inferior frontal gyrus, or cognitive responses in the ventromedial prefrontal gyrus. (The brain structures responsible for these responses.) Certinaly more expensive, but more certain.

0. Wouldn’t a metal detector make more sense?

If you are testing employees to detect which ones are the murdery ones and which ones aren’t, you might want to test whether they are bringing a tool of murder with them. Because once they’re found out, they might want to murder you. This scene should be rewritten such that Leon leaps across the desk and strangles Holden, IMHO. It would make him, and other blade runners, seem much more feral and unpredictable.

(OK, those aren’t interface issues but seriously wtf. Onward.)

1. Labels, people

Controls needs labels. Especially when the buttons have no natural affordance and the costs of experimentation to discover the function are high. Remembering the functions of unlabeled controls adds to the cognitive load for a user who should be focusing on the person across the table. At least an illuminated button helps signal the state, so that, at least, is something.

 2. It should be less intimidating

The physical design is quite intimidating: The way it puts a barrier in between the blade runner and subject. The fact that all the displays point away from the subject. The weird intricacy of the camera, its ominous HAL-like red glow. Regular readers may note that the eyepiece is red-on-black and pointy. That is to say, it is aposematic. That is to say, it looks evil. That is to say, intimidating.

I’m no emotion-scientist, but I’m pretty sure that if you’re testing for empathy, you don’t want to complicate things by introducing intimidation into the equation. Yes, yes, yes, the machine works by making the subject feel like they have to defend themselves from the accusations in the ethical dilemmas, but that stress should come from the content, not the machine.

2a. Holden should be less intimidating and not tip his hand

While we’re on this point, let me add that Holden should be less intimidating, too. When Holden tells Leon that a tortoise and a turtle are the same thing, (Narrator: They aren’t) he happens to glance down at the machine. At that moment, Leon says, “I’ve never seen a turtle,” a light shines on the pupil and the iris contracts. Holden sees this and then gets all “ok, replicant” and becomes hostile toward Leon.

In case it needs saying: If you are trying to tell whether the person across from you is a murderous replicant, and you suddenly think the answer is yes, you do not tip your hand and let them know what you know. Because they will no longer have a reason to hide their murderyness. Because they will murder you, and then escape, to murder again. That’s like, blade runner 101, HOLDEN.

3. It should display history 

The glance moment points out another flaw in the interface. Holden happens to be looking down at the machine at that moment. If he wasn’t paying attention, he would have missed the signal. The machine needs to display the interview over time, and draw his attention to troublesome moments. That way, when his attention returns to the machine, he can see that something important happened, even if it’s not happening now, and tell at a glance what the thing was.

4. It should track the subject’s eyes

Holden asks Leon to stay very still. But people are bound to involuntarily move as their attention drifts to the content of the empathy dilemmas. Are we going to add noncompliance-guilt to the list of emotional complications? Use visual recognition algorithms and high-resolution cameras to just track the subject’s eyes no matter how they shift in their seat.

5. Really? A bellows?

The bellows doesn’t make much sense either. I don’t believe it could, at the distance it sits from the subject, help detect “capillary dilation” or “ophthalmological measurements”. But it’s certainly creepy and Terry Gilliam-esque. It adds to the pointless intimidation.

6. It should show the actual subject’s eye

The eye color that appears on the monitor (hazel) matches neither Leon’s (a striking blue) or Rachel’s (a rich brown). Hat tip to Typeset in the Future for this observation. His is a great review.

7. It should visualize things in ways that make it easy to detect differences in key measurements

Even if the inky, dancing black blob is meant to convey some sort of information, the shape is too organic for anyone to make meaningful readings from it. Like seriously, what is this meant to convey?

The spectrograph to the left looks a little more convincing, but it still requires the blade runner to do all the work of recognizing when things are out of expected ranges.

8. The machine should, you know, help them

The machine asks its blade runner to do a lot of work to use it. This is visual work and memory work and even work estimating when things are out of norms. But this is all something the machine could help them with. Fortunately, this is a tractable problem, using the mighty powers of logic and design.

Pupillary diameter

People are notoriously bad at estimating the sizes of things by sight. Computers, however, are good at it. Help the blade runner by providing a measurement of the thing they are watching for: pupillary diameter. (n.b. The script speaks of both iris constriction and pupillary diameter, but these are the same thing.) Keep it convincing and looking cool by having this be an overlay on the live video of the subject’s eye.

So now there’s some precision to work with. But as noted above, we don’t want to burden the user’s memory with having to remember stuff, and we don’t want them to just be glued to the screen, hoping they don’t miss something important. People are terrible at vigilance tasks. Computers are great at them. The machine should track and display the information from the whole session.

Note that the display illustrates radius, but displays diameter. That buys some efficiencies in the final interface.

Now, with the data-over-time, the user can glance to see what’s been happening and a precise comparison of that measurement over time. But, tracking in detail, we quickly run out of screen real estate. So let’s break the display into increments with differing scales.

There may be more useful increments, but microseconds and seconds feel pretty convincing, with the leftmost column compressing gradually over time to show everything from the beginning of the interview. Now the user has a whole picture to look at. But this still burdens them into noticing when these measurements are out of normal human ranges. So, let’s plot the threshold, and note when measurements fall outside of that. In this case, it feels right that replicants display less that normal pupillary dilation, so it’s a lower-boundary threshold. The interface should highlight when the measurement dips below this.

Blush

I think that covers everything for the pupillary diameter. The other measurement mentioned in the dialogue is capillary dilation of the face, or the “so-called blush response.” As we did for pupillary diameter, let’s also show a measurement of the subject’s skin temperature over time as a line chart. (You might think skin color is a more natural measurement, but for replicants with a darker skin tone than our two pasty examples Leon and Rachel, temperature via infrared is a more reliable metric.) For visual interest, let’s show thumbnails from the video. We can augment the image with degree-of-blush. Reduce the image to high contrast grayscale, use visual recognition to isolate the face, and then provide an overlay to the face that illustrates the degree of blush.

But again, we’re not just looking for blush changes. No, we’re looking for blush compared to human norms for the test. It would look different if we were looking for more blushing in our subject than humans, but since the replicants are less empathetic than humans, we would want to compare and highlight measurements below a threshold. In the thumbnails, the background can be colored to show the median for expected norms, to make comparisons to the face easy. (Shown in the drawing to the right, below.) If the face looks too pale compared to the norm, that’s an indication that we might be looking at a replicant. Or a psychopath.

So now we have solid displays that help the blade runner detect pupillary diameter and blush over time. But it’s not that any diameter changes or blushing is bad. The idea is to detect whether the subject has less of a reaction than norms to what the blade runner is saying. The display should be annotating what the blade runner has said at each moment in time. And since human psychology is a complex thing, it should also track video of the blade runner’s expressions as well, since, as we see above, not all blade runners are able to maintain a poker face. HOLDEN.

Anyway, we can use the same thumbnail display of the face, without augmentation. Below that we can display the waveform (because they look cool), and speech-to-text the words that are being spoken. To ensure that the blade runner’s administration of the text is not unduly influencing the results, let’s add an overlay to the ideal intonation targets. Despite evidence in the film, let’s presume Holden is a trained professional, and he does not stray from those targets, so let’s skip designing the highlight and recourse-for-infraction for now.

Finally, since they’re working from a structured script, we can provide a “chapter” marker at the bottom for easy reference later.

Now we can put it all together, and it looks like this. One last thing we can do to help the blade runner is to highlight when all the signals indicate replicant-ness at once. This signal can’t be too much, or replicants being tested would know from the light on the blade runner’s face when their jig is up, and try to flee. Or murder. HOLDEN.

For this comp, I added a gray overlay to the column where pupillary and blush responses both indicated trouble. A visual designer would find some more elegant treatment.

If we were redesigning this from scratch, we could specify a wide display to accomodate this width. But if we are trying to squeeze this display into the existing prop from the movie, here’s how we could do it.

Note the added labels for the white squares. I picked some labels that would make sense in the context. “Calibrate” and “record” should be obvious. The idea behind “mark” is an easy button for the blade runner to press when they see something that looks weird, like when doctors manually annotate cardiograph output.

Lying to Leon

There’s one more thing we can add to the machine that would help out, and that’s a display for the subject. Recall the machine is meant to test for replicant-ness, which happens to equate to murdery-ness. A positive result from the machine needs to be handled carefully so what happens to Holden in the movie doesn’t happen. I mentioned making the positive-overlay subtle above, but we can also make a placebo display on the subject’s side of the interface.

The visual hierarchy of this should make the subject feel like its purpose is to help them, but the real purpose is to make them think that everything’s fine. Given the script, I’d say a teleprompt of the empathy dilemma should take up the majority of this display. Oh, they think, this is to help me understand what’s being said, like a closed caption. Below the teleprompt, at a much smaller scale, a bar at the bottom is the real point.

On the left of this bar, a live waveform of the audio in the room helps the subject know that the machine is testing things live. In the middle, we can put one of those bouncy fuiget displays that clutters so many sci-fi interfaces. It’s there to be inscrutable, but convince the subject that the machine is really sophisticated. (Hey, a diegetic fuiget!) Lastly—and this is the important part—An area shows that everything is “within range.” This tells the subject that they can be at ease. This is good for the human subject, because they know they’re innocent. And if it’s a replicant subject, this false comfort protects the blade runner from sudden murder. This test might flicker or change occasionally to something ambiguous like “at range,” to convey that it is responding to real world input, but it would never change to something incriminating.

This way, once the blade runner has the data to confirm that the subject is a replicant, they can continue to the end of the module as if everything was normal, thank the replicant for their time, and let them leave the room believing they passed the test. Then the results can be sent to the precinct and authorizations returned so retirement can be planned with the added benefit of the element of surprise.

OK

Look, I’m sad about this, too. The Voight-Kampff machine is cool. It fits very well within the art direction of the Blade Runner universe. This coolness burned the machine into my memory when I saw this film the first dozen times, but despite that, it just doesn’t stand up to inspection. It’s not hopeless, but does need a lot of thinkwork and design to make it really fit to task, and convincing to us in the audience.

Blade Runner (1982)

Whew. So we all waited on tenterhooks through November to see if somehow Tyrell Corporation would be founded, develop and commercialize general AI, and then advance robot evolution into the NEXUS phase, all while in the background space travel was perfected, Off-world colonies and asteroid mining established, global warming somehow drenched Los Angeles in permanent rain and flares, and flying cars appear on the market. None of that happened. At least not publicly. So, with Blade Runner squarely part of the paleofuture past, let’s grab our neon-tube umbrellas and head into the rain to check out this classic that features some interesting technologies and some interesting AI.

Release date: 25 Jun 1982

The punctuation-challenged crawl for the film:

“Early in the 21st Century, THE TYRELL CORPORATION advanced Robot evolution into the NEXUS phase—a being virtually identical to a human—known as a Replicant. [sic] The NEXUS 6 Replicants were superior in strength and agility, and at least equal in intelligence, to the genetic engineers who created them. Replicants were used Off-world as slave labor, in the hazardous exploration and colonisation of other planets. After a bloody mutiny by a NEXUS 6 combat team in an Off-world colony, Replicants were declared illegal on Earth—under penalty of death. Special police squads—BLADE RUNNER UNITS—had orders to shoot to kill, upon detection, any trespassing Replicants.

“This was not called execution. It was called retirement.”

Four murderous replicants make their way to Earth, to try and find a way to extend their genetically-shortened life spans. The Blade Runner named Deckard is coerced by his ex-superior Bryant and detective Gaff out of retirement and into finding and “retiring” these replicants.

Deckard meets Dr. Tyrell to interview him, and at Tyrell’s request tests Rachel on a Voight-Kampff machine, which is designed to help blade runners tell replicants from people. Deckard and Rachel learn that she is a replicant. Then with Gaff, he follows clues to the apartment of one exposed replicant, Leon, where he finds a synthetic snake scale in the bathtub and a set of photographs in a drawer. Using a sophisticated image inspection tool in his home, he scans one of the photos taken in Leon’s apartment, until he finds the reflection of a face. He prints the image to take with him.

He takes the snake scale to someone with an electron microscope who is able to read the micrometer-scale “maker’s serial number” there. He visits the maker, a person named “the Egyptian,” who tells Deckard he sold the snake to Taffey Lewis. Deckard visits Taffey’s bar, where he sees Zhora, another of the wanted replicants, perform a stage act with a snake. She matches the picture he holds. He heads backstage to talk to her in her dressing room, posing as a representative of the “American Federation of Variety Artists, Confidential Committee on Moral Abuses.” When she finishes pretending to prepare for her next act, she attacks him and flees. He chases and retires her. Leon happens to witness the killing, and attacks Deckard. Leon has the upper hand but Deckard is saved when Rachel appears from the crowd and shoots Leon in the head. They return to his apartment. They totally make out.

Meanwhile, Roy has learned of a Tyrell employee named Sebastian who does genetic design. On orders, Pris befriends Sebastian and dupes him into letting her into his apartment. She then lets Roy in. Sebastian figures out that they are replicants, but confesses he cannot help them directly. Roy intimidates Sebastian into arranging a meeting between him and Dr. Tyrell. At the meeting, Tyrell says there is nothing that can be done. In fury, Roy kills Tyrell and Sebastian.

The police investigating the scene contact Deckard with Sebastian’s address. Deckard heads there, where he finds, fights, and retires Pris. Roy is there, too, but proves too tough for Deckard to retire. Roy could kill Deckard but instead opts to die peacefully, even poetically. Witnessing this act of grace, Deckard comes to appreciate the “humanity” of the replicants, and returns home to elope with Rachel.

In the last scene, Gaff hints to Deckard with this unicorn origami that Deckard himself is a replicant.


P.S. This series uses “The Final Cut” edit of the movie, so I don’t have to hear that wretchedly-scripted voiceover from the theatrical release. If you can, I recommend seeing that version.

IMDB Icon
v
iTunes