Tag Archives: blue

Routing Board

by

10

When the two AIs Colossus and Guardian are disconnected from communicating with each other, they try and ignore the spirit of the human intervention and reconnect on their own. We see the humans monitoring Colossus’ progress in this task on big board in the U.S. situation room. It shows a translucent projection map of the globe with white dots representing data centers and red icons representing missiles. Beneath it, glowing arced lines illustrate the connection routes Colossus is currently testing. When it finds that a current segment is ineffective, that line goes dark, and another segment extending from the same node illuminates.

For a smaller file size, the animated gif has been stilled between state changes, but the timing is as close as possible to what is seen in the film.

Forbin explains to the President, “It’s trying to find an alternate route.”

A first in sci-fi: Routing display 🏆

First, props to Colossus: The Forbin Project for being the first show in the survey to display something like a routing board, that is, a network of nodes through which connections are visible, variable, and important to stakeholders.

Paul Baran and Donald Davies had published their notion of a network that could, in real-time, route information dynamically around partial destruction of the network in the early 1960s, and this packet switching had been established as part of ARPAnet in the late 1960s, so Colossus was visualizing cutting edge tech of the time.

This may even be the first depiction of a routing display in all of screen sci-fi or even cinema, though I don’t have a historical perspective on other genres, like the spy genre, which is another place you might expect to see something like this. As always, if you know of an earlier one, let me know so I can keep this record up to date and honest.

A nice bit: curvy lines

Should the lines be straight or curvy? From Colossus’ point of view, the network is a simple graph. Straight lines between its nodes would suffice. But from the humans’ point of view, the literal shape of the transmission lines are important, in case they need to scramble teams to a location to manually cut the lines. Presuming these arcs mean that (and not just the way neon in a prop could bend), then the arcs are the right display. So this is good.

But, it breaks some world logic

The board presents some challenges with the logic of what’s happening in the story. If Colossus exists as a node in a network, and its managers want to cut it off from communication along that network, where is the most efficient place to “cut” communications? It is not at many points along the network. It is at the source.

Imagine painting one knot in a fishing net red and another one green. If you were trying to ensure that none of the strings that touch the red knot could trace a line to the green one, do you trim a bunch of strings in the middle, or do you cut the few that connect directly to the knot? Presuming that it’s as easy to cut any one segment as any other, the fewer number of cuts, the better. In this case that means more secure.

The network in Colossus looks to be about 40 nodes, so it’s less complicated than the fishing net. Still, it raises the question, what did the computer scientists in Colossus do to sever communications? Three lines disappear after they cut communications, but even if they disabled those lines, the rest of the network still exists. The display just makes no sense.

Before, happy / After, I will cut a Prez

Per the logic above, they would cut it off at its source. But the board shows it reaching out across the globe. You might think maybe they just cut Guardian off, leaving Colossus to flail around the network, but that’s not explicitly said in the communications between the Americans and the Russians, and the U.S. President is genuinely concerned about the AIs at this point, not trying to pull one over on the “pinkos.” So there’s not a satisfying answer.

It’s true that at this point in the story, the humans are still letting Colossus do its primary job, so it may be looking at every alternate communication network to which it has access: telephony, radio, television, and telegraph. It would be ringing every “phone” it thought Guardian might pick up, and leaving messages behind for possible asynchronous communications. I wish a script doctor had added in a line or three to clarify this.

  • FORBIN
  • We’ve cut off its direct lines to Guardian. Now it’s trying to find an indirect line. We’re confident there isn’t one, but the trouble will come when Colossus realizes it, too.

Too slow

Another thing that seems troubling is the slow speed of the shifting route. The segments stay illuminated for nearly a full second at a time. Even with 1960s copper undersea cables and switches, electronic signals should not take that long. Telephony around the world was switched from manual to automatic switching by the 1930s, so it’s not like it’s waiting on a human operating a switchboard.

You’re too slow!

Even if it was just scribbling its phone number on each network node and the words “CALL ME” in computerese, it should go much faster than this. Cinematically, you can’t go too fast or the sense of anticipation and wonder is lost, but it would be better to have it zooming through a much more complicated network to buy time. It should feel just a little too fast to focus on—frenetic, even.

This screen gets 15 seconds of screen time, and if you showed one new node per frame, that’s only 360 states you need to account for, a paltry sum compared to the number of possible paths it could test across a 38 node graph between two points.

Plus the speed would help underscore the frightening intelligence and capabilities of the thing. And yes I understand that that is a lot easier said than done nowadays with digital tools than with this analog prop.

Realistic-looking search strategies

Again, I know this was a neon, analog prop, but let’s just note that it’s not testing the network in anything that looks like a computery way. It even retraces some routes. A brute force algorithm would just test every possibility sequentially. In larger networks there are pathfinding algorithms that are optimized in different ways to find routes faster, but they don’t look like this. They look more like what you see in the video below. (Hat tip to YouTuber gray utopia.)

This would need a lot of art direction and the aforementioned speed, but it would be more believable than what we see.

What’s the right projection?

Is this the right projection to use? Of course the most accurate representation of the earth is a globe, but it has many challenges in presenting a phenomenon that could happen anywhere in the world. Not the least of these is that it occludes about half of itself, a problem that is not well-solved by making it transparent. So, a projection it must be. There are many, many ways to transform a spherical surface into a 2D image, so the question becomes which projection and why.

The map uses what looks like a hand-drawn version of Peirce quincuncial projection. (But n.b. none of the projection types I compared against it matched exactly, which is why I say it was hand-drawn.) Also those longitude and latitude lines don’t make any sense; though again, a prop. I like that it’s a non standard projection because screw Mercator, but still, why Peirce? Why at this angle?

Also, why place time zone clocks across the top as if they corresponded to the map in some meaningful way? Move those clocks.

I have no idea why the Peirce map would be the right choice here, when its principle virtue is that it can be tessellated. That’s kind of interesting if you’re scrolling and can’t dynamically re-project the coastlines. But I am pretty sure the Colossus map does not scroll. And if the map is meant to act as a quick visual reference, having it dynamic means time is wasted when users look to the map and have to orient themselves.

If this map was only for tracking issues relating to Colossus, it should be an azimuthal map, but not over the north pole. The center should be the Colossus complex in Colorado. That might be right for a monitoring map in the Colossus Programming Office. This map is over the north pole, which certainly highlights the fact that the core concern of this system is the Cold War tensions between Moscow and D.C. But when you consider that, it points out another failing. 

Later in the film the map tracks missiles (not with projected paths, sadly, but with Mattel Classic Football style yellow rectangles). But missiles could conceivably come from places not on this map. What is this office to do with a ballistic-missile submarine off of the Baja peninsula, for example? Just wait until it makes its way on screen? That’s a failure. Which takes us to the crop.

Crop

The map isn’t just about missiles. Colossus can look anywhere on the planet to test network connections. (Even nowadays, near-earth orbit and outer space.) Unless the entire network was contained just within the area described on the map, it’s excluding potentially vital information. If Colossus routed itself through through Mexico, South Africa, and Uzbekistan before finally reconnecting to Guardian, users would be flat out of luck using that map to determine the leak route. And I’m pretty sure they had a functioning telephone network in Mexico, South Africa, and the Balkan countries in the 1960s.

This needs a complete picture

SInce the missiles and networks with which Colossus is concerned are potentially global, this should be a global map. Here I will offer my usual fanboy shout-outs to the Dymaxion and Pacific-focused Waterman projection for showing connectedness and physical flow, but there would be no shame in showing the complete Peirce quincuncial. Just show the whole thing.

Maybe fill in some of the Pacific “wasted space” with a globe depiction turned to points of interest, or some other fuigetry. Which gives us a new comp something like this.

I created this proof of concept manually. With more time, I would comp it up in Processing or Python and it would be even more convincing. (And might have reached London.)

All told, this display was probably eye-opening for its original audience. Golly jeepers! This thing can draw upon resources around the globe! It has intent, and a method! And they must have cool technological maps in D.C.! But from our modern-day vantage point, it has a lot to learn. If they ever remake the film, this would be a juicy thing to fully redesign.

IQ Testing

by

1

When Joe is processed after his arrest, he is taken to a general IQ testing facility. He sits in a chair wearing headphones. A recorded voice asks, “If you have one bucket that holds two gallons, and another bucket that holds five gallons, how many buckets do you have?” Into a microphone he says, incredulous that this is a question, “Two?” The recorded voice says, “Thank you!”

IDIOCRACY-IQ11

Joe looks to his left to see another subject is trying to put a square blue peg into the middle round hole of a panel and of course failing. Joe looks to his right, to see another subject with a triangular green peg in hand that he’s trying to put into the round middle hole in his interface. Small colored bulbs above each hole are unlit, but they match the colors of the matching blocks, so let’s presume they illuminate when the correct peg is inserted. When you look closely, it’s also apparent that the blocks are tethered to the panel so they’re not lost, and each peg is tethered directly below its matching hole. So there are lots and lots of cues that would let a subject figure it out. And yet, they are not. The subject to Joe’s right even eyes Joe suspiciously and turns his body to cover his test so Joe won’t try and crib…uh…“answers.”

Idiocracy_iq03

Comedy

The comedy in the scene comes from how rudimentary these challenges are. Most toddlers could complete the shape test. Even if you couldn’t figure out the shapes, you could match the colors, i.e. the blue object goes in the hole under the blue bulb. Most preschoolers could answer the spoken challenge. It underscores the stupidity of this world that generalized IQ tests for adults test below grade school levels.

IQ Testing

Since Binet invented the first one in 1904, IQ testing has a long, and problematic past (racism and using it to justify eugenic arguments, just for instance) but it can have a rational goal: How do we measure the intelligence of a set of people (students in a classroom, or applicants to intelligence jobs) for strategic decisions about aptitude, assistance, and improvement? But intelligence is a very slippery concept, and complicated to study much less test. The good news in this case is that the citizens of Idiocracy don’t have very sophisticated intellects, so very basic tests of intelligence should suffice.

Some nice things

So, that said, the shape test has some nice aspects. The panel is angled so the holes are visible and targetable, without being so vertical it’s easy to drop the pegs while manipulating them. The panel is plenty thick for durability and cleaning. The speech-to-text tech seems to work perfectly, unlike the errors and bad design that riddle most technologies in Idiocracy.

Idiocracy_iq02

A garden path match

There’s an interesting question of affordances in the device. You can see in the image above that the yellow round block fits just fine in the square hole. Ordinarily, a designer would want to prevent errors like this by, say, increasing the diameter of the round peg (and its hole) so that it couldn’t be inserted into the square hole. That version of the test would just test the time it took by even trial-and-error to match pegs to their matching holes, then you could rank subjects by time-to-completion. But by allowing the round peg to fit in the square hole, you complicate the test with a “garden path” branch where some subjects can get lost in what he thinks is a successful subtask. This makes it harder to compare subjects fairly, because another subject might not have wandered down this path and paid an unfair price in their time-to-complete.

Another complication is that this test has so many different clues. Do they notice the tethers? Do subjects notice the colored bulbs? (What about color blind subjects?) Having it test cognitive skills as well as fine-motor manipulation skills as well as perception skills seems quite complicated and less likely to enable fair comparisons. 

We must always scrutinize IQ tests because people put so much stock in them and it can be very much to an individual’s detriment. Designers of these tests ought to instrument them carefully for passive and active feedback about when the test itself is proving to be problematic.

Challenging the “superintelligent?”

A larger failing of the test is that it doesn’t challenge Joe at all. All his results would tell him is that he’s much much more intelligent than these tests are built for. Fair enough, there’s nothing in the world of Idiocracy which would indicate a need to test for superintelligence among the population, but this test had to be built by someone(s), generations ago. Could they not even have the test work on someone as smart as themselves? That’s all it would need to test Joe. But we live in a world that should be quite cautious about the emergence of a superintelligence. It would be comforting to imagine that we could test for that. Maybe we should include the Millennium Problems at the end of every test. Just in case.

GOPad.png

Another Idiot Test

As “luck” would have it, Trump tweeted an IQ test just this morning. (I don’t want to link to it to directly add any fuel to his fire, but you can Google it easily.) It’s an outrageous political video ad. As you watch it:

  • Do you believe that a single anecdote about a troubled, psychotic individual is generalizable to everyone with brown skin? Or even to everyone with brown skin who is not American and seeking legal asylum in the U.S.?
  • Do you ignore the evidence of the past decades (and the last week) that show it’s conservative white males who are much more of a problem? (Noting that vox is a liberal-leaning publication, but look at the article’s citations.)
  • Can you tell that the war drums under the ad are there only to make you feel scared, appealing to your emotions with cinematic tricks?
  • Do you uncritically fall for implicature and the slippery slope fallacy?

If the answers to all these are yes, well, sorry. You’ve failed an IQ test put to you by one of the most blatantly racist political ads since WIllie Horton. (Not many ads warrant a deathbed statement of regret, but that one did.) Maybe it’s best you take the rest of the week off treating yourself. Leave town. Take a road trip somewhere. Eat some ice cream.

For the rest of you, congratulations on passing the test. We have 5 days until the election. Kick the racist bastards and the bastards enabling the racist bastards out.

Sleeping pods

by

2

Use

Joe and Rita climb into the pods and situate themselves comfortably. Officer Collins and his assistant approach and insert some necessary intravenous chemicals. We see two canisters, one empty (for waste?) and one filled with the IV fluid. To each side of the subject’s head is a small raised panel with two lights (amber and ruby) and a blue toggle switch. None of these are labeled. The subjects fall into hibernation and the lids close.

Collins and his assistant remove a cable labeled “MASTER” from the interface and close a panel which seals the inputs and outputs. They then close a large steel door, stenciled “TOP SECRET,” to the hibernation chamber.

Idiocracy_pods04

The external interface panel includes:

  • A red LED display
  • 3 red safety cover toggle switches labeled “SET 1” “SET 2” and “SET 3.”
  • A 5×4 keypad
    • 0-9 numbers
    • Letters A–F
    • Four unlabeled white buttons

500 years later, after the top secret lab is destroyed, the pods become part of the mountains of garbage that just pile up. Sliding down an avalanche of the stuff, the pods wind up in a downtown area. Joe’s crashes through Frito’s window. At this moment the pod decides enough is enough and it wakes him. Clamps around the edge unlock. The panel cover has fallen off somewhere, and the LED display blinks the text, “unfreezing.” Joe drowsily pushes the lids open and gets out.

Its purpose in the narrative

This is a “segue” interface, mostly useful in explaining how Joe and Rita are transported safely 500 years in the future. At its base, all it needs to convey is:

  • Scienciness (lights and interfaces, check)
  • See them pass into sleep (check)
  • See why how they are kept safe (rugged construction details, clamped lid, check)
  • See the machine wake them up (check)

Is it ideal?

The ergonomics are nice. A comfortable enough coffin to sleep in. And it seems…uh…well engineered, seeing as how it winds up lasting 500 times its intended use and takes some pretty massive abuse as it slides down the mountains of garbage and through Frito’s window into his apartment. But that’s where the goodness ends. It looks solid enough to last a long long time. But there are questions.

From Collins’ point of view:

  • Why was it engineered to last 500 years, but you know, fail to have any of its interior lights or toggle switches labeled? Or have something more informative on the toggles than “SET 1”?
  • How on earth did they monitor the health of the participants over time? (Compare Prometheus’ hibernation screens.) Did they just expect it to work perfectly? Not a lot of comfort to the subjects. Did they monitor it remotely? Why didn’t that monitoring screen arouse the suspicions of the foreclosers?
  • How are subjects roused? If the procedure is something that Collins just knows, what if something happens to him? That information should be somewhere on the pod with very clear instructions.
  • How does it gracefully degrade as it runs out of resources (power, water, nutrition, air, water storage or disposal) to keep it’s occupants alive? What if the appointed person doesn’t answer the initial cry for help?

From the hibernators’ point of view:

  • How do the participants indicate their consent to go into hibernation? Can this be used as an involuntary prison?
  • How do they indicate consent to be awakened? (Not an easy problem, but Passengers illustrates why it’s necessary.)
  • What if they wake early? How do they get out or let anyone know to release them?
  • Why does the subject have to push the lid if they’re going to be weak and woozy when they waken? Can’t it be automatic, like the hibernation lids in Aliens?
  • How does the sleeper know it’s safe to get out? Certainly Joe and Rita expected to wake up in the military laboratory. But while we’re putting in the effort to engineer it to last 500 years, maybe we could account for the possibility that it’s somewhere else.
  • Can’t you put me at ease in the disorientating hypnopompic phase? Maybe some soothing graphic on the interior lid? A big red label reading, “DON’T PANIC” with an explanation?
  • Can you provide some information to help orient me, like where I am and when I am? Why does Joe have to infer the date from a magazine cover?

From a person-in-the-future point of view

  • How do the people nearby know that it contains living humans? That might be important for safekeeping, or even to take care in case the hibernators are carrying some disease to which the population has lost resistance.
  • How do we know if they’ve got some medical conditions that will need specialized care? What food they eat? Whether they are dangerous?
  • Can we get a little warning so we can prepare for all this stuff?

Is the interface believable?

Oh yes. Prototypes tend to be minimum viable thing, and usability lags far behind basic utility. Plus, this is military, known to be tough people expecting their people to be tough people without the need for civilian niceties. Plus, Collins didn’t seem too big on “details.” So very believable.

Idiocracy_surveillance14

Note that this doesn’t equate to the thing itself being believable. I mean, it was an experiment meant to last only a year. How did it have the life support resources—including power—to run for 500 times the intended duration? What brown fluid has the 273,750,000 calories needed to sustain Luke Wilson’s physique for 500 years? (Maya Rudoph lucks out needing “only” 219,000,000.) How did it keep them alive and prevent long-term bedridden problems, like pressure sores, pneumonia, constipation, contractures, etc. etc.?
See? Comedy is hard to review.

Fight US Idiocracy: Donate to close races

Reminder: Every post in this series includes some U.S.-focused calls to action for readers to help reverse the current free fall into our own Idiocracy. In the last post I provided information about how to register to vote in your state. DO THAT.
If you accidentally missed the deadline (and triple check because many states have some way to register right up to and including election day, which is 06 NOV this year), there are still things you can do. Sadly, one of the most powerful things feels crass: Donate money to close campaigns. Much of this money is spent reaching out to undecided voters via media channels, and that means the more money the more reach.

close_districts.png
ActBlue_logo.png


There are currently 68 highly competitive seats—those considered a toss up between the two parties or leaning slightly toward one. You can look at the close campaigns and donate directly, or you can donate to Act Blue, and let that organization make the call. That’s what I did. Just now. Please join me.

The Cookie

by

4

In one of the story threads, Matt uses an interface as part of his day job at Smartelligence to wrangle an AI that is the cloned a mind of a client named Greta. Matt has three tasks in this role. 

  1. He has to explain to her that she is an artificial intelligence clone of a real world person’s mind. This is psychologically traumatic, as she has decades of memories as if she were a real person with a real body and full autonomy in the world.
  2. He has to explain how she will do her job: Her responsibilities and tools.
  3. He has to “break” her will and coerce her to faithfully serve her master—who is the the real-world Greta. (The idea is that since virtual Greta is an exact copy, she understands real Greta’s preferences and can perform personal assistant duties flawlessly.)

The AI is housed in a small egg-shaped device with a single blue light camera lens. The combination of the AI and the egg-shaped device is called “The Cookie.” Why it is not called The Egg is a mystery left for the reader, though I hope it is not just for the “Cookie Monster” joke dropped late in the episode.

Communication in & out

The blue light illuminates when the AI’s attention is on a person in the environment. She can hear through a microphone embedded in the device. She can speak only with someone who is wearing a paired headset. Matt wears one during training. Without a paired headset, the AI cannot directly communicate with the outside world, only control other technologies in the house.

Black_Mirror_Cookie_headset.png

 

There is a fully immersive way for Matt to participate in the virtual world that will be discussed in the Mind Crimes post.

To keep any chat threads focused, subsequent posts will discuss separately:

It’s going to be a dark few posts. Sorry about that. This is Black Mirror, after all. On the upside, Jon Hamm have us two delightful reaction gifs across these scenes. I shall share them anon.

Black_Mirror_Cookie_33.png

Cyberspace: Bulletin Board

by

2

Johnny finds he needs a favor from a friend in cyberspace. We see Johnny type something on his virtual keyboard, then selects from a pull down menu.

JM-35-copyshop-Z-animated

A quick break in the action: In this shot we are looking at the real world, not the virtual, and I want to mention how clear and well-defined all the physical actions by actor Keanu Reeves are. I very much doubt that the headset he is wearing actually worked, so he is doing this without being able to see anything.

Will regular users of virtual reality systems be this precise with their gestures? Datagloves have always been expensive and rare, making studies difficult. But several systems offer submillimeter gestural tracking nowadays: version 2 of Microsoft Kinect, Google’s Soli, and Leap Motion are a few, and much cheaper and less fragile than a dataglove. Using any of these for regular desktop application tasks rather than games would be an interesting experiment.

Back in the film, Johnny flies through cyberspace until he finds the bulletin board of his friend. It is an unfriendly glowing shape that Johnny tries to expand or unfold without success.

JM-36-bboard-A-animated

After some more virtual typing, the bulletin board reveals itself as a cube that spins and expands. It doesn’t fill the entire screen, but does reveal the face of Strike, the owner of the bulletin board. His face is stylized as if by a real time image processing filter of the type built into most static image editors today. Strike tells Johnny to go away.

JM-36-bboard-B-animated

Johnny doesn’t give up and the conversation continues. The cube now expands to fill the screen, with Johnny looking into the cube and Strike’s face on the back wall.

JM-36-bboard-C

Johnny raises his hands and makes a threatening gesture, saying that he could crash Strike’s entire system. In cyberspace, his fingertips now have blades.

JM-36-bboard-D-animated

The face retreats in cyberspace, becoming smaller and further away. I’d like to think that Strike leaned back, and that has been mapped into a cyberspace equivalent move. The real world gesture carries its meaning to cyberspace.

A short while ago the Yakuza leader Shinji ordered the tracker to “initiate the virus.” It is at this point that we see the effect, with the cube carrying the image of Strike melting away under a bright light.

JM-36-bboard-E

While visual representations of cyber attacks are common in books and now TV and films, real world computer designers complain that no system under attack would waste processing power on rendering special effects. This is true for the defenders, but the attackers might want to show their power with a flashy display. Or perhaps these visual effects are generated by Johnny’s own cyberspace system, the 2021 equivalent of today’s warning message that a web site certificate cannot be verified. It’s certainly more attention-grabbing than a small padlock icon disappearing from one corner of your browser window.

At this point the Yakuza arrive in reality, and Jane takes the headset off and drags Johnny out of the shop.

Cyberspace: Newark Copyshop

by

1

The transition from Beijing to the Newark copyshop is more involved. After he travels around a bit, he realizes he needs to be looking back in Newark. He “rewinds” using a pull gesture and sees the copyshop’s pyramid. First there is a predominantly blue window that unfolds as if it were paper.

jm-35-copyshop-a-animated

And then the copyshop initial window expands. Like the Beijing hotel, this is a floor plan view, but unlike the hotel it stays two dimensional. It appears that cyberspace works like the current world wide web, with individual servers for each location that can choose what appearance to present to visitors.

Johnny again selects data records, but not with a voice command. The first transition is a window that not only expands but spins as it does so, and makes a strange jump at the end from the centre to the upper left.

jm-35-copyshop-c-animated

Once again Johnny uses the two-handed expansion gesture to see the table view of the records.

jm-35-copyshop-d

Johnny searches again, but either because there are so few records or because they’re in English, he doesn’t use voice commands. Instead he just runs his fingers over the cells, which highlight as he does so. Again this would be familiar to a current day spreadsheet user.

jm-35-copyshop-e

The contents of the cell are, once more, not useful. Johnny dismisses the copyshop with a sweeping arm  gesture which slides the “window” off the right of the screen.

jm-35-copyshop-f-animated

Aside: At normal viewing speed, it looks like the window disappears and, as would be the case in a 1995 or current day desktop system, reveals the previously-displayed windows underneath. Stepping through frame by frame shows that actually it reveals an identical copy of  the sliding content! Graphics programmers have always tried hard to avoid such visual glitches, but sometimes they slip into production code anyway.

Next

At this point in the plot, Johnny hasn’t found the images he so desperately needs. He thinks for a moment, and decides to contact the owner of a local bulletin board. Unknown to him, he has also been located by the Pharmakom tracker. Shinji and the Yakuza are on the way, and Shinji orders “initiate the virus.” 

Cyberspace: Beijing Hotel

by

1

After selecting its location from a map, Johnny is now in front of the virtual entrance to the hotel. The virtual Beijing has a new color scheme, mostly orange with some red.

jm-33-hotel-a

The “entrance” is another tetrahedral shape made from geometric blocks. It is actually another numeric keypad. Johnny taps the blocks to enter a sequence of numbers.

The tetrahedral keypad

jm-33-hotel-b

Note that there can be more than one digit within a block. I mentioned earlier that it can be difficult to “press” with precision in virtual reality due to the lack of tactile feedback. Looking closely, here the fingers of Johnny’s “hands” cast a shadow on the pyramid, making depth perception easier.

Something is wrong, and Johnny receives an electric shock.

jm-33-hotel-c

He reacts as if the shock is real, pulling his hands back and cursing.

In the 1980s and 1990s cyberpunk books such as Neuromancer and Hardwired and roleplaying games such as Cyberpunk and ShadowRun suggested that future virtual reality systems would be able to physically attack users, the dreaded “Black ICE”. While the more vigilant Internet copyright enforcers would probably be in favour, it seems unlikely that the liability lawyers at any computer manufacturer would allow a product that could electrocute users to be released, or that users would agree to put something like that on their hands. So this is most likely  just Johnny expressing the same frustration as a current day video gamer who loses a life in a first person shooter.

The last necessary step before being granted access is, for some reason, to reshape the pyramid.

jm-32-beijing-b-animated

Here the pyramid serves as a combination lock or puzzle as well as a keypad. It’s not obvious, but Johnny does make a small 3D rotating gesture on the entire pyramid before pulling and pushing blocks around. You can also see a second layer of structure underneath the moving shapes.

Is this an effective security system? Not really. Two-factor authentication systems rely both on knowingsomething, here a numeric code, and either havingsomething, such as a specific mobile phone or token generator, or beingsomeone, with a specific fingerprint. Reshaping the blocks is just a second thing the would-be user must know, and is just as vulnerable to being guessed as the numeric code. On the other hand, it might be enough to keep out simple-minded attacks that only try the first step.

The floorplan

The “interior” of the hotel site is first displayed as a flat plan view. This builds up incrementally, a transition known among VR developers since the film Tron came out as “rezzing up”. The completed plan then rotates into a 3D structure.

jm-34-inside-a-animated

We hear the voice feedback announce “General accounts selected” but don’t see how Johnny did this. A window expands out, and Johnny splits it in half to reveal some tabular data.

jm-34-inside-b-animated

The fax and phone records are displayed in a simple tabular view, which would not look out of place on any 1995 or indeed current day desktop computer spreadsheet. There’s no need to use 3D graphics for such this.

jm-34-inside-d

There are new interface elements here, overlaying the tabular data in pink. At the top we can read SEARCH > FAX CHARGES: FOUND. And on the right is a set of inscrutable numbers with headings GRID, LEVEL, MENU, and XYZ. This could be some orientation within the data, but it doesn’t make sense. In the lower-left we see a label for elevation, with data as “coordinates in sector 4.”

Below that a 9-key arrangement with arrow shapes. Perhaps this is a navigation aid for people using conventional 2D desktop interfaces rather than full virtual reality equipment, allowing them to move around by clicking the onscreen arrows or pressing the equivalent keys. If the keys are similar to those used in computer games, the up and down arrow keys move forward or backwards and the left and right keys rotate, assuming movement is predominantly in the horizontal plane. The other keys might be for banking or vertical movement.

Johnny searches for the outgoing fax. He does not use any graphical gestures for this, instead specifying the search date and time ranges by speaking. Words and operators are more precise than graphic symbols for this kind of database query, but typing on a virtual keyboard would be more awkward than speech.

When the particular table cell is found, he uses the fingertips of both hands to expand the contents, one of the standard gestures described in the Make It So book.

Not surprisingly for a Beijing hotel, the internal records are not in English. Johnny again uses a voice command to ask for translation.

jm-34-inside-e

The hotel record is just the metadata, not the actual images he’s looking for, suggesting that “fax” system is fully digital and the faxes themselves are treated like modern email messages and deleted once sent. The metadata does tell Johnny that the images were faxed to a online copyshop in Newark. Since it is network connected, Johnny can jump straight to it in cyberspace.

Cyberspace: the hardware

by

3

And finally we come to the often-promised cyberspace search sequence, my favourite interface in the film. It starts at 36:30 and continues, with brief interruptions to the outside world, to 41:00. I’ll admit there are good reasons not to watch the entire film, but if you are interested in interface design, this will be five minutes well spent. Included here are the relevant clips, lightly edited to focus on the user interfaces.

Click to see video of The cyberspace search.

Click to see Board conversation, with Pharmakom tracker and virus

First, what hardware is required?

Johnny and Jane have broken into a neighbourhood computer shop, which in 2021 will have virtual reality gear just as today even the smallest retailer has computer mice. Johnny clears miscellaneous parts off a table and then sits down, donning a headset and datagloves.

jm-30-hardware-a

Headset

Headsets haven’t really changed much since 1995 when this film was made. Barring some breakthrough in neural interfaces, they remain the best way to block off the real world and immerse a user into the virtual world of the computer. It’s mildly confusing to a current day audience to hear Johnny ask for “eyephones”, which in 1995 was the name of a particular VR headset rather than the popular “iPhone” of today.

Throughout this cyberspace sequence the virtual reality system Johnny uses gives vocal feedback, usually just confirming what has happened or repeating information visible in cyberspace. Johnny will also use voice commands himself. Jane seemingly can’t hear this feedback, as she has no idea what is happening other than what Johnny tells her. No earbuds or headphones are visible, but nearly all headsets then and now incorporate audio output as well as visual display so presumably sound is the function of the silver bulges at the back of the headset.

Dataglove

Datagloves are less common today. These track the position and orientation of the hands as they move, in this particular case to the bending of individual fingers. In 1995 this was done with magnetic or ultrasonic trackers on each hand and various fibre optic or potentiometer bend sensors on the finger joints, all built into a rather bulky glove. Today this can be done passively by a video camera, for example the Microsoft Kinect or Leap Motion Controller. With these technologies it’s not even necessary to paint dots on the fingers, which unlike faces have convenient gaps in between the points of interest.

Johnny mostly keeps his arms horizontal just above the table surface, but we will occasionally see him reach up. As chapter 5 of Make It So points out, trying to operate a vertical touch screen or gesture interface for any length of time is exhausting, and the same would be true if the VR system required him to frequently lift his hands and arms above the conventional keyboard height.

System status

There is also a system status display on the table.

jm-30-hardware-b

Various indicators light up as Johnny gets ready. It would be helpful if this were mirrored to the headset, so Johnny could at least see which components are working or not without removing it.

My first impression was that the grid on the table might be some kind of optical tracking aid. Then I remembered that this is a worktable, and protective table mats with a grid pattern printed on them are sold in craft and hardware shops. Not everything in the future needs to be advanced technology.

Voice feedback

As Johnny performs his various actions in cyberspace, another synthesized voice gives him constant feedback, most often telling him which actions and objects have been selected. I suggest this is for new users, who may be confused about exactly what they can and cannot do in virtual reality. (Of course, it is also very useful for telling us the audience what is happening.) Johnny himself is not a new VR users, but since this is a system assembled straight out of the box he gets the default setting. Over time a voice constantly telling you what you’ve done probably becomes irritating, which is why earlier systems were not so chatty.

The tracker

We see a second person in cyberspace during this sequence, although only briefly. This is the Pharmakom tracker, who is trying to locate Johnny and Jane for the Yakuza.

jm-30-hardware-c

He too wears a headset and gloves, but also has a one piece earphone and microphone. He uses this not for voice commands, but for a phone connection to Shinji, the Yakuza leader in a car.

He is standing in front of a lectern type display.

jm-30-hardware-d

This shows a street map, with the red cross hairs presumably the location being examined. Current day VR systems often mirror what the headset is showing to a more conventional display as this is very useful in testing and debugging. Note also the rows of unmarked buttons on either side. I’ll discuss these and similar buttons below.

Having him stand is an interesting choice. The advantage of standing in VR is that it allows the participant to bend and turn more freely, using body motion as an input as well as hands and head. The disadvantages are that this is more tiring, and that with the headset blocking the real world, it’s very easy to bump into things. The first commercial VR game, “Dactyl Nightmare” by W Industries, had a waist-high padded fence around the player to stop them falling over or walking too far and breaking the cables.

jm-14-vr1000

VR1000 restored by Simon Marston

Here the tracker is risking a painful bruised knee. Perhaps he is a standing desk enthusiast who believes the other health benefits make it worthwhile.

The Curious Unmarked Buttons…

A recurring hardware interface in Johnny Mnemonic is the grid of unmarked buttons. There were two in the upload hotel suite, the image grabber, and the fax machine. And here the lectern display used by the tracker has more of the same.

I can’t recall any others like this, with one exception: the Pixar animated short “Lifted”, which has a vast array of unmarked identical switches. But that was a deliberate caricature, making fun of terribly designed and confusing interfaces.

Research tells us that labelled buttons and keys are the best for learning and use, from computers and phones to their software equivalents on modern touchscreen phones. Even the buttons on consumer remote controls are marked, however cryptic the symbols may be. The only unmarked buttons in current day regular use are those used around the edges of displays for ATMs and in aircraft cockpits. Here the meaning of these “soft buttons” will be shown by the text or graphic displayed nearby.

jm-14-unmarked
Image by the author

But this isn’t possible for the unmarked buttons in Johnny Mnemonic, which either don’t have screens or don’t have buttons next to the screen.

…Are a platform for virtual buttons

Perhaps the buttons on the lectern are unmarked because they’re intended for use in cyberspace. If the computer system generating the virtual reality is aware of the lectern’s location in relation to the user, it could generate labels within the virtual reality that the user would perceive as exactly where the physical buttons are. The buttons would then provide actual tactile feedback for location and when pressed. 

Video Phone Calls

by

2

The characters in Johnny Mnemonic make quite a few video phone calls throughout the film, enough to be grouped in their own section on interfaces.

The first thing a modern viewer will note is that only one of the phones resembles a current day handheld mobile. This looks very strange today and it’s hard to imagine why we would ever give up our beloved iPhones and Androids. I’ll just observe that accurately predicting the future is difficult (and not really the point) and move on.

More interesting is the variety of phones used. In films from the 1950s to the 1990s, everyone uses a desk phone with a handset. (For younger readers: that is the piece you picked up and held next to your ear and mouth. There’s probably one in your parents’ house.) The only changes were the gradual replacement of rotary dials by keypads, and some cordless handsets. In 21st century films everyone uses a small sleek handheld box. But in Johnny Mnemonic every phone call uses a different interface.

New Darwin

First is the phone call Johnny makes from the New Darwin hotel.

jm-3-phone-hotel-c-adjusted

As previously discussed, Johnny is lying in bed using a remote control to select numbers on the onscreen keypad. He is facing a large wall mounted TV/display screen, with what looks like a camera at the top. The camera is realistic but unusual: as Chapter 10 of Make It So notes, films very rarely show the cameras used in visual communication.

Taxi

The second phone call takes place in Newark, as Johnny rides in a taxi from the airport. Since this is a moving vehicle rather than a room, it shows that wireless videophones also exist. We don’t see how the call is made, just the conversation. Johnny is looking at and speaking into a small screen in front of his seat.

jm-10-taxi-a-adjusted

Quick aside: The blue lines at the bottom of the screen are a street map, with the glowing dot being the taxi. While it’s not the focus of this particular interface, it’s interesting that this map seems to be fixed with the indicator moving sideways. Aircraft and now car navigators use a moving map with the indicator moving up for forward. But this is for the passenger rather than the driver so doesn’t need to be particularly useful. And it’s blue, so must be advanced.

At the other end is Ralphie, who is using a desk screen with a keyboard.

jm-10-taxi-b-adjusted

We get to see things from Ralphie’s end. His keyboard only has ten keys in two rows of five. Ralphie touches the middle key in the bottom row to end the call.

Is this a dedicated phone rather than a computer? The only full-sized keyboards we see in Johnny Mnemonic are part of systems implied to be outdated or salvaged. Perhaps by 2021 voice recognition is good enough to handle most input. Or perhaps by 2021 status indicators have changed and once again nobody who considers themselves important would have a QWERTY keyboard on their desk, leaving others to do the more “menial” typing.

Shinji’s mobile

There is a cyberspace sequence (discussed in a separate post) during which there is a conversation between a Pharmakom tracker and Shinji, the leader of the Yakuza searching for Johnny, who is in en route by car. Shinji’s phone seems to be just like a current day mobile, if perhaps a little smaller than we’re used to.

jm-12-1-car-phone

Takahashi’s desk phone

Takahashi, head of Pharmakom in Newark, has a desktop screen too. This is a general purpose computer which at various times displays video of his daughter and a corporate database entry about Anna, the Pharmakom founder.

jm-15-takahashi-a-adjusted

There is no keyboard, but later we will see that the desk surface has hand gesture tracking capability. Here the screen displays an onscreen video phone window and numeric keypad, similar to what we saw in the New Darwin sequence, but Takahashi doesn’t use that interface. Instead he just says “Get me Karl” and the phone dials the recipient automatically.

Takahashi doesn’t prefix his command with a control phrase such as“Siri” or “Computer” which would imply that the computer is always listening. For an executive with a private office this would be reasonable: who else could he be addressing? A second possibility is that the computer does voice recognition and would not respond to commands from anyone else.

Street Preacher’s Phone

As before, the recipient has chosen to show a video splash screen on connection instead of a live video feed.

jm-15-takahashi-b-adjusted

“Karl” is more commonly known as Street Preacher and works within a church of sorts. We don’t know whether this is genuine religion belief on his part or a cover operation. His phone system is built into a large book, which I thought was intended to be a Bible but Chris identifies as a 16th century ecclesiastical history. There are no controls visible, but we see Karl “pick up” by opening the book so perhaps he “hangs up” by closing it again. Otherwise it could be operated purely by voice.

jm-15-takahashi-c-adjusted

Public phone

Earlier in the film, Johnny picked up an “Infobahn 3000” handset with built in phone keypad.

jm-16-phonebooth-a

His next phone call is from a public phone booth. On screen we see the now familiar videophone keypad. (Apparently this time in cyan, although it’s a very minor color shift.). To the right of the screen are physical buttons, some of which are labelled “start” “stop” and “pause” so perhaps duplicate the onscreen controls. Johnny begins by borrowing Jane’s phone card and swiping it through the payment slot.

jm-16-phonebooth-b-adjusted

The red Infobahn handset is connected to Jane’s card by a cable, although we don’t see Johnny doing this. Johnny types on the handset keypad rather than using the onscreen controls, presumably doing some hacking through the interface.

At first sight it seems unlikely that the phone system could be hacked through an EFTPOS card reader. However there is a long and unhappy history of programmers leaving backdoors and unused functionality in products, often excused with “Well, nobody else knows about it”, which are then exploited. Payment cards themselves often have embedded integrated circuits. This particular hack is not completely implausible.

jm-16-phonebooth-c

When the Pharmakom splash screen appears, Johnny types again on the handset. He is manipulating the internal company phone system to gain access to a number that normally would not be available to the public.

jm-16-phonebooth-e-adjusted

The new number connects Johnny to a surprised corporate type who wants to know how Johnny got through.

jm-16-phonebooth-g-adjusted

We’ll learn later on that this gentleman is not at all who he seems to be. For now, note that Johnny talks and listens directly to the screen and speakers in the phone booth, not the handset he is holding.

Spider phone

Just before his brain is scanned by Spider, Johnny tries to make another call. This time he uses a typical 1990s computer CRT display and keyboard. He wears a conventional looking earpiece and microphone, and there is a small camera mounted on top of the display. He types the number on the keyboard and reaches a Pharmakom receptionist, but Johnny is interrupted.

jm-19-old-phonecall-adjusted

Van call

The last phone call is made by Johnny to Pharmakom again. This time he is in Spider’s van, which doesn’t have a built in phone like the taxi we saw earlier. He uses the handset for audio and a small portable screen for video. There must be a wireless transmitter and receiver somewhere, but it isn’t obvious.

jm-22-puppet-call-a-adjusted
jm-22-puppet-call-b-adjusted

Johnny doesn’t realise that he is actually talking to Takahashi, the head of Pharmakom, through a puppet avatar, which I’ll talk about in the next post.