Cyberspace: the hardware

And finally we come to the often-promised cyberspace search sequence, my favourite interface in the film. It starts at 36:30 and continues, with brief interruptions to the outside world, to 41:00. I’ll admit there are good reasons not to watch the entire film, but if you are interested in interface design, this will be five minutes well spent. Included here are the relevant clips, lightly edited to focus on the user interfaces.

Click to see video of The cyberspace search.

Click to see Board conversation, with Pharmakom tracker and virus

First, what hardware is required?

Johnny and Jane have broken into a neighbourhood computer shop, which in 2021 will have virtual reality gear just as today even the smallest retailer has computer mice. Johnny clears miscellaneous parts off a table and then sits down, donning a headset and datagloves.

jm-30-hardware-a

Headset

Headsets haven’t really changed much since 1995 when this film was made. Barring some breakthrough in neural interfaces, they remain the best way to block off the real world and immerse a user into the virtual world of the computer. It’s mildly confusing to a current day audience to hear Johnny ask for “eyephones”, which in 1995 was the name of a particular VR headset rather than the popular “iPhone” of today.

Throughout this cyberspace sequence the virtual reality system Johnny uses gives vocal feedback, usually just confirming what has happened or repeating information visible in cyberspace. Johnny will also use voice commands himself. Jane seemingly can’t hear this feedback, as she has no idea what is happening other than what Johnny tells her. No earbuds or headphones are visible, but nearly all headsets then and now incorporate audio output as well as visual display so presumably sound is the function of the silver bulges at the back of the headset.

Dataglove

Datagloves are less common today. These track the position and orientation of the hands as they move, in this particular case to the bending of individual fingers. In 1995 this was done with magnetic or ultrasonic trackers on each hand and various fibre optic or potentiometer bend sensors on the finger joints, all built into a rather bulky glove. Today this can be done passively by a video camera, for example the Microsoft Kinect or Leap Motion Controller. With these technologies it’s not even necessary to paint dots on the fingers, which unlike faces have convenient gaps in between the points of interest.

Johnny mostly keeps his arms horizontal just above the table surface, but we will occasionally see him reach up. As chapter 5 of Make It So points out, trying to operate a vertical touch screen or gesture interface for any length of time is exhausting, and the same would be true if the VR system required him to frequently lift his hands and arms above the conventional keyboard height.

System status

There is also a system status display on the table.

jm-30-hardware-b

Various indicators light up as Johnny gets ready. It would be helpful if this were mirrored to the headset, so Johnny could at least see which components are working or not without removing it.

My first impression was that the grid on the table might be some kind of optical tracking aid. Then I remembered that this is a worktable, and protective table mats with a grid pattern printed on them are sold in craft and hardware shops. Not everything in the future needs to be advanced technology.

Voice feedback

As Johnny performs his various actions in cyberspace, another synthesized voice gives him constant feedback, most often telling him which actions and objects have been selected. I suggest this is for new users, who may be confused about exactly what they can and cannot do in virtual reality. (Of course, it is also very useful for telling us the audience what is happening.) Johnny himself is not a new VR users, but since this is a system assembled straight out of the box he gets the default setting. Over time a voice constantly telling you what you’ve done probably becomes irritating, which is why earlier systems were not so chatty.

The tracker

We see a second person in cyberspace during this sequence, although only briefly. This is the Pharmakom tracker, who is trying to locate Johnny and Jane for the Yakuza.

jm-30-hardware-c

He too wears a headset and gloves, but also has a one piece earphone and microphone. He uses this not for voice commands, but for a phone connection to Shinji, the Yakuza leader in a car.

He is standing in front of a lectern type display.

jm-30-hardware-d

This shows a street map, with the red cross hairs presumably the location being examined. Current day VR systems often mirror what the headset is showing to a more conventional display as this is very useful in testing and debugging. Note also the rows of unmarked buttons on either side. I’ll discuss these and similar buttons below.

Having him stand is an interesting choice. The advantage of standing in VR is that it allows the participant to bend and turn more freely, using body motion as an input as well as hands and head. The disadvantages are that this is more tiring, and that with the headset blocking the real world, it’s very easy to bump into things. The first commercial VR game, “Dactyl Nightmare” by W Industries, had a waist-high padded fence around the player to stop them falling over or walking too far and breaking the cables.

jm-14-vr1000

VR1000 restored by Simon Marston

Here the tracker is risking a painful bruised knee. Perhaps he is a standing desk enthusiast who believes the other health benefits make it worthwhile.

The Curious Unmarked Buttons…

A recurring hardware interface in Johnny Mnemonic is the grid of unmarked buttons. There were two in the upload hotel suite, the image grabber, and the fax machine. And here the lectern display used by the tracker has more of the same.

I can’t recall any others like this, with one exception: the Pixar animated short “Lifted”, which has a vast array of unmarked identical switches. But that was a deliberate caricature, making fun of terribly designed and confusing interfaces.

Research tells us that labelled buttons and keys are the best for learning and use, from computers and phones to their software equivalents on modern touchscreen phones. Even the buttons on consumer remote controls are marked, however cryptic the symbols may be. The only unmarked buttons in current day regular use are those used around the edges of displays for ATMs and in aircraft cockpits. Here the meaning of these “soft buttons” will be shown by the text or graphic displayed nearby.

jm-14-unmarked
Image by the author

But this isn’t possible for the unmarked buttons in Johnny Mnemonic, which either don’t have screens or don’t have buttons next to the screen.

…Are a platform for virtual buttons

Perhaps the buttons on the lectern are unmarked because they’re intended for use in cyberspace. If the computer system generating the virtual reality is aware of the lectern’s location in relation to the user, it could generate labels within the virtual reality that the user would perceive as exactly where the physical buttons are. The buttons would then provide actual tactile feedback for location and when pressed. 

Brain Upload

Once Johnny has installed his motion detector on the door, the brain upload can begin.

3. Building it

Johnny starts by opening his briefcase and removing various components, which he connects together into the complete upload system. Some of the parts are disguised, and the whole sequence is similar to an assassin in a thriller film assembling a gun out of harmless looking pieces.

jm-6-uploader-kit-a

It looks strange today to see a computer system with so many external devices connected by cables. We’ve become accustomed to one piece computing devices with integrated functionality, and keyboards, mice, cameras, printers, and headphones that connect wirelessly.

Cables and other connections are not always considered as interfaces, but “all parts of a thing which enable its use” is the definition according to Chris. In the early to mid 1990s most computer user were well aware of the potential for confusion and frustration in such interfaces. A personal computer could have connections to monitor, keyboard, mouse, modem, CD drive, and joystick – and every single device would use a different type of cable. USB, while not perfect, is one of the greatest ever improvements in user interfaces.

Why not go wireless? Wireless devices remove the need for a physical connection, but this means that anyone, not just you, could potentially connect. So instead of worrying about whether we have the right kind of cable, we now worry about the right kind of Bluetooth pairing and WiFi encryption password scheme. Mobile wireless devices also need their own batteries, which have to be charged. So wireless may seem visually cleaner, but comes with its own set of problems.

As of early 2016 we have two new standards, Lightning and USB-C, that are orientation-independent (only fifty years after audio cables), high bandwidth, and able to transmit power to peripherals as well. Perhaps by 2021 cables will have made a comeback as the usual way to connect devices.

2. Explaining it

Johnny explains the process to the scientists. He needs them to begin the upload by pushing a button, helpfully labelled “start”, on the gadget that resembles an optical disk drive. There’s a big red button as well, which is not explained but would make an excellent “cancel” button.

jm-6-uploader-kit-b

It would be simpler if Johnny just did this himself. But we will shortly discover that the upload process is apparently very painful. If Johnny had his hands near the system, he might involuntarily push another button or disturb a cable. So for them, having a single, easily differentiated button to press minimizes their chance of messing it up.

1. Making codes

He also sticks a small black disk on the hotel room’s silver remote control. The small disk is evidently is a wireless controller or camera of some kind. The scientists must watch the upload progress counter, and as it approaches the end, use this modified remote to grab three frames from the TV display, which will become the “access code” for the data. (More on this below.)

jm-6-uploader-kit-x

None of the buttons on this remote have markings or labels, but neither Johnny nor the scientist who will be using it are bothered. Perhaps this hotel chain tries to please every possible guest by not favouring any particular language? But even in that case, I’d expect there to be some kind of symbols on the buttons and a multilingual manual to explain the meaning of each. Maybe Johnny spends so much time in hotel suites that he has memorised the button layout?

Short of a mind reading remote that can translate any button press into “what the user intended”, I have to admit this is a terrible interface.

(There is a label on the black disk, but I have no idea what it means or even which script that is. Anyone?)

0. Go go go

Johnny plugs in his implant, puts on a headset with more cables, and bites down on a mouthguard. He’s ready.

jm-6-uploader-kit-d

The scientist pushes the start button and the upload begins. Johnny sees the data stream in his headset as a flood of graphics and text.

jm-7-uploading-a

Why does he need the headset when there is a direct cable connection to the implant? The movie doesn’t make it explicit. It could be related to the images used as the access code. (More on this below.) Perhaps the images need to be processed by the recipient’s own optic nerve system for more reliable storage?

Still, in the spirit of apologetics we should try to find a better explanation than “an opportunity for 1995 cutting edge computer generated graphics.” Perhaps it is a very flashy progress indicator? Older computer systems had blinking lights on disk drives to indicate activity, copied on some of today’s USB sticks. Current-day file upload or download GUIs have progress bars. As processing and graphics capabilities increase, it will be possible for software to display thumbnails or previews of the actual data being transferred without slowing down.

Unfortunately there is an argument against this, which is that the obvious upload progress indicator is a numeric display counting gigabytes down to zero, and it makes a fast chirping sound as a sonic indicator as well. The counter shows the data flowing at gigabytes per second, the entire upload lasting about a minute. There’s also the problem that it’s not Johnny who is interested in knowing whether the upload is scientific data rather than, say, a video collection; but the scientists, and they can’t see it.

jm-7-uploading-b

As the counter drops below one hundred, the scientist points the remote with black disk at the TV display, currently showing a cartoon, and presses the middle button. The image from the TV appears overlaid on the data stream to Johnny. This is a little odd, because Johnny assured the scientists that he wouldn’t know what the access codes were himself. Maybe these brief flashes are not enough time for him to remember these particular images among the gigabytes of visual content. But the way they’re shown to us, I’ll bet you can remember them when they come up again later in the plot.

jm-7-uploading-d

Two more images are grabbed before the counter stops. When the upload finishes, the three images are printed out. (In the original film this is shown upside down, so I have rotated the image.)

jm-7-uploading-f-rotated

Tagged

So what are the images for? The script isn’t clear. I suggest that the images are being used as the equivalents of very large random numbers for whatever cryptography scheme protects the data against unauthorised access. Some current day systems use the timing of key presses and mouse movements as a source of randomness because humans simply can’t move their fingers with microsecond precision. Here, the human element makes it impossible to predict exactly which frame is chosen.

Humans also find images much easier to recognise than hundred digit numbers. Anyone who has seen the printout will be able to say whether a particular image is part of the access code or not with a high degree of confidence. In computer systems today, Secure Shell, or ssh, is a widely used encrypted terminal program for secure access to servers. Recent versions of ssh have a ‘randomart’ capability which shows a small ASCII icon generated from the current cryptographic key to everyone who logs on. If this ASCII icon appears different, this alerts everyone that the server key has been changed.

There’s one potential usability problem with the whole “pick three random images” mechanism. The last frame was grabbed when the counter was very close to zero. What would have happened if he had been too slow and missed altogether? Wouldn’t it be more reliable to have the upload system automatically grab the images rather than rely on a human? Chris suggests that maybe it secretly did grab three images that could have used without human input, but privileged the human input since it was more reliably random.

Quick aside: You may be asking, if images would be so wonderful, why aren’t we using them in this way already? It’s because our current security systems need not just very large random numbers, but very large random numbers with particular mathematical properties such as being prime. But let’s cut Johnny Mnemonic some slack,  saying that by 2021 we may have new algorithms.

OK, back to the plot.

-1. Sharing the codes

The access codes are to be faxed from Beijing to Newark, although this gets interrupted by the Yakuza intruders. This is yet another device with unmarked buttons.

jm-7-uploading-g

This device makes the same beeps and screeches as a 1990s analog fax machine. Since we’ll later learn that all the fax messages and phone calls are stored digitally in cyberspace, this must be a skeuomorphism, the old familiar audio tones now serving just as progress indicators.

As with other audio output, the tones allow the user to know that the transmission is proceeding and when it ends without having to pay full attention to the device. On the other hand, there is potential for confusion here as the digital upload is (presumably) much faster. Most current day computer systems could upload three photos, even in high resolution, well before the sequence of tones would complete. Users would most likely wait longer than actually necessary before moving on to their next task.

-2. Washing up

During the upload Johnny clenches his fists and bites his mouthguard. When the upload finishes, he retreats to the bathroom in considerable pain. At one point blood flows from his nose, and he swipes his hand over the tap to wash it down the drain. The bathroom announces that the water temperature is 17 degrees. We’ll come back to this later.

jm-8-bathroom-tap

As Make It So emphasises in the chapter on brain interfaces, there is nothing in our current knowledge to suggest that writing or reading memories to or from a human brain would be painful. On the other hand, we know that information in the brain isthe shape of the neurons in the brain. Who knows what side effects will happen as those neurons are disconnected and reconnected as they need to be? We don’t know, so can’t really say whether it would hurt or not.

-3. Escaping the Yakuza

As mentioned in a prior post, while he is in the bathroom, the motion detector Johnny installed on the hotel door isn’t very effective and the Yakuza break in, kill everyone else, and acquire the second of the three access code images. Johnny escapes with the first image and flies to Newark, North America.