In the five minute sequence about cyberspace, we’ve seen a variety of interface elements and actions. The single most impressive quality of the Johnny Mnemonic cyberspace is, to me, just how flexible and multi-modal it is.
Cyberspace was envisioned as a 3D spatial metaphor. Here we’ve seen that with specialized hardware, you can fly with hand gestures or voice commands. Or if you’re in a hurry, you can teleport directly. Destinations can be specified precisely with numbers or graphically by pointing. Other actions can be expressed by pointing and other gestures, or by typing in text or numbers, or by speaking.
The primary output channel is obviously visual, but voice feedback is used to indicate successful actions rather than cluttering up the screen. The visual style at the top level of interaction is three dimensional, but internal systems use more conventional 2D interfaces when 3D would be confusing or excessive. Most “applications” run in what is currently called full screen mode, but there is some evidence that the user could have multiple systems appearing side by side in a kind of three-dimensional window manager.
It’s been two decades since Johnny Mnemonic was released, but the film version of cyberspace still looks like an interface that would be both fun and effective. (OK, replace the datagloves with modern motion capture cameras.) That’s quite a lot of staying power, and it’s still worth seeing the film for these five minutes alone.
***
(And now, finally, after writing far more words than I’d originally intended, there are no more interfaces left to analyze in the film. Roll credits in the Report Card, coming up next.)
The transition from Beijing to the Newark copyshop is more involved. After he travels around a bit, he realizes he needs to be looking back in Newark. He “rewinds” using a pull gesture and sees the copyshop’s pyramid. First there is a predominantly blue window that unfolds as if it were paper.
And then the copyshop initial window expands. Like the Beijing hotel, this is a floor plan view, but unlike the hotel it stays two dimensional. It appears that cyberspace works like the current world wide web, with individual servers for each location that can choose what appearance to present to visitors.
Johnny again selects data records, but not with a voice command. The first transition is a window that not only expands but spins as it does so, and makes a strange jump at the end from the centre to the upper left.
Once again Johnny uses the two-handed expansion gesture to see the table view of the records.
Johnny searches again, but either because there are so few records or because they’re in English, he doesn’t use voice commands. Instead he just runs his fingers over the cells, which highlight as he does so. Again this would be familiar to a current day spreadsheet user.
The contents of the cell are, once more, not useful. Johnny dismisses the copyshop with a sweeping arm gesture which slides the “window” off the right of the screen.
Aside: At normal viewing speed, it looks like the window disappears and, as would be the case in a 1995 or current day desktop system, reveals the previously-displayed windows underneath. Stepping through frame by frame shows that actually it reveals an identical copy of the sliding content! Graphics programmers have always tried hard to avoid such visual glitches, but sometimes they slip into production code anyway.
Next
At this point in the plot, Johnny hasn’t found the images he so desperately needs. He thinks for a moment, and decides to contact the owner of a local bulletin board. Unknown to him, he has also been located by the Pharmakom tracker. Shinji and the Yakuza are on the way, and Shinji orders “initiate the virus.”
After selecting its location from a map, Johnny is now in front of the virtual entrance to the hotel. The virtual Beijing has a new color scheme, mostly orange with some red.
The “entrance” is another tetrahedral shape made from geometric blocks. It is actually another numeric keypad. Johnny taps the blocks to enter a sequence of numbers.
The tetrahedral keypad
Note that there can be more than one digit within a block. I mentioned earlier that it can be difficult to “press” with precision in virtual reality due to the lack of tactile feedback. Looking closely, here the fingers of Johnny’s “hands” cast a shadow on the pyramid, making depth perception easier.
Something is wrong, and Johnny receives an electric shock.
He reacts as if the shock is real, pulling his hands back and cursing.
In the 1980s and 1990s cyberpunk books such as Neuromancer and Hardwired and roleplaying games such as Cyberpunk and ShadowRun suggested that future virtual reality systems would be able to physically attack users, the dreaded “Black ICE”. While the more vigilant Internet copyright enforcers would probably be in favour, it seems unlikely that the liability lawyers at any computer manufacturer would allow a product that could electrocute users to be released, or that users would agree to put something like that on their hands. So this is most likely just Johnny expressing the same frustration as a current day video gamer who loses a life in a first person shooter.
The last necessary step before being granted access is, for some reason, to reshape the pyramid.
Here the pyramid serves as a combination lock or puzzle as well as a keypad. It’s not obvious, but Johnny does make a small 3D rotating gesture on the entire pyramid before pulling and pushing blocks around. You can also see a second layer of structure underneath the moving shapes.
Is this an effective security system? Not really. Two-factor authentication systems rely both on knowingsomething, here a numeric code, and either havingsomething, such as a specific mobile phone or token generator, or beingsomeone, with a specific fingerprint. Reshaping the blocks is just a second thing the would-be user must know, and is just as vulnerable to being guessed as the numeric code. On the other hand, it might be enough to keep out simple-minded attacks that only try the first step.
The floorplan
The “interior” of the hotel site is first displayed as a flat plan view. This builds up incrementally, a transition known among VR developers since the film Tron came out as “rezzing up”. The completed plan then rotates into a 3D structure.
We hear the voice feedback announce “General accounts selected” but don’t see how Johnny did this. A window expands out, and Johnny splits it in half to reveal some tabular data.
The fax and phone records are displayed in a simple tabular view, which would not look out of place on any 1995 or indeed current day desktop computer spreadsheet. There’s no need to use 3D graphics for such this.
There are new interface elements here, overlaying the tabular data in pink. At the top we can read SEARCH > FAX CHARGES: FOUND. And on the right is a set of inscrutable numbers with headings GRID, LEVEL, MENU, and XYZ. This could be some orientation within the data, but it doesn’t make sense. In the lower-left we see a label for elevation, with data as “coordinates in sector 4.”
Below that a 9-key arrangement with arrow shapes. Perhaps this is a navigation aid for people using conventional 2D desktop interfaces rather than full virtual reality equipment, allowing them to move around by clicking the onscreen arrows or pressing the equivalent keys. If the keys are similar to those used in computer games, the up and down arrow keys move forward or backwards and the left and right keys rotate, assuming movement is predominantly in the horizontal plane. The other keys might be for banking or vertical movement.
Johnny searches for the outgoing fax. He does not use any graphical gestures for this, instead specifying the search date and time ranges by speaking. Words and operators are more precise than graphic symbols for this kind of database query, but typing on a virtual keyboard would be more awkward than speech.
When the particular table cell is found, he uses the fingertips of both hands to expand the contents, one of the standard gestures described in the Make It So book.
Not surprisingly for a Beijing hotel, the internal records are not in English. Johnny again uses a voice command to ask for translation.
The hotel record is just the metadata, not the actual images he’s looking for, suggesting that “fax” system is fully digital and the faxes themselves are treated like modern email messages and deleted once sent. The metadata does tell Johnny that the images were faxed to a online copyshop in Newark. Since it is network connected, Johnny can jump straight to it in cyberspace.
Cyberspace is usually considered to be a 3D spatial representation of the Internet, an expansion of the successful 2D desktop metaphor. The representation of cyberspace used in books such as Neuromancer and Snow Crash, and by the film Hackers released in the same year, is an abstract cityscape where buildings represent organisations or individual computers, and this what we see in Johnny Mnemonic. How does Johnny navigate through this virtual city?
Gestures and words for flying
Once everything is connected up, Johnny starts his journey with an unfolding gesture. He then points both fingers forward. From his point of view, he is flying through cyberspace. He then holds up both hands to stop.
Both these gestures were commonly used in the prototype VR systems of 1995. They do however conflict with the more common gestures for manipulating objects in volumetric projections that are described in Make It So chapter 5. It will be interesting to see which set of gestures is eventually adopted, or whether they can co-exist.
Later we will see Johnny turn and bank by moving his hands independently.
We also see him using voice commands, saying “hold it” to stop forward motion immediately. Later we see him stretch one arm out and bring it back, apparently reversing a recent move.
In cyberpunk and related fiction users fly everywhere in cyberspace, a literal interpretation of the spatial metaphor. This is also how users in our real world MUD and MOO cyberspaces start. After a while, travelling through all the intermediate locations between your start and destination gets tedious. MUDs and MOOs allow teleporting, a direct jump to the desired location, and the cyberspace in Johnny Mnemonic has a similar capability.
Gestures for teleporting
Mid sequence, Johnny wants to jump to the Beijing hotel where the upload took place. To do this, he uses a blue geometric shape at the lower left of his view, looking like a high tech, floating tetrahedron. Johnny slowly spins this virtual object using repeated flicking gestures with his left hand, with his ring and middle fingers held together.
It looks very similar to the gesture used on a current-day smartphone to flick through a photo album or set of application icon screens. And in this case, it causes a blue globe to float into view (see below).
Johnny grabs this globe and unfolds it into a fullscreen window, using the standard Hollywood two handed “spread” gesture described in Chapter 5 of Make It So.
The final world map fills the entire screen. Johnny uses his left hand to enter a number on a HUD style overlay keypad, then taps on the map to indicate China.
I interpret this as Johnny using the hotel phone number to specify his destination. It would not be unusual for there to be multiple hotels with the same name within a city such as Beijing, but the phone number should be unique. But since Johnny is currently in North America, he must also specify the international dialing code or 2021 equivalent, which he can do just by pointing. And this is a well-designed user interface which accepts not only multimodal input, but in any order, rather than forcing the user to enter the country code first.
Keyboards and similar physical devices often don’t translate well into virtual reality, because tactile feedback is non-existent. Even touch typists need the feeling of the physical keyboard, in particular the slight concavity of the key tops and the orientation bumps on the F and J keys, to keep their fingers aligned. Here though there is just a small grid of virtual numbers which doesn’t require extended typing. Otherwise this is a good design, allowing Johnny to type a precise number and just point to a larger target.
Next
After he taps a location, the zoomrects indicate a transition into a new cyberspace, in this case, Beijing.
The opening shot of Johnny Mnemonic is a brightly coloured 3D graphical environment. It looks like an abstract cityscape, with buildings arranged in rectangular grid and various 3D icons or avatars flying around. Text identifies this as the Internet of 2021, now cyberspace.
Strictly speaking this shot is not an interface. It is a visualization from the point of view of a calendar wake up reminder, which flies through cyberspace, then down a cable, to appear on a wall mounted screen in Johnny’s hotel suite. However, we will see later on that this is exactly the same graphical representation used by humans. As the very first scene of the film, it is important in establishing what the Internet looks like in this future world. It’s therefore worth discussing the “look” employed here, even though there isn’t any interaction.
Cyberspace is usually equated with 3D graphics and virtual reality in particular. Yet when you look into what is necessary to implement cyberspace, the graphics really aren’t that important.
MUDs and MOOs: ASCII Cyberspace
People have been building cyberspaces since the 1980s in the form of MUDs and MOOs. At first sight these look like old style games such as Adventure or Zork. To explore a MUD/MOO, you log on remotely using a terminal program. Every command and response is pure text, so typing “go north” might result in “You are in a church.” The difference between MUD/MOOs and Zork is that these are dynamic multiuser virtual worlds, not solitary-player games. Other people share the world with you and move through it, adventuring, building, or just chatting. Everyone has an avatar and every place has an appearance, but expressed in text as if you were reading a book.
guest>>@go #1914 Castle entrance A cold and dark gatehouse, with moss-covered crumbling walls. A passage gives entry to the forbidding depths of Castle Aargh. You hear a strange bubbling sound and an occasional chuckle. Obvious exits: path to Castle Aargh (#1871) enter to Bridge (#1916)
Most impressive of all, these are virtual worlds with built-in editing capabilities. All the “graphics” are plain text, and all the interactions, rules, and behaviours are programmed in a scripting language. The command line interface allows the equivalent of Emacs or VI to run, so the world and everything in itcan be modified in real time by the participants. You don’t even have to restart the program. Here a character creates a new location within a MOO, to the “south” of the existing Town Square:
laranzu>>@dig MyNewHome laranzu>> @describe here as “A large and spacious cave full of computers” laranzu>> @dig north to Town Square
The simplicity of the text interfaces leads people to think these are simple systems. They’re not. These cyberspaces have many of the legal complexities found in the real world. Can individuals be excluded from particular places? What can be done about abusive speech? How offensive can your public appearance be? Who is allowed to create new buildings, or modify existing ones? Is attacking an avatar a crime? Many 3D virtual reality system builders never progress that far, stopping when the graphics look good and the program rarely crashes. If you’re interested in cyberspace interface design, a long running textual cyberspace such as LambdaMOO or DragonMUD holds a wealth of experience about how to deal with all these messy human issues.
So why all the graphics?
So it turns out MUDs and MOOs are a rich, sprawling, complex cyberspace in text. Why then, in 1995, did we expect cyberspace to require 3D graphics anyway?
The 1980s saw two dimensional graphical user interfaces become well known with the Macintosh, and by the 1990s they were everywhere. The 1990s also saw high end 3D graphics systems becoming more common, the most prominent being from Silicon Graphics. It was clear that as prices came down personal computers would soon have similar capabilities.
At the time of Johnny Mnemonic, the world wide web had brought the Internet into everyday life. If web browsers with 2D GUIs were superior to the command line interfaces of telnet, FTP, and Gopher, surely a 3D cyberspace would be even better? Predictions of a 3D Internet were common in books such as Virtual Reality by Howard Rheingold and magazines such as Wired at the time. VRML, the Virtual Reality Markup/Modeling Language, was created in 1995 with the expectation that it would become the foundation for cyberspace, just as HTML had been the foundation of the world wide web.
Twenty years later, we know this didn’t happen. The solution to the unthinkable complexity of cyberspace was a return to the command line interface in the form of a Google search box.
Abstract or symbolic interfaces such as text command lines may look more intimidating or complicated than graphical systems. But if the graphical interface isn’t powerful enough to meet their needs, users will take the time to learn how the more complicated system works. And we’ll see later on that the cyberspace of Johnny Mnemonic is not purely graphical and does allow symbolic interaction.
