Zed-Eyes

In the world of “White Christmas”, everyone has a networked brain implant called Zed-Eyes that enables heads-up overlays onto vision, personalized audio, and modifications to environmental sounds. The control hardware is a thin metal circle around a metal click button, separated by a black rubber ring. People can buy the device with different color rings, as we see alternately see metal, blue, and black versions across the episode.

To control the implant, a person slides a finger (thumb is easiest) around the rim of a tiny touch device. Because it responds to sliding across its surface, let’s say the device must use a sensor similar to the one used in The Entire History of You (2011) or the IBM Trackpoint,

A thumb slide cycles through a carousel menu. Sliding can happen both clockwise and counterclockwise. It even works through gloves.

HUD_menu.gif

The button selects or executes the selected action. The complete list of carousel menu options we see in the episode are: SearchCameraMusicMailCallMagnifyBlockMapThe particular options change across scenes, so it is context-aware or customizable. We will look at some of the particular functions in later posts. For now, let’s discuss the “platform” that is Zed-eyes.

Analysis

There’s not much to discuss about the user interface. The carousel a mature, if constrained, interface model familiar to anyone who has used an iPod. We know the constraints and benefits of such a system, and the Zed-Eyes content seems to fit this kind of interface well.

Hardware

The main question about the hardware is that is must be very very easy to lose or misplace. It would make sense for the Zed-Eyes to help you locate it when you need help, but we don’t see a hint of this in the show.

I think the little watch-battery form factor is a bad design. It’s easy to lose and hard to find and requires a lot of precision to use. Since this exists in a world with very high fidelity image recognition and visual processing, better would be to get rid of input hardware altogether.

Let the user swipe with their thumb across their index finger (or really, any available surface) and have the HUD read that as input. To distinguish real-world interactions that should not have consequence—like swiping dust off a computer—from input meant for the HUD, it could track the user’s visual focal point. When the user’s eyes focus on the empty space in the air right above where they’re swiping, the system knows swiping is meant to affect the interface.

With this kind of interaction there would be no object to lose, and of course save whatever entity provides this service the costs of the hardware and maintenance.

We must note that such a design might not play well cinematically, as viewers might not understand what was happening at first, but understanding the hardware is not critical to understanding the plot-critical effects of using the technology.

Cyborgs in social space

A last question is about the invisibility of the technology. This can cause problems when a user is known to be hearing, but functionally deaf because they are listening to music loudly, and the people around them can’t tell that. Someone could be speaking to the user and believe their non-response is disrespect. It could cause safety problems as, say, a bicyclist barrels towards them on a sidewalk, ringing their bell, expecting the user to move. This can allow privacy abuse as a user can take pictures in circumstances that should be private.

Joe, the moment he is taking a picture of Beth.

One solution would be to make the presence of the tech and interactions quite visible. Glowing pupils and large, obvious gestural control, for example. But in a world where everyone has the technology, the Zed-Eyes can simply limit the behavior of photographs to permitted places, times, and according to the preferences of the people in the photograph. If someone is listening to music and functionally deaf, a real time overlay could inform people around them. This guy is listening to music. If a place is private, the picture option could be disabled with feedback to the user of this. Sorry, pictures are not allowed here.

The visibility we want for ubiquitous technology can be virtual, and provide feedback to everyone involved.

3 of 3: Brain Hacking

The hospital doesn’t have the equipment to decrypt and download the actual data. But Jane knows that the LoTeks can, so they drive to the ruined bridge that is the LoTek home base. As mentioned earlier under Door Bombs and Safety Catches the bridge guards nearly kill them due to a poorly designed defensive system. Once again Johnny is not impressed by the people who are supposed to help him.

When Johnny has calmed down, he is introduced to Jones, the LoTek codebreaker who decrypts corporate video broadcasts. Jones is a cyborg dolphin.

jm-24-jones-animated

Jones has not just an implant like Johnny or an augmented nervous system like Jane, but a full neural brain interface that gives him active control. The thing behind his eye and under the cable can rotate, and he can also direct and control an external microwave radar dish. In the background there are a lot of cables and blinking lights apparently connecting Jones to the LoTek video broadcast gear.

For his part, Johnny is sitting in a chair, upper head strapped into a helmet-like brain scanner. This one is very big and clunky, perhaps because it is salvaged old technology or perhaps because this is not just a passive scanner, so needs additional elements and power to actively modify the brain.

jm-26-download-b

When this starts operating, we see the same strobing white light flashes that the first scanner used.

J-Bone, the LoTek leader, uses a handheld camera to feed the first access code image into the system. This is yet another piece of talking technology, announcing that the first image has been loaded.

jm-26-download-c

The captured image is processed to remove the perspective keystoning, and displayed on one of three small panels on the wall, side by side. That specialised displays are made solely for displaying three images suggests that this form of access code is a standard method of data protection in 2021. The other two panels display rolling static.

jm-26-download-d-adjusted

Wait…static?

Why is there static in a 2021 system for displaying computer images? It’s not just because they’re analog: old CRT computer monitors went blank if there was nothing to display. This is a missing or scrambled signal.

Since the LoTeks rely on scavenged technology, it’s quite likely that they are the last people on the planet still using coax video cables. Another possibility is that this is a deliberate imitation, as we saw earlier with the digital fax machine that made analog sounds. Computer graphics programmers are constantly wondering whether the screen is black because they didn’t draw anything, or black because they accidently drew everything in that color. The rolling static makes it clear that there is no image to display, not that the image is blank.

The first download attempt is interrupted by the Yakuza attacking the bridge. There’s some equipment damage, but by the end of the fight Johnny and company have recovered the second access code image.

jm-26-download-f-adjusted

Still not enough, but Johnny now attempts to “hack his own brain” which is successful (discussed below). The data is finally downloaded and the LoTeks broadcast the cure for NAS worldwide.

Tech Tease

The hacking and downloading take place in another virtual reality space, the internal representation of the implant. These sequences are action-packed and filled with eye catching visuals. If we wanted, there’s much that could be written about, from the visual representations of hacking used in film and TV to the advisability of transmitting vital scientific data through a video encoder. But we never get to see the interface!

Instead, we see Johnny just sit and do nothing other than maintain a death grip on the chair armrests and try not to grind his teeth into fragments. According to the running commentary on the hack provided by J-Bone, Johnny is performing actions in VR. It’s possible that the LoTek brain scanner is a true brain interface that gives him active control by thought alone with no sound or audio experience.

But this is evidently high grade encryption, which could only be broken by an expert hacker. Without visible controls for the brain scanner, the expert hacker would need to be using a direct brain interface. And the hacker would naturally have their own avatar. The only person present who definitely meets all these requirements is not Johnny, but Jones.

Could Jones be really doing all the work? In the original short story it was Jones, and here he’s certainly doing something in virtual reality. Johnny would make a useful distraction, and J-Bone might deliberately mislead the non-LoTek bystanders to keep Jones a secret.

jm-26-download-g

Whether it’s Johnny or Jones, we only get to see what happens, not how. Rather than end on this disappointing note, I’ll now jump back to discuss the more rewarding interfaces for phone calls and cyberspace search sequence. 

The secret of the tera-keyboard

GitS-Hands-01

Many characters in Ghost in the Shell have a particular cybernetic augmentation that lets them use specially-designed keyboards for input.

Triple-hands

To control this input device, the user’s hands are replaced with cybernetic ones. Normally they look and behave like normal human hands. But when needed, the fingers of these each split into three separate mini-fingers, which can move independently. These 30 spidery fingerlets triple the number of digits at play, dancing across the keyboard at a blinding 24 positions per second.

GitS-Hands-02

The tera-keyboard

The keyboards for which these hands were built have eight rows. The five rows nearest the user have single symbols. (QWERTY English?) Three rows farthest from the user have keys labeled with individual words. Six other keys at the top right are unlabeled. Each key glows cyan when pressed and is flush with the board itself. In this sense it works more like a touch panel than a keyboard. The board has around 100 keys in total.

GitS-Hands-03

What’s nifty about the keyboard itself is not the number of keys. Modern keyboards have about that many. What’s nifty is that you can see these keyboards are massively chorded, with screen captures from the film showing nine keys being pressed at once.

GitS-Hands-04

Let’s compare. (And here I owe a great mathematical debt of thanks to Nate Clinton for his mastery of combinatorics.) The keyboard I’m typing this blog post on has 104 keys, and can handle five keys being pressed at once, i.e, a base key like “S” and up to four modifier keys: shift, control, option, and command. If you do the math, this allows for 1600 different keypresses. That’s quite a large range of momentary inputs.

But on the tera-keyboard you’re able to press nine keys at once, and more importantly, it looks like any key can be chorded with any other key. If we’re conservative in the interpretation and presume that 9 keys must be pressed at once—leaving 6 fingerlets free to move into position for the next bit of input—that still adds up to a possible 2,747,472,247,520 possible keypresses (≈2.7 trillion). That’s about nine orders of magnitude more than our measley 1600. At 24 keypresses per second, that’s a data rate of 6.5939334e+13 per second.

GitS-Hands-05

So, ok, yes, fast, but it only raises the question:

What exactly is being input?

It’s certainly more than just characters. Unicode‘s 110,000 characters is a fraction of a fraction of this amount of data, and it covers most of the world’s scripts.

Is it words? Steven Pinker in his book The Language Instinct cites sources estimating the number of words in an educated person’s vocabulary is around 60,000. This excludes proper names, numbers, foreign words, any scientific terms, and acronyms, so it’s pretty conservative. Even if we double it, we’re still around the number of characters in Unicode. So even if the keyboard had one keypress for every word the user could possibly know and be thinking at any particular moment, the typist would only be using a fragment of its capacity.

typing

The only thing that nears this level of data on a human scale is the human brain. With a common estimate of 100 billion neurons, the keyboard could be expressing the state of it’s users brain, 24 times a second, distinguishing between 10 different states of each neuron.

This also bypasses one of the concerns of introducing an input mechanism like this that requires active manipulation: The human brain doesn’t have the mechanisms to manage 30 digits and 9-key-chording at this rate. To get it to where it could manage this kind of task would need fairly massive rewiring of the brain of the user. (And if you could do that, why bother with the computer?)

But if it’s a passive device, simply taking “pictures” of the brain and sharing those pictures with the computer, it doesn’t require that the human be reengineered, just re-equipped. It requires a very smart computer system able to cope with and respond to that kind of input, but we see that exact kind of artificial intelligence elsewhere in the film.

The “secret”

Because of the form factor of hands and keyboard, it looks like a manual input device. But looking at the data throughput, the evidence suggests that it’s actually a brain interface, meant to keep the computer up to date with whatever the user is thinking at that exact moment and responding appropriately. For all the futurism seen in this film, this is perhaps the most futuristic, and perhaps the most surprising.

GitS-Hands-06