Fritzes 2024 Winners

So I missed synchronizing the Fritzes with the Oscars. By like, a lot. A lot a lot. That hype curve has come and gone. (In my defense, it’s been an intensely busy year.) I don’t think providing nominees and then waiting to reveal winners makes sense now, so I’ll just talk about them. It was another year where there weren’t a lot of noteworthy speculative interfaces, from an interaction design point of view. This is true enough that I didn’t have enough candidates to fill out my usual three categories of Believable, Narrative, and Overall. So, I’m just going to do a round-up of some of the best interfaces as I saw them, and at the end, name an absolute favorite.

The Kitchen

In a dystopian London, the rich have eliminated all public housing but one last block known as The Kitchen. Izi and Benji live there and are drawn together by the death of Benji’s mother, who turns out to be one of Izi’s romantic partners from the past. The film is full of technology, but the one part that really struck me was the Life After Life service where Izi works and where Benji’s mom’s funeral happens. It’s reminiscent of the Soylent Green suicide service, but much better done, better conceived. The film has a sci-fi setting, but don’t expect easy answers and Marvel-esque plot here. This film about relationships amid struggle and ends quite ambiguously.

The funerary interfaces are mostly translucent cyans with pinstripe dividing lines to organize everything. In the non-funerary the cyan is replaced with bits of saturated red. Everything funerary and non- feels as if it has the same art direction, which lends to reading the interfaces extradiegetically, but maybe that’s part of the point?

Pod Generation

This dark movie considers what happens if we gestated babies in technological wombs called pods.  The interactions with the pod are all some corporate version of intuitive, as if Apple had designed them. (Though the swipe-down to reveal is exactly backwards. Wouldn’t an eyelid or window shade metaphor be more natural? Maybe they were going for an oven metaphor, like bun in the oven? But cooking a child implications? No, it’s just wrong.)

The design is largely an exaggeration of Apple’s understated aesthetic, except for the insane, giant floral eyeball that is the AI therapist. I love how much it reads like a weirdcore titan and the characters are nonplussed, telegraphing how much the citizens of this world have normalized to inhumanity. I have to give a major ding to the iPad interface by which parents take care of their fetuses, as its art direction is a mismatch to everything else in the film and seems quite rudimentary, like a Flash app circa 1998.

Before I get to the best interfaces of the year, let’s take a moment to appreciate two trends I saw emerging in 2023. That of hyperminimalist interfaces and of interface-related comedy.

Hyperminimalist interfaces

This year I noticed that many movies are telling stories with very minimal interfaces. As in, you can barely call them designed since they’re so very minimalist. This feels like a deliberate contrast to the overwhelming spectacle that permeates, say, the MCU. They certainly reduce the thing down to just the cause and effect that are important to the story. Following are some examples that illustrate this hyperminimalism.

This could be a cost-saving tactic, but per the default New Criticism stance of this blog, we’ll take it as a design choice and note it’s trending.

Shout-out: Interface Comedy

I want to give a special shout-out to interface-related comedy over the past year.

Smoking Causes Coughing

The first comes from the French gonzo horror sci-fi Smoking Causes Coughing. In a nested story told by a barracuda that is on a grill being cooked, Tony is the harried manager of a log-processing plant whose day is ruined by her nephew’s somehow becoming stuck in an industrial wood shredder. Over the scene she attempts to reverse the motor, failing each time, partly owing to the unlabeled interface and bad documentation. It’s admittedly not sci-fi, just in a sci-fi film, and a very gory, very hilarious bit of interface humor in an schizoid film.

Guardians of the Galaxy 3

The second is Guardians of the Galaxy 3. About a fifth of the way into the movie, the team spacewalks from the Milano to the surface of Orgocorp to infiltrate it. Once on the surface, Peter, who still pines for alternate-timeline Gamora, tries to strike up a private conversation with her. The suits have a forearm interface featuring a single row of colored stay-state buttons that roughly match the colors of the spacesuits they’re wearing. Quill presses the blue one and tries in vain to rekindle the spark between him and Gamora in a private conversation. But then a minute into the conversation, Mantis cuts in…

  • Mantis
  • Peter you know this is an open line, right?
  • Peter
  • What?
  • Mantis
  • We’re listening to everything you’re saying.
  • Drax
  • And it is painful.
  • Quill
  • And you’re just telling me now‽
  • Nebula
  • We were hoping it would stop on its own.
  • Peter
  • But I switched it over to private!
  • Mantis
  • What color button did you push?
  • Peter
  • Blue! For the blue suit!
  • Drax
  • Oh no.
  • Nebula
  • Blue is the open line for everyone.
  • Mantis
  • Orange is for blue.
  • Peter
  • What‽
  • Mantis
  • Black is for orange. Yellow is for green. Green is for red. And red is for yellow.
  • Drax
  • No, yellow is for yellow. Green is for red. Red is for green.
  • Mantis
  • I don’t think so.
  • Drax
  • Try it then.
  • Mantis (screaming)
  • HELLO!
  • Peter writhes in pain
  • Mantis
  • You were right.
  • Peter
  • How the hell and I supposed to know all of that?
  • Drax
  • Seems intuitive.

The Marvels

A third comedy bit happens in The Marvels, when Kamala Khan is nerding out over Monica Rambeau’s translucent S.H.I.E.L.D. tablet. She says…

  • Khan
  • Is this the new iPad? I haven’t seen it yet.
  • Rambeau
  • I wish.
  • Khan
  • Wait, if this is all top secret information, why is it on a clear case?

Rambeau has no answer, but there are, in fact, some answers.

Anyway, I want to give a shout-out to the writers for demonstrating with these comedy bits some self-awareness and good-natured self-owning of tropes. I see you and appreciate you. You are so valid.

Best Interfaces of 2023

But my favorite interfaces of 2023 come from Spider-Man: Across the Spider-Verse. The interfaces throughout are highly stylized (so might be tough to perform the detailed analysis, which is this site’s bread-and-butter) but play the plot points perfectly.

In Across the Spider-Verse, while dealing difficulties with his home life and chasing down a new supervillain called The Spot, Miles Morales learns about The Society. The Society is a group of (thousands? Tens of thousands? of) Spider-people of every stripe and sort from across the Multiverse, whose overriding mission is to protect “canon” events in each universe that, no matter how painful, they believe are necessary to keep the fabric of reality from unraveling. It’s full of awesome interfaces.

Lyla is the general artificial intelligence that has a persistent volumetric avatar. She’s sassy and disagreeable and stylish and never runs, just teleports.

The wrist interfaces—called the Multiversal Gizmo—worn by members of The Society all present highly-contextual information with most-likely actions presented as buttons, and, as needed, volumetric alerts. Also note that Miguel’s Gizmo is longer, signaling his higher status within The Society.

Of special note is volumetric display that Spider Gwen uses to reconstruct the events at the Alchemax laboratory. The interface is so smart, telegraphs its complex functioning quickly and effectively, and describes a use that builds on conceivable but far-future applications of inference. The little dial that pops up allowing her to control time of the playback reminds me of Eye of Agamatto (though sadly I didn’t see evidence of the important speculative time-control details I’d provided in that analysis). The in-situ volumetric reconstruction reminds me of some of the speculative interfaces I’d proposed in the review of Deckard’s photo inspector from Blade Runner, and so was a big thrill to see.

All of the interfaces have style, are believable for the diegesis, and contribute to the narrative with efficiency. Congratulations to the team crafting these interfaces, and if you haven’t seen it yet, what are you waiting for? Go see it. It’s in a lot of places and the interfaces are awesome. (For full disclosure, I get no kickback from these referral links.)

Hackers (1995)

Our third film is from 1995, directed by Iain Softley.

Hackers is about a group of teenage computer hackers, of the ethical / playful type who are driven by curiosity and cause no harm — well, not to anyone who doesn’t deserve it. One of these hackers breaks into the “Gibson” computer system of a high profile company and partially downloads what he thinks is an unimportant file as proof of his success. However this file is actually a disguised worm program, created by the company’s own chief of computer security to defraud the company of millions of dollars. The security chief tries to frame the hackers for various computer crimes to cover his tracks, so the hackers must break back into the system to download the full worm program and reveal the true culprit.

The film was made in the time before Facebook when it was common to have an online identity, or at least an online handle (nick), distinct from the real world. Our teenage hacker protagonists are:

  • Crash Override, real name Dade.
  • Acid Burn, real name Kate.
  • Cereal Killer, Lord Nikon, and Phantom Phreak, real names not given.
  • Joey, the most junior, who doesn’t have a handle yet.

As hackers they don’t have a corporate budget, so use a variety of personal computers rather than the expensive SGI workstations we saw in the previous films. And since it’s the 1990s, their network connections are made with modems over the analog phone system and important files will fit on 1.44 megabyte floppy disks. 

The Gibson, though, is described as “big iron”, a corporate supercomputer. Again this was the 1990s when a supercomputer would be a single very big and very expensive computer, not thousands of PC CPUs and GPUs jammed into racks as in the early 21st C. A befitting such an advanced piece of technology it has a three dimensional file browsing interface which is on display both times the Gibson is hacked.

First run

The first hack starts at about 24 minutes. Junior hacker Joey  has been challenged by his friends to break into something important such as a Gibson. The scene starts with Joey sitting in front of his Macintosh personal computer and reviewing a list of what appear to be logon or network names and phone numbers. The camera flies through a stylised cyberspace representation of the computer network, the city streets, then the physical rooms of the target company (which we will learn is Ellingson Minerals), and finally past a computer operator sitting at a desk in the server room and into the 3D file system. This single “shot” actually switches a few times between the digital and real worlds, a stylistic choice repeated throughout the film. Although never named in the film this file system is the “City of Text” according to the closing credits.

Joey looks down on the City of Text. Hackers (1995)

The file system is represented as a virtual cityscape of skyscraper-like blocks. The ground plane looks like a printed circuit board with purple traces (lines). The towers are simple box shapes, all the same size, as if constructed from blue tinted glass or acrylic plastic. Each of the four sides and the top shows a column of text in white lettering, apparently the names of directories or files. Because the tower sides are transparent the reverse facing text on the far sides is also visible, cluttering the display.

This 3D file system is the most dynamic of those in this review. Joey flies among the towers rather than walking, with exaggerated banking and tilting as he turns and dives. At ground level we can see some simple line graphics at the left as well as text.

Joey flies through the City of Text, banking as he changes direction. Hackers (1995)

The city of text is even busier due to animation effects. Highlight bars move up and down the text lists on some panes. Occasionally a list is cleared and redrawn top to bottom, while others cycle between two sets of text. White pulses flow along the purple ground lanes and fly between the towers. These animations do not seem to be interface elements. They could be an indicator of overall activity with more pulses per second meaning more data being accessed, like the blinking LED on your Ethernet port or disk drive. Or they could be a screensaver, as it was important on the CRT displays of the 1990s to not display a static image for long periods as it would “burn in” and become permanent.

Next there is a very important camera move, at least for analysing the user interface. So far the presentation has been fullscreen and obviously artificial. Now the camera pulls back slightly to show that this City of Text is what Joey is seeing on the screen of his Macintosh computer. Other shots later in the film will make it clear that this is truly interactive, he is the one controlling the viewpoint.

Joey looks at a particular list of directories/files on one face of a skyscraper. Hackers (1995)

I’ll discuss how this might work later in the analysis section. For now it’s enough to remember that this is a true file browser, the 3D equivalent of the Macintosh Finder or Windows File Explorer.

While Joey is exploring, we cut to the company server room. This unusual activity has triggered an alarm so the computer operator telephones the company security chief at home. At this stage we don’t know that he’s evil, but he does demand to be addressed by his hacker handle “The Plague” which doesn’t inspire confidence. (The alarm itself shows that a superuser / root / administrator account is in use by displaying the password for everyone to see on a giant screen. But we’re not going to talk about that.) 

Joey wants to prove he has hacked the Gibson by downloading a file, but by the ethics of the group it shouldn’t be something valuable. He selects what he thinks will be harmless, the garbage or trash directory on a particular tower. It’s not very clear but there is another column of text to the right which is dimmed out.

Joey selects the GARBAGE directory and a list of contents appears. Hackers (1995)

There’s a triangle to the right of the GARBAGE label indicating that it is a directory, and when selected a second column of text shows the files within it. When Joey selects one of these the system displays what today would be called a Live Tile in Windows, or File Preview in the Mac Finder. But in this advanced system it’s an elaborate animation of graphics and mathematical notation.

Joey decides this is the file he wants and starts a download. Since he’s dialled in through an old analog phone modem, this is a slow process and will eventually be interrupted when Joey’s mother switches his Macintosh off to force him to get some sleep.

Joey looks at the animation representing the file he has chosen. Hackers (1995)

Physical View

Back in the server room of Ellingson Minerals and while Joey is still searching, the security chief AKA “The Plague” arrives. And here we clearly see that there is also a physical 3D representation of the file system.

The Plague makes a dramatic entrance into the physical City of Text. Hackers (1995)

Just like the virtual display it is made up of rectangular towers made of blue tinted glass or plastic, arranged on a grid pattern like city skyscrapers. Each is about 3 metres high and about 50cm wide and deep. Again matching the virtual display, there is white text on all the visible sides, being updated and highlighted. However there is one noticeable difference, the bottom of each tower is solid black.

What are the towers for? Hackers is from 1995, when hard drives and networked file servers were shoebox- to pizza-box-sized, so one or two would fit into the base of each tower. The physical displays could be just blinkenlights, an impressive but not particularly useful visual display, but in a later shot there’s a technician in the background looking at one of the towers and making notes on a pad, so they are intended to show something useful. My assumption is that each tower displays information about the actual files being stored inside, mirroring the virtual city of text shown online.

When he reaches the operator’s desk, The Plague switches the big wall display to the same 3D virtual file system.

The Plague on the left and the night shift operator watch what Joey is doing on a giant wall screen. Hackers (1995)

He uses an “echo terminal” command to see exactly what Joey is doing, so sees the same garbage directory and that the file is being copied. We’ll later learn that this seemingly harmless file is actually the worm program created by The Plague, and that discovering it had been copied was a severe shock. Here he arranges for the phone connection to be traced and Joey questioned by his government friends in the US Secret Service (which at the time was responsible for investigating some computer security incidents and crimes), setting in motion the main plot elements.

Tagged: animated, architecture, big screens, busted!, control room, cyan, doorway, drama, eavesdropping, emergency, flashing, flying, glow, hacking, industrial espionage, labeling, monitoring, navigating, orange, purple, security, surveillance, terminal, translucency, translucent display, wall interface

Second run

After various twists and turns our teenage hackers are resolved to hack into the Gibson again to obtain a full copy of the worm program which will prove their innocence. But they also know that The Plague knows they know about the worm, Ellingson Minerals is alerted, and the US Secret Service are watching them. This second hacking run starts at about 1 hour 20 minutes.

The first step is to evade the secret service agents by a combination of rollerblading and hacking the traffic lights. (Scenes like this are why I enjoy the film so much.) Four of our laptop-wielding hackers dial in through public phone booths. The plan is that Crash will look for the file while Acid, Nikon, and Joey will distract the security systems, and they are expecting additional hacker help from around the world.

We see a repeat of the earlier shot flying through the streets and building into the City of Text, although this time on Crash’s Macintosh Powerbook.

Crash enters the City of Text. Hackers (1995)

It seems busier with many more pulses travelling back and forth between towers, presumably because this is during a workday.

The other three start launching malware attacks on the Gibson. Since the hacking attempt has been anticipated, The Plague is in the building and arrives almost immediately.

The Plague walks through the physical City of Text as the attack begins. Hackers (1995)

The physical tower display now shows a couple of blocks with red sides. This could indicate the presence of malware, or just that those sections of the file system are imposing a heavy CPU or IO load due to the malware attacks.

This time The Plague is assisted by a full team of technicians. He primarily uses a “System Command Shell” within a larger display that presumably shows processor and memory usage. It’s not the file system, but has a similar design style and is too cool not to show:

The Plague views system operations on a giant screen, components under attack highlighted in red on the right. Hackers (1995)

Most of the shots show the malware effects and The Plague responding, but Crash is searching for the worm. His City of Text towers show various “garbage” directories highlighted in purple, one after the other.

Crash checks the first garbage directory, in purple. Other possible matches in cyan on towers to the right. Hackers (1995)

What’s happening here? Most likely Crash has typed in a search wildcard string and the file browser is showing the matching files and folders.

Why are there multiple garbage directories? Our desktop GUIs always show a single trashcan, but under the hood there is more than one. A multiuser system needs at least one per user, because otherwise files deleted by Very Important People working with Very Sensitive Information would be visible, or at least the file names visible, to everyone else. Portable storage devices, floppy disks in Hackers and USB drives today, need their own trashcan because the user might still expect to be able to undelete files even if it has been moved to another computer. For the same reason a networked drive needs its own trashcan that isn’t stored on the connecting computer. So Crash really does have to search for the right garbage directory in this giant system.

As hackers from around the world join in, the malware effects intensify. More tower faces, both physical and digital, are red. The entire color palette of the City of Text becomes darker.

Crash flies through the City of Text, a skyscraper under siege. Hackers (1995)

This could be an automatic effect when the Gibson system performance drops below some threshold, or activated by the security team as the digital equivalent of traffic cones around a door. Anyone familiar with the normal appearance of the City of Text can see at a glance that something is wrong and, presumably, that they should log out or at least not try to do anything important.

Crash finds the right file and starts downloading, but The Plague hasn’t been fully distracted and uses his System Command Shell to disconnect Crash’s laptop entirely. Rather than log back in, Crash tells Joey to download the worm and gives him the full path to the correct garbage directory, which for the curious is root/.workspace/.garbage (the periods are significant, meaning these names should not normally be displayed to non-technical users).

We don’t see how Joey enters this into the file browser but there is no reason it should be difficult. Macintosh Finder windows have a clickable text search box, and both the Ubuntu Desktop Shell and Microsoft Windows start screen will automatically start searching for files and folders that match any typed text.

Joey downloads the worm, this time all of it. The combined malware attacks crash The Gibson. Unfortunately the secret service agents arrive just in time to arrest them, but all ends well with The Plague being exposed and arrested and our hacker protagonists released.

Tagged: 3D rendering, animation, architecture, big screens, blue, bright is more, call to action, color cue, command and control, control room, crisis, cyan, dark, defense, flashing, flowing, flying, glow, hacking, industrial espionage, keyboard, mission, motion cue, navigating, nerdsourcing, personal computer, red, red is warning, search, search, status indicator, threshold alert, translucency, translucent display, trap, trash, wall mounted, yellow

Analysis

How believable is the interface?

The City of Text has two key differences from the other 3D file browsers we’ve seen so far.

  1.  It must operate over a network connection, specifically over a phone modem connection, which in the 1990s would be much slower than any Ethernet LAN.
  2. This 3D view is being rendered on personal computers, not specialised 3D workstations. 

Despite these constraints, the City of Text remains reasonably plausible.

Would the City of Text require more bandwidth than was available? What effect would we expect from a slow network connection? It’s a problem when copying files, upload or download, but much less so for browsing a file system. The information being passed from the Gibson to the 3D file browser is just a list of names in each directory and a minimal set of attributes for each, not the file contents. In 1995 2D file browsers on personal computers were already showing icons, small raster images, for each file which took up more memory than the file names. The City of Text doesn’t, so the file data would certainly fit in the bandwidth available.

The flying viewpoint doesn’t require much bandwidth either. There is no avatar or other representation of the user, just an abstract viewpoint. Only 9 numbers are needed to describe where you are and what you’re looking at in 3D space, and predictive techniques developed for games and simulations can reduce the network bandwidth required even more.

Networked file systems and file browsers already existed in 1995, for example FTP and Gopher, although with pure text interfaces rather than 3D or even 2D graphics. The only missing component would be the 3D viewpoint coordinates.

PCs in the 1990s, especially laptops, rarely had any kind of 3D graphics acceleration and would not have been able to run the Jurassic Park or Disclosure 3D file browsers. The City of Text, though, is much less technically demanding even though it displays many more file and folder names.

Notice that there is no hidden surface removal, where the front sides of a 3D object hide those that are further away. There’s no lighting, with everything rendered in flat colors that don’t depend on the direction of the sun or other light sources, and no shadows. There are no images or textures, just straight lines and plain text. And finally everything is laid out on an axis-aligned grid; meaning all the graphics are straight up/down, left/right, or forwards/back; and all the towers and text are the same size. Similar shortcuts were used in 1990s PC games and demo scene animations, such as the original Doom in which players could look from side to side but not up or down.

I’m not saying that the City of Text on a 1990s PC or laptop would be easy, especially on Joey’s Macintosh LC, but it is plausible.

Alas the worm animation shown when that particular file is selected is not possible. We see fractal graphics and mathematical notation in 3D, and it’s a full screen image rather than a simple file icon. Whether it’s a pre-rendered animation or being generated on the fly there’s way too much to push through a modem connection, even though at the time “full screen” meant a lot less pixels than now in the 21st C.

The physical towers were also not possible. Three metre high flat screen displays didn’t exist in 1995, and I don’t see how that many projectors could be installed in the ceiling without interfering with each other.

How well does the interface inform the narrative of the story?

Hackers is a film all about computers and the people who work with them, and therefore must solve the problem (which still exists today) of making what is happening visible and understandable to a non-technical audience. Director Iain Softley said he wanted a metaphorical representation of how the characters perceived the digital world, not a realistic one. Some scenes use stylised 2D graphics and compositing to create a psychedelic look, while the 3D file browser is meant to be a virtual equivalent to the physical city of New York where Hackers is set. At least for some viewers, myself included, it works.

The worm animation also works well. Joey is looking for an interesting file, a trophy, and the animation makes it clear that this is indeed an extraordinary file without needing to show the code.

The physical towers, though, are rather silly. The City of Text is meant to be metaphorical, a mental landscape created by hackers, so we don’t need a physical version.

How well does the interface equip the character to achieve their goals?

The City of Text is very well suited to the character goals, because they are exploring the digital world. Looking cool and having fun are what’s important, not being efficient.

Now if you’ll excuse me, I have a rollerblading lesson before the next review…

The Cookie: Matt’s controls

When using the Cookie to train the AI, Matt has a portable translucent touchscreen by which he controls some of virtual Greta’s environment. (Sharp-eyed viewers of the show will note this translucent panel is the same one he uses at home in his revolting virtual wingman hobby, but the interface is completely different.)

Black_Mirror_Cookie_18.png

The left side of the screen shows a hamburger menu, the Set Time control, a head, some gears, a star, and a bulleted list. (They’re unlabeled.) The main part of the screen is a scrolling stack of controls including Simulated Body, Control System, and Time Adjustment. Each has an large icon, a header with “Full screen” to the right, a subheader, and a time indicator. This could be redesigned to be much more compact and context-rich for expert users like Matt. It’s seen for maybe half a second, though, and it’s not the new, interesting thing, so we’ll skip it.

The right side of the screen has a stack of Smartelligence logos which are alternately used for confirmation and to put the interface to sleep.

Mute

When virtual Greta first freaks out about her circumstance and begins to scream in existential terror, Matt reaches to the panel and mutes her. (To put a fine point on it: He’s a charming monster.) In this mode she cannot make a sound, but can hear him just fine. We do not see the interface he uses to enact this. He uses it to assert conversational control over her. Later he reaches out to the same interface to unmute her.

The control he touches is the one on his panel with a head and some gears reversed out of it. The icon doesn’t make sense for that. The animation showing the unmuting shows it flipping from right to left, so does provide a bit of feedback for Matt, but it should be a more fitting icon and be labeled.

Cookie_mute
Also it’s teeny tiny, but note that the animation starts before he touches it. Is it anticipatory?

It’s not clear though, while she is muted, how he knows that she is trying to speak. Recall that she (and we) see her mouthing words silently, but from his perspective, she’s just an egg with a blue eye. The system would need some very obvious MUTE status display, that increases in intensity when the AI is trying to communicate. Depending on how smart the monitoring feature was, it could even enable some high-intensity alert system for her when she needs to communicate something vital. Cinegenically, this could have been a simple blinking of the blue camera light, though this is currently used to indicate the passage of time during the Time Adjustment (see below.)

Simulated Body

Matt can turn on a Simulated Body for her. This allows the AI to perceive herself as if she had her source’s body. In this mode she perceives herself as existing inside a room with large, wall-sized displays and a control console (more on this below), but is otherwise a featureless white.

Black_Mirror_Cookie_White_Room.png

I presume the Simulated Body is a transitional model—part of a literal desktop metaphor—meant to make it easy for the AI (and the audience) to understand things. But it would introduce a slight lag as the AI imagines reaching and manipulating the console. Presuming she can build competence in directly controlling the technologies in the house, the interface should “scaffold” away and help her gain the more efficient skills of direct control, letting go of the outmoded notion of having a body. (This, it should be noted, would not be as cinegenic since the story would just feature the egg rather than the actor’s expressive face.)

Neuropsychology nerds may be interested to know that the mind’s camera does, in fact, have spatial lags. Several experiments have been run where subjects are asked to imagine animals as seen from the side and then timed how long it took them to imagine zooming into the eye. It takes longer, usually, for us to imagine the zoom to a elephant’s eye than a mouse’s because the “distance” is farther. Even though there’s no physicality to the mind’s camera to impose this limit, our brain is tied to its experience in the real world.

Black_Mirror_Cookie_Simulated_Body.png

The interface Matt has to turn on her virtual reality is confusing. We hear 7 beeps while the camera is on his face. He sees a 3D rendering of a woman’s body in profile and silhouette. He taps the front view and it fills with red. Then he taps the side view and it fills with red. Then he taps some Smartelligence logos on the side with a thumb and then *poof* she’s got a body. While I suspect this is a post-actor interface, (i.e. Jon Hamm just tapped some things on an empty screen while on camera and then the designers had to later retrofit an interface that fit his gestures) this multi-button setup and three-tap initialization just makes no sense. It should be a simple toggle with access to optional controls like scaffolding settings (discussed above.)

Time “Adjustment”

The main tool Matt has to force compliance is a time control. When Greta initially says she won’t comply, (specifically and delightfully, she asserts, “I’m not some sort of push-button toaster monkey!”) Then he uses his interface to make it seem like 3 weeks pass for her inside her featureless white room. Then again for 6 months. The solitary confinement makes her crazy and eventually forces compliance.

Cookie_settime.gif

The interface to set the time is a two-layer virtual dial: Two chapter rings with wide blue arcs for touch targets. The first time we see him use it, he spins the outer one about 360° (before the camera cuts away) to set the time for three weeks. While he does it, the inner ring spins around the same center but at a slower rate. I presume it’s months, though the spatial relationship doesn’t make sense. Then he presses the button in the center of the control. He sees an animation of a sun and moon arcing over an illustrated house to indicate her passage of time, and then the display. Aside: Hamm plays this beat marvelously by callously chomping on the toast she has just help make.

Toast.gif

Improvements?

Ordinarily I wouldn’t speak to improvements on an interface that is used for torture, but as this could only affect a general AI that is as yet speculative, and it couldn’t be co-opted to torture real people since time travel doesn’t exist, so I think this time it’s OK. Discussing it as a general time-setting control, I can see three immediate improvements.

1. Use fast forward models

It makes most sense for her time sentence to end automatically and automatically return to real-world speed. But each time we see the time controls used, the following interaction happens near the end of the time sentence:

  • Matt reaches up to the console
  • He taps the center button of the time dial
  • He taps the stylized house illustration. In response it gets a dark overlay with a circle inside of it reading “SET TIME.” This is the same icon seen 2nd down  in the left panel.
  • He taps the center button of the time dial again. The dark overlay reads “Reset” with a new icon.
  • He taps the overlay.

Please tell me this is more post-actor interface design. Because that interaction is bonkers.

Cookie_stop.gif

If the stop function really needs a manual control, well, we have models for that that are very readily understandable by users and audiences. Have the whole thing work and look like a fast forward control rather than this confusing mess. If he does need to end it early, as he does in the 6 months sentence, let him just press a control labeled PLAY or REALTIME.

2. Add calendar controls

A dial makes sense when a user is setting minutes or hours, but a calendar-like display should be used for weeks or months. It would be immediately recognizable and usable by the user and understandable to the audience. If Hamm had touched the interface twice, I would design the first tap to set the start date and the second tap to set the end date. The third is the commit.

3. Add microinteraction feedback

Also note that as he spins the dials, he sees no feedback showing the current time setting. At 370° is it 21 or 28 days? The interface doesn’t tell him. If he’s really having to push the AI to its limits, the precision will be important. Better would be to show the time value he’s set so he could tweak it as needed, and then let that count down as time remaining while the animation progresses.

Cookie_settime.gif

Effectiveness subtlety: Why not just make the solitary confinement pass instantly for Matt? Well, recall he is trying to ride a line of torture without having the AI wig out, so he should have some feedback as to the duration of what he’s putting her through. If it was always instant, he couldn’t tell the difference between three weeks and three millennia, if he had accidentally entered the wrong value. But if real-world time is passing, and it’s taking longer than he thinks it should be, he can intervene and stop the fast-forwarding.

That, or of course, show feedback while he’s dialing.

Near the end of the episode we learn that a police officer is whimsically torturing another Cookie, and sets the time-ratio to “1000 years per minute” and then just lets it run while he leaves for Christmas break. The current time ratio should also be displayed and a control provided. It is absent from the screen.

Black_Mirror_Cookie_31.png

Add psychological state feedback

There is one “improvement” that does not pertain to real world time controls, and that’s the invisible effect of what’s happening to the AI during the fast forward. In the episode Matt explains that, like any good torturer, “The trick of it is to break them without letting them snap completely,” but while time is passing he has no indicators as to the mental state of the sentience within. Has she gone mad? (Or “wigged out” as he says.) Does he need to ease off? Give her a break?

I would add trendline indicators or sparklines showing things like:

  • Stress
  • Agitation
  • Valence of speech

I would have these trendlines highlight when any of the variables are getting close to known psychological limits. Then as time passes, he can watch the trends to know if he’s pushing things too far and ease off.

Ministry of Art detector gate

ChildrenofMen-Artgate-01

Jumping back in the film a bit, we’re going to visit the Ministry of Art. When Theo goes there to visit his brother, after the car pulls to the front of the secured building, Theo steps out and walks toward a metal-detector gate.

Its quite high, about 3 meters tall. The height helps to reinforce the notion that this is a public space.

  1. This principle, that short ceilings are personal, and high ceilings are public, is I believe a well-established one in architectural design. Read the Alexandrian pattern if you’d like to read more about it.
  2. Is it a public space? It is, since it’s a Ministry. But it isn’t, since he joins his brother in what looks like a rich person’s private dining room. I was always a bit confused by what this place was meant to be. Perhaps owning to The Dark Times, Nigel has cited Minister rights and cordoned off part of the Tate Modern to live in. If anyone can explain this, please speak up.
  3. On the downside, the height makes the text more out of sight and harder to read by the people meant to be reading it.

The distance is balanced by the motion graphics of the translucent sign atop the gate. Animated red graphics point the direction of ingress, show a security stripe pattern, and provide text instructions.

Motion is a very strong attention-getting signal, and combined with the red colors, does all the attention-getting that the height risks. But even that’s not a critical issue, as there is of course a guard standing by to ensure his understanding and compliance.

ChildrenofMen-Artgate-05

Note that there is no interaction here (which is the usual filter for this blog), but since I’m publishing an interview with the designer of this and the Kubris interface soon, I thought I’d give it a quick nod.

Stark Tower monitoring

Since Tony disconnected the power transmission lines, Pepper has been monitoring Stark Tower in its new, off-the-power-grid state. To do this she studies a volumetric dashboard display that floats above glowing shelves on a desktop.

Avengers-Reactor-output03

Volumetric elements

The display features some volumetric elements, all rendered as wireframes in the familiar Pepper’s Ghost (I know, I know) visual style: translucent, edge-lit planes. A large component to her right shows Stark Tower, with red lines highlighting the power traveling from the large arc reactor in the basement through the core of the building.

The center of the screen has a similarly-rendered close up of the arc reactor. A cutaway shows a pulsing ring of red-tinged energy flowing through its main torus.

This component makes a good deal of sense, showing her the physical thing she’s meant to be monitoring but not in a photographic way, but a way that helps her quickly locate any problems in space. The torus cutaway is a little strange, since if she’s meant to be monitoring it, she should monitor the whole thing, not just a quarter of it that has been cut away.

Flat elements

The remaining elements in the display appear on a flat plane. Continue reading

Carrier Control

The second instantiation of videochat with the World Security Council that we see is  when Fury receives their order to bomb the site of the Chitauri portal. (Here’s the first.) He takes this call on the bridge, and rather than a custom hardware setup, this is a series of windows that overlay an ominous-red map of the world in an app called CARRIER CONTROL. These windows represent a built-in chat feature for discussing this very topic. There is some fuigetry on the periphery, but our focus is on these windows and the conversation happening through them.

Avengers-fury-secure-transmission01

In this version of the chat, we are assured that it is a SECURE TRANSMISSION by a legend across the top of each, but there is not the same level of assurance as in the videoconference room. If it’s still HOTP, Fury isn’t notified of it. There’s a tiny 01_AZ in the upper right of every screen, but it never changes and is the same for each participant. (An homage to Arizona? Lighter Andrew Zink? Cameraman Arthur Zajac?) Though this is a more desperate situation, you imagine that the need for security is no less dire. Having that same cypher key would be comforting if it is in fact a policy.

Different sizes of windows in the app seem to indicate a hierarchy, since the largest window is the fellow who does most of the talking in both conferences, and it does not change as others speak. Such an automated layout would spare Fury the hassle of having to manage multiple windows, though visually these look more like individual objects he’s meant to manipulate. Poor affordances.

dismiss

The only control we see is when Fury dismisses them, and to do this he just taps at the middle of the screen. The teleconference window is “push wiped” by a satellite view of New York City. Fine, he feels like punching them. But…

a) How does he actually select something in that interface without a tap?

b) A swipe would have been more meaningful, and in line with the gestural pidgin I identified in the gestural chapter of the book.

And of course, if this was the real world, you’d hope for better affordances for what can be done on this window across the board.

So though mostly effective, narratively, could use some polish.

The HoverChair Social Network

WallE-SocialNetwork03

The other major benefit to the users of the chair (besides the ease of travel and lifestyle) is the total integration of the occupant’s virtual social life, personal life, fashion (or lack-thereof), and basic needs in one device. Passengers are seen talking with friends remotely, not-so-remotely, playing games, getting updated on news, and receiving basic status updates. The device also serves as a source of advertising (try blue! it’s the new red!).

A slight digression: What are the ads there for? Considering that the Axiom appears to be an all-inclusive permanent resort model, the ads could be an attempt to steer passengers to using resources that the ship knows it has a lot of. This would allow a reprieve for heavily used activities/supplies to be replenished for the next wave of guests, instead of an upsell maneuver to draw more money from them. We see no evidence of exchange of money or other economic activity while on-board the Axiom

OK, back to the social network.

Security?

It isn’t obvious what the form of authentication is for the chairs. We know that the chairs have information about who the passenger prefers to talk to, what they like to eat, where they like to be aboard the ship, and what their hobbies are. With that much information, if there was no constant authentication, an unscrupulous passenger could easily hop in another person’s chair, “impersonate” them on their social network, and play havoc with their network. That’s not right.

It’s possible that the chair only works for the person using it, or only accesses the current passenger’s information from a central computer in the Axiom, but it’s never shown. What we do know is that the chair activates when a person is sitting on it and paying attention to the display, and that it deactivates as soon as that display is cut or the passenger leaves the chair.

We aren’t shown what happens when the passenger’s attention is drawn away from the screen, since they are constantly focused on it while the chair is functioning properly.

If it doesn’t already exist, the hologram should have an easy to push button or gesture that can dismiss the picture. This would allow the passenger to quickly interact with the environment when needed, then switch back to the social network afterwards.

And, for added security in case it doesn’t already exist, biometrics would be easy for the Axiom. Tracking the chair user’s voice, near-field chip, fingerprint on the control arm, or retina scan would provide strong security for what is a very personal activity and device. This system should also have strong protection on the back end to prevent personal information from getting out through the Axiom itself.

Social networks hold a lot of very personal information, and the network should have protections against the wrong person manipulating that data. Strong authentication can prevent both identity theft and social humiliation.

Taking the occupant’s complete attention

While the total immersion of social network and advertising seems dystopian to us (and that’s without mentioning the creepy way the chair removes a passenger’s need for most physical activity), the chair looks genuinely pleasing to its users.

They enjoy it.

But like a drug, their enjoyment comes at the detriment of almost everything else in their lives. There seem to be plenty of outlets on the ship for active people to participate in their favorite activities: Tennis courts, golf tees, pools, and large expanses for running or biking are available but unused by the passengers of the Axiom.

Work with the human need

In an ideal world a citizen is happy, has a mixture of leisure activities, and produces something of benefit to the civilization. In the case of this social network, the design has ignored every aspect of a person’s life except moment-to-moment happiness.

This has parallels in goal driven design, where distinct goals (BNL wants to keep people occupied on the ship, keep them focused on the network, and collect as much information as possible about what everyone is doing) direct the design of an interface. When goal-driven means data driven, then the data being collected instantly becomes the determining factor of whether a design will succeed or fail. The right data goals means the right design. Wrong data goals mean the wrong design.

Instead of just occupying a person’s attention, this interface could have instead been used to draw people out and introduce them to new activities at intervals driven by user testing and data. The Axiom has the information and power, perhaps even the responsibility, to direct people to activities that they might find interesting. Even though the person wouldn’t be looking at the screen constantly, it would still be a continuous element of their day. The social network could have been their assistant instead of their jailer.

One of the characters even exclaims that she “didn’t even know they had a pool!”. Indicating that she would have loved to try it, but the closed nature of the chair’s social network kept her from learning about it and enjoying it. By directing people to ‘test’ new experiences aboard the Axiom and releasing them from its grip occasionally, the social network could have acted as an assistant instead of an attention sink.

WallE-SocialNetwork05

Moment-to-moment happiness might have declined, but overall happiness would have gone way up.

The best way for designers to affect the outcome of these situations is to help shape the business goals and metrics of a project. In a situation like this, after the project had launched a designer could step in and point out those moments were a passenger was pleasantly surprised, or clearly in need of something to do, and help build a business case around serving those needs.

The obvious moments of happiness (that this system solves for so well) could then be augmented by serendipitous moments of pleasure and reward-driven workouts.

We must build products for more than just fleeting pleasure

WallE-SocialNetwork09

As soon as the Axiom lands back on Earth, the entire passenger complement leaves the ship (and the social network) behind.

It was such a superficial pleasure that people abandoned it without hesitation when they realized that there was something more rewarding to do. That’s a parallel that we can draw to many current products. The product can keep attention for now, but something better will come along and then their users will abandon them.

WallE-SocialNetwork07

A company can produce a product or piece of software that fills a quick need and initially looks successful. But, that success falls apart as soon as people realize that they have larger and tougher problems that need solving.

Ideally, a team of designers at BNL would have watched after the initial launch and continued improving the social network. By helping people continue to grow and learn new skills, the social network could have kept the people aboard the Axiom it top condition both mentally and physically. By the time Wall-E came around, and life finally began to return to Earth, the passengers would have been ready to return and rebuild civilization on their own.

To the designers of a real Axiom Social Network: You have the chance to build a tool that can save the world.

We know you like blue! Now it looks great in Red!

The Hover Chair

WallE-HoverChair05

The Hover Chair is a ubiquitous, utilitarian, all-purpose assisting device. Each passenger aboard the Axiom has one. It is a mix of a beach-side deck chair, fashion accessory, and central connective device for the passenger’s social life. It hovers about knee height above the deck, providing a low surface to climb into, and a stable platform for travel, which the chair does a lot of.

A Universal Wheelchair

We see that these chairs are used by everyone by the time that Wall-E arrives on the Axiom. From BNL’s advertising though, this does not appear to be the original. One of the billboards on Earth advertising the Axiom-class ships shows an elderly family member using the chair, allowing them to interact with the rest of the family on the ship without issue. In other scenes, the chairs are used by a small number of people relaxing around other more active passengers.

At some point between the initial advertising campaign and the current day, use went from the elderly and physically challenged, to a device used 24/7 by all humans on-board the Axiom. This extends all the way down to the youngest children seen in the nursery, though they are given modified versions to more suited to their age and disposition. BNL shows here that their technology is excellent at providing comfort as an easy choice, but that it is extremely difficult to undo that choice and regain personal control.

But not a perfect interaction

Continue reading

Otto’s Manual Control

WallE-Otto02

WallE-Otto06

When it refused to give up authority, the Captain wrested control of the Axiom from the artificial intelligence autopilot, Otto. Otto’s body is the helm wheel of the ship and fights back against the Captain. Otto wants to fulfil BNL’s orders to keep the ship in space. As they fight, the Captain dislodges a cover panel for Otto’s off-switch. When the captain sees the switch, he immediately realizes that he can regain control of the ship by deactivating Otto. After fighting his way to the switch and flipping it, Otto deactivates and reverts to a manual control interface for the ship.

The panel of buttons showing Otto’s current status next to the on/off switch deactivates half its lights when the Captain switches over to manual. The dimmed icons are indicating which systems are now offline. Effortlessly, the captain then returns the ship to its proper flight path with a quick turn of the controls.

One interesting note is the similarity between Otto’s stalk control keypad, and the keypad on the Eve Pod. Both have the circular button in the middle, with blue buttons in a semi-radial pattern around it. Given the Eve Pod’s interface, this should also be a series of start-up buttons or option commands. The main difference here is that they are all lit, where the Eve Pod’s buttons were dim until hit. Since every other interface on the Axiom glows when in use, it looks like all of Otto’s commands and autopilot options are active when the Captain deactivates him.

A hint of practicality…

The panel is in a place that is accessible and would be easily located by service crew or trained operators. Given that the Axiom is a spaceship, the systems on board are probably heavily regulated and redundant. However, the panel isn’t easily visible thanks to specific decisions by BNL. This system makes sense for a company that doesn’t think people need or want to deal with this kind of thing on their own.

Once the panel is open, the operator has a clear view of which systems are on, and which are off. The major downside to this keypad (like the Eve Pod) is that the coding of the information is obscure. These cryptic buttons would only be understandable for a highly trained operator/programmer/setup technician for the system. Given the current state of the Axiom, unless the crew were to check the autopilot manual, it is likely that no one on board the ship knows what those buttons mean anymore.

WallE-Otto03

Thankfully, the most important button is in clear English. We know English is important to BNL because it is the language of the ship and the language seen being taught to the new children on board. Anyone who had an issue with the autopilot system and could locate the button, would know which button press would turn Otto off (as we then see the Captain immediately do).

Considering that Buy-N-Large’s mission is to create robots to fill humans’ every need, saving them from every tedious or unenjoyable job (garbage collecting, long-distance transportation, complex integrated systems, sports), it was both interesting and reassuring to see that there are manual over-rides on their mission-critical equipment.

…But hidden

The opposite situation could get a little tricky though. If the ship was in manual mode, with the door closed, and no qualified or trained personnel on the bridge, it would be incredibly difficult for them to figure out how to physically turn the ship back to auto-pilot. A hidden emergency control is useless in an emergency.

Hopefully, considering the heavy use of voice recognition on the ship, there is a way for the ship to recognize an emergency situation and quickly take control. We know this is possible because we see the ship completely take over and run through a Code Green procedure to analyze whether Eve had actually returned a plant from Earth. In that instance, the ship only required a short, confused grunt from the Captain to initiate a very complex procedure.

Security isn’t an issue here because we already know that the Axiom screens visitors to the bridge (the Gatekeeper). By tracking who is entering the bridge using the Axiom’s current systems, the ship would know who is and isn’t allowed to activate certain commands. The Gatekeeper would either already have this information coded in, or be able to activate it when he allowed people into the bridge.

For very critical emergencies, a system that could recognize a spoken ‘off’ command from senior staff or trained technicians on the Axiom would be ideal.

Anti-interaction as Standard Operating Procedure

WallE-Otto05

The hidden door, and the obscure hard-wired off button continue the mission of Buy-N-Large: to encourage citizens to give up control for comfort, and make it difficult to undo that decision. Seeing as how the citizens are more than happy to give up that control at first, it looks like profitable assumption for Buy-N-Large, at least in the short term. In the long term we can take comfort that the human spirit–aided by an adorable little robot–will prevail.

So for BNL’s goals, this interface is fairly well designed. But for the real world, you would want some sort of graceful degradation that would enable qualified people to easily take control in an emergency. Even the most highly trained technicians appreciate clearly labeled controls and overrides so that they can deal directly with the problem at hand rather than fighting with the interface.

The Excessive Machine

When Durand-Durand captures Barbarella, he places her in a device which he calls the “Excessive Machine. She sits in a reclining seat, covered up to the shoulders by the device. Her head rests on an elaborate red leather headboard. Durand-Durand stands at a keyboard, built into the “footboard” of the machine, facing her.

The keyboard resembles that of an organ, but with transparent vertical keys beneath which a few colored light pulse. Long silver tubes stretch from the sides of the device to the ceiling. Touching the keys (they do not appear to depress) produces the sound of a full orchestra and causes motorized strips of metal to undulate in a sine wave above the victim.

When Durand-Durand reads the strange sheet music and begins to play “Sonata for Executioner and Various Young Women,” the machine (via means hidden from view) removes Barbarella’s clothing piece by piece, ejecting them through a tube in the side of the machine near the floor. Then in an exchange Durand-Durand reveals its purpose…

  • Barbarella
  • It’s sort of nice, isn’t it?
  • Durand-Durand
  • Yes. It is nice. In the beginning. Wait until the tune changes. It may change your tune as well.
  • Barbarella
  • Goodness, what do you mean?
  • Durand-Durand
  • When we reach the crescendo, you will die of pleasure. Your end will be swift, but sweet, very sweet.

As Durand-Durand intensifies his playing, Barbarella writhes in agony/ecstasy. But despite his most furious playing, he does not kill Barbarella. Instead his machine fails dramatically, spewing fire and smoke out of the sides as its many tubes burn away. Barbarella is too much woman for the likes of his technology.

I’m going to disregard this as a device for torture and murder, since I wouldn’t want to improve such a thing, and that whole premise is kind of silly anyway. Instead I’ll regard it as a BDSM sexual device, in which Durand-Durand is a dominant, seeking to push the limits of an (informed, consensual) submissive using this machine. It’s possible that part of the scene is demonstration of prowess on a standardized, difficult-to-use instrument. If so, then a critique wouldn’t matter. But if not…Since the keys don’t move, the only variables he’s controlling are touch duration and vertical placement of his fingers. (The horizontal position on each key seems really unlikely.) I’d want to provide the player some haptic feedback to detect and correct slipping finger placement, letting him or her maintain attention on the sub who is, after all, the point.