Tag Archives: disambiguation

Browsing Files in 3D

by

0

Be forewarned—massive spoilers ahead. (The graphic shows the Millennium Falcon sporting a massive spoiler.)

What’s this all about?

The origin story here is that I wanted to review Hackers, a film I enjoy and Chris describes as “awesome/ly bad”. However, Hackers isn’t science fiction. Well, I could argue that it is set in an alternate reality where computer hackers are all physically attractive with fashionable tastes in music and clothing, but that isn’t enough. The film was set firmly in the present day (of 1995) and while the possibilities of computer hacking may be exaggerated for dramatic purposes, all the computers themselves are quite conventional for the time. (And therefore appear quaint and outdated today.)

With the glorious exception of the three dimensional file storage system on the “Gibson” mainframe. This fantastic combination of hardware and software was clearly science fiction then, and remains so today. While one futuristic element is not enough to justify a full review of Hackers, it did start us thinking. The film Jurassic Park also has a 3D file system navigator, which wasn’t covered in depth by either the book or the online review. And when Chris reached out to the website readers, they provided more examples of 3D file systems being added to otherwise mundane computer systems.

So what we have here is a review of a particular interface trope rather than an entire film or TV show: the three dimensional file browsing and navigation interface.

Scope

This review is specifically limited to interfaces that are recognisably file systems, where people search for and manipulate individual documents or programs. Cyberspace, a 3D representation of the Internet or World Wide Web, is too broad a topic, and better covered in individual reviews such as those for Ghost in the Shell and Johnny Mnemonic.

I also originally intended to only include non-science fiction films and shows but Jurassic Park is an exception. Jurassic Park has been reviewed, both in the book and on the website, but the 3D file system was a comparatively minor element. It is included here as a well known example for comparison.

The SciFiInterfaces readership also provided examples of research papers for 3D file system browsing and navigation—rather more numerous than actual production systems, even today. These will inform the reviews but not be discussed individually.

Because we are reviewing a topic, not a particular film or TV show, the usual plot summaries will be shortened to just those aspects that involve the 3D file system. As a bonus, we can also compare and contrast the different interfaces and how they are used. The worlds of Ghost in the Shell and Johnny Mnemonic are so different that it would be unfair to judge individual interfaces against each other, but for this review we are considering 3D file systems that have been grafted onto otherwise contemporary computer systems, and used by unaugmented human beings to perform tasks familiar to most computer users.

Sources

Having decided on our topic and scope, the properties for review are three films and one episode of a TV show.

Jurassic Park, 1993

“I know this!” and Jurassic Park is so well known that I assume that you do too. We will look specifically at the 3D file system that is used by Lex in the control room to reactivate the park systems.

Disclosure, 1994

This film about corporate infighting includes a virtual reality system, complete with headset, glove, laser trackers, and walking surface, which is used solely to look for particular files.

Hackers, 1995

As mentioned in the introduction this film revolves around the hacking of a Gibson mainframe, which has a file system that is both physically three dimensional and represented on computer screens as a 3D browser.

A bar chart. The x-axis is every year between and including 1902 to 2022. The y-axis, somewhat humorously, shows 2-place decimal values up to 1. Three bars at 1.00 appear at 1993, 1994, and 1995. One also appears in 2016, but has an arrow pointing back to the prior three with a label, “(But really, this one is referencing those.)”

All three of these films date from the 1990s, which seems to have been the high point for 3D file systems, both fictional and in real world research.

Community, season 6 episode 2, “Lawnmower Maintenance and Postnatal Care” (2016)

In this 21st century example, the Dean of a community college buys an elaborate virtual reality system. He spends some of his time within VR looking for, and then deleting, a particular file. 

Clockwise from top left: Jurassic Park (1993), Disclosure (1994), Hackers (1995), Community (2016)

And one that almost made it

File browsing in two dimensions is so well established in general-purpose computer interfaces that the metaphor can be used in other contexts. In the first Iron Man film, at around 52 minutes, Tony Stark is designing his new suit with an advanced 3D CAD system that uses volumetric projection and full body gesture recognition and tracking. When Tony wants to delete a part (the head) from the design, he picks it up and throws it into a trashcan.

Tony deletes a component from the design by dropping it into a trashcan. Iron Man (2008)

I’m familiar with a number of 2D and 3D design and drawing applications and in all of them a deleted part quietly vanishes. There’s no more need for visual effects than when we press the delete key while typing.

In the film, though, it is not Tony who needs to know that the part has been deleted, but the audience. We’ve already seen the design rendering moved from one computer to another with an arm swing, so if the head disappeared in response to a gesture, that could have been interpreted as it being emailed or otherwise sent somewhere else. The trashcan image and action makes it clear to the audience what is happening.

So, that’s the set of examples we’ll be using across this series of posts. But before we get into the fiction, in the next post we need to talk about how this same thing is handled in the real world.

Kimoyo Beads

by

3

One of the ubiquitous technologies seen in Black Panther is the kimoyo bead. They’re liberally scattered all over the movie like tasty, high-tech croutons. These marble-sized beads are made of vibranium and are more core to Wakandan’s lives than cell phones are to ours. Let’s review the 6 uses seen in the film.

1. Contact-EMP bombs

We first see kimoyo beads when Okoye equips T’Challa with a handful to drop on the kidnapper caravan in the Sambisa forest. As he leaps from the Royal Talon, he flings these, which flatten as they fall, and guide themselves to land on the hoods of the caravan. There they emit an electromagnetic pulse that stops the vehicles in their tracks. It is a nice interaction that does not require much precision or attention from T’Challa.

2. Comms

Wakandans wear bracelets made of 11 kimoyo beads around their wrists. If they pull the comms bead and place it in the palm, it can project very lifelike volumetric displays as part of realtime communication. It is unclear why the bead can’t just stay on the wrist and project at an angle to be facing the user’s line of sight, as it does when Okoye presents to tribal leaders (below.)

We see a fascinating interaction when T’Challa and W’Kabi receive a call at the same time, and put their bracelets together to create a conference call with Okoye.

The scaled-down version of the projection introduces many of the gaze matching problems identified in the book. Similarly to those scenes in Star Wars, we don’t see the conversation from the other side. Is Okoye looking up at giant heads of T’Challa and W’Kabi? Unlikely. Wakanda is advanced enough to manage gaze correction in such displays.

Let me take a moment to appreciate how clever this interaction is from a movie maker’s perspective. It’s easy to imagine each of them holding their own bead separately and talking to individual instances of Okoye’s projection. (Imagine being in a room with a friend and both of you are on a group call with a third party.) But in the scene, she turns to address both T’Challa and W’Kabi. Since the system is doing body-and-face gaze correction, the two VP displays would look slightly different, possibly confusing the audience into thinking these were two separate people on the call. Wakandans would be used to understanding these nuances, but us poor non-Wakandan’s are not.

Identical Okoyes ensures (at least) one of the displays is looking at something weird. It’s confusing.
This is confusing.
Having gaze correction so both Okoyes are looking at T’Challa when she’s talking to him makes it look like there are two different characters. It’s confusing.
This is also confusing.

The shared-display interaction helps bypass these problems and make the technology immediately understandable and seamless.

Later Shuri also speaks with Okoye via communication bead. During this conversation, Shuri removes another bead, and tosses it into a display to show an image and dossier of Killmonger. Given that she’s in her lab, it’s unclear why this gesture is necessary rather than, say, just looking toward a display and thinking, “Show me,” letting the AI Griot interpret from the context what to display.

A final communication happens immediately after as Shuri summons T’Challa to the the lab to learn about Killmonger. In this screenshot, it’s clear that the symbol for the comms bead is an asterisk or star, which mimics the projection rats of the display, and so has some nice semantics to help users learning which symbols do what.

3. Presentation

 In one scene, Okoye gives the tribe rulers a sitrep using her kimoyo beads as a projector. Here she is showing the stolen Wakandan artifact. Readers of the book will note the appearance of projection rays that are standard sci-fi signals that what is seen is a display. A lovely detail in the scene is how Okoye uses a finger on her free hand to change the “slide” to display Klawe. (It’s hard to see the exact gesture, but looks like she presses the projection bead.) We know from other scenes in the movie that the beads are operated by thought-command. But that would not prevent a user from including gestures as part of the brain pattern that triggers an event, and would make a nice second-channel confirmation as discussed in UX of Speculative Brain-Computer Inputs post.

4. Remote piloting

When T’Challa tours Shuri’s lab, she introduces him to remote access kimoyo beads. They are a little bigger than regular beads and have a flared, articulated base. (Why they can’t just morph mid-air like the ones we see in the kidnapper scene?) These play out in the following scene when the strike team needs to commandeer a car to chase Klawe’s Karavan. Oyoke tosses one on the hood on a parked car, its base glows purple, and thereafter Shuri hops into a vibranium-shaped simulacrum of the car in her lab, and remotely operates it.

A quick note: I know that the purple glow is there for the benefit of the audience, but it certainly draws attention to itself, which it might not want to do in the real world.

In the climactic battle of the tribes with Killmonger, Shuri prints a new bracelet and remote control bead for Agent Ross. She places the bracelet on him to enable him to remote pilot the Royal Talon. It goes by very quickly, and the scene is lit quite sparsely, but the moment she puts it on him, you can see that the beads are held together magnetically.

5. Eavesdropping

When Agent Ross is interrogating the captured Klawe, we get a half-second shot to let us know that a kimoyo bead has been placed on his shoulder, allowing T’Challa, Okoye, and Nakia to eavesdrop on the conversation. The output is deliveredby a flattened bone-conducting speaker bead behind their left hears.

6. Healing

Later in the scene, when Killmonger’s bomb grievously wounds Agent Ross in his spine, T’Challa places one of Nakia’s kimoyo beads onto the wound, stabilizing Ross long enough to ferry him to Wakanda where Shuri can fully tend to him. The wound conveniently happens to be kimoyo-bead sized, but I expect that given its shape-shifting powers, it could morph to form a second-skin over larger wounds.

I wondered if kimoyo beads were just given to Wakandan royalty, but it’s made clear in the scene where T’Challa and Nakia walk through the streets of Birnin Zana that every citizen has a bracelet. There is no direct evidence in the film, but given the pro-social-ness throughout, I want to believe that all citizens have free access to the beads, equipping each of them to participate equitably in the culture.

So, most of the interaction is handled through thought-command with gestural augmentation. This means that most of our usual concerns of affordances and constraints are moot. The one thing that bears some comment is the fact that there are multiple beads on the bracelet with different capabilities. How does a user know which bead does what?

As long as the beads can do their job in place on the wrist, I don’t think it matters. As long as all of the beads are reading the user’s thoughts, only the one that can respond need respond. The others can disregard the input. In the real world you’d need to make sure that one thought isn’t interpretable as multiple things, a problem discussed on my team at IBM as disambiguation. Or if they are you must design an interaction where the user can help disambiguate the input, or tell the system which meaning they intend. We never this edge case in Black Panther. 

It seems that some of the beads have specialized functions that cannot be performed by another, each has several symbols engraved into it, the indentions of which glow white for easy identification. The glow is not persistent across all uses, so it must be either context-aware and/or a setting that users can think to change. But even when not lit, the symbols are clear, and clearly distinguishable, so once the user learns the symbols, the labeling should help.

Black Votes Matter

Today is an important day in the United States. It’s election day 2020. Among one of the most important days in U.S. politics, ever. Among Trump’s litany of outrageous lies across his presidency is this whopper: “I have done more for Black Americans than anybody, except for the possible exception of Abraham Lincoln.” (Pause for your spit take and cleaning your screen.)

As infuriating and insulting as this statement is emotionally (like, fuck you for adding “possible” in there, like it’s somehow possible that you’ve done more than freed our black citizens from slavery, you maggot-brained, racist, malignant narccicist) let’s let the Brookings institute break down why, if you believe Black Lives Matter, you need to get out there and vote blue all the way down the ticket.

https://www.cnn.com/2020/11/02/us/ocoee-massacre-100th-anniversary-trnd/index.html

You should read that whole article, but some highlights/reminders

  • Trump ended racial sensitivity training, and put a ban on trainings that utilize critical race theory
  • Hate crimes increased over 200% in places where Trump held a campaign rally in 2016
  • He dismissed the Black Lives Matters movement, said there were “fine people” among white supremacist groups, and rather than condemning the (racist, not gay) Proud Boys, told them to “stand by.”
  • Not a single one of his 53 confirmed appeals court judges circuit justices is black.
  • The criminal mishandling of the COVID-19 pandemic has killed twice as many black Americans as it has white Americans. (Don’t forget he fired the pandemic response team.)

If you are reading this on election day, and have not already done so, please go vote blue. Know that if you are in line even when the polls officially closed, they have to stay open for the entire line to vote. If you have voted, please help others in need. More information is below.

https://www.vote.org/

If you are reading this just after election day, we have every evidence that Trump is going to try and declare the election rigged if he loses (please, please let it be when he loses to a massive blue waver). You can help set the expectation among your circle of friends, family, and work colleagues that we won’t know the final results today. We won’t know it tomorrow. We may have a better picture at the end of the week, but it will more likely take until late November to count everyone’s vote, and possibly until mid December to certify everyone’s vote.

And that’s what we do in a liberal democracy. We count everyone’s vote, however long that takes. To demand it in one day during a pandemic is worse than a toddler throwing a “I want it now” tantrum. And we are so very sick of having a toddler in this position.

By Christian Bloom

Brain Upload

by

6

Once Johnny has installed his motion detector on the door, the brain upload can begin.

3. Building it

Johnny starts by opening his briefcase and removing various components, which he connects together into the complete upload system. Some of the parts are disguised, and the whole sequence is similar to an assassin in a thriller film assembling a gun out of harmless looking pieces.

jm-6-uploader-kit-a

It looks strange today to see a computer system with so many external devices connected by cables. We’ve become accustomed to one piece computing devices with integrated functionality, and keyboards, mice, cameras, printers, and headphones that connect wirelessly.

Cables and other connections are not always considered as interfaces, but “all parts of a thing which enable its use” is the definition according to Chris. In the early to mid 1990s most computer user were well aware of the potential for confusion and frustration in such interfaces. A personal computer could have connections to monitor, keyboard, mouse, modem, CD drive, and joystick – and every single device would use a different type of cable. USB, while not perfect, is one of the greatest ever improvements in user interfaces. Continue reading →

Time circuits (which interface the Flux Capacitor)

by

4

BttF_137Time traveling in the DeLorean is accomplished in three steps. In the first, he traveler turns on the “time circuits” using a rocking switch in the central console. Its use is detailed in the original Back to the Future, as below.

In the second, the traveler sets the target month, day, year, hour, and minute using a telephone keypad mounted vertically on the dashboard to the left, and pressing a button below stoplight-colored LEDs on the left, and then with an extra white status indicator below that before some kind of commit button at the bottom.
 BttF_135

In the third, you get the DeLorean up to 88 miles per hour and flood the flux capacitor with 1.21 gigawatts of power.

Seems simple.

It’s not… Continue reading →

Course-correction

by

5

The stage managers’ main raison d’être is to course-correct if and when victims begin to deviate from the path required of the ritual.

This begins with the Prep team, long before the victims enter the stage. For example, Jules’ hair dye and Marty’s laced pot. These corrections become more necessary and intense once the victims go on stage.

Making sure there are sexy times

The ritual requires that a sexy young couple have sexy times on stage before they suffer and die. “The mood” can be ruined by many things, but control has mechanisms to cope with most of them. We see three in the movie.

Temperature

The temperature can’t be too hot or too cold, but this isn’t something that can be set and forgot. What counts as the right temperature is a subjective call for the people involved and their circumstances, such as being drunk, or amount and type of clothes worn. Fortunately, the video-audio panopticon lets the stage managers know when a victim speaks about this directly, and do something about it. The moment Jules complains, for instance, Sitterson is able to reach over to a touch-screen display and tap the temperature a few degrees warmer.

Sitterson heats things up.

The gauge is an interesting study. It implies a range possible between 48 and 92 degrees Fahrenheit, each of which is uncomfortable enough to encourage different behaviors in the victims, without the temperature itself being life-threatening.

Moreover, we see that it’s a “blind” control. Before Sitterson taps it, he is only shown the current temperature as a blue rectangle that fills up four bars and that it is exactly 64 degrees. But if he knew he wanted it to be 76 degrees, what, other than experience or training, tells him where he should touch to get to that desired new temperature? Though the gauge provides immediate feedback, it still places a burden on his long-term memory. And for novice users, such unlabeled controls require a trial-and-error method that isn’t ideal. Even the slim area of white coloring at the top, which helpfully indicates temperatures warmer than cooler, appears too late to be useful.

Better would be to have the color alongside or under the gauge with smaller numbers indicated along its length such that Sitterson could identify and target the right temperature on the first try.

Libido

The next thing that can risk the mood is a lack of a victim’s amorous feelings. Should someone not be “feeling it,” Control can pipe sex pheromones to areas on stage. We see Hadley doing this by operating a throttle lever on the electronic-era control panel. After Hadley raises this lever, we see small plumes of mist erupt from the mossy forest floor that Jules and Curt are walking across.

Hadley introduces pheromones to the forest air.

This control, too, is questionable. Let’s first presume it’s not a direct control, like a light switch, but more of a set-point control, like a thermostat. Similar to the temperature gauge above, this control misses some vital information for Hadley to know where to set the lever to have the desired amount of pheromone in the air, like a parts-per-million labeling along the side. Perhaps this readout occurs on a 7-segment readout nearby or a digital reading on some other screen, but we don’t see it.

There is also no indication about how Hadley has specified the location for the pheromone release. It’s unlikely that he’s releasing this everywhere on stage, lest this become a different sort of ritual altogether. There must be some way for him to indicate where, but we don’t see it in use. Perhaps it is one of the lit square buttons to his right.

An interesting question is why the temperature gauge and pheromone controls, which are similar set-point systems, use not just different mechanisms, but mechanisms from different eras. Certainly such differentiation would help the stage managers’ avoid mistaking one for the other, and inadvertently turn a cold room into an orgy, so perhaps it is a deliberate attempt to avoid this kind of mistake.

Lights

The final variable that stands in the way of Jules’ receptiveness (the authors here must acknowledge their own discomfort in having to write about this mechanistic rape in our standard detached and observational tone) is the level of light. After she complains that it is too dark, Hadley turns a simple potentiometer and the “moonlight” on a soft bed of moss behind them grows brighter.

Control responds to Jules’ objection to the darkness.

This, too, is a different control than the others; though it controls what is essentially a floating-point variable. But since it is more of a direct control than the other two, its design as a hard-stop dial makes sense, and keeps it nicely differentiated from the others.

Marty’s Subliminal Messages

Over the course of the movie, several times we hear subliminal messages spoken to directly control Marty. We never see the inputs used by Control, but they do, at least on one occasion, actually influence him, and is one of the ways the victims are nudged into place.

Marty breaks the fourth wall

In addition to Dana & Curt’s almost not getting it on, another control-room panic moment comes when Marty accidentally breaks a lamp and finds one of the tiny spy cameras embedded throughout the cabin. Knowing that this level of awareness or suspicion could seriously jeopardize the scenario, Hadley bolts to a microphone where he says, “Chem department, I need 500 ccs of Thorazine pumped into room 3!”

Marty finds a spy camera

Hadley speaks a command to the Chem department

Careful observers will note while watching the scene that a menu appears on a screen behind him as he’s stating this. The menu lists the following four drugs.

  • Cortisol (a stress hormone)
  • Pheromones (a category of hormonal social signals, most likely sex pheromones)
  • Thorazine (interestingly, an antipsychotic known to cause drowsiness and agitation)
  • Rhohyptase (aka Rhohypnol, the date rape drug)

Given that content, the timing of the menu is curious. It appears, overlaid on the victim monitoring screen, the moment that Hadley says “500.” (Before he can even specify “Thorazine.”) How does it appear so quickly? Either there’s a team in the Chem department also monitoring the scene, and who had already been building a best-guess menu for what Hadley might want in the situation and they just happened to push it to Hadley’s screen at that moment; Or there’s an algorithmic voice- and goal-awareness system that can respond quickly to the phrase “500 ccs” and provide the top four most likely options. That last one is unlikely, since…

  • We don’t see evidence of it anywhere else in the movie
  • Hadley addresses the Chem department explicitly
  • We’d expect him to have his eyes on the display, ready to make a selection on its touch surface, if this was something that happened routinely

But, if we were designing the system today with integrated voice recognition capabilities, it’s what we’d do.

Curt suggests they stick together

After the attack begins on the cabin itself, Curt wisely tells the others, “Look, we’ve got to lock this place down…We’ll go room by room, barricade every window and every door. We’ve got to play it safe. No matter what happens, we have to stay together.” Turns out this is a little too wise for Hadley’s tastes. Sitterson presses two yellow, back-lit buttons on his control panel to open vents in the hallway, that emit a mist. As Curt passes by the vents and inhales, he pauses, turns to the others and says, “This isn’t right…This isn’t right, we should split up. We can cover more ground that way.”

Sitterson knocks some sense out of Curt.

This two-button control seems to indicate drug (single dose) and location, which is sensible. But if you are asking users to select from different variables, it’s a better idea to differentiate them by clustering and color, to avoid mistakes and enable faster targeting.

Locking the doors

Once the victims are in their rooms, Hadley acknowledges it’s time to, “Lock ‘em in.” Sitterson flips a safety cover and presses a back-lit rocker switch, which emits a short beep and bolts the doors to all the victims’ rooms at the same time.

Sitterson bolts the victims’ doors.

Marty in particular notices the loud “clunk” as the bolts slide into place. He tests the door and is confounded when he finds it is, in fact, locked tight. Control’s earlier concern about tipping their hand seems to matter less and less, since this is a pretty obvious manipulation.

The edge of the world

Bolted doors pale in comparison to the moment when Curt, Dana, and Holden violently encounter the limits of the stage. After the demolition team seals the tunnel to prevent escape that way, Curt tries to jump the ravine to the other side so he can fetch help. Unfortunately for him, the ravine is actually an electrified display screen, showing a trompe-l’œil illusion of the far side. By trying to jump the ravine, Curt unwittingly commits suicide by slamming into it.

Curt slams into the edges of the “world” of the cabin.

The effect of the screen is spectacular, full of arcs zipping along hexagonal lines and sparks flying everywhere. Dana and Holden rush to the edge of the cliff to watch him tumble down its vast, concave surface. It seems that if you’ve come this far, Control isn’t as concerned about tipping its hand as it is finishing the job.