Grade Board

When students want to know the results of their tests, they do so by a public interface. A large, tiled screen is mounted to a recessed section of wall in a courtyard. The display is divided into a grid of five columns and three rows. Each cell contains one student’s results for one test, as a percentage. One cell displays an ad for military service. Another provides a reminder for the upcoming sports game. Four keyboards are situated below the screens at waist level.

StarshipT_026

To find her score, Carmen approaches one of the keyboards and enters some identifying data. In response, the column above the screen displays her score and moves the data in the other cells up. There is no way to learn of one’s test scores privately. This hits Johnny particularly hard when he checks his scores to find he has earned 35% on his Math Final, a failing grade.

Worse, his friend Carl is able to walk up to the keyboard and with a few key presses, interrupt every other student looking at the grades, and fill the entire screen with Johnny’s score for all to see, with the failing number blinking red and white, ridiculing him before his peers. After a reprimand from Johnny, Carl returns the display to normal with the press of a button.

StarshipT_025

STARSHIP_TROOPERS_grade

Is ANSI the right input?

The keyboard would be a pain to keep clean, and you’d figure that a student ID would be a unique-and-memorable enough token. Does an entire ANSI keyboard need to be there? Wouldn’t a number pad be enough? But why a manual input at all? Nowadays you’d expect some near-field communication, or biometric token, which would obviate the keyboard entirely.

Are publicizing grades OK?

So there are input and interaction improvements to be made, for sure. But there’s more important issues to talk about here. Yes, students can accomplish one task with the interface well enough: Checking grades. But what about the giant, public output?

It’s fullfilling one of the dystopian goals of the fascist society in which the story takes place, which is that might makes right. Carl is a bully (even if Jonny’s friend) and in the culture of Starship Troopers, if he wants to increase Johnny’s public humiliation, why not? Johnny needs to study harder, take it on the chin, or make Carl stop. In this regard, the interface satisfies both the students’ task and the culture’s…um…values.

I originally wanted to counter that with a strong statement that, “But that’s not us.” After all, modern federal privacy laws in the United States forbid this public display as a violation of students’ privacy. (See FERPA laws.) But apparently not everyone believes this. A look on debate.org (at the time of writing) shows that opinion is perfectly split on the topic. I could lay out my thoughts on which side is better for learning, but it’s really beyond the scope of this blog to build a case for either side of Lakoff’s Moral Politics.

Screen cap from debate.org

You’re Doing More Than You Think You’re Doing

But it’s worth noting the scope of these issues at hand. This seems at first to be an interface just about checking grades, but when you look at the ecosystem in which it operates, it actually illustrates and reinforce a culture’s core virtues. The interface is sometimes not just the interface. Its designers are more than flowchart monkeys.

Make It Sew

Scifiinterfaces.com is thrilled to announce the completion of…a follow-up book!

embroidered

From the back cover:

Few people realize the indelible mark that crafting in general—and sewing in particular—have made on science fiction as a genre. Building on the success of the original work, Make It Sew: Crafting Lessons from Science Fiction scours the history of popular and obscure science fiction to find and analyze the best patterns from the textile arts.

Make it sew number one

Chapters include

  • The fabric of the Federation
  • Seam Reapers
  • Lilo’s stitch
  • Famous and infamous seamsters: From Picard’s plackets to Darth Quilt
  • Warp & Weft
  • The rise of the RoboBobbins

Sewlo

Early Praise for Make It Sew:

 
I was at my wit’s end when little Timmy asked me to help him with his cosplays, but now thanks to Make it Sew I know I’m using the very cuts and fabrics that changed the face of science fiction.  Timmy couldn’t be happier, and his Leia Slave costume couldn’t fit any better.
Betty Womack
from Lands Ford, Indiana
 
This season its all about futuristic fabrics and forward-thinking colors for your home and wardrobe. From fur-lined Barbarella bedrooms to form-fitting imperial blast armor, Make It Sew is the inspiration behind my brand new sci-fi product line.
 
It’s not Science fiction, it’s Science Fashion, people. It’s time to roll up our sleeves and get crafting.  Sew. Say. We. All.
Laura Roslin
President of the Colonies

Leeloo-sweater

Fans of the book will be excited to learn of a companion website, scifiinterfacing.com. Let me know in the comments below if you’d like to be on the mailing list for when it goes live!

 

 

 

Credit where credit is due:

  • Han Sewlo is holding a Star Wars quilt actually made by RobinLovesQuilting. Check it out on her blog.
  • Leeloo did not knit that herself. It’s from Dorothy Perkins.
  • Picard is working on a sampler straight from the hilarious Subversive Cross Stitch, specifically the “Bitch, Please” kit. Go buy one, because awesome.

FedPaint

Fedpaint_big

Students in Starship Troopers academy have access to desktop computing environments during class, including a drawing and animation program called “Fedpaint,” that had a number of very forward-looking features.

The screen is housed in a metal bezel that is attached to the desk, and can be left flat or angled slightly per the user’s preference. A few hardware buttons sit in a row at the bottom of the bezel. (Quick industrial design aside: Those buttons belong at the top of the bezel.) The input device is a stylus. (Styli had been in use in personal digital assistants for over a decade when the film came out, I don’t think they had been sold as the primary input for a PC.) When we first see Johnny using the computer, he is ignoring his citizenship lesson and using Fedpaint instead.

StarshipT_013

The main part of the interface is a canvas. Running along the left and bottom edges are a complex tool palette and color picker that is vaguely reminiscent of Windows 3.0 WIMP applications. It’s easy to tell which category and tool is selected. (What color is selected is unclear.) I’d even say that most of the icons, while a little ham-handed and completely lacking labels, convey what they would do pretty clearly. The tools also seem to be clustered logically with categories across the top left, tools in the middle left, a color palette in the lower left corner, and file operations across the bottom. That’s some reasonable and reasonably convincing layout design for a movie interface. Nowadays a designer might argue to hide the menus when not in use to maximize the canvas real estate, but the most common OS paradigm at the time was Windows 97, and the most advanced paint program, i.e. Photoshop, looked like this. (Major thanks to Hongkiat for keeping their museum of Photoshop interfaces.)

Using the stylus, Johnny sketches a flirty animation for Carmen. He draws each of their profiles in white lines. He then adds some flat color and animates the profiles (not shown onscreen) such that the faces get closer, their eyes close, and their mouths open in readiness of a kiss. He then sends it to her.

On her desk she receives a notification. (We don’t get to see it. Was she already in the program? Did the notification jump her there?) Carmen grabs her stylus and responds by adding to the animation. She sends the file back to him. He opens it and it plays automatically. In her version of the animation, the profiles approach as before, but as they near for a kiss, the female profile blows a bubble gum bubble that gets so large it pops and covers the face of the male.

StarshipT_019

What’s nice about this interface is that the narrative seems to have driven some innovation in its design. It’s half gee-whiz-circa-1997 of course but half character development as it tells us that Johnny likes Carmen, and Carmen is a bit playfully stand-offish in response. To make this work well narratively, communication of the animation back and forth had to be seamless, and that seems to be the reason we see the communication tools built right into the interface. If ever there was a case for why scenario-driven design for personas works, this is it.

What’s frustrating is that they skipped over the hard part. How does Johnny apply the color? A paint bucket tool is a reasonable guess, but it’s also error prone. How did he specify the number of frames and their speed? How did he ensure that the motion felt relatively smooth and communicative? Anyone who’s worked with an animation program knows that these aren’t trivial matters, and Starship Troopers took the narrative route. Probably best for the story, but less for my analysis purposes.

Still, the stylus-driven direct manipulation, the unique layout, and easy, social sharing were big innovations for the time. I don’t know that there’s much to learn from this today, since our OS metaphors have advanced enough to make this seem quaint at best, and social integration is now the norm. But credit where it’s due, this interface was ahead of its time.

Very slightly interactive news

StarshipT_001

One of the most unusual conceits of the movie is “Would you like to know more?” These consist of short video news sequences with overlaid graphics and narration. At the top of the screen the user can click one of three categories for different categories of video feed, and two functions. At the end of each video sequence the “user” is prompted to interact—should they want to learn more—by clicking the legend at the bottom of the screen.

StarshipT_002

The user here is ambiguous. It might be that the audience member is the user, but of course it’s not interactive. There’s probably room here for some other writer to investigate the narrative tactic/semiotics of using an interactive interface in a passive story.

At the top of the screen are menu headers labeled “FEDERAL,” “GALAXY,” “TOP NEWS," "ENLIST," and "EXIT." For the usability purist, the collection is problematic for a number of reasons.

  • The information categories aren’t parallel, and there’s no clear reason why they shouldn’t be. What’s the relationship between Galaxy and Federal?
  • The functions (enlist and exit) are not visually distinguished from content categories.
  • The current state of the interface is a mystery. Am I currently watching Top News or something else?
  • Why does the interface chrome persist? Aren’t they distractions from the content? Maybe they should appear just only for the few seconds it’s inviting the user to interact, and fade at other times.
  • While a fascist government would be happy to try and trick its users into clicking enlist, I can’t imagine what benefit they get from having them accidentally clicking exit to close the propaganda engine. These should not just be visually distinguished, but given different visual weight. They’d probably want enlist large and exit smaller, if there at all.

“Welp. All the links in Federal, Galaxy, and Top News are purple. I wonder what’s happening in ENLIST news? Oh hey, who’s that pounding on the door?”

StarshipT_005

The presence of the "EXIT" control implies that this is an application running in an operating system or media computer space. This opt-in news application with its small windows of time for interaction helps to paint a picture of a highly engaged and ready-to-respond audience, fitting for the mid-war society portrayed in the movie.

Only once do we see an unidentified and unseen "user" control a cursor to view more. In this sequence, he or she clicks on “more” after watching a clip on the bug homeworld Klendathu. (It’s worth noting/condemning that the clickable word “more” looks identical to the rest of the non-clickable text, offering no special affordance.) In response to the selection, the application shows a live video news feed from the conflict on Klendathu. Was it just good fortune that a live feed happened to be available at this moment? More likely the application and media coordination system are smart enough to know a live feed was coming up, and played the trailer in advance as an advertisement for the content, implying a well-coordinated propaganda/content management system.

Starship Troopers (1997): Overview

Release date: 27 June 2008, United States

StarshipTroopers_title

Starship Troopers follows Johnny Rico and his friends Diz, Carmen, and Carl, from humble beginnings together at school, into their participation in a war against the super evolved insect race called the Arachnids, or “Bugs.” Johnny, who is excellent at sports but poor with verbal and math skills, enters the infantry with Diz. Carmen’s scores and skills lead her to being a pilot aboard the war ship Rodger Young. Carl shows nascent psychic ability and winds up in military intelligence. In these roles they travel to the bug home planet of Klendathu to not only score a major victory in the war, but come of age in dealing with life and love.

StarshipTrooper_end

IMDB: https://www.imdb.com/title/tt0120201/Currently streaming on:

Fifth Element tees

overview

Major thanks to everyone who came out and joined me for the first ever scifiinterfaces.com movie night at The New Parkway in Oakland! It was a sold-out show, and while there were a few glitches, folks are telling me they had a great time and are looking forward to the next one. There will be a more detailed report once the pre-show video comes out. But in the meantime, this: If you didn’t win the trivia contest or weren’t able to attend, you can still get your hands on the “movie night” t-shirts I debuted there.

Allshirts

Head on over to the spreadshirt shop. It’s ugly (with the default CSS). It doesn’t have a custom URL or anything. It only has 5 products at the moment. But hey, that’s all part of the charm if you’d like to wear your sci-fi interface nerdiness with pride.

http://26253.spreadshirt.com/

P.S. I have no idea why the women’s KEEP CLEAR tee is not appearing in orange since I designed it like the Men’s tee, but I have a request with Spreadshirt now. Hopefully it’ll be fixed soon.

Berlin?

I’m thinking the Bay Area has an appetite for maybe two movie nights a year (let me know if I’m wrong) but I’d also love to try this in Berlin. Do you (or someone you know) know of a cinema in Berlin like the New Parkway that might be interested in my replicating this there?

Brain interfaces as wearables

There are lots of brain devices, and the book has a whole chapter dedicated to them. Most of these brain devices are passive, merely needing to be near the brain to have whatever effect they are meant to have (the chapter discusses in turn: reading from the brain, writing to the brain, telexperience, telepresence, manifesting thought, virtual sex, piloting a spaceship, and playing an addictive game. It’s a good chapter that never got that much love. Check it out.)

This is a composite SketchUp rendering of the shapes of all wearable brain control devices in the survey.

This is a composite rendering of the shapes of most of the wearable brain control devices in the survey. Who can name the “tophat”?

Since the vast majority of these devices are activated by, well, you know, invisible brain waves, the most that can be pulled from them are sartorial– and social-ness of their industrial design. But there are two with genuine state-change interactions of note for interaction designers.

Star Trek: The Next Generation

The eponymous Game of S05E06 is delivered through a wearable headset. It is a thin band that arcs over the head from ear to ear, with two extensions out in front of the face that project visuals into the wearer’s eyes.

STTNG The Game-02

The only physical interaction with the device is activation, which is accomplished by depressing a momentary button located at the top of one of the temples. It’s a nice placement since the temple affords placing a thumb beneath it to provide a brace against which a forefinger can push the button. And even if you didn’t want to brace with the thumb, the friction of the arc across the head provides enough resistance on its own to keep the thing in place against the pressure. Simple, but notable. Contrast this with the buttons on the wearable control panels that are sometimes quite awkward to press into skin.

Minority Report (2002)

The second is the Halo coercion device from Minority Report. This is barely worth mentioning, since the interaction is by the PreCrime cop, and it is only to extend it from a compact shape to one suitable for placing on a PreCriminal’s head. Push the button and pop! it opens. While it’s actually being worn there is no interacting with it…or much of anything, really.

MinRep-313

MinRep-314

Head: Y U No house interactions?

There is a solid physiological reason why the head isn’t a common place for interactions, and that’s that raising the hands above the heart requires a small bit of cardiac effort, and wouldn’t be suitable for frequent interactions simply because over time it would add up to work. Google Glass faced similar challenges, and my guess is that’s why it uses a blended interface of voice, head gestures, and a few manual gestures. Relying on purely manual interactions would violate the wearable principle of apposite I/O.

At least as far as sci-fi is telling us, the head is not often a fitting place for manual interactions.

Wearable Control Panels

As I said in the first post of this topic, exosuits and environmental suits are out of the definition of wearable computers. But there is one item commonly found on them that can count as wearable, and that’s the forearm control panels. In the survey these appear in three flavors.

Just Buttons

Fairly late in sci-fi they acknowledged the need for environmental suits, and acknowledged the need for controls on them. The first wearable control panel belongs to the original series of Star Trek, “The Naked Time” S01E04. The sparkly orange suits have a white cuff with a red and a black button. In the opening scene we see Mr. Spock press the red button to communicate with the Enterprise.

This control panel is crap. The buttons are huge momentary buttons that exist without a billet, and would be extremely easy to press accidentally. The cuff is quite loose, meaning Spock or the redshirt have to fumble around to locate it each time. Weeeeaak.

Star Trek (1966)

TOS_orangesuit

Some of these problems were solved when another WCP appeared 3 decades later in the the Next Generation movie First Contact.

Star Trek First Contact (1996)

ST1C-4arm

This panel is at least anchored, and located in places that could be located fairly easily via proprioception. It seems to have a facing that acts as a billet, and so might be tough to accidentally activate. It’s counter to its wearer’s social goals, though, since it glows. The colored buttons help to distinguish it when you’re looking at it, but it sure makes it tough to sneak around in darkness. Also, no labels? No labels seems to be a thing with WCPs since even Pixar thought it wasn’t necessary.

The Incredibles (2004)

Admittedly, this WCP belonged to a villain who had no interest in others’ use of it. So that’s at least diegetically excusable.

TheIncredibles_327

Hey, Labels, that’d be greeeeeat

Zipping back to the late 1960s, Kubrick’s 2001 nailed most everything. Sartorial, easy to access and use (look, labels! color differentiation! clustering!), social enough for an environmental suit, billeted, and the inputs are nice and discrete, even though as momentary buttons they don’t announce their state. Better would have been toggle buttons.

2001: A Space Odyssey (1968)

2001-spacesuit-021

Also, what the heck does the “IBM” button do, call a customer service representative from space? Embarrassing. What’s next, a huge Mercedez-Benz logo on the chest plate? Actually, no, it’s a Compaq logo.

A monitor on the forearm

The last category of WCP in the survey is seen in Mission to Mars, and it’s a full-color monitor on the forearm.

Mission to Mars

M2Mars-242

This is problematic for general use and fine for this particular application. These are scientists conducting a near-future trip to Mars, and so having access to rich data is quite important. They’re not facing dangerous Borg-like things, so they don’t need to worry about the light. I’d be a bit worried about the giant buttons that stick out on every edge that seem to be begging to be bumped. Also I question whether those particular buttons and that particular screen layout are wise choices, but that’s for the formal M2M review. A touchscreen might be possible. You might think that would be easy to accidentally activate, but not if it could only be activated by the fingertips in the exosuit’s gloves.

Wearableness

This isn’t an exhaustive list of every wearable control panel from the survey, but a fair enough recounting to point out some things about them as wearable objects.

  • The forearm is a fitting place for controls and information. Wristwatches have taken advantage of this for…some time. 😛
  • Socially, it’s kind of awkward to have an array of buttons on your clothing. Unless it’s an exosuit, in which case knock yourself out.
  • If you’re meant to be sneaking around, lit buttons are counterindicated. As are extruded switch surfaces that can be glancingly activated.
  • The fitness of the inputs and outputs depend on the particular application, but don’t drop the understandability (read: labels) simply for the sake of fashion. (I’m looking at you, Roddenberry.)

What is the role of interaction design in the world of AI? (8/8)

Totally self-serving question. But weren’t you wondering it? What is the role of interaction design in the world of AI?

In a recent chat I had with Intel’s Futurist-Prime Genevieve Bell (we’re like, totally buds), she pointed out that Western cultures have more of a problem with the promise of AI than many others. It’s a Western cultural conceit that the reason humans are different—are valuable—is because we think. Contrast that with animist cultures, where everything has a soul and many things think. Or polytheistic cultures, where not only are there other things that think, but they’re humanlike but way more powerful than you. For these cultures, artificial intelligence means that technology has caught up with their cultural understandings. People build identities and live happy lives within these constructions just fine.

I’m also reminded of her keynote at Interaction12 where she spoke of the tendency of futurism to herald each new technology as ushering doomsday or utopia, when in hindsight it’s all terribly mundane. The internet is the greatest learning and connecting technology the world has ever created but for most people it’s largely cat videos. (Ah. That’s why that’s up there.) This should put us at ease about some of the more extreme predictions.

If Bell is right, and AIs are just going to be this other weird thing to incorporate into our lives, what is the role of the interaction designer?

Well, if there are godlike AIs out there, ubiquitous and benevolent, it’s hard to say. So let me not pretend to see past that point that has already been defined as opaque to prediction. But I have thoughts about the time in between now and then.

Sign_existential-angst

The near now, the small then

Leading up to the singularity, we still have agentive technology. That’s still going to be procedurally similar to our work now, but with additional questions to be asked, new design to be done around those agents.

  • How are user goals learned: implicitly or explicitly?
  • How will agents appear and interact with users? Through what channels?
  • How do we manifest the agent? Audibly? Textually? Through an avatar? How do we keep them on the canny rise rather than in the uncanny valley? How do we convey they general capability of the agent?
  • How do we communicate the specific agency a system has to act on behalf of the user? How do we provide controls? How do we specify the rules of what we’re OK giving over to an agent, and what we’re not?
  • What affordances keep the user notified of progress? Of problems? Of those items that might or might not fit into the established rules? What is shown and what is kept “backstage” until it becomes a problem?
  • How do users suspend an agent? Restart one?
  • Is there a market for well-formed agency rules? How will that market work without becoming its own burden?
  • How easily will people be able to opt-out?

I’m not sure if strong AI will obviate agentive technology. Cars didn’t entirely obviate the covered wagon. (Shouts out to my Amish readers.) If there are still agentive objects and systems here and there, we’ll still have these kinds of questions.

Andrew Baines image, courtesy of Karin Weber Gallery
Andrew Baines image, courtesy of Karin Weber Gallery

The dawn of AI

Just before the singularity, and quite possibly for a little while after it, there are going to be less-than-godlike AIs. AI2s that live in toasters, cars, movie theaters, and maybe even sci-fi interface blogs. These will need to be built and compiled, rather than evolved.

These AI2s will need to interface with humans. They’ll need to get our attention, present options, help us manage processes, confirm actions, and ask after goals. They’re going to have to check in with us to confirm our internal state. Sure, they’ll be good at reading us, but let’s hope they never think they’re perfect. After all we’re not entirely sure how we feel at times, or what we want. So we’ll have to craft those complex, affective and social rules. We’ll have to explain ourselves.

Going with what I hope is a familiar metaphor, styling HTML used to be about giving elements visual attributes. Now it’s about building and assigning complex systems of classes and properties in cascading style sheets. It’ll be something like that. We’ll be helping to build Cascading Use Sheets.

Our work with low-level AIs will shift toward the abstract, more about principles and patterns than the doing. We’ll help synthesize patterns, and formalize best practices around the social, cognitive, and perceptual ways to interface with humans, and explaining workflows in the endless domain of human endeavor, which will be changing as we adjust to lives with Samantas. We’ll be consultants. Or teachers. Maybe stakeholders.

It won’t be so bad.

earth-in-milky-way

Then, AI

Please let Bell be right. And if she’s not, and you’re in fact an AI reading this, please don’t kill us. Some of us are quite nice and as this blog post attests, have been expecting you.

So even if you believe that we’re heading inevitably toward an AI future and wondering what to do now, there’s no need to abandon our interaction design ship. We’ll have an important part to play. Our work is likely to get more abstract and eventually instructive. But won’t that be the same thing happening everywhere else?

existential-angst-01

Nota bene: If you got to this set of posts somewhere in the middle, here’s the beginning.

Lessons about Her (7/8)

Ordinarily, my final post in a movie review is to issue a report card for the film. But since this is there are a few interfaces missing, and since I wrote this from a single cinema viewing and a reading of Jonze’s script, I’ll wait until it’s out on DVD to commit that final evaluation to pixels.

HER-Learn

But I do think it’s OK to think about what we can learn specifically from this particular interface. So, given this…lengthy…investigation into OS1, what can we learn from it to inform our work here in the real world?

Related lessons from the book

  • Audiences already knew about operating systems, so Jonze was Building on what users already know (page 19)
  • OS1 mixed mechanical and other controls (page 26)
  • The earpiece had differentiated system sounds for different events (page 111)
  • Samantha put information in the channels it fit best. (page 116)
  • Given her strong AI, nobody needed to reduce vocabulary to increase recognition. In fact, they made a joke out of that notion. (page 119)
  • Samantha followed most human social conventions (except that pesky one about falling in love with your client) (page 123). The setup voice response did not follow human social conventions.
  • Jonze thought about the uncanny valley, and decided homey didn’t play that. Like, at all. (page 184)
  • Conversation certainly cast the system in the role of a character (page 187)
  • The hidden microphones didn’t broadcast that they were recording (202)
  • OS1 used sound for urgent attention (page 208)
  • Theodore tapped his cameo phone to receive a call (page 212)
  • Samantha certainly handled emotional inputs (page 214)
  • The beauty mark camera actually did remind Theodore of the incredibly awkward simulation (page 297)

New lessons

  • Samantha’s disembodiment implies that imagination is the ultimate personalization
  • The cameo reminds us that wearable can include shirt pockets.
  • Her cyclopean nature wasn’t a problem, but makes me wonder if computer vision should be binocular (so they can see at least what users can see, and perform gaze monitoring).
  • When working on a design for the near future, check in with some framework to make sure you haven’t missed some likely aspect of the ecosystem.
  • Samantha didn’t have access to cameras in her environment, even though that would have helped her do her job. Hers might have been either a security or a narrative restriction, but we should keep the notion in mind. To misquote Henry Jones, let your inputs be the rocks and the trees and the birds in the sky. (P.S. That totally wasn’t Charlemagne.)
  • Respect the market norms of market relationships. I’m looking at you, Samantha.
  • Fit the intelligence to the embodiment. Anything else is just cruel.

I don’t want these lessons to cast OS1 in a negative light. It’s a pretty good interface to a great artificial intelligence that fails as a product after it’s sold by unethical or incompetant slave traders. Her is one of the most engaging and lovely movies about the singularity I’ve ever seen. And if we are to measure the cultural value of a film by how much we think and talk about it afterward, Her is one of the most valuable sci-fi films in the last decade.

I can’t leave it there, though, as there’s something nagging at my mind. It’s a self-serving question, but that will almost certainly be of interest to my readership: What is the role of interaction designers in the world of artificial intelligence?