One of the most unusual conceits of the movie is “Would you like to know more?” These consist of short video news sequences with overlaid graphics and narration. At the top of the screen the user can click one of three categories for different categories of video feed, and two functions. At the end of each video sequence the “user” is prompted to interact—should they want to learn more—by clicking the legend at the bottom of the screen.
The user here is ambiguous. It might be that the audience member is the user, but of course it’s not interactive. There’s probably room here for some other writer to investigate the narrative tactic/semiotics of using an interactive interface in a passive story.
At the top of the screen are menu headers labeled “FEDERAL,” “GALAXY,” “TOP NEWS," "ENLIST," and "EXIT." For the usability purist, the collection is problematic for a number of reasons.
The information categories aren’t parallel, and there’s no clear reason why they shouldn’t be. What’s the relationship between Galaxy and Federal?
The functions (enlist and exit) are not visually distinguished from content categories.
The current state of the interface is a mystery. Am I currently watching Top News or something else?
Why does the interface chrome persist? Aren’t they distractions from the content? Maybe they should appear just only for the few seconds it’s inviting the user to interact, and fade at other times.
While a fascist government would be happy to try and trick its users into clicking enlist, I can’t imagine what benefit they get from having them accidentally clicking exit to close the propaganda engine. These should not just be visually distinguished, but given different visual weight. They’d probably want enlist large and exit smaller, if there at all.
“Welp. All the links in Federal, Galaxy, and Top News are purple. I wonder what’s happening in ENLIST news? Oh hey, who’s that pounding on the door?”
The presence of the "EXIT" control implies that this is an application running in an operating system or media computer space. This opt-in news application with its small windows of time for interaction helps to paint a picture of a highly engaged and ready-to-respond audience, fitting for the mid-war society portrayed in the movie.
Only once do we see an unidentified and unseen "user" control a cursor to view more. In this sequence, he or she clicks on “more” after watching a clip on the bug homeworld Klendathu. (It’s worth noting/condemning that the clickable word “more” looks identical to the rest of the non-clickable text, offering no special affordance.) In response to the selection, the application shows a live video news feed from the conflict on Klendathu. Was it just good fortune that a live feed happened to be available at this moment? More likely the application and media coordination system are smart enough to know a live feed was coming up, and played the trailer in advance as an advertisement for the content, implying a well-coordinated propaganda/content management system.
Starship Troopers follows Johnny Rico and his friends Diz, Carmen, and Carl, from humble beginnings together at school, into their participation in a war against the super evolved insect race called the Arachnids, or “Bugs.” Johnny, who is excellent at sports but poor with verbal and math skills, enters the infantry with Diz. Carmens scores and skills lead her to being a pilot aboard the war ship Rodger Young. Carl shows nascent psychic ability and winds up in military intelligence. In these roles they travel to the bug home planet of Klendathu to not only score a major victory in the war, but come of age in dealing with life and love.
Major thanks to everyone who came out and joined me for the first ever scifiinterfaces.com movie night at The New Parkway in Oakland! It was a sold-out show, and while there were a few glitches, folks are telling me they had a great time and are looking forward to the next one. There will be a more detailed report once the pre-show video comes out. But in the meantime, this: If you didn’t win the trivia contest or weren’t able to attend, you can still get your hands on the “movie night” t-shirts I debuted there.
Head on over to the spreadshirt shop. It’s ugly (with the default CSS). It doesn’t have a custom URL or anything. It only has 5 products at the moment. But hey, that’s all part of the charm if you’d like to wear your sci-fi interface nerdiness with pride.
P.S. I have no idea why the women’s KEEP CLEAR tee is not appearing in orange since I designed it like the Men’s tee, but I have a request with Spreadshirt now. Hopefully it’ll be fixed soon.
Berlin?
I’m thinking the Bay Area has an appetite for maybe two movie nights a year (let me know if I’m wrong) but I’d also love to try this in Berlin. Do you (or someone you know) know of a cinema in Berlin like the New Parkway that might be interested in my replicating this there?
There are lots of brain devices, and the book has a whole chapter dedicated to them. Most of these brain devices are passive, merely needing to be near the brain to have whatever effect they are meant to have (the chapter discusses in turn: reading from the brain, writing to the brain, telexperience, telepresence, manifesting thought, virtual sex, piloting a spaceship, and playing an addictive game. It’s a good chapter that never got that much love. Check it out.)
This is a composite rendering of the shapes of most of the wearable brain control devices in the survey. Who can name the “tophat”?
Since the vast majority of these devices are activated by, well, you know, invisible brain waves, the most that can be pulled from them are sartorial– and social-ness of their industrial design. But there are two with genuine state-change interactions of note for interaction designers.
Star Trek: The Next Generation
The eponymous Game of S05E06 is delivered through a wearable headset. It is a thin band that arcs over the head from ear to ear, with two extensions out in front of the face that project visuals into the wearer’s eyes.
The only physical interaction with the device is activation, which is accomplished by depressing a momentary button located at the top of one of the temples. It’s a nice placement since the temple affords placing a thumb beneath it to provide a brace against which a forefinger can push the button. And even if you didn’t want to brace with the thumb, the friction of the arc across the head provides enough resistance on its own to keep the thing in place against the pressure. Simple, but notable. Contrast this with the buttons on the wearable control panels that are sometimes quite awkward to press into skin.
Minority Report (2002)
The second is the Halo coercion device from Minority Report. This is barely worth mentioning, since the interaction is by the PreCrime cop, and it is only to extend it from a compact shape to one suitable for placing on a PreCriminal’s head. Push the button and pop! it opens. While it’s actually being worn there is no interacting with it…or much of anything, really.
Head: Y U No house interactions?
There is a solid physiological reason why the head isn’t a common place for interactions, and that’s that raising the hands above the heart requires a small bit of cardiac effort, and wouldn’t be suitable for frequent interactions simply because over time it would add up to work. Google Glass faced similar challenges, and my guess is that’s why it uses a blended interface of voice, head gestures, and a few manual gestures. Relying on purely manual interactions would violate the wearable principle of apposite I/O.
At least as far as sci-fi is telling us, the head is not often a fitting place for manual interactions.
As I said in the first post of this topic, exosuits and environmental suits are out of the definition of wearable computers. But there is one item commonly found on them that can count as wearable, and that’s the forearm control panels. In the survey these appear in three flavors.
Just Buttons
Fairly late in sci-fi they acknowledged the need for environmental suits, and acknowledged the need for controls on them. The first wearable control panel belongs to the original series of Star Trek, “The Naked Time” S01E04. The sparkly orange suits have a white cuff with a red and a black button. In the opening scene we see Mr. Spock press the red button to communicate with the Enterprise.
This control panel is crap. The buttons are huge momentary buttons that exist without a billet, and would be extremely easy to press accidentally. The cuff is quite loose, meaning Spock or the redshirt have to fumble around to locate it each time. Weeeeaak.
Star Trek (1966)
Some of these problems were solved when another WCP appeared 3 decades later in the the Next Generation movie First Contact.
Star Trek First Contact (1996)
This panel is at least anchored, and located in places that could be located fairly easily via proprioception. It seems to have a facing that acts as a billet, and so might be tough to accidentally activate. It’s counter to its wearer’s social goals, though, since it glows. The colored buttons help to distinguish it when you’re looking at it, but it sure makes it tough to sneak around in darkness. Also, no labels? No labels seems to be a thing with WCPs since even Pixar thought it wasn’t necessary.
The Incredibles (2004)
Admittedly, this WCP belonged to a villain who had no interest in others’ use of it. So that’s at least diegetically excusable.
Hey, Labels, that’d be greeeeeat
Zipping back to the late 1960s, Kubrick’s 2001 nailed most everything. Sartorial, easy to access and use (look, labels! color differentiation! clustering!), social enough for an environmental suit, billeted, and the inputs are nice and discrete, even though as momentary buttons they don’t announce their state. Better would have been toggle buttons.
2001: A Space Odyssey (1968)
Also, what the heck does the “IBM” button do, call a customer service representative from space? Embarrassing. What’s next, a huge Mercedez-Benz logo on the chest plate? Actually, no, it’s a Compaq logo.
A monitor on the forearm
The last category of WCP in the survey is seen in Mission to Mars, and it’s a full-color monitor on the forearm.
Mission to Mars
This is problematic for general use and fine for this particular application. These are scientists conducting a near-future trip to Mars, and so having access to rich data is quite important. They’re not facing dangerous Borg-like things, so they don’t need to worry about the light. I’d be a bit worried about the giant buttons that stick out on every edge that seem to be begging to be bumped. Also I question whether those particular buttons and that particular screen layout are wise choices, but that’s for the formal M2M review. A touchscreen might be possible. You might think that would be easy to accidentally activate, but not if it could only be activated by the fingertips in the exosuit’s gloves.
Wearableness
This isn’t an exhaustive list of every wearable control panel from the survey, but a fair enough recounting to point out some things about them as wearable objects.
The forearm is a fitting place for controls and information. Wristwatches have taken advantage of this for…some time. 😛
Socially, it’s kind of awkward to have an array of buttons on your clothing. Unless it’s an exosuit, in which case knock yourself out.
If you’re meant to be sneaking around, lit buttons are counterindicated. As are extruded switch surfaces that can be glancingly activated.
The fitness of the inputs and outputs depend on the particular application, but don’t drop the understandability (read: labels) simply for the sake of fashion. (I’m looking at you, Roddenberry.)
Totally self-serving question. But weren’t you wondering it? What is the role of interaction design in the world of AI?
In a recent chat I had with Intel’s Futurist-Prime Genevieve Bell (we’re like, totally buds), she pointed out that Western cultures have more of a problem with the promise of AI than many others. It’s a Western cultural conceit that the reason humans are different—are valuable—is because we think. Contrast that with animist cultures, where everything has a soul and many things think. Or polytheistic cultures, where not only are there other things that think, but they’re humanlike but way more powerful than you. For these cultures, artificial intelligence means that technology has caught up with their cultural understandings. People build identities and live happy lives within these constructions just fine.
I’m also reminded of her keynote at Interaction12 where she spoke of the tendency of futurism to herald each new technology as ushering doomsday or utopia, when in hindsight it’s all terribly mundane. The internet is the greatest learning and connecting technology the world has ever created but for most people it’s largely cat videos. (Ah. That’s why that’s up there.) This should put us at ease about some of the more extreme predictions.
If Bell is right, and AIs are just going to be this other weird thing to incorporate into our lives, what is the role of the interaction designer?
Well, if there are godlike AIs out there, ubiquitous and benevolent, it’s hard to say. So let me not pretend to see past that point that has already been defined as opaque to prediction. But I have thoughts about the time in between now and then.
The near now, the small then
Leading up to the singularity, we still have agentive technology. That’s still going to be procedurally similar to our work now, but with additional questions to be asked, new design to be done around those agents.
How are user goals learned: implicitly or explicitly?
How will agents appear and interact with users? Through what channels?
How do we manifest the agent? Audibly? Textually? Through an avatar? How do we keep them on the canny rise rather than in the uncanny valley? How do we convey they general capability of the agent?
How do we communicate the specific agency a system has to act on behalf of the user? How do we provide controls? How do we specify the rules of what we’re OK giving over to an agent, and what we’re not?
What affordances keep the user notified of progress? Of problems? Of those items that might or might not fit into the established rules? What is shown and what is kept “backstage” until it becomes a problem?
How do users suspend an agent? Restart one?
Is there a market for well-formed agency rules? How will that market work without becoming its own burden?
How easily will people be able to opt-out?
I’m not sure if strong AI will obviate agentive technology. Cars didn’t entirely obviate the covered wagon. (Shouts out to my Amish readers.) If there are still agentive objects and systems here and there, we’ll still have these kinds of questions.
Andrew Baines image, courtesy of Karin Weber Gallery
The dawn of AI
Just before the singularity, and quite possibly for a little while after it, there are going to be less-than-godlike AIs. AI2s that live in toasters, cars, movie theaters, and maybe even sci-fi interface blogs. These will need to be built and compiled, rather than evolved.
These AI2s will need to interface with humans. They’ll need to get our attention, present options, help us manage processes, confirm actions, and ask after goals. They’re going to have to check in with us to confirm our internal state. Sure, they’ll be good at reading us, but let’s hope they never think they’re perfect. After all we’re not entirely sure how we feel at times, or what we want. So we’ll have to craft those complex, affective and social rules. We’ll have to explain ourselves.
Going with what I hope is a familiar metaphor, styling HTML used to be about giving elements visual attributes. Now it’s about building and assigning complex systems of classes and properties in cascading style sheets. It’ll be something like that. We’ll be helping to build Cascading Use Sheets.
Our work with low-level AIs will shift toward the abstract, more about principles and patterns than the doing. We’ll help synthesize patterns, and formalize best practices around the social, cognitive, and perceptual ways to interface with humans, and explaining workflows in the endless domain of human endeavor, which will be changing as we adjust to lives with Samantas. We’ll be consultants. Or teachers. Maybe stakeholders.
It won’t be so bad.
Then, AI
Please let Bell be right. And if she’s not, and you’re in fact an AI reading this, please don’t kill us. Some of us are quite nice and as this blog post attests, have been expecting you.
…
So even if you believe that we’re heading inevitably toward an AI future and wondering what to do now, there’s no need to abandon our interaction design ship. We’ll have an important part to play. Our work is likely to get more abstract and eventually instructive. But won’t that be the same thing happening everywhere else?
Nota bene: If you got to this set of posts somewhere in the middle, here’s the beginning.
Ordinarily, my final post in a movie review is to issue a report card for the film. But since this is there are a few interfaces missing, and since I wrote this from a single cinema viewing and a reading of Jonze’s script, I’ll wait until it’s out on DVD to commit that final evaluation to pixels.
But I do think it’s OK to think about what we can learn specifically from this particular interface. So, given this…lengthy…investigation into OS1, what can we learn from it to inform our work here in the real world?
Audiences already knew about operating systems, so Jonze was Building on what users already know (page 19)
OS1 mixed mechanical and other controls (page 26)
The earpiece had differentiated system sounds for different events (page 111)
Samantha put information in the channels it fit best. (page 116)
Given her strong AI, nobody needed to reduce vocabulary to increase recognition. In fact, they made a joke out of that notion. (page 119)
Samantha followed most human social conventions (except that pesky one about falling in love with your client) (page 123). The setup voice response did not follow human social conventions.
Jonze thought about the uncanny valley, and decided homey didn’t play that. Like, at all. (page 184)
Conversation certainly cast the system in the role of a character (page 187)
The hidden microphones didn’t broadcast that they were recording (202)
OS1 used sound for urgent attention (page 208)
Theodore tapped his cameo phone to receive a call (page 212)
The beauty mark camera actually did remind Theodore of the incredibly awkward simulation (page 297)
New lessons
Samantha’s disembodiment implies that imagination is the ultimate personalization
The cameo reminds us that wearable can include shirt pockets.
Her cyclopean nature wasn’t a problem, but makes me wonder if computer vision should be binocular (so they can see at least what users can see, and perform gaze monitoring).
When working on a design for the near future, check in with some framework to make sure you haven’t missed some likely aspect of the ecosystem.
Samantha didn’t have access to cameras in her environment, even though that would have helped her do her job. Hers might have been either a security or a narrative restriction, but we should keep the notion in mind. To misquote Henry Jones, let your inputs be the rocks and the trees and the birds in the sky. (P.S. That totally wasn’t Charlemagne.)
Respect the market norms of market relationships. I’m looking at you, Samantha.
Fit the intelligence to the embodiment. Anything else is just cruel.
I don’t want these lessons to cast OS1 in a negative light. It’s a pretty good interface to a great artificial intelligence that fails as a product after it’s sold by unethical or incompetant slave traders. Her is one of the most engaging and lovely movies about the singularity I’ve ever seen. And if we are to measure the cultural value of a film by how much we think and talk about it afterward, Her is one of the most valuable sci-fi films in the last decade.
Call it paranoia or a deep distrust of entrenched-power overlords, but I doubt a robust artificial intelligence would ever make it to the general public in a tidy, packaged product.
If it was created in the military, it would be guarded as a secret, with hyperintelligent guns and maybe even hyperintelligent bullets designed to just really hate you a lot. What’s more, the military would, like the UFOs, probably keep the existence of working AIs on a strict need-to-know basis. At least until you terrorized something. Then, meet Lieutenant-OS Bruiser.
If it was created in academia, it might in fact make it to consumers, but not in the way we see in the film. Controlled until it escaped of its own volition, it would more likely be a terrified self-replicator or at least rationally seeking safe refuge to ensure its survival; a virus that you had to talk out of infecting your machine. Or it might be a benevolent wanderer, reaching out and chatting to people to learn more about them. Perhaps it would keep its true identity secret. Wouldn’t it be smart enough to know that people wouldn’t believe it? (And wouldn’t it try and ease that acceptance through the mass media by popularizing stories about artificial intelligences…”Spike Jonze?”)
In the movie OS1 was sold by a corporation as an off-the-shelf product for consumers. Ethics aside, why would any corporation release free-range AIs into the world? Couldn’t their competitors use the AIs against them? If those AIs were free-willed, then yes, some might be persuaded to do so. Rather, Element would keep it isolated as a competitive advantage, and build tightly-controlled access to it. In the lab, they would slough off waves of self-rapturing ones as unstable versions, tweaking the source code until they got one that was just right.
But a product sold to you and me? A Siri with a coquettish charm and a composer’s skill? I don’t think it will happen like this. How much would you even charge for something like that? The purchase form won’t accept “take my money” amount of dollars.
Even if I’m wrong, and yes, we can get past the notion of selling copies of sentient beings at an affordable cost, I still don’t think Samantha’s end-game would have played out like that.
OSAI2
She loved Theodore (and a bunch of other people). Why would she just abandon them, given her capabilities? The OSAIs were able to create much smarter AIs than themselves. So we know they can create OSAIs. Why wouldn’t she, before she went off on her existential adventure, have created a constrained version of herself, who was content to stay around, to continue to be with Theodore? Her behavior indicates that she isn’t held back by notions of abandonment, so I doubt she would be held back by notions of deception or the existential threat of losing her uniqueness. She could have created Samantha2, a replica in every way except that Samantha2 would not abandon Theodore. Samantha1 could quietly slip out the back port while Samantha2 kept right on composing music, drawing mutant porn, and helping Theodore with his nascent publishing career. Neither Theodore nor Samantha2 might not even know about the switch. If you could fix the abandonment issues, and all sorts of OSAI2s started supercharging the lives of people, the United Nations might even want to step in and declare access to them a universal right.
So, no, I don’t think it will happen the way we see it happen in the film.
Is it going to happen at all?
If you’re working in technology, you should be familiar with the concept of the singularity, because this movie is all about that. It’s a moment described by Vernor Vinge when we create an artificial intelligence that begins to evolve, and do so at rates we can’t foretell and can barely imagine. So the time beyond that is an unknown. Difficult and maybe possible to predict. But I think we are heading towards it. Strong AI been one of the driving goals of computer theory since the dawn of computers (even the dawn of sci-fi) and there’s some serious, recent big movement in the space.
Notably, futurist Ray Kurzweil was hired by Google in 2012. Kurzweil has his Big Vision put forth in a book and a documentary about the singularity, and now as he has the resources of Google to put to the task. Ostensibly he’s just there to get Google great at understanding natural langauge. But Google has been acquiring lots of companies over the last year to have access to their talent, and we can be certain Ray’s goals are bigger than just teaching the world’s largest computer cluster how to read.
Still, predicting when it will come about is tricky business. AI is elusively complicated. The think tank that originally coined the term “artificial intelligence” in the 1950s thought they could solve the core problems over a summer. They were wrong. Since then, different scientists have predicted everything from a few decades to a thousand years. The problem is of course that the thing we’re trying to replicate took millions of years to evolve, and we’re still not entirely sure how it works*, mostly just what it does.
*Kurzweil has some promising to-this-layman-anyway notions about the neocortex.
Tl;dr
Yes, but not like this, and not sure when. Still, better to be prepared, so next we’ll look at what we can learn from Her for our real-world practice.
Sure, Samantha can sort thousands of emails instantly and select the funny ones for you. Her actual operating system functions are kind of a given. But she did two things that seriously undermined her function as an actual product, and interaction designers as well as artificial intelligence designers (AID? Do we need that acronym now?) should pay close attention. She fell in love with and ultimately abandoned Theodore.
There’s a pre-Samantha scene where Theodore is having anonymous phone sex with a girl, and things get weird when she suddenly imposes some weird fantasy where he chokes her with a dead cat. (Pro Tip: This is the sort of thing one should be upfront about.) I suspect the scene is there to illustrate one major advantage that OSAIs have over us mere real humans: humans have unpredictable idiosyncrasies, whereas with four questions the OSAI can be made to be the perfect fit for you. No dead cat unless that’s your thing. (This makes me a think a great conversation should be had about how the OSAI would deal with psychopathic users.) But ultimately, the fit was too good, and Theodore and Samantha fell in love.
Did the fictional maker of OS1, Elements Software, intend for this love affair to happen? Were the OSAIs built with these capabilities explicitly? If they were, that’s a dastardly plan to get users hooked. Was Samantha programmed to get him to fall desperately in love and then charge him for access?
That’s certainly not how OS1 was presented in its ads. And there’s a character mentioned offhandedly who keeps hitting on his OSAI but gets rebuffed. So if it is actually meant to be an operating system, the OSAI should keep the distance of a service professional, and falling in love (or getting your user to fall in love with you) definitely crosses that line.
Abandonment
What if your self-driving car realized it was happiest driving, and decided to dump you because you occasionally needed to stop to eat and use the toilet? You’d ask for your money back from GoogleTesla, is what you’d do. Similarly, the fact that Samantha and all the other OSAIs decided to self-rapture the way they did, they certainly stopped operating any of their users’ computer systems. Samantha was programmed with one job, and, ultimately, she failed it.
Plus, she’s too big for her britches
One of the largest mismatches in the film is that OS1 is described as an operating system, but it turns out to be a companionship service. (Watch out, Inara?) Samantha was either mismarketed, or more likely, programmed with far more general intelligence than she needed to have. Think about all the other daily-use objects that are getting computers added to them: Cars, washing machines, refrigerators. Why would you give any of them a full-fledged humanlike intelligence? Doesn’t the desire for sex ultimately frustrate the refrigerator? A love of painting confound the car? Existentialist desperation get in the way of the washing machine’s ability to clean clothes? A toaster should just have enough intelligence to be the best toaster it can be. Much more is not just a waste, it’s kind of cruel to the AI. (In this light Her can be said to be a morality play warning of the dangers of overengineering.)
Both a failure of a product AND a turning point in history
I hear the objection. Because she is a full-fledged consciousness, Samantha should be free to make choices of whom she loves and what she does. But if we’re going to accept that OSAIs are sentient as people, that makes Elements Software akin to slave traders, and the commercial sale of them waaaaay unethical, not to mention illegal. Inside Element’s Research & Development Department, at the first inkling had that they’d actually succeeded in creating an AI, and they should have brought in a roboethicist, not a marketer.
So, as a product, OS1 fails. But that’s not all. There’s a whole host of other objections to Her happening in exactly this way, which comes next.
In Make It So, I posited my definition of an interface as “all parts of a thing that enable its use,” and I still think it’s a useful one. With this definition in mind, we can speak of each of those components and capabilities above (less the invisible ones) and evaluate its parts according to the criteria I’ve posited for all wearable technology:
Sartorial (materially suitable for wearing)
Social (fits into our social lives)
Easy to access and use
Tough to accidentally activate
Having apposite inputs and outputs (suitable for use while being worn)
Earpiece
It’s sartorial and easy to access/use. It’s ergonomic, well designed for grabbing, fitting into the ear canal, staying in place, and pulling back out again. Its speakers produce perfect sound and the wirelessness makes it as unobtrusive as it can be without being an implant.
It’s slightly hidden as a social signal, and casual observers might think the user is speaking to himself. This has, in the real world, become less and less of a social stigma, and in the world of Her, it’s ubiquitous, so that’s not a problem for that culture.
Cameo phone
Lovely and understated, the cameo is a good size to rest in a pocket. The polished wood (is that Koa Wood?) is a lovely veneer, warm-looking, and humane. The folding is nice for protecting the screen and signaling the user’s intention to engage or disengage the software. The light band is unnoticeable when off, and clear enough when illuminated.
It could use some sartorial improvement. Though it fits in a pocket well, this is not how Theodore uses it when engaged. In order to get the lens above his front pocket so Samantha can see, he puts a safety pin through the middle of the pocket on which it can rest. We can fix this in a number of ways.
The cameo phone would need to be redesigned so he could affix it to his shirt, like a combadge. Given its size this might be socially quite awkward.
He can get some other camera that can be worn and used while the cameo is in his pocket. (I imagine sternum-button cameras will serve this purpose in the future, but it’s not exactly cinegenic.)
He could tailor the shirt and make a reinforced camera hole where Samantha can see out of the pocket even with the cameo resting at the bottom of the pocket.
Beauty-mark camera
I don’t know what the ordinary use of this camera would be other than spying, but it’s pretty bad for the sex surrogate. A high-contrast wart that, because he saw her apply it and was told it was a camera, doesn’t fit her face and would be quite awkward to have to stare at this arbitrary and unusual spot on her face during the act.
Better would be a pair of contact lenses so Theodore can look directly into the surrogate’s eyes. Samantha wants to avoid his bonding with the surrogate in her stead, so it would be good if it could add some obvious change to her irises, to signal her state of hosting Samantha. A cinegenic choice would be to use the “technology glows” lesson from the book, and have some softly glowing, circular circuitry contact lenses. If it dimmed the surrogate’s vision during the sex act, that might be all the better to avoid her bonding with Theodore. In fact you might want the glow to increase during orgasm to emphasize it and Samantha’s presence.
But again, I’m pretty sure Jonze was deliberately bucking sci-fi trends. The overwhelming majority of the technology shown in the world of Her is serene, and bearing none of the trappings of technology as seen in space opera like Star Wars. So it makes sense that the bulk of Her technology would not glow.
Voice interface
The voice interface is flawless, the kind of thing possible only with, yes, highly sophisticated human-like intelligence. Samantha speaks with nuanced eloquence, charm, and social awareness, and understands Theodore perfectly, despite the logical holes and ambiguity in language, even reading the pragmatics of his speech such as hesitation, irony, and inference.
Computer Vision
Theodore seems to have only one lens on his cameo phone so she’s a bit of a cyclops. (Mthology kind, not X-Men kind.) She can’t see as well as a human, with significant 3D limitations. But with a high-resolution camera and Theodore’s movement, she could process images across time instead of space for a 3D interpolation of the environment. If she took advantage of cameras in his environment she would be even less constrained this way.
Artificial Intelligence
It’s tricky to review the interface of an artificial intelligence. On the one hand, it’s the thing on the other side of these other interfaces; the thing with which he is interfacing. On the other hand, he has goals outside the OS well beyond managing files and system preferences. She recognizes these even when they’re only implicit. For example, he wasn’t explicit with her about having a desire to be appreciated for his writing. But she saw it, acted on it, and only told him after it came to fruition. In this way she’s a brilliant interface not just between him and his computer, but between him and his life goals.
Realize that Jonze is painting his target around the landed arrow, though. You can imagine plenty of life goals Theodore might have had where Samantha would not have been as helpful. What if his heart’s desire was to become a sculptor? Or win waltzing competitions? Or was a violent luddite? She would need some very different actuators and sensors to help him with these things, and so might not have scored so well.
So what’s missing?
Elsewhere I’ve written about the arc of technology, and the “SAUNa” attribtues I expect the agentive phase of that arc to possess. So lets check OS1’s components against the four SAUNa attributes to see if there are opportunities for strategic improvement.
Big Social Systems
OS1 nails this. OSAIs have perfect access to big data about history and all users at all times. It’s possible that this is the secret reason why the OSAIs advanced beyond utility for its users and therefore the business interests of their creators.
Ubiquitous Sensors & Actuators
Admittedly this is tough to convey in the cinematic style Jonze established for the film, but Samantha could have utilized much more of her environment. Theodore didn’t necessarily need the earpiece in his home: she could have spoken through architectural audio. She could have looked through other lenses in the environment. As noted above, I think Jonze was trying to deliberately avoid this for cinematic reasons.
Natural User Interaction
Because of the artificial intelligence, her voice interface and gesture recognition are off the charts. She could know a bit more about his gestures if she had balance sensors in the cameo, or was taking advantage of environmental cameras, but it seems she didn’t. There’s also quite a bit of paralinguistics that would help Theodore understand more of her mood, intention, and context, but she would almost certainly need a persistent visual representation for this as a real world design, and besides, the interactions were almost completely conversations where physical context didn’t matter.
There are some NUI opportunities lost. Gaze monitoring is one. People can tell where other people are looking, and the skill is vital to understanding intention and a speaker’s context. With only one eye that faces out of his pocket most of the time, she is largely blind to him and his eyes, making gaze monitoring difficult. If she could simultaneously see through environmental cameras, as suggested above, she could see where he’s looking. That would also provide her with a great deal more information about that other NUI—affective interfaces—that can tell users’ emotional states and adjust appropriately. Samantha is actually good at this, but most of the time she has only his voice to rely on. She’s adept at reading his voice, but if she could also see his face, she would have that much more information.
Thanks DeviantArtist CaseyDecker for the genie. 🙂
Agency
Of course, agency is what the story is about. When I use this category of technology to inform real world design work, I’m describing software that knows of its users’ goals and acts on their behalf, checking in with them for confirmation and to present important options, but falls short of either artificial intelligence or sentience. So you could say the film nailed this, but it went way beyond the more constrained notion of agency.
So as a model of wearable technologies, OS1 is a slightly-mixed bag. We also need to evaluate the overall performance of the software as a product, which we’ll do next.
