The following is the draft of a section from my forthcoming book, The City Is Here For You To Use, concerning various ways in which networked devices are used to furnish the mobile pedestrian with a layer of location-specific information superimposed onto the forward view — “augmented reality,” in other words. (The context is an extended discussion of four modes in which information is returned from the global network to the world so it may be engaged, considered and acted upon, which is why the bit here starts in medias res.)
As you see it here, the section is not quite in its final form; it hasn’t yet been edited for meter, euphony or flow, and in particular, some of the arguments toward the end remain too telescoped to really stand up to much inspection. Nevertheless, given the speed at which wearable AR is evolving, I thought it would be better to get this out now as-is, to garner your comments and be strengthened by them. I hope you enjoy it.
One seemingly potent way of returning networked information to the world would be if we could layer it directly over that which we perceive. This is the premise of so-called augmented reality, or AR, which proposes to furnish users with some order of knowledge about the world and the objects in it, via an overlay of informational graphics superimposed on the visual field. In principle, this augmentation is agnostic as to the mediating artifact involved, which could be the screen of a phone or tablet, a vehicle’s windshield, or, as Google’s Glass suggests, a lightweight, face-mounted reticle.
AR has its conceptual roots in informational displays developed for military pilots in the early 1960s, at the point when the performance of enemy fighter aircraft began to overwhelm a human pilot’s ability to react. In the fraught regime of jet-age dogfighting, even a momentary dip of the eyes to a dashboard-mounted instrument cluster could mean disaster. The solution was to project information about altitude, airspeed and the status of weapons and other critical aircraft systems onto a transparent pane aligned with the field of vision, a “head-up display.”
This notion turned to have applicability in fields beyond aerial combat, where the issue wasn’t so much reaction time as it was visual complexity. One early AR system was intended to help engineers make sense of the gutty tangle of hydraulic lines, wiring and control mechanisms in the fuselage of an airliner under construction; each component in the otherwise-hopeless confusion was overlaid with a visual tag identifying it by name, and colored according to the system it belonged to.
Other systems were designed to help people manage situations in which both time and the complexity of the environment were sources of pressure — for example, to aid first responders in dispelling the fog and chaos they’re confronted with upon arrival at the scene of an emergency. One prototype furnished firefighters with visors onto which structural diagrams of a burning building were projected, along with symbols indicating egress routes, the position of other emergency personnel, and the presence of electric wiring or other potentially dangerous infrastructural elements.
The necessity of integrating what were then relatively crude and heavy cameras, motion sensors and projectors into a comfortably wearable package limited the success of these early efforts — and this is to say nothing of the challenges posed by the difficulty of establishing a reliable network connection to a mobile unit. But the conceptual heavy lifting done to support these initial forays produced a readymade discourse, waiting for the day augmentation might be reinstantiated in smaller, lighter, more capable hardware.
That is a point we appear to have arrived at with the advent of the smartphone. As we’ve seen, the smartphone handset can be thought of as a lamination together of several different sensing and presentation technologies, subsets of which can be recombined with one another to produce distinctly different ways of engaging networked information. Bundle a camera, accelerometer/gyroscope, and display screen in a single networked handset, and what you have in your hands is indeed an artifact capable of sustaining rudimentary augmentation. Add GPS functionality and a three-dimensional model of the world — either maintained onboard the device, or resident in the cloud — and a viewer can be offered location-specific information, registered with and mapped onto the surrounding urban fabric.
In essence, phone-based AR treats the handset like the transparent pane of a cockpit head-up display: you hold it before you, its camera captures the forward-facing view, and this is rendered on the screen transparently but for whatever overlay of information is applied. Turn and the on-screen view turns with you, tracked (after a momentary stutter) by the grid of overlaid graphics. And those graphics can provide anything the network can: identification, annotation, direction or commentary.
It’s not hard to see why developers and enthusiasts might jump at this potential, even given the sharp limits imposed by the phone as platform. We move through the world and we act in it, but the knowledge we base our movements and actions on is always starkly less than what it might be. And we pay the price for this daily, in increments of waste, frustration, exhaustion and missed opportunity. By contrast, the notion that everything the network knows might be brought to bear on someone or -thing standing before us, directly there, directly present, available to anyone with the wherewithal to sign a two-year smartphone contract and download an app — this is a deeply seductive idea. It offers the same aura of omnipotence, that same frisson of godlike power evoked by our new ability to gather, sift and make meaning of the traces of urban activity, here positioned as a direct extension of our own senses.
Why not take advantage of this capability? After all, the richness and complexity of city life confronts us with any number of occasions on which the human sensorium could do with a little help.
Let a few hundred neurons in the middle fusiform gyrus of the brain’s right hemisphere be damaged, or fail to develop properly in the first place, and the result is a disorder called prosopagnosia, more commonly known as faceblindness. As the name suggests, the condition deprives its victims of the ability to recognize faces and associate them with individuals; at the limit, someone suffering with a severe case may be entirely unable to remember what his or her loved ones look like. So central is the ability to recognize others to human socialization, though, that even far milder cases cause significant problems.
Sadly, this is something I can attest to from firsthand experience. Like an estimated 2.5% of the population, I suffer from the condition, and even in the relatively attenuated form I’m saddled with, my broad inability to recognize people has caused more than a few experiences of excruciating awkwardness. At least once or twice a month I run into people on the street who clearly have some degree of familiarity with me, and find myself unable to come up with even a vague idea of who they might be; I’ll introduce myself to a woman at a party, only to have her remind me (rather waspishly, but who can blame her) that we’d worked together on a months-long project. Deprived of contextual cues — the time and location at which I usually meet someone, a distinctive hairstyle or mode of dress — I generally find myself no more able to recognize former colleagues or students than I can complete strangers. And as uncomfortable as this can be for me, I can only imagine how humiliating it is for the person on the other end of the encounter.
I long ago lost track of the number of times in my life at which I would have been grateful for some subtle intercessionary agent: something that might drop a glowing outline over the face of someone approaching me and remind me of his or her name, the occasion on which we met last, maybe even what we talked about on that occasion. It would spare both of us from mortification, and shield my counterpart from the inadvertent but real insult implied by my failure to recognize them. So the ambition of using AR in this role is lovely — precisely the kind of sensitive technical deployment I believe in, where technology is used to lower the barriers to socialization, and reduce or eliminate the awkwardnesses that might otherwise prevent us from better knowing one another.
But it’s hard to imagine any such thing being accomplished by the act of holding a phone up in front of my face, between us, forcing you to wait first for me to do so and then for the entire chain of technical events that must follow in order to fulfill the aim at the heart of the scenario. The device must acquire an image of your face with the camera, establish the parameters of that face from the image, and upload those parameters to the cloud via the fastest available connection, so they may be compared with a database of facial measurements belonging to known individuals; if a match is found, the corresponding profile must be located, and the appropriate information from that profile piped back down the connection so it may be displayed as an overlay on the screen image.
Too many articulated parts are involved in this interaction, too many dependencies — not least of which is the coöperation of a Facebook, a Google, or some other enterprise with a reasonably robust database of facial biometrics, and that is of course wildly problematic for other reasons. Better I should have confessed my confusion to you in the first place.
Perhaps a less technologically-intensive scenario would be better suited to the phone as platform for augmentation? How about helping a user find their way around the transit system, amidst all the involutions of the urban labyrinth?
Here we can weigh the merits of the use case by considering an actual, shipping product, Acrossair’s Nearest Subway app for the iPhone, first released in 2010. Like its siblings for London and Paris, Nearest Tube and Nearest Metro, Nearest Subway uses open location data made available by the city’s transit authority to specify the positions of transit stops in three-dimensional space. On launch, the app loads a hovering scrim of simple black tiles featuring the name of each station, and icons of the lines that serve it; the tiles representing more distant stations are stacked atop those that are closer. Rotate, and the scrim of tiles rotates with you. Whichever way you face, you’ll see a tile representing the nearest subway station in the direction of view, so long as some outpost of the transit network lies along that bearing in the first place.
Nearest Subway is among the more aesthetically appealing phone-based AR applications, eschewing junk graphics for simple, text-based captions sensitively tuned to the conventions of each city’s transit system. If nothing else, it certainly does what it says on the tin. It is, however, almost completely worthless as a practical aid to urban navigation.
When aimed to align with the Manhattan street grid from the corner of 30th Street and First Avenue, Nearest Subway indicates that the 21st Street G stop in Long Island City is the closest subway station, at a distance of 1.4 miles in a north-northeasterly direction.
As it happens, there are a few problems with this. For starters, from this position the Vernon Boulevard-Jackson Avenue stop on the 7 line is 334 meters, or roughly four New York City blocks, closer than 21st Street, but it doesn’t appear as an option. This is either an exposure of some underlying lacuna in the transit authority’s database — unlikely, but as anyone familiar with the MTA understands implicitly, well within the bounds of possibility — or more probably a failure on Acrossair’s part to write code that retrieves these coordinates properly.
Just as problematically, the claimed bearing is roughly 55 degrees off. If, as will tend to be the case in Manhattan, you align yourself with the street grid, a phone aimed directly uptown will be oriented at 27 degrees east of due north, at which point Nearest Subway suggests that the 21st Street station is directly ahead of you. But it actually lies on an azimuth of 82 degrees; if you took the app at its word, you’d be walking uptown a long time before you hit anything even resembling a subway station. This is most likely to be a calibration error with the iPhone’s compass, but fairly or otherwise Nearest Subway shoulders the greater part of the blame here — as anyone familiar with computational systems has understood since the time of Babbage, if you put garbage in, you’ll get garbage out.
Furthermore, since by design the app only displays those stations roughly aligned with your field of vision, there’s no way for it to notify you that the nearest station may be directly behind your back. Unless you want to rotate a full 360 degrees, then, and make yourself look like a complete idiot in the process, the most practical way to use Nearest Subway is to aim the phone directly down, which makes a reasonably useful ring of directional arrows and distances pop up. (These, of course, could have been superimposed on a conventional map in the first place, without undertaking the effort of capturing the camera image and augmenting it with a hovering overlay of theoretically compass-calibrated information.)
However unfortunate these stumbles may be, they can all be resolved, addressed with tighter code, an improved user interface or a better bearing-determination algorithm. Acrossair could fix them all, though — enter every last issue in a bug tracker, and knock them down one by one — and that still wouldn’t address the primary idiocy of urban AR in this mode: from 30th Street and First Avenue, the 21st Street G stop is across the East River. You need to take a subway to get there in the first place. However aesthetically pleasing an interface may be, using it to find the closest station as the crow flies does you less than no good when you’re separated from it by a thousand meters of water.
Finally, Nearest Subway betrays a root-level misunderstanding of the relationship between a citydweller and a transportation network. In New York City, as in every other city with a complex underground transit system, you almost never find yourself in a situation where you need to find the station that’s nearest in absolute terms to begin with; it’s far more useful to find the nearest station on a line that gets you where you want to go. Even at the cost of cluttering what’s on the screen, then, the very first thing the would-be navigator of the subway system needs is a way to filter the options before them by line.
I raise these points not to park all of the blame at Acrossair’s door, but to suggest that AR itself is badly unsuited to this role, at least when handled in this particular way. It takes less time to load and use a map than it does to retrieve the same information from an augmentive application, and the map provides a great deal more of the context so necessary to orienting yourself in the city. At this point in technological evolution, then, more conventional interface styles will tend to furnish a user with relevant information more efficiently, with less of the latency, error and cruft that inevitably seem to attend the attempt to superimpose it over the field of vision.
If phone-based augmentation performs poorly as social lubricant or aid to urban navigation, what about another role frequently proposed for AR, especially by advocates in the cultural heritage sector? This use case hinges on the argument that by superimposing images or other vestiges of the past of a place directly over its present, AR effectively endows its users with the ability to see through time.
This might not make much sense at all in Songdo, or Masdar, or any of the other new cities now being built from scratch on greenfield sites. But anyone who lives in a place old enough to have felt the passage of centuries knows that history can all too easily be forgotten by the stones of the city. Whatever perturbations from historical events may still be propagating through the various flows of people, matter, energy and information that make a place, they certainly aren’t evident to casual inspection. An augmented view returning the layered past to the present, in such a way as to color our understanding of the things all around us, might certainly prove to be more emotionally resonant than any conventional monument.
Byzantium, old Edo, Roman Londinium, even New Amsterdam: each of these historical sites is rife with traces we might wish to surface in the city occupying the same land at present. Locales overwhelmed by more recent waves of colonization, gentrification or redevelopment, too, offer us potent lenses through which to consider our moment in time. It would surely be instructive to retrieve some record of the jazz- and espresso-driven Soho of the 1950s and layer it over what stands there at present; the same goes for the South Bronx of 1975. But traversed as it was during the twentieth century by multiple, high-intensity crosscurrents of history, Berlin may present the ultimate terrain on which to contemplate recuperation of the past.
This is a place where pain, guilt and a sense of responsibility contend with the simple desire to get on with things; no city I’m familiar with is more obsessively dedicated to the search for a tenable balance between memory and forgetting. The very core of contemporary Berlin is given over to a series of puissant absences and artificially-sustained presences, from the ruins of Gestapo headquarters, now maintained as a museum called Topography of Terror, to the remnants of Checkpoint Charlie. A long walk to the east out leafy Karl-Marx-Allee — Stalinallee, between 1949 and 1961 — takes you to the headquarters of the Stasi, the feared secret police of the former East Germany, also open to the public as a museum. But there’s nowhere in Berlin where the curious cost of remembering can be more keenly felt than in the field of 2,711 concrete slabs at the corner of Ebertstrasse and Hannah-Arendt-Strasse. This is the Memorial to the Murdered Jews of Europe, devised by architect Peter Eisenman, with early conceptual help from the sculptor Richard Serra.
Formally, the grim array is the best thing Eisenman has ever set his hand to, very nearly redemptive of a career dedicated to the elevation of fatuous theory over aesthetic coherence; perhaps it’s the Serra influence. But as a site of memory, the Monument leaves a great deal to be desired. It’s what Michel Foucault called a heterotopia: something set apart from the ordinary operations of the city, physically and semantically, a place of such ponderous gravity that visitors don’t quite know what to make of it. On my most recent visit, the canyons between the slabs rang with the laughter of French schoolchildren on a field trip; the children giggled and flirted and shouted to one another as they leapt between the stones, and whatever the designer’s intent may have been, any mood of elegy or commemoration was impossible to establish, let alone maintain.
Roughly two miles to the northeast, on the sidewalk in front of a doner stand in Mitte, is a memorial of quite a different sort. Glance down, and you’ll see the following words, inscribed into three brass cubes set side by side by side between the cobblestones:
Ermordet in Auschwitz: that is, on specific dates in November of 1942 and March of the next year, the named people living at this address were taken across this very sidewalk and forcibly transported hundreds of miles east by the machinery of their own government, to a country they’d never known and a facility expressly designed to murder them. The looming façades around you were the last thing they ever saw as free people.
It’s in the dissonance between the everyday bustle of Mitte and these implacable facts that the true horror resides — and that’s precisely what makes the brass cubes a true memorial, indescribably more effective than Eisenman’s. The brass cubes, it turns out, are Stolpersteine, or “stumbling blocks,” a project of artist Gunter Demnig; these are but three of what are now over 32,000 that Demnig has arranged to have placed in some 700 cities. The Stolpersteine force us to read this stretch of unremarkable sidewalk in two ways simultaneously: both as a place where ordinary people go placidly about their ordinary business, just as they did in 1942, and as one site of a world-historical, continental-scale ravening.
The stories etched in these stones are the kind of facts about a place that would seem to yield to a strategy of augmentation. The objection could certainly be raised that I found them so resonant precisely because I didn’t see them every day, and that their impact would very likely fade with constant exposure; we might call this the evil of banality. But being compelled to see and interpret the mundane things I did in these streets through the revenant past altered my consciousness, in ways subtler and longer-lasting than anything Eisenman’s sepulchral array of slabs was able to achieve. AR would merely make the metaphor literal — in fact, it’s easy for me to imagine the disorienting, decentering, dis-placing impact of having to engage the world through a soft rain of names, overlaid onto the very places from which their owners were stolen.
But once again, it’s hard to imagine this happening via the intercession of a handset. Nor are the qualities that make smartphone-based AR so catastrophically clumsy, in virtually every scenario of use, particularly likely to change over time.
The first is the nature of functionality on the smartphone. As we’ve seen, the smartphone is a platform on which each discrete mode of operation is engaged via a dedicated, single-purpose app. Any attempt at augmenting the environment, therefore, must be actively and consciously invoked, to the exclusion of other useful functionality. The phone, when used to provide such an overlay, cannot also and at the same time be used to send a message, look up an address, buy a cup of coffee, or do any of the other things we now routinely expect of it.
The second reservation is physical. Providing the user with a display surface for graphic annotation of the forward view simply isn’t what the handset was designed to do. It must be held before the eyes like a pane of glass in order for the augmented overlay to work as intended. It hardly needs to be pointed out that this gesture is not one particularly well-suited to the realities of urban experience. It has the doubly unappealing quality of announcing the user’s distraction and vulnerability to onlookers, while simultaneously ensuring that the device is held in the weak grip of the extended arm — a grasp from which it may be plucked with relative ease.
Taken together, these two impositions strongly undercut the primary ostensible virtue of an augmented view, which is its immediacy. The sole genuine justification for AR is the idea that information is simply there, copresent with that you’re already looking at and able to be assimilated without thought or effort.
That sense of effortlessness is precisely what an emerging class of wearable mediators aims to provide for its users. The first artifact of this class to reach consumers is Google’s Glass, which mounts a high-definition, forward-facing camera, a head-up reticle and the microphone required by the natural-language speech recognition interface on a lightweight aluminum frame. While Glass poses any number of aesthetic, practical and social concerns — all of which remain to be convincingly addressed, by Google or anyone else — it does at least give us a way to compare hands-free, head-mounted AR with the handset-based approach.
Would any of the three augmentation scenarios we explored be improved by moving the informational overlay from the phone to a wearable display?
A system designed to mitigate my prosopagnosia by recognizing faces for me would assuredly be vastly better when accessed via head-mounted interface; in fact, that’s the only scenario of technical intervention in relatively close-range interpersonal encounters that’s credible to me. The delay and physical awkwardness occasioned by having to hold a phone between us goes away, and while there would still be a noticeable saccade or visual stutter as I glanced up to read your details off my display, this might well be preferable to not being remembered at all.
That is, if we can tolerate the very significant threats to privacy involved, which only start with Google’s ownership of or access to the necessary biometric database. There’s also the question of their access to the pattern of my requests, and above all the one fact inescapably inherent to the scenario: that people are being identified as being present in a certain time and place, without any necessity whatsoever of securing consent on their part. By any standard, this is a great deal of risk to take on, all to lubricate social interactions for 2.5% of the population.
Nearest Subway, as is, wouldn’t be improved by presentation in the line of sight. Given what we’ve observed about the way people really use subway networks, information about the nearest station in a given direction wouldn’t be of any greater utility when splashed on a head-up display than it is on the screen of a phone. Whatever the shortcomings of this particular app, though, they probably don’t imply anything in particular about the overall viability of wearable AR in the role of urban navigation, and in many ways the technology does seem rather well-suited to the wayfinding challenges faced by the pedestrian.
Of the three scenarios considered here, though, it’s AR’s potential to offer novel perspectives on the past of a place that would be most likely to benefit from the wearable approach. We would quite literally see the quotidian environment through the lens of a history superimposed onto it. So equipped, we could more easily plumb the psychogeographical currents moving through a given locale, better understand how the uses of a place had changed over time, or hadn’t. And because this layer of information could be selectively surfaced — invoked and banished via voice command, toggled on or off at will — presenting information in this way might well circumvent the potential for banality through overfamiliarization that haunts even otherwise exemplary efforts like Demnig’s Stolpersteine.
And this suggests something about further potentially productive uses for augmentive mediators like Glass. After all, there are many kinds of information that may be germane to our interpretation of a place, yet effectively invisible to us, and historical context is just one of them. If our choices are shaped by dark currents of traffic and pricing, crime and conviviality, it’s easy to understand the appeal of any technology proposing that these dimensions of knowledge be brought to bear on that which is seen, whether singly or in combination. The risk of bodily harm, whatever its source, might be rendered as a red wash over the field of vision; point-by-point directions as a bright and unmistakable guideline reaching into the landscape. In fact any pattern of use and activity, so long as its traces were harvested by some data-gathering system and made available to the network, might be made manifest to us in this way.
Some proposed uses of mediation are more ambitious still, pushing past mere annotation of the forward view to the provision of truly novel modes of perception — for example, the ability to “see” radiation at wavelengths beyond the limits of human vision, or even to delete features of the visual environment perceived as undesirable. What, then, keeps wearable augmentation from being the ultimate way for networked citizens to receive and act on information?
The approach of practical, consumer-grade augmented reality confronts us with a interlocking series of concerns, ranging from the immediately practical to the existential.
A first set of reservations centers on the technical difficulties involved in the articulation of an acceptably high-quality augmentive experience. We’ve so far bypassed discussion of these so we could consider different aspects of the case for AR, but ultimately they’re not of a type that allows anyone to simply wave them away.
At its very core, the AR value proposition subsists in the idea that interactions with information presented in this way are supposed to feel “effortless,” but any such effortlessness would require the continuous (and continuously smooth) interfunctioning of a wild scatter of heterogeneous elements. In order to make good on this promise, a mediation apparatus would need to fuse all of the following elements: a sensitively-designed interface; the population of that interface with accurate, timely, meaningful and actionable information; and a robust, high-bandwidth connection to the networked assets furnishing that information from any point in the city, indoors or out. Even putting questions of interface design to the side, the technical infrastructure capable of delivering the other necessary elements reliably enough that the attempt at augmentation doesn’t constitute a practical and social hazard in its own right does not yet exist — not anywhere in North America, anyway, and not this year or next. The hard fact is that for a variety of reasons having to do with national spectrum policy, a lack of perceived business incentives for universal broadband connectivity, and other seemingly intractable circumstances, these issues are nowhere near being ironed out.
In the context of augmentation, as well, the truth value of representations made about the world acquires heightened significance. By superimposing information directly on its object, AR arrogates to itself a peculiar kind of claim to authority, a claim of a more aggressive sort than that implicit in other modes of representation, and therefore ought to be held to a higher standard of completeness and accuracy. As we saw with Nearest Subway, though, an overlay can only ever be as good as the data feeding it, and the augurs in this respect are not particularly reassuring. Right now, Google’s map of the commercial stretch nearest to my apartment building provides labels for only four of the seven storefront businesses on the block, one of which is inaccurately identified as a restaurant that closed many years ago. If even Google, with all the resources it has at its disposal, struggles to provide its users with a description of the streetscape that is both comprehensive and correct, how much more daunting will other actors find the same task?
Beyond this are the documented problems with visual misregistration and latency that are of over a decade’s standing, and have not been successfully addressed in that time — if anything, have only been exacerbated by the shift to consumer-grade hardware. At issue is the mediation device’s ability to track rapid motions of the head, and smoothly and accurately realign any graphic overlay mapped to the world; any delay in realignment of more than a few tens of milliseconds is conspicuous, and risks causing vertigo, nausea and problems with balance and coordination. The initial release of Glass, at least, wisely shies away from any attempt to superimpose such overlays, but the issue must be reckoned with at some point if useful augmentive navigational applications are ever to be developed.
Another set of concerns centers on the question of how long such a mediator might comfortably be worn, and what happens after it is taken off. This is of especial concern given the prospect that one or another form of wearable AR might become as prominent in the negotiation of everyday life as the smartphone itself. There is, of course, not much in the way of meaningful prognostication that can be made ahead of any mass adoption, but it’s not unreasonable to build our expectations on the few things we do know empirically.
Early users of Google’s Glass report disorientation upon removing the headset, after as few as fifteen minutes of use — a mild one, to be sure, and easily shaken off, from all accounts the sort of uneasy feeling that attends staring overlong at an optical illusion. If this represents the outer limit of discomfort experienced by users, it’s hard for me to believe that it would have much impact on either the desirability of the product or people’s ability to function after using it. But further hints as to the consequences of long-term use can be gleaned from the testimony of pioneering researcher Steve Mann, who has worn a succession of ever-lighter and more-capable mediation rigs all but continuously since the mid-1980s. And his experience would seem to warrant a certain degree of caution: Mann, in his own words, early on “developed a dependence on the apparatus,” and has found it difficult to function normally on the few occasions he has been forcibly prevented from accessing his array of devices.
When deprived of his set-up for even a short period of time, Mann experiences “profound nausea, dizziness and disorientation”; he can neither see clearly nor concentrate, and has difficulty with basic cognitive and motor tasks. He speculates that over many years, his neural wiring has adapted to the continuous flow of sensory information through his equipment, and this is not an entirely ridiculous thing to think. At this point, the network of processes that constitutes Steve Mann’s brain — that in some real albeit reductive sense constitutes Steve Mann — lives partially outside his skull.
The objection could be made that this is always already the case, for all of us — that some nontrivial part of everything that make us what we are lives outside of us, in the world, and that Mann’s situation is only different in that much of his outboard being subsists in a single, self-designed apparatus. But if anything, this makes the prospect of becoming physiologically habituated to something like Google Glass still more worrisome. It’s precisely because Mann developed and continues to manage his own mediation equipment that he can balance his dependency on it with the relative freedom of action enjoyed by someone who for the most part is able to determine the parameters under which that equipment operates.
If Steve Mann has become a radically hybridized consciousness, at least he has a legitimate claim to ownership and control over all of the places where that consciousness is instantiated. By contrast, all of the things a commercial product like Glass can do for the user rely on the ongoing provision of a service — and if there’s anything we know about services, it’s that they can be and are routinely discontinued at will, as the provider fails, changes hands, adopts a new business strategy or simply reprioritizes.
A final set of strictly practical concerns have to do with the collective experience of augmentation, or what implications our own choice to be mediated in this way might hold for the experience of others sharing the environment.
For all it may pretend to transparency, literally and metaphorically, any augmentive mediator by definition imposes itself between the wearer and the phenomenal world. This, of course, is by no means a quality unique to augmented reality. It’s something AR has in common with a great many ways we already buffer and mediate what we experience as we move through urban space, from listening to music to wearing sunglasses. All of these impose a certain distance between us and the full experiential manifold of the street, either by baffling the traces of it that reach our senses, or by offering us a space in which we can imagine and project an alternative narrative of our actions.
But there’s a special asymmetry that haunts our interactions with networked technology, and tends to undermine our psychic investment in the immediate physical landscape; if “cyberspace is where you are when you’re on the phone,” it’s certainly also the “place” you are when you text or tweet someone while walking down the sidewalk. I’ve generally referred to what happens when someone moves through the city while simultaneously engaged in some kind of remote interaction as a condition of “multiple adjacency,” but of course it’s really no such thing: so far, at least, only one mode of spatial experience can be privileged at a given time. And if it’s impossible to participate fully in both of these realms at once, one of them must lose out.
Watch what happens when a pedestrian first becomes conscious of receiving a call or a text message, the immediate damming they cause in the sidewalk flow as they pause to respond to it. Whether the call is made hands-free or otherwise doesn’t really seem to matter; the cognitive and emotional investment in what transpires in the interface is what counts, and this investment is generally so much greater than it is in the surroundings that street life clearly suffers as a result. The risk inherent in this divided attention appears to be showing up in the relevant statistics in the form of an otherwise hard-to-account-for upturn in accidents involving pedestrian fatalities, where such numbers had been falling for years. This is a tendency that is only likely to be exacerbated by augmentive mediation, particularly where content of high inherent emotional involvement is concerned.
At this moment in time, it would be hard to exaggerate the appeal the prospect of wearable augmentation holds for its vocal cohort of enthusiasts within the technology community. This fervor can be difficult to comprehend, so long as AR is simply understood to refer to a class of technologies aimed at overlaying the visual field with information about the objects and circumstances in it.
What the discourse around AR shares with other contemporary trans- and posthuman narratives is a frustration with the limits of the flesh, and a frank interest in transcending them through technical means. To advocates, the true appeal of projects like Google’s Glass is that they are first steps toward the fulfillment of a deeper promise: that of becoming-cyborg. Some suggest that ordinary people mediate the challenges of everyday life via complex informational dashboards, much like those first devised by players of World of Warcraft and similar massively multiplayer online role-playing games. The more fervent dream of a day when their capabilities are enhanced far beyond the merely human by a seamless union of organic consciousness with networked sensing, processing, analytic and storage assets.
Beyond the profound technical and practical challenges involved in achieving any such goal, though, someone not committed to one or another posthuman program may find that they have philosophical reservations with this notion, and what it implies for urban life. These may be harder to quantify than strictly practical objections, but any advocate of augmentation technologies who is also interested in upholding the notion of a city as a shared space will have to come to some reckoning with them.
Anyone who cares about what we might call the full bandwidth of human communication — very much including transmission and reception of those cues vital to understanding, but only present beneath the threshold of conscious perception — ought to be concerned about the risk posed to interpersonal exchanges by augmentive mediation. Wearable devices clearly have the potential to exacerbate existing problems of self-absorption and mutual inconsideration. Although in principle there’s no reason such devices couldn’t be designed to support or even enrich the sense of intersubjectivity, what we’ve seen about the technologically-mediated pedestrian’s unavailability to the street doesn’t leave us much room for optimism on this count. The implication is that if the physical environment doesn’t fully register to a person so equipped, neither will other people.
Nor is the body by any means the only domain that the would-be posthuman subject may wish to transcend via augmentation. Subject as it is to the corrosive effects of entropy and time, forcing those occupying it to contend with the inconvenient demands of others, the built environment is another. Especially given current levels of investment in physical infrastructure in the United States, there is a very real risk that those who are able to do so will prefer retreat behind a wall of mediation to the difficult work of being fully present in public. At its zenith, this tendency implies both a dereliction of public space and an almost total abandonment of any notion of a shared public realm. This is the scenario imagined by science-fiction author Vernor Vinge in Rainbows End (2006), in which people interact with the world’s common furniture through branded thematic overlays of their choice; it’s a world that can be glimpsed in the matter-of-factly dystopian videos of Keiichi Matsuda, in which a succession of squalid environments come to life only when activated by colorful augmentive animations.
The most distressing consequences of such a dereliction would be felt by those left behind in any rush toward augmentation. What happens when the information necessary to comprehend and operate an environment is not immanent to that environment, but has become decoupled from it? When signs, directions, notifications, alerts and all the other instructions necessary to the fullest use of the city appear only in an augmentive overlay, and as is inevitably the case, that overlay is available to some but not others? What happens to the unaugmented human under such circumstances? The perils would surely extend beyond a mere inability to act on information; the non-adopter of a particularly hegemonic technology almost always places themselves at jeopardy of being seen as a willful transgressor of norms, even an ethical offender. Anyone forgoing augmentation, for whatever reason, may find that they are perceived as somehow less than a full member of the community, with everything that implies for the right to be and act in public.
The deepest critique of all those lodged against augmented reality is sociologist Anne Galloway’s, and it is harder to answer. Galloway suggests that the discourse of computational augmentation, whether consciously or otherwise, “position[s] everyday places and social interactions as somewhat lacking or in need of improvement.” Again there’s this Greshamization, this sense of a zero-sum relationship between AR and a public realm already in considerable peril just about everywhere. Maybe the emergence of these systems will spur us to some thought as to what it is we’re trying so hard to augment. Philip K. Dick once defined reality as “that which refuses to go away when you stop believing in it,” and it’s this bedrock quality of universal accessibility — to anyone at all, at any time of his or her choosing — that constitutes its primary virtue. If nothing else, reality is the one platform we all share, a ground we can start from in undertaking the arduous and never-comfortable process of determining what else we might agree upon. To replace this shared space with the million splintered and mutually inconsistent realities of individual augmentation is to give up on the whole pretense that we in any way occupy the same world, and therefore strikes me as being deeply inimical to the urban project as I understand it. A city in which the physical environment has ceased to function as a common reference frame is, at the very least, terribly inhospitable soil for democracy, solidarity or simple fellow-feeling to take root in.
It may well be that this concern is overblown. There is always the possibility that augmented reality never will amount to very much, or that after a brief period of consideration it’s actively rejected by the mainstream audience. Within days of the first significant nonspecialist publicity around Google Glass, Seattle dive bar The 5 Point became the first commercial establishment known to have enacted a ban on the device, and if we can fairly judge from the rather pungent selection of terms used to describe Glass wearers in the early media commentary, it won’t be the last. By the time you read these words, these weak signals may well have solidified into some kind of rough consensus, at least in North America, that wearing anything like Glass in public space constitutes a serious faux pas. Perhaps this and similar AR systems will come to rest in a cultural-aesthetic purgatory like that currently occupied by Bluetooth headsets, and if that does turn out to be the case, any premature worry about the technology’s implications for the practice of urban democracy will seem very silly indeed.
But something tells me that none of the objections we’ve discussed here will prove broadly dissuasive, least of all my own personal feelings on the subject. For all the hesitations anybody may have, and for all the vulnerabilities even casual observers can readily diagnose in the chain of technical articulations that produces an augmentive overlay, it is hard to argue against a technology that glimmers with the promise of transcendence. Over anything beyond the immediate near term, some form of wearable augmentive device does seem bound to take a prominent role in returning networked information to the purview of a mobile user at will, and thereby in mediating the urban experience. The question then becomes what kind(s) of urbanity will be produced by people endowed with this particular set of capabilities, individually and collectively, and how we might help the unmediated contend with cities unlike any they have known, enacted for the convenience of the ambiguously transhuman, under circumstances whose depths have yet to be plumbed.
Notes on this section
 Grüter T, Grüter M, Carbon CC (2008). “Neural and genetic foundations of face recognition and prosopagnosia”. J Neuropsychol 2 (1): 79–97.
 For early work toward this end, see http://www.cc.gatech.edu/~thad/p/journal/augmented-reality-through-wearable-computing.pdf. The overlay of a blinking outline or contour used as an identification cue, incidentally, has long been a staple of science-ﬁctional information displays, showing up in pop culture as far back as the late 1960s. The earliest appearance I can locate is 2001: A Space Odyssey (1968), in which the navigational displays of both the Orion III spaceplane and Discovery itself relied heavily on the trope — this, presumably, because they were produced by the same contractor, IBM. See also Pete Shelley’s music video for “Homosapien” (1981) and the traverse corridors projected through the sky of Blade Runner’s Los Angeles (1982).
 As always, I caution the reader that the specifics of products and services, their availability will certainly change over time. All comments here regarding Nearest Subway pertain to v1.4.
 See discussion of “Superplonk” in [a later section]. http://m.spectrum.ieee.org/podcast/geek-life/profiles/steve-manns-better-version-of-reality
 At the very least, user interface should offer some kind of indication as to the confidence of a proffered identification, and perhaps how that determination was arrived at. See [a later section] on seamfulness.
 Azuma, “Registration Errors in Augmented Reality,” 1997.
 See Governors Highway Safety Association, “Spotlight on Highway Safety: Pedestrian Fatalities by State,” 2010. http://www.ghsa.org/html/publications/pdf/spotlights/spotlight_ped.pdf; similarly, a recent University of Utah study found that the act of immersion in a conversation, rather than any physical aspect of use, is the primary distraction while driving and talking on the phone. That hands-free headset may not keep you out of a crash after all. http://www.informationweek.com/news/showArticle.jhtml?articleID=205207840
 A story on the New York City-based gossip site Gawker expressed this point of view directly, if rather pungently: “If You Wear Google’s New Glasses, You Are An Asshole.” http://gawker.com/5990395/if-you-wear-googles-new-glasses-you-are-an-asshole
 The differentiation involved might be very fine-grained indeed. Users may interact with informational objects that exist only for them and for that single moment.
 The first widespread publicity for Glass coincided with Google’s release of a video on Wednesday, 20th February, 2013; The 5 Point announced its ban on 5th March. The expressed concerns center more on the device’s data-collection capability than anything else: according to owner Dave Meinert, his customers “don’t want to be secretly filmed or videotaped and immediately put on the Internet,” and this is an entirely reasonable expectation, not merely in the liminal space of a dive bar but anywhere in the city. See http://news.cnet.com/8301-1023_3-57573387-93/seattle-dive-bar-becomes-first-to-ban-google-glass/
Wagner James Au, who would know, has what in a better world would be an incendiary piece in the latest Wired. Au’s piece lays it all right out there, regarding the meaning and purpose of virtual reality.
As VR’s leading developers straight-up admit in the piece, its function is to camouflage the inequities and insults of an unjust world, by offering the masses high-fidelity simulations of the things their betters get to experience for real. Here’s the money quote, no pun intended: “[S]ome fraction of the desirable experiences of the wealthy can be synthesized and replicated for a much broader range of people.” (That’s John Carmack speaking, for future reference.)
I always want to extend to those I disagree with some presumption of good will. I don’t think it’s either healthy or productive or pleasant for the people around me to spend my days in a permanent chokehold of high dudgeon. And I always want to leave some room for the possibility that someone might have been misunderstood or misquoted. But Au is a veteran reporter on this topic; I think it’s fair to describe his familiarity with the terrain, and the players, as “comprehensive.” So I rather doubt he’s mischaracterized Carmack’s sentiments, or those of Oculus Rift founder Palmer Luckey. And what those sentiments amount to is outright barbarism — is nothing less than moral depravity.
The idea that all we can do is accede to a world of permanent, vertiginous inequity — inequity so entrenched and so unchallengeable that the best thing we can do with our technology is use it as a palliative and a pacifier — well, this is everything I’m committed to working against. Thankfully there are others who are also doing that work, who understand the struggle as the struggle. Thankfully, I think most of us still understand Carmack’s stated ambition as vile. We do, right?
I’ll have more to say about the uses of VR (and its cousin augmented reality, or AR) shortly.
The following “interview” with me appears in the July/August 2009 issue of Interactions magazine, the ACM’s journal on interaction design. I say “interview” because it’s basically an edit on the sprawling chat Tish Shute had with me for her site, back in February of this year; as we know, even minor editorial alterations can produce disproportionate shifts in tone and emphasis, and that’s certainly the case here.
I should say from the outset that I don’t have much use for the ACM, and in particular greatly dislike their stance on access to publications, which flies in the face of my own conviction that the point (and power) of knowledge is to share it. Accordingly, I’m republishing the piece in its entirety here. For the sake of accuracy, I’ve left the editorial characterization of me and my work intact, but you should never, ever construe this as an endorsement of same. As ever, I hope you enjoy it.
“At the end of the world, plant a tree”
Six questions for Adam Greenfield
Adam Greenfield is Nokia’s head of design direction for service and user-interface design, and the author of Everyware: The dawning age of ubiquitous computing and the upcoming The City Is Here For You To Use. He is also an impactful speaker and articulate blogger, and has become a major authority in the thinking about the impact of future ubiquitous technologies on people and society.
In a lengthy interview with Tish Shute recently published on UgoTrade.com, Greenfield ranged over topics including augmented reality, Usman Haque’s Pachube project, the networked book, the networked city, and what to do at the end of the world.
The interview is dense and rich, with many of the questions raised relevant to our audience. We asked Adam to expand on some of his answers for Interactions Magazine.
TS: The legal scholar Eben Moglen has identified three elements of privacy: anonymity, secrecy and most importantly autonomy. How do you see Moglen’s three elements being worked out in a ubiquitously networked world? Are there ways we could design ubiquitous systems that might support personal autonomy?
AG: If we accept for the moment a definition of autonomy as a feeling of being master of one’s own fate, then absolutely yes. One thing I talk about a good deal is using ambient situational awareness to lower decision costs – that is, to lower the information costs associated with arriving at a choice presented to you, and at the same time mitigate the opportunity costs of having committed yourself to a course of action. When given some kind of real-time overview of all of the options available to you in a given time, place and context – and especially if that comes wrapped up in some kind of visualization that makes anomaly detection a matter of instantaneous gestalt, to be grasped in a single glance – your personal autonomy is tremendously enhanced. Tremendously enhanced.
What do I mean by that? It’s really simple: you don’t head out to the bus stop until your phone tells you a bus is a minute away, and you don’t walk down the street where more than some threshold number of muggings happen – in fact, by default it doesn’t even show up on your maps – and you don’t eat at the restaurant whose forty-eight recent health code violations cause its name to flash red in your address book. And all these decisions are made possible because networked informatics have effectively rendered the obscure and the hidden transparent to inquiry. And there’s no doubt in my mind that life is thusly made just that little bit better.
But there’s a cost – there’s always a cost. Serendipity, solitude, anonymity, most of what we now recognize as the makings of urban savoir faire: it all goes by the wayside. And yes, we’re richer and safer and maybe even happier with the advent of the services and systems I’m so interested in, but by the same token we’re that much poorer for the loss of these intangibles. It’s a complicated trade-off, and I believe in most places it’s one we’re making without really examining what’s at stake.
So as to how this local autonomy could be deployed in Moglen’s more general terms, I don’t know, and I’m not sure anyone does. Because he’s absolutely right: Bernard Stiegler reminds us that the network constitutes a global mnemotechnics, a persistent memory store for planet Earth, and yet we’ve structured our systems of jurisprudence and our life practices and even our psyches around the idea that information about us eventually expires and leaves the world. Its failure to do so in the context of Facebook and Flickr and Twitter is clearly one of the ways in which the elaboration of our digital selves constrains our real-world behavior. Let just one picture of you grabbing a cardboard cutout’s breast or taking a bong hit leak onto the network, and see how the career options available to you shift in response.
This is what’s behind Anne Galloway’s calls for a “forgetting machine.” An everyware that did that – that massively spoofed our traces in the world, that threw up enormous clouds of winnow and chaff to give us plausible deniability about our whereabouts and so on – might give us a fighting chance.
TS: Early theorizing of a “calm,” “invisible” ubicomp seems out of synch with the present-day reality of services like Twitter and Facebook, where active, engaged, contact-driven users continually manage their networked identity. How will the processes of contact and identity-sharing that have seemingly captured the popular imagination be or not be part of the city that is Here For You To Use?
AG: Let’s remember that ubicomp itself, as a discipline, has largely moved on from the Weiserian discourse of “calm technology”; Yvonne Rogers, for example, now speaks of “proactive systems for proactive people.” You can look at this as a necessary accommodation with the reality principle, which it is, or as kind of a shame – which it also happens to be, at least in my opinion. Either way, though, I don’t think anybody can credibly argue any longer that just because informatic systems pervade our lives, designers will be compelled to craft encalming interfaces to them. That notion of Mark Weiser’s was never particularly convincing, and as far as I’m concerned it’s been thoroughly refuted by the unfolding actuality of post-PC informatics.
All the available evidence, on the contrary, supports the idea that we will have to actively fight for moments of calm and reflection, as individuals and as collectivities. And not only that, as it happens, but for spaces in which we’re able to engage with the Other on neutral turf, as it were, since the logic of “social media” seems to be producing Big Sort-like effects and echo chambers. When given the tools that allow us to do so, we seem to surround ourselves with people who look and think and consume like we do, and the result is that the tools allowing us to become involved with anything but the self, or selves that strongly resemble it, are atrophying.
So when people complain about K-Mart and Starbucks and American Eagle Outfitters coming to Manhattan, and how it means the suburbanization of the city, I have to laugh. Because the real suburbanization is the smoothening-out of our social interaction until it only encompasses the congenial. A gated community where everyone looks and acts the same? That’s the suburbs, wherever and however it instantiates, and I don’t care how precious and edgy your tastes may be. Richard Sennett argued that what makes urbanity is precisely the quality of necessary, daily, cheek-by-jowl confrontation with a panoply of the different, and as far as I can tell he’s spot on.
We have to devise platforms that accommodate and yet buffer that confrontation. We have to create the safe(r) spaces that allow us to negotiate that difference. The alternative to doing so is creating a world of ten million autistic, utterly atomic and mutually incomprehensible tribelets, each reinforced in the illusion of its own impeccable correctness: duller than dull, except at the flashpoints between. And those become murderous. Nope. Unacceptable outcome.
TS: What new imaginings or possibilities do you see when pixels anywhere are linked to everyware?
AG: Limitless opportunities for product placement. Commercial insertions and injections, mostly.
Beyond that: one of the places where shallowly Weiserian logic breaks down is in thinking that the platforms we use now disappear from the world just because ubiquitous computing has arrived. We’ve still got radio, for example – OK, now it’s satellite radio and streaming Internet feeds, but the interaction metaphor isn’t any different. By the same token, we’re still going to be using reasonably conventional-looking laptops and desktop keyboard/display combos for a while yet. The form factor is pretty well optimized for the delivery of a certain class of services, it’s a convenient and well-assimilated interaction vocabulary, none of that’s going away just yet. And the same goes for billboards and “TV” screens.
But all of those things become entirely different propositions in everyware world: more open, more modular, ever more conceived of as network resources with particular input and output affordances. We already see some signs of this with Microsoft’s recent “Social Desktop” prototype – which, mind you, is a very bad idea as it currently stands, especially as implemented on something with the kind of security record that Windows enjoys – and we’ll be seeing many more.
If every display in the world has an IP address and a self-descriptor indicating what kind of protocols it’s capable of handling, then you begin to get into some really interesting and thorny territory. The first things to go away, off the top of my head, are screens for a certain class of mobile device – why power a screen off your battery when you can push the data to a nearby display that’s much bigger, much brighter, much more social? – and conventional projectors.
Then we get into some very interesting issues around large, public interactive displays – who “drives” the display, and so forth. But here again, we’ll have to fight to keep these things sane. It’s past time for a public debate around these issues, because they’re unquestionably going to condition the everyday experience of walking down the street in most of our cities. And that’s difficult to do when times are hard and people have more pressing concerns on their mind.
TS: The science-fiction writer David Brin sees two potential futures: in the first, the government watches everybody, and in the second everybody watches everybody. (The latter he calls sousveillance.) It has been suggested by the artificial-intelligence enthusiast Ben Goertzel that providing an artificial intelligence with access to a massive datastore fed by ubicomp is the first step toward effective sousveillance.
What do you think the role of AI in ubicomp will be? Is it worth thinking about what the first important application of such technologies might be?
AG: I don’t believe that artificial intelligence as the term is generally understood – which is to say, a self-aware, general-purpose intelligence of human capacity or greater – is likely to appear within my lifetime, or for a comfortably long time thereafter.
Having said that, Goertzel seems to be making the titanic (and enormously difficult to justify) assumption that a self-aware artificial intelligence would share any perspectives, goals, priorities or values whatsoever with the human species, let alone with that fraction of the human species that could use a little help in countering watchfulness from above. “Hooking [an] AI up to a massive datastore fed by ubicomp” sounds to me more like the first step toward enslavement…if not outright digestion.
Sousveillance – the term is Steve Mann’s, originally – doesn’t imply “everybody watching everybody” to me, anyway, so much as a consciously political act of turning infrastructures of observation and control back on those specific institutions most used to employing same toward their own prerogatives. Think Rodney King, think Oscar Grant.
TS: You seem to be skeptical about the role everyware can play in sustainable living. And yet at the moment it seems that – in the hacker and business communities at least – the role of everyware in reducing carbon footprint/energy management, etc., is the great green hope.
Will everyware enable or hinder fundamental changes at the level of culture and identity necessary to support the urgent global need “to consume less and redefine prosperity”?
AG: I’m not skeptical about the potential of ubiquitous systems to meter energy use, and maybe even incentivize some reduction in that use – not at all. I’m simply not convinced that anything we do will make any difference.
Look, I think we really, seriously screwed the pooch on this. We have fouled the nest so thoroughly and in so many ways that I would be absolutely shocked if humanity comes out the other end of this century with any level of organization above that of clans and villages. It’s not just carbon emissions and global warming, it’s depleted soil fertility, it’s synthetic estrogens bio-accumulating in the aquatic food chain, it’s our inability to stop using antibiotics in a way that gives rise to multiple drug resistance in microbes.
Any one of these threats in isolation would pose a challenge to our ability to collectively identify and respond to it, as it’s clear anthropogenic global warming already does. Put all of these things together, assess the total threat they pose in the light of our societies’ willingness and/or capacity to reckon with them, and I think any moderately knowledgeable and intellectually honest person has to conclude that it’s more or less “game over, man” – that sometime in the next sixty years or so a convergence of Extremely Bad Circumstances is going to put an effective end to our ability to conduct highly ordered and highly energy-intensive civilization on this planet, for something on the order of thousands of years to come.
So with all apologies to Bruce Sterling, I just don’t buy the idea that we’re going to consume our way to Ecotopia. Nor is any symbolic act of abjection on my part going to postpone the inevitable by so much as a second, nor would such a sacrifice do anything meaningful to improve anybody else’s outcomes. I’d rather live comfortably – hopefully not obscenely so – in the years we have remaining to us, use my skills as they are most valuable to people, and cherish each moment for what it uniquely offers.
Maybe some people would find that prospect morbid, or nihilistic, but I find it kind of inspiring. It becomes even more crucial that we not waste the little time we do have on broken systems, broken ways of doing things. The primary question for the designers of urban informatics under such circumstances is to design systems that underwrite autonomy, that allow people to make the best and wisest and most resonant use of whatever time they have left on the planet. And who knows? That effort may bear fruit in ways we have no way of anticipating at the moment. As it says in the Qu’ran, gorgeously: “At the end of the world, plant a tree.”
TS: The concept of autonomy is signaled clearly in the title you have chosen for your next book, The City Is Here For You To Use, and seems to be a consistent theme in your writing. While you have in the past (notably in Everyware) discussed the possible constraints to presentation of self and threats to a flexible identity posed by ubiquitous computing, your next book signals optimism. What are your key grounds for this optimism?
AG: It’s not optimism so much as hope. Whether it’s well-founded or not is not for me to decide. I guess I just trust people to make reasonably good choices, when they’re both aware of the stakes and have been presented with sound, accurate decision-support material.
Putting a fine point on it: I believe that most people don’t actually want to be dicks. We may have differing conceptions of the good, our choices may impinge on one another’s autonomy. But I think most of us, if confronted with the humanity of the Other and offered the ability to do so, would want to find some arrangement that lets everyone find some satisfaction in the world. And in its ability to assist us in signaling our needs and desires, in its potential to mediate the mutual fulfillment of same, in its promise to reduce the fear people face when confronted with the immediate necessity to make a decision on radically imperfect information, a properly-designed networked informatics could underwrite the most transformative expansions of people’s ability to determine the circumstances of their own lives.
Now that’s epochal. If that isn’t cause for hope, then I don’t know what is.
I’ve long been of the opinion that there are terms of art floating through the various interaction design and user-experience conversations I’m a party to that should never, ever be exposed to the end user. That is, however useful they are to us as designers, they’re so technical, jargony or obscure that they should neither show up in a product or service interface, nor its documentation, nor all but the most granular and geek-centric of its marketing materials. By no means am I alone in thinking this; I’d imagine this is as close to conventional wisdom as one can get in this disjoint field.
What I might have missed, though, is that there are other people even within a design organization who might not sling the lingo with such ease. Not that these folks are in any way limited or less than fully competent at their jobs, it’s just that they’re not as au courant regarding the minutiae of my own particular field as I might like. (And why should they be? It’s not like I live and breathe market-segmentation strategy.)
Case in point: I must say that it’s been surprisingly difficult, in various conversations with folks not immersed in the IxD space, to get across the essential distinction between context-aware applications and location-based services (LBS).
Everyone gets LBS, more or less: it’s the ground on which Interaction Design Cliché No. 2 is built. You’re in a particular place, and there are things your device can do in this place that it’s not capable of elsewhere. Straightforward enough, right?
But what is “context” if not location? What could that possibly mean? It turns out that this is not at all an obvious distinction, and that understanding what an interface designer might mean by “context-aware” is actually built on, uh, shared context. (It’s like rain on your wedding day, or summink.) And since that shared context is, in this case, absent, it behooves the designer who wants to work effectively in a heterogeneous organization to do a better job of explaining these important ideas to everyone else around them.
In fact, designer Mac Funamizu has actually nailed two separate things here. The first demonstrates precisely what I, at least, mean when I use the words “context aware”: but for some residual core of basic functionality, the device’s capabilities and available interface modalities at any given moment are largely if not entirely determined by the other networked objects around it*. If you pair the device with a text, it’s a reader; at the checkstand, it provides a friendly POS interface; aimed at the skyline, it augments reality.
Why this argument is so self-evident to longterm IxD folks and so relatively hard for anyone else to grok is, I believe, a function of the fact that we already take for granted the (rather significant) assumption from which it proceeds: that the greater part of the places and things we find in the world will be provided with the ability to speak and account for themselves. That they’ll constitute a coherent environment, an ontome of self-describing networked objects, and that we’ll find having some means of handling the information flowing off of them very useful indeed.
That the world, of course, looks nothing like this at present is a given. I do think it’s coming, though, as the marginal cost of instrumenting reality-at-large dives below the value derived from harnessing such dataflows in aggregate. Accept that, and the utility of an easy-to-use context-aware mediator like the one here depicted should become very clear indeed. (Inside baseball: let me make it absolutely clear, however, that I don’t believe anything like the semantic Web as its apologists currently understand it will ever exist.)
The second thing Mac got right is more subtle, and it’s a line about the evolution of mobile devices that I think is deeply correct. It’s that the device is of almost no importance in and of itself, that its importance to the person using it lies in the fact that it’s a convenient aperture to the open services available in the environment, locally as well as globally.
Mac happens to have interpreted this metaphor particularly literally, but there’s nothing wrong with that; it’s certainly a defensible choice. The business lesson that drops out of it, though – and of course I would think this – is that the crafting of an impeccable user experience is virtually the only differentiator left to a would-be player in this market, with clear implications for allocation of organizational effort and resources.
At any rate, I find Mac’s vision infinitely more convincing than another, far weaker near-future UI concept that’s been making the rounds lately. The idea of mobile device as context-sensitive pane isn’t even “futuristic” to me, to be honest: just a very convenient way to explain to the people around me what it is I believe we now have to build, and how, and why. Thanks, Mac.
[*UPDATED: After a second reading, I though I should make it clear here that a more usual definition within the field would stipulate reference to the state of the systems to which an application is coupled, whether technical, physical or social. See my (long) explanation here.]
The French magazine Chronic’art recently did me the honor of interviewing me on topics ubiquitous. I thought you might like to have a look at what came of it – if nothing else, it’s certainly revealing as to the differing perceptions of these technologies from one culture to another. I’ll caution you, though: it’s long. (I’ve provided links where I thought they might be useful.) Enjoy.
1. What is Ubimedia and how can you define it?
Well, let me first make it clear that the word “ubimedia” was the translator’s choice, and not my own. (I would have preferred “ubiciel” as a more direct translation of “everyware,” primarily because it avoids all of the unwanted resonances that inevitably come along with “-media.”)
Having said that, what I am trying to describe is a post-PC milieu in which many of the objects and surfaces of everyday life – tables, sweaters, bicycles, coffee cups, doorways, shoes and so on – have been endowed with the capacity to sense, store, process, display and transmit information.
To me, there are two clear and fundamental implications of this turn away from the “computer.” The first is that the menus, windows and mouses we rely upon in our interactions with PCs will no longer make any particular sense – that this class of computational devices and services will require an entirely new way of thinking about interaction and interface. And the second is that all kinds of ordinary situations, contexts and transactions we’ve never previously conceived of as being technical will be remade on the logics of digital information processing.
2. With [Everyware], you told us that, from a status of «users» we become «subjects», in every meanings of the word: a subject with an interiority, an experience, a subjectivity, but also like the «subject» of something, subordinated to something. This idea of an abandon by relegation creates a new environment where «machines» are in charge of all the context. Is this the achievement of a scenario described several times in Sci-Fi and in philosophy of the inversion of the domination man vs. machine?
I don’t believe this to be the case. All of the systems I describe in Everyware are still devised by some human institution or agency, in pursuit of their own all-too-human ends. Such domination as exists – and I have no doubt that there will continue to be domination – will be that of humans over other humans, merely achieved through technical ends.
I simply meant that PCs are something we use with full consciousness of our own agency: we choose to use them at times and places of our own choosing, and when we do so they are at the center of our attention. This is not at all necessarily the case with everyware, and describing the human being involved as a “subject” rather than a “user” is a way of highlighting the distinction.
3. Is [everyware] the achievement of a Terminator-like scenario: our enslavement by the machine?
Not by the machine, no – but perhaps by the logics inscribed in digital information-processing systems.
My concern is primarily that certain qualities necessary (or convenient) in the design of such systems – explicitness of social orderings, for example, or strongly binary representations of state – are incommensurate with the ways in which we ordinarily live our lives. Whether we come to regard this as “enslavement” will depend on the degree to which our everyday experiences are colored by the presence of such systems, and the quality of their design.
4. Can we say, in other words, that [everyware] is creating an urban environment like in Minority Report and what does it mean for you?
Everyware doesn’t “create” anything in and of itself; I think our experience of it, whether in the urban environment, the domestic, or wherever else, is strongly determined by the choices we make as societies, and to a lesser extent as individuals.
But if we look at the current choices our societies have made around technologies of surveillance and display, then, yes, I do believe it’s possible to predict some things about near-future urban environments in the developed world. It’s already possible, for example, to track people’s movements via their mobile phones, even when the phones are switched off; it’s only a matter of time before somebody fuses that capability with relational-database-driven “behavioral marketing,” and serves consumers with dynamic, targeted advertising much like that depicted in Minority Report. Certainly organizations like JCDecaux are acutely aware of the possibilities.
Similarly, the ostensible requirements of public safety in a post-9/11, post-7/7, post-Bali, post-Madrid world are already invoked as justifications for more (and more intimate) tracking and surveillance, and here too all the necessary technology is already in place. All of the event passes for the Beijing Olympics, for example, are provisioned with RFID tags, and the Chinese government has been reasonably forthright about expressing their interest in using this capability to track the movements of visitors to the Games, and even in modeling higher-level behavior based on what is known about patterns of those movements.
And how we might avoid the less pleasant implications of these scenarios? To my mind, that will have a great deal to do with the regulatory environment, and with the controls over such exploitation that we demand of our representatives in government.
5. Arthur C. Clarke said «any sufficiently advanced technology is indistinguishable from magic». Do you think [everyware] can lead to that kind of impression? Perhaps to visitors from the past…?
I’ve described the signature interaction pattern of everyware as “information processing dissolving in behavior,” where there isn’t any visible token of the elaborate transaction that is taking place between a user and one or more technical systems. And it’s true that – when everything works properly, anyway – there’s undeniably something magical about this, something effortless. In fact, that’s a large part of why you’d want to deploy ubiquitous systems in the first place, that quality of lightness they’d impart to everyday interactions.
So to a certain degree it makes sense that interaction designers are resorting to metaphors of magic, animism, ensoulment and so on, in their attempt to render the inherent complexity of these systems comprehensible to their users – see, particularly, the work of my friend Mike Kuniavsky. The trouble with this line of thinking, as far as I am concerned, is that it tends to be disempowering. It turns something which can and should be understood by the people using it into something more closely resembling “spooky action at a distance,” and leads to a belief on the part of users that the systems around them are “black boxes” whose inner workings are both occult and bound to remain that way.
Put bluntly, I don’t think this mystification is an acceptable outcome.
6. [Everyware] is restoring an «augmented reality», ie. the colonization of the tangible reality by the virtual one. In order to make the co-existing of objects of the physical world in the Ubimedia, you talk about the necessity of creating bridges between atoms and bits. RFID, bar-codes were created in this direction. Why is it important?
Well, if you’re interested in performing computational operations on things in the real world, whether those actions involve location in time and space or anything else, those things first need to have a representation in the virtual, an informational token that will stand in for them.
At this point, it’s trivial to endow just about any designed object with this capacity from the beginning. But there are all kinds of things in the world where for whatever reason it would be problematic to build this functionality in: cats, pallets of toothpaste, old cars…RFID and 2D barcodes are simple, low-cost ways to import such real objects into virtuality wholesale.
7. In your book, you talk about the Octopus system in Hong-Kong, the E-Pass in the United States, we can talk at least about the Navigo system in the Parisian underground. Do those experiences prefigure our [everyware] future?
These systems aren’t robust ubicomp, perhaps, in the Weiserian mode, but I do think that all the promise and all the peril of everyware are already fully present in them.
8. Mark Weiser and John Seely Brown have earlier predicted the «arrival of calm technology», i.e. the fact that the more computers penetrate our life, the more they have to disappear. Can you explain this paradox?
Firstly, I should point out that I never did share Weiser and Seely Brown’s optimism regarding this.
Their argument was that because, as a consequence of Moore’s Law, information sensing and processing devices would be everywhere in our daily environment, their designers would be compelled to make them “encalming as well as informing.” That, in other words, they’d be designed to a far, far higher standard than anything else we use. This is an argument predicated on an assumption of good faith on the part of all designers that I regard as thoroughly unwarranted, given everything we know about the process and the politics of information-technological development. I just don’t see any evidence to support it.
Further, as Yvonne Rogers has recently pointed out, in addition to its inherent impracticality, an “encalming” agenda may not even have been a desirable goal for ubiquitous systems in the first place; she calls, rather, for “proactive systems for proactive people,” and I think this is altogether a wiser and more practicable direction for the field. That the Weiserian vision is still inspiring to a great many researchers and designers is undeniable, but I don’t think it’s any longer useful as a practical setting-forth of agendas for development.
9. What about the New Songdo project, in South Korea, the «ubiquitous town»? Can you call to our mind its reality?
I must be honest here and admit that, in all the time since I first began doing research for Everyware in mid-2005, I have not seen anything that substantiates or makes good on the claims the developers and promoters of New Songdo have made for the city. There is, anyway, nothing available on the Web that might convince me that the emergent reality of New Songdo in any way resembles the vision set forth for it.
Whether this is due to budgetary or technical constraints or shortfalls or some other reason entirely is still unclear, at least to me. I think it does stand as a cautionary tale, though, to all those who think the “city of the future” is something which can be achieved in a single stroke.
10. From this point of view, is the ubimedia against the predictions of the 1990’s virtual reality gurus, or «cyberspace», as William Gibson described it in Neuromancer?
In a sense, everyware is simply cyberspace everted – turned inside out and superimposed over the ordinary. And, at least as far as I’m concerned, the ordinary everyday world is an infinitely richer and more interesting place than any virtual reality.
11. If we suppose we can resolve the interoperability issues by an unique standard, can’t we fear a bigger danger, ie. the arrival of a total control society in which every hint of personal life would be stocked and listed by a one and unique entity, capable of blending all databases (data-mining)?
To me, the primary concern presented by data-mining – more formally, online relational analysis, fused to new techniques of information visualization – is not so much that one central authority can come to enjoy “total informational awareness,” although that is a valid fear, as that we will all come to enjoy such awareness. That is truly novel and truly problematic, in ways that run much deeper than a merely techno-Orwellian scenario.
12. Is the privacy issue now obsolete in the [everyware] era? Can we accept this idea and adapt ourselves to it?
13. How do we manage our history and our image in a[n everyware] environment, where every hint of individual life are stocked and listed?
I see questions 12. and 13. as two facets of the same concern.
I don’t, myself, believe in the slightest that privacy is a quaint value, or one that it is no longer possible to support either technically or socially. The famous and by-now routine assertions that “privacy is history” and that we should simply “get over it” are, to my mind, cop-outs, betrayals.
What I do believe, however, is that some of the foundational notions that our social arrangements and our jurisprudence have been built on – the reasonable expectation that one is anonymous when in public space, for example – have turned out to be local to the particular state of technosocial affairs in which they originated, rather than eternal truths. So if we agree that things like privacy and anonymity are worth keeping, we’ll now have to fight for them.
Some people clearly do not feel that these are values worth upholding, that we should instead consciously architect “a future of openness, trust and mutual accountability” from the tools and components we now have available. And it’s foreseeable that there might in fact be some upside to such transparency – we might no longer demand of our celebrities and political figures, for example, that they meet some utterly unrealistic template of ethical purity and physical perfection. For myself, I tend to think that the unpleasant consequences of total transparency will tend to outweigh whatever nominal relaxation we enjoy of the invidious expectations we hold of each other.
As to how we manage the presentation of self in a world where there is such easy slippage between our various masks, I simply don’t know. Dishonesty is the great lubricant without which any society rapidly becomes unbearable – imagine having to disclose your actual feelings about all your friends, relatives, neighbors and coworkers to them, in real time – and unfortunately, the advocates of technologically-driven universal transparency seem to persistently undervalue its utility as an aid to social cohesion.
14. In economical terms, the ubimedia means a battle for the «computing colonization of everyday objects» between the big names of the high-tech industry. What is the situation nowadays ? What are the most implicated companies in this strategy? Videogames developers like Sony and Microsoft? Internet’s giants like Google or Yahoo?
I wouldn’t say that any of these organizations have made any particularly interesting inroads with regard to objects we encounter in everyday life – with the partial exception of Google on mobile devices, which does profoundly affect the nature of conversations or situations in which it’s invoked.
We can look, instead, to services like OnStar, which is one of the very few interesting and successful things that General Motors has done in the last quarter-century. Under the influence of OnStar, GM increasingly understands the passenger vehicle as a sensor and a computational platform. Sadly, it has yet to affect the way they design or produce or distribute their cars, but it unquestionably represents the endowment of something familiar with information-processing power.
I might also mention Nike’s Nike+ iPod Sport Kit, which in effect transforms an ordinary running shoe into a biometric monitor and enables it to radiate performance data to the World Wide Web, where that data becomes a social object and a social pivot. Crucially, I understand that most of the interaction design was done by Nike rather than Apple, which suggests to me something of where the relevant skills involved in integrating everyday life with everyware are going to come from.
15. On one hand, the arrival of an unique standard would obviously favoring the development of [everyware]; on the other hand, this would mean a new monopolistic situation, like the Microsoft’s one and that is the most feared thing. Is this discrepancy the major problem of [everyware’s] development?
There is certaintly an argument to be made that in the realm of ubiquitous computing, the power – the grandeur – lies with the party or parties who determine and dictate the technical standards for the interoperability of components, systems and services.
But the beautiful thing is that no one organization seems any longer to be able to compel users’ acquiescence to their mandates. Microsoft and Sony have each shot themselves in the foot on more than one occasion by insisting that their users buy into some closed and proprietary standard that nobody had asked for: do a search for “ATRAC” or “PlaysForSure” if you doubt it.
After humiliating and expensive blunders like these, I’d hope that even the Microsofts and Sonys of the world come around to thinking that the future belongs more to open standards, as rigorously defined by Ken Krechmer – if for no other reason that any distributed system presenting as large an “attack surface” as the kind of everyware we’re talking about truly needs the enhanced security that goes hand-in-hand with openness.
And openness, in turn, acts to minimize the influence of large organizations. We’ve seen, for example, that where Web services are concerned, it’s independent developers using or promulgating open standards that have led the way. I don’t expect ubiquity to play out any differently.
16. You talk about an imposed «multiplicity» the ubi-computers developers would hardly manage. In this point of view, might [everyware], in the first place, increase some conformity, some unbearable consensus?
I think we’re getting two separate issues tangled up here. The first is the heterogeneity of the discrete technical systems that comprise the ubiquitous ecosystem, which is what I’m primarily referring to when I speak of “multiplicity.” The everyware milieu is comprised, in its total extent, by your mobile phone, by entirely ordinary desktop and laptop PCs through which information enters the network, by devices like Nike+, by RFID-mediated urban interfaces like Oyster or Octopus or EZPass…all of these systems are points of entry for information that can then be grabbed and manipulated and acted upon by other systems.
In the past, it was felt that this heterogeneity constituted a hard problem for the would-be developers of ubiquity. For example, a building-management system might rely on one centralized computational process, with limited resources available to track and manage all these sorts of assets and respond to their comings and goings, their crashes, their sudden availabilities and equally abrupt ruptures. I’m the furthest thing from an expert in computer science, but as far as I understand it, this is no longer thought of as being particularly problematic, particularly as computational resources, actual processing cycles, are now distributed across a networked “cloud” of devices.
The second issue is one of the heterogeneity of development organizations. Especially as each of the components that feed the ubiquitous ecosystem become “lighterweight,” simpler and cheaper, the number of people and organizations able to participate in building them explodes. I’ve in the past described this diversity of developers as simultaneously everyware’s greatest hope and its greatest weakness – it lends the ecosystem as a whole a much-needed flexibility and “hybrid vigor,” but it also makes it infinitely more challenging to achieve any consensus as to what might constitute “ethical” or “responsible” development.
17. Are iRoom and, generally speaking, the «event heap» model, answers to this multiplicity challenge? What is that?
These are obsolete models. As I mention, we’d be more likely, these days, to speak of “cloud computing.”
18. You talk in your book about the body as a databank and not anymore as a receptacle of impressions (the «SenseWear» patch of Bodymedia). Does the induction of machines transform the nature of man?
No, I don’t think anything quite so profound. I do think, however, that it changes our behaviors – what we do with our bodies, and why, and how – in quite significant ways.
My own personal example has to do with Nike+. Formerly very easy-going as a runner – a devotee of running as a kind of moving meditation and a space of pure relaxation – I have become quite compulsively competitive now that my performance statistics are captured and radiated to the World Wide Web. It’s no longer acceptable, somehow, that I’m forty-five seconds slower than my friend over the course of a mile, even though I’ve five long years older than him; it’s no longer entirely tolerable that anybody in my Nike+ orbit puts in more weekly miles than I do. And what’s more, I no longer run at all unless I have my Nike+ with me – why would I, when that would be an uncaptured and thus a “wasted” effort?
As you’ll probably have inferred from the way in which I’m discussing this turn of events, I’m deeply ambivalent about it. Unquestionably, I run more, and faster, than I did before I opened my life to networked biometrics. But it’s not quite the same as it was before. It means something different, and it feels different.
19. You recommend a flashback system (the famous CTRL-Z) in every utilization of the ubimedia, in order to avoid any irreversible disaster. An option so obvious we hardly imagine it doesn’t exist on every ubiquitous system, present or future. Does this «second chance» option make life like a videogame where it’s always possible to start again a fresh game?
But this is already the case: there’s a guy named Mitchell Wade who has some really good data on the generational divide between those of us who grew up before and the advent of widespread videogaming. His findings demonstrate – to my satisfaction, anyway – that people who grow up gaming are far less risk-averse than those of us who did not.
At any rate, I’m not particularly worried about making things too soft and forgiving by introducing such functionality. We can already see that the hassle of clumsily-devised everyware is likely to be so significant that any mitigation we can provide as designers will be both merciful and welcome.
20. Another solution: you talk about the possibility to say no to [everyware], say «no» to a system we don’t want it to manage things for us. Isn’t it a little bit naive? Or pragmatically unmanageable?
In public, yes, it is already almost certainly too late, as we’ve seen: try moving through an environment like central London without having your traces captured and parsed! But in private space, in our own homes at the very least, I continue to maintain that we should be able to shut it all down at will.
Whether someone who choses to opt out in such a manner, at least for any length of time, will be able to meet emerging social criteria for “performance,” will be able to keep their nostrils above the waterline in the ferociously competitive world we seem to be building, I can’t say. That’s a question for all of us to answer, at the level of the societies we belong to, and it’s beyond the ability of designers to affect.
21. Isn’t it a little bit utopist to imagine we can master or orientate our ubiquitous future? Aren’t machines out of control yet?
Again, I think that we’re a long way from needing to worry about autonomous machines. I’m much more concerned about various social and political agendas being advanced via ubiquitous technology, and therefore appearing neutral and “natural.”
22. Can we imagine a life separate from this ubiquitous world you describe? Isn’t this a new form of luddism?
No, not at all. I believe that it’s meaningless to say that one is “pro-technology” or “anti-technology” – the idea is absurd to me. There are, instead, discrete technologies, and we can each of us choose which we desire to invite into our lives.
That is, those of us who are empowered to choose. Not everyone will be so lucky. What I see happening is a “digital divide” much more pervasive and worrisome than that which ordinarily concerns commentors on technology. Generally, we see this divide depicted as one where the wealthy have access and the disadvantaged and those without resources lack it. What I see everyware doing is turning this supposition on its head, so that it is the wealthy, the privileged and the powerful solely who are able to abscond from visibility to the network.
We already see some signs of this in the mobile space: Blyk‘s business model, for example, points pretty clearly to a time when the less-privileged have access to advertising-supported “free” connectivity that comes with strings very much attached, while those who can afford it are subject to no such restraint.
23. [Everyware] means also the arrival of order and precision (on what machines are built) in our world made of chaos, disorder, uncertainty…How do you explain this paradox? What can be the consequences? Can we talk about the inversion of the dominating vs. dominated model, the mastering vs. loss of control one?
We know this about digital systems, that their representations of the world and of situations in it are binary, granular, stepped: on or off, one or zero, yes or no. By contrast, when I think of something like my feelings for my closest friends as they’ve evolved over time, they’re in constant flux, inchoate, inarticulate. In short: exceedingly difficult to represent as the state of a digital system.
So when I contemplate the specter of all the profoundly analogue relations of everyday life remade as just that, I cringe. I think, particularly, of Deleuze’s “Postscript on the Societies of Control,” where discipline itself has become nomadic, something immanent in the world. But, again, I don’t see this as domination by machines so much as it is the subordination of will and desire to a set of inappropriate logics that have nothing to do with the way the human sensorium and psyche experience the world.
24. Doesn’t those ordinate systems favor and increase disorder in the real world, as everything influences everything everywhere (Butterfly effect or «event heap»)?
I would not be in the slightest surprised to see catastrophic errors and defaults cascading through poorly-devised ubiquitous systems.
25. Bernard Stiegler talks about a «global mnemonic system» to underline the fact that, in the future, nothing will be isolated from anything. What is this environment made of information always accessible from anyone? Are we going to be better informed? And what this environment will change for us?
What we’re contemplating is a world where just about any human activity undertaken generates its own discrete data trail, which is then subject to aggregation, analysis, persistent storage, representation and an output which is in turn potentially located just about anywhere. I find it unthinkable that this would not transform our epistemologies both formal and personal.
Will we be better informed? In some shallow sense, yes. My wife, my friends and I already interrupt our conversations to Google an unfamiliar name, or look something up on Wikipedia: it’s what the iPhone is for. But as I understand him, anyway, Stiegler’s concern is with the exteriorization of knowledge and with the consequences of that exteriorization for the recuperation of desire. That’s a much deeper question and I don’t think we as yet have any solid sense yet what either our psyches or our societies come to resemble under such conditions.
26. Why isn’t [everyware] real yet? What restraint its development?
I would argue quite strongly that it is real – that for all that it may not particularly resemble what Mark Weiser had in mind, everyware is already all around us, hiding in plain sight. I hate to resort to the Gibsonian cliché, but I believe that we may not necessarily perceive it as such because it’s “not yet evenly distributed.”
27. One difficulty of [everyware], you said, is that people don’t see it as useful, they don’t need it. Can you explain that?
Sure. It’s the gulf between the way people ordinarily perceive their everyday needs, and the kind of thing a development organization might contrive as part of a “business requirements analysis.”
Nobody says to themselves, “Hmmm, what I really need is a ubiquitous household management system.” They might say “I’d like it if all of the digital devices in my life to spoke to one another.” But far more likely is for people to express their needs and desires in high-level terms, like wanting to know when traffic is particularly bad, how to get to the restaurant where they’re meeting a friend, where their children are at the present moment, where they stashed last winter’s scarf and gloves when the weather turned pleasant…
It’s the task of the designer to understand desires like these, to use all of the empathy and insight they have at their disposal to produce systems and services that allow people to fulfill their desires without needing to attend overmuch to the tool itself.
28. You define five basic principles, five warnings in order to develop safely [everyware]. We easily imagine you know the laws of Asimov…Do you think a supranational organization is possible, in order to enforce those rules and avoid bad surprises?
Given that only the tiniest minority of those developing ubiquitous systems do so in consciousness that this is in fact what they’re doing – they are far more likely to concieve of themselves as developing modular, discrete componentry, that’s only after the fact laminated into a distributed, networked system – it’s hard to imagine that any such call will prove to be terribly effective. Nevertheless, I think it’s valuable simply to set baseline expectations for what we might call an etiquette of ubiquitous interactions, and I myself would certainly like to see some group of developers take this challenge on.
29. What kind of organization could enact and enforce those rules?
I think we’ve already seen a few examples of an organizational topology that would work: open-source development, the rise of Wikipedia, what Clay Shirky might call “organizing without organizations.” I also generally use standards-based Web development as having set a felicitous precedent for global standards arising out of the work of activist developers.
30. We see more and more evolving machines whose functions evolve in time, and we think ubimedia will nurture those developments: the roles of machines will change from the ones imagined by their developers (from service functions to control functions, or matching information functions, on behalf of a State or companies). How to manage and control this?
[Must have missed this question – I don’t see that I answered it. : . ) ]
31. For Norbert Wiener, when humans are in front of machines, they act as they have an internal force. Some point of view later developed by Byron Reeves and Clifford Nass in 1996 in The Media Equation (people act as computers were humans). We can see this in 2001: A Space Odyssey (HAL 9000), Ghost In The Shell or The Matrix. How do you analyze this need for humans to identify themselves to machines? What does it reveal?
I can’t speak to what it might mean philosophically, I’m just not qualified. Empirically, however, people unquestionably seem to have some deep-seated need to impose an anthropomorphism on the artifacts around them – and not even digital ones: Fabien Girardin and Nicolas Nova, in their forthcoming pamphlet Sliding Friction, demonstrate that frequently all that is necessary is for “eyebrows” to be added before we perceive some piece of machinery or architecture to have a personality.
As Reeves and Nass showed us, though, the more interactive an artifact is, the harder this anthropomorphism is to escape. We ascribe emotions to interactive artifacts, we treat them as social beings and coevals. What it implies for the designers of everyware is that, again, it might not be such a bad idea if the devices around us observed the same standards of social concourse that a well-socialized human adult would: discretion, tact, implicit forgivenness of trivial blunders, and so on. It’s long past time, in other words, for the designers of interactive systems to acquaint themselves with the social-psychological literature; I generally give my students Goffman to start with.
32. In the same direction, you see that machine is perceived as a «social tool». Isn’t this a lure? An overestimated function of the machine which reveals more a lack of knowledge and control of the computing object? Some inferiority complex?
I don’t think it’s so much an inferiority complex as it is the misallocation of resources, ambitions and desires. Let’s use digital information technology to do what it does best, and not install it where it can only do harm. I see the social sphere, generally, as one of the latter circumstances – see my post called “Antisocial Networking” for a much more thorough treatment of this theme.
33. We see two opposites views on new technologies between West and East. The first is more about technique (optical fiber, high quality broadband connection…) and is still bind to fix PC. On the other hand, Japan is all about mobility, as we see in the success of mobile phone (the «keitai» or «ubiquitous communicator»). Is [everyware] more able to flourish in Asia than in the West?
The stereotype is that audiences in East Asia are on the one hand somehow less concerned with individual privacy, and on the other hand more invested in notions of technical progress on the national level. And it’s true that in places like South Korea, Japan and Singapore we have these very high-level ubiquitous initiatives on the part of government, very tightly coordinated with manufacturers and their marketing activities.
I question, though, whether these initiatives ultimately do any good. My sense is that they tend to lock developers into prematurely-calcified visions of what ubiquity is or could be, and have rather less to do with anything that people actually want or need. On the other hand, getting around Hong Kong or Seoul with their RFID-enabled transit systems is a pleasure that no metropolitan system in the United States can remotely match, so it may well be that there’s something to be said for the coordinated approach.
34. In this direction, does [everyware] have many ways of developing itself? Many ways of access?
I believe that there are and will continue to be as many everywares as there are local cultures that adopt it. If nothing else, in very many places around the world, the desktop computer has never been the primary means of accessing networked information, so it’s foolish to assume that the PC will loom particularly large in the future of these local conditions.
What I believe to be very foolish are comments like those of Qualcomm’s COO, Sanjay Jha, that the pervasive Internet experience somehow “needs” to resemble that which we’re familiar with from the desktop. Here is a place where a hundred flowers will most assuredly bloom, each exquisitely adaptive to its own time, place, audience and context.
35. Technique isn’t neutral, as we see with Heidegger, Habermas or McLuhan. For the last skeptics, isn’t [everyware] the ultimate proof they’re right?
This is a live question for me.
On the one hand, I think it’s beyond question that technical systems are generally presented as neutral and universal, in such a way as to disguise the valuations embedded in them, where you already know that I believe these valuations to have an outsize impact on our freedom to choose and experience.
But I’m also sympathetic to Bruno Latour’s point, pushing back against some of the things that Langdon Winner has asserted, that artifacts can not be said to have politics in any meaningful sense. His argument is that, whatever political desire is inscribed in the design of an artifact, the set of circumstances those artifacts are deployed in are invariably so dynamic as to undermine the designer’s intention in short order. I suppose we’ll have to wait and see who proves to be more correct in the ubiquitous context.