It is important to realize that before Like.com was helping people find shoes and watches online, its technology was the core of Riya, a photo sharing and search site that allowed users to upload, tag and search images based on facial recognition technology. Users simply uploaded their photo library to Riya, tag faces in a handful of the photos, and then Riya’s facial recognition technology took over and attempt to automatically tag different faces it “recognizes” so that you don’t have to. Riya’s vision was to expand this facial recognition beyond an individual’s library, and eventually reach every digital photo scattered across the Web.

Over 4 years ago, I wrote about the privacy implications of Riya’s facial recognition technology here and here. I warned then:

Sure, it would be great if I could easily search for pictures of me on the web, but do I want others to be able to do that as well? Could this provide a useful tool for stalkers looking for a particular person? Abusive partners trying to track down their victim? (Note that Riya also uploads and indexes all the metadata related to your photos, including the date and time is was taken, when it was uploaded, etc. Users can also tag and search photos based on location). …there are externalities once all of the images of our daily lives (and their related metadata) are uploaded to the Internet, indexed, searchable, and accessible to all.

Google almost bought Riya back in 2006, but acquired Neven Vision instead, and soon integrated basic facial recognition into its image search results, and, later, into Picasa and Orkut.

With the purchase of Like.com, Google obtains the original Riya technology. Coupled with other recent social networking related acquisitions, Google appears poised to make a run at Facebook, with Riya providing the means to automate photo tagging at a scale Riya could only have dreamed of. What remains to be seen is how carefully Google will consider the privacy implications of unleashing powerful face recognition technology on throngs of users.

Face.com is launching a facial recognition application called Photo Finder to allow Facebook users to search their photos — and photos of their friends — to learn, recognize, and tag familiar faces. There are numerous stories on this launch (each seemingly building from the same press material, each magically offering 100 invites to the private alpha of the app), and the details can’t be confirmed, but essentially the app will scan all your photos and those in your social graph and automatically tag you and your friends. You can then confirm, deny, or edit these tags, and even set up a watchlist to notify you when the app finds new photos of you (or a friend).

This kind of facial recognition system isn’t new, even in social networking contexts (see Polar Rose). But Face.com’s particular approach has some unique features. Photo Finder claims to fully respect existing Facebook privacy settings. It only scans photos that have been made viewable to you, and only allows your friends to scan the photos you allow them to see. Further, the tags generated by Photo Finder are only viewable in Photo Finder, and only be your friends who are also using Photo Finder. They do not become co-mingled with Facebook’s own tagging system.

While these features help mitigate some of the initial privacy fears of having the ability to automatically scan and recognize photos on Facebook, it doesn’t absolve all my concerns.

As I’ve written elsewhere, the deployment of robust face recognition tools for use by everyday people on social media platforms presents unique privacy concerns. Previously Facebook gave me some (but limited) power to control the searchability of my likeness within the millions of photos they host. If someone uploads a photo of me and decides to tag that image with my name, I can be easily searchable. However, I am notified of this, and  can remove that tag. While the photo remains, I gain some “privacy via obscurity” since few will know that photo exists and that I’m in it.

But with systems like Photo Finder, photos containing my likeness can be more easily discovered, and I receive no notification when someone has identified me using the system (unless I’m also automatically scanning new photos to find my face). Photo Finder works to reduce the information friction that might have prevented people from finding random photos of me on the social Web.

It is good that Face.com has designed Photo Finder with some privacy-protecting features, but we must not be lulled into thinking that no privacy threats exist.

• Scientific American has a nice special issue dedicated to “the future of privacy.” Nothing new here for most privacy scholars, but it is a nice treatment of the issues that is approachable to those who don’t spend every breathing moment thinking about privacy and surveillance theory. (Also very good for undergraduate courses!)
• Colorado Law School professor Paul Ohm has released an important new article on “The Rise and Fall of Invasive ISP Surveillance,” where he argues that “Nothing in society poses as grave a threat to privacy as the Internet Service Provider.”
• Google released a new version of Picasa, that now includes facial recognition technology to help you identify friends and family in your pictures without requiring you to tag them by hand each time you see them. Similar to Riya, Picasa’s facial recognition technology will ask you to identify people in your pictures that you haven’t tagged yet. Once you do and start uploading more pictures, Picasa starts suggesting tags for people based on the similarity between their face in the picture and the tags you already put in place for them. (I’ll blog more about this separately soon.)
• Google also released its own Web browser, Chrome. Many saw conspiracy when Google made the (bone-headed) mistake of simply copying its standard EULA to the Chrome site, which erroneously stated that users grant “Google a perpetual, irrevocable, worldwide, royalty-free, and non-exclusive license to reproduce, adapt, modify, translate, publish, publicly perform, publicly display and distribute any Content which you submit, post or display on or through, the Services.” Of course, Google confirmed this was an error, and changed it. Meanwhile, others showed great concern over whether Google would be snooping on browser activity. Lauren Weinstein and Google’s own Matt Cutts helped diffuse those concerns.
• Speaking of browsers, the new version of Internet Explorer (IE8) includes an “InPrivate” mode that lets users control whether or not IE saves their browsing history, cookies, and other potentially sensitive data. This is in line with Safari’s “Private Browsing” feature, but has not avoided all criticism.
• As an iPhone user, I’ve often wondered whether the device “phones home” and what kind of usage statistics might be be shared with Apple. (Recall how the Mini-Store iTunes update from a few years ago caused a stir due the automatic transmission of users’ listening habits to Apple.) Turns out that the iPhone does take periodic screenshots of everything you do in order to make that “shrinking screen” effect work when you press the home key. While presumably that image isn’t stored or transmitted, Wired points out the larger concern: “The phone presumably deletes the image after you close the application. But anyone who understands data is aware that in most cases, deletion does not permanently remove files from a storage device.” Apple should make transparent how this works, where these images reside on the phone, and the process under which they are deleted from memory (including the cache).
• My friend and colleague Anders Albrechtslund has published an excellent article Surveillance and Ethics in Film: Rear Window and The Conversation in the Journal of Criminal Justice and Popular Culture 15(2), pp. 129-144.
• Finally, I’m sad to hear that Sivacracy is going on indefinite hiatus. The silver lining here is that Siva is inching closer to completion of his book The Googlization of Everything.

It seems Google is starting to take advantage of their acquisition of the facial recognition company Neven Vision a while back. First step to distinguish a face from a non-face. Next step perhaps to distinguish between faces? Then what?

This post is an attempt to collect and organize some thoughts on how the rise of so-called Web 2.0 technologies bear on privacy and surveillance studies. After presenting a few examples of unintended consequences of Web 2.0 that bear on privacy and surveillance, I will introduce the term “netaveillance,” which might provide a useful concept around which a more robust theory of surveillance about the Web 2.0 phenomena might be built.

The rhetoric surrounding the Web 2.0 movement presents certain cultural claims about media, identity, and technology. It suggests that everyone can and should use new Internet technologies to organize and share information, to interact within communities, and to express oneself. It promises to empower creativity, to democratize media production, and to celebrate the individual while also relishing the power of collaboration and social networks. Websites such as Flickr, Wikipedia, del.icio.us, MySpace, and YouTube are all part of this apparent second-generation Internet phenomenon, which has spurred a variety of new services and communities – and venture capitalist dollars.

This cartoon of a room full of people arguing at a cocktail party after someone mentioned the provocative theories of Marshall McLuhan reminds me of today’s emotional debates over the relative impact – and even the very existence – of Web 2.0. Many hail Web 2.0 as the “new wisdom of the web,” and “a new cultural force based on mass collaboration,” while others deride it as merely a marketing jingo, “amoral,” and even an extension of Marxist ideology.

This last notion, the relationship between Web 2.0 and Marxism, was suggested by Andrew Keen, one of the loudest provocateurs of the Web 2.0 ideology. Keen has received considerable criticism for making comparisons between the Web 2.0 meme and Marxism, but, between the vitriol, he does make some valid points about the utopianism and solipsism that seems to underlie much of the Web 2.0 discourse. In particular, he criticizes the fervent commitment to technological progress:

The ideology of the Web 2.0 movement was perfectly summarized at the Technology Education and Design (TED) show in Monterey, last year, when Kevin Kelly, Silicon Valley’s über-idealist and author of the Web 1.0 Internet utopia Ten Rules for The New Economy, said:“Imagine Mozart before the technology of the piano. Imagine Van Gogh before the technology of affordable oil paints. Imagine Hitchcock before the technology of film. We have a moral obligation to develop technology.”

But where Kelly sees a moral obligation to develop technology, we should actually have–if we really care about Mozart, Van Gogh and Hitchcock–a moral obligation to question the development of technology. [emphasis added]

This moral obligation to question the development of technology compels Keen to identify some of the unintended consequences of the emergence of Web 2.0 infrastructures, including the flattening of culture, the overabundance of amateur authors and producers, and narcissism run wild.

As I begin to study the Web 2.0 meme from the perspective of privacy and surveillance theory, a different set of unintended consequences emerges, including shifts in the flow of personal information that might threaten personal privacy in ways much more damaging than Keen’s concern that content is now made and distributed by mere amateurs instead of honed professionals.

For example, Web 2.0 applications often rely on rich metadata to create value in information, such as the geotagging of images uploaded to Flickr. While it might be useful and fun to have locational data automatically associated with your images, considerable privacy concerns emerge as an externality. For instance, law enforcement officials can simply search for all photos online matching the location & timing of a certain political rally in order to broaden their ability to keep records of who was present. Or, combined with the development of facial recognition technologies with shared online photos, stalkers (or other annoying folks) might soon be able to search for a certain person’s face, and discover the GPS coordinates of the coffee shop they seem to be pictured in every Tuesday morning. Someone even developed a tool, FlickerInspector, to facilitate this kind of mining of the datastreams users leave behind on Flickr.

Of course, one doesn’t need a fancy application like FlickerInspector to reap the benefits of the new datastreams facilitated by Web 2.0 applications. Inherent in Web 2.0 evangelism is an overall faith in the network to be the processing platform: users are encouraged to put as much of their lives as possible online, to divulge and share their personal lives, their professional development, their favorite websites, their music, their friendships, their appointments, and even where they’ve connected to wi-fi. If you know a person’s “handle” on one Web 2.0 site (“michaelzimmer” at del.icio.us), you probably can find them on many more (Plazes, LibraryThing).

The prevalence of sharing so many details of one’s life through various Web 2.0 and social networking sites, and the relative ease of finding users across these services, leads to a second key externality: the rise of amateur data-mining. Fueled by the power and reach of Web search engines, it seems anyone can now engage in the kind of tracking and data-mining of user’s online activities that was once possibly only by the most powerful of computer systems.

An interesting case of amateur data mining made possible through Web 2.0 involves “Don, the camera thief.” The blog BoingBoing posted a story of a woman who lost her camera while on vacation, but was contacted by the family who happened to find it. Unfortunately – and oddly – the family who found it refused to return the camera because their child liked it so much. BoingBoing thought the actions by the finders of the camera were “shameful.” A few days after posting this, BoingBoing received an e-mail from someone who claimed his name was “Don Deveny,” purportedly a Canadian lawyer, who implied that the post was illegal and that BoingBoing was liable for making it. The folks at BoingBoing doubted the legitimacy of the email (the word “lawyer” was misspelled, for example), and decided to see what he could find out about “Don.”

They first contacted many of the law societies in Canada, none of whom had any record of a “Don Deveny” licensed to practice law in Canada. (by the way, it is illegal to pretend to be a lawyer). From their e-mail exchange, they were able to isolate the writer’s real e-mail address from the message headers, and through a Google search, located other pages that contain that address. That led them to a profile page for a user of the website called “Canada Kick A**” who shared the very same e-mail address. That profile page had a different person’s name (perhaps “Don’s” real name?), and also listed a location and profession for the user (he’s not a lawyer). It didn’t take much to figure out (or at least get a better clue) as to who this e-mailer was, and his profile page on a Web 2.0-inspired discussion board made it much easier.

Readers of BoingBoing did some amateur data mining of their own: a commenter at the original camera owner’s blog seemed to share many of the same sentiments of “Don,” along with many of the same spelling errors. This commenter used a different screen name, but when asked to identify himself, also said he was a lawyer. Another reader then discovered that a user with that same screen name recently bid on memory cards at eBay that would have been used in the stolen camera. More amateur data mining ensued, and discovered another user profile at a different discussion forum with the same user name and same “favorite sites” listed in the signature file. And this page included a photo of the user: Is this “Don” our camera thief?

Another example of the ease of amateur data mining with the help of Web 2.0 services is the outing of Lonelygirl15. Lonelygirl15 was the mysterious girl leaving video confessions on YouTube, garnering a huge following of devoted fans, yet know one knew who she was or if they were really just a kid’s video diary or perhaps a large hoax or advertising campaign. After some amateur data mining, the truth came out:
A reader was surfing an article on Lonelygirl15 at a random website when he came across a comment that linked to a private MySpace page that was allegedly that of the actress who plays Lonelygirl15. Since the profile was set to “private,” very little information one could glean from the page. However, when he queried Google for that particular MySpace user name, “jeessss426,” he was able to access Google’s cache from the page a few months ago when it was still public. A lot of the details of the girl’s background quickly emerged: She was an actress from a small city in New Zealand who had moved to Burbank recently to act. The name on the profile was “Jessica Rose.” When he happened to query Google image search for “Jessica Rose New Zealand” he was instantly rewarded with two cached thumbnail photos of Lonelygirl15, a.k.a. Jessica Rose, from a New Zealand talent agency that had since removed the full size versions. A search on Yahoo for “jeessss426” also turned up various pictures from her (probably forgotten) ImageShack photo sharing account. Lonelygirl15 was revealed.

Little effort was needed to link up the various e-mails, user names, personal data flows, and photos shared across blogs, discussion forums and other Web 2.0-style sites to track down “Don the camera thief” or “LoneyGirl15”. Moving more and more of our activities to Web 2.0 makes it harder to remain anonymous, and the myth of “security through obscurity” seems to be disappearing as various crumbs of our true identity are being scattered across the Web 2.0 landscape.

A final externality of Web 2.0 relates to a new form of informational voyeurism that these platforms enable. While Web 2.0 sites have enjoyed incredible growth and heavy viral participation, only a small fraction of overall users actually use the services to upload content – the vast majority just likes to lurk and watch. According to one report, only 0.16 percent of YouTube’s total traffic is made up of users who upload videos. Similarly, only 0.2 percent of Flickr’s regular users are there to upload photos. And slick new tools emerge daily to facilitate the surveillance and voyeurism of people’s daily activities. For example, “feeds” on Facebook allow users to be notified immediately when a friend updates their profile (changing their mood, their friend list, their relationship status, etc), dodgeball helps users find friends (and unknown friends of friends) within a 10 block radius of their present location, DiggSpy allows real-time monitoring of user’s activities on the popular news ranking site Digg, and Twitter has quickly emerged as the hottest new voyeuristic service, allowing users to share text snippets of their day-to-day activities, and monitor others’ streams of the mundane details of their lives (such as “a whole gang of women with dogs just walked past my window”).

What seems to be emerging is a new form of voyeuristic surveillance of people’s everyday lives, fueled by Web 2.0. This has been referred to varyingly as “peer-to-peer surveillance” or even as a new kind of “participatory panopticon.” Yet these terms – and the theories embedded within them – seem insufficient to fully grasp the significance of the emergence of this new voyeurism of the mundane. Surveillance, of course, implies the “watching over” of subjects from above, with an explicit power relationship between the watchers and those placed under its gaze. Trying to describe surveillance as “peer-to-peer” suggests a flattening of the power relationship that is counter to its very definition. Similarly, the notion of a “participatory panopticon” is at the same time redundant and contradictory. Foucault revealed how panoptic power becomes internalized by the subjects, thus, they necessarily “participate” in their own subjugation. Yet the top-down power relationship within the panoptic structure remains. The participation by the subjects does not make them equal with the watchers. Yet the informational voyeurism associated with Web 2.0 seems to imply a balance between the users: one shares their data streams in order to improve the overall worth of the network, coupled with the presumption that they’ll be able to observe and leverage others’ streams as well.

This notion resembles that of “equiveillance,” a state of equilibrium between the top-down power of surveillance, and the resistant bottom-up watching of sousveillance. Yet, this notion implies merely a balance in access to surveillance information, and is focused more on how to reach some kind of harmonious relationship with our rising surveillance society. With the informational voyeurism of Web 2.0, however, the goal isn’t to resist or come to terms with the power yielded by traditional surveillance, but rather to participate in a widespread and open sharing of the mundane details of one’s daily life. To give one’s peers a glimpse into one’s own personal universe.

These snapshots of the minutia of people’s lives have been compared to the Japanese concept of “neta”, the tidbits of people’s lives that are shared with family and friends as a kind of social currency. The Japan Media Review (an affiliate of Annenberg’s Online Journalism Review) recently made an insightful connection between “neta” and Web 2.0 voyeurism:

In Japanese, “material” for news and stories is called “neta.” The term has strong journalistic associations, but also gets used to describe material that can become the topic of conversation among friends or family: a new store seen on the way to work; a cousin who just dropped out of high school; a funny story heard on the radio. Camera phones provide a new tool for making these everyday neta not just verbally but also visually shareable.

As the mundane is elevated to a photographic object, the everyday is now the site of potential news and visual archiving. Sending camera-phone photos to major news outlets and moblogging are one end of a broad spectrum of everyday and mass photojournalism using camera phones. What counts as newsworthy, noteworthy and photo-worthy spans a broad spectrum from personally noteworthy moments that are never shared (a scene from an escalator) to intimately newsworthy moments to be shared with a spouse or lover (a new haircut, a child riding a bike). It also includes neta to be shared among family or peers (a friend captured in an embarrassing moment, a cute pet shot) and microcontent uploaded to blogs and online journals. The transformation of journalism through camera phones is as much about these everyday exchanges as it is about the latest headline.

Building on this Japanese concept of “neta,” I propose a new kind of “veillance” has emerged with Web 2.0 infrastructures: “netaveillance”. Netaveillance can be defined as the process of openly and purposefully providing an almost continual stream of the details of one’s daily life – the mundane, the profane, and the vain – through Web-based technologies, coupled with the ability to capture similar data streams from one’s peers. Netaveillance constitutes an emerging ecosystem of personal data flows – not the exceptional information meant to be protected from state or commercial surveillance, but the free and open sharing of the minutiae of our lives.

My conceptualization of netaveillance is, to be sure, in its most nascent of stages. Much work needs to be done to contemplate how it relates to existing theories of privacy and surveillance, how power relations between and among participants might still exist, how such data flows could be captured by state or commercial interests, and so on. Theorizing and understanding netaveillance is no small task, but it might provide a new language and framework from which to understand the informational voyeurism and related unintended consequences of the Web 2.0 phenomenon.

Whether you want to bring it up at a cocktail party is up to you.

Polar Rose and future developments that make facial recognition available to the masses risk encroaching on people’s privacy, warns Yaman Akdeniz, director of the UK non-profit group Cyber-Rights & Cyber-Liberties.

“Although this sounds like a great idea, I would not like to be searchable in this way, or so easily tracked without my consent,” says Akdeniz. The database compiled by Polar Rose is similar to the kind of biometric database some governments wish to use, he points out.

“I wonder whether they have a right to build such a database,” says Akdeniz, he suggests people think twice before embracing such potentially intrusive tools, and consider which photos of themselves they allow online.

Others agree. Simon Davies, director of the campaign group Privacy International and a specialist in technology and privacy at the London School of Economics, UK, says face-searching technology is valuable but must be used responsibly.

He fears Polar Rose could help identity thieves or stalkers, or even be used by the police to monitor protesters. “They could use the service to find where people have been, what their activities are, or who they associate with,” he says.

Search engines should allow users to prevent their photos being searched, says Davies. “There should be a way to put code in a webpage that signals you want to opt out,” he told New Scientist.

I don’t have time to add much more, but I’ve blogged about the privacy and surveillance concerns with widespread adoption of facial recognition search engines here, here, and here.

[via Pogo Was Right]

…if social networking is the future of online communication and Google is going to use Social Network profile information to personalize their search engines, but does not have a way of identifying the owner of the profile on some social networks due to no advertising in the network or no Google partnerships, what is one object or identifer that most users or members of Social Networks, Blogs, and Forums post that Google can use Neven Vision to help define that user? Their photo.

[via Google Operating System]

Combined with Riya, where users have upoaded over 7 million personal photos to Riya’s servers and tagged and labeled the subject’s faces to be searchable via Riya’s facial recognition technology, we are creeping closer and closer to a user-created, user-supported, and user-identified database of images of the population. The increased aggregation and tagging of my image has value externalities that have yet to be fully theorized. As I’ve warned before:

Sure, it would be great if I could easily search for pictures of me on the web, but do I want others to be able to do that as well? Could this provide a useful tool for stalkers looking for a particular person? Abusive partners trying to track down their victim? (Note that Riya also uploads and indexes all the metadata related to your photos, including the date and time is was taken, when it was uploaded, etc. Users can also tag and search photos based on location). …there are externalities once all of the images of our daily lives (and their related metadata) are uploaded to the Internet, indexed, searchable, and accessible to all.

Now Riya is moving ahead by applying the same facial recognition technology (and the facial profiles already learned from those 7 million uploads) to all images found on the web. While they couch the benefits in terms of being able to search for particular fabric patterns of rugs online (yawn), such a service is much more likely to be used to find more pictures of a person’s face you saw on MySpace or Flickr, or of an ex-girlfriend you’re trying to track down. As I warned before:

Sure, it would be great if I could easily search for pictures of me on the web, but do I want others to be able to do that as well? Could this provide a useful tool for stalkers looking for a particular person? Abusive partners trying to track down their victim? (Note that Riya also uploads and indexes all the metadata related to your photos, including the date and time is was taken, when it was uploaded, etc. Users can also tag and search photos based on location). …there are externalities once all of the images of our daily lives (and their related metadata) are uploaded to the Internet, indexed, searchable, and accessible to all.

[via Search Engine Watch]