Annotated Bibliography
Papers on Data Surveillance: Theory, Practice & Policy

© Xamax Consultancy Pty Ltd,  1995-2008
Photo of Roger Clarke

During the last 30 years, I've done a lot of work in the area of information privacy and data surveillance, and published many papers.

This document provides an overview of the whole body of my work. This page is updated to 31 December 2006. Papers since then are indexed on my What's New Page.

Each paper is designed so as to be readable as a standalone document, and hence there is duplication within the papers, particularly in the introductory sections.

The following alternative indexes are available:

This page indexes papers written by me. I also provide assistance in locating resources more generally.


INTRODUCTION

My published work in dataveillance and information privacy comprises well over 100 papers, including over a dozen in refereed journals, many invited and contributed papers published in conference proceedings, and occasional papers on topical matters. Many of them are available on the web, including everything I've published since 1994.

The papers span a very wide range of topics, and are described in the following sections:

  1. Theory:
    1. Dataveillance and Privacy
    2. Identification, Anonymity and Pseudonymity
    3. National Identification Schemes
    4. The Authentication of Identity and Other Assertions
    5. Privacy-Invasive Technologies (the PITs)
    6. Biometrics in Particular
    7. The Internet ('e-Privacy')
    8. Privacy-Enhancing Technologies (PETs)
  2. Organisational Strategy and Practices:
    1. Strategy
    2. Privacy Impact Assessment (PIAs)
    3. Privacy Policy Statements (PPS)
  3. Policy:
    1. Policy Issues
    2. Proposals for Policy Measures
    3. Critiques of Policy Measures
    4. Organisational Self-Regulation
    5. Professional Responsibilities
    6. Broader Issues
  4. Australia:
    1. Current Issues
    2. The Australia Card and Its Progeny, 1985-
    3. The Privacy Act 1988, as amended in 1990 and 2001
    4. Other Papers Prior to 1988

Many of the papers are relevant to more than one of these areas, and hence many appear multiple times in this document. They are introduced in logical order, and this is in some cases not the same as the chronological order of publication.


1. THEORY

The first cluster of papers establishes the theory of dataveillance. In part, these papers recapitulate knowledge developed by other people, but they also contain a lot of original contributions.


1.1 Dataveillance and Privacy

An introduction and summary is in 'Introduction to Dataveillance and Information Privacy, and Definitions of Terms' (1997). See also 'What's Privacy?' (2006).

The base for my extensive work in this area was laid in a paper entitled 'Information Technology and Dataveillance' (1988, published in a major US journal). Physical and electronically enhanced monitoring of individuals and groups is expensive. The paper shows how information technology is enabling those old techniques to be replaced by highly automated, and therefore much cheaper, systematic observation of data about people. This new form of monitoring, whose descriptor I abbreviated to 'dataveillance', is potentially highly privacy-invasive.

A second paper, 'Human Identification in Information Systems: Management Challenges and Public Policy Issues' (1994, published in a leading international journal), further develops a vital aspect of the argument: the nature of human identification as it is applied within information systems. Remarkably, there are very few works in any academic literature which address the question of such uses of human identity, and for this reason the paper had a long gestation period (from 1985 until 1994).

A later paper Chip-Based ID: Promise and Peril updated and extended the analysis of the implications of identification cards (1997, presented at an international conference in Montreal).

The paper 'The Digital Persona and Its Application to Data Surveillance' (1994, published in a leading international journal) introduced a new concept, the 'digital persona', as a tool in the analysis of behaviour on the 'net. It applies the tool, together with data surveillance theory, to predict the monitoring of the 'real-life' behaviour of individuals and groups through their net behaviour. The concept of 'nym' is related to (but not identical to) what I mean by 'digital persona'. Nyms are addressed in the following section.

A further paper examined Person-Location and Person-Tracking Technologies, initially for presentation at the 1999 conference of Privacy and Data Protection Commissioners. A revised version was published in a leading international journal.

A paper published in a major U.S. journal argued that Internet Privacy Concerns Confirm the Case for Intervention (1999), and that that last renegade among advanced western nations would find it necessary to establish a statutory framework for privacy protections in the private sector. This paper also specified the requirements of a genuinely 'co-regulatory' approach.

The serious inadequacies in the 'Fair Information Practices' model of privacy protection were stated in 'Beyond the OECD Guidelines: Privacy Protection for the 21st Century' (January 2000), which identified the many additional steps urgently necessary if a collapse in consumer and citizen confidence were to be avoided.

An invited paper reviewed the abject failure of the intelligentsia to notice the rise of privacy-invasive technologies: 'While You Were Sleeping ... Surveillance Technologies Arrived' (January-February 2001).

I took the opportunity of an invited presentation to a New Zealand conference to undertake a retrospective on 'Dataveillance - 15 Years On', supported by a slide-set (March 2003), and to update it further in Have We Learnt To Love Big Brother? (2005) and Social Impacts of Transport Surveillance (2006, with Marcus Wigan).

The following two are less pretentious than the above papers, but they contain some important ideas:


1.2 Identification, Anonymity and Pseudonymity

A short tutorial explaining the key concepts in this vital (and seriously misunderstood) area is in 'Identification and Authentication Fundamentals' (May 2004).

In the area of identity, identifiers and identification, the paper on 'Human Identification in Information Systems' (1994) established the foundation for this thread of my work.

A long series of papers investigates alternatives to identification, both anonymity (which precludes the discovery of identity, and hence compromises the public interest in accountability), and pseudonymity (which makes the discovery of identity intentionally difficult, but not impossible). The best couple of papers to read are probably 'Identified, Anonymous and Pseudonymous Transactions: The Spectrum of Choice' (1999) and section 2.4 of Authentication: A Sufficiently Rich Model to Enable e-Business (2001).

Papers relating to national identification schemes are in the following section. Other papers, in chronological order, are as follows:


1.3 National Identification Schemes

See:


1.4 The Authentication of Identity and Other Assertions

The concept of authentication has been seriously misunderstood. A series of papers considers both the specific concept of 'identity authentication', and the authentication of various other kinds of assertions.

An introduction is provided by 'Identification and Authentication Fundamentals' (2003).

The most comprehensive papers are these:

A closely related area on which I've published a succession of papers is public key infrastructure to support digital signatures.

The full series of papers is as follows:


1.5 Privacy-Invasive Technologies (the PITs)

I coined the term 'PIT' in late 1998. An explanation is in Introducing PITs and PETs: Technologies Affecting Privacy (March 2001). See also my PITs and PETs Resources Site (February 2001). The remainder of this section provides references to papers on particular technologies that adversely affect privacy.

An early paper, 'Database Retrieval Technology and Subject Access Principles' (1984; with Graham Greenleaf, published in the Australian Computer Journal, but not available on the web), is concerned with data retrieval technology. It examines the scope for a particular form of database technology to render impractical an established privacy-protective mechanism, the so-called 'subject access principle'. This case study exemplifies the way in which developments in information technology undermine privacy protection laws. It provides a basis for understanding the impact of other developments, such as 'reverse access' to telephone directories, monitoring of energy usage, textual analysis, 'data mining', and the discovery of individual characteristics through the analysis of seemingly anonymous, statistical collections.

Field work, undertaken in both the United States and Australia between 1987 and 1992, resulted in a series of papers on Computer Matching:

The paper 'Trails in the Sand' (1996) highlights the wide range of transaction trails that people leave behind them.

Another technique, Profiling, was examined in 'Profiling: A Hidden Challenge to the Regulation of Data Surveillance' (1993). A later paper examines 'Customer Profiling and Privacy: Implications for the Finance Industry' (1997).

A comprehensive paper examines Direct Marketing (February 1998), including mail, fax, outbound tele(phone)-marketing, and Internet marketing. Related topics that have been addressed include spam, cookies and emailing lists.

During the period 1985-87, the Australian Government developed a proposal to implement a central database, whose purpose was the facilitation of dataveillance of all residents of that country. 'Just Another Piece of Plastic for Your Wallet: The Australia Card' (1987, published in an international technology policy journal) provides a carefully documented description and analysis of the proposal, a distillation of the issues, and a political history of the proposal's development and ultimate fate.

Although the Australia Card proposal was withdrawn in the face of dramatically negative public opinion, the momentum that dataveillance applications of information technology had attained within the Commonwealth public sector was scarcely affected. 'The Resistible Rise of the National Personal Data System' (1992, published in a leading American journal of computers and law) documents a number of developments during the following three years. It is primarily a political history, expressed within the context set by the theory of dataveillance.

Considerable effort has been invested in the area of chip-cards, including:

Another area of importance has been public key infrastructure (PKI) to support digital signatures. A long series of papers published in that area is indexed here.

Other specific threats have been examined, as follows:


1.6 Biometrics in Particular

Biometric technologies endeavour to identify people, or to authenticate assertions about people, based on some physical characteristic. Most biometrics technologies are incapable of working effectively in the real world, and most technologies (and many companies) have failed. So-called 'facial recognition' technology doesn't do anything of the kind, is of very dubious usability for authentication, and is completely untenable for identification – despite many statements by salesmen and Government Ministers who pretend it's a solution to terrorism, and to law and order more generally.

A very few biometrics technologies – essentially only fingerprints and iris (and in narrowly constrained contexts perhaps hand geometry) – have any credibility at all. But each of these also has a vast array of weaknesses, and their effectiveness is enormously dependent on the environment in which they are applied.

Whether they are effective or not, biometric technologies are extraordinarily threatening to freedoms of the individual. The papers here examine the wide range of myths that have been perpetrated by biometrics suppliers and their industry associations, and that are sustained by national security and law enforcement agencies because those myths suit their purposes.

The most important papers are:

Other significant presentations and notes are:

Other papers and presentations include:

The following resource is also available:

Early background papers include:


1.7 The Internet (e-Privacy')

I've published about 50 papers in this area, and have described and linked to them from a separate document.


1.8 Privacy-Enhancing Technologies (PETs)

An explanation of the concept of 'PETs' is in Introducing PITs and PETs: Technologies Affecting Privacy (March 2001). Here is another accessible introduction to the key ideas. And here is a paper on Business Cases for Privacy-Enhancing Technologies (June 2005).

An early series of papers addresses security aspects:

A series of papers has considered technologies whose purpose is to directly support privacy. This work is primarily in:

I was active in the original W3C Working Group on P3P (Platform for Privacy Preferences). Unfortunately, the initiative fell so far short of its aspirations that it is not worthy of the name 'PET'. All of the following appeared in Privacy Law & Policy Reporter:


2. ORGANISATIONAL STRATEGY AND PRACTICE

In the 1970s, government agencies and corporations resisted calls for privacy protections. During the intervening decades, many of them have come to recognise privacy as a factor that can harm their business, and that therefore needs to be addressed in a positive manner.

I have performed many consultancy assignments in this area during the last two decades. This section identifies published papers that are addressed specifically to business enterprises and government agencies, firstly in matters of general strategic significance, and secondly in the specific area of privacy impact assessments (PIAs).


2.1 Strategy

The primary papers I've published on privacy as a strategic factor for corporations and government agencies are as follows:

Guidance on specific matters of a strategic nature is provided in:


2.2 Privacy Impact Assessment (PIAs)

I have performed a considerable number of consultancies in this area (see client list). This included the preparation of guidelines for the performance of PIAs as long ago as 1998, and the review of guidelines prepared by government agencies.

My published papers in the area are:


2.3 Privacy Policy Statements (PPS)

A Privacy Policy Statement (PPS) is a web-page that makes statements about the web-site owner's privacy policies. The device is talked about by business and government as if the concept mattered, even though its significance is very low. Nonetheless, some guidance is provided in:


3. POLICY

It is widely claimed that information technology is becoming pervasive, and is giving rise to an 'information economy' and an 'information society'. If that is the case, then its impacts will be substantial, and must be managed. This part of my collection of papers is concerned with public policy regarding dataveillance.


3.1 Policy Issues

The main policy issues are identified in the following papers:

I identified what I then saw as the Five Most Vital Privacy Issues, (July 1997), for the Montreal Daily, Le Devoir, prior to the International Conference on Privacy, Montreal.

A resource-page has been maintained since 1996, citing surveys of public attitudes to privacy issues.


3.2 Proposals for Policy Measures

Many of my papers pursue positive arguments about what policy measures should look like. The most important of these are:

Others include:


3.3 Critiques of Policy Measures

A paper on the OECD's Data Protection Guidelines (1989) provides a template for evaluating laws, and proposals for laws, and has been applied in a number of countries.

Many of my papers examine the inadequacies of the 'official response' by Parliaments and Governments, which involves providing mere 'fair information practices' protections. The intellectual basis underlying thse arguments is in 'Beyond the OECD Guidelines: Privacy Protection for the 21st Century' (January 2000).

The following comments were made on the dreadful Privacy Amendment (Private Sector) Act 2000. It passed into law in December 2000, and became [in]effective on 21 December 2001:

A specific matter of concern is public key infrastructure (PKI), and the use of digital signatures for identity authentication. A long list of papers is indexed above. Two that are particularly oriented towards the policy aspects are 'eAuthentication: Where's the Public Interest?' (2003), and Identity Management; and PIAs (2004).

Another concern is 'ENUM' (March 2003).

Yet another is the mindless enthusiasm for biometrics that has arisen as a result of the dominance of 'national security' over both civil liberties and logic, aided and abetted by fraudulent representations by technology providers. A specific critique is at:

Papers between 1996 and 1999 were:

Older papers that are now of primarily historical interest are as follows:


3.4 Organisational Self-Regulation

Organisational self-regulation without a layer of statutory regulation underpinning it is not a form of privacy protection, but a mere pretence. This is addressed in a number of places, including:

A major paper provides guidance to corporations and government agencies concerning 'Privacy and Dataveillance; and Organisational Strategy' (1996, a keynote presentation at an Australian Conference).

Observations concerning a particular industry code are provided in 'Smart move by the smart card industry: The Smart Card Industry's Code of Conduct - Part I' (1996) and Part II (1997). Both were published in the Privacy Law & Policy Reporter.


3.5 Professional Responsibilities

A paper entitled 'Economic, Legal and Social Implications of Information Technology' (1988, published as an 'Issues and Opinions' piece in a leading US journal) provides a discussion of the ethics of academic endeavour in the information systems discipline. It argues that information technology's impacts are so great that detached observation is an inadequate stance for an information systems academic.

Information systems researchers must engage themselves in their subject-matter, and extend themselves beyond mere description and explanation, and even beyond the prediction of the outcomes of artefact design and interventions in organisations and society. Information systems researchers are irretrievably involved in the process of engineering organisations and society, and cannot meaningfully sustain the pretext that they are entirely uninterested in, and unaffected by, the processes around them. These issues are examined in 'Data Surveillance: Theory, Practice and Policy' (July 1997). That paper argues that policy issues in general, and information privacy in particular, are not only an appropriate area of focus for information systems researchers, but that they are also capable being approached in a sufficiently disciplined manner.

Here is the Australian Computer Society's ELSIC Committee, which I chaired 1984-95, and of which I continue to be an active member.

An outline is provided of 'Information Systems Audit & Information Privacy' (1997), and resources are provided for Privacy Impact Assessments (1998) and (2004).

Other papers on this topic include:

Self-regulation by research professional in the I.S. discipline is considered in considered in 'Information Privacy in a Globally Networked Society: Implications for Research', plus slide-set (December 2002). The contributions to the panel session were later published as 'Information Privacy in a Globally Networked Society: Implications for IS Research' (October 2003).

A further paper comments on the social responsibility of engineers, plus PowerPoint slides (2002).


3.6 Broader Issues

Like other technologies, computing and telecommunications are capable of being applied to the benefit of humanity as a whole, or of particular interest groups within society. Use of information technology by the politically powerful as a means of exercising control over the thoughts and actions of members of the public, is a matter of especial concern to those living in democracies.

A paper entitled 'Information Technology: Weapon of Authoritarianism or Tool of Democracy?' (1994; presented at the World Congress of computing academics) identifies critical implications of information technology for democracies. It represented a response to a paper submitted to the conference by a senior government executive of a country that had previously been dominated by the U.S.S.R., and that has no tradition of democracy as it is known in 'western' countries.

This paper's importance is that it lifts the application of the theory of dataveillance from the individual and social levels to the political level, and is a first, tentative step toward the building of a bridge between the theory of dataveillance (developed, as it has been, largely from within the information systems discipline), towards broader theories arising in anthropology, sociology and political science.

Other papers that consider broader issues related to dataveillance include:


4. AUSTRALIA

This section identifies papers that deal with matters specific to Australia.


4.1 Current Issues

A summary paper in this area is a history of privacy in Australia (December 1998). This is supplemented by a running document on current developments, and an update. Because privacy-invasive technologies and applications are so rampant, it's extremely difficult to keep up ...

Australia risks following down the same mindless trail as the U.S. in relation to security in locations such as airline terminals. A paper was prepared on the Smartgate 'face recognition' scheme trialled by Customs at Mascot airport (February 2003). Submissions were also made re Passport Biometrics, on behalf of the Australian Privacy Foundation, on 17 February, 25 February, 2 March, and 15 March (all in 2004).

Another concern is the lack of consideration of privacy throughout the telecommunications industry, including in relation to ENUM (March 2003).


4.2 The Australia Card and Its Progeny, 1985-

Papers on National Identification schemes generally are in a separate section, above. This section addresses the specifics of the ongoing attempts by executives in Australian government agencies and Australian politicians to implement extremist social control mechanisms in this country.

During the period 1985-87, the Commonwealth Government developed a proposal to implement a central database of the Australian population, whose purpose, expressed in terms of the theory developed in this body of work, was the facilitation of dataveillance of all residents of this country. 'Just Another Piece of Plastic for Your Wallet: The Australia Card' (1987, published in an international technology policy journal) provides a carefully documented description and analysis of the proposal, a distillation of the issues, and a political history of the proposal's development and ultimate fate.

Another paper on the topic was 'National Identification Scheme - Costs and Benefits' (1986, published in an Australian journal).

Although the Australia Card proposal was withdrawn in the face of dramatically negative public opinion, the momentum that dataveillance applications of information technology had attained within the Commonwealth public sector was scarcely affected. 'The Resistible Rise of the National Personal Data System' (1992, published in an American journal of computers and law) documents a number of developments during the following three years. It is primarily a political history, expressed within the context set by the theory of dataveillance.

The untrustworthiness of a mere Prime Minister's assurances are underlined in 'The Tax File Number Scheme: A Case Study of Political Assurances and Function Creep' (1991, published in an Australian journal).

An attempt by law enforcement interests to create a highly privacy-intrusive scheme is documented in 'LEAN Times Ahead: The Proposed Law Enforcement Access Network' (1992).

The reasons for public concerns about government agencies are summarised in 'Why the Public Is Scared of the Public Sector' (1993, presented at an Australian conference).

The justification for a major computer matching scheme is shown to be incompetent or fraudulent, in 'Matches Played Under Rafferty's Rules: The Parallel Data Matching Program Is Not Only Privacy-Invasive But Economically Unjustifiable As Well' (1993, published in Privacy Law & Policy Reporter).

Echoes recur continually. In April 1999, the Commonwealth Auditor-General, Pat Barrett, sought to revive the process of extending the Tax File Number, this time into the Medicare arena (ANAO 1999). His proposal was then considered by the House of Representatives Standing Committee on Economics, Finance and Public Administration (EFPA). Too pressed for time to provide yet another time-wasting submission to yet another government committee, I submitted a brief letter (November 1999).

During the 1990s, bureaucrats achieved merger of all government benefits schemes, and correlation of identifiers, by means of Centrelink. Despite resistance going back 15 years, they eventually succeeded in combining the health insurance and pharmaceutical benefits schemes in to a single agency. Then in the early 2000s, the centralist's dream of a 'super-ministry' was achieved, euphemistically called Human Services, which enable the benefit schemes, health systems and child support agency to be drawn even closer together.

Meanwhile, a succession of attempts were made to upgrade drivers' licences into a much broader social control scheme. See Submission to Queensland Transport in relation to the Proposed Smartcard-Based Driver's Licence (2003).

Then in mid-2005, the Queensland Premier Beattie, in an endeavour to shift media attention away from serious problems in his State, suggested that an Australia Card was needed. The Prime Minister ran with it. A cluster of identity schemes was progressively drawn into one being developed by the 'Human Services' Minister dubbed the Access Card.

Advocacy groups and the media have caused a great deal of pertrubation in the statements made by politicians and bureaucrats about the scheme. The resolve with which it is being pursued by the Government appears set to bring down another Minister in due course.

Documentation about the schemes, the continual changes in Government pronouncements, and the privacy advocacy campaigns, is at:

Parallel State Government schemes include:

Papers on the topic are as follows:


4.3 The Privacy Act 1988, as amended in 1990 and 2001

The Privacy Act, passed in December 1988, was carefully analysed in the following papers, which were distributed among a small community of interested researchers and government agencies:

A paper on 'Consumer Credit Reporting and Information Privacy Regulation', summarised the situation in the lead-up to the 1989 extensions to the Privacy Act (1989). The resultant legislation was described in 'Privacy Regulation of Consumer Credit Reporting' (June 1989, published in an Australian journal, not available on the web).

The disastrously bad amendment Bill of 2000 was analysed and severely criticised:

It came into force on 21 December 2001, exacerbating rather than solving problems. For subsequent analyses, see:


4.4 Other Papers Prior to 1990

This section is provided primarily as a matter of record, most recent first.

During 1989-90, a series of papers was prepared under consultancy assignments for the Commonwealth Privacy Commissioner, on:

During the period 1985-1990, several Policy Papers were prepared for the Australian Computer Society, on:

The situation in health care is discussed in 'Current Health Care Information Privacy Issues' (1990, invited paper at an Australian conference).

A series of assessments of earlier proposals for privacy regulation was published:

During 1978-80, a series of papers were prepared for (now High Court Justice) Michael Kirby, in relation to the design of the OECD Guidelines.

During 1976-77, a series of papers was prepared while I was a Research Officer the N.S.W. Privacy Committee. This included a set of Guidelines in relation to Personal Data Systems which would stand up very well today.

During 1972-75, several papers were prepared on behalf of the N.S.W. Branch of the Australian Computer Society. At that time, it appeared that the N.S.W. Government might push through privacy protection legislation that could have been harmful to the then immature computer industry. I led the professional association's lobbying against unreasonable regulation. I was too successful: N.S.W. passed no substantive law until 1998, and then came up with what was until December 2000 the world's worst privacy protection legislation ...



These community service pages are a joint offering of the Australian National University (which provides the infrastructure), and Roger Clarke (who provides the content).
The Australian National University
Visiting Professor, Faculty of
Engineering and Information Technology
Xamax Consultancy Pty Ltd, ACN: 002 360 456
78 Sidaway St, Chapman ACT 2611 AUSTRALIA
Tel: +61 2 6288 1472, 6288 6916

Created: 17 August 1997 - Last Amended: 30 October 2007 by Roger Clarke - Site Last Verified: 15 February 2005
This document is at www.anu.edu.au/people/Roger.Clarke/DV/AnnBibl.html
Mail to Webmaster   -    © Xamax Consultancy Pty Ltd, 1995-2006   -    Privacy Policy