Why it’s time to rethink the problem of secrecy

These notes were prepared for forthcoming talks at the Australia-New Zealand School of Government in Wellington on May 16 and Melbourne on May 21; at the University of Tasmania Law School on May 23; and at the Lee Kuan Yew School of Public Policy in Singapore on May 27.

We all recognize that excessive secrecy is a threat to democracy. But technological changes of recent years have fundamentally changed the nature of the “secrecy problem.”   Today, we need a new way of thinking about secrecy that recognizes the advent of systems of public surveillance and control that span the public and private sector; that are supported by durable alliances of politicians, bureaucrats and politicians; and whose design and operation are practically unintelligible to most citizens.

The conventional way of talking about government secrecy assumed that the main threat to individual well-being came from governments rather than from private actors. In fact, it was often assumed that there was a commonality of interest between individual citizens and businesses, since both wanted to constrain arbitrary action by government. And it made strong assumptions about the ability of individuals to comprehend and use information once they received it. Information was contained in easily understood paper documents. Any document that revealed misconduct — the so-called “smoking gun” — could be published easily and usually spoke for itself. Such revelations could be expected to trigger outrage and eventually reform.  (See my 2006 book from Cambridge University Press, Blacked Out: Government Secrecy in the Information Age.)

This way of talking about government secrecy was the status quo in the United States in the decades immediately following the Second World War. It reflected the anti-statist American political culture. It took for granted the primitive methods of record-keeping that prevailed at the time, a high level of public literacy, and a system of politics in which power was widely dispersed, allowing coalitions to easily mobilize against perceived governmental abuses.

This world is disappearing because of extraordinary advances in information technology. The cost of computer processing and data storage capabilities has plummeted over the last four decades, so that computing power is more widely dispersed. Software has also improved, along with networking and sensor technologies. The capacity to harvest, analyze, and distribute information has increased immensely. Moreover, we are still only in the early stages of building a fully wired world.

There is one force that is driving this process of technological innovation: competition. It is manifested in three forms. The first is security competition between states and non-state actors. Governments, insurgents, and terrorists exploit new technologies, and improve technologies, in the attempt to bolster their own security and undermine their adversaries. The second form of competition is between commercial and financial enterprises, both seeking to expand and consolidate control over markets. And the third form of competition, in democratic systems, is between political candidates, or between political parties, who exploit new technologies in their drive to increase influence over voters.

One result of these different forms of competition has been the emergence of complex systems of public and private surveillance and control. Governments, for example, have strong incentives to improve capacity for domestic surveillance, in large part because new technologies give enemies the ability to penetrate within national borders, either by cyberattacks or recruitment of disaffected citizens. Businesses also have strong incentives to gather data about individuals so that they can adjust production, and market their products more effectively. And politicians have similar incentives: more information about potential voters allows them to tailor their appeals more exactly, and mobilize voters more efficiently.

Three distinctive features of these new surveillance systems are already clear. First, these systems are capital intensive. It takes large sums of money to build the equipment and hire the people needed to collect and analyze data. As a result, it is difficult for new competitors — emergent states or non-state actors, new businesses, new candidates or parties — to gain a foothold against established rivals.

Second, public and private surveillance systems are tightly integrated. For example, governments (including national security agencies) often rely on data collected by businesses, while businesses rely on data harvested and released by governments. Meanwhile, candidates and political parties rely on governmental and commercial data for the purposes of campaigning. Competition between established players within each of the three domains is intense. But this competition also produces a shared interest between domains (shared, that is, between bureaucrats, businessmen, and politicians) in maintaining robust and integrated systems for collecting data about the public at large.

Third, these new systems are complicated. They are not easily understood by people who do not have advanced technical knowledge. (And this includes politicians who have formal responsibility for oversight of some of these systems.) Not only are the systems themselves incomprehensible to the average person, but the data that is collected within those systems is so massive and complicated in structure, that it could not be easily understood even if it was made publicly available. To put it another way, the general public may be reverting to a state of functional illiteracy, in the sense that many citizens are unable to comprehend either the systems themselves, or the data that is harvested by those systems. We no longer live in a world of simple paper records that speak for themselves.

We should be worried about what these changes might do to the character of the advanced democracies. The idea of personal anonymity — the notion that an individual can remain unknown in important respects — is becoming obsolete. Governments, businesses, and political parties are rapidly improving their capacity to anticipate and influence individual behavior. Moreover, there does not appear to be any Madisonian system of “faction against faction” working to protect the individual. On the contrary, all major factions (bureaucrats, business people, and partisans) share an interest in extending surveillance capabilities. And the process that is operating to undermine personal anonymity is, from the point of view of the typical citizen, incomprehensible. This must have implications for the capacity to maintain state legitimacy. Legitimacy is predicated on consent, and consent is predicated on the ability to understand how the governing system works.

It is difficult to see how we can avoid the advent of a world that is characterized by expansive and integrated systems of surveillance and control. One response has been to call for stronger privacy laws that give individuals the right to opt-out of data collection projects, or at least control the purposes for which their personal information can be used. But there are several practical reasons why this approach is unlikely to curtail the growth of surveillance systems. There are now so many ways that personal information is collected, any system of providing consent to collection would be very time-consuming for individuals. It would also be difficult to explain concisely all the ways in which information might be used. In addition, even well-informed individuals have a habit of making poor judgments about the release of personal information. And increasingly, direct collection is not necessary; important information about a particular person can be inferred from the analysis of data harvested from other sources.

(Some people argue that individuals should be given a property right in their personal information, so that the unauthorized collection and use of that information could be treated as theft. This is an attempt to bolster the conventional approach to privacy protection, however this approach would still be vulnerable to all of the problems noted in the preceding paragraph.)

A second response to the growth of surveillance falls within the realm of politics. The hope is that new information technologies will dramatically reduce the cost of mobilizing individuals to resist growing governmental and corporate power, and check the growth of enveloping systems of surveillance and control.

For example, web-based technologies have been linked to the success of the both the anti-globalization movement and the populist political movement in the United States in the late 1990s and early 2000s, as well as the Occupy movement in 2011. But the hope that new technologies will generate new sources of countervailing power may be unrealistic. There are two difficulties. The first is that the effectiveness of these movements is limited by their “headless” or “leaderless” structure. The second is that participation in these movements is still constrained by the economics of collective action. That is, there are incentives for individuals to stand by as others contribute the labor that is needed to make bureaucracies and corporations accountable.

A third response to the growth of surveillance is more subversive. The aim is to cause the collapse of new surveillance structures through hacking or leaking. For example, some radicals hoped that the string of high-profile disclosures from WikiLeaks in 2010, and Edward Snowden in 2013, would “bring the system down.” But the system has proved to be remarkably resilient. The establishment has been effective in isolating and punishing leakers, discouraging copycat leakers, and managing the fallout from unauthorized disclosures. Moreover, leaks and other forms of subversion do nothing to alter the incentives that encourage governments, corporations and politicians to build surveillance systems. The three forms of competition — security, commercial, and political — still provide powerful motivations for the expansion of surveillance and control.

If we are, in fact, heading toward a world of more extensive and integrated surveillance and control, what implications does this have for our way of talking about secrecy? I am not sure that I have the answers. But I think we will have to recognize three points.

First, the notion that the task is primarily one of challenging governmental secrecy has to be abandoned. The problem is preserving individual autonomy and privacy, which can be threatened as much by private action, particularly when that private action involves the harvesting of information that is passed on to public actors. Transparency measures must allow individuals to understand and resist public and private actions that pose an unjustifiable threat to autonomy and privacy.

Second, a new approach to secrecy must recognize that most citizens will be unable to understand much of the information that might be released through disclosure mechanisms — whether that information describes the architecture of surveillance and control, or consists of data that is harvested through that architecture. Increasingly, individuals will have to rely on trusted intermediaries to demand and interpret information from public and private entities. And those intermediaries will have to make larger investments in their own capabilities than might have been necessary fifty years ago.

The last point is also the most challenging. We will have to grapple with the economics of accountability. That is, how do we create incentives for individuals and organizations to take the steps that are necessary for any disclosure system to work — for example, to serve as a trusted intermediary, or mobilize against governmental or corporate misconduct? The mainstream media used to have commercial incentives to serve as intermediaries, but their business model has been undermined by new information technologies. Opposition parties also had incentives to serve as intermediaries, but today they also depend increasingly on public and private surveillance systems. And individual citizens often under-invest in political action, hoping to free-ride on the work of others.

So who now has a motive, economic or political, that is strong enough to provide an adequate amount of investment in promoting the accountability of public and private enterprises? As systems of surveillance and control become more extensive, we will need more robust accountability mechanisms — that is, stronger laws to provide access to information, as well trusted intermediaries who use those laws and mobilize against abuses of power. Oversight will be more difficult, and more money will have to be spent on accountability mechanisms. But who will have the commercial or political incentive to make the appropriate investment?

Alasdair Roberts is the Jerome L. Rappaport Professor of Law and Public Policy at Suffolk University Law School. These notes are prepared for talks at the Australia and New Zealand School of Government in May 2014.