In the shadow of the state (1)

Thinking through the paradox of polities: the state as protector and predator.

In a previous post on Helen Dale’s Substack, I discussed the paradox of polities (or the paradox of politics). We need the state to protect us from social predators but the state itself is the most dangerous of social predators.

The first, and greatest, of historical sociologists, Ibn Khaldun (1332-1406) nicely articulated this paradox:

Mutual aggression of people in cities and towns is thus averted by the authorities and by the government, which hold back the masses under their control from attacks and aggression against each other. They are thus prevented by the influence of force and governmental authority from mutual injustice, save such injustice as comes from the ruler himself.

The Muqaddimah, I.2.7.

Tingey injury law firm, on Unsplash.

Political economist Mancur Olson’s analysis of the ruler as a stationary bandit is another approach that considers the same underlying tensions between state-as-protector and state-as-predator.

Farmers within state societies were typically in a protection-predation relationship with their lords and rulers. Rulers and lords extracted income from farmers — engaging in social predation to do so — while seeking to protect the farmers who were their sources of revenue. The local knight of the manor — along with squires and other soldiers in his service — was landlord, local magistrate, local police unit and local military unit, all rolled up into one.

The highly successful Hungarian response to the second Mongol invasion of Hungary, in the winter of 1285-6, displayed the protective role of castles and knights quite vividly. Smashing a Mongol invasion while outnumbered — c.40,000 Mongols were crushingly defeated by c.30,000 Hungarians — was a notable feat.

That states were broadly protective is quite clear. Not only have state societies generally had much lower rates of homicide than non-state societies — something that shows up strikingly in the genetic record — they have also been marked by population growth, much greater social complexity, and much greater ability to mobilise resources, compared to non-state societies.

So far, so clear.

The three-way shadow

What is less clear is how much of that achievement is done by the state and how much occurs in the shadow of the state. Which is to say, how much is a direct result of what the state — and the state apparat — does and how much comes from what folk do, given that the state exists and it does what it does.

The shadow of the state operates in three ways. First, the possibility of deliberately attracting the attention of the state: for instance, taking folk to the state courts. In Magna Carta, the one thing the barons wanted the King to do more of, was provide more frequent circuits by royal judges.

Second, being able to do things because of the level of peace, and other benefits, the state provides. Third, organising things so as to avoid the state.

The last covers what is often called the informal economy, ranging from things simply outside the view of the state (what we might call white markets), things that may be illegal but the rules are not enforced or otherwise evaded (which we will call grey markets), to things that are illegal to own or provide where the state makes serious attempts to enforce the ban (black markets).

Informal markets can be very large. Up until the legalisation of private property in 2004, the dramatic surge in Chinese economic growth from 1978 onwards was overwhelmingly a matter of grey markets.

Chinese commerce operated on an ever-greater scale while being — quite systematically — illegal. It had to operate, therefore, without recourse to the courts. Moreover, it did so with dramatic success.

The experience of China from 1978 to 2004 belies a lot of glib claims about what the state “has to do” for large-scale commerce to operate. This point applies to informal markets more generally. Across the globe, a great deal of commerce operates without being able to use state courts.

As E.P. Stringham points out in Private Governance, there tends to be all-or-nothing view of what it is necessary for states to do. The claim being that there are a range of things that will not happen unless the state does it.

The more one looks, the less true that appears. Indeed, it is hard to find anything that states provide that have not also been provided by private operators: up to and including adjudication, policing and defence.

The areas claimed to be in the category of “have to be provided” by the state are typically various forms of coordination goods:

club goods (e.g. group-membership) where folk can be excluded, but can jointly consume the good;

common-pool resources (such as fisheries and irrigation systems) where folk cannot be easily excluded, but one’s use of the good precludes another doing so; and

public goods (such as group defence) where folk cannot be easily excluded but can jointly consume the good.

The problem with “the state has to do it” folk is they typically treat the matter as if social action is a matter of either the state or of atomised agents. This is flatly not true.¹

Political economist Elinor Ostrom built a career — one that culminated in a Nobel memorial (2009) —  demonstrating that this all-or-nothing view was systematically wrong. Local communities have a long history of developing all sorts of ways to successfully manage common-pool resources such as irrigation systems.

Bundling with territory

Even with public goods, states provide them by bundling them with territory. That is, they create a giant club — a legal country, with set borders and rights of residence — and they provide public goods to the residents of said country.

This pattern — create a club (a group able to police membership), evolve rules for the club, provide services to the club — is a pattern that private actors can, and do, engage in all the time.

A striking example is that stock markets were not creations of the law, nor of the state. They were clubs created by private actors who evolved their own rules. Rules that ignored the state: they may even have involved some level of agreement to actively ignore the rules of the state. Stock markets are particularly salient examples of private governance, but not remotely unusual ones.

The costly state

Once one becomes aware of how much private governance there is — and how much of the evolution of commerce has remarkably little to do with the state — the benefits the existence of states do provide becomes more of a puzzle. Especially as there are all sorts of ways in which states impose costs on their societies.

For instance, it is very clear that Latin America is systematically poorer than Anglo-America because Latin American states — and the imperial Spanish and Portuguese states that they emerged from — have systematically increased transactions costs within their societies. The higher level of transaction friction² reduces the level of transactions and has done so year after year. As commercial transactions are overwhelmingly transactions for mutual benefit, this higher transaction friction is a systematic social cost.

The imperial British state — and the settler societies that emerged from it — have been systematically more commerce-friendly. This meant that the two sets of states have been on quite different economic trajectories, with the higher transaction friction states being much poorer societies, even though European settlement in Latin America is a century or more older than in Anglo-America.

The much higher levels of informal markets in Latin American states — and their societies’ much lower levels of economic activity, compared to Anglo-America — are directly connected. Informal markets typically operate to avoid the transaction frictions imposed by the state.

It is also clear that there is a general tendency for states to become less functional over time. If they are put under enough competitive pressure, this tendency can be counteracted, or even reversed. This is essentially the story of Europe — and its offshoot societies — from the beginning of the Commercial Revolution in the C11th onwards. Nevertheless, the underlying tendency of states to become less functional over time is very clear.

Ibn Khaldun built an entire analysis of social order on the basis of increasing state dysfunction over time. The way empires tend to collapse, including the dynastic cycle of China, also shows this tendency. Mancur Olson’s distributional coalitions tend to increase in their negative effects over time. The take-off to mass prosperity came from the interaction between (pdf) ideas, technology and the break up of local rent-seeking distributional coalitions. Command economies, precisely because the state attempts to do so much, have particularly strong tendencies to increasing dysfunction.

In our own time, the developed democracies — despite their increasing productive capacities — are showing clear signs of increasing state dysfunction. A pattern that is weakening commitment to democracy.

Border regions

While states bundled public goods with territory, they rarely had border control in the modern sense, typically lacking the administrative resources to do so. If they were going to have significant border control, that generally meant building a wall. On the other hand, faced with enough migratory pressure, that is true in the modern world as well.

Bundling public goods with territory clarified revenue collection and those patterns of deference we call authority. That territory worked as a basis for revenue and authority we can see from how many wars were fought to protect or seize territory. A considerable amount of military effort was also expended to protect borders from raiders.

A ruler would boast about their capacity to conquer territory, as that would not only express their power, it would also say that raiders were further away.

The Ottomans utilised raiding as part of their system of conquest. Ghazis — warriors for Islam generated by Islamic polygyny and whose activities were sanctified by Islam — would operate on the borders, degrading the enemy’s borders by raiding, killing, enslaving, so driving people away, and depressing economic activity. This would reduce the revenue the rival state could extract.

At some point, the Ottoman Army would move in and conquer the territory. Settlers would be brought in, the ghazis would move to the new border, rinse-and-repeat.

This systematised the sanctified raiding and polygyny of Islam. Such adaptive systematisation of Islamic patterns was a feature of Ottoman rule, and was the basis of the Ottoman state being the most successful Islamic state, and one of the longest-lasting Muslim dynasties.

This pattern took the Ottomans from the interior of Anatolia to the borders of Vienna. It was only brought to a halt by the Habsburg creation of the Military Frontier: fortified agropastoral villages who elected their own magistrates and officers, who had various tax concessions and were able to raid back.

Borders — particularly those with pastoralist peoples — were notoriously contested lands. Fortifications tended to be concentrated on borders.

A level of de-population of borders was sometimes accepted, to provide a buffer zone. This was particularly so if the state sought to make bearing arms a monopoly restricted to agents of the state.

Alternatively, border lords and residents would be granted more expansive autonomy — hence  marcher lords and bishops and nobles with palatine authority — and/or border residents had formalised raiding rights. The Habsburg Military Frontier was a particularly extensive combination of borderer autonomy and raiding rights that proved highly successful.

States would seek to control otherwise unprofitable border areas in order to protect core areas. The contrast between borderlands and the core of states provides an indicator of where the state advantage for social complexity and resource mobilisation comes from.

State dynamics

As the existence of a state provides persistent benefits across history, and across various forms of the state, those benefits must flow from general features of being a functioning state.

The fundamental feature of a state — what is required to have it continue to exist — is the ability to routinely extract revenue. When one examines when and how states provide public goods, it has been typically tied to maintaining or improving revenue extraction.

Even in our own time, there is a strong tendency for states to under-provide policing services to fiscal-sink localities: localities that cost more in expenditure than they provide in revenue. This pattern is very clear in the Americas — accounting for its very high homicide rates — but can also be seen in Europe (e.g. French banilieue).

Conversely, states have often gone to considerable efforts to protect or otherwise promote trade, because of its revenue benefits. Not least that taxing outsiders can be less fraught than taxing locals, though the trick is to generate a balance of protection and predation that maximises revenue. But balancing predation and protection is a perennial difficulty for states.³

It is why Parliamentary states were persistently able to have higher taxes than autocratic states: Parliaments provided better information about what level of taxes were acceptable; and taxes obtained with the consent of the major players in a society where more safely and easily extracted. With such bargaining, higher trade-offs between provision of public goods and revenue extraction could be achieved. Hence rulers were willing to engage in the bother of Parliaments.

The more say that the wider society has over what the state does, then the more the state does, continues to be a strong tendency in political life. It is why culturally homogeneous societies tend to have higher levels of state provision: there is more agreement about what the state should do, and how, and better feedback between the state apparat and society, creating much less social friction in its operation.

One reason why we are seeing more political polarisation and alienation is that those feedbacks are breaking down as our societies become more varied and the gulf between the epistemic industries — education, academe, media, IT — and the general public becomes larger. It is increasingly clear that the contemporary administrative/managerial class has very different perspectives than the general public on a wide range of issues.

Administering the institutional commons

One way in which state societies were able to mobilise far greater resources than non-state societies is quite clear: taxing generates surplus — i.e. resources above subsistence. The extraction of resources via taxation before they could be turned into supporting more babies created surplus on an unrivalled scale.

This is demonstrated by the monumental creations of state societies: particularly the labour-service autocracies such as pharaonic Egypt, the Khmer Empire, various Chinese dynasties. The pyramids and temples of Egypt; the constructions of Angkor Wat and Angkor Thom; the Great Wall(s) of China; all manifest the surplus creation-by-extraction and resource mobilisation capacities of such states.

So, if states provide public goods by bundling then with territory — and if private governance operates on the basis of clubs — then a state operates as a club of clubs. It administers, indeed creates, an institutional commons. That is, it creates a territorial space within which institutions operate and evolve.

Without a state, there is no such institutional commons. Hence violence operates more incessantly in non-state societies and the level of trade — while it does occur — is much lower, as is any level of surplus.

How a state administers the institutional commons varies widely. It can do it in a way that is systematically hostile to commerce, or in a way that is relatively indifferent to commerce, or in a way that actively fosters commerce, or some mixture thereof.

After the Glorious Revolution of 1688, the English Parliament — the British Parliament after the Act of Union (1707) — operated as what Joel Mokyr and John Nye call a meta-institution: an institution that managed/made-the-rules-for other institutions. The King/Queen-in-Parliament acted as the central clearinghouse for all statutes and public spending.

This is what Parliamentary Supremacy actually meant: that the British state had a single, representative, deliberative body that administered the institutional commons. For various reasons, it came to do that in a way that broke-up local distributional coalitions and created a liberal economic order increasingly favourable to (domestic) commerce. Hence Britain became the cutting-edge society for the Great Enrichment, the development of mass-prosperity societies.

Creating an institutional commons

But Parliamentary Supremacy was a particular way of administering the institutional commons. Administering — indeed, creating — the institutional commons is what all states do. They do it so that they can routinely extract revenue, and so exist.

They do it within their territorial ambit of authority. When the habitual deference we call authority fails, so does the state. The classic problem of borderlands has precisely been the intrusion of those who lack such habitual deference.

In creating such an institutional commons — based on habitual deference — states can routinely extract surplus, the population can increase, institutions can grow, social complexity can develop. People can operate in the shadow of the state: whether attracting its attention; using what it provides; or avoiding it; but still taking advantage of the institutional commons created by the state.

The various forms of clubs that generate private governance can also operate. For people have an answer to the question: whose rules apply to, who manages, the social space with folk with whom, or about some matter, that is not covered by some private governance club I am a member of? The rules of the state apply, the management of the state applies, is the default answer.

The state is the institutional default that manages and creates the institutional commons within particular territories. A concert of states can create elements of an institutional commons internationally. This is what folk are referring to when they talk of the “rules-based international order”. Note, these rules are not law proper — despite pretences to the contrary — as there are no systematic remedies, but it is still a form of institutional commons.

How a state does creates and manages an institutional commons — including how well or badly — can and does vary hugely. But one merely has to look to the contested borderlands of history, and to the vast limitations of non-state societies, to see how important the creation and management of the institutional commons is.

We want to live in an institutional commons, to have the benefits of it. Yet an authority powerful enough to create and manage the institutional commons is an authority powerful enough to be the most dangerous social predator. Which is why the paradox of polities is always with us, and can never be solved, just managed more or less well.

References

Ronald Coase & Ning Wang, How China Became Capitalist, Palgrave Macmillan, 2013.

David Frye, Walls: A History of Civilization in Blood and Brick, Faber & Faber, 2018.

Monika Karmin, et al., ‘A recent bottleneck of Y chromosome diversity coincides with a global change in culture,’ Genome Resources, 2015 Apr;25(4):459-66.

Lawrence Keeley, War Before Civilization: The Myth of the Peaceful Savage, Oxford University Press, 1997.

Meir Kohn, ‘An Alternative Theoretical Framework for Economics,’ Cato Journal, Vol. 41, No. 3 (Fall 2021).

Joram Mayshar, Omer Moav, Zvika Neeman, ‘Geography, Transparency, and Institutions,’ American Political Science Review, June 2017, 111, 3. 622-636.

Joram Mayshar, Omer Moav, Luigi Pascali, ‘The Origin of the State: Land Productivity or Appropriability?,’Journal of Political Economy, April 2022, 130, 1091-1144.

Yascha Mounk and Roberto Foa, ‘The Danger of Deconsolidation: The Democratic Disconnect,’ Journal of Democracy, vol. 27, no. 3, July 2016, 5-17.

Joel Mokyr and John V.C. Nye, ‘Distributional Coalitions, the Industrial Revolution, and the Origins of Economic Growth in Britain,’ Southern Economic Journal, (2007), 74: 50-70.

Robert Neuwirth, The Stealth of Nations: The Global Rise of the Informal Economy, Pantheon Books, 2011.

Mancur Olson, The Rise and Decline of Nations: Economic Growth, Stagflation and Social Rigidities, Yale University Press, 1982.

Mancur Olson, Power and Prosperity: Outgrowing Communist and Capitalist Dictatorships, Basic Books, 2000.

Elinor Ostrom, Governing the Commons: The evolution of institutions for collective action, Cambridge University Press, [1990] 2011.

James C. Scott, Against the Grain: A Deep History of the Earliest States, Yale University Press, 2017.

Edward Peter Stringham, Private Governance: Creating Order in Economic and Social Life, Oxford University Press, 2015.

Chenggang Xu, ‘The Fundamental Institutions of China’s Reforms and Development’, Journal of Economic Literature, 2011, 49:4, 1076–1151.

Jan Luiten van Zanden, Eltjo Buringh, and Maarten Bosker, ‘The rise and decline of European parliaments, 1188–1789,’ The Economic History Review, 65, 3 (2012), 835–861.

Tian Chen Zeng, Alan J. Aw & Marcus W. Feldman, ‘Cultural hitchhiking and competition between patrilineal kin groups explain the post-Neolithic Y-chromosome bottleneck,’ Nature Communications, 2018, 9:2077.

1. It is also not true that states are rational calculators and maximisers of the public good.

2. Transaction friction = rate at which transaction costs slow down transacting.

3. Ibn Khaldun noticed that there were ranges over which tax rates and tax revenues moved in opposite directions (aka the Laffer Curve):

It should be known that at the beginning of a dynasty, taxation yields a large revenue from small assessments. At the end of the dynasty, taxation yields a small revenue from large assessments.

The Muqaddimah, I.3.36.