Predictive policing and social credit — democracies aren't so different either: the convergence of surveillance technology
Contents
Predictive policing and social credit systems aren't exclusive to authoritarian states. The justifications and legal procedures differ, but at the level of technique — flagging "risky" people and places from historical data — algorithmic policing in democracies isn't all that different.
Background — why this topic now
China's social credit system and the surveillance apparatus in Xinjiang are frequently cited as the archetypal case of authoritarian surveillance. Comparable techniques used in democracies — London's Metropolitan Police "Gangs Matrix," or the predictive-policing software used in several U.S. cities — get relatively less attention.
A report summarizing a 2024 expert workshop from Stanford's Freeman Spogli Institute (FSI) Global Digital Policy Incubator (GDPi) makes this point explicitly: law enforcement agencies in many democracies are increasingly using predictive-policing tools to forecast risk and allocate resources, and this is raising concerns about opacity and discriminatory impact (Stanford FSI).
This piece answers a question that cuts across cases usually discussed separately: is the line dividing surveillance technology a matter of political regime, or something else?
Key data & current state
- UK Metropolitan Police's Gangs Matrix: As of 2018, 3,806 people were listed, 80% of whom were aged 12–24. One analysis found 87% of those listed were from Asian and other minority-ethnic backgrounds, and 78% were Black (The Conversation, 2018). The UK Information Commissioner's Office (ICO) issued an enforcement notice over the system's "function creep," lack of transparency, and inaccuracy (Computer Weekly, 2018).
- Oakland's PredPol algorithm (US): An analysis by Kristian Lum and William Isaac of the Human Rights Data Analysis Group (HRDAG) found that when the seemingly race-neutral PredPol algorithm was trained on historical drug-crime data, Black neighborhoods were flagged for patrol at roughly twice the rate of white neighborhoods — even though public-health statistics showed little difference in drug use rates across racial and income groups in Oakland (Undark, 2017).
Interpretation: both cases learned from "crime records," not "crime" itself. Sending more patrols to a given area increases the number of recorded incidents there, and that increased count then becomes the justification for the next round of patrol allocation — a feedback loop.
- What China's social credit system actually looks like: There is no single, unified, nationwide scoring system. It's a fragmented structure mixing regional pilot programs, government blacklists, corporate compliance databases, and private credit scoring (such as Alibaba-affiliated Zhima Credit) (Legal Clarity). The most concrete sanction is the blacklist. Citing data from China's National Credit Information Disclosure System, one report notes that in 2018 alone, millions of people classified as credit-delinquent had their ability to buy plane or train tickets restricted (AFR).
Deep dive
1) Why they resemble each other at the technique level — how the algorithms actually work
PredPol-style, area-based predictive policing and China's predictive-policing systems share the same mathematical skeleton. Both feed in historical data, score the risk level of a given area or person, and allocate patrol, enforcement, or sanction resources based on that score. Whether the input data is arrest records or credit-transaction records, the same feedback loop operates: "you find more where the state already looked."
Analysis: this loop isn't a technical flaw so much as an inherent feature of the design. Places under heavier surveillance produce more anomaly signals, and those signals then justify more surveillance. Regardless of regime, the same math produces the same outcome.
2) The justifications and procedures differ — so why do the outcomes look similar?
Democracies cite legal procedure, judicial review, and civil-society oversight as their justification. But as the Gangs Matrix case shows, when the algorithm's own criteria are opaque, that justification only kicks in after the fact. In the UK, musician Awate Suleiman and others, together with Liberty, sued the Metropolitan Police, and the ICO issued its enforcement notice — but all of this came after the system had already been running for years.
Authoritarian states don't presuppose this kind of procedure at all. A 2024 European Parliament report cites Xinjiang as the archetypal case combining AI-driven surveillance, predictive policing, and detention (European Parliament). Whether procedure exists determines whether after-the-fact correction is even possible — but while the algorithm is actually running, who gets flagged shows a strikingly similar pattern (concentrated among minorities and low-income groups) in both systems.
3) Technology exports and supply chains — convergence that crosses political regimes
The Stanford FSI workshop report notes that low-cost Chinese surveillance technology has been exported to other authoritarian regimes, such as Venezuela's "Homeland" system, underpinning digital repression there. But democratic law enforcement agencies also frequently procure facial-recognition and behavior-analysis technology from private vendors. This convergence of technique follows cross-border commercial technology supply chains rather than political ideology — what matters isn't who sells the technology, but who uses it with no checks in place.
Implications for Korea
In 2023, Seoul announced a plan to replace all of its CCTV cameras citywide with "intelligent CCTV" by 2026. These cameras detect behaviors like loitering, falling, or assault and automatically send footage to a control center when a risky situation is detected (Seoul Media Hub, 2023). This isn't yet at the "predict risky individuals" stage of the UK's Gangs Matrix or the US's PredPol, but the underlying direction — training on historical data to flag anomaly signals — sits on the same trajectory.
Before a similar debate takes off domestically, designing an audit system that lets outside parties verify the algorithm's criteria, along with an appeals process for misdetection or bias, up front is cheaper than the litigation and trust-rebuilding costs that come later.
Outlook & variables to watch
- Positive variables: precedents of after-the-fact judicial and regulatory intervention are accumulating, such as Liberty's lawsuit over the Gangs Matrix and the ICO's enforcement notice in the UK. The EU AI Act imposes impact-assessment and transparency obligations on high-risk AI systems used for policing purposes.
- Risk variables: the commercial supply chains for facial-recognition and behavior-analysis technology trade regardless of political regime. When "safety" as a justification overrides procedural checks, transparency can get pushed aside even in democracies.
- Checkpoints: whether external audits and effectiveness verification are mandatory when predictive policing or intelligent CCTV is adopted, and whether the results of algorithmic impact assessments are made public.
Conclusion
- The line dividing surveillance technology is transparency and accountability, not political regime.
- At the technique level, democracies and authoritarian states are converging, but whether after-the-fact judicial and civil-society oversight exists still differs by regime.
- Since domestic discussion of intelligent CCTV and predictive policing in Korea is still at an early stage, it's cheaper to think through audit and transparency systems first, before the technology itself.
References
Contents
Related posts
Why Can't AI Do Just This One Thing? — The Concept of the 'Jagged Frontier'
Why AI outperforms humans on some tasks but actually hurts performance on other, seemingly similar ones — explained through the Harvard-BCG field experiment and the theory of 'asymmetry of verification,' along with criteria for deciding how far to trust AI on a given task.
Why AI Regulation Differs by Country — Three Paths Rooted in Political Systems
The EU legislates, the US relies on corporate pledges, and China uses registration. These different shapes of AI regulation stem not from differing attitudes toward AI, but from how each country's political system is built to make and enforce rules in the first place.
When Models Become Commodities, Where Does the Money Flow? — Moats in the AI Value Chain
Model prices keep falling while model companies' revenue surges, and 89% of firms use AI but only 37% feel a profit impact. This piece traces that gap across the AI value chain's layers.
The AI bubble debate: why it recurs every few months — patterns from the railway and dot-com bubbles
Railway and dot-com bubbles shared one pattern: infrastructure came before monetization, and crashes sorted winners from losers rather than killing the tech. Why the AI bubble debate keeps recurring.