How London's Planning System Got Buried in Duplicate Images — and Why It Matters Now

London's 33 borough councils are sitting on a collective backlog of roughly 4.2 million duplicate image files embedded in planning application records, according to figures compiled by the Greater London Authority's digital infrastructure unit earlier this year. The problem is not new. But a March 2026 directive from the GLA, effective 1 September, is demanding that every borough adopt a standardised image deduplication protocol before any new planning data feeds into the Mayor's unified London Development Database — and the clock is running out.

The timing matters because Keir Starmer's government has staked serious political capital on accelerating housebuilding, with planning reform central to the Labour manifesto and the Housing and Planning Act 2025 now in force. Sadiq Khan's office is under pressure to show that the capital's planning infrastructure can actually process the volume of applications that national housing targets require. A system clogged with redundant image data is not a bureaucratic footnote. It is a practical brake on approval speeds.

How the Problem Was Built, File by File

The roots go back to around 2003, when boroughs began digitising paper planning files in earnest. The London Borough of Southwark's planning department, one of the earliest to go fully digital, used a batch-scanning system that automatically created duplicate entries whenever a file was re-opened or re-submitted. The same issue afflicted Islington, Tower Hamlets, and most other inner-London boroughs within a few years. Nobody at the time established a cross-borough standard for image hashing or deduplication.

By 2015, the problem had compounded. The Planning Portal — the national submission gateway managed by TerraQuest Solutions and used by applicants across England — introduced a new document upload system that preserved file names as submitted rather than assigning unique identifiers. An architect submitting six versions of the same site elevation photograph, each named "photo1.jpg", would generate six separate entries in a borough's back-end database. Multiplied across tens of thousands of applications annually, the arithmetic becomes punishing quickly.

The GLA's digital unit estimated in a January 2026 internal review that Hackney Council alone holds approximately 340,000 duplicate image records across live and archived cases. Wandsworth, which has processed a high volume of permitted development applications along the Nine Elms corridor near Battersea Power Station, has a figure closer to 290,000. Neither borough currently has dedicated resource to address it manually.

What the September Deadline Actually Requires

The GLA's directive, formally titled the London Planning Data Standards Update 3.1, requires boroughs to implement image deduplication using a perceptual hashing algorithm — a technique that identifies visually identical or near-identical photographs even when file names differ. Boroughs have the option to procure commercial tools or use an open-source library already tested by the GLA's own team at City Hall on Queen Victoria Street.

Several boroughs are already moving. Camden Council signed a contract worth £180,000 in May 2026 with a specialist data management firm to run a retrospective clean on its entire planning archive, covering cases back to 2001. Lewisham has taken a different approach, partnering with King's College London's Urban Analytics group to run a pilot deduplication exercise on 12 months of recent applications first.

The practical stakes extend beyond tidy databases. Planning officers searching application records spend an estimated 18 minutes per case, on average, navigating duplicate image sets when assessing heritage impact or neighbour objection documents, according to a 2025 survey by the Planning Officers Society. Eliminating that friction is not trivial when London boroughs collectively processed just under 90,000 major and minor planning applications in 2024-25.

Boroughs that miss the September deadline will not lose planning powers, but their application data will be excluded from the Mayor's consolidated feed into the national Planning Data Platform, which Homes England uses to track delivery against housing targets. That exclusion has teeth. Councils with slower approval records are already scrutinised under the Housing and Planning Act's intervention provisions. For planning teams in boroughs such as Brent and Barking and Dagenham, where housing delivery pressure is acute, falling off the GLA's data platform would be a significant political embarrassment heading into the 2027 local elections.