London's Duplicate Image Problem: How the Capital Stacks Up Against New York and Amsterdam

London's planning and public records infrastructure is sitting on hundreds of thousands of duplicate digital images, many of them filed across the Greater London Authority's development portal and the 32 individual borough systems that still operate independently. The backlog is creating real friction for architects, developers and residents trying to navigate housing applications at a moment when the Starmer government is betting its entire domestic legacy on getting homes built faster.

The problem surfaced prominently earlier this year when Southwark Council's planning portal — handling some of the heaviest application loads in England, covering the Elephant and Castle regeneration zone and the Old Kent Road corridor — reported that its document management system contained multiple instances of the same site photography and heritage images filed under different reference numbers. Processing staff were spending time manually cross-checking submissions rather than advancing decisions. Southwark is not alone.

Why This Matters Now

The timing is awkward for City Hall. Mayor Sadiq Khan's London Plan sets a target of 52,000 new homes per year across the capital. Government pressure to hit that figure has pushed planning departments to digitise faster than their quality-control processes can keep pace. When a single planning application for a mid-size Hackney development can generate 400 or more separate image files — site photos, heritage assessments, environmental surveys — the duplicate rate compounds quickly. Industry observers have estimated that across major UK local authorities, duplicate and near-duplicate files account for between 15 and 25 percent of total document storage, though no single authoritative national figure has been published.

The GLA's Digital Planning programme, launched formally in 2023 and backed by central government funding through the Department for Levelling Up's successor body, has been piloting automated deduplication tools in partnership with three pilot boroughs: Croydon, Tower Hamlets, and Haringey. The tools use image-hash comparison to flag identical or near-identical files before they are formally registered in the system. Tower Hamlets — which covers the Whitechapel and Bethnal Green areas where development pressure is intense — began its pilot in January 2026.

How London Compares Globally

New York City's Department of City Planning rolled out a system-wide deduplication layer across its Digital City Record portal in 2024, after a review found the borough-level upload system was generating significant redundancy in Brooklyn and Queens planning files. The city moved to a centralised hash-check at the point of upload, meaning duplicates are rejected automatically before they enter the archive. The result was a measurable reduction in storage costs and faster retrieval times for staff. Amsterdam's Omgevingsloket — the Netherlands' integrated environment and planning portal — operates a single national upload gateway that handles deduplication at source, meaning individual municipalities never hold their own redundant copies.

London's structure is different and the comparison is instructive. Because English planning remains a function of individual local authorities rather than a centralised national body, each of London's 33 planning authorities — the 32 boroughs plus the City of London Corporation — maintains its own document store. That fragmentation is the root cause. The GLA has no direct power to compel boroughs to adopt unified systems, only to incentivise them through funding conditions attached to programmes like the Digital Planning pilot.

The Bartlett School of Planning at University College London has been tracking the issue as part of broader research into planning digitalisation across European capital cities. The school's work, ongoing since 2024, places London behind Amsterdam and Copenhagen but ahead of Paris, where legacy data structures in the national permis de construire system create similar but arguably more entrenched duplication problems.

For developers and residents, the practical advice is straightforward: when submitting planning applications to any London borough, standardise image file naming conventions and use the Planning Portal's central upload function rather than emailing files directly to case officers. Direct email submissions are more likely to be duplicated across internal systems. The GLA has published guidance on its Digital Planning pages — updated in March 2026 — that sets out best-practice file management for applicants. Boroughs not yet in the pilot are expected to be offered expanded access to the deduplication tools from early 2027, subject to the pilot's evaluation results due at the end of this year.