London's Fight Against Duplicate Images in Public Records: How the Capital Compares to New York, Amsterdam and Singapore

Thousands of duplicate images are jamming London's public planning portals, slowing down housing applications and frustrating architects, developers and residents trying to navigate an already overloaded system. The problem is not unique to the capital, but a comparison with New York, Amsterdam and Singapore suggests London is falling behind on the technical fixes that other major cities have already begun rolling out.

The issue matters right now because the Starmer government has staked significant political capital on accelerating housing delivery through planning reform. Every bottleneck in the system — including the mundane, unglamorous problem of bloated digital archives stuffed with redundant image files — pushes timelines back and adds cost. The Greater London Authority estimates that the London Development Database, which tracks major planning applications across all 33 boroughs, now hosts document packages running well into the gigabytes for individual sites, partly because applicants routinely resubmit identical supporting photographs and site plans under different file names.

Where London Is Now

The most visible pressure points are in the inner boroughs. Southwark Council's planning portal, which handles applications for major regeneration zones including the Old Kent Road Opportunity Area, has been flagged internally for hosting substantial volumes of duplicate image content across multi-phase applications. In Tower Hamlets, where development activity around Whitechapel and Poplar has intensified since 2023, planning officers have described spending significant staff time manually cross-referencing image submissions — work that image-deduplication software could automate.

The London Digital Twin programme, run under the GLA's Smart London initiative, is the most relevant infrastructure project addressing this. It aims to create a unified spatial data layer for the capital, and deduplication of visual records is a component of that work. But the programme's phased rollout, which began in earnest in 2024, has focused first on 3D modelling and sensor data rather than the planning archive backlog. Meanwhile, Hackney Council piloted an automated document-management system in 2025 that included basic image hash-matching to flag duplicates at the point of submission — a modest but practical step.

The costs add up. Cloud storage for local authority planning portals across London's 33 boroughs is not centrally reported, but sector analysts at public-sector technology consultancy Socitm have previously noted that document storage costs for large urban authorities can run to six figures annually, with redundant data a significant driver. Reducing duplicate image volumes by even 30 percent across a borough's archive can meaningfully cut retrieval times and storage bills.

How Other Cities Are Handling It

New York City's Department of City Planning rolled out an automated deduplication layer within its NYC Zoning Application Portal in late 2024, using perceptual hashing to identify near-identical images submitted across related applications. The system flags duplicates before they enter the archive rather than cleaning up after the fact. Amsterdam's Omgevingsloket — the Netherlands' national planning application platform adopted city-wide — applies similar logic and has the advantage of being a single centralised system rather than London's fragmented borough-by-borough patchwork. Singapore's Urban Redevelopment Authority has gone furthest, integrating image deduplication into its CorNet e-Submission system, which processes building plan submissions, and cross-references visual data against a live digital twin of the city updated quarterly.

London's structural problem is its structure itself. Thirty-three planning authorities, each with its own portal, procurement cycle and IT budget, make a coordinated technical fix far harder to deliver than in Singapore's unitary system or Amsterdam's nationally standardised platform. The GLA has no direct power to compel boroughs to adopt common document-management standards, only the ability to incentivise through grant conditions.

For developers and architects submitting applications now, the practical advice is straightforward: audit your own submission packages before uploading. Remove duplicate site photographs, consolidate design and access statements into single image-embedded PDFs, and use unique file names tied to application reference numbers. That won't fix the systemic problem, but it reduces the chance of your application sitting in a manual review queue. A broader solution — whether through a GLA-coordinated procurement of deduplication tooling or a push to consolidate borough portals onto a shared platform — will depend on political will and budget decisions that have not yet been made.