London's Duplicate Image Problem: How the Capital Stacks Up Against New York, Berlin and Tokyo

London's planning and heritage agencies are sitting on a growing backlog of duplicate images — tens of thousands of near-identical photographs stored redundantly across borough portals, Historic England's archive and the Greater London Authority's own digital infrastructure — and the effort to clean them up has become a quiet but contested technical and bureaucratic project with real money attached to it.

The problem is not abstract. Duplicate imagery wastes server capacity, slows searches for planning officers and distorts heritage assessments when the same photograph of a building appears under multiple reference numbers. The GLA's digital transformation team acknowledged the issue internally as far back as 2024, when a cross-borough audit of planning portal submissions flagged that roughly one in five uploaded images was a duplicate or near-duplicate of material already held elsewhere in the system. The audit covered submissions from thirty-two London boroughs processed through the Planning Portal's national infrastructure.

What London Is Doing — and Where It's Falling Behind

The City of London Corporation has piloted a deduplication programme using perceptual hashing tools across its conservation area image libraries since January 2026. The technology assigns a fingerprint to each photograph and flags near-matches for manual review before archiving. Southwark Council began a similar exercise in March covering images attached to the Elephant and Castle regeneration scheme, one of the densest clusters of planning submissions in inner London over the past decade.

Neither programme is yet operating at scale. The City of London's pilot covers roughly 14,000 images — a fraction of the estimated 800,000 photographs held across all London borough planning databases, according to figures presented at a Local Government Association digital infrastructure seminar in April 2026. Southwark's project is staffed by a single contract data officer working a three-day week.

Compare that with New York City, where the Department of City Planning deployed an automated deduplication layer across its ZOLA land-use mapping system in 2023, processing upward of 2.3 million images over eighteen months and reducing redundant storage by an estimated 34 percent. Berlin's Senatsverwaltung für Stadtentwicklung completed a full deduplication pass of its Stadtplan image repository in late 2024, aided by federal digitalisation funding under Germany's Onlinezugangsgesetz programme. Tokyo's Bureau of Urban Development has embedded deduplication checks at the point of upload since 2022, meaning duplicate images are rejected before they enter the archive rather than discovered years later.

London has no equivalent upload-stage filter in any of its thirty-three planning authorities. Each borough manages its own image storage, and while the Planning Portal provides a shared submission interface, deduplication is left to individual councils. That fragmentation is the core of the problem.

Why It Matters Now — and What Comes Next

The timing is not incidental. The Starmer government's Planning and Infrastructure Bill, currently before Parliament, places new obligations on local authorities to digitise and publish planning records in standardised, machine-readable formats. The Department for Housing, Communities and Local Government has set a compliance deadline of April 2028 for all English councils. Duplicate image data will complicate that process: a record containing multiple versions of the same photograph cannot be cleanly converted into a standardised open-data format without prior cleaning.

Historic England, which maintains the National Heritage List for England from its offices on Waterhouse Square in Holborn, has flagged the issue in its 2025-26 annual digital strategy, noting that archive quality across local authority submissions remains inconsistent. The organisation has not yet published a deduplication standard for heritage imagery, though work on one is understood to be ongoing.

For planning officers and researchers working with London's built environment records, the practical advice for now is straightforward: when submitting images to any borough portal, use unique filenames tied to specific dates and site references, and avoid re-uploading photographs from previous applications. Several boroughs, including Hackney and Lambeth, have begun issuing updated submission guidance along those lines as interim measures while the broader technical fix remains pending. The gap between London's current position and what New York or Berlin have already achieved is measurable in years, not months.