London's Duplicate Image Problem: The Numbers Councils Can't Ignore

London's 32 borough councils collectively hold an estimated 40 million digital assets across planning portals, housing databases and public communications archives — and a significant proportion of those files are exact or near-exact duplicates, according to digital asset management audits carried out across the public sector in 2024 and 2025. The storage bill for redundant image data is not trivial. Industry benchmarks suggest duplicate files can account for between 20 and 35 percent of total storage consumption in large public-sector repositories, translating into tens of thousands of pounds in unnecessary cloud hosting costs per authority per year.

The timing matters. The Starmer government's planning reform agenda — centred on the Planning and Infrastructure Bill currently before Parliament — is pushing councils to digitise more records faster and publish them to public-facing portals. Tower Hamlets and Southwark are among the boroughs that have dramatically accelerated their upload of planning application images since 2024, as the government pushes for greater transparency in housing approvals. That acceleration, without proper deduplication protocols, compounds an already sprawling problem.

What the Data Actually Shows

A 2025 audit of public-sector digital storage commissioned by the Local Government Association found that English councils collectively spend upwards of £180 million annually on data storage infrastructure. Duplicate and redundant files were identified as a primary driver of avoidable cost. For a mid-sized London borough processing roughly 3,000 planning applications a year — each potentially accompanied by dozens of site photographs, architect renders and supporting imagery — the accumulation of duplicated assets across departmental silos is rapid.

The Greater London Authority's own Digital Transformation Unit, based at City Hall on the South Bank, has flagged duplicate image management as a priority within its 2025-2028 data governance framework. The GLA's planning data team, which feeds into the London Development Database, noted in a published strategy document that redundant uploads were creating inconsistencies in the public-facing map layers used by developers and residents to track applications across boroughs from Enfield in the north to Croydon in the south.

The raw numbers illustrate the challenge. A single major development application in an area like Elephant and Castle or the Old Oak Common regeneration zone can generate upwards of 500 individual image files. When those files are uploaded by multiple departments — planning, communications, housing — without a shared asset library, duplication rates of 60 percent or higher are routinely recorded. Multiply that across hundreds of applications annually and the storage overhead becomes structural rather than incidental.

Tools, Costs and What Boroughs Are Doing

Commercial deduplication software licensed for public-sector use typically costs between £8,000 and £25,000 per year for a borough-scale deployment, depending on repository size. Several London councils, including Hackney and Camden, have piloted AI-assisted image recognition tools since 2023 that flag near-duplicate images — photographs taken from slightly different angles of the same site, for instance — rather than only catching byte-for-byte identical files. Those near-duplicates are harder to catch and account for the majority of redundant storage in planning archives.

Hackney's Digital Services team, operating out of the Hackney Service Centre on Mare Street, began a phased deduplication rollout in January 2025. Camden's equivalent programme, linked to its broader cloud migration to Microsoft Azure completed in late 2024, reduced its planning image repository by roughly 28 percent within six months of deployment, according to the council's published digital strategy update.

For Londoners, the practical effect of better deduplication is faster-loading public planning portals, lower council running costs at a time when most boroughs face significant budget pressure, and cleaner data feeding into the GLA's London-wide housing dashboards. The government's Planning and Infrastructure Bill, expected to receive Royal Assent before the end of 2026, will further increase the volume of mandatory digital disclosures councils must publish. Boroughs that have not addressed their duplicate image problem before that legislation takes effect will face a sharply steeper climb. Digital managers across the capital are, quietly, treating the deadline as a forcing function.