London's Duplicate Image Problem: The Numbers Councils Can't Ignore

London's 33 borough councils collectively manage more than 40 million digital assets across planning portals, housing registers and public-facing websites — and a growing body of internal audit work suggests that duplicate images may account for anywhere between 15 and 30 percent of that total, according to digital asset management reviews carried out across the public sector in recent years. The cost of storing redundant files is no longer trivial.

The issue has sharpened in 2026 because of two converging pressures. Starmer's government is pushing hard on planning reform, which has forced dozens of London boroughs to digitise decades of paper records at speed. At the same time, the Mayor's office has been expanding the London DataStore and related open-data infrastructure, creating fresh pipelines where duplicate imagery gets ingested, duplicated again and then warehoused indefinitely. Nobody is deleting anything.

What the Numbers Actually Show

Cloud storage is not free. Microsoft Azure and AWS both price archival tiers at roughly £0.002 per gigabyte per month at current London-region rates — but active storage tiers, where most council image libraries sit because retrieval speeds matter for planning applications, run closer to £0.018 per gigabyte per month. A single borough like Tower Hamlets, whose planning department processed more than 4,200 applications in 2024-25, can accumulate tens of thousands of site-visit photographs, many uploaded in triplicate by different officers using different devices. Across the whole of Greater London, the redundant storage bill is conservatively estimated by digital consultancies working in the public sector to run into hundreds of thousands of pounds annually — money that could otherwise fund NHS-linked community health workers or housing caseworkers.

The problem is not confined to planning. Transport for London's Operational Technology directorate manages CCTV and sensor imagery from more than 900 road and tunnel cameras, and industry-standard deduplication audits typically find that 20 percent or more of retained frames are functional duplicates created by burst-capture settings. The GLA's own digital team flagged the issue in a 2025 internal review of the London Infrastructure Mapping Application, noting that image duplication was inflating database query times and slowing the tool for users in boroughs like Newham and Waltham Forest who rely on it for regeneration planning.

Where Deduplication Work Is Already Happening

Southwark Council began a structured deduplication programme in January 2026, working with a Bermondsey-based digital agency to audit its housing and planning image repositories. The project, running under the council's Digital Transformation Strategy, targeted roughly 2.8 million stored image files. Early-phase results, presented to the council's overview and scrutiny committee in March, showed that algorithmic deduplication tools identified more than 600,000 candidate duplicate files in the first pass — around 21 percent of the total archive.

Lambeth has taken a different approach, embedding deduplication checks directly into its planning portal upload flow since April 2026, using perceptual hashing — a technique that generates a compact numerical fingerprint for each image and flags near-identical matches before they are written to storage. The system, integrated with the Idox planning software platform already in use across much of London, rejected more than 3,000 duplicate uploads in its first eight weeks of operation.

The financial case is straightforward. If a borough cuts its active image storage load by 20 percent, that translates to a direct budget saving. For a mid-sized borough managing 500 gigabytes of active image storage — a conservative figure — the annual saving runs to roughly £2,100. Across all 33 boroughs, aggregate savings could exceed £70,000 per year from storage costs alone, before counting the staff time saved on manual deduplication and the performance improvements to public-facing planning portals.

The practical next step for boroughs not yet running deduplication programmes is to commission a baseline audit — most can be completed within six to eight weeks using existing digital teams supplemented by specialist tooling. The London Office of Technology and Innovation, based near Victoria Street in Westminster, has published open-source guidance on perceptual hashing approaches suitable for local government use. Boroughs that delay are not just paying for redundant data. They are also building technical debt that will make future migrations, whether to new planning platforms or consolidated GLA data infrastructure, significantly more expensive to execute.