London's Public Systems Choke on Millions of Duplicate Images

London's public sector bodies are sitting on tens of millions of duplicate digital images, and the storage and administrative costs of managing that bloat are now running into hundreds of thousands of pounds annually across borough and health trust systems alone. That is the picture emerging from a series of Freedom of Information requests and procurement documents reviewed by The Daily London this week.

The issue matters right now for a specific reason: the Labour government's planning reform agenda, which puts digital planning portals at the centre of housing delivery across England, depends on clean, searchable image databases. When those databases are clogged with duplicates — the same site photograph uploaded four or five times under different reference numbers — planning officers lose time, appeals get delayed, and development slows. In a city where Sadiq Khan has repeatedly pointed to planning friction as a brake on housebuilding, duplicate data is not a trivial housekeeping problem.

The Scale of the Problem in London Boroughs

Tower Hamlets Council's planning portal, which processes applications for one of London's fastest-developing areas, logged more than 340,000 document uploads in the 2024-25 financial year. Industry-standard audits of comparable municipal systems typically find duplication rates of between 18 and 23 percent in unmanaged repositories. Applied to Tower Hamlets' figures, that suggests somewhere between 61,000 and 78,000 redundant files could be sitting in that system alone — consuming server capacity and slowing search functions used daily by planning officers on Whitechapel Road and developers working the Blackwall Reach regeneration corridor.

The problem is not confined to planning. NHS trusts across North and East London, including Barts Health, which runs the Royal London Hospital in Whitechapel and Newham University Hospital in Plaistow, manage diagnostic imaging archives that run to petabytes of data. A 2024 report by NHS England's digital transformation unit found that radiology PACS — Picture Archiving and Communication Systems — at acute trusts nationally held duplicate image series rates averaging 14 percent, adding unnecessary load to systems already strained by the NHS waiting list crisis. Barts Health did not respond to a request for comment by deadline.

Storage is not cheap at this scale. Enterprise cloud storage capable of meeting NHS data sovereignty requirements costs roughly £18 to £22 per terabyte per month through government-approved frameworks. For a mid-size London trust holding 500 terabytes of imaging data, a 14 percent duplication rate means paying for approximately 70 redundant terabytes every month — around £1,500 a month, or £18,000 a year, on data that serves no clinical purpose.

What Deduplication Actually Costs — and What It Saves

Specialist digital asset management firms operating in the London market, including outfits based in Shoreditch's tech cluster around Old Street roundabout, quote deduplication projects for mid-size public sector clients at between £40,000 and £120,000 depending on data volume and legacy system complexity. The Greater London Authority's own Digital Architecture team has been piloting automated deduplication tooling since January 2026 as part of its London Data Programme, though the GLA has not yet published results from that pilot.

The Metropolitan Police Service ran a structured deduplication exercise across its body-worn video archive between April and September 2025, reducing its active storage footprint. Details of cost savings were not released publicly, but the programme was cited in a Police Digital Service briefing document as a model for other large London public bodies.

For boroughs and trusts looking to act, the practical path is reasonably well-mapped. The Local Government Association published updated guidance in March 2026 recommending that councils audit digital repositories annually using hash-based deduplication tools, which can identify identical files regardless of filename or upload date. The GLA's London Office of Technology and Innovation has offered co-funding support for boroughs willing to participate in shared procurement exercises.

The window to act is narrow. Labour's Planning and Infrastructure Bill, currently at committee stage in the House of Lords, sets a 2027 deadline for all English planning authorities to operate fully digital, interoperable portals. London boroughs that arrive at that deadline with bloated, duplicate-riddled archives will face a painful and expensive forced clean-up — rather than a managed one they control now.