How London's Planning Archives Ended Up Full of the Same Image Twice: The Story Behind the Duplicate Problem

The Greater London Authority holds tens of thousands of digital images tied to planning applications, heritage assessments, and development proposals stretching back to the early 2000s. A significant portion of those records, according to planning professionals familiar with the system, contain duplicate image files — the same photograph or architectural render uploaded multiple times under different reference numbers, bloating databases and, more critically, creating confusion during planning reviews and public consultations.

This is not a trivial housekeeping problem. With Mayor Sadiq Khan's administration pushing hard on housing delivery under the London Plan and the Starmer government demanding faster planning decisions from local authorities, the integrity of digital records has become a live operational issue. Duplicate images slow down document searches, inflate storage costs, and in some cases have led to planning officers reviewing the wrong version of a site photograph during application assessments.

How the Duplication Problem Built Up Over Two Decades

The roots go back to the early days of digital planning submissions. Before the Planning Portal introduced its current unified upload system, London's 33 boroughs each ran their own document management processes. Islington Council, for instance, operated a separate in-house system through much of the 2000s, as did the London Borough of Southwark. When the GLA began centralising certain strategic planning records after 2008, files were migrated in bulk. Nobody systematically deduplicated.

The problem compounded further when individual boroughs began scanning legacy paper records as part of the government's digital-by-default push after 2013. Tower Hamlets alone digitised thousands of documents related to Canary Wharf-area developments that had originally been submitted in hard copy. Those scans were often uploaded alongside the original digital files that applicants had already submitted electronically, creating parallel records of the same image with no automated flag to catch the overlap.

By 2019, the Planning Advisory Service — the Local Government Association body that audits planning department performance — had identified digital record management as a systemic weakness across English local authorities. The GLA's own digital infrastructure review, carried out in the spring of 2022, flagged that certain planning application databases contained image duplication rates estimated at roughly 30 percent of total stored files, though the figure varied significantly between departments.

What Comes Next for London's Planning Records

The practical consequences have landed hardest on the teams handling major development corridors. Along the Thames Tideway zone — stretching from Battersea Power Station east through to Greenwich — multiple large-scale mixed-use schemes have generated application files running to hundreds of documents each. Navigating those records efficiently matters when planning committees at Lambeth, Wandsworth, and Lewisham are all working on adjacent sites under tight statutory determination deadlines.

The GLA's Digital Planning Programme, which received funding through the Department for Levelling Up, Housing and Communities before that department's 2023 restructure into what is now the Ministry of Housing, Communities and Local Government, has been developing deduplication protocols since late 2024. The programme is working with Planning London Datahub — the GLA's central data repository based in City Hall on Kamal Chunchie Way in Newham — to run automated hash-matching across image files to identify and flag exact and near-exact duplicates before human review.

For planning officers and members of the public engaging with London development proposals, the practical advice is straightforward. When accessing application documents via the Planning Portal or individual borough websites, check the document reference date and submission batch number rather than relying on image filenames, which are frequently auto-generated and non-specific. If the same photograph appears under two different reference codes in a public consultation file, it is worth querying the relevant borough planning department directly in writing — that query becomes part of the application record.

The broader fix is structural. Full deduplication across the GLA's strategic planning image archive is projected to complete by early 2027, according to the programme's published roadmap. Whether that timeline holds will depend partly on resourcing decisions that have not yet been finalised in the current spending review cycle.