Publicly accessible data and computer vision for retrofit planning: evidence from London social housing

Buildings account for approximately 34% of global CO 2 emissions, placing the built environment at the centre of climate action. Ageing social housing represents a particularly under-studied challenge: in the United Kingdom, around 78% of the building stock predates 1980, and nearly 60% of buildings carry poor energy performance ratings. Systematic retrofit planning is further hampered by building data that is incomplete and fragmented across disparate sources. This paper presents a pilot study investigating data-driven approaches to support social housing retrofit planning across several London boroughs. Rather than proposing a deployable pipeline or a novel neural-network architecture, its purpose is to test what can be inferred from currently accessible property and image data, and to identify the technical and organisational difficulties that arise when doing so. Using publicly accessible property datasets, computer vision training data, and street-level imagery, convolutional neural networks were fine-tuned to predict two retrofit-relevant building characteristics: wall construction type and architectural style. These two tasks were selected as diagnostic cases: wall construction is directly relevant to fabric-first retrofit decisions, while architectural style provides a visually observable proxy for construction age and illustrates the opportunities and risks of using existing non-retrofit image datasets. In-distribution (ID) accuracy reached up to 94%; however, performance declined substantially when models were evaluated against out-of-distribution (OOD) property samples provided by local councils. A structured analysis of the ID-OOD image domain shift revealed a gap in terms of pixel statistics, frequency profile, and semantic content that is consistent with the performance drop for each task. This highlights the generalisation challenges inherent in applying models trained on publicly accessible data to real-world contexts, and underscores the need for broader, purpose-built labelled datasets. A stakeholder workshop with representatives from local government, retrofit organisations, and industry consultancies complemented the technical findings. Discussions surfaced practical barriers to data use in the retrofit sector, including inconsistent data formats, limited inter-organisational data sharing, and a lack of coordination across the supply chain. The findings suggest that publicly accessible housing data and machine learning hold genuine promise for large-scale urban retrofit characterisation, but that realising this potential will require greater data standardisation, coordinated investment in labelled datasets, and closer collaboration across the sector.