Subjective Street Scene Perceptions for Shanghai with Large-Scale Application of Computer Vision and Machine Learning

EasyChair Preprint no. 6166

Abstract

An increasing number of new studies emerged to apply computer vision (CV) to street view imagery (SVI) dataset to objectively extract the view indices of various streetscape features such as trees to proxy urban scene qualities. However, human perceptions (e.g., imageability) have a subtle relationship to visual elements which cannot be fully captured using view indices. Conversely, subjective measures using survey and interview data explain more human behaviors. However, the effectiveness of integrating subjective measures with the SVI dataset has been less discussed. To address this, we integrated crowdsourcing, CV, and machine learning (ML) to subjectively measure four important perceptions suggested by classical urban design theory. We first collected experts’ ratings on sample SVIs regarding the four qualities which became the training labels. CV segmentation was applied to SVI samples extracting streetscape view indices as the explanatory variables. We then trained ML models and achieved high accuracy in predicting the scores. We found a strong correlation between the predicted complexity score and the density of urban amenities and services Point of Interests (POI), which validates the effectiveness of subjective measures. In addition, to test the generalizability of the proposed framework as well as to inform urban renewal strategies, we compared the measured qualities in Pudong to other five renowned urban cores worldwide. Rather than predicting perceptual scores directly from generic image features using a convolution neural network, our approach follows what urban design theory suggested and confirms various streetscape features affecting multi-dimensional human perceptions. Therefore, its result provides more interpretable and actionable implications for policymakers and city planners.

Keyphrases: Computer Vision., Global comparison., human perception., Street view image., Subjective measure.

@Booklet{EasyChair:6166,
  author = {Waishan Qiu and Wenjing Li and Xun Liu and Xiaokai Huang},
  title = {Subjective Street Scene Perceptions for Shanghai with Large-Scale Application of Computer Vision and Machine Learning},
  howpublished = {EasyChair Preprint no. 6166},

  year = {EasyChair, 2021}}