A Study on an Evaluation of the Managed Residential Environment Improvement Project Using Deep-Learning Model

1. Research Background and Objectives

The managed residential environment improvement project targets detached and multi-family housing areas, released renewal areas, areas planned for renewal, renewal promotion districts, and retention of renewal promotion districts for the comprehensive residential revitalization including social and economic aspects along with improvements in the physical environment (Seoul Metropolitan Government Department of Residential Environment Improvement, 2020). Unlike the simple top-down urban regeneration method, the project also supports the cost of repairing aging residences and the maintenance of key elements for streetscape improvement. Also, support for necessary aspects are expanded by differentiating project elements according to the target area and the development direction for sustainable development is established taking into consideration the historical and cultural elements of the target area, so the project expands the regeneration subject through the participation of residents to establish a safe and comfortable residential environment and the project aims for an effective city regeneration that is tangible and reflects the requirements of the residents. In comparison to renewal projects like redevelopment and reconstruction, the risk of eviction of residents is less, thus, the settlement of existing residents is guaranteed while enabling the improvement of the streetscape and the general environment.

Especially, the significance lies in the provision of residential environment improvement projects tailored for each target site through support of resident life and improvement of the physical environment such as the establishment of criminal activity prevention plans, repavement of roads, and removal of walls, which are closely related to the satisfaction of the residents. Most project target sites are residential areas with poor residential environments that have been neglected and left unattended compared to other residential areas of Seoul due to insufficient development profit or cancellation of the redevelopment and reconstruction plans, and carrying out projects targeting such areas have received high praise for realizing universal housing welfare. However, in contrast to other renewal projects like redevelopment and reconstruction, the managed residential environment improvement project frequently does not result in distinct physical changes after, making it difficult to assess whether the project objectives have been successfully achieved. Also, there is the disadvantage that it is not possible to carry out an objective evaluation of whether the actual environment has been improved after the completion of a project. Due to this aspect, preference for such projects is low in comparison to redevelopment and reconstruction projects.

Until June 2020, a total of 626 redevelopment or reconstruction projects in Seoul have been completed or are in progress. However, despite managed residential environment improvement projects making up 44% of the urban regeneration projects of Seoul, these projects have only been implemented in 84 target sites.¹⁾

Some target sites have been removed despite being set as project target sites due to lack of resident participation.²⁾ Even today in 2020, 7 years after the first project completion in 2013, discussion and public hearings have been attempted with regard to the post evaluation of projects and research related to the evaluation of these projects are lacking as well.

Research has been consistently conducted to evaluate streetscapes and their improvement, and such research has mostly evaluated elements that can be used to determine improvements to the physical environment or through surveys. However, since most utilize qualitative methods like surveys and field studies, the scope of analysis is limited along with the limitation of being labor intensive (Geunduk Park, Soo-gi Lee, 2018)

Meanwhile, with the rapid advancement of big data, machine learning, and deep learning technologies, such technologies are being applied in various fields of research. In the field of urban planning, there are research on the analysis of street environment through deep learning. In previous research, methods of evaluation of landscape and actual green space rate, which analysis of was restricted, have been presented because deep learning has facilitated the analysis of visual elements and detailed quantitative assessment. Therefore, evaluation of managed residential environment improvement projects, which are accompanied by changes in the physical environment, using deep learning will also become possible.

The core of the project is in the real improvement of the living environment of residents and the improvement of the streetscape through changes in the physical environment is an important means to attain this goal. However, not only is it difficult to determine improved areas due to the nature of the project where physical environment improvement is less than that of redevelopments and reconstructions, but an objective method of evaluation for the street environment improvement is lacking and moreover it is difficult to consider the project as sustainable when doubts about the effectiveness of the project can arise.

In order to maintain the momentum of the project and prove its effectiveness, objective and quantitative verification of the effects is necessary. This project results in gradual physical changes during the long project period, so residents have difficulty intuitively recognizing such changes. Additionally, the amount of changes is small, making it difficult to adequately express the effects through the perception of residents or subjective surveys. Thus, the ability to quantitatively represent the landscape improvement effect using an objective evaluation method will play a critical role in increasing the reliability and sustainability of the project.

In this study, street images of the target site of the managed residential environment improvement project target obtained using Naver Street View (NSV) were evaluated through image prediction deep learning analysis and the results analyzed. Also, the physical environment changes from the project implementation was quantitatively analyzed by comparing with areas of similar conditions without the project implementation. Improvements of the streetscape is one of the most visually apparent changes to the managed residential environment improvement project and such improvements can be used to intuitively determine the effect of the project.

In this study, improvements to the streetscape from the project were quantitatively evaluated and verified based on the deep learning analysis results to present its applicability as a method of evaluating the streetscape improvement from the managed residential environment improvement project. Furthermore, the effects were measured in terms of the changes to the physical environment in order to contribute to the setting of the direction of future projects.

1. Streetscape Evaluation

Research have actively been pursued with the aim of evaluating the changes made to the street environment. Most users were given the opportunity to experience the streets and evaluation was carried out using qualitative methods such as a survey. The streetscape evaluation method using a survey can clearly and intuitively reveal the experience of the user and by allowing the users to directly experience the streets, accurate results can be obtained as diverse elements such as harmony and surrounding in addition to the elements that constitute the streets. A study (Byung-sook Choi et al., 1999) that proposed a temporal and spatial methodology to obtain objectified evaluation results of the culture village cultivation project carried out targeting street spaces stated that subjective evaluation as well as objective evaluation are necessary whilst considering the policy and administrative aspects along with the physical aspects necessary for evaluation. A study presented evaluation items for the post evaluation of projects and a survey method based on indices predicted to be influential (Yeon-kyung Cho et al., 2009), and another study aimed to increase objectivity by evaluating the improvements to the street environment through the assignment of weights per index, utilizing environmental perception characteristics (Eun-yeon Koo et al., 2011). These studies employed methods including repeated surveys for increased reliability, more detailed questions, and verifying survey participants by categorizing them in greater detail such as worker, commute to school, and passing through.

Other studies analyzed evaluations by each case through the structuralization of the street environment evaluation content (Gung-ji-hee Nam, So-hyun Park, 2009) and carried out evaluations mainly regarding the physical elements without depending on qualitative methods like surveys (Yoon-nam Chung et al., 2014). Evaluation was performed using the assignment of weighting for each physical environment element or whether they exist or not and whether the standards are appropriate or not, and the objective was to improve objectivity by presenting detailed directions for physical improvements for each target site. Not only that, a variety of methods for evaluating street environment improvement were used including evaluation of psychologically impacting elements and physical elements at the project target site (Jae-hyuk Yang, Kang-hee Lee, 2009).

Such literature mostly selected surveys for the evaluation of streetscape and tried to overcome the objectivity limitation of surveys through various means conducted together. Guaranteeing objectivity by generalizing the subjectivity that can appear from surveys is important for the evaluation of projects and determination of improvements to the streetscape including the assignment of weighting, expansion of the survey group, classification of whether having experience in the streets, and consideration of psychological factors. However, the sample number cannot be indefinitely increased for some of the groups that have experienced the streets. Also, while it is common for surveys to target people with experience, it cannot be said that everyone will have had the same street experience as the street environment was restricted.

Evaluating only with the components without experience of the streets is not qualitative in comparison to studies based on surveys, and while evaluation is more convenient and there could be less chance of interference from subjectivity, the factors of perception and harmony that an individual obtains through experience are excluded, so it cannot be considered that the street environment was completely evaluated.

The managed residential environment improvement project has numerous project target sites and there already exists a number of target sites for which the project has been completed, so experiencing the past street environment or conducting a survey at this point in time is not possible and invalid, making it impractical to compare of the streetscape before and after the project through surveys. Moreover, due to the project producing little physical environmental changes, objectively reflecting the streetscape changed through the project solely using the research methods employed in previous studies is not possible and difficult to quantitatively represent the streetscape change.

2. Deep Learning

Deep learning is a method that has been researched for an extended period of time (Selfridge, 1959). The limitations in computational hardware at the time restricted the active usage of deep learning, but the recent enhancements in computing performance and accuracy has resulted in the application of deep learning in a diverse range of fields. In particular, there have been advancements centered on artificial neural networks that have led to rapid development in voice recognition, image object recognition, and object detection (LeCun et al., 2015; Keun-duck Park, Soo-ki Lee, 2018).

Urban research utilizing deep learning are being carried out. Starting from simply categorizing land coverage using satellite images, applications have developed to the level of being able to determine the deterioration and development of regions (Marcos et al,. 2018; Wurm et al,. 2019). A variety of attempts have are being made including assessing the London landscape (Seresinhe et al., 2017) and identifying slums using image data (Ibrahim et al., 2019). In Korea, various applications have been pursued including constructing a deep learning model using open source data and Google Street View (GSV) and conducting physical environment analysis using that model to predict pedestrian satisfaction (Keun-duck Park, Soo-ki Lee, 2018), analysis of the visual green space rate to analyze its effects on walking time(Dong-hwan Ki, 2020), application in analyzing the effects of urban organizations on the traffic accidents of pedestrians on neighborhood streets (Soo-hoon Park et al., 2020), and predicting housing prices (Hae-jung Jeon, Hye-sun Yang, 2019).

Deep learning analysis does not require field surveys and additional processing of the completed data as was done in previous research. If the data is available, analysis can be performed without temporal and spatial restrictions, overcoming the labor-intensive limitation of previous studies.

Due to such high applicability and convenience, studies have been carried out using deep learning in the evaluation of streetscape and analysis of non-physical elements. Deep learning is actively applied in various studies where perception of the cityscape was predicted using street images (Dubey et al., 2016), visual characteristics were extracted from street images to predict housing prices and population density (Arietta et al., 2014), and a deep learning model was trained using survey data which was then used to predict non-physical elements that compose a city (Naik et al., 2016).

These studies commonly handle more data than landscape evaluation research in the past and presents results that is relevant for a wider range of target sites to overcome the limitations of previous methodology.

As the managed residential environment improvement project is not a project that produces remarkable physical changes, it is difficult to clearly recognize changes to the streetscape. There are also numerous project target sites, so while it is difficult to evaluate the effects of the project using the same methods of landscape evaluation research in the past, applying deep learning analysis enables quantitative analysis of images before and after the project through transfer learning using street images, which is fitting for the evaluation of streetscape changes.

1. Research Scope and Data

In this study, the Naver Street View (NSV) data from 2010 to 2020 and the managed residential environment improvement project plan report provided through the Seoul Metropolitan Government urban regeneration portal. NSV data can be obtained for corresponding time period before and after the project and comparison with the plan report allowed for the intuitive and accurate acquiring of information on whether the project actually was carried out.

As shown in <Table 1>, among the 28 target sites that completed the Seoul Metropolitan Government managed residential environment improvement project up until June 2020, 828 NSV data from 19 target sites were extracted and used after removing 9 target sites for which NSV data acquisition was difficult due to excessive narrow roads and lands that are isolated from roads. However, as shown in <Table 2>, the detailed project elements are not uniformly applied to all the streets existing within the project target sites, so the plan report of the project target site was checked as shown in <Figure 1> and compared with the NSV to select only the streets that the project was actually implemented on as view spots. View spots were selected so that they are simultaneously locations that reflect the application of the detailed project elements as well as locations where the view angle and location of the data before and after the project match.

Table 1.

Target site list

Table 2.

Detailed project element by target site

Figure 1.

Example of process for specifying View Spots and extracting street imagesSource: Redrawn based on Dobonggu (2017)

In order to acquire the data before and after the project using the NSV and exclude the subjectivity of the researcher, the most recent images for before and after time periods were obtained to exclude physical environmental changes caused from factors other than the project. Also, the season of the project target site, the image recording angle, and image location were matched since they can affect the accuracy of the project before and after analysis using deep learning. Analysis using deep learning was limited to the streetscape images and analysis of non-physical elements was restricted, thus, the analysis results were based on the ascertainable visual elements among the project elements.

2. Deep Learning Analysis of the Streetscape Changes

This study employed a deep learning model to evaluate the streetscape of areas where the managed residential environment improvement project was implemented. The deep learning model was trained for landscape image prediction using Naver Street View (NSV) based on the CNN deep learning model after training using Place Pulse 2.0 data.

Place Pulse 2.0 is a collection of street view images of 110,988 locations in 56 cities of 28 countries recorded between 2007 and 2012 that 81,630 people of 162 countries categorized and evaluated based on the 6 indices of safe, lively, beautiful, wealthy, boring, and depressing, making Place Pulse 2.0 advantageous in gaining sufficient objectivity.

In this study, the project was to be evaluated using streetscape image recognition, thus, quantified data was necessary. Since the Place Pulse 2.0 used in the training only provided the superior choice between two compared images rather than numerical data, the Microsoft Trueskill algorithm was used to score Place Pulse 2.0 as quantification through scoring was necessary for the project evaluation (Herbrich et al., 2007; Dubey et al., 2016).

When an image is inputted into a CNN model, data is computed through a series of filters called layers to obtain the classification value and there are 3 layers: convolution, pooling, and fully-connected. In the convolution layer, characteristics of the image are extracted through the activation function like Relu and the dimension of the extracted data is limited in the pooling layer. Afterwards, the weighting is adjusted in the fully-connected layer and the output is predicted. For high prediction accuracy, a large number of labeled image data is necessary, but to overcome this, CNN models that have already been trained such as VGG16, InceptionV3, and ResNet152 were fine tuned for use after carrying out transfer learning for this study(Shin et al,. 2016).

For the deep learning model used in this study, transfer learning was conducted on the convolution layer of the VGG16 model, which completed pre-training using the ImageNet data set with classification annotations such as animal and object, for the streetscape evaluation using the analysis framework of Zhang et al.(2018) that predicted landscape images based on survey data. The VGG16 model is a highly reliable model that was developed by the Oxford Visual Group and was 1st runner-up in the 2014 ImageNet recognition competition. The model offers satisfactory performance and convenient usage.

Additionally, for the training of the deep learning model, overfitting was prevented by using 75% as training data and 25% as validation data for each index.

For “safe”, “lively”, “beautiful”, “wealthy”, “depressing”, and “boring”, 17,918 images, 16,396 images, 14,873 images, 13,438 images, 12,163 images, and 11,935 images were used in the training, respectively, and the accuracy of this model was adequately above 75% for “safe”, “lively”, “beautiful”, and “wealthy”, which had accuracies of 81%, 79%, 80%, and 77%, respectively, while the accuracy was low for “depressing” and “boring”, which had accuracies of 74% and 69%, respectively. The AUC, which represents the overall performance of the model, was 0.89 for “safe”. The AUC was similar to the accuracy order of the indices(see <Table 3>).

Table 3.

Prediction Model Performance

In this study, streetscape images were quantitatively evaluated using the method shown in <Figure 2> and the comparison of the street view images before and after the project at the same street and location was used to evaluate the streetscape changes of the area where the managed residential environmental improvement project was implemented.

Figure 2.

Example of deep learning application method used in studies

Among the 6 indices, “boring” and “depressing” were difficult to intuitively translate their meaning into Korean unlike the other 4 indices. Due to the limitation of “boring” and “depressing” being abstract, they have lower relevance with the streetscape changes and the achievement of the project goal through the managed residential environment improvement project investigated in this study. Compared to the other indices, their training data amount is smaller and their accuracies and AUC are lower, so these were excluded and the 4 indices of “safe”, “lively”, “beautiful”, and “wealthy” were used to evaluate the streetscape before and after the project, analyze the variation of each index value, and determine the Z value and P-value using the paired sample Wilcoxon signed rank test.

3. Correlation and Factor Analysis

The managed residential environment improvement project included aesthetic elements as necessary elements that can change or improve the street environment like crime prevention through environmental design(CPTED) and road repavement. However, the same landscape improvement elements are not applied for all target sites and they are applied differently according to the characteristics and objective of each target site.

The landscape of target sites is created through the harmony of such elements, and in order to determine whether the elements applied through the managed residential environment improvement project actually contributed to the improvement of the streetscape through their interaction, analysis of the variations of the 4 index values of “safe”, “lively”, “beautiful”, and “wealthy” as well as the effect of each index. Therefore, the effects of each index on the other indices were analyzed using the Spearman’s rank correlation coefficient based on the deep learning analysis result.

Changes for each index of the landscape before and after the project can be analyzed through the deep learning analysis, but there is the limitation that the possibility of natural streetscape changes due to the passing of time cannot be completely excluded. In this study, Naver Street View(NSV) data of the same time period was obtained for and comparatively analyzed between target sites where the managed residential environment improvement project was implemented and target sites with similar characteristics as shown in <Table 4>.

Table 4.

Reciprocal comparison by similar target site and project implementation site

Changes in the streetscape was also observed from the NSV data of target sites where the project was not implemented caused by activities unrelated to the implementation of the project including residents painting walls, planting and arrangement of trees, and home improvements and repairs, but for an objective comparison, the data was used regardless of the degree of the changes.

1. Streetscape Improvement Evaluation

Deep learning analysis was performed using 828 NSV data, and the analysis result revealed that all the indices for after the project implementation improved significantly compared to before the project as shown in <Table 5>.

Table 5.

Result of target site Wilcoxon signed rank test

The Wilcoxon signed rank test conducted based on the deep learning analysis result showed that all indices increased after the project compared to before the project. The increases varied by index but all the indices increased significantly by a minimum of 0.07 except for the “beautiful” index. In addition, the P-values of all the indices except for “beautiful” were 0.0000, showing that the results were significant. Especially, in line with the common goal of cultivating a safe and relaxing residential environment for the managed residential environment improvement project, the Z value of the evaluation index “safe”, which represents the feeling of safety from the street images, was 5.03 and the Z value of the evaluation index “wealthy”, which represents the feeling of affluence, was 5.19. The changes in these indices were greater than the other indices. The Z value for the index “beautiful” was a somewhat low 2.99, which is considered the result of the small amount of physical changes that result from the project and the characteristics of the most of the target sizes being deteriorating low-rise residential areas. Thus, it was determined that the street environment improved objectively and quantitatively through the project, and despite the relatively small amount of physical changes in comparison to other renewal projects, significant changes are produced. Also, it was found that the all indices improved after the project when compared to before the project for the 19 target sites analyzed using deep learning. Although the characteristics of each project target site vary, considering the small proportion of target sites with remarkable physical changes, it was found that the streetscape image improves positively regardless of the degree of the physical change when the project is implemented.

2. Correlation Between Indices

The safe, lively, beautiful, and wealthy indices derived through the deep learning analysis did not form a normal distribution, so analysis was carried out using the Spearman’s rank correlation coefficient, which is a non-parametric correlation verification method. As shown in the analysis result of <Table 6>, all indices were found to be correlated.

Table 6.

Spearman’s rank correlation coefficient

The significance probability between indices was found to be significant at 0.000 for all 4 indices. Analysis showed that all the indices had a positive correlation to each other.

When compared with the other indices, the "beautiful" and "lively" indices had somewhat lower correlations, but overall the correlation was high at 0.550 and greater. In particular, the correlation coefficient of the "safe" and "wealthy" indices was 0.797, which was the largest positive correlation.

All the indices had positive correlations with each other in addition to the “safe” and “wealthy” indices, and it can be considered that each street environment element improved through the managed residential environment improvement project used as the basis for the index calculations interacted positively. Therefore, regardless of the degree of the changes in the detailed elements, physical environment, or the focus of the project, the target site of the managed residential environment improvement project implementation, effects can be attained in various aspects in terms of the streetscape improvement. This suggests that through the implementation of the project, positive effects other than the intended objectives can be simultaneously obtained, showing the possibility of producing effects greater than the quantitatively represented values.

3. Factor Analysis

The target sites of the managed residential environment improvement project implementation and similar target sites without the project implementation were analyzed for the same time period. For a fair evaluation, areas that planned for a redevelopment and reconstruction project in the past under the same selection conditions of the managed residential environment improvement project but was cancelled were chosen for the similar target sites. As shown in <Table 7>, target sites without project implementation in comparison to target sites with the implementation showed smaller changes in all the index values and no significant results. On the other hand, the target sites with project implementation exhibited increases for all 4 measurement indices and the Wilcoxon signed rank test was conducted to find that the result is significant.

Table 7.

Comparison result between non-implementation target site and implementation target site

All the Z values for the target sites without project implementation were less than 1.000. It is difficult to interpret this result as showing clear change, but when compared to the Z values for the project implemented target sites all being 2.000 and greater, there is a substantial difference. Similarly for the P-value, the target sites without project implementation resulted in statistical insignificance. Meanwhile, the result value for the target sites with project implementation was very significant at 0.000~0.006.

All index values for the target sites without project implementation also increased slightly with time. Changes to the streetscape of the target sites without project implementation used as comparison during the progress of the project cannot be completely controlled, so factors that affect the streetscape including the voluntary improvement and repair of homes by residents with the passing of time can improve the streetscape even for target sites without project implementation. However, it is difficult to consider this causing significant change. The analysis result means that with time the indices can change due to the occurrence of natural change, but also that it is not a significant result. This excludes the possibility of intentional index changes by the subjectivity of the researcher, and it can be considered that the project objectively contributed to the improvement of the street landscape.