A Study on the Machine Learning-Based Apartment Price Index

1. Research on House Price Indices

House price indices can broadly be classified into transaction-based indices and appraisal-based indices (Lee and Lee, 2008; Park, 2009; and Geltner, 2011). According to Geltner (2011), transaction-based indices can further be classified into methods that use statistical models and those that simply sum up the average or median values of transaction prices at every point. Indices based on average or median values have the advantage of not requiring analysts’ statistical knowledge and computing performance. However, they cannot facilitate comparisons among points due to the uncontrolled heterogeneity of houses transacted at each point. Statistical models are used to control houses’ heterogeneity, and two representative types are the repeat sales index and the hedonic price index. The repeat sales index controls the heterogeneity of houses through the method of producing an index by using trading pairs of identical houses that have been transacted at least twice. In this approach, it is assumed that changes in the quality of houses do not occur over time (Barr et al., 2017). The hedonic price index relies on the hedonic price model, where the sales prices of houses are the dependent variable, while various characteristics of houses serve as explanatory variables. By incorporating houses' characteristics as explanatory variables in the model, the cross-sectional fluctuations of house prices can be controlled. Transaction point information is entered in the model as a dummy variable, then serial fluctuations in house prices are captured based on the regression coefficient of transaction point dummy variables (Geltner, 2011). Additionally, with the hedonic price index, there is also the time-varying parameter model, where a model consisting of identical combinations of characteristic variables at each point is estimated, and identical characteristic values are entered in each cross-section model to estimate houses’ potential prices, thus calculating the index (Lee, 2007).

One major drawback of the transaction-based index is the requirement to secure enough transactions. Park (2007) demonstrated this issue through simulations in a market with low transaction frequency, where a hedonic price index was generated. The results showed that when transaction samples were inadequate, the volatility of the index increased, and the influence of outliers became significant. Consequently, when small areas such as towns (eup), townships (myeon), and neighborhoods (dong) or station areas are considered as the spatial scope of index production or when transactions decrease during real estate downturns, there is a possibility of encountering substantial noise and producing an unstable index due to the failure to obtain sufficient transactions. In relation to this, there has been an increase in the demand for subdivided indices at the level of small areas, driven by issues such as neighborhood-specific real estate policies. Song et al. (2020) have also argued that, to counter the instability of the index, corrective methods such as moving averages or sample overlaps should be employed.

Next, the appraisal-based index constitutes a method in which individual house prices are regularly appraised, and the price index is calculated through the flow of housing market capitalization produced with the appraisal price per house as the basic price. According to Suh (2009), in South Korea, KB Kookmin Bank (hereafter “KB”) and Real Estate 114 (R114) have produced price indices based on appraisal prices. R114 has investigated the upper and lower limit prices per apartment complex and per area through real estate agents. Similarly, KB has utilized the method of investigating market prices for every point based on transaction prices in cases where transactions occurred. When there are no transactions, they rely on cases of nearby transactions collected by real estate agents in relevant areas. As of April 2022, a total of 36,300 sample houses nationwide were being tracked.

With the appraisal-based method, under the premise that there exist accurate appraisal prices at each point, the price index can be produced in a comparatively intuitive manner. Unlike the transaction-based index, a stable index flow can be constructed even when the number of transaction samples is insufficient. However, the following drawbacks have been pointed out. First, the scope of price index production is limited to certain groups of houses that are regularly appraised (Hill and Steurer, 2017). This is because a vast number of appraisers is necessary in order to appraise potential prices at points where transactions have not occurred for the total houses. As a result, fluctuations in the prices of houses other than sample houses are not adequately reflected in the index. The second issue that has been identified is the phenomenon of smoothing. This arises from the conservative approach taken by human appraisers when evaluating individual houses. They tend to base their assessments on past appraisal cases of the same houses, leading to a tendency to make cautious adjustments. Geltner (1991) has explained this phenomenon with the following “partial adjustments model”:

is the true appraisal price of house i at point t, and is the purely random appraisal error. However, , the final appraisal price of house i at point t, is influenced by , the final appraisal price at the immediately preceding point, and the degree of such influence changes according to αⁱ, which signifies “confidence” regarding the house’s appraisal price at the immediately preceding point i. The parameter αⁱ has a value ranging from 0 to 1. When α αⁱ is 1, then the immediately preceding appraisal price is disregarded. On the contrary, when αⁱ is 0, then appraisal is not performed at the current point, and the immediately preceding appraisal price is extended without any adjustment. Geltner (1991) has argued that, no matter how accurate individual house price appraisals are, unless prices are completely recalculated at every point, smoothing will persist in the index consisting of appraised houses. Lee and Lee (2008) have supported this notion, revealing that αⁱ is measured to be 0.35 or 0.48 in KB’s appraisal-based house price index, which demonstrates the existence of the smoothing phenomenon. This smoothing presents advantages such as reduced sensitivity to outliers in house transactions and the potential for decreased noise in the price index. However, there are drawbacks associated with this approach. One such drawback is the difficulty in capturing turning points in the housing market and using this index as a basis for timely policy establishment. This is due to their inherent lag, exhibiting a time delay of approximately 1-2 quarters compared to the current market situation (Lee and Lee, 2008; Hill and Steurer, 2017; and Song et al., 2020). In addition, Eriksen et al. (2019) have also pointed out the possible existence of the problem of a conflict of interest. This occurs when certified public appraisers, in the process of carrying out their duties, may introduce biases in house price appraisals due to influence from interested parties such as licensed real estate agents in the areas.

2. Research on ML-based Price Indices

In recent years, there have been attempts to construct price indices through ML techniques, which exhibit higher accuracy than traditional statistical models, to overcome the limitations of existing house price indices. Barr et al. (2017) created a house price index for Los Angeles during 2000-2016 at the individual ZIP code level. To achieve this, a cross-section regression model where house transaction prices were calculated per quarter was constructed through the gradient boosting method, an ML technique. For comparison purposes, the median transaction price index and the repeat sales index were also calculated under identical conditions. Pairs of repeated transactions of identical houses were extracted from a randomly separated test set, and price increase rates predicted for respective trading pairs through actual price increase rates and index were contrasted to evaluate the accuracy of the index. The analyses revealed that noise was less, and accuracy was higher in the ML-based price index than in median prices and the repeat sales index. However, the price index in this research was generated solely for individual ZIP codes and did not extend to create a price index for the entire area.

Bae and Yu (2018b), using the random forests and the ANN models, both ML techniques, calculated individual apartment prices per point using the Gangnam-gu district in Seoul as their focus. They then computed the geometric mean of the increase rates in comparison to the reference point for each sample, resulting in the generation of a price index for each point. Here, data on transaction prices per point and per sample house in the past n months were used to construct a cross-section regression model, and the numbers of months were designated as 1 month, 3 months, 6 months, and 12 months, respectively, in conducting tests. According to the analysis results, volatility was larger when the learning data covered shorter periods. Conversely, when the period was longer, volatility decreased, resulting in higher stability. However, it became more challenging to reflect the latest trends, and the index tended to exhibit smoothing. In addition, when the house price trend index announced by the Korea Real Estate Board (REB), the REB’s transaction-based price index, and the ML-based index were compared, they found that the ML-based index exhibited greater volatility than the existing indices. Consequently, it was pointed out that there was a need to supplement sample prices with qualitative analysis conducted by researchers.

Kim et al. (2022) constructed a model explaining apartment prices in 25 autonomous districts in Seoul through ANN and random forests. Through their analyses, they found error rates to be the lowest for the random forest model and produced a price index based on the trained random forest model. This research performed modeling through the method of entering region and time dummies along with apartment characteristic variables in the model. A price index was then produced by adjusting the values of the region and time dummies while fixing characteristic variables using average values. The results showed that the index derived from this research, which performed modeling based on transaction prices, exhibited a greater range of fluctuations compared to the existing KB and REB house price indices.

3. Chapter Summary

Previous research on house price indices can be largely classified into transaction-based indices, which perform statistical modeling based on transaction samples, and appraisal-based indices, in which price indices are generated in a manner similar to composite stock market indices, assuming the existence of a directly appraised price flow for each point and individual house. However, each type of indices has its own strengths and weaknesses. Transaction-based indices can provide a more accurate representation of the market situation, but their effectiveness relies on the number of transaction samples or house transactions, which affects the statistical significance of index values. Consequently, it is difficult to produce stable indices for recessions when transactions drop and for small areas, thus leading to the generation of index flow noise that is not related to actual price fluctuations. In contrast, appraisal-based indices are less sensitive to the number of transactions but require humans’ direct appraisals. This makes it difficult to produce such an index for all houses, introduces smoothing effects, and can give rise to the problem of conflict of interests.

To overcome the limitations of previous research, this study aims to develop a house price valuation model using ML and, subsequently, construct a price index. By doing so, this study intends to address issues that arise from human involvement in price appraisals and resolve the problem of instability, which is a drawback of transaction-based indices. In recent times, there has been increasing research focusing on building more sophisticated house price indices using ML techniques compared to existing methodologies. However, existing research has focused on calculating transaction prices per point using cross-section models that do not incorporate serial information (Barr et al., 2017; Bae and Yu, 2018b). Alternatively, some studies have included serial information in the model such as time dummy variables but have primarily emphasized the overall model's predictive accuracy rather than exploring how relevant variables explain house prices in a serial manner (Kim et al., 2022). Because a price index represents the serial flow of prices, constructing a model that explains the fundamental prices of the index requires an examination of both longitudinal modeling and its impact. Consequently, this study seeks to construct an estimation model through apartment price explanatory variables including time information, to review the model’s accuracy in predictions, and to show that the model can model the serial dimension of house prices. Through this analysis, this study will discuss the distinctive characteristics of the proposed price index.

1. Artificial Neural Networks (ANNs)

ANNs are a type of ML technique with various structures designed to suit different research objectives such as recurrent neural networks (RNNs) and convolutional neural networks (CNNs). This study uses the basic multilayer perceptron structure. The model consists of the input layer, hidden layer, and output layer. Data entered in the network are multiplied by the weights of each layer and then transmitted to the next layer through the non-linear activation function. The model calculates losses by comparing the produced values in the final output layer with transaction prices. The model’s weights are updated in a way that minimizes total losses during training, allowing the model to abstract and learn non-linear relationships among the variables. Here, a model with two or more hidden layers is separately classified as a deep neural network (DNN), and the process of training such a DNN is referred to as "deep learning." Since the 2010s, advancements in computing power and model learning techniques have made deep learning possible and led to its widespread adoption. Deep learning has been demonstrated to outperform traditional linear regression models in terms of prediction performance (Chollet, 2017; and Loo, 2019).

Two main drawbacks of ML techniques that have been pointed out are that they are black box models and can be easily overfitted. Unlike traditional statistical models where relationships among variables can be expressed and interpreted as regression coefficients, neural networks in black box models consist of numerous weights so that it is difficult to express relationships among variables with a single slope value. Thus, this study aims to demonstrate the model's ability to capture the temporal aspect of prices by analyzing the patterns of model prediction errors. Overfitting refers to a phenomenon where the model is overly optimized to the training data, leading to a decline in its ability to make accurate predictions on new, unseen data. To prevent overfitting, a portion of a training data set (henceforth referred to as the “training set”) is separated as a validation data set (henceforth referred to as the “validation set”). Prediction errors are then evaluated using both the training set and the validation set during the model's learning process. The weights are adjusted until errors in the validation set are minimized, establishing the model's final configuration. Subsequently, if prediction errors on the test data set (henceforth referred to as the “test set”) are similar to validation set-based prediction errors, then overfitting is considered to be absent, and the model will be finalized.

2. Serial Autocorrelation Testing Methodology

To demonstrate that the model can effectively capture the temporal dimension of transaction prices, this study aims to compare the serial autocorrelation of transaction prices and prediction errors for individual houses in the test set. It can be simply expressed as follows:

Here, t is the time period, y_t is the transaction price, X_t is the explanatory variable including the transaction time, ϵ_t is the model prediction error, ρ is the first-order autocorrelation coefficient, and e_t is simple white noise that does not include autocorrelation. If ρ₀, which is the autocorrelation coefficient of the price time series, exhibits a significant positive value, but ρ₁, which is the autocorrelation coefficient of the prediction error time series, is either 0 or significantly smaller than ρ₀, then the model can be considered to explain effectively the autocorrelation of transaction prices.

However, with transaction information from multiple houses, the unique non-serial characteristics of individual houses can act as fixed effects, causing individual effects unrelated to the time series to persist in the model's prediction errors. This can be expressed as follows:

i represents an individual house, y_i,t is the transaction price of each house at each point, X_i,t is the explanatory variable fluctuating serially, Z_i is the fixed explanatory variable not changing serially, μ_i is the unexplained fixed effects of individual houses, and ϵ_i,t is individual and serial residuals. To test serial autocorrelation in such panel data, the first-difference method proposed by Wooldridge (2002) and Drukker (2003) can be applied. When the above expression is first-differentiated, it is as follows:

When the expression is differentiated in this manner, fixed effects are eliminated, leaving only serial explanatory variables and residuals. According to Wooldridge (2002), the correlation coefficient of the differentiated time series and first-order lag time series of residuals, denoted as Corr Corr(Δϵ_i,t, Δϵ_i,t-1), becomes -0.5 when ϵ_i,t does not have serial autocorrelation. Based on this, Drukker (2003) has demonstrated that it is possible to test the panel data’s serial autocorrelation by regressing the difference time series of model residuals on the first-order lags and examining whether the coefficient value is -0.5.

Consequently, this study aims to confirm whether the ML model has appropriately performed serial modeling by obtaining the first-order autocorrelation coefficient of transaction prices and prediction errors while simultaneously conducting tests based on Wooldridge’s and Drukker’s method to eliminate individual houses' fixed effects.

3. Price Index Methodology

In this study, the transaction prices of individual houses at each point are predicted through a model that has learned serial information on house prices, and the number of households is weighted to calculate market capitalization, based on which a price index is constructed. The logic behind the price index is an application of the method used for the S & P 500 index, which has been made publicly available by S & P Dow Jones Indices, LLC. The price index is based on the Laspeyres method but has been modified to ensure that any change to stocks constituting the index at individual points results in the revision of standard market capitalization. This modification prevents any disruption in the temporal continuity of the index. The specific expressions are as follows.

First, the basic formula of the Laspeyres index, where the index value at the reference point is established as 100, is expressed as follows:

I is the index, P is the price, Q is the number of households, i is an individual house, 0 is the reference period, and t is the point at which the index is calculated. While this expression assumes no change to the index’s constituent stocks, the total number of houses can vary due to factors such as new constructions or demolitions. This can disrupt the continuity of the index flow when calculating the simple market capitalization ratio. When the index is devised only based on houses that exist throughout the total time series to prevent such disruption, the prices of newly constructed and demolished houses are not reflected in the index. To address this issue, the basic Laspeyres index is modified as follows. First, market capitalization at every point is established as the numerator, and the divisor that can fluctuate at every point is established as the denominator:

D_t is the divisor at point t. Because Q_i,t is not fixed at the reference point in the modified index, market capitalization at every point in the area becomes the numerator. The divisor is adjusted each time there is a newly constructed house or a demolished house, thus allowing the index to continue without the index value being affected by changes to market capitalization caused by changes to the total amount. When the completion of the construction of a new house at point t is presumed, it is as follows:

Here, s represents the house newly constructed at point t and is separate from i, which is a set of houses that existed at point t-1 as well. If the divisor is not changed, fluctuations in market capitalization because of the presence of a newly constructed house can cause the index to rise considerably. Therefore, the divisor is adjusted so that constituent items up to point t-1, the index produced through the divisor at point t-1, and the index produced including newly incorporated items will be identical. When expressed as an expression, it is as follows:

D_t, the divisor adjusted at point t, is not actually used in calculations of the index at point t but is used from point t+1 onward. Therefore, if a new house is incorporated at point t, first, I_t, which is the index at point t, is calculated through the divisor and constituent items up to point t-1, disregarding the new house. Next, the divisor is adjusted by dividing market capitalization at point t, recalculated after incorporating the new house, by I_t. Subsequently, when there are no changes to houses’ constituent items from point t + 1 onward, the divisor newly adjusted at point t is used in this method.

In cases in which a house is excluded due to its demolition, the order of adjusting the divisor is inverted and correction is performed so that the index produced through houses remaining even after point t and the price index produced including houses excluded from point t onward will be identical. When a house demolished at point t is established as r, this is expressed as an expression as follows:

Consequently, when cases of the coexistence of newly constructed and demolished houses at each point are hypothesized and generalized, the two expressions can be combined and expressed as follows:

1. Data

The focus of object analysis in this study was limited to apartments because apartments are relatively standardized assets in South Korea, and transactions of identical apartments are more frequent than those of detached houses, making it easier to create the price index (Jeong, 2014; and Kim et al., 2015). The source of transaction prices for the model’s learning process was the apartment sales data provided by the Ministry of Land, Infrastructure and Transport (MOLIT) of South Korea. The data covered cases of transactions in Seoul from January 2006 to December 2022, and, after excluding canceled transactions, a total of 1,184,083 cases of transactions were used. Furthermore, to control characteristics specific to each apartment area, data from the apartment complex database provided by the South Korean prop-tech company Zigbang was used. Additionally, to calculate market capitalization, which forms the basis of the price index, information from Zigbang on the number of households per apartment complex area was utilized.

The model’s explanatory variables included the longitude and latitude coordinates of each apartment complex, the number of years since construction completion, the floor-area ratios, the total number of households per apartment complex, the apartment areas, the numbers of rooms, the numbers of bathrooms, and the transaction dates. The dependent variable was the sales price per area. Specifically, the longitude and latitude coordinates, the number of years since construction completion, the floor-area ratios, and the total number of households were determined for each apartment complex. The areas, the numbers of rooms, and the numbers of bathrooms were determined based on the area of each house in an apartment complex. Additionally, for the transaction date variable, January 1, 2006, when apartment transactions were first disclosed, was set as day 1, and this number was incremented by 1 for each subsequent day. Although this study’s price index was calculated every month, transaction date information was entered daily because, in the case of large apartment complexes with frequent transactions, price fluctuations can occur even at intervals shorter than a month. Furthermore, unlike earlier research based on the hedonic price model, variables such as administrative division dummies or season dummies were not included. Instead, spatiotemporal characteristics were explained through the geographic coordinate and transaction date variables. This decision was made due to the overfitting phenomenon that occurred when dummy variables were included in the model, exceeding the permissible level. Tables 1 and 2 provide a summary and descriptive statistics of the data sets, respectively.

Table 1.

Summary of dataset

Table 2.

Descriptive statistics

2. Model Configuration

This study utilized Python’s neural networks library Keras (ver. 2.4.0) and employed Scikit-Learn's StandardScaler function to standardize the input variables, thereby enhancing learning efficiency (Chollet, 2017). To find the optimal neural network model, it is necessary to explore various hyperparameter combinations including the model's structure, number of training epochs, and loss functions. As there is no predetermined rule, multiple combinations need to be tested repeatedly (Lenk et al., 1997). Through literature review and repeated learning and searches, this study obtained the optimal hyperparameter combinations, which are summarized in Table 3. To prevent the model from being overfitted to the training data, the data set was randomly divided into the training set, validation set, and test set. During the training process of a total of 300 epochs, the model at the point where errors in the validation set were minimized was selected as the final model. Additionally, the separated test set was used to confirm the error rates of the final model.

Table 3.

Summary of Neural Network Hyperparameters

1. Model Training

Figures 1 and 2 display the learning progress in the full model and the reduced model, respectively, without the transaction date variable. After 300 epochs of learning, the mean absolute percentage error (MAPE) value for the validation set, which was separated from the training set, converged to around 5% in the full model that included transaction dates as an input variable. However, in the case of the reduced model, the MAPE value for the validation set remained above 6%, failing to reach the same level as the full model.

Figure 1.

Training progress of full model

Figure 2.

Training progress of reduced model without trade date variable

Meanwhile, when the graph of learning progress in the full model in Figure 1 is examined, the training set-based MAPE value continuously dropped to around 4.2%, whereas the validation set-based MAPE value tended to converge at approximately 5.1%, failing to fall below this threshold. This indicates a potential overfitting of the model to the training set. Consequently, the 299th epoch, where the validation set-based MAPE value was minimized to 5.115%, was chosen as the point for establishing the final model. To assess the model's accuracy further, the test set, distinct from both the training set and the validation set, was utilized. The MAPE value from the test set amounted to 4.98%, lower than the validation set-based MAPE value, indicating that the general prediction performance of the model for new and unseen data was acceptable. Moreover, for the model in which information on transaction dates was not included, the test set-based MAPE value was also measured through the model at the epoch minimizing the validation set-based MAPE value, and the result was 6.49%. To determine the statistical significance of the difference between the two MAPE values, paired sample t-tests were conducted on the average difference between absolute percentage errors produced from the models. The t-value was 129.967, and the p-value was 0.000, confirming that the decrease in the model error rate through the usage of the transaction date variable was statistically significant.

2. Serial Autocorrelation Test

To prove the research hypothesis that if the model effectively captured the temporal dimension of apartment transaction prices, serial autocorrelation would decrease considerably in model prediction errors in comparison with the transaction price flow, serial autocorrelation of prices and prediction errors in the test set was confirmed. As the test set encompasses diverse apartments, housing units of the same areas in individual apartment complexes were considered identical houses, and each transaction in the test set was matched with immediately preceding transactions of the same house.²⁾ The total number of test set transactions amounted to 236,817 cases. However, the first-point transactions of individual houses were excluded from the analyses due to the unavailability of immediately preceding transaction prices. Consequently, matched transaction data on 213,799 cases were analyzed. The regression results for the price per area and prediction errors of each sale in terms of the immediately preceding cases are presented in Table 4. According to the results of the analyses, for price per area time series, the first-order autocorrelation coefficient and the R-squared value were high, amounting to 1.0222 and 0.956, respectively. This indicates a strong linear correlation between transaction price time series and the immediately preceding prices of identical houses. However, in the case of the model’s prediction error time series, the first-order autocorrelation coefficient was 0.1547, and the R-squared value was 0.023, respectively. If temporal modeling were not performed at all, model prediction errors would exhibit serial correlation with immediately preceding transactions, as with transaction price time series. In such a case, it would be possible to obtain both a high R-squared value and an autocorrelation coefficient approximating 1.0. However, the results show that, through the model, the temporal dimension of transaction prices was effectively modeled. In addition, the non-zero regression coefficient value of prediction errors further confirmed that the model significantly explained the temporal dimension but could not completely explain it.

Table 4.

Regression result: test-set price per area and prediction error to their time lags

Meanwhile, because the test set in this study includes diverse apartments, individual apartments’ fixed effects may have influenced the results. To verify serial autocorrelation after eliminating fixed effects from these panel data, the Wooldridge-Drukker test method was applied. For both transaction price and prediction error time series, differences from immediately preceding transactions of the same houses were calculated to remove fixed effects, and relevant difference time series were regressed to immediately preceding transactions. The results are recorded in Table 5. According to Wooldridge (2002), the regression coefficient approaches 0 when strong serial autocorrelation exists, indicating a random walk pattern. In contrast, the coefficient approaches -0.5 when there is no autocorrelation, representing white noise time series. According to the results, the regression coefficient for price per area approximated 0, indicating the presence of strong autocorrelation. For the prediction error time series, the regression coefficient was -0.4136. While this explained serial correlation in terms of the model considerably, the regression coefficient of prediction errors significantly fell short of -0.5, indicating that some degree of serial correlation remained. This result of the Wooldridge-Drukker method-based test is consistent with the analyses of the original time series.

Table 5.

Regression result: test-set price per area and prediction error to their time lags (1st-difference)

3. Evaluation of Potential Price Prediction

Figure 3 compares between “actual transaction prices” recorded in the training set and “predicted values” generated by the trained ANN model for an 84.9 m² Ricenz Apartment in Jamsil-dong, Songpa-gu, Seoul from 2011 to December 2022. The solid line represents the transaction price flow recorded in the training set, with the immediately preceding prices extended for points without transactions. The dotted line represents model prediction prices at every point, and the graph confirms that model prediction prices fluctuate even at points when transactions have not occurred. The round dots represent the transaction prices included in the test set. The data in the test set have not been used in training the model. Thus, this study considers them as “unobserved potential prices.”

Figure 3.

Model-predicted, actual, and potential price: Jamsil Ricenz Apartment 84.9 m2

To prove the hypothesis that temporal price modeling improved potential price predictions for points without transactions, this study compared the error rates of the transaction price flow per apartment complex area³⁾ in the training set with the predicted price flow produced by the model trained on potential prices. The analysis included 231,998 cases out of a total of 236,817 cases in the test set, excluding 4,819 cases that did not have the immediately preceding transaction prices of identical houses in the training set. According to the results, the difference between the potential prices of the test set and the actual transaction price flow had a MAPE value of 6.24%, while the difference between the same potential prices and the model's predicted prices had a MAPE value of 4.90%, resulting in an error rate reduction of approximately 1.34%p through ML modeling. This shows that, in comparison with the simple extension of immediately preceding transaction prices, estimated prices through modeling resulted in a decrease in the error rate by approximately 1.34%p. To confirm the statistical significance of this difference, a paired sample t-test was performed, yielding a t-value of 104.137 and a p-value of 0.000, indicating that the decrease in the error rate because of the model was statistically significant. Consequently, it can be confirmed that the flow of the model's estimated prices is closer to unobserved potential prices than the flow of simple real transactions, which are noisy and may not capture changes in value when there is no transaction.

4. Price Index Based on Model-Estimated Prices

Price trends for individual apartment complexes’ area types were estimated using the trained model. The estimated price per area was then multiplied by the area to calculate estimated prices for each point. Trends in apartments’ market capitalization, obtained by weighting estimated sales prices with the numbers of households in areas per apartment complex, were aggregated to produce the monthly price index. The reference point of monthly price estimations was the first of each month, and the prices for January 2006-December 2022 were estimated. When there were newly constructed and/or demolished houses, the divisor was modified to ensure index continuity, following the method presented in Chapter III. The total number of apartment complexes in Seoul included in the calculation was 7,463, and the number of area types amounted to a total of 46,024.

In Figure 4, the ANN model’s estimated price-based price index (henceforth referred to as the “model-estimated price index”), KB’s house price trend index, the REB’s house price trend index, and the REB’s transaction-based price index for all apartments in Seoul have been reconverted so that the figure for January 2017 would be 100 and are compared.

Figure 4.

House price index comparison (Seoul, apartment sales, 2017.01=100)

In the overall serial trends, the model-estimated price index exhibited trends most similar to those in the REB’s transaction-based price index. The REB’s house price trend index showed the lowest overall price volatility, and KB’s house price trend index exhibited trends in between. This was presumably because while KB’s and the REB’s house price trends reflected in the indices not only transactions but also bids regarding sample houses, the model-estimated price index and the REB’s transaction-based price index were based on identical data sets, only using transactions of the total houses.

In detail, through index movements during major recessions in 2008, 2018, and 2022, it can be confirmed that the REB’s transaction-based price index is more sensitive than the model-estimated price index. The model-estimated price index captured only fluctuations on a level similar to those of KB’s and the REB’s house price trend indices, thus confirming the existence of smoothing of the index. This tendency, in relation to analysis results showing the partial existence of autocorrelation with identical houses’ immediately preceding transaction prices in the model’s predictions of serial transaction prices earlier, is interpreted as being because of the existence of the conservative price appraisal tendency explained by Geltner’s (1991) partial adjustments model also in model-estimated prices.

Meanwhile, the model-estimated price index is produced by aggregating the price flow on the level of individual apartment areas. As for the flow of basic prices, there is a tendency for transactions to fluctuate more gently than such a flow, as illustrated in the example in Figure 3. Consequently, the model-estimated price index can be produced stably even for small areas with insufficient transaction samples such as towns, townships, and neighborhoods instead of a wide scope such as the entire Seoul. Figure 5 shows the production of the model-estimated price index for 14 legal neighborhoods in Gangnam-gu, Seoul as an example. The diagram confirms that, even for the scope of small areas, it is possible to suppress noise and create a smooth price index flow.

Figure 5.

Model prediction-based index: Dong-level, Gangnam-gu, Seoul (2017.01=100)4)