Current Issue

Urban planning refers to a series of processes for establishing a desirable future image of a city and implementing it (Ministry of Land, Infrastructure and Transport, 2024). Since urban planning secures rationality and fairness through expert opinions and deliberation, hearing opinions from related organizations, and reflecting residents’ opinions during the establishment process, citizen participation is an essential element of urban planning (Lee, 2009). However, urban planning in Korea is often limited to unidirectional transmission of planning information or limited opinion collection (Lee and Kim, 2012). In addition, traditional urban planning, centered on experts, has been criticized for having limitations in reflecting citizens’ demands or local characteristics. In addition, the urban planning resident participation system has a stiff and closed structure that does not sufficiently reflect social needs, and it also leads to social conflicts because it fails to consider the region and its residents (Lee et al., 2022). In this context, the need for establishing plans through citizen participation is being discussed, and the 2030 Seoul Master Plan, which emphasizes citizen participation in the early stage of urban planning, was established. However, since the citizens who participated in the process did not sufficiently understand the technical and legal aspects of urban planning, questions were raised regarding the quality of the opinions presented, and there was a problem that the opinions of a small group of participants were over-represented (Kim, 2014). In addition, problems occurred in the process of inducing citizen participation such as the complexity in analyzing opinions and reflecting them to policies, cost incurred, and temporal and spatial constraints. Therefore, it is necessary to find effective ways to address these problems and maximize the benefits of citizen participation.

To solve these problems, Large Language Models (LLMs), which have recently been attracting attention with the development of artificial intelligence (AI), can be used. Based on their natural language processing capabilities, LLMs can understand and generate various types of text, and through this, they can quickly process and analyze complex information. Since the introduction of ChatGPT, a conversational AI developed by OpenAI in 2022, LLMs have been utilized in various fields (Park et al., 2023). Recently developed LLMs can perform tasks at levels similar to human levels, so they can quickly perform a wide range of tasks such as translation, free Q&A, essay writing, and computer programming (Kasneci et al., 2023). In particular, LLM agents that have independence and perform tasks according to specific purposes can be created and utilized in various fields such as work and research. Recently, various studies have demonstrated that LLM agents can be developed and utilized to replace the role of actual humans in urban planning, traffic management, and voting preference verification (Fu et al., 2024; Yang et al., 2024; Zhang et al., 2024). Accordingly, various approaches utilizing AI LLMs are being proposed to promote citizen participation and improve efficiency in urban planning, and it is becoming possible to overcome existing limitations and collect and reflect the opinions of more citizens (Wang et al., 2023; Gao et al., 2024). The utilization of LLMs offers the potential to enhance the transparency and participation of urban planning, thus enabling the implementation of a democratic and open urban planning process through real-time data analysis and feedback. However, there are several limitations and problems in the introduction of this new technology. For example, there are biases that may occur in the process of collecting opinions using LLMs, data quality issues, and uncertainty about how various stakeholders including citizens will accept AI. Therefore, it is a critical research task to clearly analyze the possibilities and limitations of citizen-participatory urban planning using an LLM.

The present study has the following purposes. First, this study seeks methods of applying an LLM more effectively to derive opinions on urban planning from specific citizens living in specific areas using the AI GPT-4o model and prompt engineering. In particular, this study aims to develop prompts that can be reflected in urban planning by representing specific citizens in the research area. Second, various prompt composition methods are compared through prompt engineering techniques, and the impact of each composition method on the agent's discussion content is evaluated. Finally, the limitations that may arise in the opinion derivation process using an LLM are analyzed, and based on that, directions for how AI-based citizen-participatory urban planning can be improved are suggested.

Accordingly, the main research questions to be addressed in this study are as follows. First, “Can an LLM be used to generate results that represent the discussion process during citizen participation in urban planning?” Second, “Can an LLM actually represent residents of a specific area and derive the right conclusion based on appropriate knowledge?” and “How do differences in prompts through prompt engineering affect the LLM results, and how can this be applied to urban planning?” The goal of this study is to answer these questions and ultimately contribute to expanding citizen participation in the urban planning process through prompt engineering, alleviating blind spots in resident-participatory urban planning, and finding better urban planning methods.

Citizen participation in urban planning is one way citizens can actively participate in the urban planning process, express their opinions, and affect the decision-making process (Haklay et al., 2018). Furthermore, citizen participation can directly influence the government’s policy-making and execution processes and plays a role in monitoring and expanding support for administrative agencies (Lee et al., 2015). This means that citizen participation goes beyond the mere act of participation; it has the institutional significance that it directly influences policy-making processes and serves as an administrative watchdog. In particular, citizen participation increases the rationality and publicness of urban planning and reflects diverse opinions and values in the plan, thereby contributing to sustainable collaborative planning (Kim, 2010). In other words, citizen participation is significant in that it allows urban planning decision-makers to collect a wider range of opinions from diverse citizens and reflect them in policies to produce comprehensive and acceptable results (Shipley and Utz, 2012). In this way, citizen participation in the planning process can contribute to increasing the effectiveness and efficiency of public policies, and therefore requires a careful approach to the design and implementation of the process.

Today, citizen participation in urban planning is carried out in various forms, such as public hearings, surveys, and online platforms (Yang, 2012). The Seoul Metropolitan Government offers a website to publicize the urban planning process and continuously collect citizens’ opinions and operates a participation panel composed of some citizens to monitor and assess the implementation of urban planning. In particular, such citizen participation can be facilitated through visualization and analysis of spatiotemporal data. Specifically, spatial information technologies such as Public Participation Geographic Information Systems (PPGIS) can be utilized to induce direct participation of citizens in urban planning and demonstrate that citizens’ opinions can be reflected in the urban planning process (Koh, 2006; Haklay et al., 2018).

However, there are still some areas that require improvement in current urban planning. Citizen participation is often limited to a formal procedure to satisfy legal requirements, and it is difficult to exert real influence. Specifically, it has been pointed out that the results of citizen participation are limited to the opinion collection stage and that opinions are not sufficiently reflected in the decision-making stage (Lee et al., 2015). In the process of establishing the 2030 Master Plans for Cheongju, Suwon, and Seoul, the level of citizen participation was lower than the participation plan, and the participation of ordinary citizens was limited (Jeon and Lee, 2016). In particular, since citizen participation of the marginalized is further limited in this process, sufficient citizen participation of various classes needs to be ensured in future urban planning (Mahjabeen et al., 2009). Finally, issues related to the time and costs that are incurred in the process of citizen participation in urban planning have also been continuously raised. As seen in the case of Seoul City’s ‘living area plan,’ a lot of time and resources are required to effectively elicit citizen participation, which in turn leads to high costs; therefore, it is necessary to design the future citizen participation process in consideration of these issues (Yang, 2018).

The development of LLMs began with language models. Language models predict the position of words in a sentence by probabilistically calculating how natural a sentence is, and natural language processing and analysis using the models began to draw attention (Jones, 1994). However, traditional language models have had difficulties in identifying words with low frequency or first appearance as well as overfitting issues and challenges in capturing complex language (Rosenfeld, 2000; Brown, 2020). To overcome these problems, LLMs, which are advanced language models that apply AI-based structures, have emerged (Chang et al., 2024; Hadi et al., 2023). LLMs are effective in identifying the relevance and context of all words in a sentence by learning patterns and relationships between words, phrases, and sentences through large text data sets. Accordingly, LLMs have begun to be utilized in various fields while overcoming the limitations of conventional models used for natural language processing, and their usability has recently been attracting more attention with the emergence of generative artificial intelligence (GAI). Specifically, GAI is capable of analyzing and generating various forms of data, such as text, images, and voice, based on prompts directly entered by the user (Brühl, 2023).

In this context, ‘prompt engineering’ has emerged as a process for using GAI. This is a process of specifying prompts and optimizing the input prompts to generate the desired response in GAI including LLMs, and it plays an important role in deriving high-level results from GAI systems based on natural language processing and LLMs (Marvin et al., 2023; Fulford and Ng, 2023). It has been proven that the performance of GAI models can be improved through prompt engineering and more realistic and appropriate answers to various problems can be provided, contributing to high-quality question answering (Bsharat et al., 2023; Strobelt et al., 2022; Abukhalaf et al., 2023). Studies have suggested that prompt engineering can maximize the potential of LLMs and propose specific utilization methods specialized in various fields.

LLMs trained with reinforcement learning via human feedback can consistently generate human-like outputs (Kalla et al., 2023; Tessler et al., 2024). They also have flexible characteristics to perform various tasks, so they can be used for high-level tasks such as answering questions or making inferences based on the answers (Zhu et al., 2023). Accordingly, the use of LLMs has been increasing in recent urban planning and design studies.

A study using ChatGPT, which enables LLM-based chat, evaluated the quality of 50 urban plans being implemented in the United States through prompt engineering and compared the results with the evaluations actually conducted by humans. The results showed that the evaluation by ChatGPT was consistent with the traditional content analysis approach by 68% (Fu et al., 2023). Furthermore, a study was conducted that combined prompt engineering and geo-knowledge to recognize geographic location descriptions in social media data and utilize them for disaster response and control (Hu et al., 2023). The study demonstrated that this method outperformed the existing named entity recognition method by more than 40%. Furthermore, a spatio-temporal LLM (ST-LLM) called UrbanGPT was recently proposed by combining an LLM and spatio-temporal data (Li et al., 2024). The model exhibited generalized performance in various urban environments and showed high prediction accuracy even in areas that were not used for model training. In this way, LLMs have been applied and utilized in various fields of urban planning and facilitate not only the evaluation of existing urban plans but also spatial analysis and data structuring (Zhu et al., 2024; Quan and Lee, 2025). In summary, LLMs can provide higher efficiency and accuracy in planning and data collection in the field of urban planning. In particular, there are implications in that the use of LLMs can increase the consistency and objectivity of evaluation and reduce the time and cost of the data collection process compared to existing methods.

Recently, the use of LLM-based agents is drawing attention, because they are capable of providing an environment for real-time response and problem solving, especially in dynamic and complex urban environments (Ruan et al., 2023; Ni et al., 2024; Masterman et al., 2024). Specifically, LLM-based agents can be utilized to perform roles on behalf of real humans. A study conducted on a method for generating personal mobility data using an LLM-based agent demonstrated that agent-based generated data can reflect real spatiotemporal activity patterns at a high level compared to real mobility data (Wang et al., 2024). The agents can replace the discussion process by configuring a multi-agent through multiple combinations. In a study that built a multi-agent-based resident collaboration framework for Beijing, it was found that the land use plan generated through the framework received higher scores than plans prepared by actual human experts in terms of resident satisfaction, inclusiveness indicators, and service accessibility (Zhou et al., 2024).

The uniqueness of this study is that, unlike the existing methodologies for promoting citizen participation in the urban planning process, it focuses on prompt engineering using an LLM. In particular, this study proposes a more efficient and comprehensive citizen opinion collection method utilizing an LLM capable of digitizing and automating offline-based opinion collection methods such as surveys, public hearings, and interviews. Specifically, this study utilizes the GPT-4o model, which has proven excellent performance as the latest LLM. It is expected that the model can represent citizens with various characteristics so that it can reflect various citizen opinions regardless of time and place. In particular, LLMs such as GPT-4o can remember previous chats and generate answers based on previous content when a user asks a follow-up question (Barua, 2024). The use of an agent based on this will enable new content to be derived more effectively and meaningfully based on previous content.

In addition, while previous studies have used AI merely as a data analysis tool or only generated fragmentary content, this study is different in that it demonstrates that AI can be directly used for discussions representing citizens. This allows for the derivation of various perspectives and interests related to urban planning in a specific region and enables more in-depth analysis. Specifically, the discussion-type prompts presented by the GPT-4o model are expected to directly influence urban planning decision-making by helping local residents organize and express their thoughts more easily by representing their opinions, and furthermore, by systematically analyzing their opinions.

Lastly, this study is unique in the sense that it presents specific ways to utilize an LLM by analyzing how the interactions between agents within an LLM occur, through various prompt engineering practices. This will contribute to presenting a new methodology for sustainable urban planning by combining three important elements: AI, urban planning, and citizen participation.

The spatial scope of this study was set to the Wangsimni-Haengdang area in Seongdong-gu, Seoul, which is in the Northeast Zone 1 designated by the 2030 Seoul Living Area Plan. Seongdong-gu is a composite space where residential areas, commercial areas, and public facilities are densely located (Cho and Chi, 2020). In line with these spatial characteristics, the Seoul Metropolitan Government is implementing policies such as improving the residential environment, creating jobs and economic vitality, encouraging youth entrepreneurship, and consulting small business owners (Seoul Metropolitan Government, 2014; Seoul Metropolitan Government, 2015). In particular, various facilities within the Wangsimni-Haengdang living area are closely related to the daily living of residents; thus, the area was selected as the target area of the study. This study is based on a spatial unit to reflect the opinions of residents at the scale of the local living area rather than a higher-level urban plan on a broad scale. In this context, this study set the Wangsimni-Haengdang living area, a small-scale local living area, in the Northeast Zone, as the research scope based on the '2030 Seoul Living Area Plan.' The 2030 Seoul Living Area Plan is a follow-up plan to the Urban Master Plan, and it presents the direction of living area development and urban management concept by reflecting the characteristics of the living area and the opinions of residents. This plan is divided into zones and local living areas, and this study focuses on local living areas.

Next, GPT-Knowledge was built by utilizing local news crawling data and basic living area data to input information through prompt engineering and induce context-based responses. GPT-Knowledge is a pre-training input interface that helps to input structured and unstructured data into the GPT-4o model in a structured form, and it is useful for increasing the model's information accessibility and understandability (Yang et al., 2022). GPT-Knowledge was used to analyze the difference from the Zero-shot model in which responses are derived without prior information. Specifically, the local news data used in this study was from April 2023 to September 2024, after the lifting of the indoor and public transportation mask-wearing obligation due to COVID-19. This was set to exclude the impact of the epidemic and to consider the daily living standards of ordinary citizens. To collect the content, data crawling was conducted with 'Wangsimni-Haengdang Living Facility' as a keyword, and the data was organized in the order of date, news organization, headline, and short text under the title. Finally, out of the 881 total pieces of news data collected, 125 pieces of news crawling data were used after excluding redundant items. For the basic living area data, the content corresponding to the local resident participation workshop of the ‘2030 Seoul Living Area Plan’ and the content corresponding to the Wangsimni-Haengdang local living area were converted to text to be used in the study (Seoul Metropolitan Government, 2018). The news and basic living area data were processed into JSON files. The files include structured data that the LLM model can effectively understand and are composed in the form of key-value pairs (Bsharat et al., 2023). An example of the JSON file used in this study is attached in the <Appendix>.

This study was designed to analyze the effect of the components of the prompt on the quality of the results in the process of deriving LLM-based resident-participatory urban planning. First, this study conducted discussions using an AI agent, set a prompt representing the opinions of actual citizens as an agent, and designed the experiment by specifying three conditions for the agent:

• (1) Whether the age, status, and number of agents are considered
• (2) Whether specific facilities in the area are considered
• (3) Whether news data on the actual local living area is used in training and reflected

Through this process, this study analyzes the qualitative differences in the results of LLM-based urban planning and derives conditions under which the most effective and realistic urban planning is derived.

In this study, evaluation indicators were set up for evaluating the results generated through prompt engineering. Existing natural language processing models used consistency, conformity, relevance, and evidence as human evaluation items for evaluating the quality of generated text (Wang et al., 2020; Fabbri et al., 2021). Based on the previous reports, six evaluation indicators were selected in this study to evaluate how well the LLM-generated answers reflect the actual local characteristics, and these were set with a focus on how consistently they operate according to the agent and prompt conditions. Each item is scored out of 10 points, with points deducted based on specific indicators such as accuracy and logical consistency.

In this study, the prompt was specified at a total of 12 levels according to the prompt specification process presented in <Table 1> and then utilized. The prompt result evaluation was conducted based on the results of Lv10, Lv11, and Lv12, the highest specification levels, by varying only the number of agents in a situation where all available data such as news and living areas were included.

First, the summary of the Lv10 prompt results, obtained by giving specific conditions and setting the number of agents to 2, are shown below, and the full text is attached in the <Appendix>.

In ‘Statement 1,’ the college student agent in their 20s expressed frustration over the lack of study spaces and cultural and resting places for college students. In response, the landlord agent in their 60s in Statement 1 empathized with the college students’ sense of the problem and mentioned the need to solve the problem along with the main tasks he thought of. In addition, in subsequent speeches, the agents expressed their opinions based on the personality assigned to each agent for each topic and presented their opinion on the other agent’s argument. In particular, in ‘Statement 3', the landlord agent in their 60s generated thoughts about the college student agent’s argument in association of the landlord’s own problem, thereby deepening the discussion. In the first round, the agents generated their separate opinions on living service facilities. In the second round, the agents mentioned the negative effects of redevelopment and created a discussion on local development that embraces various classes in the area. In the third round, the agents generated the opinion that outsiders should also be taken into consideration, considering that Wangsimni is a transportation hub. Therefore, the results showed that the agents asserted needs based on the positions given to them and developed the discussions related to living service facilities in consideration of not only the residents in the local community but also outsiders visiting the living area.

Next, in the Lv11 prompt in which the number of agents was increased to five, the interactions were relatively few. A summary of the Lv11 prompt results is show below, and the full text is attached in the <Appendix>.

In the three rounds of discussion, there was a tendency to generate local issues corresponding to each agent's background. In ‘Statement 1', the college student agent in their 20s mentioned learning spaces and youth-friendly facilities, and in the following statements, the agent was arguing only about the youth community centers and startup support centers according to the agent’s own needs, indicating that there were no significant interactions with other agents. Regarding the discussion content, there was an advantage that the opinions of various residents were heard. The five agents set in the Lv11 prompt argued about youth, education, infrastructure, commerce, and green space and generated discussion results based on local characteristics and facilities existing within the research scope. The college student agent in their 20s, who emphasized study spaces and youth-friendliness, mentioned cultural spaces, startup support centers, and AR/VR-based learning facilities, and the office worker agent in their 30s mentioned local characteristics such as those of Wangsimni Station and emphasized the harmony of urban infrastructure. These results suggest that universal contents were generated based on the pre-training of the GPT-4o model.

Finally, the results of the Lv12 prompt in which the number of discussion agents was set to 10 are described below.

In Lv12, unlike Lv10 and Lv11, the agent's speech was performed only once, and independent results were generated for each agent without interaction. This may be because the amount that can be generated at one session is limited during the prompt generation process. However, the age and occupation characteristics of individual agents were well reflected like Lv10 and Lv11, and the types of necessary facilities were generated differently for individual agents.

In summary, the evaluation results showed that the content and contextual quality of the results generated by the AI increased depending on the degree of specificity of the prompt. This indicates that the agent's characteristics may be well reflected according to the set-up conditions and suggests that it is possible to represent a specific class of citizens through prompt engineering. However, as the number of set agents increased, the interactivity was lowered. To solve this problem, it may be necessary to subdivide the prompt engineering process or employ an LLM that has been trained with the discussion process.

In this study, to evaluate the results according to the level of specificity of the prompt, six indicators were set, as shown in <Table 3>: ‘logical consistency,’ ‘accuracy,’ ‘local context reflection,’ ‘persona reflection,’ ‘prompt reflection,’ and ‘interactivity.’

To compare the prompt results from Lv1 to Lv12 based on these six indicators, 20 evaluators residing in Seongdong-gu and majoring in urban planning were selected for evaluation. The 20 evaluators consist of undergraduate, master's, and doctoral students majoring in urban planning. The evaluation was carried out by evaluating the results of the 12 combinations of ChatGPT4o prompt scenarios using the six indicators. The prompts and the full text results from Lv1 to Lv12 were provided to individual evaluators, who were requested to provide subjective evaluations for the six indicators on a Likert scale score ranging from 1 to 10 (Van der Lee et al., 2021). Specifically, the evaluators were asked to provide scores starting from the basic 10 points and deducting points according to conditions, and the average response scores were calculated for each prompt and indicator. The evaluation results for each indicator are shown in <Table 4> and <Figure 2>, and the specific details for each indicator are shown below.

Figure 2. 
Summary of expert ratings on prompt-generated results

This study analyzed the possibilities and limitations of citizen participation in urban planning using the GPT-4o model and prompt engineering. Specifically, through prompt content specification, a discussion process representing citizen participation was presented under various conditions, and the generated results were evaluated with various indicators. Regarding prompt content specification, the accuracy, local context reflection, and logical consistency tended to increase as discussion conditions, age and occupation of agents, and local context data were added step by step. This suggests that such specified prompts can generate good results by inducing participants to recognize the realistic problems of the area more accurately and to logically present feasible improvement measures. On the contrary, it was confirmed that as the number of agents increases, the interactivity between the personas decreased, and problems such as decreased local specificity in the generated results occurred. Through this, it was demonstrated that a small number of agents can be more effectively utilized to present more specific results when a single model is used. The major implications of the study are described below.

First, an LLM-based urban planning process may be established to set up virtual personas reflecting various backgrounds and occupations in order to collect broader and more in-depth opinions. Through this, substantial citizen participation may be expected. Due to the limitations of the existing method, it is difficult to reflect the background and environment of individual citizens in an actual resident participation workshop. However, the LLM-based agent employed in this study can effectively derive various perspectives and interests by setting specific conditions. This methodology supports more comprehensive decision-making in the urban planning process and suggests the possibility of reflecting the voices of various citizens. It is also suggested that the new methodology can replace or supplement citizen participation through prompt engineering in future urban planning processes.

Second, an LLM may be utilized to provide real-time data analysis and feedback systems and propose a dynamic and adaptive urban planning process. In this study, social requirements were considered by reflecting local issues and residents’ demands to the agents by utilizing news data and local living area data. In other words, an LLM enables flexible plans to be prepared that adapt to the changing urban environment by reflecting real-time issues and providing immediate feedback using various data including text. When such additional information is considered, more detailed results may be presented for target areas of urban planning. Therefore, various data may be actively utilized in prompt engineering to establish a more precise and complex urban planning process.

Third, the LLM-based urban planning approach can contribute to cost saving and efficiency improvement. While the existing citizen participation process is time-consuming and costly, LLMs can automatically collect and analyze citizens’ opinions, thereby making decision support more efficient. This approach simplifies the overall urban planning process and allows for the reflection of diverse citizen opinions in an economical way.

In summary, this study suggests that citizen participation-type urban planning using an LLM can reflect diverse citizens’ voices and promote sustainable urban development and that LLM can be used as an important tool in urban planning.

The present study has the following limitations.

First, although the local context reflection slightly differed depending on the prompt, universal results that can be applied to other regions were presented in general. In the case of LLM models, it is difficult to specifically confirm the source of the presented answers due to the model limitations. Therefore, additional prompt engineering processes such as inputting the type of data to be used may be needed in the future to accurately reflect local characteristics. In addition, the problems and improvement plans presented in the discussion results were generally similar. This may be considered as the consistency of the GPT-4o model, but it may be the result of training the model by repeatedly entering similar prompts. Further verification of the model’s consistency can be carried out through future experimentation.

Second, the discussion process was not sufficiently considered in this study, and the discussion process specified in this study insufficiently represented the discussion process during actual resident participation workshops attended by about 10 participants for each administrative dong. Specifically, the prompt results when the number of agents was set to five or ten tended to only present their own opinions without showing empathy or opposition to each other’s arguments, despite the fact that the discussion method was specifically presented. These results reveal the limitations of creating multiple agents from a single model. Furthermore, it was confirmed that the more agents, the lower the quality of the generated results in terms of various result indicators. To solve this problem, further research may need to be conducted such as using multiple model agents or presenting different discussion methods through a single model.

Finally, this study attempted to precisely identify the detailed differences between the evaluation items using a 10-point Likert scale. Since the rating was performed in this study by deducting points according to the degree of compliance with the evaluation items based on a maximum of 10 points, the direction of interpretation may differ depending on the respondent's evaluation criteria, and there may be differences in the interpretation when compared with existing studies. In addition, although the agents were defined comprehensively so that the application subjects of the study were broad, due to the limitations of the preliminary study, the results were compared and analyzed through expert evaluation rather than a survey conducted with similar subjects, which may limit the generalization of the results and suggest the need to consider the similarity of human evaluators in future studies.

In future studies, the limitations of the present study may be overcome, and a more specific discussion process may be provided by specifying the input data and further reviewing the prompt engineering process. In addition, higher performance may be achieved through fine-tuning or retrieval-augmented generation (RAG). Fine-tuning is a method of performing training by adding a dataset to an existing LLM to ensure task suitability in a specific field, and RAG is a methodology for improving the ability to identify information relationships among relevant facts by connecting an external database for a specific field. Through this, prompts may be further specified to improve the quality of LLM-based discussion results, reduce universality of generated results, and prevent hallucination that generates incorrect information.