Land | Free Full-Text | Enhancing Urban Landscape Design: A GAN-Based Approach for Rapid Color Rendering of Park Sketches
1. Introduction
The construction of urban living environments is a crucial aspect of realizing urban ecological civilization. It directly impacts residents’ quality of life and the sustainable development of cities. Park green spaces, as essential elements for enhancing the ecological environment and city image, are among the city’s most vital resources. In the planning process of urban green spaces, designers integrate various design concepts, such as site space layout, landscape nodes, and road traffic, into hand-drawn plans. These plans directly express the designer’s thoughts, and the rendered colorful plans illustrate the design effect. This process aids communication with multiple parties and enhances the design’s quality and level. However, the current workflow from the research stage to the color flat drawing stage is often time-consuming. Therefore, improving the efficiency of plan rendering is a significant challenge in the current design process and the primary motivation and goal of this study.
In recent years, generating images with appropriate colors and textures based on sketches has become a research hotspot. The advancement of machine learning has greatly facilitated designers. With the evolution of artificial intelligence, many painting tasks are now accomplished using deep-learning technology. Generative adversarial networks, an innovative architecture, have found widespread application. However, landscape design drawings often struggle to render satisfactory results due to the irregularity of elements, intricate details, and limited available corresponding data. Currently, designers still need to manually complete it through software such as Photoshop, relying on their perception of color matching and the coordination of different detail textures. If deep learning can assist designers in quickly completing the design, it will significantly improve design efficiency and generate more application value. However, the drawing of landscape design diagrams makes it difficult to render better results due to defects such as the weak regularity of elements, complexity of details, and less corresponding data available. Currently, designers are still required to complete the design manually through Photoshop and other software based on the perception of color matching and the coordination of different details and textures. If deep learning can assist designers to quickly complete the design, the design efficiency will be greatly improved and generate more application value.
To address these issues, this study proposes an automatic plan rendering system based on generative adversarial networks, combining the features of machine learning and landscape planning and design. We use hand-drawn line-draft plans and color plan design drawings for training. Compared to the edge extraction commonly used in other studies, hand-drawn line-draft maps are more likely to reflect the actual semantic structure of images, conform to the aesthetics and habits of designers, and facilitate us in screening out the generation algorithm most suitable for the actual design process. After selecting the appropriate algorithm, we built a data enhancement module to optimize the rendering system. The main contributions of this paper are as follows: (1) Based on Pix2pix and CycleGAN, we built a fully automated park plan rendering system and screened out algorithms suitable for the design process. (2) By employing data augmentation models, we were able to expand our small sample size and optimize the training process. (3) We analyzed the differences and similarities in the results generated by various algorithms and sample data volumes. (4) We utilized a novel approach to visualize urban park green spaces, using hand-drawn line drafts as experimental data. (5) Our proposed automated rendering system enhanced the speed and quality of sketch rendering in landscape design. Integrating this system into the design development and drawing production stages of traditional landscape design processes can improve the overall design efficiency.
2. Related Work
Our work is primarily related to two research areas: image coloring and conditional generative adversarial networks. This section provides a comprehensive review of the pertinent literature in these fields.
2.1. Image Coloring
Existing methods primarily focus on the coloring of grayscale painting images or line drawings, with less emphasis on the rendering of design drawings. In the landscape architecture domain, rendering design sketches involves transforming hand-drawn sketches into images with color, texture, and shadow effects to enhance the expressiveness and appeal of the design. This differs significantly from standard line-drawing coloring, as the lines and colors in design drawings carry landscape element information. These elements not only reflect the designer’s creativity and thought process but also the functionality and form of the design. Therefore, rendering landscape design sketches requires a comprehensive consideration of the design’s semantics, style, and rules, rather than merely coloring the lines.
2.2. Conditional Generative Adversarial Networks
2.2.1. GAN Generation
2.2.2. Pix2pix and CycleGAN
3. Methodology
3.1. Analytical Framework
During the dataset construction phase, the quality and quantity of the dataset significantly influence the design learning of the algorithm. Given the challenge of obtaining paired park plan layout and design scheme data, we enhanced the black-and-white line-sketch data using CycleGAN. This enhancement expanded the original 652 pairs of park data to a dataset of 1699 pairs, thereby constructing a more diverse and accurate database for subsequent generation experiments.
In the model construction phase, we employed two neural networks, Pix2pix and CycleGAN, for image generation. The corresponding sketch and design scheme datasets were simultaneously input into the neural network for initial training of the small-sample data rendering system. However, the output results were suboptimal, leading us to add a data enhancement module based on CycleGAN. Consequently, we constructed an optimized plan rendering system using the expanded large-sample data.
During the application phase, the black-and-white line sketch is input into the model. The algorithm then automatically generates a rendering design scheme of the corresponding style within seconds.
The following sections will provide a detailed discussion of each step.
3.2. Data Collection and Preparation
3.3. Sketch Rendering Model
Our approach employs Pix2pix and CycleGAN to establish an automated plan rendering system. These are prevalent techniques in style transfer-related research, yet they exhibit substantial differences in their learning methodologies.
3.3.1. Supervised Learning Based on Pix2pix: Working Principle and Process
3.3.2. Supervised Learning Working Principle and Process Based on CycleGAN
3.4. Training
During the pretraining stage, the Pix2pix and CycleGAN algorithms are employed to train the hand-drawn sketch data, each with an equal sample size. The data are processed separately according to the format required by the different algorithms, leading to the initial construction of two small-sample line-draft plan rendering generation systems.
In the data augmentation stage, this study intends to use the CycleGAN algorithm as the foundation. It acquires 348 pairs of line-draft sketches and design drawings from various channels to construct a data augmentation system. Once the data augmentation system is established, we input 1047 design drawings, which lack corresponding line sketches, into the system and obtain 1047 black-and-white line sketches. These algorithmically generated black-and-white line sketches closely resemble human hand-drawn levels in terms of light and dark relationships, line strokes, and other aspects.
Finally, our data are augmented from the original 652 pairs of black-and-white line sketches and design scheme data to 1699 pairs of data samples. By retraining CycleGAN with the augmented dataset, we obtain the optimized design sketch rendering generation system.
3.5. Testing
To compare the efficiency of the generation systems built using different algorithms and varying sizes of data samples, this study selected five black-and-white line drafts as the test set. In the testing section, selected samples were inputted into the variously trained stages of CycleGAN and Pix2pix models, yielding post-test images.
The experimental samples selected are primarily categorized into medium-scale (samples a, b, c) and small-scale park line drafts (samples d, e), each exhibiting distinct characteristics. Planting: samples c and e primarily consist of point-shaped trees, while sample a involves a large cloud tree. The remaining samples comprise a mixed planting of cloud trees and point-shaped trees. Roads and Paving: samples a, b, and c all possess clear main ring roads and branch road structures, exhibiting pronounced spatial opening and closing characteristics. Conversely, samples d and e combine large-area paving and roads. Other Layout Elements: all five samples include lawns of varying areas. Apart from sample d, the rest of the samples contain water bodies.
4. Results and Analysis
To evaluate the performance of our developed model, we grouped and compared the test results of five samples. We assessed these results from a landscape design perspective using the following criteria: (1) Whether the color at the edges of the line drawings is clear; (2) Whether details like pavements, roads, and nodes are complete; (3) Whether the colors of trees, lawns, etc., are reasonable and diverse; (4) Whether the overall style is aesthetically pleasing and diverse.
This section primarily analyzes the results of two sets of comparative experiments. The first is a comparison of the small-sample rendering systems of the Pix2pix model and the CycleGAN model. The second is a comparison of the rendering systems of three different data volumes of the CycleGAN model.
4.1. Algorithm Comparison Evaluation
In the small-sample test results of CycleGAN, it can be observed that this model can achieve different brightness and changes in the rendering processing of plants (such as samples a, b, c). Simultaneously, it has a certain distinction in the color of paving, water bodies, and lawns (such as samples b, d, e). However, it also has certain shortcomings: (1) It cannot clearly render the color of the water body (such as samples a, b, e). (2) Some trees appear inappropriately blue or purple (such as samples c and d). (3) The boundary color of some roads is blurred (such as samples a, b, c).
In contrast to CycleGAN, the rendering images generated by Pix2pix lack details and real textures, the colors are relatively uneven, and the possibility of blurring is much greater. These problems are because supervised learning needs to use a one-to-one dataset to let the computer learn the transformation logic in it. However, there is a certain gap between the hand-drawn line draft and the design drawing which cannot be completely aligned. Pix2pix makes it difficult to understand the connection between the two, and the results generated are of poor quality.
In the research of style transfer in the field of architectural design, related technologies, such as Pix2pix and CycleGAN, have become mainstream. However, few studies have compared and evaluated the results of the two algorithms in the field. Through the experiments and analysis of this study, the results of CycleGAN in line-draft rendering image generation are more stable and accurate than Pix2pix.
4.2. Data Volume Expansion Comparison Evaluation
Compared with the small-sample CycleGAN model mentioned earlier, the test results of the data expansion model have the following advantages: (1) It can better distinguish the color of road elements and their boundaries. (2) The expression of tree colors is more accurate, showing different brightness and saturation of green. However, it also has certain disadvantages: (1) Some cloud trees and point-shaped trees show mode collapse in rendering (such as samples a, d, e). (2) It is unable to render the color of water body elements accurately.
Moreover, our optimized system can more accurately and beautifully express rendering results for line drafts of different scales and has a certain style, mainly manifested as: (1) The rendering of the road structure is clear, primarily light gray. (2) The treatment of trees is shown as greens of different brightness, saturation, and transparency, and some lawns have a uniform gradient effect (such as samples a, c, e). (3) There is a certain distinction for the water body, and the rendering effect is significantly higher than the small sample, showing a blue–gray color (such as samples a and b). (4) The overall rendering effect shows a unique style of unified tone.
The main defects of this model are: (1) The clarity of the water body rendering color is insufficient. (2) The diversity of expressions for paving, nodes, etc., needs to be improved.
Following a comparative analysis, it has been observed that our generation results have significantly improved in several aspects after the data enhancement: (1) For identical plant elements, different brightness, saturation, and green textures can be generated in various positions rather than generating plants of different colors. (2) The interference of noise to the algorithm is relatively reduced, enabling it to accurately identify paving, water bodies, roads, etc., and distinguish color elements. (3) The rendering effect is more accurate and exhibits a certain style.
The experiment demonstrates that this study effectively expands the experimental samples using data enhancement methods, optimizes the model training effect, and generates more realistic and aesthetically pleasing park plan design drawings.
5. Discussion
This paper proposes a line-sketch rendering park design system based on Pix2pix and CycleGAN algorithms, which realizes the conversion from black-and-white flat line-sketch drawings to color texture rendering drawings. By utilizing a pretrained model to augment limited sample data, this approach overcomes the constraints of sparse and low-quality data in this field. It involves training an optimized model to enhance the accuracy and variety of generated images, thereby establishing an automated plan rendering system. Additionally, this research has advanced the evolution of human–computer collaborative design workflows within the landscape industry.
This study’s proposed model enhances the rendering efficiency and can generate images with rich color features. It is capable, to a certain extent, of accurately differentiating various elements in park designs, leading to the production of effective design drawings. Diverging from the traditional approach of coloring based on the image block function and adding details and texture, our model discerns coloring rules from a multitude of hand-drawing design schemes. It extracts the interplay of the texture, shape, color tone, and placement of landscape elements. Consequently, the model can flexibly render plans based on input parameters, enabling the swift generation of results.
Our research has made progress in rendering design drawings, but it has limitations. The overall rendering effect depends on the quality of the drawings; rough or blurry drawings can lead to distorted results. Our method has yet to adapt to different design styles and aesthetics. Specific limitations include: (1) It is mainly used for small- to medium-scale parks, and may not be accurate for larger landscape designs. (2) It is unable to finely control data postenhancement. Future research could improve in several areas: First, vectorizing hand-drawn sketches to improve the rendering quality and adapt to different design stages and styles. Second, targeting landscape elements in design drawings for specific generation and optimization to enhance the accuracy and intelligence. Third, enhancing model interactivity, integrating with the designer’s operational workflow for real-time rendering, quick design adjustments, and improved communication and experience.
While our results indicate that conditional generative adversarial networks are effective in rendering design sketches, this research has certain limitations: (1) The design schemes selected are primarily for small- and medium-sized parks, which may limit the model’s ability to generate local nodes in larger-scale park designs accurately. (2) The enhancement of the algorithm does not allow for detailed constraints of the data. In future work, we aim to explore ways to refine the data enhancement methods and improve the algorithm. Our goal is to achieve multiscale plan rendering, thereby enhancing the rendering efficiency and diversifying styles. (3) Our algorithm exhibits limitations in generating vegetation, particularly in the crucial aspect of color diversity. This limitation is not just a technical issue, but also reflects the algorithm’s inadequacy in understanding and reproducing the true richness and variety of colors in natural vegetation. Specifically, for vegetation colors other than green, our algorithm fails to capture the subtle nuances and diversity of plant colors found in nature. This issue touches on the profound challenge of bridging algorithmic design with the accurate representation of natural colors, necessitating a better balance between the development of algorithms and a deeper understanding of design theory.
6. Conclusions
This paper introduces a method leveraging generative adversarial networks (GANs) designed to rapidly produce rendered drawings from line-drawing plan designs. This method aids designers in swiftly conceptualizing design scenes. The research investigates the influence of the volume and quality of design data, as well as the impact of different algorithms on the generation results driven by algorithmic processes. Additionally, it involves the development of corresponding auxiliary design tools for evaluation.
This paper makes the following two breakthroughs:
Firstly, it addresses the critical challenge of data scarcity in this field. In response, the study introduces a data augmentation model based on generative adversarial networks (GANs) that effectively expands the range of small-sample hand-drawn line-sketch plan drawings.
Secondly, the prevalent color rendering techniques, which primarily depend on edge extraction, risk omitting crucial details in hand-drawn sketches. This loss is particularly evident in the design’s structure and texture, significantly hindering designers’ capacity to convey their fundamental concepts. To solve these problems, this study has developed a model for rendering directly against hand-drawn line drawings. The model aims to improve work efficiency and generate high-quality color-flat drawings that can be applied in industrial environments.
The experimental results demonstrate the main contributions of this study: (1) The development of a model based on Pix2pix and CycleGAN for rapidly generating diverse and rich design schemes from black-and-white hand-drawn sketches, significantly enhancing work efficiency. (2) The validation of the scheme’s rationality and accuracy through objective evaluation, confirming its applicability in actual design processes and fostering the integration of artificial intelligence technology with landscape design. (3) Empirical evidence from data volume comparison underscores the importance of data augmentation in improving the model quality, leading to the creation of an optimized model with expanded data. (4) The use of hand-drawn sketches as experimental data aligns more closely with the practical application in landscape design processes. This study stands out for its emphasis on practical applications, in contrast to earlier research that mainly followed a computer science research paradigm with a sole focus on image accuracy. It primarily evaluates the generated results from a designer’s viewpoint, highlighting its relevance to real-world scenarios. The results generated by the algorithm are expressed as low saturation in color saturation. However, in the actual design workflow, this artistic expression style is more conducive to the expression of designers’ thinking and is more suitable for use as actual effect drawings. In the field of landscape design, this study enhances the rendering speed and quality of designs, improving collaboration and communication between designers and clients. While hand-drawn line drafts are indeed the primary embodiment of a designer’s conceptualization in the tangible realm, the application of color plays a pivotal role in visual communication. For instance, in the realm of botanical landscaping, while line drafts delineate the hierarchical structure among plants, it is the application of color that critically conveys the harmony of the planting scheme. Similarly, in landscape design illustrations, line drafts establish the relational dynamics among design elements; however, the infusion of color, although not altering the design itself, significantly aids in the accentuation of design areas and focal points, thereby holding substantial value for both the designers and evaluators of multifaceted design proposals.
This research aims to enhance the rendering segment within the intelligent workflow of landscape design. The implementation of more efficient rendering technologies is expected to accelerate the iterative thinking process of designers and enable the vivid presentation of preliminary designs to stakeholders. By reducing the time spent on early design iterations, the overall speed and efficiency of the design process can be significantly improved.
This research initiates an inquiry into the intelligent rendering processes within landscape design. Addressing the challenges encountered in this study, future investigations will delve into specific areas, such as applying constrained color rendering to plant-specific projects and developing renderings in a variety of stylistic floor plans. These topics will direct our forthcoming research efforts, indicating a commitment to advancing the field of landscape design through the integration of innovative rendering techniques. To be more specific, in our future work, we plan to broaden the scope of our design schemes, incorporating the rendering of large-scale urban park line drawings. We will focus on controlling the details of pavements, buildings, and trees to ensure that the local colors in the output results are layered and the textures are richer. Additionally, the model will be fine-tuned to better adapt to various depths in plan sketch inputs, thereby enhancing its efficiency in providing inspiration for designers. We also intend to update related technical methods to achieve superior outcomes. Unlike previous research, this technological process is designed to integrate seamlessly with actual projects, assisting designers in rapidly developing ideas and providing timely feedback to users. This approach is likely to influence the operational modes and personnel composition of some companies. Many related factors arising from this new mode remain to be explored and discussed in future research.