3.1. Extraction of Fabric Colors

As an important carrier of information transmission, images enable people to perceive directly through vision. Color space, as a different representation of image color structure, is divided into hardware oriented and object oriented types based on the different objects. The foundation of color information research is color space, which quantifies people’s subjective perception of color into specific numerical values and provides a basis for color expression ^[13,14]. However, the choice of color space can affect the accuracy of image color reconstruction. The color space selected for image color reconstruction must have both independence and uniformity. Independence requires that the components of the color space do not affect each other, and changes in one component will not cause changes in other components ^[15,16]. Uniformity requires that the equal values of each component in the color space can produce approximately consistent visual changes. There are multiple ways to express color space, and existing color spaces include Red- Green-Blue (RGB), International Commission on Illumination Color Space (CIE-lab and Lab), and Hue-Saturation-Value (HSV), etc. ^[17]. Fig. 1 shows the ICRA for implementing graphic design research.

Fig. 1 shows the flowchart of the color reconstruction algorithm, which mainly includes FCE, color interaction selection, matching color generation, and multi-objective collaborative color transfer. For fabrics, conventional color extraction methods are not applicable. Most fabrics have complex organizational structures, yarn textures, and pattern patterns on their surfaces, which affect subsequent color extraction. To eliminate the influence of organizational structure and yarn texture structure in color extraction, an FCE method based on texture filtering and DPC is used. Firstly, clear fabric texture images are input, and then denoising and smoothing are performed using a bilateral filtering algorithm. In fabric image acquisition, image signals may be contaminated by noise due to environmental and optical sensing instrument limitations. Fabric images are preprocessed using bilateral filtering algorithms. Bilateral filtering, as a non-linear filtering technique, is similar to Gaussian filtering in that its middle pixel value is replaced by the weighted average of pixels within the window. Bilateral filtering is improved based on Gaussian filtering, taking into account not only the geometric differences of pixels but also the differences between each pixel and the center pixel value. For points with significant differences in pixel values compared to neutral points, smaller weights are assigned. The operation of Gaussian filtering is represented by formula (1).

In formula (1), $I_{gf}(p)$ refers to the Gaussian filtering operation. $S(p)$ stands for a matrix window centered on point $p$, with a size of $k \times k$. $x_p$ and $y_p$ refer to the horizontal and vertical coordinates of $p$, respectively. $x_q$ and $y_q$ stand for the horizontal and vertical coordinates of point $q$, respectively. $N_p$ stands for a standardized output value. $I_{gf}(p)$ refers to the pixel value output after $p$ Gaussian filtering. $\sigma_s$ refers to the Gaussian kernel function’s bandwidth constant in a spatial domain. The operation of bilateral filtering is represented by formula (2).

In formula (2), $I_{bf}(p)$ refers to the pixel value after $p$ filtering output. $W_p$ refers to the standardized output value. $\sigma_r$ refers to the Gaussian kernel function’s bandwidth constant in a color domain. The fabric image processed by a bilateral filter needs to be smoothed by a rolling guide filter to smooth the texture of the fabric organization. Rolling guided filters can ensure the accuracy of pattern edge structures while removing image textures. The process of rolling guided filtering is represented by formula (3).

In formula (3), $I_{bf}(p)$ refers to the pixel value output at point $p$ after Gaussian blur. Then, by using the joint bilateral filtering formula, the blurred contour edges are restored through iterative calculation in Fig. 2.

Fig. 2 shows the texture removal and edge restoration process of rolling guided filtering. After Gaussian blur, the edge information of the image is removed. After multiple iterations, the blurry edges gradually become clear. However, the lost edge information in the image cannot be restored, resulting in a lack of complete details in the final iteration result. To ensure the integrity of the fabric contour, a calculation method that can effectively distinguish the texture of the fabric organization from the edges of the pattern is used. The Sobel operator is used to calculate the pixel gradient of the preprocessed fabric image, represented by formulas (4) and (5) ^[18].

Fig. 2. Rolling guided filter texture removal and edge restoration process.

In formula (4), $F_x$ refers to the gradient value in the horizontal direction. $\otimes$ refers to the convolution operator. The vertical gradient is represented by formula (5).

In formula (5), $F_y$ refers to the gradient value in the vertical direction. The Sobel operator has two sets of $3 \times 3$ matrices. The horizontal and vertical templates are convolved with the image in a plane to obtain approximate brightness gradients for the horizontal and vertical directions. Then, the gradient sum of all pixels within the sliding window is calculated using formula (6).

In formula (6), $T_x(p)$ and $T_y(p)$ refer to the gradient values of $p$ in the $x$ and $y$ directions, respectively. $K_x$ and $K_y$ refer to standardized outputs. $T(p)$ refers to the gradient value of the region centered on point $p$. The larger the $T(p)$, the greater the likelihood that point $p$ is the contour edge point. $\theta_p$ refers to the direction of the gradient in the region at point $p$, with a value range of $[-90^\circ, 90^\circ]$. After clarifying the texture of the fabric, DPC is used for image color clustering. Formulas (7) to (10) are the steps for DPC.

In formula (7), $\rho_i$ refers to the estimated value of pixel $i$. $N$ refers to the total pixels. $d_{ij}$ refers to the color difference value between pixel $i$ and $j$. $\eta$ refers to the quantity of points where $d_c$ is greater than point $i$’s color difference value. $d_c$ represents the distance between the center point, which is the distance between each color feature point in the fabric sample and its cluster center. This parameter is used to determine the density of points and select appropriate clustering centers for clustering. This density estimation method is relatively rough. To obtain more accurate estimates, the Gaussian kernel density estimation method is used, represented by formula (8).

Then, the minimum color difference $\delta_i$ between points with higher density and pixel point $i$ is calculated using formula (9).

For the point having the highest density among all pixels, $\delta_i$ is the maximum color difference with other pixels, represented by formula (10).

In formula (10), $M$ represents the total number of neighboring points or feature points involved in the calculation. After the above calculations, a decision map is drawn and cluster centers are selected. Then, the distance from each point to each cluster center is calculated, and each point is classified as the closest cluster center to it. The FCE method proposed in this study is used for color extraction of silk fabrics. Fig. 3 shows the extraction effect.

Fig. 3. Silk fabric pattern edge extraction process.

Fig. 3 shows the edge extraction of silk fabric patterns. Fig. 3(a) shows the original image of silk fabric. Fig. 3(b) presents the extraction results of the Sobel operator. Fig. 3(c) shows the regional gradient information. Fig. 3(d) shows the fabric image after texture elimination. From this image, the original silk fabric image is transformed into a texture image with accurate edges.

3.2. Generation of Matching Color Tables

After FCE, a matching CB is generated. The generation of this table comprehensively considers the limitations of user interaction and visual differences, which is a combinatorial optimization problem. The generation of matching CB in image color transfer requires certain constraints to be met, therefore it has a high degree of randomness. In the generation of matching CB, the directed migration is solved using interactive color selection, such as assigning color values to a certain area of a graphic design. Color is selected based on the target sub-image of the target design image and the relevant areas of the template image. After selecting the template and target images’ colors, it is necessary to reconstruct these image colors based on the color structure relationship. Fig. 4 shows the color structure relationship.

Fig. 4. Color structure relation.

Fig. 4 is a schematic diagram of the color structure relationship. Visual differences also have a significant impact on the generation of matching CB. It is necessary to control the visual difference of the target image CB within a certain range and maintain the contrast matrix’s similarity to the maximum extent. In Fig. 4, L1 represents the unprocessed color chart, L2 represents controlling visual differences, and L3 represents the processed color chart. If this visual difference of CB is greater than its proportion difference, Z-score standardization is required, represented by formula (11).

In formula (11), $c^*$ refers to the standardized value. $c$ represents the original value associated with the standardization process, where the original value is used to calculate color differences during the standardization process. $\mu$ refers to the average value of all sample data. $\delta$ refers to the standard deviation of all sample data. After standardizing all sample data, the root Mean Square Error (MSE) $E_c$ is represented by formula (12).

In formula (12), $n$ refers to the quantity of samples. However, the overall effect of CB cannot be considered from the comparison of CB. The visual difference between these color points added to CB and CB’s overall mean can be comprehensively considered. The overall CB contrast is represented by formula (13).

In formula (13), $E_{c1}$ refers to the mean of CB and CB’s difference. $E_{c2}$ refers to color points’ contrast difference in CB. $E_{ca}$ refers to the overall CB contrast. The matching effect of CB is quantified using formula (14).

In formula (14), $E$ refers to the energy value that quantifies the matching effect of CB. Based on the target CB, continuous iterative optimization of CB selection is carried out. The background color in the design is relatively single, and the brightness needs to be adjusted to adapt to the overall effect. The brightness value adjustment function is represented by formula (15).

In formula (15), $C_{tl}$ and $C_{ml}$ refer to CB’s average brightness. $I_{tl}$ and $I_{ml}$ refer to the average brightness of the target and template images. $P_{tl}$ refers to the original brightness of the target image. $P_{ml}$ indicates the brightness value adjusted after color matching. Fig. 5 shows the brightness adjustment process.

Fig. 5. Brightness adaptive process.

Fig. 5 shows the brightness adaptive process. The target image can be adjusted based on the brightness of CB. After re-matching the colors, some information may be lost, which can be repaired by matching the missing pixels with boundary points.

4.1. Performance Analysis of Fabric Color Extraction Algorithm

The study has established selection criteria to ensure that the selected samples are representative. A total of 100 different types of fabrics are selected, and the samples are sourced from various types of fabrics, including silk, cotton, and synthetic fibers. Various fabric samples are collected from local textile markets and online fabric suppliers to ensure that these samples cover multiple colors, textures, and constructions. Samples with clear fabric texture and rich colors are selected from the preliminary collection. These samples have been evaluated by professional designers to ensure their diversity in color and texture. The selected fabric samples are classified by fabric type and color for subsequent experimental analysis. The FCE algorithm is conducted in the laboratory. The study selects Mean Pixel Accuracy (MPA) and Mean Intersection over Union (MIoU) as the test indicators. Table 1 shows the laboratory environment settings.

Fig. 6. Convergence result of DPC algorithm on data set.

Table 1 shows the laboratory environment settings. The study selects 100 fabric texture images and divides them into training and testing sets based on a 7:3 ratio. On the training and testing sets, the iteration of DPC is set to 100 in Fig. 6.

Fig. 6 presents DPC’s convergence results on the training and testing sets. As the iteration increased, the loss value of DPC gradually converged. In the first 40 iterations, DPC quickly converged to 0.95 on the training set and 0.87 on the test set. After 100 iterations, DPC gradually converged to 0.36 on the test set. The study compared DPC with other clustering algorithms. Fig. 7 shows the MPA results.

Fig. 7. MPA results of multiple algorithms.

Fig. 7 shows the MPA results of multiple algorithms. Mean Shift is a density-based non-parametric clustering algorithm that assumes that datasets of different cluster classes conform to different probability density distributions. The sample points will eventually converge at the maximum local density. Points that converge to the same local maximum are considered members of the same cluster class. Density-Based Spatial Clustering of Applications with Noise (DBSCAN) is a representative density-based clustering method ^[19]. This method defines a cluster as the minimal set of densely connected points, delineating areas through the application of a sufficiently high density threshold into clusters. Furthermore, it enables the discovery of clusters exhibiting disparate forms within a noisy spatial database. Fuzzy C-Means (FCM), as a clustering algorithm for fuzzy mathematical models, can handle irregular datasets and datasets of different sizes ^[20]. K-means is an iterative clustering analysis algorithm that can be used for color extraction and visualization of color data. DPC has a high MPA after multiple iterations, reaching 94.9%. The lower MPA values of Mean Shift and DBSCAN indicate that these two algorithms have poor pixel accuracy values. The above algorithms are tested using MIoU in Fig. 8.

Fig. 8. MIoU results of multiple algorithms.

Fig. 8(a) presents the MIoU of Mean Shift, DPC, and DBSCAN. Fig. 8(b) shows the MIoU of FCM and K-means. The MIoU values of the above algorithms increased with iteration. The MIoU value of Mean Shift gradually stabilized at 90.1% after 100 iterations. After 100 iterations of FCM and K-means, the MIoU values were higher, at 90.8% and 90.7%, respectively. The MIoU value of DPC was the highest, reaching 91.1%. In summary, DPC had strong advantages over other algorithms in fabric color and texture extraction, which better met the needs of color reconstruction.

4.2. Performance Analysis of Image Color Reconstruction Algorithm

The proposed image color reconstruction method was tested using Mean Absolute Error (MAE), MSE, Peak Signal-to-noise Ratio (PSNR), and SSIM as evaluation metrics. The proposed ICRA was compared with other similar algorithms using U-Net and Generative Adversarial Networks (GAN). The results are shown in Fig. 9.

Fig. 9. Comparison results of MSE and MASE for various algorithms.

Fig. 9 shows the MSE and MASE results of multiple algorithms. The Mean Shift Clustering algorithm is based on the gradient rise strategy of probability density function for clustering, which is intuitive, simple, and efficient. Mask Region-based Convolutional Neural Network (Mask R-CNN) is a deep learning algorithm used for instance segmentation and object detection. U-Net is a convolutional neural network used for image segmentation. GAN is widely used in image generation by learning the distribution of training data through generative and discriminative networks. The proposed algorithm had the lowest MAE and MSE, while Mask R-CNN had the highest MAE and MAE. The MSE and MAE of Mean Shift and U-Net were also higher, indicating that Mask R-CNN, Mean Shift, and U-Net had larger errors and poorer image reconstruction results. Fig. 10 shows the SSIM and PSNR of various algorithms.

Fig. 10. Results of comparison between SSIM and PSNR of various algorithms.

Fig. 10 (a) shows the SSIM results of multiple algorithms. With the increase of Gaussian noise, the SSIM of various algorithms increased. The SSIM of GAN was the lowest, indicating that the reconstructed image using GAN had lower similarity to the original image. The proposed image reconstruction method had a high SSIM, indicating a good reconstruction effect. Fig. 10 (b) presents the PSNR for multiple methods. This proposed algorithm had a higher PSNR compared to Mask R-CNN, indicating that the pixel quality of the reconstructed images by these two algorithms was better. The PSNR of GAN and Mean Shift was lower, indicating poor pixel quality of the reconstructed image. Table 2 presents the results of various algorithm metrics.

Table 2 presents the multiple algorithm indicators. This proposed algorithm had excellent performance with the lowest MSE of 8.915. However, other algorithms had an MSE higher than 20, with Mean Shift achieving an MSE of 46.787, indicating poor regression performance and poor performance in image reconstruction. For MAE, the Mask R-CNN reached the highest of 57.611, indicating that the algorithm had a significant reconstruction error in the image. The proposed algorithm had the lowest MAE, only 7.224, indicating a small error and the best reconstructed image effect. For PSNR and SSIM, Mask R-CNN had higher values compared to the proposed algorithm. This proposed algorithm had PSNR and SSIM values of 0.875 and 28.733, respectively, indicating a high similarity between real and reconstructed images, which effectively achieved image reconstruction tasks. To further validate the effectiveness of the proposed algorithm, tests were conducted on two different datasets. Dataset 1 is a silk fabric dataset, which contains images of 100 different styles of silk fabrics with rich color and texture features. Dataset 2 is a cotton fabric dataset, which contains 100 different types of cotton fabrics, with samples including various colors and patterns. The test results are shown in Table 3.

According to Table 3, the proposed method exhibits high performance indicators on different types of fabric datasets, indicating that the algorithm has good adaptability and generalization ability. The results of the silk fabric dataset are slightly better than those of the cotton fabric dataset. This may be due to the more complex texture and richer colors of silk fabrics, which provide a wider range of testing conditions for the performance of algorithms.