A hybrid discrete state transition algorithm for combinatorial optimization problems

The discrete state transition algorithm (DSTA) has been wildly applied to deal with combinatorial optimization problems. However, its low convergence accuracy limits its application in large-scale optimization problems. Aiming at the convergence performance and search intensity of the algorithm, a hybrid discrete state transition algorithm (HDSTA) is proposed in this work by introducing tabu search and elite solution set. Firstly, a searching mechanism with the integration of DSTA and tabu search (TS) is established, which allows moving to adjacent solutions at an increased cost to escape from the local optimum. Specifically, a tabu list as adaptive memory is adopted to avoid the loop when deviating from local optima. Secondly, an elite solution set is introduced to integrate the information of the previous optimal solution and the global optimal solution, and the search strategy is modified to expand the range and diversity of candidate solutions. Finally, the proposed HDSTA is verified according to the real data on two well-known optimization problems (staff assignment problem and traveling salesman problem) and the real data of an industrial case. The experimental results show the effectiveness of the proposed algorithm in large-scale optimization problems.


Introduction
Combinatorial optimization problems are encountered in almost all branches of engineering, including system control, process management, parameter setting, etc. Generally speaking, the combinatorial optimization problems are NP-hard. When the problem scale is small, accurate algorithms are adopted to obtain exact solutions step by step, for example, branch-and-cut approach (Lin and Tian, 2021) and Hooke-Jeeves pattern search (Tabassum et al., 2021). However, these methods mainly take exponential time to run. As the scale of the problem increases, more and more attention has been paid to stochastic algorithms which can obtain approximate solutions within the acceptable computation time. The existing stochastic methods mainly include evolutionary algorithms and metaheuristic algorithms. The evolutionary methods consist of evolution strategies (Hassanzadeh et al., 2021), genetic programming (Zhang et al., 2019a) and genetic algorithm (Wang and Sobey, 2020), while metaheuristic methods contain particle swarm optimization, simulated annealing, differential evolution algorithm (Abdel-Basset et al., 2021;Houssein et al., 2021;Wang et al., 2021), etc.
The state transition algorithm (STA) is originally proposed by Zhou et al., based on structuralist learning (Zhou et al., 2012). The concept of the STA is to treat the solution as a state and the process of updating as a state transition, through which a new state will be created. It generates candidate solutions by the cyclic utilization of four special operators. The greedy criterion is used to update the current optimal solution until the termination condition of the iteration is met. For PID control problem, (Zhang et al., 2018;Zhang et al., 2019b) fuzzy fractional order PID controller is adjusted by the state transition algorithm to achieve the optimal control of zinc powder addition. The traditional state transition algorithm is an unconstrained optimization method and applicable to continuous optimization problems. Therefore, a discrete state transfer algorithm (DSTA) is proposed to solve the discrete optimization problems . The existing results show that DSTA is an effective and promising algorithm for combinatorial optimization problems. However, when the problem scale is large, the DSTA is excellent in convergence rate but poor in convergence accuracy, and it is easy to get trapped into local optima. Dong et al. (2016) proposed the concept of quadratic state transition to solve the assignment problems, which expanded the scope of candidate solutions and increased the diversity of candidate solutions. In (Wang et al., 2016), a quantum state transition algorithm is used to solve the job shop scheduling problems. It combines the shift decoding and position exchange coding to map the solution space, and the non-local optimal solution tolerance mechanism is proposed to improve the convergence accuracy. Unfortunately, in the above studies, the generation of the next population is only related to the optimal solution of the previous generation, which shows slow convergence rate and low convergence accuracy in the later search stage.
In this paper, a hybridization search framework which combines DSTA and tabu search (TS) is proposed to solve the combinatorial optimization problems. The DSTA is used to be the global search scheme, and the TS is introduced to act as a local search scheme. The primary contributions of this article can be summarized as follows: (1) A hybrid discrete state transition algorithm is proposed by combining the tabu search process with discrete state transition algorithm. A search framework combining DSTA and tabu search is proposed. The tabu list is introduced as a short-term memory and can be updated based on the recently accessed solutions. (2) A new approach for the population in DSTA is developed to produce feasible solutions, called elite solution set, which integrates the information of previous optimal solution and global optimal solution to speed up the search process. (3) The effectiveness of HDSTA is verified on various combinatorial optimization problems, including the staff assignment problems, traveling salesman problems, and multi-AGV task assignment problems based on real-world industrial data.
The rest of the paper is organized as follows. Section 2 presents the principle of the discrete state transition algorithm. The hybrid discrete state transition algorithm is explained in Section 3. In Section 4, the results of various experiments are presented to evaluate the performance of the proposed method. Finally, the paper is summarized in Section 5.

The discrete state transition algorithm
In order to introduce the main principle of the discrete state transition algorithm, the following discrete optimization problems are considered: To facilitate solving the discrete optimization problems and accelerate the search rate, the subscript representation is usually used to encode the candidate solutions. The values of the n-dimensional decision variable K κ 1 , κ 2 , /, k m { }correspond to the numbers 1, 2, /, m { } . A three-dimensional example of the subscript representation is shown in Figure 1, and the different values of x correspond with six encoding methods. The subscript representation is expandable, which can make corresponding changes according to the specific form of the discrete optimization problems. Moreover, the subscript representation can also reduce the difficulty of solving the partial discrete optimization problems.
The DSTA designs four special geometric operators shown in Figure 2.  (1) Swap transformation operator where, A swap k ∈ R n×n is a random Boolean matrix. It has the ability to exchange m a positions in x k , and m a is the swap factor with the function of controlling the maximum number of exchangeable positions.
(2) Shift transformation operator where, A shif t k ∈ R n×n is called a shift transformation matrix. m b is the shift factor to control the number of positions to be shifted. The shift transformation operator can move m b positions behind another specified positions.
(3) Symmetry transformation operator where, A sym k ∈ R n×n is called a symmetric transformation matrix. m c is a constant to determine the maximum length of subsequent positions. The symmetric transformation operator has ability to symmetrically exchange consecutive m c positions.
(4) Substitute transformation operator where, A sub k ∈ R n×n is called a substitute transformation matrix, and m d is a constant to determine the maximum number of positions to be substituted. The substitute transformation operator can be regarded as global exploration, and the positions of substituted elements are created randomly.

Hybrid discrete state transition algorithm
The state transition algorithm is wildly used in optimization problems because of its excellent performance in exploring complex solution space (global search). However, it is easily stuck in the optimum local area in large-scale problems. Contrarily, TS has superiority in local search (Ding et al., 2015) to help DSTA escape from the local optimum. TS allows transmitting from a better solution in the present iteration to a worse solution (Berlin´ska and Przybylski, 2021). Therefore, a hybrid state transition algorithm (HDSTA) is proposed by combining state transition algorithm with tabu search method. The HDSTA balances searches by combining DSTA and TS with the initialization scheme, resulting in higher quality solutions.

The main scheme of HDSTA
The algorithm starts with an initialization of the elite solution set φ i , and tentative multiple local optimal solutions are sorted in ascending order of the value of the objective function, where the solution in elite solution set is denoted by p n . Candidate solutions are generated by four special geometric operators defined in Eqs 2-5, and then the update strategies based on TS are introduced to select optimal solutions. Hybridization is carried out by inserting the TS into the DSTA framework to explore a local search space.
The purpose of introducing TS here is to balance global decentralized search with local area capabilities. TS considers a worse solution, which increases the risk of falling into a cyclic search. The main challenge in avoiding cyclic search lies in how to establish a mechanism for relevant historical information. Therefore, a tabu list with short-term memory is used to prevent duplicate responses, and the current solution can be displaced to the next one only if the new solution is not on the tabu list and all conditions are satisfied. One solution from the elite solutions is selected and sets as the current solution, and a certain number of candidates are selected from the neighbors of the current solution. Only if the candidate solution is much more adaptive than the solution in current elite solution set will it be FIGURE 3 Flow chart of the hybrid discrete state transition algorithm.

Frontiers in Energy Research
frontiersin.org accepted by new elite solution set. Conversely, if there is no candidate solution that satisfies this type, the next solution is selected from the elite solution set and its corresponding move is added to the tabu list. By iterating the above steps until the termination condition is met, the algorithm will stop. The flow chart of the proposed HDSTA is shown in Figure 3. The detailed steps are described below: Detailed steps of HDSTA.
Step 1: encoding combinatorial optimization problem using task sequence Step 2: setting parameters and generating the initialized population Step 3: evaluate the fitness value Step 4: is the maximal iteration reached? If reached, go to Step 7, else, go to Step 5 Step 5: reproduction using four special transformation operators defined in Eqs 2-5 Step 6: run TS neighborhood (insert, swapping), the go to Step 3 Step 7: output best solution

The elite solution set
In the traditional DSTA, the population is only related to the optimal solution of the previous generation. It is excellent in convergence rate but its probability of falling into local optimum increases. To address the limitations, the elite solution set with the information of previous optimal solution is proposed.
In initial population of the traditional DSTA, all solutions are positioned at the one side of search space. When this side of search space is the property of local optima, the algorithm will converge faster. Therefore, the initialization method of elite solution set is proposed to generate solutions located in different search spaces. The solution set Q 0 is initialized and the upper limit of the number of solutions in the set is defined as m, which is stored in ascending order of the objective function value. After creating the initial population, the fitness evaluation is performed by an objective function. During the iteration process, all the elite solutions searched in each generation are compared with the solutions stored in Q 0 . If there is a better solution, it will be replaced so as to ensure that the elite solution set Q 0 contains all the elite individuals in the search process. The initialization of the elite solution set is given in Algorithm 1. Here, S e is the number of solutions in the population, S best is the optimal solution in the population and Q 0 is the elite solution set.

Tabu search
TS is a widely used local search strategy based on the neighborhood search. Inspired by (Umam et al., 2022), the framework of this paper consists of the neighborhood structure, tabu list and move attribution. Firstly, the length of tabu list and the aspiration criteria are determined. Then, the maximum iteration of DSTA is taken as the termination condition. Next, using a neighborhood search with insertion and swapping methods makes a move. Finally, solutions are evaluated and updated with the best solution which is non-tabu of the search results till the termination condition is met.
The neighborhood structure determines the quality of the solution generated by the search, and also affects the search efficiency and convergence speed of the algorithm. In this paper, the state itself or the process of state transition is regarded as a tabu object. For example, when a state changes from a to b, the state b or the state transition a → b can be regarded as a tabu object, which is simple and easy to understand.
The tabu list affects the search process through the length of the list. The increase of list size causes a wider search scope, and more solutions are listed. The proposed HDSTA adopts the variable tabu list mode with the length of the list between the minimum and maximum values, and the length of tabu list increases from minimum to maximum gradually in the process. Whenever a better solution is obtained or the elite solution set is updated, the length of tabu list is increased to support deeper exploration in this area.
The candidate solution set consists of the better neighbor solutions in the neighbor solution set. The better neighbor solutions are selected by the fitness value function, then, the candidate solutions satisfying the forbidden and amnesty criteria are selected as the new current solutions in the candidate solution set. The mobility attribute is the key pillar for TS to find the optimum solution. The proposed algorithm defines two conditions. The first condition is that if a move of a member leads to a solution with higher cost than the best answer already obtained, then the member is removed from the tabu list. The second one is that if the structure of the method and tabu list in the stage of the algorithm is such that there is no possibility of moving to a member. In this case, the member closest to the exit is removed from the list.

Experimental results and discussion
The effectiveness of the proposed HDSTA is verified through three experiments, including the staff assignment problems, traveling salesman problems and a large-scale industry case using real-world data. All experiments are implemented on a computer with Intel Core i7-9700 CPU 4.8 GHz and 16 GB RAM. The population size is set to 50 for all experiments of different problems. The parameters, including elite solution set size, tabu list size and maximum iterations, are set to 30, 120 and 200 respectively. The best indicates the minimum value of results, the worst means the maximum value of results, and then it follows the mean (average value), st.dev (standard deviation) and time/s (computation time). These common statistics can indicate the Frontiers in Energy Research frontiersin.org convergence accuracy, search efficiency and stability of the algorithm.

Results analysis for staff assignment problem
Staff assignment problem is a well-known combinatorial optimization problem in human resources department (Mo et al., 2020), which can be described formally as follows: n employees are assigned to m locations, where the cost of the i th employee traveling to the k th location is defined by c ik . The communication time between employee i and employee j is denoted by t ij , and f kg represents the communication cost. It is required that each location must be assigned one employee, and each employee only travel to one location. The mathematical model can be expressed as follows: The matrix coefficients of different dimensions are introduced. Both c ik and t ij are random integers ranging from 1 to 100, and f kg is a random decimal ranging from 0 to 1. To verify the performance of the proposed HDSTA, it is compared with the discrete state transfer algorithm (DSTA) and the simulated annealing algorithm (SA). The cases of 20, 30, 40 and 50 dimensions are generated, and each algorithm runs 20 times for each case independently.
The performance of the different methods is illustrated in Table 1. It can be seen that the best and average values of the results when solving the above assignment problems with four different dimensions using HDSTA are better than those of SA and DSTA. In addition, the HDSTA algorithm takes less time to solve each problem than the simulated annealing algorithm and is close to that of DSTA. The above results reflect the effectiveness of the algorithm proposed by HDSTA algorithm.

Results analysis for traveling salesman problem
Traveling salesman problem (TSP) is another well-known combinatorial optimization problem. To test the performance of the HDSTA, the simulated annealing algorithm (SA) is compared with DSTA based on standard TSP data (Tagne Fute et al., 2022).
From the results shown in Table 2, it can be concluded that HDSTA outperforms SA and DSTA in almost all performances. HDSTA obtains better results in all four standard cases. Taking the att48.tsp standard case as an example, the best result is shown in Figure 4.   Table 2 shows the results of different standard cases. The standard deviation of HDSTA is lower than that of SA and DSTA, which proves that the proposed algorithm has a relatively strong stability. In addition, both HDSTA and DSTA have better advantages in terms of computation time than SA.

Results analysis for real-world industrial cases
Automated guided vehicles (AGV) are widely used to improve warehouse transport efficiency (Cheikhrouhou and Khoufi, 2021). Multi-AGV task assignment problem is to assign and sequence tasks for each AGV, with the main objective of minimizing the sum of costs through efficient AGV fleet management. When all warehouses in the layout are fixed and the cost of using AGV is only related to the moving distance, the task assignment problem is a special traveling salesman problem, whose polynomial can be expressed as below .
where K is the set of all vehicles with |K| m. i, j are nodes. d ij represents the distance from node i to j. x ijk , i, j ∈ V, k ∈ K is a binary variable used by vehicle k if the edge between node i and node j is 1, otherwise it is 0. The real AGV system of a warehouse in Changsha, China is used as the experimental platform. The system consists of 15 AGV and 39 warehouses, and the layout of the warehouses is illustrated in Figure 5. Practitioner feedback indicates that periods of 50 requests

FIGURE 4
The best route of att48.tsp obtained by HDSTA.

FIGURE 5
The layout of the warehouse. using over 5 AGV are regarded as busy periods (Li et al., 2022). Taking the number of AGV and task distances into consideration, the objective can be transformed into obtaining a near-optimal solution in a reasonable amount of time and determining its degree of approximation according to its NP-hard property.
To verify the effectiveness of HDSTA, 50-160 tasks were randomly generated, and AGV numbers range from 5 to 15. In the experiments, 10 groups of tasks were generated and ran 10 times for each case, and the mean value and computation time were assessed. In industrial applications, the average value and computation time are the most important performance indicators. In each case, HDSTA is compared with the adaptive large neighborhood search algorithm (HALNS) (Singh et al., 2022), and the pre-programmed algorithm (PPA) (Maza and Castagna, 2005). The performance and computation time of HALNS, PPA and HDSTA for 120 sets of task assignments assigned by 20 case types are shown in Tables 3. Compared with PPA and HALNS, HDSTA has a lower mean value and less computation time.

Conclusion
HDSTA is proposed to solve the large-scale combinatorial optimization problems. The TS method is introduced to guide the local search of DSTA to achieve the balance between the exploration capabilities and the exploitation capabilities. Furthermore, a new approach for the population generation of DSTA has been adapted. It avoids that all solutions are positioned at one side of the search space, resulting in too fast convergence. The experimental results of three large-scale combinatorial optimization problems show that the proposed method has better solving accuracy than the other two algorithms. The HDSTA can be further applied to more industrial cases. In addition, this research can be expanded by combining the DSTA with another local search algorithm for future works under multiple objectives.

Data availability statement
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.