A novel approach to optimizing any given mathematical function, called the MOdified REinforcement Learning Algorithm (MORELA), is proposed. Although Reinforcement Learning (RL) is primarily developed for solving Markov decision problems, it can be used with some improvements to optimize mathematical functions. At the core of MORELA, a sub-environment is generated around the best solution found in the feasible solution space and compared with the original environment. Thus, MORELA makes it possible to discover global optimum for a mathematical function because it is sought around the best solution achieved in the previous learning episode using the sub-environment. The performance of MORELA has been tested with the results obtained from other optimization methods described in the literature. Results exposed that MORELA improved the performance of RL and performed better than many of the optimization methods to which it was compared in terms of the robustness measures adopted.
References
[1]
Kwon, Y.D., Kwon, S.B. and Kim, J. (2003) Convergence Enhanced Genetic Algorithm with Successive Zooming Method for Solving Continuous Optimization Problems. Computers and Structures, 81, 1715-1725.
[2]
Hamzacebi, C. (2008) Improving Genetic Algorithms’ Performance by Local Search for Continuous Function Optimization. Applied Mathematics and Computation, 196, 309-317.
[3]
Baskan, O., Haldenbilen, S., Ceylan, H. and Ceylan, H. (2009) A New Solution Algorithm for Improving Performance of Ant Colony Optimization. Applied Mathematics and Computation, 211, 75-84.
[4]
Kiran, M.S., Gündüz, M. and Baykan, O.M. (2012) A Novel Hybrid Algorithm Based on Particle Swarm and Ant Colony Optimization for Finding the Global Minimum. Applied Mathematics and Computation, 219, 1515-1521.
[5]
Valian, E., Tavakoli, S. and Mohanna, S. (2014) An Intelligent Global Harmony Search Approach to Continuous Optimization Problems. Applied Mathematics and Computation, 232, 670-684.
[6]
Yu, S., Zhu, S., Ma, Y. and Mao, D. (2015) A Variable Step Size Firefly Algorithm for Numerical Optimization. Applied Mathematics and Computation, 263, 214-220.
[7]
Shelokar, P.S., Siarry, P., Jayaraman, V.K. and Kulkarni, B.D. (2007) Particle Swarm and Ant Colony Algorithms Hybridized for Improved Continuous Optimization. Applied Mathematics and Computation, 188, 129-142.
[8]
Kao, Y.-T. and Zahara, E. (2008) A Hybrid Genetic Algorithm and Particle Swarm Optimization for Multimodal Functions. Applied Soft Computing, 8, 849-857.
[9]
Seckiner, S.U., Eroglu, Y., Emrullah, M. and Dereli, T. (2013) Ant Colony Optimization for Continuous Functions by Using Novel Pheromone Updating. Applied Mathematics and Computation, 219, 4163-4175.
[10]
Hsieh, Y.-Z. and Su, M.-C. (2016) A Q-Learning-Based Swarm Optimization Algorithm for Economic Dispatch Problem. Neural Computing and Applications, 27, 2333-2350. https://doi.org/10.1007/s00521-015-2070-1
[11]
Samma, H., Lim, C.P. and Saleh, J.M. (2016) A New Reinforcement Learning-Based Memetic Particle Swarm Optimizer. Applied Soft Computing, 43, 276-297.
[12]
Walraven, E., Spaan, M.T.J. and Bakker, B. (2016) Traffic Flow Optimization: A Reinforcement Learning Approach. Engineering Applications of Artificial Intelligence, 52, 203-212.
[13]
Tozer, B., Mazzuchi, T. and Sarkani, S. (2017) Many-Objective Stochastic Path Finding Using Reinforcement Learning. Expert Systems with Applications, 72, 371-382.
[14]
Sutton, R.S. and Barto, A.G. (1998) Reinforcement Learning: An Introduction. The MIT Press, Cambridge, MA, USA; London, England.
[15]
Ozan, C. (2012) Dynamic User Equilibrium Urban Network Design Based on Modified Reinforcement Learning Method. Ph.D. Thesis, The Graduate School of Natural and Applied Sciences, Pamukkale University, Denizli, Turkey. (In Turkish)
[16]
Abdulhai, B. and Kattan, L. (2003) Reinforcement Learning: Introduction to Theory and Potential for Transport Applications. Canadian Journal of Civil Engineering, 30, 981-991. https://doi.org/10.1139/l03-014
[17]
Bazzan, A.L.C., Oliviera, D. and Silva, B.C. (2010) Learning in Groups of Traffic Signals. Engineering Applications of Artificial Engineering, 23, 560-568.
[18]
Vanhulsel, M., Janssens, D., Wets, G. and Vanhoof, K. (2009) Simulation of Sequential Data: An Enhanced Reinforcement Learning Approach. Expert Systems with Applications, 36, 8032-8039.
[19]
Kaelbling, L.P., Littman, M.L. and Moore, A.W. (1996) Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research, 4, 237-285.
[20]
Liu, F. and Zeng, G. (2009) Study of Genetic Algorithm with Reinforcement Learning to Solve the TSP. Expert Systems with Applications, 36, 6995-7001.
[21]
Maravall, D., De Lope, J. and Martin, H.J.A. (2009) Hybridizing Evolutionary Computation and Reinforcement Learning for the Design of Almost Universal Controllers for Autonomous Robots. Neurocomputing, 72, 887-894.
[22]
Chen, Y., Mabu, S., Shimada, K. and Hirasawa, K. (2009) A Genetic Network Programming with Learning Approach for Enhanced Stock Trading Model. Expert Systems with Applications, 36, 12537-12546.
[23]
Wu, J., Xu, X., Zhang, P. and Liu, C. (2011) A Novel Multi-Agent Reinforcement Learning Approach for Job Scheduling in Grid Computing. Future Generation Computer Systems, 27, 430-439.
[24]
Derhami, V., Khodadadian, E., Ghasemzadeh, M. and Bidoki, A.M.Z. (2013) Applying Reinforcement Learning for Web Pages Ranking Algorithms. Applied Soft Computing, 13, 1686-1692.
[25]
Khamis, M.A. and Gomaa, W. (2014) Adaptive Multi-Objective Reinforcement Learning with Hybrid Exploration for Traffic Signal Control Based on Cooperative Multi-Agent Framework. Engineering Applications of Artificial Intelligence, 29, 134-151.
[26]
Ozan, C., Baskan, O., Haldenbilen, S. and Ceylan, H. (2015) A Modified Reinforcement Learning Algorithm for Solving Coordinated Signalized Networks. Transportation Research Part C, 54, 40-55.
[27]
Liu, B., Wang, L., Jin, Y.-H., Tang, F. and Huang, D.-X. (2005) Improved Particle Swarm Optimization Combined with Chaos. Chaos, Solitons and Fractals, 25, 1261-1271.
[28]
Sun, W. and Dong, Y. (2011) Study of Multiscale Global Optimization Based on Parameter Space Partition. Journal of Global Optimization, 49, 149-172. https://doi.org/10.1007/s10898-010-9540-x
[29]
Chen, C., Chang, K. and Ho, S. (2011) Improved Framework for Particle Swarm Optimization: Swarm Intelligence with Diversity-Guided Random Walking. Expert Systems and Applications, 38, 12214-12220.
[30]
Toksari, M.D. (2009) Minimizing the Multimodal Functions with Ant Colony Optimization Approach. Expert Systems and Applications, 36, 6030-6035.
[31]
Chelouah, R. and Siarry, P. (2000) Tabu Search Applied to Global Optimization. European Journal of Operational Research, 123, 256-270.
[32]
Tutkun, N. (2009) Optimization of Multimodal Continuous Functions Using a New Crossover for the Real-Coded Genetic Algorithms. Expert Systems and Applications, 36, 8172-8177.
