Using reinforcement learning in genome assembly: in-depth analysis of a Q-learning assembler

Kleber Padovani^1*Rafael Cabral Borges^2*Roberto Xavier²André Carlos Ponce De Leon Ferreira De Carvalho³Anna Reali³Annie Chateau⁴Ronnie Alves^2,5

• ¹University of the State of Amazonas, Manaus, Brazil
• ²Vale Institute of Technology, Belém, Pará, Brazil
• ³Universidade de Sao Paulo, So Paulo, Brazil
• ⁴Universite de Montpellier, Montpellier, France
• ⁵Universidade Federal do Para, Belm, Brazil

Genome assembly remains an unsolved problem, and de novo strategies (i.e., those run without a reference) are relevant but computationally complex tasks in genomics. Although de novo assemblers have been previously successfully applied in genomic projects, there is still no 'best assembler', and the choice and setup of assemblers still rely on bioinformatics experts. Thus, as with other computationally complex problems, machine learning has emerged as an alternative (or complementary) way to develop accurate, fast and autonomous assemblers. Reinforcement learning has proven promising for solving complex activities without supervision, such as games, and there is a pressing need to understand the limits of this approach to 'real-life' problems, such as the DNA fragment assembly problem. In this study, we analyze the boundaries of applying machine learning via reinforcement learning (RL) for genome assembly. We expand upon the previous approach found in the literature to solve this problem by carefully exploring the learning aspects of the proposed intelligent agent, which uses the Q-learning algorithm. We improved the reward system and optimized the exploration of the state space based on pruning and in collaboration with evolutionary computing (>300% improvement). We tested the new approaches on 23 environments. Our results suggest the unsatisfactory performance of the approaches, both in terms of assembly quality and execution time, providing strong evidence for the poor scalability of the studied reinforcement learning approaches to the genome assembly problem. Finally, we discuss the existing proposal, complemented by attempts at improvement that also proved insufficient. In doing so, we contribute to the scientific community by offering a clear mapping of the limitations and challenges that should be taken into account in future attempts to apply reinforcement learning to genome assembly.