This example implementation for Q learning uses an R library built by PennState University. The details of this library can be found at link below: https://methodology.psu.edu/downloads/qlearning
qlaci library zip can be downloaded at the link below: https://methodology.psu.edu/downloads/qlaci
This library for Reinforcement learning techniques is evolving. This code will be updated as and when any new implementations are published in the CRAN repository.
Reference from the above link: The qlaci (Q-learning with adaptive confidence intervals) R package can be used with data from a sequential, multiple assignment, randomized trial (SMART) to design an adaptive intervention. The qlaci R package requires R 2.15, available for free download. This is the recommended platform for running this package. At the time of release, R version 3.0.x was released recently. We will only support installation of qlaci on R 2.15.x.