1. ABSTRACT
  2. TABLE OF CONTENTS
  3. LIST OF FIGURES
  4. LIST OF TABLES
  5. ACKNOWLEDGMENTS
  6. 1. INTRODUCTION
  7. 2. REINFORCEMENT LEARNING SYSTEMS
  8. 3. THE ADVANTAGE UPDATING ALGORITHM
    1. Advantage Updating
  9. 4. A LINEAR QUADRATIC REGULATOR PROBLEM
  10. 5. Q-LEARNING WITH SMALL TIME STEPS
  11. 6. SIMULATION RESULTS
  12. 7. CONVERGENCE OF ADVANTAGE UPDATING
  13. 8. IMPLEMENTATION ISSUES
  14. 9. CONCLUSION
  15. 10. BIBLIOGRAPHY
  16. APPENDIX A: NOTATION
  17. APPENDIX B: LQR CONSTANTS