Conditional Abstraction Trees for Sample-Efficient Reinforcement Learning

Mehdi Dadvar, Rashmeet Kaur Nayyar, Siddharth Srivastava

Research output: Contribution to journalConference articlepeer-review

Abstract

In many real-world problems, the learning agent needs to learn a problem's abstractions and solution simultaneously. However, most such abstractions need to be designed and refined by hand for different problems and domains of application. This paper presents a novel top-down approach for constructing state abstractions while carrying out reinforcement learning (RL). Starting with state variables and a simulator, it presents a novel domain-independent approach for dynamically computing an abstraction based on the dispersion of temporal difference errors in abstract states as the agent continues acting and learning. Extensive empirical evaluation on multiple domains and problems shows that this approach automatically learns semantically rich abstractions that are finely-tuned to the problem, yield strong sample efficiency, and result in the RL agent significantly outperforming existing approaches.

Original languageEnglish (US)
Pages (from-to)485-495
Number of pages11
JournalProceedings of Machine Learning Research
Volume216
StatePublished - 2023
Externally publishedYes
Event39th Conference on Uncertainty in Artificial Intelligence, UAI 2023 - Pittsburgh, United States
Duration: Jul 31 2023Aug 4 2023

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering
  • Statistics and Probability

Fingerprint

Dive into the research topics of 'Conditional Abstraction Trees for Sample-Efficient Reinforcement Learning'. Together they form a unique fingerprint.

Cite this