Robotic Manipulation Datasets for Offline Compositional Reinforcement Learning

CoRL Workshop on Pre-training Robot Learning, 2022
PDF   Code  

Abstract: Offline reinforcement learning (RL) is a promising direction that allows RL agents to be pre-trained from large datasets avoiding recurrence of expensive data collection. To advance the field, it is crucial to generate large-scale datasets. Compositional RL is particularly appealing for generating such large datasets, since 1) it permits creating many tasks from few components, and 2) the task structure may enable trained agents to solve new tasks by combining relevant learned components. This paper provides four offline RL datasets for simulated robotic manipulation created using the 256 tasks from CompoSuite. Each dataset is collected from an agent with a different degree of performance, and consists of 256 million transitions. We provide training and evaluation settings for assessing an agent’s ability to learn compositional task policies. Our benchmarking experiments on each setting show that current offline RL methods can learn the training tasks to some extent, but are unable to extract their compositional structure to generalize to unseen tasks, showing a need for further research in offline compositional RL.

Recommended citation:

@inproceedings{mendez2022composuite, 
 author = {Hussing, Marcel and Mendez-Mendez, Jorge and Kent, Cassandra and Eaton, Eric}, 
 booktitle = {CoRL 2022 Workshop on Pre-training Robot Learning}, 
 title = {Robotic Manipulation Datasets for Offline Compositional Reinforcement Learning}, 
 year = {2022} 
}