A NetHack Learning Environment Language Wrapper for Autonomous Agents

Nikolaj Goodger; Peter Vamplew; Cameron Foale; Richard Dazeley

doi:10.5334/jors.444

A NetHack Learning Environment Language Wrapper for Autonomous Agents

Journal of Open Research Software

Volume 11 (2023): Issue 1

By: Nikolaj Goodger , Peter Vamplew , Cameron Foale and Richard Dazeley

Open Access

|Jun 2023

Abstract

This paper describes a language wrapper for the NetHack Learning Environment (NLE) [1]. The wrapper replaces the non-language observations and actions with comparable language versions. The NLE offers a grand challenge for AI research while MiniHack [2] extends this potential to more specific and configurable tasks. By providing a language interface, we can enable further research on language agents and directly connect language models to a versatile environment.

References

Küttler H, Nardelli N, Miller A, Raileanu R, Selvatici M, Grefenstette E, Rocktäschel T. The nethack learning environment. In Larochelle H, Ranzato M, Hadsell R, Balcan MF, Lin H (eds.), Advances in Neural Information Processing Systems. 2020; 33: 7671–7684. Curran Associates, Inc. URL https://proceedings.neurips.cc/paper/2020/file/569ff987c643b4bedf504efda8f786c2-Paper.pdf.
Search in Google Scholar Back to article
Samvelyan M, Kirk R, Kurin V, Parker-Holder J, Jiang M, Hambro E, Petroni F, Kuttler H, Grefenstette E, Rocktäschel T. Minihack the planet: A sandbox for open-ended reinforcement learning research. In Vanschoren J, Yeung S (eds.), Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. 2021; 1. URL https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/file/fa7cdfad1a5aaf8370ebeda47a1ff1c3-Paper-round1.pdf.
Search in Google Scholar Back to article
Hambro E, Mohanty S, Babaev D, Byeon M, Chakraborty D, Grefenstette E, Jiang M, Daejin J, Kanervisto A, Kim J, Kim S, Kirk R, Kurin V, Küttler H, Kwon T, Lee D, Mella V, Nardelli N, Nazarov I, Ovsov N, Holder J, Raileanu R, Ramanauskas K, Rocktäschel T, Rothermel D, Samvelyan M, Sorokin D, Sypetkowski M, Sypetkowski M. Insights from the neurips 2021 nethack challenge. In Kiela D, Ciccone M, Caputo B (eds.), Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track, volume 176 of Proceedings of Machine Learning Research. 06–14 Dec 2022; 41–52. PMLR. URL https://proceedings.mlr.press/v176/hambro22a.html.
Search in Google Scholar Back to article
Ruder S, Peters ME, Swayamdipta S, Wolf T. Transfer learning in natural language processing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Tutorials. June 2019; 15–18. Minneapolis, Minnesota: Association for Computational Linguistics. URL https://aclanthology.org/N19-5004. DOI: 10.18653/v1/N19-5004
Open DOI Search in Google Scholar Back to article
Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S, Herbert-Voss A, Krueger G, Henighan T, Child R, Ramesh A, Ziegler D, Wu J, Winter C, Hesse C, Chen M, Sigler E, Litwin M, Gray S, Chess B, Clark J, Berner C, McCandlish S, Radford A, Sutskever I, Amodei D. Language models are few-shot learners. In Larochelle H, Ranzato M, Hadsell R, Balcan MF, Lin, H (eds.), Advances in Neural Information Processing Systems. 2020; 33: 1877–1901. Curran Associates, Inc. URL https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf.
Search in Google Scholar Back to article
Schick, T, Schütze H. It’s not just size that matters: Small language models are also few-shot learners. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. June 2021; 2339–2352. Online. Association for Computational Linguistics. URL https://aclanthology.org/2021.naacl-main.185. DOI: 10.18653/v1/2021.naacl-main.185
Open DOI Search in Google Scholar Back to article
Reid M, Yamada Y, Gu SS. Can wikipedia help offline reinforcement learning? 2022. URL https://arxiv.org/abs/2201.12122.
Search in Google Scholar Back to article
Li S, Puig X, Paxton C, Du Y, Wang C, Fan L, Chen T, Huang D-A, Akyürek E, Anandkumar A, Andreas J, Mordatch I, Torralba A, Zhu Y. Pre-Trained Language Models for Interactive Decision-Making. arXiv e-prints, art. arXiv:2202.01771; February 2022.
Search in Google Scholar Back to article
Hill F, Mokra S, Wong N, Harley T. Human instruction-following with deep reinforcement learning via transfer-learning from text; 2020. URL https://arxiv.org/abs/2005.09382.
Search in Google Scholar Back to article
Zhou X, Zhang Y, Cui L, Huang D. Evaluating commonsense in pre-trained language models; 2019. URL https://arxiv.org/abs/1911.11931.
Search in Google Scholar Back to article
Goodger N, Vamplew P, Foale C, Dazeley R. Language representations for generalization in reinforcement learning. In Vineeth NB and Ivor T, (eds.), Proceedings of The 13th Asian Conference on Machine Learning. 2021; 157: 390–405. Virtual.
Search in Google Scholar Back to article
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Ukasz Kaiser Ł, Polosukhin I. Attention is all you need. In Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds.), Advances in Neural Information Processing Systems. 2017; 30. Curran Associates, Inc. URL https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.
Search in Google Scholar Back to article
Narasimhan K, Kulkarni T, Barzilay R. Language understanding for text-based games using deep reinforcement learning. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. September 2015; 1–11. Lisbon, Portugal: Association for Computational Linguistics. DOI: 10.18653/v1/D15-1001
Open DOI Search in Google Scholar Back to article
He J, Chen J, He X, Gao J, Li L, Deng L, Ostendorf M. Deep reinforcement learning with a natural language action space. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). August 2016; 1621–1630. Berlin, Germany. Association for Computational Linguistics. URL https://aclanthology.org/P16-1153. DOI: 10.18653/v1/P16-1153
Open DOI Search in Google Scholar Back to article
Côté M-A, Kádár Á, Yuan X, Kybartas B, Barnes T, Fine E, Moore J, Tao RY, Hausknecht M, El Asri L, Adada M, Tay W, Trischler A. Textworld: A learning environment for text-based games. CoRR, abs/1806.11532; 2018. DOI: 10.1007/978-3-030-24337-1_3
Open DOI Search in Google Scholar Back to article
Jansen PA, Côté M. Textworldexpress: Simulating text games at one million steps per second. CoRR, abs/2208.01174; 2022. DOI: 10.48550/arXiv.2208.01174
Open DOI Search in Google Scholar Back to article
Jiang M, Luketina J, Nardelli N, Minervini P, Torr PHS, Whiteson S, Rocktäschel T. Wordcraft: An environment for benchmarking commonsense agents. In Workshop on Language in Reinforcement Learning (LaRel); 2020. URL https://github.com/minqi/wordcraft.
Search in Google Scholar Back to article
Koch K, McLean J, Segev R, Freed M, Berry II M, Balasubramanian V, Sterling P. How much the eye tells the brain. Current biology: CB. 08 2006; 16: 1428–34. DOI: 10.1016/j.cub.2006.05.056
Open DOI Search in Google Scholar Back to article
Petrenko A, Huang Z, Kumar T, Sukhatme G, Koltun V. Sample factory: Egocentric 3d control from pixels at 100000 fps with asynchronous reinforcement learning. In ICML; 2020. DOI: 10.1016/j.cub.2006.05.056
Open DOI Search in Google Scholar Back to article
Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, Davison J, Shleifer S, von Platen P, Ma C, Jernite Y, Plu J, Xu C, Le Scao T, Gugger S, Drame M, Lhoest Q, Alexander M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. October 2020; 38–45. Online. Association for Computational Linguistics. URL https://www.aclweb.org/anthology/2020.emnlp-demos.6. DOI: 10.18653/v1/2020.emnlp-demos.6
Open DOI Search in Google Scholar Back to article

Articles in this issue

DOI: https://doi.org/10.5334/jors.444 | Journal eISSN: 2049-9647

Journal RSS Feed

Language: English

Submitted on: Oct 25, 2022

Accepted on: Mar 16, 2023

Published on: Jun 13, 2023

Published by: Ubiquity Press

In partnership with: Paradigm Publishing Services

Publication frequency: 1 issue per year

Keywords:

Autonomous Agents,

Generalization,

Language Models,

Language Representations,

NetHack,

Reinforcement Learning,

Transfer Learning

© 2023 Nikolaj Goodger, Peter Vamplew, Cameron Foale, Richard Dazeley, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Volume 11 (2023): Issue 1