Alexander Nikulin

Hi there! I am a PhD student at MIPT, studying Offline Reinforcement Learning. I'm also working as a Senior Research Scientist at AIRI, publishing papers and supervising students. Before AIRI, I worked at Tinkoff AI. I'm best known for my work as a core developer of the CORL library and the XLand-MiniGrid environment. Before that, I completed a Master's degree in Machine Learning and Data Analysis and a Bachelor's degree in Sociology and Social Informatics at the Higher School of Economics. I completed internships at JetBrains Research, VK and TUHH during this time.

Google Scholar Github Twitter CV

a.p.nikulin at tinkoff dot ai | hsehowuhh at gmail dot com

XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning

Alexander Nikulin, Ilya Zisman, Alexey Zemtsov, Viacheslav Sinii, Vladislav Kurenkov, Sergey Kolesnikov
Preprint, 2024
PDF • Code

XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX

Alexander Nikulin, Vladislav Kurenkov, Ilya Zisman, Artem Agarkov, Viacheslav Sinii, Sergey Kolesnikov
Datasets and Benchmarks Track at Neural Information Processing Systems (NeurIPS), Poster, 2024
Intrinsically Motivated Open-ended Learning Workshop at Neural Information Processing Systems (NeurIPS), 2023
PDF • Code

In-Context Reinforcement Learning for Variable Action Spaces

Viacheslav Sinii, Alexander Nikulin, Vladislav Kurenkov, Ilya Zisman, Sergey Kolesnikov
International Conference on Machine Learning (ICML), Poster, 2024
PDF • Code

Emergence of In-Context Reinforcement Learning from Noise Distillation

Ilya Zisman, Vladislav Kurenkov, Alexander Nikulin, Viacheslav Sinii, Sergey Kolesnikov
International Conference on Machine Learning (ICML), Poster, 2024
PDF • Code

Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement Learning

Shengyi Huang, Quentin Gallouédec, Florian Felten, Antonin Raffin, Rousslan Fernand Julien Dossa, Yanxiao Zhao, Ryan Sullivan, Viktor Makoviychuk, Denys Makoviichuk, Mohamad H. Danesh, Cyril Roumégous, Jiayi Weng, Chufan Chen, Md Masudur Rahman, João G. M. Araújo, Guorui Quan, Daniel Tan, Timo Klein, Rujikorn Charakorn, Mark Towers, Yann Berthelot, Kinal Mehta, Dipam Chakraborty, Arjun KG, Valentin Charraut, Chang Ye, Zichen Liu, Lucas N. Alegre, Alexander Nikulin, Xiao Hu, Tianlin Liu, Jongwook Choi, Brent Yi
Preprint, 2024
PDF • Code

Anti-Exploration by Random Network Distillation

Alexander Nikulin, Vladislav Kurenkov, Denis Tarasov, Sergey Kolesnikov
International Conference on Machine Learning (ICML), Poster, 2023
PDF • Poster • Code

Revisiting the Minimalist Approach to Offline Reinforcement Learning

Denis Tarasov, Vladislav Kurenkov, Alexander Nikulin, Sergey Kolesnikov
Neural Information Processing Systems (NeurIPS), Poster, 2023
Workshop on Reincarnating Reinforcement Learning at International Conference on Learning Representations (ICLR), 2023
PDF • Code

CORL: Research-oriented Deep Offline Reinforcement Learning Library

Denis Tarasov, Alexander Nikulin, Dmitry Akimov, Vladislav Kurenkov, Sergey Kolesnikov
Datasets and Benchmarks Track at Neural Information Processing Systems (NeurIPS), Poster, 2023
3rd Offline Reinforcement Learning Workshop at Neural Information Processing Systems (NeurIPS), 2022
★ 1k+ stars on the Github ★
PDF • Code • Code (old version) • IT's Tinkoff Talk • PyCon Russia 2023 Talk

Katakomba: Tools and Benchmarks for Data-Driven NetHack

Vladislav Kurenkov, Alexander Nikulin, Denis Tarasov, Sergey Kolesnikov
Datasets and Benchmarks Track at Neural Information Processing Systems (NeurIPS), Poster, 2023
PDF • Code

Q-Ensemble for Offline RL: Don't Scale the Ensemble, Scale the Batch Size

Alexander Nikulin, Vladislav Kurenkov, Denis Tarasov, Dmitry Akimov, Sergey Kolesnikov
3rd Offline Reinforcement Learning Workshop at Neural Information Processing Systems (NeurIPS), 2022
PDF • Code

Let Offline RL Flow: Training Conservative Agents in the Latent Space of Normalizing Flows

Dmitriy Akimov, Vladislav Kurenkov, Alexander Nikulin, Denis Tarasov, Sergey Kolesnikov
3rd Offline Reinforcement Learning Workshop at Neural Information Processing Systems (NeurIPS), 2022
PDF • Code

MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learned

Anssi Kanervisto, Stephanie Milani, Karolis Ramanauskas, Nicholay Topin, Zichuan Lin, Junyou Li, Jianing Shi, Deheng Ye, Qiang Fu, Wei Yang, Weijun Hong, Zhongyue Huang, Haicheng Chen, Guangjun Zeng, Yue Lin, Vincent Micheli, Eloi Alonso, François Fleuret, Alexander Nikulin, Yury Belousov, Oleg Svidchenko, Aleksei Shpilman
NeurIPS 2021 Competitions and Demonstrations Track, in Proceedings of Machine Learning Research (PMLR) 176:13-28, 2022
★ Awarded with a $200 prize for the Community Support nomination ★
PDF

Machine learning models for photonic crystals band diagram prediction and gap optimisation

Alexander Nikulin, Ilya Zisman, Manfred Eich, Alexander Yu. Petrov, Alexander Itin
Photonics and Nanostructures-Fundamentals and Applications 52, 101076, 2022
PDF

Most of the talks and sources are in my native language - Russian.

2024

Blog post about XLand-100B dataset, largest to date in in-context RL.

Short Forbes interview, about my story and the research for which I won the Yandex ML Prize (video in russian).

Democratizing Meta-RL Research talk presenting XLand-MiniGrid library (video in russian)

2023

Was awarded with Yandex ML Prize (Ilya Segalovich Award) in the First publication nomination.

Gave small informal interview about my research at Fall Into ML 2023 conference (video in russian).

Short 10min talk about my SAC-RND paper at Fall Into ML 2023 conference (video in russian)

20x Faster Uncertainty Estimation for Offline RL talk presenting my SAC-RND paper (video in russian)

PyCon Russia 2023 talk presenting CORL library first major public release (video in russian)

Interview about my path to the research as a self-taught programmer with a non-traditional background (sociology)

Small interview about the research process and collaborations with students in our TinkoffLab

2022

IT's Tinkoff talk presenting CORL library (video in russian)

Final presentation of results achieved during research internship at tinkoff.ai (video in russian)

2021

ML seminars for students at JetBrains Research [1, 2, 3, 4]

Some random stuff will be here in the future.