Agent Architecture

The agent is split into four sub agents which depend on each other with the goal to improve the final performance. What follows is a description and an illustration on how the agents interact with each other.

Overview of the binbinchen agent. Read from top left to bottom right. (Source: original binbinchen agent presentation from binbinchen)

The first two agents, the Teacher and the Tutor are greedy expert agents responsible for generating a reduced actions space(RAS) and experience in form of (action, observation) pairs.

With the experience from the Tutor, the Junior is then going to train a neural network which tries to clone the behaviour of the Tutor.

Finally the trained neural network of the Junior is used by senior as a starting point for training the Senior which tries to choose actions less greedily and with consideration of the future.

Together with some expert rules, like reconnection of lines the Senior is then used as the final agent, reaching the best performance. That agent is defined in curriculumagent.submission.my_agent.

Paper

If you want a detailed explanation of each agent and their mechanisms, we again refer to the paper Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents

Baseline and Pipeline overview

Considering that the full Teacher-Tutor-Junior-Senior pipeline might seem a little bit excessive, we provide within the package a baseline module, described in L2RPN-Baseline. Here you can first initialize the Baseline and either import and retrain an already existing model, or train all of the steps yourself.