Self Play Competitive Training With Dynamic Opponent Modeling

via “self-play competitive training with dynamic opponent modeling”

* ⭐ 02/2022: [Magnetic control of tokamak plasmas through deep reinforcement learning](https://www.nature.com/articles/s41586-021-04301-9%E2%80%A6)

Unique: Implements dynamic opponent selection based on skill-matched pairings from a maintained population of agent checkpoints, creating an implicit curriculum where agents face progressively stronger opponents as they improve, rather than training against fixed or random opponents

vs others: Produces more diverse and robust racing strategies than single-agent RL or training against fixed opponents because competitive pressure drives agents to discover novel tactics and counter-strategies continuously

via “opponent modeling and belief inference”

Paper on imperfect information games

Unique: Implements incremental Bayesian belief updating specifically for game contexts, allowing real-time refinement of opponent models as new information arrives, rather than batch retraining approaches used in general ML

vs others: More sample-efficient than pure neural network opponent modeling because it leverages game-theoretic structure and explicit probability distributions, enabling faster adaptation with limited game history

via “adaptive difficulty scaling based on player skill”

Unique: Uses model selection as the primary difficulty lever rather than implementing depth-limited search or move filtering, allowing the same codebase to serve multiple skill levels without chess-specific tuning. This is simpler to implement but less precise than traditional engine difficulty controls.

vs others: Simpler to implement than Lichess's depth-based difficulty (which requires a specialized engine), but less granular and less predictable in difficulty progression.