Grant: Alliance Grants

The modern world relies heavily on the correct operation of communication networks such as the Internet. Involving human cyber experts in typical penetration tests is expensive and cumbersome. Machine Learning in general, and Reinforcement Learning in particular, hold great promise here. Trained autonomous agents may exhibit human-level or even superior intelligence in the choice of their actions. Training such autonomous agents requires a suitable training environment. Simulation-based environments typically are abstract and high-level, making it difficult to transfer a trained agent into real networks. Emulation-based training environments usually suffer from very slow learning rates, limiting their use to small-scale networks. We are working with a unique training and evaluation environment for cyber agents that unifies both real (emulated) and simulated cyber networks with a complete cross-training loop. Training agents is a time-consuming task. We need to improve the training efficiency to scale up towards more realistic scenarios. Furthermore, the learning process is typically not general but trains agents to solve a specific task for a given network. We require training approaches that allow agents to generalize and perform well when the environment changes. One important characteristic of cybersecurity systems is that when a new task is learned, we should avoid forgetting the previous knowledge, otherwise the systems become vulnerable to previous attacks. This research described in this proposal will address these shortcomings. By carefully designing the training scenarios, we can learn complex scenarios incrementally and faster. Applying results from continual learning and decision transformer models as foundational models, we can train agents that will demonstrate generalizability. Ultimately, advanced on both fronts will result in a training environment that allows us to train realistic agents for non-trivial scenarios that perform well across diverse scenarios.