From: 2023-11-24 10:15 to 12:00
Place: Lecture Hall M:B in M-huset, Ole Römers väg 1, Lund
Contact: albin [dot] heimerson [at] control [dot] lth [dot] se

Title: Learning to Control the Cloud
Speaker: Albin Heimerson
Opponent: Professor lvona Brandic, Vienna University of Technology, Austria
Advisor: Professor Johan Eker Lund University
Where: Lecture Hall M:B in M-huset, Ole Römers väg 1, Lund
When: Friday November 24th, 10:15
Zoom:  https://lu-se.zoom.us/j/61709012994

Abstract:
The cloud industry's rapid growth has raised concerns about energy consumption in the underlying infrastructure. Efficient resource management and control are crucial, alongside reducing hardware energy usage. As cloud systems become more complex, intelligent and automated controllers are increasingly needed.

Cloud environments involve diverse consumers sharing resources, creating dynamic and unpredictable loads. Intelligent automation benefits both consumers and providers. Reinforcement learning, a type of machine learning suited for sequential decision-making, offers potential solutions. We explore applying reinforcement learning to control cloud systems with complex objectives, addressing practical usability concerns. 

We begin with joint control of cooling and load balancing in datacenters, showing that it can reduce energy  consumption while maintaining server temperature thresholds. Reinforcement learning efficiently manages cooling and surpasses standard methods, even in more realistic simulations. 

Next we consider automatic scaling of interconnected microservices in a cloud environment. We create proactive reinforcement learning controllers that optimize resource allocations by understanding job dynamics, enhancing the performance compared to reactive controllers.

Transitioning to model-based control, we use a fluid model of a microservice application to optimize a load balancing policy. By applying automatic differentiation over a cost function based on model predictions, we can optimize the load balancing policy using gradient descent. While the fluid model is accurate within training data limits, we show how to extend it with a neural network to improve performance outside the training data.