Adaptive control strategies to support novel paradigms of predictive energy management and optimisation in smart buildings

Objective of the activity:

Exploit the potentialities of Deep Reinforcement Learning applied to a variety of HVAC system configuration

Framework of the activity:

Reinforcement Learning (RL) is an alternative approach to model-based control where an agent learns the optimal control policy by interacting with the controlled environment through a delayed reward mechanism. In ideal conditions a model-free DRL agents should be directly employed on the controlled environment to gradually learn the optimal control policy. However, since this process may take considerable amount of time leading to a poor control performance in its first implementation period. A common approach explored in the scientific literature to overcome this problem is to pre-train offline the RL agent using a surrogate model of the environment.

In this perspective BAEDA lab develops and analyzes RL control agents in a sofisticated simulation environement combining Python and EnergyPlus considering different architectures of the agents: single-agent and multi-agent. Moreover, different training methodologies are investigaterd: offline training, online training and imitation learning.

Figure: Simplified schematics of the implementation of a Deep Reinforcement Learning controller in the built environment.

Relevant publications on this topic:

Objective of the activity:

Define robust comparison among model-free and model-based control solutions

Framework of the activity:

The built environment is extremely heterogeneous where every building behaves as a unique entity. In order to identify the optimal control solution for each application in fundamental to develop robust comparisons between different techniques capable to highlight strengths and weakness of the different approaches. This process can support researchers and industry in moving towards the implementation more advanced control strategies for energy management in buildings.

BAEDA lab investigates different control approaches including Reinforcement Learning and Model Predictive Control. The characterization of the performance of such techniques is performed for different buildings and HVAC configurations and is carried out in dedicated simulation environments leveraging software like EnergyPlus, Python and Matlab.

Figure: Scheme of the different approaches of Model Predictive Control and Deep Reinforcement Learning

Objective of the activity:

Enable the penetration of data-driven models and advanced control strategies in buildings with a limited amount of operational data

Framework of the activity:

Advanced data-driven control strategies rely on complex models to describe and predict building and systems dynamics. The training process of these models - such as deep neural networks - requires the collection of large amount of historical data that are not usually available for new or refurbished buildings. As a consequence one of the main challenge towards a large-scale implementation of such controllers consists in the high information availability required, which limits their scalability and application.

In this context BAEDA lab conducts research activities aimed at implementing and testing transfer-learning frameworks to ease and enable the penetration of data-driven models and advanced control strategies in buildings with a limited amount of operational data.

Figure: Application of transfer-learning frameworks for the control of the built environment

Relevant publications on this topic:

Objective of the activity:

Implementation of adaptive controllers in real-world testbeds.

Framework of the activity:

One of the main barriers to the adoption of advanced control strategies in buildings is the lack of real-world testbeds capable to demonstrate to industry operators their effectiveness in achieving cost-effective performance compared to classical control strategies.

In this perspective BAEDA lab develops the in-field implementation of adaptive and predictive control strategies in real world environments including both university test facilities and real buildings in collaboration with our research partners. The implemented solution are designed taking care of every aspect of the development process from the conceptualization to the integration into the energy management system software thanks to the latest development in IoT and Cloud technologies.

Figure: The four pillars of real life implementation of advance controls in buildings