A Data-Driven Approach Towards the Application of Reinforcement Learning Based HVAC Control

Authors

  • Constantin Falk ic3@Smart Production, University of Applied Sciences Wildau, Germany
  • Tarek El Ghayed ic3@Smart Production, University of Applied Sciences Wildau, Germany
  • Ron van de Sand ic3@Smart Production, University of Applied Sciences Wildau, Germany
  • Jörg Reiff-Stephan ic3@Smart Production, University of Applied Sciences Wildau, Germany

Abstract

Refrigeration applications consume a significant share of total electricity demand, with a high indirect impact on global warming through greenhouse gas emissions. Modern technology can help reduce the high power consumption and optimize the cooling control. This paper presents a case study of machine-learning for controlling a commercial refrigeration system. In particular, an approach to reinforcement learning is implemented, trained and validated utilizing a model of a real chiller plant. The reinforcement-learning controller learns to operate the plant based on its interactions with the modeled environment. The validation demonstrates the functionality of the approach, saving around 7% of the energy demand of the reference control. Limitations of the approach were identified in the discretization of the real environment and further model-based simplifications and should be addressed in future research.

Dimensions

M. M. Mabkhot et al., “Mapping industry 4.0 enabling technologies into united nations sustainability development goals”, Sustainability 13 (2021) 2560.

VDMA e.V. Allgemeine Lufttechnik, “Energiebedarf f¨ur K¨altetechnik in Deutschland: Eine Absch¨atzung des Energiebedarfs von K¨altetechnik in Deutschland nach Einsatzgebieten”, (2017).

K. Mason & S. Grijalva, “A review of reinforcement learning for autonomous building energy management”, Computers Electrical Engineering 78 (2019) 300.

S. Liu & G. P. Henze, “Evaluation of Reinforcement Learning for Optimal Control of Building Active and Passive Thermal Storage Inventory”, Journal of Solar Energy Engineering 129 (2006) 215.

Z. Guo, A. R. Co man & P. Barooah, “Reinforcement Learning for Optimal Control of a District Cooling Energy Plant”, 2022 American Control Conference (ACC) (2022) 3329.

J. R. V´azquez-Canteli & Z. Nagy, “Reinforcement learning for demand response: A review of algorithms and modeling techniques”, Applied Energy 235 (2019) 1072.

T. G. Hovgaard, L. F. S. Larsen & J. B. Jorgensen, “Flexible and cost efficient power consumption using economic MPC a supermarket refrigeration benchmark”, 2011 50th IEEE Conference on Decision and Control and European Control Conference (2011) 848.

T. G. Hovgaard, L. F. Larsen, K. Edlund & J. B. Jørgensen, “Model predictive control technologies for efficient and flexible power consumption in refrigeration systems”, Energy 44( (2012) 105.

R. Halvgaard, L. Vandenberghe, N. K. Poulsen, H. Madsen & J. B. Jorgensen, “Distributed Model Predictive Control for Smart Energy Systems”, IEEE Transactions on Smart Grid 7 (2016) 1675.

C. Fan & Y. Ding, “Cooling load prediction and optimal operation of HVAC systems using a multiple nonlinear regression model”, Energy and Buildings 197 (2019) 7.

G. Henze & J. Schoenmann, “Evaluation of Reinforcement Learning Control for Thermal Energy Storage Systems”, HVAC&R Research 9 (2003) 259.

D. Ernst, M. Glavic, F. Capitanescu & L. Wehenkel, “Reinforcement learning versus model predictive control: a comparison on a power system problem”, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 39 (2009) 517.

A. Beghi, M. Rampazzo & S. Zorzi, “Reinforcement Learning Control of Transcritical Carbon Dioxide Supermarket Refrigeration Systems”, IFAC-PapersOnLine 50 (2017) 13754.

T. Schreiber, S. Eschweiler, M. Baranski & D. M¨uller, “Application of two promising Reinforcement Learning algorithms for load shifting in a cooling supply system”, Energy and Buildings 229 (2020) 110490.

T. Schreiber, A. Schwartz & D. Muller, “Towards an intelligent HVAC system automation using Reinforcement Learning”, Journal of Physics: Conference Series 2042 (2021) 012028.

D. Zhang & Z. Gao, “Improvement of Refrigeration Efficiency by Combining Reinforcement Learning with a Coarse Model”, Processes 7 (2019) 967.

Y. Li, Y. Wen, D. Tao & K. Guan, “Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning”, IEEE transactions on cybernetics 50 (2020) 2002.

L. Yu, Y. Sun, Z. Xu, C. Shen, D. Yue, T. Jiang & X. Guan, “Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings”, IEEE Transactions on Smart Grid 12 (2021) 407.

Y. Chen, L. K. Norford, H. W. Samuelson & A. Malkawi, “Optimal control of HVAC and window systems for natural ventilation through reinforcement learning”, Energy and Buildings 169 (2018) 195.

S. Qiu, Z. Li, Z. Li & X. Zhang, “Model-free optimal chiller loading method based on Q-learning”, Science and Technology for the Built Environment 26 (2020) 1100.

S. Qiu, Z. Li, R. He, J. Li, and Z. Li, “How does the control logic influence the establishment of a data-driven chiller model?”, Journal of Physics: Conference Series 2006 (2021) 012002.

L. P. Kaelbling, M. L. Littman & A.W. Moore, “Reinforcement learning: A survey”, Journal of artificial intelligence research 4 (1996) 237.

C. C. White & D. J. White, “Markov decision processes”, European Journal of Operational Research 39 (1989) 1.

C. J. C. H. Watkins, Learning from delayed rewards (1989).

C. J. C. H.Watkins & P. Dayan, “Q-learning”, Machine Learning 8 (1992) 279.

S. A. Khalil, “Performance Evaluation and Statistical Analysis of Solar Energy Modeling: A Review and Case Study”, Journal of the Nigerian Society of Physical Sciences 4 (2022) 911.

G. Cerbe & G. Wilhelms, ”Technische Thermodynamik: Theoretische Grundlagen und praktische Anwendungen”, Carl Hanser Verlag GmbH Co KG (2021).

F. O. Aweda, J. A. Akinpelu, T. K. Samson, M. Sanni & B. S. Olatinwo, “Modeling and Forecasting Selected Meteorological Parameters for the Environmental Awareness in Sub-Sahel West Africa Stations”, Journal of the Nigerian Society of Physical Sciences 4 (2022) 820.

K. H. Yu, Y. A. Chen, E. Jaimes, W. C. Wu, K. K. Liao, J. C. Liao, K. C. Lu, W. J. Sheu & C. C. Wang, “Optimization of thermal comfort, indoor quality, and energy-saving in campus classroom through deep Q learning”, Case Studies in Thermal Engineering 24 (2021) 100842.

Published

2023-02-24

How to Cite

A Data-Driven Approach Towards the Application of Reinforcement Learning Based HVAC Control. (2023). Journal of the Nigerian Society of Physical Sciences, 5(1), 1244. https://doi.org/10.46481/jnsps.2023.1244

Issue

Volume 5, Issue 1, February 2023

Section

Special Issue : 3rd biennial AScIN conference OAU,  Nigeria
Copyright & Licensing

Copyright (c) 2023 Constantin Falk, Tarek El Ghayed , Ron van de Sand, Jörg Reiff-Stephan

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

A Data-Driven Approach Towards the Application of Reinforcement Learning Based HVAC Control. (2023). Journal of the Nigerian Society of Physical Sciences, 5(1), 1244. https://doi.org/10.46481/jnsps.2023.1244