Reinforcement learning applied to an electric water heater: From theory to practice

F. Ruelens, B.J. Claessens, S. Quaiyum, B. De Schutter, R. Babuska, and R. Belmans, "Reinforcement learning applied to an electric water heater: From theory to practice," IEEE Transactions on Smart Grid, vol. 9, no. 4, pp. 3792-3800, 2018.

Electric water heaters have the ability to store energy in their water buffer without impacting the comfort of the end user. This feature makes them a prime candidate for residential demand response. However, the stochastic and non-linear dynamics of electric water heaters, makes it challenging to harness their flexibility. Driven by this challenge, this paper formulates the underlying sequential decision-making problem as a Markov decision process and uses techniques from reinforcement learning. Specifically, we apply an auto-encoder network to find a compact feature representation of the sensor measurements, which helps to mitigate the curse of dimensionality. A well-known batch reinforcement learning technique, fitted Q-iteration, is used to find a control policy, given this feature representation. In a simulation-based experiment using an electric water heater with 50 temperature sensors, the proposed method was able to achieve good policies much faster than when using the full state information. In a lab experiment, we apply fitted Q-iteration to an electric water heater with eight temperature sensors. Further reducing the state vector did not improve the results of fitted Q-iteration. The results of the lab experiment, spanning 40 days, indicate that compared to a thermostat controller, the presented approach was able to reduce the total cost of energy consumption of the electric water heater by 15%.

 * Online version of the paper
 * Corresponding technical report: pdf file (1.86 MB)
      Note: More information on the pdf file format mentioned above can be found here.

Bibtex entry:

        author={F. Ruelens and B.J. Claessens and S. Quaiyum and B. {D}e Schutter and R. Babu{\v{s}}ka and R. Belmans},
        title={Reinforcement learning applied to an electric water heater: From theory to practice},
        journal={IEEE Transactions on Smart Grid},

Go to the publications overview page.

This page is maintained by Bart De Schutter. Last update: March 21, 2022.