Reference:
F. Ruelens,
B.J. Claessens,
S. Quaiyum,
B. De Schutter,
R. Babuska, and
R. Belmans,
"Reinforcement learning applied to an electric water heater: From
theory to practice," IEEE Transactions on Smart Grid, vol. 9,
no. 4, pp. 3792-3800, 2018.
Abstract:
Electric water heaters have the ability to store energy in their water
buffer without impacting the comfort of the end user. This feature
makes them a prime candidate for residential demand response. However,
the stochastic and non-linear dynamics of electric water heaters,
makes it challenging to harness their flexibility. Driven by this
challenge, this paper formulates the underlying sequential
decision-making problem as a Markov decision process and uses
techniques from reinforcement learning. Specifically, we apply an
auto-encoder network to find a compact feature representation of the
sensor measurements, which helps to mitigate the curse of
dimensionality. A well-known batch reinforcement learning technique,
fitted Q-iteration, is used to find a control policy, given this
feature representation. In a simulation-based experiment using an
electric water heater with 50 temperature sensors, the proposed method
was able to achieve good policies much faster than when using the full
state information. In a lab experiment, we apply fitted Q-iteration to
an electric water heater with eight temperature sensors. Further
reducing the state vector did not improve the results of fitted
Q-iteration. The results of the lab experiment, spanning 40 days,
indicate that compared to a thermostat controller, the presented
approach was able to reduce the total cost of energy consumption of
the electric water heater by 15%.