Human-in-the-Loop Reinforcement Learning for Energy-Efficient Smart Home Recommendations

Smart homes generate a continuous stream of sensor, occupancy and appliance data that is well suited to data-driven energy optimisation, yet the majority of deployed home energy management systems still rely on static rule-based control and manual scheduling. Such systems adapt poorly to evolving occupant routines, seasonal variation and conflicting preferences among household members, which results either in avoidable energy waste or in comfort losses. This paper presents a recommender system whose policy is learned entirely from occupant feedback: a Proximal Policy Optimization (PPO) agent is trained online from accept and reject signals, with no pre-collected preference corpus, and is paired with a deterministic rule engine that supplies a behavioural prior during the cold-start period. The system is embedded in a complete Internet of Things (IoT) architecture comprising a layered application server, a relational data model and an Android client. The agent observes a nine-dimensional state vector combining temporal, environmental, device-state and rule-baseline features, and emits a device action together with a confidence score and a natural-language justification. Recommendations are validated against a priority-ordered safety rule set and reconciled across occupants through a combined weighted-average and majority-voting strategy before being delivered to the user. The nine PPO hyperparameters were tuned by Bayesian optimisation using a Tree-structured Parzen Estimator over a lightweight surrogate environment. The system was evaluated on a synthetic but behaviourally grounded one-year dataset generated at a thirty-minute resolution for a three-occupant, six-room dwelling, under three user-feedback regimes and against a rule-based baseline and a Deep Q-Network (DQN) variant. Under full acceptance the system reduced annual consumption by 14.2% on average, reaching 27.8% in the most favourable month, while under a realistic mixed-feedback regime it delivered 6.4% average annual savings with an agent confidence of approximately 74%. PPO outperformed the DQN variant in every month of the simulated year, with the gap widening in periods dominated by continuous set-point control. The results indicate that policy-gradient reinforcement learning driven by human feedback and anchored by a rule-based prior is an effective basis for adaptive, comfort-preserving energy recommendations in residential settings.