Robust residential load forecasting with manual feature engineering

kirish xususiyatlari sifatida tarixiy yuk va harorat ma’lumotlaridan foydalangan holda o‘rganildi, tasdiqlandi va sinovdan o‘tkazildi. Takrorlanuvchi qatlamlarga xususiyatni avtomatik ravishda chiqarish CNN modeli yordamida amalga oshirildi. Kalit so‘zlar. bashorat qilish modeli, kiritish xususiyatlari, xususiyatlarni ajratib olish, qo‘lda usullar, modelning ishlashi, ishonchliligi. Introduction. The efficiency of the power systems is an important characteristic that measures consumers' utilized share of the total produced energy. Power loss in the entire system is accumulated due to the transmission and distribution loss on the lines, loss in substations, and transforming points. However, another major factor contributes to the mass of loss. It is a sudden drop of consumers on the network especially at night. Consumption patterns of all consumers are not always stable and residential users are very stochastic. Energy traders and bulk producers are sensitive to this situation as the deactivation of the production blocks is expensive and energy is lost in the system. The solution is to redirect the produced energy to the other consumers. Thus, robust management of energy flow requires accurate forecasting of future power demand. Various horizons of forecasting are deployed in the power system and short-time load forecasting (STLF) is essential for scheduling power generation and distribution. Accurate forecasting is mandatory for a robust scheduling system. That’s why power industry experts and forecasting researchers are interested in developing diverse methods of forecasting improvement. Short-term load forecasting has a long evolution starting from statistical AR, ARMA, ARIMA, and SARIMA techniques to linear regression, support vector machines, and neural networks. However, accurate demand prediction remains a challenging issue among researchers. In particular, complex non-linearity and seasonality patterns need to be considered while load profiling “Фан ва технологиялар тараққиёти” Илмий–техникавий журнал №2/2025 181 Информатика и информационно-коммуникационные системы at an individual household scale. Many factors affect the energy consumption of households, such as weather conditions, installed appliances, thermal properties of the building, and most importantly the consumption behavior of each resident [1]. Hence, the focus of most works is to extract features in energy usage patterns that help improve the proposed model's performance in terms of the model’s complexity, training time, and forecasting accuracy [2]. The feature selection and extraction process has become a hot topic among authors globally, which is selecting or creating representative variables manually or automatically that are relevant to an output feature and perfect to use for building a forecasting model [3]. Yearly and seasonal models were proposed in earlier work of I. Koprinska [4] for very short-term load forecasting. The authors tested four machine learning algorithms comparing their performance using seasonality in the input data. Proposed models achieved better results. However, it can be noted as a drawback that benchmarking was performed on the data that has a strong obvious seasonality pattern. Lagged time series data is selected using autocorrelation to the former time series and 50 input features were created from the historical power data. Thus the models became very complex and the training parameters were too much. A mutual information-based feature selection technique was successfully deployed in [5] where mutual dependence of the input feature and the output value is measured. In addition, the author used recursive feature elimination, and elastic net to automate the feature selection procedure. A recent work by [6] automatic feature extraction methods for forecasting solar radiation to estimate renewable energy source output. Empirical mode decomposition was applied to decompose solar radiation data and the continuous wavelet transform technique was used to reconstruct the input signal and deep feature extraction. The RRelief algorithm was performed to rank the importance of created features. A convolutional neural network (CNN) is applied as an automatic feature extraction layer in many works. In [7] historical load, outside temperature, cooling load, and gas consumption were used as input data. CNN attention model ran over the input data to automatically create new features as load impact factors. The proposed model was compared against support vector regressor, random forest regressor, CNN integrated RNN models, and backpropagation neural network. Motivating by the above-mentioned methods we propose a manual feature extraction technique applied before RNN-based models. The capability of the proposed method has been compared against 1D-CNN, CNN-integrated RNN models, and simple RNN-based models. Input data analysis and preprocessing. Data preprocessing is vital for finding valuable features and relationships between input patterns and output values. Figure 1. Scaled power and temperature data. “Развитие науки и технологий” Научно–технический журнал №2/2025 182 Информатика ва ахборот –коммуникацион тизимлар Methods for finding hidden features and patterns allow proper modeling of long and shortterm changes. In this research work, we explored power consumption and weather data of the Andijan region, Uzbekistan. Power data consists of aggregate residential electricity consumption with a daily sample over five years. Weather data has a rich set of climatic information like daily minimum, maximum, and average temperature, humidity, wind speed, and direction. However, we used only the temperature feature as it summarizes the effects of other factors on power usage. Thus base models have two input features. Manually derived features were created from the average daily temperature. Cooling degree day term allows obtaining impacts of the daily temperature on the cooling or heating load. Input data has minor missing values so it should be recovered using an exponential moving average (EMA) method [8]. 𝐸𝑀𝐴 = (𝑋𝑖 − 𝐸𝑀𝐴𝑖−1) ∗ 𝐾 + 𝐸𝑀𝐴𝑖−1 , (1) here, 𝑋𝑖 is the value of 𝑖𝑡ℎ timestep, 𝐾 is the smoothing coefficient equal to 2 𝑛 + 1 , n is the length of the period under consideration. The current value of the EMA depends on the value of the EMA of the previous day. So, the initial value of EMA starts from the particular previous day, it is equal to simple moving (SMA) and is defined as follows: 𝑆𝑀𝐴 = 𝑋1 + 𝑋2 + ...+ 𝑋𝑛 𝑛 , (2) Numeric features in the cleaned data were scaled with the widely used statistical MinMax scaling method to transform it to an appropriate range (-1,1) for RNN models. Scaled historical load and temperature data are illustrated in fig. 1. 3. Proposed methodology. Deep learning methods, especially sequential models such as LSTM, Bidirectional LSTM, and GRU became popular for modeling sequential tasks. Recurrent neural networks are alternatives to feedforward networks that can memorize long sequences with their internal memory. The internal structures of LSTM and GRU [9] allow them to remember historical data in the memory. A proposed short-term load forecasting framework is shown in Figure 2. A vanishing/exploding gradient problem of vanilla RNNs is solved on LSTM, GRU, and Bidirectional LSTM networks. Thus, the proposed framework deployed them to model load using input features. Power and weather data are first cleaned and concatenated. A new feature was manually derived from the temperature data. Figure 2.Energy consumption forecasting framework based on sequential models. As this work is devoted to predicting residential power usage, the main part of the electrical energy is consumed by climatic appliances in households. Hence, the degree day term is incorporated to represent the effect of the outside weather temperature on the heating and cooling load. Spinoni et. al. studied the climatic change in Europe from 1982 to 2011 and observed movement in cooling, heating, and an average degree in the continent [10]. “Фан ва технологиялар тараққиёти” Илмий–техникавий журнал №2/2025 183 Информатика и информационно-коммуникационные системы We adapted the degree day term to our local condition and manually created two new F1 and F2 features that are differences between daily average temperature and CDD, HDD values. We chose CDD and HDD values when they allowed us to achieve the maximum Pearson correlation value between F1, F2 and daily load (Fig. 3). New features concatenated to the load and temperature data, normalized and split to the train and test sets. RNN-based models trained on the manual feature-added data are labeled FE-LSTM, FE-GRU, and FE-BiLSTM. For comparison, RNN-based models are adapted and trained on two baseline historical power and temperature features. Another alternative, a CNN network is applied before these three RNN families as an automatic feature extractor. Finally, the last model was built on a widely used 1D-CNN algorithm. Overall, we have a total of 10 models to be compared for the best performance (Table 1). Mean average error (MAE) and mean average percentage error (MAPE) statistical methods are used to test the performance of the models. Models were built, trained and tested 10 times on the same data in order to check their robustness by analyzing probability distribution of their MAE and MAPE values. As in the previous works, the same hyper-parameters were selected for each model: Adam optimizer, mean uared error loss function, and learning rate is 0.0002. The experiments were performed using open source Keras library and Python language on Google’s standard colaboratory environment with CPU hardware. Results and analysis. All models were trained on the same five-year data and tested on the last 250-day data. The performance of the models in the regression problems is well represented using MAE and MAPE statistical metrics between actual and forecasted values [11]. These metrics show the average of the absolute and percentage error in the considered range of the data respectively. Point forecasting does not always give true conclusions about the model's certainty due to the computer's stochastic nature of the model’s parameter calculation. Recent research works prove that a probabilistic forecast provides a range of possible cases and the likelihood of each output occurring. Statistical measures that divide the range of a data distribution into quantile intervals with specified probabilities are well in analyzing the robustness of each model [12]. Figure 3. The block diagram of the proposed manual feature extraction algorithm. “Развитие науки и технологий” Научно–технический журнал №2/2025 184 Информатика ва ахборот –коммуникацион тизимлар Figure 4-5 shows the mean absolute error of each model's prediction on training and test sets. It can be seen from the figures that the performance of the RNN family not only improved but also became robust as the prediction errors are very dense around the center. Average errors decreased from around daily85-90 KWh to 42-41 KWh in the training set. Additional features allowed models to be trained more accurately and the interquartile range (IQR) of error of featureenhanced models is very close. Baseline models, 1D-CNN, and CNN-integrated models have wider IQR and very far error from the median value, representing the models' uncertainty. Table 1. Models performance on the train and test sets. Train set Test set MAE MAPE MAE MAPE Simple-LSTM 93,2 0,74 126,7 1,13 Simple-GRU 86,4 0,7 125,3 1,11 Simple-BiLSTM 83 0,63 120,7 1,05 CNN-LSTM 92,1 0,73 128,5 1,15 CNN-GRU 85,7 0,68 118,6 1,03 CNN-BiLSTM 85 0,67 119,6 1,05 1D-CNN 117,3 0,97 180 1,57 FE-LSTM 52,2 0,42 76,6 0,48 FE-GRU 52,7 0,42 76,9 0,49 FE-BiLSTM 51,5 0,41 73,8 0,43 Models Figure 4. The mean absolute error of the models on the training data set. Figure 5.The mean absolute percentage error of the models on the training data set. “Фан ва технологиялар тараққиёти” Илмий–техникавий журнал №2/2025 185 Информатика и информационно-коммуникационные системы Models showed similar capability on the test set. Manual features allowed models to generalize better from the data. MAE of the GRU and LSTM on the test data decreased from daily 123-125 KWh to 76,6-76,9 KWh. BiLSTM model showed even better results and its average MAE dropped to below 74 KWh. A convolution layer contributed more to the LSTM network compared to GRU and BiLSTM as it has a narrower error range. However, it still has three outliers pointing to its slight uncertainty. 1D-CNN has a much higher error of 180 KWh daily which is 1.57% of daily total energy usage. Manual features made the models twice as accurate and robust compared to their baseline counterparts. MAPE decreased to around 0,5% of the daily power usage in the region. Figure 6. The mean absolute error of the models on the test data set. Figure 7. The mean absolute percentage error of the models on the test data set. Conclusion. This paper proposed a manual feature extraction method for day-ahead power load forecasting using RNN-family neural networks. Real-world regional load and outside temperature data were used for model training and testing. temperature data was manually processed to extract useful features to improve the models’ robustness. Baseline RNN-based models were developed using two feature input data with power and outside temperature. New features were extracted manually from temperature data as a deviation of daily average temperature from CDD and HDD. These new features caused an improvement in the models’ prediction and confidence interval. The comparison showed that feature-enhanced models are significantly accurate and precise in prediction. It is concluded that temperature data carries more hidden information and manually extracted features can be used successfully for power forecasting. “Развитие науки и технологий” Научно–технический журнал №2/2025 186 Информатика ва ахборот –коммуникацион тизимлар