Electric energy consumption is essential for promoting economic development and raising the standard of living. In contrast to other energy sources, electric energy cannot be stored for large-scale consumption. From an economic viewpoint, the supply and demand of electric energy must be balanced at any given time. Therefore, a precise forecasting of electric energy consumption is very important for the economic operation of an electric power grid.
The ability to create a forecasting model for an individual consumer can help determine the overall load on the grid for a given time.
For this post, we will classify this as a time series prediction problem and only use one variance. Check back for a post where introduce mutli variance.
The Dataset :
I used Smart Meters Texas or SMT portal to access my home electricity usage. SMTstores daily, monthly and even 15-minute intervals of energy data. Energy data is recorded by digital electric meters (commonly known as smart meters), and provides secure access to the data for customers and authorized market participants
In addition to acting as an interface for access to smart meter data, SMT enables secure communications with the customers in-home devices and provides a convenient and easy-to-use process where customers can voluntarily authorize market participants, other than the customer’s retail electric provider or third parties, access to their energy information and in-home devices.
Keras: Keras is an open source neural network library written in Python. It is capable of running on top of TensorFlow, Microsoft Cognitive Toolkit, Theano, or PlaidML. Designed to enable fast experimentation with deep neural networks, it focuses on being user-friendly, modular, and extensible
TensorFlow : TensorFlow is an open-source software library for dataflow programming across a range of tasks. It is a symbolic math library and is also used for machine learning applications such as neural networks
The Algorithm: LSTM
In our case, we will be using a variant of Recurrent Neural Network ( RNN ) called Long Short Term Memory ( LSTM), why? time series problems are a difficult type of prediction modeling, LSTMs are good at extracting input features patterns when it’s over a long period of time, LSTM have the capability of retaining it in memory, and use it to predict next sequences, this what my 3 months usage (in KWH) look like ( by 15 min interval ) :
for this example, I will only use a subset of the overall dataset, 3 days of electricity usage:
loading the dataset:
# load the dataset
dataframe = read_csv(‘AMS.csv’, usecols=, engine=’python’)
dataset = dataframe.values
dataset = dataset.astype(‘float32’)
Training Set and Test Set :
train_size = int(len(dataset) * 0.67)
test_size = len(dataset) – train_size
train, test = dataset[0:train_size,:], dataset[train_size:len(dataset),:]
model = Sequential()
model.add(LSTM(4, input_shape=(1, look_back))) model.add(Dense(1)) model.add(Activation(‘tanh’)) model.compile(loss=’mean_squared_error’, optimizer=’adam’) model.fit(trainX, trainY, epochs=50, batch_size=1) [/code]
trainPredict = model.predict(trainX)
testPredict = model.predict(testX) [/code]
for plotting we shift the training and test sets :
# shift train predictions for plotting
trainPredictPlot = numpy.empty_like(dataset)
trainPredictPlot[:, :] = numpy.nan
trainPredictPlot[look_back:len(trainPredict)+look_back, :] = trainPredict
# shift test predictions for plotting
testPredictPlot = numpy.empty_like(dataset)
testPredictPlot[:, :] = numpy.nan
testPredictPlot[len(trainPredict)+(look_back*2)+1:len(dataset)-1, :] = testPredict
if results were plotted, it would look something like the picture below:
Blue: actual usage
Orange: Test Set
if we zoom on the prediction part :
this is good forecasting, with a 0.19 RMSE
With the electricity market undergoing a revolution, load forecasts have gained much more significance spreading across other business departments like energy trading and financial planning. Accurate load forecasts are the basis for most reliability organizations operations, like Electricity Reliability Consul Of Texas more commonly known as ERCOT. Accurate load forecasting will become even more important with Smart Grids. Smart Grids create the opportunity to proactively take action at the consumer level, storage level, and generation side, to avoid a situation of energy scarcity and/ or price surge.
2 thoughts on “Using Smart Meter Data for consumer electricity usage forecasting”
Mohamed, where did you get access to this data? I am interested in working on forecasting using smart meter data too, but I’m having trouble finding data. Please send me an email: email@example.com. Thanks!
if you login to https://www.smartmetertexas.com/home , they will let pull all the data related to you smart meter, i’m not sure if other states have something similar