浅谈归一化对于LSTM进行时间序列预测的影响（附归一化代码）

首先介绍一下我们的示例数据，此数据是1949 到 1960 一共 12 年，每年 12 个月的航班乘客数据，一共 144 个数据，单位是 1000。
下载地址

数据如图所示

我们选取前60%作为训练数据，后40%作为测试数据

#头文件
import numpy as np
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
import  pandas as pd
import  os
from keras.models import Sequential, load_model

#加载数据
alldata = pd.read_csv("./international-airline-passengers.csv")
#只取数值列
alldata = alldata.iloc[:,1]
alldata = np.array(alldata,dtype='float64')
traindata = alldata[:int(len(alldata)*0.6)]
testdata = alldata[int(len(alldata)*0.6):]

LSTM进行预测的时候要对于数据进行转换，具体可以见我之前的博客
简单粗暴LSTM：LSTM进行时间序列预测
实验中所用到的函数

#转换函数
def Create_dataset(dataset,look_back):
    data_X, data_Y = [], []
    for i in range(len(dataset) - look_back - 1):
        a = dataset[i:(i + look_back)]
        data_X.append(a)
        data_Y.append(dataset[i + look_back])
    data_X = np.array(data_X)
    data_Y = np.array(data_Y)
    return  data_X,data_Y
    
#单维最大最小归一化和反归一化函数
def Normalize(list):
    list = np.array(list)
    low, high = np.percentile(list, [0, 100])
    delta = high - low
    if delta != 0:
        for i in range(0, len(list)):
            list[i] = (list[i]-low)/delta
    return  list,low,high

def FNoramlize(list,low,high):
    delta = high - low
    if delta != 0:
        for i in range(0, len(list)):
            list[i] = list[i]*delta + low
    return list

#LSTM进行训练的函数
def Train_Model(train_X,train_Y):
    model = Sequential()
    model.add(LSTM(4, input_shape=(train_X.shape[1],train_X.shape[2])))
    #Dense层的1是预测维度为1
    model.add(Dense(1))
    model.compile(loss='mean_squared_error', optimizer='adam')
    model.fit(train_X, train_Y, epochs=1000, batch_size=1, verbose=2)
    # model.save(os.path.join("DATA","LSTMBLog" + ".h5"))
    return model

---

进行预测实验

实验分为三种情况：
1.对于训练数据和测试数据分别进行归一化
2.对于训练数据进行归一化之后。使用训练数据的最大最小值（训练数据的范围）对于测试数据进行归一化
3.对于训练和测试的整体数据统一进行归一化

1.对于训练数据和测试数据分别进行归一化

#实验1
train_n,train_low,train_high = Normalize(traindata)
test_n,test_low,test_high = Normalize(testdata)
print(train_n,test_n)
#前一个值预测后一个值
train_X,train_Y = Create_dataset(train_n,look_back=1)
test_X,test_Y = Create_dataset(test_n,look_back=1)
#额外添加一个维度使train_X，test_X变为三维
train_X = np.reshape(train_X, (train_X.shape[0], train_X.shape[1], 1))
test_X = np.reshape(test_X, (test_X.shape[0], test_X.shape[1], 1))

model = Train_Model(train_X,train_Y)

train_predict = model.predict(train_X)
test_predict  = model.predict(test_X)

#反归一化
train_Y = FNoramlize(train_Y,train_low,train_high)
train_predict = FNoramlize(train_predict,train_low,train_high)
test_Y = FNoramlize(test_Y,test_low,test_high)
test_predict = FNoramlize(test_predict,test_low,test_high)

#进行绘图
plt.subplot(121)
plt.plot(train_Y)
plt.plot(train_predict)
plt.subplot(122)
plt.plot(test_Y)
plt.plot(test_predict)
plt.show()

结果如图所示：可以看到LSTM对于训练数据和预测数据都能很好的拟合

2.对于训练数据进行归一化之后。使用训练数据的最大最小值（训练数据的范围）对于测试数据进行归一化

引入新的归一化函数

def Normalize2(list,low,high):
    list = np.array(list)
    delta = high - low
    if delta != 0:
        for i in range(0, len(list)):
            list[i] = (list[i]-low)/delta
    return  list

进行实验

#实验2
train_n,train_low,train_high = Normalize(traindata)
#更新的归一化
test_n = Normalize2(testdata,train_low,train_high)
print(train_n,test_n)
#前一个值预测后一个值
train_X,train_Y = Create_dataset(train_n,look_back=1)
test_X,test_Y = Create_dataset(test_n,look_back=1)
#额外添加一个维度使train_X，test_X变为三维
train_X = np.reshape(train_X, (train_X.shape[0], train_X.shape[1], 1))
test_X = np.reshape(test_X, (test_X.shape[0], test_X.shape[1], 1))

model = Train_Model(train_X,train_Y)

train_predict = model.predict(train_X)
test_predict  = model.predict(test_X)

#反归一化
train_Y = FNoramlize(train_Y,train_low,train_high)
train_predict = FNoramlize(train_predict,train_low,train_high)
test_Y = FNoramlize(test_Y,train_low,train_high)
test_predict = FNoramlize(test_predict,train_low,train_high)

#进行绘图
plt.subplot(121)
plt.plot(train_Y)
plt.plot(train_predict)
plt.subplot(122)
plt.plot(test_Y)
plt.plot(test_predict)
plt.show()

这时预测结果如图

可以看到对于test数据预测结果发生了变化，周期没有改变，但是值发生了改变

3.对于训练和测试的整体数据统一进行归一化

#实验3
alldata_n,all_low,all_high = Normalize(alldata)
#也可以直接对alldata进行截取
train_n = Normalize2(traindata,all_low,all_high)
test_n = Normalize2(testdata,all_low,all_high)
print(train_n,test_n)

#前一个值预测后一个值
train_X,train_Y = Create_dataset(train_n,look_back=1)
test_X,test_Y = Create_dataset(test_n,look_back=1)
#额外添加一个维度使train_X，test_X变为三维
train_X = np.reshape(train_X, (train_X.shape[0], train_X.shape[1], 1))
test_X = np.reshape(test_X, (test_X.shape[0], test_X.shape[1], 1))

model = Train_Model(train_X,train_Y)

train_predict = model.predict(train_X)
test_predict  = model.predict(test_X)

#反归一化
train_Y = FNoramlize(train_Y,all_low,all_high)
train_predict = FNoramlize(train_predict,all_low,all_high)
test_Y = FNoramlize(test_Y,all_low,all_high)
test_predict = FNoramlize(test_predict,all_low,all_high)

#进行绘图
plt.subplot(121)
plt.plot(train_Y)
plt.plot(train_predict)
plt.subplot(122)
plt.plot(test_Y)
plt.plot(test_predict)
plt.show()

预测结果：

---

结论

可以看到LSTM进行预测的时候会保留趋势（周期）信息以及训练数据的高低（范围）信息。
其中
1.对于训练数据和测试数据分别进行归一化 保留周期信息
2.对于训练数据进行归一化之后。使用训练数据的最大最小值（训练数据的范围）对于测试数据进行归一化 保留周期和范围信息
3.对于训练和测试的整体数据统一进行归一化 保留周期和范围信息

在进行LSTM预测的时候，我们可以通过对于归一化的选择来进行是否进行高低信息的使用。

注：代码已上传到我的github，同时附录了单维多维归一化函数
本次实验中2.3.的预测结果也有一些区别,实验中3.的test预测结果更高一些

#这两种方式会产生不同的结果
normalize = np.arange(10,dtype='float64')
normalize = np.arange(10)
normalize.dtype = 'float64'