Keras中声音数据的自动编码器

Question

我有5种不同类别的声音样本的对数比例mel-spectrogram的2d阵列。

对于培训我已经在凯拉斯使用卷积和密集神经网络。下面的代码：

model = Sequential() 
model.add(Conv1D(80, 8, activation='relu', padding='same',input_shape=(60,108))) 
model.add(MaxPooling1D(2,padding='same',strides=None)) 
model.add(Flatten()) 
initializer=initializers.TruncatedNormal() 
model.add(Dense(200, activation='relu', kernel_initializer=initializer,bias_initializer=initializer)) 
model.add(BatchNormalization()) 
model.add(Dropout(0.8)) 
model.add(Dense(50, activation='relu', kernel_initializer=initializer,bias_initializer=initializer)) 
model.add(Dropout(0.8)) 
model.add(Dense(5, activation='softmax', kernel_initializer=initializer,bias_initializer=initializer)) 
model.compile(loss='categorical_crossentropy', 
      optimizer='adam',lr=0.01, 
      metrics=['accuracy']) 

我可以适用于这种类型的数据输入什么样的自动编码器的？什么型号？任何建议或代码示例都会有所帮助。 :)

Answer 1

因为我没有对数据性质问题的答案，所以我会假设我们有一组形状类似的二维数据（NSamples，68,108）。此外，我认为我的建议使用卷积二维而不是卷积1D的回答是是

这里是卷积自动编码器，模型，它可以使用训练有素的自动编码器，以及如何使用自动编码器最终模型：

from keras.layers.core import Dense, Dropout, Flatten, Reshape 
from keras.layers import Conv1D, Conv2D, Deconv2D, MaxPooling1D, MaxPooling2D, UpSampling2D, Conv2DTranspose, Flatten, BatchNormalization, Dropout 
from keras.callbacks import ModelCheckpoint 
import keras.models as models 
import keras.initializers as initializers 
from sklearn.model_selection import train_test_split 

ae = models.Sequential() 
#model.add(Conv1D(80, 8, activation='relu', padding='same',input_shape=(60,108))) 
#encoder 
c = Conv2D(80, 3, activation='relu', padding='same',input_shape=(60, 108, 1)) 
ae.add(c) 
ae.add(MaxPooling2D(pool_size=(2, 2), padding='same', strides=None)) 
ae.add(Flatten()) 
initializer=initializers.TruncatedNormal() 
d1 = Dense(200, activation='relu', kernel_initializer=initializer,bias_initializer=initializer) 
ae.add(d1) 
ae.add(BatchNormalization()) 
ae.add(Dropout(0.8)) 
d2 = Dense(50, activation='relu', kernel_initializer=initializer,bias_initializer=initializer) 
ae.add(d2) 
ae.add(Dropout(0.8)) 
#decodser 
ae.add(Dense(d2.input_shape[1], activation='sigmoid')) 
ae.add(Dense(d1.input_shape[1], activation='sigmoid')) 
ae.add(Reshape((30, 54, 80))) 
ae.add(UpSampling2D((2,2))) 
ae.add(Deconv2D(filters= c.filters, kernel_size= c.kernel_size, strides=c.strides, activation=c.activation, padding=c.padding,)) 
ae.add(Deconv2D(filters= 1, kernel_size= c.kernel_size, strides=c.strides, activation=c.activation, padding=c.padding,)) 
ae.compile(loss='binary_crossentropy', 
optimizer='adam',lr=0.001, 
metrics=['accuracy']) 
ae.summary() 
#now train your convolutional autoencoder to reconstruct your input data 
#reshape your data to (NSamples, 60, 108, 1) 
#Then train your autoencoder. it can be something like that: 
#X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=43) 
#pre_mcp = ModelCheckpoint("CAE.hdf5", monitor='val_accuracy', verbose=2, save_best_only=True, mode='max') 
#pre_history = ae.fit(X_train, X_train, epochs=100, validation_data=(X_val, X_val), batch_size=22, verbose=2, callbacks=[pre_mcp]) 

#model 
model = models.Sequential() 
#model.add(Conv1D(80, 8, activation='relu', padding='same',input_shape=(60,108))) 
model.add(Conv2D(80, 3, activation='relu', padding='same',input_shape=(60, 108, 1))) 
model.add(MaxPooling2D(pool_size=(2, 2), padding='same',strides=None)) 
model.add(Flatten()) 
initializer=initializers.TruncatedNormal() 
model.add(Dense(200, activation='relu', kernel_initializer=initializer,bias_initializer=initializer)) 
model.add(BatchNormalization()) 
model.add(Dropout(0.8)) 
model.add(Dense(50, activation='relu', kernel_initializer=initializer,bias_initializer=initializer)) 
model.add(Dropout(0.8)) 
model.add(Dense(5, activation='softmax', kernel_initializer=initializer,bias_initializer=initializer)) 
model.compile(loss='categorical_crossentropy', 
optimizer='adam',lr=0.001, 
metrics=['accuracy']) 
#Set weights    
model.layers[0].set_weights(ae.layers[0].get_weights())  
model.layers[3].set_weights(ae.layers[3].get_weights()) 
model.layers[4].set_weights(ae.layers[4].get_weights()) 
model.layers[6].set_weights(ae.layers[6].get_weights()) 
model.summary() 
#Now you can train your model with pre-trained weights from autoencoder 

像这样的模型是有用的，我有MNIST数据集，并改善与自动编码器的初始权重模型的精度与随机权初始化

但是与模型相比较，我会建议使用几个卷积/去卷积层，可能是3或者莫因为根据我的经验，具有3个或更多卷积层的卷积自动编码器比1卷积层更有效。事实上，一个卷积层我甚至不能看到任何改善的准确性有时

更新：

我检查自动编码器通过的Emanuela提供的数据，我也有不同的自动编码器架构检查它没有任何成功

我有关的假设是，该数据不包含任何显著的特点，它可以通过自动编码器，甚至CAE来区分

但是，它看起来像我的假设数据的2维性质由几乎达到99.99％，验证准确的证实：

然而，在同一时间，训练数据的97.31％的准确率可以表明与数据集的潜在问题，所以它看起来像一个好主意，修改它

另外，我会建议使用网络的合奏。你可以训练，例如10个网络具有不同的验证数据和最投票类别

分配类别项目在这里是我的代码：

from keras.layers.core import Dense, Dropout, Flatten 
from keras.layers import Conv2D, BatchNormalization 
from keras.callbacks import ModelCheckpoint 
from keras.optimizers import Adam 
from sklearn.model_selection import train_test_split 
import keras.models as models 
import keras.initializers as initializers 
import msgpack 
import numpy as np 

with open('SoundDataX.msg', "rb") as fx,open('SoundDataY.msg', "rb") as fy: 
    dataX=msgpack.load(fx) 
    dataY=msgpack.load(fy) 

num_samples = len(dataX) 
x = np.empty((num_samples, 60, 108, 1), dtype = np.float32) 
y = np.empty((num_samples, 4), dtype = np.float32) 

for i in range(0, num_samples): 
    x[i] = np.asanyarray(dataX[i]).reshape(60, 108, 1) 
    y[i] = np.asanyarray(dataY[i]) 

X_train, X_val, y_train, y_val = train_test_split(x, y, test_size=0.2, random_state=43) 

#model 
model = models.Sequential() 
model.add(Conv2D(128, 3, activation='relu', padding='same',input_shape=(60, 108, 1))) 
model.add(Conv2D(128, 5, activation='relu', padding='same',input_shape=(60, 108, 1))) 
model.add(Conv2D(128, 7, activation='relu', padding='same',input_shape=(60, 108, 1))) 
model.add(Flatten()) 
initializer=initializers.TruncatedNormal() 
model.add(Dense(200, activation='relu', kernel_initializer=initializer,bias_initializer=initializer)) 
model.add(BatchNormalization()) 
model.add(Dropout(0.8)) 
model.add(Dense(50, activation='relu', kernel_initializer=initializer,bias_initializer=initializer)) 
model.add(Dropout(0.8)) 
model.add(Dense(4, activation='softmax', kernel_initializer=initializer,bias_initializer=initializer)) 
model.compile(loss='categorical_crossentropy', 
optimizer=Adam(lr=0.0001), 
metrics=['accuracy']) 
model.summary() 
filepath="weights-{epoch:02d}-{val_acc:.7f}-{acc:.7f}.hdf5" 
mcp = ModelCheckpoint(filepath, monitor='val_acc', verbose=2, save_best_only=True, mode='max') 
history = model.fit(X_train, y_train, epochs=100, validation_data=(X_val, y_val), batch_size=64, verbose=2, callbacks=[mcp])