Kerasでユーザー定義した損失関数をモデルにコンパイルさせるとエラーが出てしまう。
私は現在Kerasを使って学習モデルを構築し、その学習モデルに対して独自に定義した損失を適用したいと考えています。
学習モデル自体は一度構築していて、Kerasに元々備わっている損失関数を使った場合に、学習が実行されることは既に確認しました。
しかし、その学習モデルに対して私が以下のように独自に定義した損失を適用すると以下のようなエラーが出てしまい、うまくいきません。
どうすれば下のように定義された損失関数をエラー無しで私のモデルに適用できますでしょうか?
ご教授頂けると助かります。
Kerasに元々備わっている損失関数を用いた場合のコード
model.compile(optimizer='rmsprop', loss='categorical_crossentropy')
これはfitした時に上手く学習が走った。
自分で定義した損失で学習を走らせる場合のコード
#KL-divergence annealingのコールバックの定義
from keras import backend as K
#hp_lambdaはkl_lossの係数として用意する。
hp_lambda = K.variable(0) # default values
from keras import callbacks
class AneelingCallback(callbacks.Callback): #callbacks.Callbackはhttps://keras.io/ja/callbacks/#callbackを参照したい。
'''Aneeling theano shared variable.
# Arguments
schedule(関数): a function that takes an epoch index as input
(integer, indexed from 0) and returns a new
learning rate as output (float).
'''
def __init__(self, schedule, variable):
super(AneelingCallback, self).__init__()
self.schedule = schedule
self.variable = variable #hp_lambdaにあたるもの
def on_epoch_begin(self, epoch, logs={}):
assert hasattr(self.model.optimizer, 'lr'), \
'Optimizer must have a "lr" attribute.'
value = self.schedule(epoch)
assert type(value) == float, 'The output of the "schedule" function should be float.'
K.set_value(self.variable, value) #上のvalueで得た値をvariableにセットする。
#print(K.eval(self.variable)) #後で消す行。kl_lossの係数をepochごとに知りたい...。
#epochごとにkl_lossの係数を変更する関数
def schedule(epoch):
if epoch < 11:
return 0
else:
return (epoch-10)/epochs
aneeling_callback = AneelingCallback(schedule, hp_lambda)
#コールバックの定義終わり
# 損失関数をこのモデルに加える。損失関数を定義。
def loss(inputs, time, outputs):
from keras.losses import categorical_crossentropy
reconstruction_loss = categorical_crossentropy(K.flatten(inputs), K.flatten(outputs))
#reconstruction_loss *= NUM_INPUT_DIM*NUM_TIMESTEPS
reconstruction_loss *= vector_dim
z_mean, z_log_var, _ = encoder_model([inputs,time])#関数の最初に書くべきか...?
kl_loss = 1 + z_log_var - K.square(z_mean) - K.exp(z_log_var)
kl_loss = K.sum(kl_loss, axis=-1)
kl_loss *= -aneeling_callback.variable
lam = 0.01 #そのままじゃうまく行かなかったので重み付け
return K.mean((1-lam)*reconstruction_loss + lam*kl_loss)
#損失関数の定義終わり
model.add_loss(loss(softmax_encoder_inputs, time_input, decoder_outputs))
# Run training
model.compile(optimizer='Adam')
エラー
Traceback (most recent call last):
File "pre_research3.py", line 293, in <module>
validation_split=0.2)
File "/home/firebird555/.pyenv/versions/3.6.6/lib/python3.6/site-packages/keras/engine/training.py", line 952, in fit
batch_size=batch_size)
File "/home/firebird555/.pyenv/versions/3.6.6/lib/python3.6/site-packages/keras/engine/training.py", line 789, in _standardize_user_data
exception_prefix='target')
File "/home/firebird555/.pyenv/versions/3.6.6/lib/python3.6/site-packages/keras/engine/training_utils.py", line 63, in standardize_input_data
'expected no data, but got:', data)
ValueError: ('Error when checking model target: expected no data, but got:', array([[[0., 0., 0., ..., 0., 0., 0.],
[0., 1., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]],
[[0., 0., 0., ..., 0., 0., 0.],
[0., 1., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]],
[[0., 0., 0., ..., 0., 0., 0.],
[0., 1., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]],
...,
[[0., 0., 0., ..., 0., 0., 0.],
[0., 1., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]],
[[0., 0., 0., ..., 0., 0., 0.],
[0., 1., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]],
[[0., 0., 0., ..., 0., 0., 0.],
[0., 1., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]]], dtype=float32))
使用するモデルの概要
Using TensorFlow backend.
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected
to
==================================================================================================
input_1 (InputLayer) (None, None, 203) 0
__________________________________________________________________________________________________
cu_dnnlstm_1 (CuDNNLSTM) [(None, None, 128), 170496 input_1[0][0]
__________________________________
________________________________________________________________
reshape_1 (Reshape) (None, 128, 10, None 0 cu_dnnlstm_1[0][0]
__________________________________________________________________________________________________
sequential_1 (Sequential) (None, 256) 1255232 reshape_1[0][0]
__________________________________________________________________________________________________
time_input (InputLayer) (None, 1) 0
__________________________________________________________________________________________________
concatenate_1 (Concatenate) (None, 257) 0 sequential_1[1][0]
time_input[0][0]
__________________________________________________________________________________________________
dense_2 (Dense) (None, 128) 33024 concatenate_1[0][0]
__________________________________________________________________________________________________
z_mean (Dense) (None, 128) 16512 dense_2[0][0]
__________________________________________________________________________________________________
z_log_var (Dense) (None, 128) 16512 dense_2[0][0]
__________________________________________________________________________________________________
z (Lambda) (None, 32) 0 z_mean[0][0]
z_log_var[0][0]
__________________________________________________________________________________________________
shifted_inputs (InputLayer) (None, 10, 203) 0
__________________________________________________________________________________________________
repeat_vector_1 (RepeatVector) (None, 1
0, 32) 0 z[0][0]
__________________________________________________________________________________________________
concatenate_2 (Concatenate) (None, 10, 235) 0 shifted_inputs[0][0]
repeat_vector_1[0][0]
__________________________________________________________________________________________________
cu_dnnlstm_2 (CuDNNLSTM) [(None, 10, 128), (N 186880 concatenate_2[0][0]
__________________________________________________________________________________________________
dense_3 (Dense) (None, 10, 203) 26187 cu_dnnlstm_2[0][0]
==================================================================================================
Total params: 1,704,843
Trainable params: 1,704,843
Non-trainable params: 0
__________________________________________________________________________________________________