私は現在Kerasを使って学習モデルを構築し、その学習モデルに対して独自に定義した損失を適用したいと考えています。
学習モデル自体は一度構築していて、Kerasに元々備わっている損失関数を使った場合に、学習が実行されることは既に確認しました。
しかし、その学習モデルに対して私が以下のように独自に定義した損失を適用すると以下のようなエラーが出てしまい、うまくいきません。
どうすれば下のように定義された損失関数をエラー無しで私のモデルに適用できますでしょうか?
ご教授頂けると助かります。

Kerasに元々備わっている損失関数を用いた場合のコード

model.compile(optimizer='rmsprop', loss='categorical_crossentropy')

これはfitした時に上手く学習が走った。

自分で定義した損失で学習を走らせる場合のコード

#KL-divergence annealingのコールバックの定義
from keras import backend as K

#hp_lambdaはkl_lossの係数として用意する。
hp_lambda = K.variable(0)  # default values

from keras import callbacks

class AneelingCallback(callbacks.Callback): #callbacks.Callbackはhttps://keras.io/ja/callbacks/#callbackを参照したい。
    '''Aneeling theano shared variable.
        # Arguments
        schedule(関数): a function that takes an epoch index as input
        (integer, indexed from 0) and returns a new
        learning rate as output (float).
        '''
    def __init__(self, schedule, variable):
        super(AneelingCallback, self).__init__()
        self.schedule = schedule
        self.variable = variable #hp_lambdaにあたるもの

    def on_epoch_begin(self, epoch, logs={}):
        assert hasattr(self.model.optimizer, 'lr'), \
            'Optimizer must have a "lr" attribute.'
        value = self.schedule(epoch)
        assert type(value) == float, 'The output of the "schedule" function should be float.'
        K.set_value(self.variable, value) #上のvalueで得た値をvariableにセットする。
        #print(K.eval(self.variable)) #後で消す行。kl_lossの係数をepochごとに知りたい...。

#epochごとにkl_lossの係数を変更する関数
def schedule(epoch):
    if epoch < 11:
        return 0
    else:
        return (epoch-10)/epochs

aneeling_callback = AneelingCallback(schedule, hp_lambda)
#コールバックの定義終わり
# 損失関数をこのモデルに加える。損失関数を定義。
def loss(inputs, time, outputs):
    from keras.losses import categorical_crossentropy
    reconstruction_loss = categorical_crossentropy(K.flatten(inputs), K.flatten(outputs))
    #reconstruction_loss *= NUM_INPUT_DIM*NUM_TIMESTEPS
    reconstruction_loss *= vector_dim
    z_mean, z_log_var, _ = encoder_model([inputs,time])#関数の最初に書くべきか...?
    kl_loss = 1 + z_log_var - K.square(z_mean) - K.exp(z_log_var)
    kl_loss = K.sum(kl_loss, axis=-1)
    kl_loss *= -aneeling_callback.variable

    lam = 0.01 #そのままじゃうまく行かなかったので重み付け
    return K.mean((1-lam)*reconstruction_loss + lam*kl_loss)
#損失関数の定義終わり
model.add_loss(loss(softmax_encoder_inputs, time_input, decoder_outputs))

# Run training
model.compile(optimizer='Adam')

エラー

Traceback (most recent call last):
  File "pre_research3.py", line 293, in <module>
    validation_split=0.2)
  File "/home/firebird555/.pyenv/versions/3.6.6/lib/python3.6/site-packages/keras/engine/training.py", line 952, in fit
    batch_size=batch_size)
  File "/home/firebird555/.pyenv/versions/3.6.6/lib/python3.6/site-packages/keras/engine/training.py", line 789, in _standardize_user_data
    exception_prefix='target')
  File "/home/firebird555/.pyenv/versions/3.6.6/lib/python3.6/site-packages/keras/engine/training_utils.py", line 63, in standardize_input_data
    'expected no data, but got:', data)
ValueError: ('Error when checking model target: expected no data, but got:', array([[[0., 0., 0., ..., 0., 0., 0.],
        [0., 1., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]],
       [[0., 0., 0., ..., 0., 0., 0.],
        [0., 1., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]],
       [[0., 0., 0., ..., 0., 0., 0.],
        [0., 1., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]],
       ...,
       [[0., 0., 0., ..., 0., 0., 0.],
        [0., 1., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]],
       [[0., 0., 0., ..., 0., 0., 0.],
        [0., 1., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]],
       [[0., 0., 0., ..., 0., 0., 0.],
        [0., 1., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        ...,
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.],
        [0., 0., 0., ..., 0., 0., 0.]]], dtype=float32))

使用するモデルの概要

Using TensorFlow backend.
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected 

to                     
==================================================================================================
input_1 (InputLayer)            (None, None, 203)    0                                            
__________________________________________________________________________________________________
cu_dnnlstm_1 (CuDNNLSTM)        [(None, None, 128),  170496      input_1[0][0]                    
__________________________________

________________________________________________________________
reshape_1 (Reshape)             (None, 128, 10, None 0           cu_dnnlstm_1[0][0]               
__________________________________________________________________________________________________
sequential_1 (Sequential)       (None, 256)          1255232     reshape_1[0][0]              



__________________________________________________________________________________________________
time_input (InputLayer)         (None, 1)            0                                            
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 257)          0           sequential_1[1][0]           


                                                                 time_input[0][0]                 
__________________________________________________________________________________________________
dense_2 (Dense)                 (None, 128)          33024       concatenate_1[0][0]              
__________________________________________________________________________________________________
z_mean (Dense)                  (None, 128)          16512       dense_2[0][0]                



__________________________________________________________________________________________________
z_log_var (Dense)               (None, 128)          16512       dense_2[0][0]                    
__________________________________________________________________________________________________
z (Lambda)                      (None, 32)           0           z_mean[0][0]                 


                                                                 z_log_var[0][0]                  
__________________________________________________________________________________________________
shifted_inputs (InputLayer)     (None, 10, 203)      0                                            
__________________________________________________________________________________________________
repeat_vector_1 (RepeatVector)  (None, 1

0, 32)       0           z[0][0]                          
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 10, 235)      0           shifted_inputs[0][0]             
                                                                 repeat_vector_1[0][0]        



__________________________________________________________________________________________________
cu_dnnlstm_2 (CuDNNLSTM)        [(None, 10, 128), (N 186880      concatenate_2[0][0]              
__________________________________________________________________________________________________
dense_3 (Dense)                 (None, 10, 203)      26187       cu_dnnlstm_2[0][0]               
==================================================================================================
Total params: 1,704,843
Trainable params: 1,704,843
Non-trainable params: 0

__________________________________________________________________________________________________