Regularisers em Keras comparada Caffe

votos
1

Eu editei um AlexNet em Keras, tentando aprender uma única classe (ou seja, cães / rostos ou não cão / faces). Tenho amostras de treinamento de corrigir imagens para o Ytrain é [1, 0] , e incorretas imagens de modo a Ytrain é [0,1] . O objetivo é saber se a imagem é um cão (por exemplo).

enquanto a formação a perda foi muito grande ~ 100.000.000

por isso eu deletei as linhas regularisers e funcionou ...

activity_regularizer=ActivityRegularizer(l1=1, l2=1),
W_regularizer=WeightRegularizer(l1=2.0,l2=0.0)))

Em Caffe o modelo funciona bem com estas linhas nas camadas de convolução:

param {
  lr_mult: 1
  decay_mult: 1
}
param {
  lr_mult: 2
  decay_mult: 0
}

Aqui é o modelo em Keras (imagem em tamanho 3,68,56):

    Xtrain = self.Xtrain
    Ytrain = self.Ytrain

    batch_size = 10
    nb_classes = 2
    nb_epoch = 3

    # input image dimensions
    img_rows, img_cols = np.shape(self.Xtrain)[2], np.shape(self.Xtrain)[3]
    #########################
    ###   AlexNet - Start  ##
    #########################
    print(Defining AlexNet...)

    model = Sequential()
    model.add(ZeroPadding2D((1,1),input_shape=(3,img_rows,img_cols)))
    model.add(Convolution2D(96, 11, 11, border_mode='valid',
                            init='glorot_normal',
                            activation='relu',
                            activity_regularizer=ActivityRegularizer(l1=1, l2=1),
                            W_regularizer=WeightRegularizer(l1=2.0,l2=0.0)))
    model.add(BatchNormalization())
    model.add(MaxPooling2D(pool_size=(3, 3),strides=(2,2)))

    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(256, 5, 5, border_mode='valid',
                            init='glorot_normal',
                            activation='relu',
                            activity_regularizer=ActivityRegularizer(l1=1, l2=1),
                            W_regularizer=WeightRegularizer(l1=2,l2=0)))
    model.add(BatchNormalization())
    model.add(MaxPooling2D(pool_size=(3, 3),strides=(2,2)))

    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(384, 3, 3, border_mode='valid',
                            init='glorot_normal',
                            activation='relu',
                            activity_regularizer=ActivityRegularizer(l1=1, l2=1),
                            W_regularizer=WeightRegularizer(l1=2,l2=0)))

    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(384, 3, 3, border_mode='valid',
                            init='glorot_normal',
                            activation='relu',
                            activity_regularizer=ActivityRegularizer(l1=1, l2=1),
                            W_regularizer=WeightRegularizer(l1=2,l2=0)))

    model.add(ZeroPadding2D((1,1)))
    model.add(Convolution2D(256, 3, 3, border_mode='valid',
                            init='glorot_normal',
                            activation='relu',
                            activity_regularizer=ActivityRegularizer(l1=1, l2=1),
                            W_regularizer=WeightRegularizer(l1=2,l2=0)))
    model.add(MaxPooling2D(pool_size=(3, 3),strides=(2,2)))

    model.add(Flatten())

    model.add(Dense(4096,
                    activation='relu',
                    init='glorot_normal',
                    activity_regularizer=ActivityRegularizer(l1=1, l2=1),
                    W_regularizer=WeightRegularizer(l1=2,l2=0)))
    model.add(Dropout(0.5))

    model.add(Dense(4096,
                    activation='relu',
                    init='glorot_normal',
                    activity_regularizer=ActivityRegularizer(l1=1, l2=1),
                    W_regularizer=WeightRegularizer(l1=2,l2=0)))
    model.add(Dropout(0.5))


    model.add(Dense(nb_classes,
                    activation='softmax',
                    activity_regularizer=ActivityRegularizer(l1=1, l2=1),
                    W_regularizer=WeightRegularizer(l1=2,l2=0)))

    #######################
    ###   AlexNet - End  ##
    #######################

    add = Adadelta(lr=0.1, rho=0.95, epsilon=1e-06, decay=0.995)
    print(Compiling AlexNet...)
    model.compile(loss='categorical_crossentropy', optimizer=add)

    print(Fitting AlexNet...)
    model.fit(Xtrain, Ytrain, batch_size=batch_size, nb_epoch=nb_epoch,
      show_accuracy=True, verbose=1, shuffle=True) 

Y é que ??

Publicado 02/02/2016 em 13:40
fonte usuário
Em outras línguas...                            

Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more