Transfer Learning Summary

本文对迁移学习在机器视觉中的实践技巧进行汇总整理 … …

@toc

Feature Extraction

在这种迁移学习模式中,预训练模型将被当作特征提取器(feature extractor),获得图像的特征表示(feature vactor)。获得特征向量之后,我们只需一个简单的分类器模型,如SVM、逻辑回归分类器、随机森林就可以完成目标分类器的设计。

VGG16倒数第二层(参数层)的输出维度为: 7x7x512 = 25,088

HDF5

抽取特征的高效存储可以选用HDF5。Hierarchical Data Format(HDF)是一种针对大量数据进行组织和存储的文件格式。经历了20多年的发展,HDF格式的最新版本是HDF5,它包含了数据模型,库,和文件格式标准。以其便捷有效,移植性强,灵活可扩展的特点受到了广泛的关注和应用。很多大型机构的数据存储格式都采用了HDF5,比如NASA的地球观测系统,MATLAB的.m文件,流体细算软件CDF,都将HDF5作为标准数据格式。HDF5本身用C实现,可以使用python的库h5py对HDF5文件进行操作。可以像操作Numpy数组一样对大型数据进行操作,比如切片,按行读取

数据在HDF5中采取分层存储方式,很像文件系统管理方式,第一级叫做组,类似于container,每个组中可以创建新的组或数据集,每一个dataset包含两部分的数据,Metadata和Data。其中Metadata包含Data相关的信息,而Data则包含数据本身。

HDF5文件存储方式

1
2
3
4
5
6
7
8
9
10
import h5py
p = "./datasets/hdf5/features.hdf5"

db = h5py.File(p)
list(db.keys())
>>> [u’features’, u’label_names’, u’labels’]
db["features"].shape
>>> (3000, 25088)
list(db["label_name"])
>>> ['cat', 'dogs', 'panda']

Fine-Tune

  • 一般而言,fine-tune在样本数据足够的情况下训练效果优于特征抽取;

    fine_tune

学习率要控制的尽量小

如何取层(Keras)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
from keras.applications import VGG16
model = VGG16(weights="imagenet", include_top=False)
model.layers
>>>
[<keras.engine.topology.InputLayer at 0x7ff06cc3b518>,
<keras.layers.convolutional.Conv2D at 0x7ff06cc3b7f0>,
<keras.layers.convolutional.Conv2D at 0x7ff06cc3b7b8>,
<keras.layers.pooling.MaxPooling2D at 0x7ff06cc3bd30>,
<keras.layers.convolutional.Conv2D at 0x7ff06cdb3b00>,
<keras.layers.convolutional.Conv2D at 0x7ff06cdb3748>,
<keras.layers.pooling.MaxPooling2D at 0x7ff06cbbb400>,
<keras.layers.convolutional.Conv2D at 0x7ff06cbcc630>,
<keras.layers.convolutional.Conv2D at 0x7ff06cb5dd68>,
<keras.layers.convolutional.Conv2D at 0x7ff06cb6d208>,
<keras.layers.pooling.MaxPooling2D at 0x7ff06cb7fdd8>,
<keras.layers.convolutional.Conv2D at 0x7ff06cb30fd0>,
<keras.layers.convolutional.Conv2D at 0x7ff06cb30e80>,
<keras.layers.convolutional.Conv2D at 0x7ff06cb52978>,
<keras.layers.pooling.MaxPooling2D at 0x7ff06caf6470>,
<keras.layers.convolutional.Conv2D at 0x7ff06cb076a0>,
<keras.layers.convolutional.Conv2D at 0x7ff06ca98dd8>,
<keras.layers.convolutional.Conv2D at 0x7ff06caaa278>,
<keras.layers.pooling.MaxPooling2D at 0x7ff06cab8e48>]

model.output
>>>
<tf.Tensor 'block5_pool/MaxPool:0' shape=(?, ?, ?, 512) dtype=float32>

如何添加层(Keras)

1
2
3
4
5
6
7
8
9
10
11
12
13
# load the VGG16 network, ensuring the head FC layer sets are left # off
baseModel = VGG16(weights="imagenet", include_top=False, input_tensor=Input(shape=(224, 224, 3)))
# add new layers
headModel = baseModel.output
headModel = Flatten(name="flatten")(headModel)
headModel = Dense(D_num, activation="relu")(headModel)
headModel = Dropout(0.5)(headModel)
headModel = Dense(classes_num, activation="softmax")(headModel)

model = Model(inputs=baseModel.input, outputs=headModel)
#freeze baseModel layers
for layer in baseModel.layers:
layer.trainable = False
  1. Typically you’ll allow your own FC head to warmup for 10-30 epochs, depending on your dataset.使用RMSprop作为优化算法;​

    1
    2
    3
    4
    5
    6
    opt = RMSprop(lr=0.001)
    model.compile(loss="categorical_crossentropy", optimizer=opt, metrics=["accuracy"])
    # train the head of the network for a few epochs (all other # layers are frozen) -- this will allow the new FC layers to
    # start to become initialized with actual "learned" values # versus pure random
    print("[INFO] training head...")
    model.fit_generator(aug.flow(trainX, trainY, batch_size=32), validation_data=(testX, testY), epochs=25, steps_per_epoch=len(trainX) // 32, verbose=1)
    1. 然后可以适当往前解冻一些层,重新训练,一般会解冻最后一层CONV,使用SGD(lr=0.001)作为优化算法;

      1
      2
      3
      # now that the head FC layers have been trained/initialized, lets # unfreeze the final set of CONV layers and make them trainable 
      for layer in baseModel.layers[15:]:
      layer.trainable = True
      1
      2
      3
      4
      5
      6
      7
      # for the changes to the model to take affect we need to recompile # the model, this time using SGD with a *very* small learning rate 
      print("[INFO] re-compiling model...")
      opt = SGD(lr=0.001)
      model.compile(loss="categorical_crossentropy", optimizer=opt, metrics=["accuracy"])
      # train the model again, this time fine-tuning *both* the final set # of CONV layers along with our set of FC layers
      print("[INFO] fine-tuning model...")
      model.fit_generator(aug.flow(trainX, trainY, batch_size=32), validation_data=(testX, testY), epochs=100, steps_per_epoch=len(trainX) // 32, verbose=1)

迁移学习的选择

主要由样本数据量以及训练目标于原目标之间的相似程度决定。

数据规模 相似数据分布 不同数据分布
小样本数据集 特征提取:FC+分类器 特征提取:低层次的Conv+分类器
大样本数据集 Fine-Tune 从头训练新的网络模型
Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now

×