A GANs Based on Sample Feature Decoding Constraint
-
摘要: 生成式对抗网络(Generative Adversarial Networks, GANs)是一种有效模拟训练数据分布的生成模型, 其训练的常见问题之一是优化JS散度(Jensen-Shannon divergence)时可能产生梯度消失问题(Vanishing gradient problem). 针对该问题, 本文提出了一种解码约束条件下的GANs, 以尽量避免JS散度近似为常数而引发梯度消失现象, 从而提高生成图像的质量. 首先利用U-Net结构的自动编码机学习出与用于激发生成器的随机向量同维度的训练样本网络中间层特征. 然后在每次对抗训练前使用本文设计的解码约束条件训练解码器. 其中, 解码器与生成器结构相同, 权重共享. 为证明模型的可行性, 推导给出了引入解码约束条件有利于JS散度不为常数的结论以及解码损失函数的类型选择依据. 为验证模型的性能, 利用CELEBA和CIFAR10数据集, 对比分析了DCGANs, LSGANs, BEGANs, WGANs, WGANsGP 及SAGANs的生成效果. 通过实验对比IS, FID和清晰度等指标发现, 本文GANs能有效提高图像生成质量, 综合性能接近SAGANs.Abstract: Generative adversarial networks (GANs) is a generative model for effectively simulating the distribution of training data. One of the common problems in training GANs is the possible vanishing gradient problem while optimizing Jensen-Shannon divergence. Aiming at the problem, a GANs under decoding constraint is proposed to avoid JS divergence approximating a constant, thus imporving the quality of generated images. Firstly, an auto-encoder structured under U-Net is utilized to learn the training sample network middle layer feature the same dimension as the random vector used for triggering generative network. Then, the decoding constraint is designed, which share the same structure and weights as that of the generative network, is used to train decoder before each adversarial training. To prove the feasibility of model, the conclusion is deduced that introducing decoding constraint is beneficial to avoid JS divergence approximating a constant and the type selection basis of decoding loss function is given. To verify the performance of the model, CELEBA and CIFAR10 dataset are used to compare and analyze the generated results of DCGANs, LSGANs, BEGANs, WGANs, WGANsGP and SAGANs. By comparing IS, FID, clarity and other index via experiment, it's discovered that the novel GANs can improve the quality of generated images, comperhensive performance close to SAGANs.
-
图 7 L2解码不限制权重实验样本, celeba
Fig. 7 L2 decoding with not restrict weight experimental samples, celeba
图 10 L2解码不限制权重实验样本, cifar10
Fig. 10 L2 decoding with not restrict weight experimental samples, cifar10
表 1 原图像与重构图像的PSNR和SSIM值统计
Table 1 PSNR & SSIM between original and reconstructed images
数据集 指标 均值 标准差 极小值 极大值 celeba PSNR 40.588 5.558 22.990 61.158 SSIM 0.9984 0.0023 0.9218 1.0000 cifar10 PSNR 46.219 6.117 28.189 66.779 SSIM 0.9993 0.0019 0.8180 1.0000 
下载: 导出CSV
表 2 celeba中不同解码实验结果
Table 2 Results of different decoding experiments in celeba
对比项 IS ( $ \sigma \times 0.01 $ )FID 清晰度均值 清晰度均值差值 训练集 2.71±2.48 0.00 107.88 0.00 正态特征 1.88±1.25 42.54 121.40 13.52 均匀特征 1.82±1.48 43.04 123.02 15.14 L1 1.99±1.53 32.95 120.16 12.28 L2* 1.69±0.97 46.08 96.88 11.00 L2(本文) 2.05±1.84 25.62 114.95 7.07 黑体表示最优值, 下同
下载: 导出CSV
表 3 cifar10中不同解码实验结果
Table 3 Results of different decoding experiments in cifar10
对比项 IS ( $ \sigma \times 0.1 $ )FID 清晰度均值 清晰度均值差值 训练集 10.70±1.47 0.00 120.56 0.00 正态特征 5.63±0.64 48.21 139.88 19.32 均匀特征 5.51±0.79 46.57 137.13 16.57 L1 5.63±0.79 42.70 138.04 17.48 L2* 4.69±0.55 79.10 119.62 0.94 L2(本文) 5.83±0.70 38.79 134.97 14.41
下载: 导出CSV
表 4 时间代价测试
Table 4 Test of time cost
数据集 模型 epoch数 总耗时/s 单位耗时/s celeba DCGANs 25 3,616.03 180.80 本文 15 2,868.33 191.22 cifar10 DCGANs 25 2,388.53 119.46 本文 15 1,859.51 123.96
下载: 导出CSV
表 5 celeba中不同GANs对比
Table 5 Comparsion of different GANs in celeba
GANs模型 epoch数 优化项 参数量( $ \times 10^6 $ )IS( $ \sigma \times 0.01 $ )FID 清晰度均值 清晰度均值差值 训练集 – – – 2.71±2.48 0.00 107.88 0.00 BEGANs 35 W距离 4.47 1.74±1.29 46.24 77.58 30.3 DCGANs 20 JS散度 9.45 1.87±1.58 50.11 124.82 16.94 LSGANs 35 Pearson散度 9.45 2.02±1.63 39.11 122.19 14.31 WGANs 35 W距离 9.45 2.03±1.75 40.31 117.15 9.27 WGANsGP 35 W距离 9.45 1.98±1.82 37.01 121.16 13.28 SAGANs1 30 W距离 10.98 2.06±1.79 21.94 109.94 2.06 SAGANs2 30 JS散度 10.98 1.99±1.79 31.04 99.57 8.31 本文 15 JS+ $ \lambda \cdot $ KL散度9.45+0.84 2.05±1.84 25.62 114.95 7.07
下载: 导出CSV
表 6 cifar10中不同GANs对比
Table 6 Comparsion of different GANs in cifar10
GANs模型 epoch数 优化项 参数量( $ \times 10^6 $ )IS( $ \sigma \times 0.1 $ )FID 清晰度均值 清晰度均值差值 训练集 – – – 10.70±1.47 0.00 120.56 0.00 BEGANs 35 W距离 3.67 5.36±0.65 107.64 80.89 39.67 DCGANs 20 JS散度 8.83 5.04±0.27 54.27 139.12 18.56 LSGANs 35 Pearson散度 8.83 5.70±0.36 43.35 135.80 15.24 WGANs 35 W距离 8.83 5.25±0.33 53.88 136.74 16.18 WGANsGP 35 W距离 8.83 5.39±0.30 50.60 139.17 18.61 SAGANs1 30 W距离 8.57 6.09±0.47 42.90 126.28 5.72 SAGANs2 30 JS散度 8.57 5.37±0.46 53.49 133.54 12.98 本文 15 JS+ $ \lambda \cdot $ KL散度8.83+0.23 5.83±0.70 38.79 134.97 14.41
下载: 导出CSV
-
[1] Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S et al. Generative adversarial nets. In: Proceedings of International Conference on Neural Information Processing Systems. Montreal, Canada: 2014. 2672−2680 [2] Mirza M, Osindero S. Conditional generative adversarial nets. arXiv preprint arXiv: 1411.1784v1, 2014 [3] Odena A. Semi-Supervised Learning with Generative adversarial networks. arXiv preprint arXiv: 1606.01583v2, 2016 [4] Odena A, Olah C, Shlens J. Conditional image synthesis with auxiliary classifier GANs. In: Proceedings of International Conference on Machine Learning. Sydney, Australia: 2017 [5] Chen X, Duan Y, Houthooft R, Schulman J, Sutskever I, Abbeel P. InfoGAN: interpretable representation learning by information maximizing generative adversarial nets. In: Proceedings of International Conference on Neural Information Processing Systems. Barcelona, Spain: 2016 [6] Donahue J, Krahenbuhl K, Darrell T. Adversarial feature learning. In: Proceedings of International Conference on Learning Representations. Toulon, France: 2017 [7] Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. In: Proceedings of International Conference on Learning Representations. San Juan, Puerto Rico: 2016 [8] Denton E, Chintala S, Szlam A, Fergus R. Deep generative image using a laplacian pyramid of adversarial networks. In: Proceedings of International Conference on Neural Information Processing Systems. Montreal, Canada: 2015.1486−1494 [9] Brock A, Donahue J, Simonyan K. Large scale GAN training for high fidelity natural image synthesis. In: Proceedings of International Conference on Learning Representations. New Orleans, USA: 2019 [10] Nguyen T D, Le T, Vu H, Phung D. Dual discriminator generative adversarial nets. In: Proceedings of Proceedings of International Conference on Neural Information Processing Systems. San Francisco, USA: 2017 [11] 张龙, 赵杰煜, 叶绪伦, 董伟. 协作式生成对抗网络. 自动化学报, 2018, 44(5): 804−810Zhang Long, Zhao Jie-Yu, Ye Xu-Lun, Dong Wei. Cooperative generative adversarial nets. Acta Automatica Sinica, 2018, 44(5): 804−810 [12] Arjovsky M, Chintala S, Bottou L. Wasserstein GAN. arXiv preprint arXiv: 1701.07875v3, 2017 [13] Mao X D, Li Q, Xie H R, Lau R Y K, Wang Z, Smolley S P. Least squares generative adversarial networks. In: Proceedings of IEEE International Conference on Computer Vision. Venice, Italy: 2017. 2813−2821 [14] Berthelot D, Schumm T, Metz L. BEGAN: Boundary equilibrium generative adversarial networks. arXiv preprint arXiv: 1703.10717v4, 2017 [15] Gulrajani I, Ahmed G, Arjovsky M, Dumoulin V, Courville A. Improved training of wasserstein GANs. In: Proceedings of International Conference on Neural Information Processing Systems. Long Beach, USA: 2017 [16] Wu J Q, Huang Z W, Thoma J, Acharya D, Gool L V. Wasserstein divergence for GANs. In: Proceedings of European Conference on Computer Vision. Munich, Germany: 2018 [17] Su J L. GAN-QP: A novel GAN framework without gradient vanishing and Lipschitz constraint. arXiv preprint arXiv: 1811.07296v2, 2018 [18] Zhao J B, Mathieu M, LeCun Y. Energy-based generative adversarial networks. In: Proceedings of International Conference on Learning Representations. Toulon, France: 2017 [19] 王功明, 乔俊飞, 乔磊. 一种能量函数意义下的生成式对抗网络. 自动化学报, 2018, 44(5): 793−803Wang Gong-Ming, Qiao Jun-Fei, Qiao Lei. A generative adversarial network in terms of energy function. Acta Automatica Sinica, 2018, 44(5): 793−803 [20] Qi G J. Loss-sensitive generative adversarial networks on lipschitz densities. arXiv preprint arXiv: 1701.06264v5, 2017 [21] Zhang H, Goodfellow I, Metaxas D, Odena A. Self-attention generative adversarial networks. In: Proceedings of International Conference on Machine Learning. Long Beach, USA: 2019 [22] Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, Bharath A A. Generative adversarial networks:an overview. IEEE Signal Processing Magazine, 2018, 35(1): 53−65 doi: 10.1109/MSP.2017.2765202 [23] Hong Y J, Hwang U, Yoo J, Yoon S. How generative adversarial networks and their variants work: an overview. ACM Computing Surveys, 2019, 52(1): Article 10, 1−43 [24] Alain G, Bengio Y. What regularized autoencoders learn from the data generating distribution. The Journal of Machine Learning Research, 2014, 15(1): 3563−3593 [25] Vincent P, Larochelle H, Bengio Y, Manzagol PA. Extracting and composing robust features with denoising autoencoders. In: proceedings of International Conference on Machine Learning. Rhineland, Germany: 2008 [26] Rifai S, Vincent P, Muller X, Glorot X, Bengio Y. Contractive auto-encoders: explicit invariance during feature extraction. In: proceedings of International Conference on Machine Learning. Washington, USA: 2011 [27] Kavukcuoglu K, Ranzato M, LeCun Y. Fast inference in sparse coding algorithms with applictions to object recognition. arXiv preprint arXiv: 1010.3467v1. 2010 [28] Goodfellow I, Bengio Y, Courville A. Deep learning. Cambridge, USA: MIT press, 2017. 502-525 [29] Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In: proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: 2015. 234−241 [30] Salimans T, Goodfellow I J, Zaremaba W, Cheung V, Radford A, Chen X. Improved techniques for training GANs. In: Proceedings of International Conference on Neural Information Processing Systems. Barcelona, Spain: 2016 [31] Xu Q T, Huang G, Yuan Y, Huo C, Sun Y, Wu F et al. An empirical study on evaluation metrics of generative adversarial networks. arXiv: 1806.07755v2, 2018 [32] Shmelkov K, Schmid C, Alahari K. How good is my GAN?. In: Proceedings of European Conference on Computer Vision. Munich, Germany: 2018 [33] Cover T M, Thomas J A. Elements of information theory. New York, USA: John Wiley & Sons Inc., 2006. 12−49 [34] Nielsen F. A family of statistical symmetric divergences based on Jensen's inequality. arXiv preprint arXiv: 1009.4004v2. 2011 -
点击查看大图
计量
- 文章访问数: 30
- HTML全文浏览量: 6
- 被引次数: 0
