Improving Complex Scene Generation by Enhancing Multi-Scale Representations of GAN Discriminatorsopen access
- Authors
- Lee, Hanbit; Lee, Sang-Goo; Park, Jaehui; Shim, Junho
- Issue Date
- Apr-2023
- Publisher
- IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
- Keywords
- Image analysis; Generative adversarial networks; Transfer learning; Feature extraction; Self-supervised learning; Generators; scene generation; self-supervised learning; transfer learning
- Citation
- IEEE ACCESS, v.11, pp 43067 - 43079
- Pages
- 13
- Journal Title
- IEEE ACCESS
- Volume
- 11
- Start Page
- 43067
- End Page
- 43079
- URI
- https://scholarworks.sookmyung.ac.kr/handle/2020.sw.sookmyung/151908
- DOI
- 10.1109/ACCESS.2023.3270561
- ISSN
- 2169-3536
- Abstract
- While recent advances of GAN models enabled photo-realistic synthesis of various object images, challenges still remain in modeling more complex image distributions such as scenes with multiple objects. The difficulty lies in the high structural complexity of scene images, where the discriminator carries a heavy burden in discriminating complex structural differences between real and fake scene images. Therefore, enhancing the discriminative capability of the discriminator could be one of the effective strategies to improve the generation performance of GAN models. In this paper, we explore ways to boost the discriminative capability by leveraging two recent paradigms on visual representation learning: self-supervised learning and transfer learning. As the first approach, we propose a self-supervised auxiliary task tailored to enhance the multi-scale representations of the discriminator. In the second approach, we further enhance the discriminator by utilizing pretrained representations from various scene understanding models. To fully utilize knowledge from multiple expert models, we propose a multi-scale feature ensemble to mix multi-sale representations. Empirical results on challenging scene datasets demonstrate that the proposed strategies significantly advance the generation performance, enabling diverse and photo-realistic synthesis of complex scene images.
- Files in This Item
-
Go to Link
- Appears in
Collections - 공과대학 > 소프트웨어학부 > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.