diff --git a/README.md b/README.md index e6865abd..0890fecb 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,11 @@ # Semantic Image Synthesis with SPADE ![GauGAN demo](https://nvlabs.github.io/SPADE//images/ocean.gif) -### [Project page](https://nvlabs.github.io/SPADE/) | [Paper](https://arxiv.org/abs/1903.07291) | [GTC 2019 demo](https://youtu.be/p5U4NgVGAwg) | [Youtube Demo of GauGAN](https://youtu.be/MXWm6w4E5q0) +# New implementation available at imaginaire repository + +We have a reimplementation of the SPADE method that is more performant. It is avaiable at [Imaginaire](https://github.com/NVlabs/imaginaire) + +### [Project page](https://nvlabs.github.io/SPADE/) | [Paper](https://arxiv.org/abs/1903.07291) | [Online Interactive Demo of GauGAN](https://www.nvidia.com/en-us/research/ai-playground/) | [GTC 2019 demo](https://youtu.be/p5U4NgVGAwg) | [Youtube Demo of GauGAN](https://youtu.be/MXWm6w4E5q0) Semantic Image Synthesis with Spatially-Adaptive Normalization.
[Taesung Park](http://taesung.me/), [Ming-Yu Liu](http://mingyuliu.net/), [Ting-Chun Wang](https://tcwang0509.github.io/), and [Jun-Yan Zhu](http://people.csail.mit.edu/junyanz/).
@@ -17,7 +21,9 @@ Copyright (C) 2019 NVIDIA Corporation. All rights reserved. Licensed under the [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode) (**Attribution-NonCommercial-ShareAlike 4.0 International**) -The code is released for academic research use only. For commercial use, please contact [researchinquiries@nvidia.com](researchinquiries@nvidia.com). +The code is released for academic research use only. For commercial use or business inquiries, please contact [researchinquiries@nvidia.com](researchinquiries@nvidia.com). + +For press and other inquiries, please contact [Hector Marinez](hmarinez@nvidia.com) ## Installation @@ -72,6 +78,10 @@ Once the dataset is ready, the result images can be generated using pretrained m 3. The outputs images are stored at `./results/[type]_pretrained/` by default. You can view them using the autogenerated HTML file in the directory. +## Generating Landscape Image using GauGAN + +In the paper and the demo video, we showed GauGAN, our interactive app that generates realistic landscape images from the layout users draw. The model was trained on landscape images scraped from Flickr.com. We released an online demo that has the same features. Please visit [https://www.nvidia.com/en-us/research/ai-playground/](https://www.nvidia.com/en-us/research/ai-playground/). The model weights are not released. + ## Training New Models New models can be trained with the following commands. diff --git a/data/cityscapes_dataset.py b/data/cityscapes_dataset.py index 1b47c425..b363273d 100644 --- a/data/cityscapes_dataset.py +++ b/data/cityscapes_dataset.py @@ -19,6 +19,7 @@ def modify_commandline_options(parser, is_train): parser.set_defaults(display_winsize=512) parser.set_defaults(label_nc=35) parser.set_defaults(aspect_ratio=2.0) + parser.set_defaults(batchSize=16) opt, _ = parser.parse_known_args() if hasattr(opt, 'num_upsampling_layers'): parser.set_defaults(num_upsampling_layers='more') diff --git a/docs/demo.html b/docs/demo.html new file mode 100644 index 00000000..7ec07d55 --- /dev/null +++ b/docs/demo.html @@ -0,0 +1,3 @@ + diff --git a/docs/index.html b/docs/index.html index 83ce8516..47104ab0 100644 --- a/docs/index.html +++ b/docs/index.html @@ -59,7 +59,7 @@ MIT   

in CVPR 2019 (Oral)

-

Paper | Code

+

Paper | Code | Online Demo App

@@ -96,8 +96,8 @@

Citation

Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu.
"Semantic Image Synthesis with Spatially-Adaptive Normalization", in CVPR, 2019. Bibtex -

+

Code

PyTorch

@@ -145,6 +145,11 @@

Code and Trained Models

Please visit our github repo.


+

Online Demo

+

We released an online demo of GauGAN, our interactive app that generates realistic landscape images from the layout users draw. The model was trained on landscape images scraped from Flickr.com. We released an online demo that has the same features. Please visit our online demo page.

+ +
+

Acknowledgement

We thank Alyosha Efros and Jan Kautz for insightful advice. Taesung Park contributed to the work during his internship at NVIDIA. His Ph.D. is supported by Samsung Scholarship.

diff --git a/models/pix2pix_model.py b/models/pix2pix_model.py index 69c86276..784cb15a 100644 --- a/models/pix2pix_model.py +++ b/models/pix2pix_model.py @@ -66,11 +66,10 @@ def create_optimizers(self, opt): if opt.isTrain: D_params = list(self.netD.parameters()) + beta1, beta2 = opt.beta1, opt.beta2 if opt.no_TTUR: - beta1, beta2 = opt.beta1, opt.beta2 G_lr, D_lr = opt.lr, opt.lr else: - beta1, beta2 = 0, 0.9 G_lr, D_lr = opt.lr / 2, opt.lr * 2 optimizer_G = torch.optim.Adam(G_params, lr=G_lr, betas=(beta1, beta2)) diff --git a/options/train_options.py b/options/train_options.py index 831b3854..e620a056 100755 --- a/options/train_options.py +++ b/options/train_options.py @@ -24,8 +24,15 @@ def initialize(self, parser): parser.add_argument('--niter', type=int, default=50, help='# of iter at starting learning rate. This is NOT the total #epochs. Totla #epochs is niter + niter_decay') parser.add_argument('--niter_decay', type=int, default=0, help='# of iter to linearly decay learning rate to zero') parser.add_argument('--optimizer', type=str, default='adam') - parser.add_argument('--beta1', type=float, default=0.5, help='momentum term of adam') - parser.add_argument('--beta2', type=float, default=0.999, help='momentum term of adam') + parser.add_argument('--beta1', type=float, default=0.0, help='momentum term of adam') + parser.add_argument('--beta2', type=float, default=0.9, help='momentum term of adam') + parser.add_argument('--no_TTUR', action='store_true', help='Use TTUR training scheme') + + # the default values for beta1 and beta2 differ by TTUR option + opt, _ = parser.parse_known_args() + if opt.no_TTUR: + parser.set_defaults(beta1=0.5, beta2=0.999) + parser.add_argument('--lr', type=float, default=0.0002, help='initial learning rate for adam') parser.add_argument('--D_steps_per_G', type=int, default=1, help='number of discriminator iterations per generator iterations.') @@ -37,7 +44,6 @@ def initialize(self, parser): parser.add_argument('--no_vgg_loss', action='store_true', help='if specified, do *not* use VGG feature matching loss') parser.add_argument('--gan_mode', type=str, default='hinge', help='(ls|original|hinge)') parser.add_argument('--netD', type=str, default='multiscale', help='(n_layers|multiscale|image)') - parser.add_argument('--no_TTUR', action='store_true', help='Use TTUR training scheme') parser.add_argument('--lambda_kld', type=float, default=0.05) self.isTrain = True return parser