>
B
Model Details
Introduction
Despite recent progress in generative image modeling, successfully generating high-resolution, diverse samples from complex datasets such as ImageNet remains an elusive goal. To this end, we train Generative Adversarial Networks at the largest scale yet attempted, and study the instabilities specific to such scale. We find that applying orthogonal regularization to the generator renders it amenable to a simple "truncation trick," allowing fine control over the trade-off between sample fidelity and variety by reducing the variance of the Generator's input. Our modifications lead to models which set the new state of the art in class-conditional image synthesis. When trained on ImageNet at 128x128 resolution, our models (BigGANs) achieve an Inception Score (IS) of 166.5 and Frechet Inception Distance (FID) of 7.4, improving over the previous best IS of 52.52 and FID of 18.6.
Benchmarking
FLOPs1.8e+21
Notes: 3e21, estimate taken from: https://www.lesswrong.com/posts/wfpdejMWog4vEDLDg/ai-and-compute-trend-isn-t-predictive-of-what-is-happening
Training
Training Code Accessibilityrepo license is Apache: https://github.com/tensorflow/tfhub.dev/blob/master/assets/docs/deepmind/models/biggan-deep-512/1.md
HardwareGoogle TPU v3
Hardware Quantity256
Size Notes: "To confirm that our design choices are effective for even larger and more complex and diverse datasets, we also present results of our system on a subset of JFT-300M (Sun et al., 2017). The full JFT-300M dataset contains 300M real-world images labeled with 18K categories. Since the category distribution is heavily long-tailed, we subsample the dataset to keep only images with the 8.5K most common labels. The resulting dataset contains 292M images – two orders of magnitude larger than ImageNet. "
Parameters
Parameters112694781
Notes: I used the publicly available implementation available at [1] There I loaded the biggan-deep512/1 model, and ran script [2] to compute the number of parameters [1] https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/biggan_generation_with_tf_hub.ipynb [2] n_params = 0 for var in module.variables: n_params += np.prod(var.shape.as_list()) pass print(n_params)
Authors
Andrew Brock, Jeff Donahue, Karen Simonyan