>
C
Model Details
Domain:
Model Access:
Open weights (unrestricted)
Citations:
896
Introduction
Text-to-Image generation in the general domain has long been an open problem, which requires both a powerful generative model and cross-modal understanding. We propose CogView, a 4-billion-parameter Transformer with VQ-VAE tokenizer to advance this problem. We also demonstrate the finetuning strategies for various downstream tasks, e.g. style learning, super-resolution, text-image ranking and fashion design, and methods to stabilize pretraining, e.g. eliminating NaN losses. CogView achieves the state-of-the-art FID on the blurred MS COCO dataset, outperforming previous GAN-based models and a recent similar work DALL-E.
Benchmarking
FLOPs2.68e+22
Notes: source: https://lair.lighton.ai/akronomicon/ archived: https://github.com/lightonai/akronomicon/tree/main/akrodb
Training
Training Code AccessibilityApache 2 license https://github.com/THUDM/CogView train script: https://github.com/THUDM/CogView/blob/main/scripts/pretrain_single_node.sh
HardwareNVIDIA Tesla V100 DGXS 16 GB
Hardware Quantity512
Size Notes: "We collected about 30 million text-image pairs from multiple channels, and built a 2.5TB new dataset (after tokenization, the size becomes about 250GB)." 250GB * (1 word / 5 bytes) = 50 billion words or 67 billion tokens So 30M text-image pairs and 50 billion words
Parameters
Parameters4000000000
Notes: "We propose CogView, a 4-billion-parameter Transformer with VQ-VAE tokenizer to advance this problem."
Authors
Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, Jie Tang
Language