As the state-of-the-art machine learning methods in many fields rely on larger
datasets, storing datasets and training models on them become significantly more
expensive. This paper proposes a training set synthesis technique for data-efficient
learning, called Dataset Condensation, that learns to condense large dataset into
a small set of informative synthetic samples for training deep neural networks
from scratch. We formulate this goal as a gradient matching problem between the
gradients of deep neural network weights that are trained on the original and our
synthetic data. We rigorously evaluate its performance in several computer vision
benchmarks and demonstrate that it significantly outperforms the state-of-the-art
methods.