Multimedia Curriculum Learning for Language Acquisition

Abstract

We explore how curriculum learning impacts language acquisition from multimedia data. We propose a new curriculum learning methods based on word concreteness aiming to strengthen the learning of concrete concepts in images. We construct a new Yoga Videos dataset to evaluate language acquisition and experiment with MS COCO [1] image captioning dataset to show the generalizability of our approaches. Extensive experimental results demonstrate the effectiveness of language curriculum and multimedia learning to accelerate learning and improve data efficiency by achieving the equivalent performance with approximately 40% less training data, especially with small-scale datasets