Abstract

In this paper, we aim to generate a video preview from a single image by proposing two cascaded networks, the Motion Embedding Network and the Motion Expansion Network. The Motion Embedding Network aims to embed the spatio-temporal information into an embedded image, called video snapshot. On the other end, the Motion Expansion Network is proposed to invert the video back from the input video snapshot. To hold the invertibility of motion embedding and expansion during training, we design four tailor-made losses and a motion attention module to make the network focus on the temporal information. In order to enhance the viewing experience, our expansion network involves an interpolation module to produce a longer video preview with a smooth transition. Extensive experiments demonstrate that our method can successfully embed the spatio-temporal information of a video into one live image, which can be converted back to a video preview. Quantitative and qualitative evaluations are conducted on a large number of videos to prove the effectiveness of our proposed method. In particular, statistics of PSNR and SSIM on a large number of videos show the proposed method is general, and it can generate a high-quality video from a single image.

Video

BibTex:

    @article{zhu-2021-video,
        author   = {Qianshu Zhu and Chu Han and Guoqiang Han and Tien-Tsin Wong and Shengfeng He},
        title    = {Video Snapshot: Single Image Motion Expansion via Invertible Motion Embedding},
        journal  = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
        month    = {December},
        year     = {2021},
        volume   = {43},
        number   = {12},
        pages    = {4491-4504}
    }