Zero-shot dense video captioning

YouCook2 is the largest task-oriented, instructional video dataset in the vision community. It contains 2000 long untrimmed videos from 89 cooking recipes; on average, each distinct recipe has 22 videos. The procedure steps for each video are annotated with temporal boundaries and described by imperative English sentences (see the example below). The videos were downloaded from YouTube and are all in the third-person viewpoint. All the videos are unconstrained and can be performed by individual persons at their houses with unfixed cameras. YouCook2 contains rich recipe types and various cooking styles from all over the world. Source: http://youcook2.eecs.umich.edu/ Image Source: https://competitions.codalab.org/competitions/20594

The ViTT dataset consists of human produced segment-level annotations for 8,169 videos. Of these, 5,840 videos have been annotated once, and the rest of the videos have been annotated twice or more. A total of 12,461 sets of annotations are released. The videos in the dataset are from the Youtube-8M dataset. An annotation has the following format: { "id": "FmTp", "annotations": [ { "timestamp": 260, "tag": "Opening" }, { "timestamp": 16000, "tag": "Displaying technique" }, { "timestamp": 23990, "tag": "Showing foot positioning" }, { "timestamp": 55530, "tag": "Demonstrating crossover" }, { "timestamp": 114100, "tag": "Closing" } ] } Source: Video Timeline Tags (ViTT)