Alex Spangher

Understanding Multimodal Procedural Knowledge by Sequencing Multimodal Instructional Manuals

The ability to sequence unordered events is evidence of comprehension and reasoning about real world tasks/procedures. It is essential for applications such as task planning and multi-source instruction summarization. It often requires thorough understanding of temporal common sense and multimodal i…