Haiyue Sheng

See, Think, Act: Teaching Multimodal Agents to Effectively Interact with GUI by Identifying Toggles

The advent of multimodal agents facilitates effective interaction within graphical user interface (GUI), especially in ubiquitous GUI control. However, their inability to reliably execute toggle control instructions remains a key bottleneck. To investigate this, we construct a state control benchmar