SaGol: Using MiniGPT-4 to Generate Alt Text for Improving Image Accessibility

Yunseo Moon; Hyunmin Lee; SeungYoung Oh; Hyunggu Jung

IJCAI 2024poster1 citations

SaGol: Using MiniGPT-4 to Generate Alt Text for Improving Image Accessibility

Yunseo Moon, Hyunmin Lee, SeungYoung Oh, Hyunggu Jung

Abstract

SaGol is an AI-powered application to improve image accessibility for people with visual impairments (PVI) users. Alternative (alt) text, a general method of web accessibility for PVI users, is text or phrases that describe images on a website in an understandable way. SaGol generates alt text with the images on the user's smartphone using a vision large language model called MiniGPT-4. SaGol searches for similar images based on the generated alt text. We evaluated the length of alt text and the search accuracy. This paper shows a potential opportunity to improve image accessibility for PVI users.

Humans and AI: HAI: Human-computer interactionHumans and AI: HAI: ApplicationsHumans and AI: HAI: Intelligent user interfaces

BibTeX

@inproceedings{ijcai2024p1023,
  title     = {SaGol: Using MiniGPT-4 to Generate Alt Text for Improving Image Accessibility},
  author    = {Moon, Yunseo and Lee, Hyunmin and Oh, SeungYoung and Jung, Hyunggu},
  booktitle = {Proceedings of the Thirty-Third International Joint Conference on
               Artificial Intelligence, {IJCAI-24}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Kate Larson},
  pages     = {8745--8748},
  year      = {2024},
  month     = {8},
  note      = {Demo Track},
  doi       = {10.24963/ijcai.2024/1023},
  url       = {https://doi.org/10.24963/ijcai.2024/1023},
}