Captioning Bosch: A Twitter Bot

Cornelia Ferner

IJCAI 2022poster0 citations

Captioning Bosch: A Twitter Bot

Abstract

The artworks by Dutch painter Hieronymus Bosch are well known for their incredible wealth of details. The popular BoschBot regularly posts small segments of the digitized paintings on Twitter, thus relieving their density and making them more accessible. CaptioningBoschBot, the Twitter bot presented in this demo, reverses the creative process of the artist: It uses the out-of-context painting segments as input for an encoder-decoder model to generate captions that interpret the painted objects. As the model was only trained on realistic, photographic images, curious interpretations of the otherworldly details can be observed. The generated captions are again posted on Twitter to encourage discussions about Bosch's masterpieces and the AI technology in general.

Application domains: Text, literature and creative languageApplication domains: Images and visual artsMethods and resources: Applications and software frameworksMethods and resources: Machine learning, deep learning, neural models, reinforcement learning

BibTeX

@inproceedings{ijcai2022p694,
  title     = {Captioning Bosch: A Twitter Bot},
  author    = {Ferner, Cornelia},
  booktitle = {Proceedings of the Thirty-First International Joint Conference on
               Artificial Intelligence, {IJCAI-22}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Lud De Raedt},
  pages     = {5011--5014},
  year      = {2022},
  month     = {7},
  note      = {AI and Arts - Exhibits},
  doi       = {10.24963/ijcai.2022/694},
  url       = {https://doi.org/10.24963/ijcai.2022/694},
}