AI Art Curation: Re-imagining the city of Helsinki in occasion of its Biennial

TL;DR

The intersection of artificial intelligence and art curation is explored through a project developed for the 2023 Helsinki Art Biennial, "New Directions May Emerge," addressing how contemporary machine learning can move beyond data management to become an active participant in curatorial practice.

Abstract

Art curatorial practice is characterized by the presentation of an art collection in a knowledgeable way. Machine processes are characterized by their capacity to manage and analyze large amounts of data. This paper envisages AI curation and audience interaction to explore the implications of contemporary machine learning models for the curatorial world. This project was developed for the occasion of the 2023 Helsinki Art Biennial, entitled New Directions May Emerge. We use the Helsinki Art Museum (HAM) collection to re-imagine the city of Helsinki through the lens of machine perception. We use visual-textual models to place indoor artworks in public spaces, assigning fictional coordinates based on similarity scores. We transform the space that each artwork inhabits in the city by generating synthetic 360 art panoramas. We guide the generation estimating depth values from 360 panoramas at each artwork location, and machine-generated prompts of the artworks. The result of this project is an AI curation that places the artworks in their imagined physical space, blurring the lines of artwork, context, and machine perception. The work is virtually presented as a web-based installation on this link http://newlyformedcity.net/, where users can navigate an alternative version of the city while exploring and interacting with its cultural heritage at scale.

Figures (7)

• Figure 1: Example artwork with CLIP Interrogator extracted text. Artwork: Lepistö, P. "Kolme vesilintua metsälammella". Courtesy of the HAM collection.
• Figure 2: Example indoor artwork (left) with predicted location (right). Artwork: Matson, A. "Asetelma". Courtesy of the HAM collection.
• Figure 3: Top: Map of Helsinki showing the public artworks $\mathbf{y}_{\mathbf{z}_{public}^*}$ in red, and the predicted locations $\mathbf{\hat{y}}_{\mathbf{z}_{indoor}^*}$ in blue. Clustered blue points illustrate how the Random Forest model does not capture variation in the data. Bottom: Updated map with the GPS-inspired similarity method. Green points indicate sea locations that did not have a 360° panoramic image.
• Figure 4: Example of the top three most similar public artworks (right) to an indoor painting sample (left). Artworks (from left to right top to bottom): Sallinen, T. "Hihhulit"; Juva, K. "Arkkienkeli Mikael"; Sörensen-Ringi, H. "Jäähyväiset"; Kaasinen, T. "Ihmisiä". Courtesy of the HAM collection.
• Figure 5: Distribution of the proposed metrics. 3 image-pair examples are showed from low, medium, and high scores. Each of them showing the original painting above, the original panorama to the left and the art panorama to the right.