AI and ML
How art programs drive innovation at Thoughtworks
Great art has the power to inspire us, to touch our souls and make us reflect on the true nature of what it is to be human. For many, such as the visually impaired, art is a closed book, inaccessible and impenetrable.
To change this, Thoughtworks has been collaborating with an artist, Lenora Rosenfield, to support the exhibition “O Relevo” (The Relief) for the 2018 Mercosul Biennial, in Porto Alegre, Brazil. There are more than 6.5 million people in Brazil that have visual impairments.
The exhibition was composed of new interpretations of famous paintings which were already presented during the ten previous editions of the Mercosul Biennial – including A Negra, by Tarsila do Amaral. Those reinterpretations were created using relief and textures to provide sensorial experiences other than visual, allowing visually impaired people to experience works of art through other senses. On top of this technique, used before by the artist, this year Thoughtworks helped create wearable and digital technologies to make the experiences even richer.
Complementing the artist’s technique and in partnership with her, Thoughtworks created wearable technologies in the shape of an “assistive glove.” The assistive glove possessed color sensors and the capacity to translate the colors of the works into sounds, whether they’re the words used to name colors (“blue”, “red”, “yellow”, etc.) or illustrative recordings of colors, the most appropriate for visually impaired people who were never exposed to colors (the sound of the sea, for example, referring to the color blue). With this apparatus, the visitors were able to “visualize” the works of Lenora with the tips of their fingers.
The union of art and technology is not new for Thoughtworks. In our New York office, for example, there is a program called the Arts Residency. In 2016, Thoughtworks collaborated with the Cyborg Foundation to develop new prototypical human “senses.” The artists Neil Harbisson and Moon Ribas – in partnership with Thoughtworks for a technological component - believe that human psychology can be “reformulated” using electronic sensors permanently integrated into the body. In the case of that experiment in Brazil, this “something more” is actually the junction of art and technology with the aim of inclusion.
Desiree Santos and Igor Balteiro, Thoughtworkers who played an integral role in the project, believe that having a diverse team was critical to its success. This enabled us to take in perspectives what we might otherwise have not considered, says Igor — for instance, considering how the system would work for those with color vision deficiency or loss of sensation in their fingertips.
Even so, not everything was plain sailing. Desiree left the initial workshop with the conviction that the solution would involve embedding a vibration sensor inside the actual work of art. However, it became clearly obvious that such a solution would not meet other requirements, like the flexibility to use it in other paintings. “It was a tremendous setback! We had to discard all that we had done, and start anew,” she says.
The team met again for another brainstorming session in order to “reorganize the mosaic”, and – now equipped with the learning of the previous attempts – we idealized a prototype based on another sense (hearing), external to the work of art (so as not to interfere in the artistic process), and at the same time, that moves with the person. And so, there came the idea of working with wearable technologies and quickly sprang up the solution of the glove which allows a person to touch the work of art. Desiree links the quest for a more flexible solution to a connection the Thoughtworks team made between software and hardware: ”We use our knowledge of software and apply it to hardware, and I consider that connection positive. My experience with software helped create better quality hardware. The sensor could have been embedded in the painting, but it wouldn’t have been such a flexible, scalable and elegant solution. It made no sense!”
“I am proud to have participated in this project,” says Desiree. “It’s wonderful to see how technology can open doors for a group of people – generally excluded – and actually, democratize the fine arts. The assistive glove users realized that the project was specially done for them, and this touches my heart!”
Technology is not neutral; it can create a better or worse world. As technologists, we need to be aware of this responsibility. At times, we need to be intentional to make technology more inclusive, instead of creating a digital divide.
At Thoughtworks, we have a goal to help create an Equitable Tech Future. This quest for a fairer technology is a path which often leads us in surprising directions. What about you, how do you see the role of technology in building a better future?
To change this, Thoughtworks has been collaborating with an artist, Lenora Rosenfield, to support the exhibition “O Relevo” (The Relief) for the 2018 Mercosul Biennial, in Porto Alegre, Brazil. There are more than 6.5 million people in Brazil that have visual impairments.
The exhibition was composed of new interpretations of famous paintings which were already presented during the ten previous editions of the Mercosul Biennial – including A Negra, by Tarsila do Amaral. Those reinterpretations were created using relief and textures to provide sensorial experiences other than visual, allowing visually impaired people to experience works of art through other senses. On top of this technique, used before by the artist, this year Thoughtworks helped create wearable and digital technologies to make the experiences even richer.
Art and technology
The works of art were developed with a new technique called “synthetic fresco” – a creation of the artist – and interpreted with the same dimensions and colors as the originals, giving impaired people ways of understanding and experimenting them through touch.Complementing the artist’s technique and in partnership with her, Thoughtworks created wearable technologies in the shape of an “assistive glove.” The assistive glove possessed color sensors and the capacity to translate the colors of the works into sounds, whether they’re the words used to name colors (“blue”, “red”, “yellow”, etc.) or illustrative recordings of colors, the most appropriate for visually impaired people who were never exposed to colors (the sound of the sea, for example, referring to the color blue). With this apparatus, the visitors were able to “visualize” the works of Lenora with the tips of their fingers.
Picture 1: Assistive glove being used to interact with a red and green world map
The union of art and technology is not new for Thoughtworks. In our New York office, for example, there is a program called the Arts Residency. In 2016, Thoughtworks collaborated with the Cyborg Foundation to develop new prototypical human “senses.” The artists Neil Harbisson and Moon Ribas – in partnership with Thoughtworks for a technological component - believe that human psychology can be “reformulated” using electronic sensors permanently integrated into the body. In the case of that experiment in Brazil, this “something more” is actually the junction of art and technology with the aim of inclusion.
Wearable technologies, agile methodologies, quick prototyping, diverse team and empathy with users
Various hypotheses were formulated by the Porto Alegre group, which also held workshops on several themes in the office, including sewing (sewing electronic circuits into clothes) and distinct technologies that were tested during the project, such as embedded vibration sensors, Arduino and others. We thought of solutions related to sensors in the paintings, vibration, sound, sensors on the fingers, among other hypotheses. Through techniques we frequently use in our day-to-day client work, such as rapid prototyping, validation of hypotheses and inclusion of real users during the whole process, we kept testing until we reached the final solution.
Picture 2: Thoughtworks team in a sewing workshop
Desiree Santos and Igor Balteiro, Thoughtworkers who played an integral role in the project, believe that having a diverse team was critical to its success. This enabled us to take in perspectives what we might otherwise have not considered, says Igor — for instance, considering how the system would work for those with color vision deficiency or loss of sensation in their fingertips.
Even so, not everything was plain sailing. Desiree left the initial workshop with the conviction that the solution would involve embedding a vibration sensor inside the actual work of art. However, it became clearly obvious that such a solution would not meet other requirements, like the flexibility to use it in other paintings. “It was a tremendous setback! We had to discard all that we had done, and start anew,” she says.
The team met again for another brainstorming session in order to “reorganize the mosaic”, and – now equipped with the learning of the previous attempts – we idealized a prototype based on another sense (hearing), external to the work of art (so as not to interfere in the artistic process), and at the same time, that moves with the person. And so, there came the idea of working with wearable technologies and quickly sprang up the solution of the glove which allows a person to touch the work of art. Desiree links the quest for a more flexible solution to a connection the Thoughtworks team made between software and hardware: ”We use our knowledge of software and apply it to hardware, and I consider that connection positive. My experience with software helped create better quality hardware. The sensor could have been embedded in the painting, but it wouldn’t have been such a flexible, scalable and elegant solution. It made no sense!”
“I am proud to have participated in this project,” says Desiree. “It’s wonderful to see how technology can open doors for a group of people – generally excluded – and actually, democratize the fine arts. The assistive glove users realized that the project was specially done for them, and this touches my heart!”
Inclusion and business
Our experience and practice have shown that taking an inclusive approach isn’t just the right thing to do; it also has business benefits among which we highlight three:- New technologies are dual. They can be applied to more different projects once created. For example, in the same office in Porto Alegre, and sharing the same laboratory, we used image processing and sensor technologies (both used in the Assistive Glove project) to generate business value in a case of measurement of product quantity in a large company, as can be seen in this article.
- The inclusion of different people through technology also means the inclusion of new potential users of new products and services. At times that is a question of legislation/regulation, like for example the creation of accessible software, and in other cases, the "side effect" is to welcome more groups of potential clients.
- We are seeing the increase of conscious consumer groups who — when purchasing products — take into account the environment, the human and animal welfare, as well as fair work relationships. For them, purchases aren’t simply questions of price and brand.
Technology is not neutral; it can create a better or worse world. As technologists, we need to be aware of this responsibility. At times, we need to be intentional to make technology more inclusive, instead of creating a digital divide.
At Thoughtworks, we have a goal to help create an Equitable Tech Future. This quest for a fairer technology is a path which often leads us in surprising directions. What about you, how do you see the role of technology in building a better future?
About Lenora Rosenfield
Gaucha from Porto Alegre, Lenora Rosenfield's "synthetic fresco" technique, created in Italy, holds the central place in her current production. With a postdoctoral degree from the University of Udine, she was a professor of painting at the Institute of Arts (UFRGS) in 1993 -2017, and for more than two decades, she was at the head of the restoration laboratory of that institution. She is a reference in the restoration of works of art such as those of Pedro Weingartner, Aldo Locatelli, Glauco Rodrigues, Francisco de Goya, Pablo Picasso, and Marc Chagall. The artist has stayed away from restoration for some years, and she retired from academic life in 2017 to devote herself more intensely to her poetics. Her works are in the collections of the main institutions of Rio Grande do Sul, and in private collections in Brazil, the United States and Europe.Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.
