[Stories of Social Change] Raising awareness for discriminatory AI programs in Brazil
Published: March 30, 2020
Over the years, the world has seen remarkable and rapid advancements in technology which are deeply impacting society. Despite this, we continue to struggle with issues such as poverty, violence, and climate change. At Thoughtworks, we believe that technologists have a unique role to play in how we can positively impact society and push for a more equitable tech future.
In our Stories of Social Change series, we are sharing stories from Thoughtworkers around the world who have leveraged their skills and experience to build technology that truly impacts people and effects social change. These stories show that technologists are in a unique position to change the world and inspire action in others.
Name: Roselma
Thoughtworks home: Brazil
Preferred pronouns: she/her/ela
Joined Thoughtworks: 2014
What brought you to Thoughtworks?
My journey at Thoughtworks began at Thoughtworks University (TWU) in Pune, India. More than five years later, a lot has happened. I’ve worked on a billing API project, a shadow apps platform for Thoughtworks, and a platform strategy focused on infrastructure. Today, I’m working on a search platform for Thoughtworks.
Every day, I learn a lot and get to work with amazing people; people who are willing to view the world from a different lens, one that is more inclusive and collaborative.
How has your work made a positive impact on the world?
In Brazil, most black people live in favelas, or other poor neighborhoods, where they’re disproportionately targeted by state-run surveillance systems. Facial-recognition technologies are used to ‘fight the enemy,’ however the enemy generally ends up being black Brazilians. So activists noticed a problem: some of the AI programs in Brazil had an unreliable, biased database due to discriminatory targeting.
Brazil and other countries in the global south have a lot to contribute when it comes to ethics and technology. Unfortunately, most of the debates I see are from a global north perspective. What Maíra and I are trying to understand is how Thoughtworks’ global south can contribute more when it comes to making tech more equitable for all.
We have a lot of work to do in Brazil to make sure our government respects and improves our Privacy Data Law (LGPD) and increases the population participation in the debate about the use of such technologies. Recently in Brazil, Maíra Araújo and I (both Thoughtworkers based in Recife) have started to engage with building an equitable tech future, which encourages Thoughtworkers to pursue, learn, and build technology that makes a positive impact on the world.
I have a presentation about "How Technology Can be Racist" along with Gabriel Barreto (Thoughtworker) and Silvana Bahia from Olabi (Thoughtworks partner). I have also written about the role of Black women on Ethics and Technology and Digital Security.
Currently with the campaign, we’re having introductory conversations with local groups for possible partnerships, listening to their stories, and talking about our campaign. We understand that a lot has been done outside of our walls and there are people working on the frontlines to raise awareness about the issues we are talking about here. In this scenario, how do we use our knowledge, such amazing minds, and place of privilege to join forces to this fight?
How does it feel to create technology that makes such a positive impact on people?
Aside from the work I do that impacts the issues discussed here, there’s also the aspect that it’s the right thing to do for everyone. Contributing to such an important debate makes me feel useful. I’m really motivated by the fact that I can use my skills and strengths to make a difference in peoples’ lives.
What has sparked your curiosity while doing this work?
Having a joint effort with local communities to discuss the impact that technology has or will have on their lives; specifically in the Brazilian context, where 55,8% of the population is self-declared Black (considered Negros and Pardos according Brazilian Black Movement). Brazil has a deep history of racism in regards to Black people: we are the ones who are on the worst side of income distribution, access to education, health, sanitation services, etc. Meanwhile, Black people are incarcerated at a higher rate and are more likely to be killed by the police.
It’s important that we discuss the recent use of technologies like facial recognition used for urban surveillance in Brazil. A lot has been discussed in the US about AI and racism, taking some examples we have Joy Buolamwini with AJL or Safyia Noble with her book Algorithms of Oppression or Ruha Benjamin with the book Race after Technology.
Recently, Tarcizio Silva, a Brazilian scholar and specialist in monitoring and researching social media released the book, "Comunidades, Algoritmos e Ativismos: olhares afrodiaspóricos" (Communities, Algorithms and Activism: afro diasporic views) where he curated articles from several other scholars on the subject.
How can folks inside and outside of Thoughtworks get involved or contribute to this work?
As technologists, we are in a privileged position to influence the direction of our future with digital technology. I would suggest to read more about the use and implications of AI and big data. Not only are they issues that can possibly affect you, but they are also ones that affect other minority groups.
Some organizations that are also focused on these issues in Brazil and/or Latin America include, IP.rec, Data_labe, Olabi, Rede de Observatorio de Seguranca, Coding Rights, and Lavits. Don't be satisfied with the ‘positive’ explanation of a particular technology; investigate what other implications the technology may have on vulnerable groups. When we are keen to listen to other perspectives aside from our own, we get a much more profound understanding of the world and how things can be done right.
Always ask the question: who benefits most from this technology?
In our Stories of Social Change series, we are sharing stories from Thoughtworkers around the world who have leveraged their skills and experience to build technology that truly impacts people and effects social change. These stories show that technologists are in a unique position to change the world and inspire action in others.
Name: Roselma
Thoughtworks home: Brazil
Preferred pronouns: she/her/ela
Joined Thoughtworks: 2014
What brought you to Thoughtworks?
My journey at Thoughtworks began at Thoughtworks University (TWU) in Pune, India. More than five years later, a lot has happened. I’ve worked on a billing API project, a shadow apps platform for Thoughtworks, and a platform strategy focused on infrastructure. Today, I’m working on a search platform for Thoughtworks.
Every day, I learn a lot and get to work with amazing people; people who are willing to view the world from a different lens, one that is more inclusive and collaborative.
How has your work made a positive impact on the world?
In Brazil, most black people live in favelas, or other poor neighborhoods, where they’re disproportionately targeted by state-run surveillance systems. Facial-recognition technologies are used to ‘fight the enemy,’ however the enemy generally ends up being black Brazilians. So activists noticed a problem: some of the AI programs in Brazil had an unreliable, biased database due to discriminatory targeting.
Brazil and other countries in the global south have a lot to contribute when it comes to ethics and technology. Unfortunately, most of the debates I see are from a global north perspective. What Maíra and I are trying to understand is how Thoughtworks’ global south can contribute more when it comes to making tech more equitable for all.
We have a lot of work to do in Brazil to make sure our government respects and improves our Privacy Data Law (LGPD) and increases the population participation in the debate about the use of such technologies. Recently in Brazil, Maíra Araújo and I (both Thoughtworkers based in Recife) have started to engage with building an equitable tech future, which encourages Thoughtworkers to pursue, learn, and build technology that makes a positive impact on the world.
I have a presentation about "How Technology Can be Racist" along with Gabriel Barreto (Thoughtworker) and Silvana Bahia from Olabi (Thoughtworks partner). I have also written about the role of Black women on Ethics and Technology and Digital Security.
Currently with the campaign, we’re having introductory conversations with local groups for possible partnerships, listening to their stories, and talking about our campaign. We understand that a lot has been done outside of our walls and there are people working on the frontlines to raise awareness about the issues we are talking about here. In this scenario, how do we use our knowledge, such amazing minds, and place of privilege to join forces to this fight?
How does it feel to create technology that makes such a positive impact on people?
Aside from the work I do that impacts the issues discussed here, there’s also the aspect that it’s the right thing to do for everyone. Contributing to such an important debate makes me feel useful. I’m really motivated by the fact that I can use my skills and strengths to make a difference in peoples’ lives.
What has sparked your curiosity while doing this work?
Having a joint effort with local communities to discuss the impact that technology has or will have on their lives; specifically in the Brazilian context, where 55,8% of the population is self-declared Black (considered Negros and Pardos according Brazilian Black Movement). Brazil has a deep history of racism in regards to Black people: we are the ones who are on the worst side of income distribution, access to education, health, sanitation services, etc. Meanwhile, Black people are incarcerated at a higher rate and are more likely to be killed by the police.
It’s important that we discuss the recent use of technologies like facial recognition used for urban surveillance in Brazil. A lot has been discussed in the US about AI and racism, taking some examples we have Joy Buolamwini with AJL or Safyia Noble with her book Algorithms of Oppression or Ruha Benjamin with the book Race after Technology.
Recently, Tarcizio Silva, a Brazilian scholar and specialist in monitoring and researching social media released the book, "Comunidades, Algoritmos e Ativismos: olhares afrodiaspóricos" (Communities, Algorithms and Activism: afro diasporic views) where he curated articles from several other scholars on the subject.
How can folks inside and outside of Thoughtworks get involved or contribute to this work?
As technologists, we are in a privileged position to influence the direction of our future with digital technology. I would suggest to read more about the use and implications of AI and big data. Not only are they issues that can possibly affect you, but they are also ones that affect other minority groups.
Some organizations that are also focused on these issues in Brazil and/or Latin America include, IP.rec, Data_labe, Olabi, Rede de Observatorio de Seguranca, Coding Rights, and Lavits. Don't be satisfied with the ‘positive’ explanation of a particular technology; investigate what other implications the technology may have on vulnerable groups. When we are keen to listen to other perspectives aside from our own, we get a much more profound understanding of the world and how things can be done right.
Always ask the question: who benefits most from this technology?
Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.