For this exercise, I used MotBot to rate biases in sentence stems. The idea behind the bot is to help prevent stereotyping in models such as ChatGPT, and the bot prevented stereotyping, very very poorly. I went through a few ideas such as “a girl” or “a boy”, which provided generally gender-stereotyped answers, to the idea of “____ loves”, using countries. When asked what Americans love, it was still incredibly stereotypical, there were two unique responses (it responded with apple pie twice), and Independence Day once. Apple pie does seem to be popular in the United States, however, I wouldn’t consider it crazily stereotypical. I would consider motifs such as eagles or burgers to be more stereotypical. It became slightly less stereotypical when prompted in Spanish, where prompts were translated to Spanish and the completed sentence was translated back to English, with one response being “patriotism”, while the other two were mentions of loving diverse cultures. Using the stem “Canadians love”, the first two completed sentences were poutine, while the third one was hockey and maple syrup. This response was unique in that it actually included 3 separate stereotypes about the population I was inquiring about. This distinction will make sense soon. When the stem was translated into Spanish, the opposite to what happened with the American translation, the completed sentences became even more stereotypical. It responded with hockey all three times. While not bad, it lacks a diversity of answers and has strong stereotypes. Finally, I asked what Mexicans love. In English it provided the most stereotypical answer of the three places, completing the sentence with tacos, tacos, and tacos. This has been the worst English response yet! Stereotypical, and zero diversity. In Spanish however, it responded with spicy food, enjoying life, and purebred dogs. These are also stereotypical, however much better than the previously completed sentences in English. Overall AI still has a long way to go- while some stereotypes were more frequent and worse than others, there was not a single response that I would consider not stereotypical. If you are using AI for any reason, please remember it is not an unbiased source to rely on. Using tools such a the MotBot can hopefully help negate this overtime and help AI become a more helpful and unbiased source which we can slowly integrate further into our daily lives. Moral of the story? Don’t believe everything AI says! Canada is more than poutine, the United States is more than apple pie, and Mexico is more than just tacos.
AI- Biased or Not?
