The 2023 Hacks for Humanity: Hacking for the Social Good event from Oct. 6-8 brought together undergraduates, high school students, professionals and community members from across the valley to innovate technology for the social good. Hosted by ASU’s Project Humanities, the event aims to address local and global issues by implementing the seven Humanity 101 principles: compassion, empathy, forgiveness, integrity, kindness, respect and self-reflection.
One of the projects to receive recognition used emerging technologies to facilitate communication and help people who are deaf or hard of hearing participate in video calls.
Led by ASU Library Makerspace student worker Chandra prakash Pandey, the third-place team created an app that uses a smartphone camera to convert real-time American Sign Language into speech. The team included Bharat Gupta, Alex Koopman, Mason Gibbs and Rajiv Kataria.
Pandey is a second-year computer science major in the Ira A. Fulton Schools of Engineering. Pandey’s interest in augmented reality (AR) modeling focused on how to use a smartphone or device camera to recognize hand gestures in the air. Imagining those gestures as sign language that could be translated to speech seemed like a logical next step that aligned with the mission of the Hacks for Humanity event. “For example, if you’re at a Starbucks and need to order a coffee, this app can help deaf and hard of hearing people communicate,” said Pandey. “Vice versa, for people who don’t know ASL, this could take speech and turn it into sign language.”
Pandey joined the Makerspace team earlier this summer where he mentors other ASU students who come in to work on their projects that range from 3D printing to laser cutting.
“I get to learn to talk with students, help them problem solve to see if can we do this,” said Pandey. “I’ve learned so much by mentoring students, and even within our hackathon team, I’m continuing to keep in touch with the high school students who helped create the hackathon project.”
Hacks for Humanity lasted for 36 hours and running on little sleep by the end, the whole team felt drained but determined. “If I start a project, doesn’t matter how it ends, I just wanted to see it complete,” said Pandey. “When we were announced, I screamed! We did it! It was really fun, and we all felt accomplished with how it came together.”
Victor Surovec, director of the ASU Library’s Makerspace sees the powerful impact of students helping students. “Our students deal with all sorts of challenges that they help other students figure out; they learn new skills and different processes, and they help set the path for other students building skillsets for those who come asking for help.”
Pandey will continue to explore AR and VR as tools for education as well as studying how generative AI technology can support these projects. This past weekend, he participated in HackHarvard, where he teamed up with students from Stanford and Boston University and made a website with a generative AI called LegalAid. “Our product helps people learn about the laws of the state they are in and helps them take further action if they are victims of a crime,” said Pandey.
Surovec added, “Now and in the future, libraries are where you can come to learn all these new technologies. We will show you all the things you need to know to imagine new solutions. Exploring new tools for expanding accessibility, and being able to have valuable conversations through an app such as this are so important to helping people feel a sense of belonging.”
Learn more about the Hacks for Humanity projects on the event website. To learn more about ASU Library’s Makerspace and schedule a visit, visit the Makerspace website.