- Run by Heart n Soul, in partnership with University of the Arts London’s Creative Computing Institute and the Royal Borough of Greenwich.
- Aimed to demonstrate how artificial intelligence (AI) could improve communication between people with learning disabilities and autistic people, and health and social care professionals – an area identified as a priority for change by people with learning disabilities and autistic people.
- Involved the development and testing of an AI app that uses multimodal machine learning to support understanding.
- Ran from November 2023 to March 2024.
Health care services can be difficult for people to navigate, and they can be even more confusing and overwhelming for people with learning disabilities and autistic people.
Heart n Soul, a creative community and arts charity, has explored how artificial intelligence (AI) might offer a way of communicating that would allow people with learning disabilities to be better understood, be able to say what they want, present themselves independently, and take a more active role in their care and support. It may also offer alternative ways of understanding and explaining jargon used in health and social care communications.
Building on a prototype that Heart n Soul had developed, the jargon-buster application, a co-design team of people with learning disabilities and autistic people and social care staff started to build and test a bespoke AI app through a series of workshops.
By integrating audio, video, images, speech recognition and generative AI, the team created a multimodal analysis and communication tool. They discovered that it is possible for people with learning disabilities and autistic people to develop their own AI systems.
The exploratory app is able to simplify complex letters, change terminology so that the majority of people can understand it, describe pictures, build up an understanding of people’s preferences, and be used by people with complex needs, including physical needs, using a simple push button system.
The project also showed how communication is an effective tool to improve relationships and enable culture change; by making simple accessible communication the norm and complicated jargon-layered communication the exception, connection and relationships are better for everyone.
This exploratory work builds on Heart n Soul’s Believe in Us project, which is part of the Common Ambition programme, which supports collaborative communities where people work together to reduce inequalities and improve health and care.
For more information about this project, please contact Heart n Soul.