Basic communication app - startconvo.co.uk

Hi everyone,

So I want to share one of the projects I’ve been working on using hype and javascript.

Background of project

I’m a practicing nurse in the UK and I’ve previously worked on a ward with people who’d had a stroke and found it difficult to talk. I’ve also worked with people with locked in syndrome who can only blink or use small movements.

As you can imagine communication can be difficult for these people. However, there’s a common method used in which a person can use an alphabet board to signal through the letters to create words and therefore communicate.

This communication idea is simple however it relies on both parties knowing the alphabet board well and both understanding how to navigate it. It’s also difficult as a person can only often give 1 signal so the system has to take this into account.

How the system works

The nurse, or person initiating the system, first goes through the alphabet vowels.

Using one signal, such as blinking, the user signals the vowel they want to select.

The nurse then states the vowel again but with the consonants that follow. For example:

if the user blinked at the letter A, then nurse would then say A B C D awaiting a signal for the letter selected. This would be the first letter of the word created.

Heres how the alphabet is layed out using this system:

A bcd
E fgh
I jklmn
O pqrst
U vwxyz

Once a letter has been selected, the process is started again to eventually make a word.

Making the system digital

Having used this method with patients in real life, it’s actually hard to use to start with. First you have to confidently know the alphabet without thinking “what’s next” which is hard when the person in front of you is trying to talk. Also, you have to write down every single letter selected otherwise you risk forgetting what was selected and then not knowing what the person is trying to say.

With the hype created web app, the device takes away this human error and the nurse only has to swipe through the letters and tap when they see a signal. Furthermore, the app stores the last letter selected in a progress bar and once a word is created, the nurse can tap the bar and the device reads out the word.

The app works best on a iPad but also works in safari and other internet browsers on a PC/Mac. It currently doesn’t work on smaller devices but will work on the larger screened iPhones.

I’ve named the app ‘Start Convo’ and it’s perfect to start those small conversations for those people who find it hard to communicate.

Please have a go by visiting www.startconvo.co.uk

Further development

I’m hoping to have a version that automatically cycles through the letters so the nurse doesn’t have to. Then they would hopefully only have to wait for the signal and tap the screen.

I’m also looking at ways in which the user themselves can trigger what letter they select using a bluetooth button or something similar.

I would also like to include word predictions and the option to save previous word + sentences in future versions.

I would love to hear what you all think about the project and feel free to share this with anyone that could potentially use it. Having seen and worked stroke survivors myself, cheap, affordable and easy to use communication apps can be really be useful to reduce stress and anxiety when trying to express your wants and wishes.

Thanks,

Chris

2 Likes

Hi Chris!

Have not looked at your app yet - but what a great project!

1 Like

I found the dragging on my iPad a bit difficult.
So auto would likely be good.

I was thinking you could have an auto option or a tap buttons to advance right or down.

1 Like

Thanks for the feedback. I think auto cycling will make it more simple to pick up and use. I agree that swiping isn’t always the best for navigation and simple tap buttons are often better :+1:

1 Like

I actually meant to say have both tap or auto as an option.

And love what you are trying to do. Well done.

1 Like

Nice work!

Predictions would be a great update. Not sure if that makes sense but the idea would be to have a language setting and then the app would query an auto-completion service (online, SAAS) or dictionary (embedded dictionary) displaying the suggestion as a choice based on the letters already provided.

The system with blinking then gets the additional step of asking WORD

WORD
A bcd
E fgh
I jklmn
O pqrst
U vwxyz

That way the patient can choose the auto-complete suggestion if it is already the right one.

Going one step further you can add the top 3 suggestions in the WORD row and handle them the way you do letters. Meaning if the patient see the desired word in the word row he blinks on WORD and then the nurse simply reads the words one by one until he confirms.

WORD suggestion1 suggestion2 suggestion3
A bcd
E fgh
I jklmn
O pqrst
U vwxyz

suggestion1-3 would be replaced by the actual words the auto complete query returns.

Finally, you could introduce a weighed auto-complete that “learns” based on the choices made previously (confirmed by tapping them after blinking) and adapts over time to the patients spelling habits (given the app is running on a per patient basis). If it’s a general device for the complete station adaptive behavior might not make sense.

2 Likes

As a complement to @MaxZieb’s thoughts about word complete…

I am thinking of a baby who does not know how to speak coherently yet.

There would be a set of “maintenance” needs a parent cycles through initially:
Hungry, thirsty, bathroom, hurt, etc.

Instead of having to spell out basic~important (and often immediate) needs all the time there could be a set of phrases for these situations, which in addition to the “standard” ones could be customized as needed.

1 Like

Maybe this helps… https://www.youtube.com/watch?v=CVClHLwv-4I

Theoretically, if the JavaScript based facial recognition software can be used to detect a blink in just one eye, kinda like a wink, then maybe your Hype project could work without so much intervention from a nurse.

1 Like

Thanks for all your suggestions. I’ll keep on developing the idea and feed back to the post on my progress :+1: