Appraising apps in SLT/P (1) – Outcome measures and building the evidence base

I’ve resisted the cliché of calling this post “App”raisal / APPraisal / app-raisal

Well I’d probably over-estimated myself at the end of my last blog post when I suggested I’d present a structured approach to appraising apps. The more I thought about it, the more there seemed to be to think about and naturally there would always be things that I’d miss out on. So in the interests of getting a post up and making the topic of appraising apps more manageable and not restricting it to just a single blog post, I’m now proposing to present questions and things to think about, more or less as they occur to me. I’ll probably aim to introduce one or two into any one blog post. This means that I can always add to my list of thoughts later, and it also gives people the opportunity to comment and make suggestions (if they are so inclined).

Results are much easier to digest in graphsSo without any further excuses and delays I’ll go directly into some of my considerations on the issue. These will not necessarily be in any particular order as I’ll aim to make things more structured (i.e. so they could fit on a couple of sides of A4) as my list becomes more well-rounded and refined.

1) How can outcomes be measured?

While I’m aware that a number of apps will have built-in progress tracking capabilities, I’d avoid the temptation to consider this as a truly valid outcome measure. As SLT/Ps we should all be aware of the tendency to very often find only item-specific treatment effects when using impairment-based therapies (particularly when working with adult clients with aphasia). Therefore, improvements in performing the app tasks cannot be assumed to reflect tangible linguistic/functional improvements. Any improvements in doing the app task may simply be a reflection of practice effects and repeated exposure to the same items over and over and over again until they have become ingrained as an automatic stimulus-response behaviour.

SLT/Ps should therefore consider what independent measures can be used to assess whether an app has brought about real speech and language improvement.

If an app is offering therapy (or practice) in spoken word retrieval through picture naming activities, we should consider whether it would be appropriate to measure improvements on a secondary measure of spoken picture naming whether it be through a published assessment with normative data or whether it be our own informal/unpublished assessment.We should even go so far as to think “well in my ‘regular’ practice, where I don’t use apps, I’d also be interested to know if my hours of therapy involving getting Patient X to name lots of pictures has led to any changes in Patient X’s sentence production and conversation, therefore I’d like to get a bit more of an extended speech sample”. The same principles apply if an app is offering practice for children in producing speech sounds correctly – we should be looking to identify an independent measure of phonology, and so on and so on for all types of skills and behaviours that an app may claim to target.

Rather than belittling the value of apps and being a chore (i.e. an additional task) to perform, if such additional outcome measures are routinely in place when carrying out ‘therapy by app’, they can only serve to strengthen the app’s claim of offering something positive. If measurable improvement is observed in something other than the app task itself, then it naturally suggests that using the app has had a positive impact. Indeed, if SLT/Ps routinely adopt such practices then they themselves are in a position to contribute to the evidence base (which is currently lacking), rather than just carrying on regardless and waiting for someone in some University somewhere to run the definitive randomised controlled trial on 100s of patients – Yes ‘research’ can be carried out within the context of routine care and reported by SLT/Ps working in clinical contexts!

Furthermore, if one is struggling to think of an appropriate independent measure it may also be an indication that the app may be targeting something not usually considered by SLT/Ps (whether due to an oversight or simply that it is considered trivial or unimportant), or alternatively, it may not be targeting anything in particular.


About chrissp1980
Currently a lecturer in speech pathology in North Queensland, Australia. I'm lecturing in acquired disorders of speech and language and also attempting to enthuse students in conducting clinically-relevant projects using principles of Evidence-Based Practice. Wish me luck!

One Response to Appraising apps in SLT/P (1) – Outcome measures and building the evidence base

  1. Pingback: My thoughts on: Reading TherAppy from Tactus Therapy « Thinking of a better blog name

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: