How many ways can self-report change measurement help

A survey 1.Have self-report change questionnaires helped psychotherapy? NO: quite the reverse Hm: they’ve helped and hindered YES: without change data we’d be dead 2.Do you use then routinely in your clinical work? YES/NO 3.If “NO”: why not …?
Learning from CORE Chris Evans EAP SARC conference Connecting Psychotherapy Practice & Research SFU Vienna 17.ii.16.
NO: quite the reverse Hm: they’ve helped and hindered YES: without change data we’d be dead 2.Do you use then routinely in your clinical work. YES/NO 3.If NO : why not … .
PBE Outcome is the O in CORE, but it’s about change Power … Challenges/problems and possible avenues. Beware of numbers – but use them. Beware of certainties – stop seeking them. Let’s plan for the future in EAP. … and there are some pictures.
So you want to help … Help clients/patients Help ourselves Psychologically and financially Help our profession, this psychotherapy thing!
CORE-OM and CORE system Clinical Outcomes in Routine Evaluation
History Roots in Society for Psychotherapy Research (SPR) inputs over the years from many other bodies: MHF, CPCT, Artemis Trust, NHS, CIMS, Leeds & Sheffield Universities MHF meeting and funding from1994 started the process Vital other funding from many sources: CPCT, Artemis, NHS (Leeds), HEFC (universities) Many people: Michael Barkham, Frank Margison, Janice Connell, John Mellor-Clark, Kerry Audin …100s more Official launch 1998 Owned by CORE System Trust (coresystemtrust.org) Still growing!
CORE history: publications Numbers from PsycINFO all word searching January 2016 H index Google search for CORE-OM : 115M hits!
familiarity with the same measures & scores on those measures) –providing comparability between studies –facilitate audit of clinical services –facilitate audit of individual therapists’ work –feed into health economic and other research analyses.
CORE system & other measures
Actually, this sort of CORE: Il c(u)ore del CORE That the person completing the measure, who might have few or no problems, or really very severe problems, might feel that someone scoring their questionnaire would gain some sense of their state over the last week.
CORE system Not just a measure but also … –Theory –Attitude –People Not just a measure but actually … –Measures, instruments Money –To fund it … –So it could be copyleft: copyright protected but free to use without any fees Support –Scanning & then computerisation
Theory & attitude Theory: –Get out of the confidentiality closet –Embrace simple quantification –It’s a core only, not a complete description – Outcome = change Attitude – Bottom up curiosity not top down politics –Share it –Compare it –It’s a core only: complement it –Support each other
Out of the closet: quantification Sharing data about therapy helps describe what’s happening Particularly important in late 20 th C & 21 st C with pressure on funding and austerity Numbers easily aggregated, fit a commoditising Zeitgeist Quantitative data almost never leaks someone’s identity so helps preserve confidentiality … … but it always loses information as it selects what to measure
Balancing the personal and the publishable Quantitative data protects confidentiality but loses richness We must always chose how to mitigate the loss There is no one right answer Psychotherapy research and related fields need much more discussion of this, and better models, need to weave quantitative and qualitative better
34 items on two sides of A4 tick box completion five response levels (0 to 4) pan-theoretical (equally disliked by all modalities and theories) simple language can score it by hand can use optical character recognition to scan it shortened forms available.
Not just a measure : CORE-OM
Not just a measure : CORE-A CORE-A (Assessment) is a conext setting, practitioner completed record form To contextualise therapy change data, not an outcome measure 2 sides of A4 for Therapy Assessment Form (CORE-TAF) 2 sides of A4 for End of Therapy form (CORE-EoT) Mostly tick box, but some text
CORE-A TAF covers … Demographics Referral history & pragmatics Relationships/support Clinical history (previous therapies & current medication) Medication at assessment Identified problems/concerns Risk Result of assessment
Demographics Dates and session data Relationships, support & social situation Previous MH service use or other formal supports Medication
Reason for referral Problem categories with duration Risk & ICD-10 Coping actions Assessment outcome
CORE-A EoT covers … Therapist ID Duration & type of therapy Frequency of sessions # sessions attended & not Type of ending Change in initial problems/concerns Change in initial risk assessment Benefits of therapy, motivation alliance & PM Medication at termination
Demographics & dates Modality/theory Format Frequency Ending
Problems: rated severity and whether focus of work Risk Benefits of therapy Medication change Follow-up
Not just a measure : short forms CORE SF-A & SF-B –18 item parallel short forms GP-CORE –15 items for general population, non-clinical surveys CORE-10 CORE-5
Not just a measure : related measures LD-CORE – for people with learning difficulties –Short (15 items) YP-CORE – for young people –11-17 age range –10 items SCORE – for families (rate family not individual) –Short (15 items) –Long (28 items)
Translations Strong translation protocols for CORE-OM and for YP-CORE Not The old way : translation/backtranslation Had the feeling of butterflies in your stomach The CORE way: multiple forward translation and a focus group & then field checking Then, and only then, do some conventional psychometric validation (=exploration) Approved translations must be done with me and follow this protocol
Počutil sem se nevzdržno osamljeno in izolirano. Počutil(-a) sem se strašno sam(-a) in osamljen(-a). Počutil(-a) sem se strašno samo in izolirano. Počutil sem se zelo osamljeno in izolirano. Počutil sem se grozno sam in osamljen. Počutil sem se strašno osamljenega/samega in izoliranega. Počutil sem se strašno samega in osamljenega. Počutil(-a) sem se grozno osamljeno in izolirano. Počutil(-a) sem se grozno sam(-a) in osamljen(-a). (Počutil(-a) sem se zelo sam(-a) in osamljen(-a). ).
Some translations completed 1.Gujarati 2.Norwegian (Bokmal) 3.Italian 4.Slovak 5.Swedish 6.Icelandic 7.Albanian 8.Greek 9.Dutch 10.Danish 11.Portuguese 12.Croatian 13.Welsh 14.German 15.Lithuanian 16.Polish 17.Turkish 18.Finnish 19.Spanish 20.Argentine Spanish 21.Xhosa 22.Romanian 23.BSL (British Sign Language)
Translations ongoing 1.Serbian 2.Bosnian 3.Kannada 4.Urdu 5.Sami 6.Maltese 7.Estonian
CORE in sign language (BSL)
I have felt tense, anxious or nervous in BSL
CORE system outputs and analyses: theory and attitude
CORE system theory Excellent for work within Evidence Based Practice (EBP) paradigm … but intended primarily to support Practice Based Evidence (PBE) and to link that to EBP
EBP model Professional Consensus (mainly from researchers) Psychotherapy Research (mainly RCT) Patient Care
PBE model Audit of Service Delivery Outcomes Benchmarking Patient Care Process, qualitative, narrative/case research
CORE: EBP/PBE complementarity Efficacy RCTs Audit of Service Delivery Outcomes Benchmarking Improved Patient Care Process, qualitative, narrative/case research Pragmatic RCTs
CORE outputs and attitude: benchmarks
Aggregation and comparison
The (now very old) dataset 6610 records (from >12k): 33 primary care NHS services 40 to 932 records per service Anonymised, voluntary Four components to the data: Therapist completed CORE-A Therapy Assessment Form (TAF) End of Therapy Form (EOT) Client completed CORE-OM At assessment and end of therapy or follow-up
Plotting data: simple proportion
Plotting data: reference lines
Plotting data: add CI for sites
Plotting data: add summary
CORE-6D is a score not a measure Uses the multidimensionality and breadth of the CORE-OM items to select six items, five MH and one more physical 1I have felt terribly alone and isolated 15I have felt panic or terror 16I made plans to end my life 21I have been able to do most things I needed to 33I have felt humiliated or shamed by other people 8I have been troubled by aches, pains or other physical problems Health economic gambling trades against longevity proved reliable Allows QALY valuation of CORE-OM scores (provided you have the item scores, not just the totals) Worst of the 33 states rated at.10 of full health: so each year lived only.1 QALY.
How long are the sessions/groups/meetings. When will this therapy finish. –Number How many of us in the room. How many sessions. –Costs/money How much will it cost. Can I afford to so this .
Bad reasons to mistrust numbers They’re difficult (often true but try living without them!) You can prove anything with statistics (not true!) They can’t measure what really matters (not perfectly, it’s true, but doesn’t stop us trying or shopping for things that matter with money) They can’t measure the subtlety of what we do (ditto, but we expect to be paid for it) I don’t understand statistics, and … … They worry me and make me feel stupid (generally true, sad really, not bad!)
Good reasons to mistrust numbers There are important aspects of mental health and psychotherapy that can’t be reduced to numbers (but anyone who claims everything can be reduced to numbers claims too much!) It is easy to make things look very persuasive with numbers and hide assumptions and problems as … … they seem so certain, seductively certain (true, so we need to guard against seduction) They are cheap and confidential so they displace other, more expensive or difficult things (true, so we need to resist that and mix methods and not assume research = numbers )
What’s good about numbers: aggregating You can do arithmetic with numbers: Add & divide and you’ve got averages You can sort them and get ranges and distributions You can do statistics with them: get a handle on uncertainty: P values & hypothesis testing Confidence intervals and precision of estimation Reliability & psychometrics: quality of fit to a measurement model
The right numbers. Are we financially viable. But must ensure that analyses make sense to whoever is given them: therapists, clients, managers, commissioners, politicians, public.
Blind alleys: Seeking (pseudo) certainties
Danger alley 1: RCTs of psychotherapies
RCTs and hierarchies of evidence Simply not true that quantitative data is best DBRCTs brilliant for pharmacology but even there they are complemented with practice based evidence (adverse event reporting) RCTs in psychotherapy can never be double blind: essentially uninterpretable Need to fight the idea that aggregated RCT evidence is the only evidence that selects therapies
Danger alley 2: Sessional measurement
(2012). Cautionary notes on power steering for psychotherapy. Canadian Psychology/Psychologie canadienne, 53(2), 131–139. doi: /a
Sessional measurement and cheap data Sessional data is great … … but not if motivated to match (very cheaply) the apparent certainties of RCTs and become the only PBE In the UK it’s used to create the illusion of no missing ending scores (IAPT) In the US (and widely) it’s going to save us from off track therapies ( power steering )
No missing ending data This was an important driver in funding for IAPT –Sessional measurement ensures outcome data –Analysed on last score is the last score basis, in missing data terms this is last score carried forward But … –This assumes a zero decay (and zero growth) model –Enforces a very intensive measurement focus on the work for client and practitioners
same for all (not necessarily, but little literature on use of idiographic or PQ steering) If a nomothetic measure like CORE-OM or OQ-45 the channel was defined by professionals (ultimately) Assumes that what’s important is conscious (or reflected in the conscious completion of the measure) Offers anxiety containment when that may not be ideal All the research to date has used the communication measure as the outcome measure.
Good sessional tracking Fine if it’s clearly outside the therapy to get a change trajectory and ending score and interpreted wisely If it’s used inside the therapy for steering and it’s not the outcome measure for the therapy, and if using it this way is congruent with theory: CBT, BT, some or much systemic therapy Some humanistic therapy Some or much eclectic/integrative therapy
Review Successes of CORE Problems and challenges for CORE and all PROMS Ways forward
Successes Good solid, acceptable nomothetic questionnaires Copyleft & helped set a trend for this Approved in NHS England but not enforced by political control, recommended measures in NHS Scotland Sensible set of short forms Good offshoots: YP-CORE, LD-CORE and SCORE & PSYCHLOPS Translations: good method produced 25 quality translations and increasing Widely used: saved NHS and others > £1M, > 150 publications and increasing Usage still growing, publications too Supported the idea of PBE as a vital complement to EBP Good software (CORE-PC then CORE-Net and now any software) Benchmark principle Mappings to and from BDI-II scores (and more coming) CORE-6D QALY scoring
Probably supported rather unthinking use of PROMs in clinical practice That’s helped malignant political control of therapies in the NHS and even more widely .
Failures/challenges (daytime version) We were naïve about how measures would be used: –seen as linked with unthinking sessional measurement –Start/end only model was always going to be too little and often not at the right times –No real use of contextual information (CORE-A) As a result: hasn’t promoted thoughtful, statistically and clinically sensible use of change measures not really linked to the qualitative processes of therapy lack of income (and other things) severely limited use of the huge CORE datasets that accrued lack of income holds back translations and their psychometric exploration
How many ways can self-report change measures help … (the EAP)
Every EAP member should be using a PROM routinely in practice But that has to be so the data are some help to someone That’s about numbers: if you see few clients, you will help the collectivity but not learn much yourself, join with others seeing few clients, use measures at sensible intervals.
Someone has to analyse the data and sensibly Take context into account: don’t compare radically different clients, nor radically different patients/clients, nor radically different therapists Combine it with other data: qualitative data.
Try to use it with del cuore del CORE Think about how you will use the data, who will see scores and who will see what analyses … Think about politics but don’t lose your integrity (please, not just PR work!) And feel clear in your guts that your clients have an understanding of what you want to do but don’t let that dominate their thinking and their therapies (remember del cuore ).
In the light of the above … Think about when it makes sense to use which form within your therapies, get follow-up if possible Weave the stories around the numerical data Join a network, link with friends and trusted colleagues, challenge each other.
Ways forward: (1) Weaving quantitative & qualitative
–Need to understand how to weave them together more creatively and robustly – rigorous idiography methods to put qualitative in higher status with quantitative sceptics.
Quantitative warp/qualitative weft
(2009). Measuring psychological outcomes after cognitive behaviour therapy in primary care: a comparison between a new patient-generated measure PSYCHLOPS (Psychological Outcome Profiles) and HADS (Hospital Anxiety and Depression Scale). Journal of Mental Health, 18, 169–177. doi: / Ashworth, M., Robinson, S. I., Evans, C., Shepherd, M., Conolly, A., & Rowlands, G. (2007). What does an idiographic measure (PSYCHLOPS) tell us about the spectrum of psychological issues and scores on a nomothetic measure (CORE-OM). Primary Care and Community Psychiatry, 12, 7–16. Ashworth, M., Robinson, S. I., Godfrey, E., Parmentier, H., Shepherd, M., Christey, J., … Matthews, V. (2005). The experiences of therapists using a new client-centred psychometric instrument, PSYCHLOPS (Psychological Outcome Profiles). Counselling & Psychotherapy Research, 5, 37–42. doi:Journal-Article Empirical-Study Qualitative-Study Journal Peer-Reviewed-Journal Ashworth, M., Robinson, S. I., Godfrey, E., Shepherd, M., Evans, C., Seed, P., … Tylee, A. (2005). Measuring mental health outcomes in primary care: the psychometric properties of a new patient-generated outcome measure, Psychlops (’Psychological Outcome Profiles’). Primary care mental health, 3, 261–270. Ashworth, M., Shepherd, M., Christey, J., Matthews, V., Wright, K., Parmentier, H., … Godfrey, E. (2004). A client-centred psychometric instrument: the development of PSYCHLOPS. Counselling & Psychotherapy Research, 4, 27–33. Barkham, M., Evans, C., Margison, F., McGrath, G., Mellor-Clark, J., Milne, D., & Connell, J. (1998). The rationale for developing and implementing core outcome batteries for routine use in service settings and psychotherapy outcome research. Journal of Mental Health, 7, 35–47. Barkham, M., Leach, C., Lucock, M., Evans, C., Margison, F., Mellor-Clark, J., … McGrath, G. (2001). Service profiling and outcomes benchmarking using the CORE-OM: Toward practice-based evidence in the psychological therapies. Journal of Consulting and Clinical Psychology, 69(2), 184– 196. Barkham, M., Gilbert, N., Connell, J., Marshall, C., & Twigg, E. (2005). Suitability and utility of the CORE-OM and CORE-A for assessing severity of presenting problems in psychological therapy services based in primary and secondary care settings. British Journal of Psychiatry, 186(MAR.), 239– 246. Barkham, M., Connell, J., Miles, J. N. V., Evans, C., Stiles, W. B., Margison, F., & Mellor-Clark, J. (2006). Dose-effect relations and responsive regulation of treatment duration: The good enough level. Journal of Consulting and Clinical Psychology, 74(1), 160–167. Barkham, M., Bewick, B., Mullin, T., Gilbody, S., Connell, J., Cahill, J., … Evans, C. (2012). The CORE-10: A short measure of psychological distress for routine use in the psychological therapies. Counselling and Psychotherapy Research, 1–11. doi: / Bracken, P., Thomas, P., Timimi, S., Asen, E., Behr, G., Beuster, C., … Yeomans, D. (2012). Psychiatry beyond the current paradigm. The British Journal of Psychiatry, 201(6), 430–434. doi: /bjp.bp El-Rufaie, O. E. F. A., & Absood, G. (1987). Validity study of the Hospital Anxiety and Depression Scale among a group of Saudi patients. British Journal of Psychiatry, 151, 687–688. Evans, C., Margison, F., & Barkham, M. (1998). The contribution of reliable and clinically significant change methods to evidence-based mental health. Evidence-Based Mental Health, 1(3), 70–72. Evans, C., Mellor-Clark, J., Margison, F., Barkham, M., Audin, K., Connell, J., & McGrath, G. (2000). CORE: Clinical Outcomes in Routine Evaluation. Journal of Mental Health, 9(3), 247–255..
(2002). Towards a standardised brief outcome measure: Psychometric properties and utility of the CORE-OM. British Journal of Psychiatry, 180(JAN.), 51–60. Evans, C., Connell, J., Barkham, M., Marshall, C., & Mellor-Clark, J. (2003). Practice-based evidence: Benchmarking NHS primary care counselling services at national and local levels. Clinical Psychology and Psychotherapy, 10(6), 374–388. Evans, C., Ashworth, M., & Peters, M. (2010). Are problems prevalent and stable in non-clinical populations. Problems and test- retest stability of a patient-generated measure, PSYCHLOPS (Psychological Outcome Profiles), in a non-clinical student sample. British Journal of Guidance & Counselling, 38(4), 431–439. doi: / Evans, C. (2010). Death, taxes, certainties, groups and communities; or NICE and the deathly hallows. Therapeutic Communities, 31, 321–327. Evans, C. (2012). The CORE-OM (Clinical Outcomes in Routine Evaluation) and its derivatives. Integrating Science and Practice, 2(2). Retrieved from Evans, C. (2012). Cautionary notes on power steering for psychotherapy. Canadian Psychology/Psychologie canadienne, 53(2), 131– 139. doi: /a Kelly, V., Holttum, S., Evans, C., & Shepherd, M. (2012). Social constructions of PSYCHLOPS (psychological outcome profiles) in the context of CBT for psychosis. Counselling and Psychotherapy Research, 1–10. doi: / Mace, C. (2006). Setting the world on wheels: some clinical challenges of Evidence-Based Practice. Group analysis, 39(3), 304–320. doi: / Margison, F. R., Barkham, M., Evans, C., McGrath, G., Clark, J. M., Audin, K., & Connell, J. (2000). Measurement and psychotherapy: Evidence-based practice and practice-based evidence. British Journal of Psychiatry, 177(8), 123–130. Mavranezouli, I., Brazier, J. E., Rowen, Donna, D., & Barkham, M. (2012). Estimating a Preference-Based Index from the Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM): valuation of CORE-6D. Medical Decision Making, 20, 321– 333. doi: / X Mavranezouli, I., Brazier, J. E., Young, T. A., & Barkham, M. (2010). Using Rasch analysis to form plausible health states amenable to valuation: the development of CORE-6D from a measure of common mental health problems (CORE-OM). Quality of Life Research, 20, 321–333. doi: /s
(2009). The NICE Depression Guidelines and the recovery model: is there an evidence base for IAPT. Journal of Mental Health, 18, 405–414. doi:doi: / Robinson, S. I., Ashworth, M., Shepherd, M., & Evans, C. (2006). In their own words: a narrative-based classification of clients’ problems on an idiographic outcome measure for talking therapy in primary care. Primary care mental health, 4, 165–174. Rogers, K. D., Young, A., Lovell, K., & Evans, C. (2013). The Challenges of Translating the Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) Into British Sign Language. Journal of Deaf Studies and Deaf Education, 18(3), 287–298. doi: /deafed/ent002 Rogers, K., Evans, C., Campbell, M., Young, A., & Lovell, K. (2013). The reliability of British Sign Language and English versions of the Clinical Outcomes in Routine Evaluation - Outcome Measure with d/Deaf populations in the UK: an initial study. Health & Social Care in the Community, n/a–n/a. doi: /hsc Sinclair, A., Barkham, M., Evans, C., Connell, J., & Audin, K. (2005). Rationale and development of a general population well-being measure: Psychometric status of the GP-CORE in a student sample. British Journal of Guidance and Counselling, 33(2), 153–173. Twigg, Elspeth, Mick Cooper, Chris Evans, Elizabeth S. Freire, John Mellor-Clark, Barry McInnes, and Michael Barkham. ‘Acceptability, Reliability, Referential Distributions, and Sensitivity to Change of the YP-CORE Outcome Measure: Replication and Refinement’. Child and Adolescent Mental Health currently online preprint only (2015). doi:doi: /camh

Economic Indicator: Definition and How to Interpret

Use Change Metrics to Reduce Change Management Risks

/thmb/3x0sL27fLZ1TrSyEjDdkDAkw

4 Self-Report Measures and Symptom Validity Tests, Psychological Testing in the Service of Disability Determination

A comparison of direct versus self-report measures for assessing physical activity in adults: a systematic review, International Journal of Behavioral Nutrition and Physical Activity

How many ways can self-report change measurement help psychotherapy? Learning from CORE Chris Evans EAP SARC conference “Connecting. - ppt download

A comparison of direct versus self-report measures for assessing physical activity in adults: a systematic review, International Journal of Behavioral Nutrition and Physical Activity

How many ways can self-report change measurement help

Change management framework – How to Measure Change - Celerity

Rethinking self-reported measure in subjective evaluation of assistive technology, Human-centric Computing and Information Sciences

[Vanek, Jeff] on . *FREE* shipping on qualifying offers. Somehow I Thought I Would Be Taller: Finding the Courage You Need to Grow

Somehow I Thought I Would Be Taller: Finding the Courage You Need to Grow