With funding from a DPUK Discovery Award, researchers at the University of Edinburgh have assessed the existing available methods of measuring cognitive change, concluding that a technique known as item response theory (IRT) offers the most promising route to accuracy and effectiveness.
Adoption of more reliable methods of measuring symptom change could enable quicker and more accurate diagnosis of dementia and mild cognitive impairment. It could also, in a research context, help determine whether a therapeutic intervention in a clinical trial is effective, as well as improving the accuracy of information contained within cohort studies – vital for helping predict who is at risk of developing dementia.
The study is published in the journal Alzheimer’s Research and Therapy.
Lead author Dr Aja Murray, a lecturer in psychology at the University of Edinburgh, said: ‘Determining whether someone has undergone meaningful change in symptoms over time is an important task in dementia research and clinical practice. But it can be challenging to know whether someone’s cognitive function has genuinely changed over time, or whether a change in scores has occurred because the measures being used aren’t reliable enough – for example, through random error or bias.’
The researchers assessed the advantages and disadvantages of available methods for assessing individual-level cognitive change over time through tests or questionnaires. These methods include classical test theory (the traditional method of statistical analysis used in this area) and item response theory (a more modern method that is better at accommodating the fact tests aren’t equally reliable for all levels of functioning). Statistically significant changes over time in these psychometric methods are measured using a framework known as a reliable change index (RCI).
A practical illustration of the different methods – and the different conclusions they can reach – was carried out using data from the Lothian Birth Cohorts, with analysis taking place in the DPUK Data Portal.
The researchers conclude that IRT-based approaches hold particular promise ‘because they have the flexibility to accommodate solutions to a wide range of issues that influence the accuracy of judgements of meaningful change’.
They write: ‘Considerable progress has been made in the development and implementation of methodologies to assess when individuals have undergone meaningful change. IRT-based RCIs hold significant promise in addressing some of the outstanding shortcomings of traditional methods – especially the dependence of measurement error on attribute level. Further empirical studies should consider adopting IRT-based RCIs as an alternative to or alongside traditional RCIs. At the same time, the field could benefit from further methodological work to further evaluate and compare different IRT-based RCIs and to make them more accessible for researchers and clinicians.’