Description of the ANDI facility
The aim of this project is to develop an Advanced Neuropsychological Diagnostics Infrastructure (ANDI): a web-based infrastructure that uses advanced, multivariate statistical techniques and a large database of normative data of healthy control subjects to solve diagnostic and outcome assessment problems in clinical research. ANDI will allow researchers to more easily locate patients in their samples who, for example, fulfill particular diagnostic criteria, are cognitively impaired or otherwise show an abnormal pattern of cognitive or behavioral characteristics, respond particularly well to a treatment, et cetera. ANDI will also be made accessible for qualified clinical practitioners to assist them in the diagnosis and treatment evaluation of individual patients.
How it works
Users will upload their patient data, i.e. a series of neuropsychological test scores and/or behavior ratings of either a group of patients or of a single patient. Subsequently, ANDI returns a report that documents for each patient separately a detailed, multivariate analysis of the profile of test scores. The report states whether the patient deviates from the norm (e.g. whether he is cognitively impaired), and which of the series of tests are responsible for this deviation from the norm (e.g. in which cognitive domains he is impaired).
The report also gives point estimates with confidence intervals of the frequency with which the patient’s score deviations occur in the normal population. The latter information is essential to decide whether there are dissociations of cognitive functions, a fundamental concept in clinical neuropsychology and cognitive brain research in general. Additionally, the report documents the size and the demographic characteristics of the sample to which the patient(s) was/were compared. The anonymous normative data will come from people who served as control subjects in numerous neuropsychological and other behavioral health investigations that have been conducted in the past decades.
Relevance to clinical behavioral sciences
Many brain diseases are characterized by behavioral symptoms, which can be devastating to the patient. It is therefore of great importance for clinicians and researchers to have sensitive methods to evaluate the behavioral symptoms necessary for diagnosis. Current behavioral rating scales and checklists are quite suitable but they are typically stand-alone instruments with stand-alone normative data. Which means that in the diagnostic procedure the tests are used separately from each other. From a statistical stance, this is a univariate approach. The diagnostic task, however, is of a multivariate nature, because with few exceptions multiple cognitive or behavioral characteristics have to be evaluated before a diagnostic or evaluative decision can be made. Clinicians generally solve this multivariate task in an intuitive manner, because of a lack of formal solutions.
During the last years powerful statistical techniques have been developed that may assist clinical researchers and practitioners to solve these multivariate diagnostic problems. Professor John Crawford of the University of Aberdeen and his colleagues in the United Kingdom, and Dr Hilde Huizenga of the University of Amsterdam and her co-workers have been particularly productive in developing multivariate techniques for statistical analysis of single case and small group data, and in adapting these techniques to the behavioral sciences. For example, Huizenga et al. (2007) devised the method of multivariate normative comparison (MNC), which compares multiple characteristics of an individual to a group of normal control subjects or some other reference group. The method takes the individual’s profile of characteristics into account, and allows drawing a decision on whether or not the individual deviates from the controls.
There seems to be sufficient proof-of-principle evidence of the superiority of the multivariate techniques over conventional methods. Yet, the research community does not use these methods intensively. One of the barriers is probably that the software programs that accompany these methods lack sufficient flexibility and user-friendliness. Another barrier is that these programs require the researcher or practitioner to upload not only his patient data, but also his own normative dataset, or descriptive statistics of a normative dataset. Once the ANDI website is in the air, these barriers will be taken away.
Many cognitive tests and other behavioral instruments of high quality are available. Researchers frequently use these instruments in varying combinations to investigate atypical, i.e. neurological or psychiatric, samples. Besides the patient groups of interest, clinical researchers often include groups of healthy control subjects in their studies. Thus, a goldmine of control data is hidden in the desks and computers of researchers. It only has to be extracted for exploitation. Fortunately, the attitudes on data sharing are changing. Large-scale projects often have formal data sharing policies, and researchers in small-scale studies are increasingly prepared to share their data. Indeed, we found many colleagues prepared – not to say enthusiastic – to participate in a consortium with the aim to fill the ANDI database.
Norms of psychological tests tend to get dated and to turn less and less applicable with the years. This is called the ‘Flynn effect’ after its discoverer, James Flynn, who showed a worldwide increase of scores on intelligence tests with 3 to 5 points per decade (Flynn, Psychol Bull 1987). Should this be of concern to us? After all, we intend to use normative data from studies that have been conducted in the past, some of them even shortly after 1990, while some of the tests we will include are quite old. Many explanations have been coined for the Flynn effect, but none of them has been definitively proven (Hiscock, J Clin Exp Neuropsychol 2007). One of the most likely explanations, however, is the fact that the mean educational level of the population has steadily risen during the 20th century, leading to higher IQ scores in younger than in older age cohorts when they take the same intelligence test. But if test scores are corrected for demographic characteristics, among which the level of education, as is custom in clinical neuropsychology, then one should largely be liberated from the nuisance caused by Flynn effects. Given that demographic correction of test scores will be a built in feature of ANDI’s analyses, we expect that Flynn effects are not a major concern.
Technical feasibility and collaboration
We were able to set up a large consortium of researchers, who agreed to contribute data to ANDI. So far, 24 research groups from 13 universities and other research institutions have agreed to participate in the ANDI consortium. The present ANDI consortium participants will provide normative data from over 65 (neuro)psychological tests and behavior rating scales with a total N of more than 10,000 healthy control persons. Many groups can provide longitudinal data.
We expect the ANDI consortium and/or the contributions of its participants will continue to grow as soon as we start to actually fill the database. As the ANDI project goes on, we will hopefully be joined by more participants, and the current participants will have collected more data.
A steering committee consisting of ANDI consortium participants has been formed. This committee will act as a sounding board of users to safeguard the versatility and user-friendliness of the ANDI website, and will assist in the dissemination of ANDI and its results. One of the advisors to the ANDI project is professor John Crawford from the University of Aberdeen, Scotland. He has given his permission to implement and apply his methods.
Technical expertise additional to what is available in our own group will be obtained from the Netherlands eScience Center. This guarantees state of the art data management, implementation of analytical algorithms and interfaces (website and web services). The centre will take care of the sustainability of the ANDI website after termination of its construction.