In prior posts, we noted that COAs play a significant role in regulatory submissions. In this post we will look closely into the clinician reported outcomes (ClinROs) type.
Medicine is both science and art, and as such it is subjective in nature, as those who practice it have their own perception of the disease and how their patients are responding to the prescribed treatment. Because approvals of regulatory submissions take into consideration patient health outcomes in regard to effectiveness and safety of the new product (or repurposed product), clinical outcome assessments determined by clinicians must follow the scientific method more than the art! Clinical assessments utilizing ClinROs must be carried out by a healthcare professional with specific training in the area in which the rating/assessment is being determined to establish confidence and to ensure reliability in the rating, especially when utilized to determine the benefits and risks of a medical intervention.
The path to having a meaningful and validated ClinRO can be long and it must follow a rigorous scientific process. The FDA has put forth guidelines and workshops to provide clarity and guidance around clinical assessment outcomes (COAs) in clinical trials. Early on, these activities were mostly focused around developing and implementing patient reported outcomes (PROs) leaving some ambiguity around the development of ClinROs. Luckily, the The Professional Society for Health Economics and Outcomes Research (ISPOR)-task force stepped in to provide a framework for developing ClinROs.
“The explicit objective in following good measurement practices is to minimize error in measurement of the concept of interest and achieve clarity in the interpretation of study results when evaluating treatment benefit.“ ISPOR task force (2017)
The ClinRO must be well defined in addressing a health concept, measurable, consistent in its interpretation, and with minimum inter-rater variability. In contrast to PROs, patient perspective in developing ClinROs is not central. Involving clinicians in the design and development of ClinROs is, however, essential. Their professional expertise, understanding of the disease and the nuances of treatment and outcomes provide critical information to the validity of ClinROs.
The development of ClinROs generally follows the PRO development process and falls under 4 major steps. When developing ClinROs to be used as endpoints in clinical trials additional input needs to be considered.
Major steps in developing ClinROs
There are three main types of ClinROs:
Readings. These are assessments where well-trained clinicians use their professional judgement to determine the results after reading tests such as pathology reports, mammography, MRI, x-ray, etc. The decision is presented in a dichotomous manner such as Yes/No and Present/Absent.
Patient input is not needed when developing Reading ClinROs. Concept of measurement, context of use, and confirmation of measurement steps must be well defined, while the content validity step is irrelevant in the development of this type of ClinRO.
Ratings. Clinicians use scales or instruments to rate the clinical assessment similar to patients using PRO instruments. These scales can be continuous or categorical measures with at least 3 levels. The knowledge and expertise of the administering clinician for Rating ClinROs is paramount to the validity of the results. Examples of such ClinROs include the Montreal Cognitive Assessment (MoCA), Karnofsky score, and Psoriasis Area and Severity Index (PASI). The development of Rating ClinROs must follow all four development steps and take into consideration inter-rater variability (see below) which can greatly impact sample size and the iterative development processes.
Clinician Global Assessments. This type of ClinRO concerns the clinician global impression and the clinician global impression of change, and it depends heavily on the professional judgement of the clinician, their knowledge of the disease, the treatment modalities, the patient’s health status, and the methods by which the health status is being assessed. This falls more under the art side of medicine than the science, and therefore when used in clinical trials, they are usually used as secondary or exploratory measures for evaluating the effectiveness of an investigational intervention. Their effectiveness could be enhanced by using them in combination with biomarkers, PROs, etc. It is essential to consider the context of use and measurement properties, which must be well defined, when developing this type of ClinROs.
ClinRO as endpoints
ClinROs are considered surrogate endpoints because they provide insight on and can indirectly measure how the patient feels or functions. However, like all other COAs, when being considered as endpoints, ClinROs must meet the fundamental requirements for an informative measure that addresses how patients feel, function, or survive. Because of that, the FDA encourages the use of PROs as primary and secondary endpoints in clinical trials because they directly provide measures on the patient’s feelings and function. This has proven to be valuable and successful in regulatory submissions. On the other hand, the use of ClinROs as primary or secondary endpoints continues to lag behind. One could argue this is because of the complexity of the clinical evaluation and outcome determination that include, understanding the disease mechanism, disease presentation and manifestation, comorbidities, environmental factors as well as the clinician’s own perception of the disease and the patient. This complex evaluation could lead to greater variability in a measured outcome between different clinicians, what is referred to as inter-rater variability. The ISPOR task force therefore has formulated the steps for developing and validating ClinROs factoring in their potential as study endpoints covered in their 2017 article.
Steps in developing ClinROs as endpoints
The pandemic has rapidly pushed the healthcare and pharmaceutical industries to adopt remote and digital tools to continue providing care and monitoring their patients. Regulatory groups have also provided greater flexibility in how healthcare and clinical trial services can be provided. In the last year alone, there has been a boom in remote patient monitoring technologies in healthcare and clinical trials such as televisit, ePRO, eConsent. This evolution is welcome and provides greater opportunities to innovate and foster more remote assessments that reduce patient burdens related to travel and time off of work for clinic visits, and more likely allows for better healthcare access and increased diversity in clinical trials.
Could ClinROs also ride the wave and be completed remotely? Is that feasible for at least a portion of them? Yes, ClinROs utilize scales, but still require patient-clinician interaction and input from these stakeholders is important. What do investigators and clinicians think about conducting remote clinical assessments? How confident are they in their reliability? At a minimum, ClinROs will have to follow a rigorous testing and validation when being converted to remote assessments. We look forward to including the scientific method to enable robust and rigorous validation to foster the move to remote digital ClinRO options!