GRSF Risk Model Development

Galileo Risk Stratification can be implemented with a pre-built model or used to calibrate a model that aligns with the capabilities of the care team, the key process indicators used to define success, and risk factors detectable in realtime with available data assets.

Model Development and Validation

The risk stratification model that is described in this document was developed and validated using the Galileo Risk Stratification Framework (GRSF). GRSF provides a “model simulation” environment in which thousands of risk level criteria combinations can be compared using historical data to estimate the relationship between a particular set of criteria and prospective targeted utilization.

The performance of a risk stratification model can be described using a historical simulation by evaluating each risk level in terms of its sensitivity and specificity to targeted prospective utilization.

Model Specificity is described by TPMPM: Targeted Per Member Per Month costs during the 3 months after risk level assignment. Alternatively, specificity can be described by %Pts w PTU: the percentage of patients with any amount of Prospective Targeted Utilization in the same period.

Model Sensitivity is described by % of All $: that is, the percentage of all targeted utilization in the subsequent 3 months that will occur on patients assigned to this risk level.

A summary of the estimated performance of the model described in this document is shown below:

GRSF provides detailed information about the performance of each risk factor that is used to determine risk level assignment. This information can be used to assess the relative value of alternative risk factors that may be considered.

The relative performance of each risk factor used in the model as well as its contribution to the overall performance of its assigned risk level is shown below.

Tuning the Risk Stratification Criteria

The provisional risk stratification model that is described in this document can be implemented “as is” on a new Medicare population. The model can be readily “re-validated” using the Galileo Risk Stratification Framework (GRSF) on the specific target population in order to determine the risk level sensitivity and specificity as well as the cohort sizes that should be expected in the course of a real-time implementation.

There are several reasons that a care team may wish to consider changes to the risk stratification criteria. The full value of risk stratification can only be realized if it is well-aligned to the care model it serves. In many cases, it is expected that the risk stratification model and the care management program co-evolve during their implementation lifecycle as they have many elements that are co-dependent. While tuning the risk stratification model, a care team may address one or more of the following considerations:

Align Risk Levels with Care Team Capabilities. Ideally, the goals and capabilities of the care team should drive the criteria for each risk level. For example, a care team with a pharmacist and social worker may wish to emphasize criteria related to medication therapies and social factors. Alternatively, a team without these roles may prefer that other themes drive risk factor selection.

It is also desirable to have the stratification model implement criteria that will align risk-similar population segments by shared management characteristics. In this manner, a team can devise a treatment protocol for each risk level. If a given recommended risk level is found to commonly include patients that are unlikely to be well-served by the treatment protocol that is developed for patients with a given risk level assignment, the risk level criteria for that risk level can be adjusted.

Adjust Risk Level Cohort Size. Risk levels should identify a target population that is sized according to the capacity of the care team. If the estimate provided by GRSF for the number of patients in a given risk level is too large for the care team to manage, the threshold applied to each criteria can be adjusted or some criteria can be eliminated.

Construct New Risk Factors that Use New Data Elements. A particular model implementation may have access to clinical data from an EHR or other data sources that can be used to develop customized risk factors. Concepts that can be readily adapted into high-value risk factors include:

• Social determinants of health such as the available of family caregivers, housing security or access to transportation.
• Clinical screening for frailty, fall risk, or mental health.
• Assessments of patient activation and compliance with treatment recommendations.
• Subjective assessments of risk documented by the care team.
• Patient reported outcomes that provide insight into ongoing effect or compliance with long term management.
• New risk factors created from these concepts can be used to augment or replace existing risk level criteria.

Using a Customized Definition of Targeted Utilization. Ideally, the definition of “targeted” utilization should be selected to match the goals of the care team and the key process indicators it will use to measure success. A poorly devised definition of “targeted” utilization can result in the selection of risk level criteria that commonly prioritize patients that are not likely to benefit from the treatments and services that the care team can provide. Further, if the patients that are selected for management are not driving the desired key process indicators, managing these patients will have a muted effect on the intended measures of success.

While the long term goal of many initiatives will include improving broad outcomes and lowering overall cost, focusing on a specific and carefully devised definition of “targeted” solution will make it less likely that the impact of the team is difficult to measure above the variation in overall cost introduced by changes in attribution and the overall risk profile of the population.

The Galileo Risk Stratification Framework (GRSF) provides a robust set of data tools that can be used to weigh the relative value of alternative risk level criteria. GRSF uses historical data for a large population to simulate the risk level assignment at monthly intervals. It then measures the targeted utilization that occurs in the months following the assignment. By performing thousands of simulations using slightly different risk factors as selection criteria, GRSF can quantify the “contribution” of each risk factor to the overall model.

For example, GRSF model simulation can be used to assess the impact of adding the risk factor “Hospitalization for a behavioral health condition in the prior 90 days” as a selection criteria for to the “Level 1” risk level.

The GRSF results can be interpreted as follows:

Overall, in the population being used for simulation, on average 9.3 patients are identified using this risk factor.

• On average, the subsequent TPMPM (Targeted Per Member Per Month cost) of these 9.3 patients is $1,263.
• However, some of these patients are already identified as “Level 1” by the other Level 1 criteria. Given these existing Level 1 criteria, on average only 5.7 patients are added to Level 1.
• The average subsequent TPMPM of the 5.7 patients that would be added to Level 1 is $425. This is considerably different than the TPMPM of the patients that meet this criteria BUT are already included in Level 1.
• If this criteria was added to Level 1, the overall TPMPM for all patients included in Level 1 would fall by $41.8. This suggests a substantive drop in Level 1 specificity if this criteria were to be included in the list of selection criteria.
• If this criteria was added to Level 1, the overall “% of All Targeted $” that is measured for all patients included in Level 1 would increase by 0.25%. This suggests a minor increase in Level 1 specificity if this criteria were to be included in the list of selection criteria.