Five essentials for evaluating predictive models

Predictive modeling uses your vast store of information to forecast future needs for medical resources. By becoming a knowledgeable purchaser and user of predictive modeling services, you can enjoy a return on your investment in the areas of care management, underwriting and benefit design.

Key Factors for RFP

There's been an explosion of predictive modeling services, each with different methodologies and technology designs. Ineffective predictive modeling— through either poor models or data—wastes your valuable resources and may have a negative impact on your members. However, by understanding how to assess the offerings and apply the technology once you have purchased it, predictive modeling can realize the promise of using information to significantly improve value in health care. The following factors can be used in a Request for Proposal (RFP) to help you select a vendor:


Always ask for the model's R-squared measurement, the commonly accepted measurement of a predictive modeling solution's accuracy. Reliable vendors will know their R-squared measurement.

Vendors should be able to demonstrate both the sensitivity and specificity of their solutions, especially for case management programs. High sensitivity indicates positive predictive value: an ability to identify most of the people who would benefit from a care management intervention. Specificity or negative predictive value is the ability to limit the number of false positives or people who would not benefit from a care management program. Sensitivity and specificity are important so you can assign resources where they're needed most.


Transparency means the ability to differentiate among the data points. For care management programs, transparency means clinicians can look underneath the risk scores to the level of individual claims so they can devise appropriate interventions. A risk score is not particularly helpful for care management nurses; they need a way to understand what's driving the risk. To this end, member profiles should include a listing of all episodes of care and the key services involved in their treatment.

To evaluate transparency in your RFP, ask whether the model is a rules-based or neural net solution. In general, you should look for rules-based models, because they match data patterns to clear clinical rules that identify such things as the disease, type of episode, co-morbid conditions, and drug treatments. In a good rules-based model, you can easily identify these risk markers.

In contrast, neural net or so-called black box algorithms are not clinically based and are technically complicated, so you have to possess real data mining expertise to understand how a specific risk score has been compiled. This robs clinicians of many of the advantages that predictive modeling should deliver for care management. Black box algorithms also make it difficult for you to check the validity of the model.


Your RFP should ask whether the vendor supports your relevant database technologies, so they can load the data quickly and reliably into their model's data mart. You should also ask if supporting databases will be exported to your care management, underwriting, and actuarial applications.

Another key question is how the model defines and groups care— by procedure, diagnosis, or episodes of care. Using fully fleshed-out episodes of care results in better predictions since the groups are clinically homogeneous. This approach takes into account all of an individual's underlying clinical factors, not simply a diagnosis or severity indicator.

Supports operational needs

The solution selected must adapt to your operational issues and must generate predictions as often as your business needs dictate. Also, the data used in the solution must be fresh, reliable, and accessible. In particular, it should be refreshed at least monthly to be available for client renewals.

Finally, the solution must be flexible enough to use the data that is available, e.g., medical only, pharmacy only, medical and pharmacy combined. It should also be able to incorporate emerging data sources, such as lab results.

Industry credibility

One of the most obvious markers of industry credibility is market penetration. The RFP should probe whether others use the solution and if they will speak to its value.

Because predictive modeling is changing and improving at a rapid rate, credibility is not just rooted in the solution itself, but in the ongoing support the vendor offers. Upgrades and support require a team that fully understands not just the technology, but also how health care works. The RFP should check whether the support offered includes an integrated team that brings together IT, clinical, actuarial, and underwriting experts.

The information in this article is subject to change without notice. This article contains proprietary information, which is protected by U.S. and international copyright. All rights are reserved. No part of this article may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, without the express written permission of Ingenix, Inc. Copyright 2006 Ingenix, Inc.

Disclosure and insurability

Disclosure is the process of revealing information. To bind reinsurance coverage, you must reveal claimant data that may not have been available at the time of underwriting. This disclosure is important for identifying chronic situations that represent known risks and is necessary because of the inherent delay between underwriting the risks and binding the coverage. Disclosure is intended to be a quick review of the latest claim activity at the time that a binder for coverage is signed.

Why is disclosure important?

When preparing a quote, the reinsurer performs a careful analysis of claim costs and trends, including an analysis of the current year's activity. A critical assumption is the degree to which this data can be considered complete. Disclosure helps the reinsurer solidify the accuracy of this assumption.

The disclosure also identifies claimants that may be categorized as chronic and, therefore, highly predictable in both the usage and the cost of services. Depending upon the level of predictable costs, certain members may become uninsurable.

Further, the reinsurer may be able to immediately employ a managed care program to assist in the management of these new claims. This, of course, would potentially benefit both you and the reinsurer.

An accurate disclosure is important to you to protect against possible denial of a claim. The disclosure statement is part of the signed binder. Therefore, without full disclosure, the reinsurer has the right to exclude serious losses that were known by the plan, but not disclosed. This may be rare, but the reinsurer does this to protect against the situation where a party knowingly withholds serious losses.

What to disclose?

The disclosure statement (sample below) identifies the claimants that need to be disclosed: any member that is expected to have covered losses that will exceed 50% of the selected specific retention. Limiting this list to those representing a potential serious loss will expedite the process; however, you need to be careful to list all members that are known to you.

No matter how long or short your list, it is critical to provide the following data for each claimant:

  • Diagnosis or diagnoses—allows the reinsurer to identify chronic or ongoing care which is highly predictable in nature.
  • Prognosis—helps identify a near term resolution versus an ongoing situation and helps identify future costs. An estimate of future costs should accompany a prognosis.
  • Charges/claim amount—identifies the magnitude of the claim.
  • Current status and future treatments—supports the information provided in the prognosis.

When these items are provided in a concise but thorough manner for each disclosed claimant, the process can usually be completed very quickly with little, if any, additional discussions of clinical details.

What are the possible outcomes?

Most likely the disclosure will reveal a normal level of catastrophic claim activity of an acute nature, which allows the reinsurer to confirm the terms as originally priced.

Another scenario is that a chronic claim is identified to have a high probability of continuing into the coverage period in question, and a separate deductible may be assigned to that claim if it is likely to exceed the retention. This has now become a known claim to both you and the reinsurer and, therefore, uninsurable. A basic premise of insurance is that known events with predictable costs are not insurable.

A third scenario is that the disclosed claim information is dramatically different from the claim information presented during the quotation process, and the reinsurer is forced either to materially modify its quoted rates or terms or to completely withdraw their quotation. This rarely occurs.

Disclosure Statement by Reinsured

(Used by Summit Re and ERC/Swiss Re)

You agree that any serious losses known by you as of the date you sign this Offer will be excluded from coverage unless previously disclosed to and accepted by ERC. Please enclose with this Offer any serious claim information that has come to your attention so that we may re-evaluate our underwriting. A "serious claim" is defined as any loss known by you for which:

  1. Charges incurred have exceeded 50% of the Specific Retention selected; or
  2. Charges are expected to exceed the Specific Retention selected due to the nature of the illness or injury; or
  3. Any Member remains hospitalized or disabled and is expected to exceed 50% of the Specific Retention.

Information submitted for each serious claim should include the diagnosis or diagnoses, prognosis, charges/claim amount, current status, and future treatments.

What are you, Summit Re?

“What is Summit Re, a broker?” ask some individuals in the industry who haven’t worked with us before. Technically, we are regulated as a Reinsurance Intermediary Broker, which is very different from the retail broker you may have dealt with before. We place reinsurance for health plans, but only for ERC/Swiss Re. And we do so much more: we’re responsible for underwriting each risk, developing and maintaining underwriting and pricing manuals, drafting contracts, processing claims and premium payments, servicing accounts, and maintaining managed care vendor relationships.

The health plan reinsurance marketplace is divided roughly in half between coverages that are delivered directly, which is the way we do business, and those placed through brokers. Which is better? Competition keeps all of us on our toes, but here are reasons we prefer direct distribution.

Deal directly with the decision-makers

Your Summit account team doesn’t just sell a coverage, it prepares and delivers the contract language, pays claims under that contract, and works with your medical management team to reduce current and future medical expenses.

Short distribution chain, low expenses

ERC/Swiss Re retains the risks it writes, so there are no back-end pool and intermediary expenses. Summit provides home office services and sales at a cost comparable to broker loads alone.

It’s a technical sale—we’re a technical company

Summit Re has 3 FSA-level actuaries and 2 CPAs that get involved in your coverage issues. We can tell you we cover LTAC days as standard inpatient days, not restricted step-down days—and be sure we pay the claims that way. Our sales cycle starts with understanding your risk, not just quoting on your current coverage.

Do you work with a retail broker today? You can still get a Summit Re quote. We compete with traditional brokers every day. The broker field is extremely competitive, but the number of reinsurers they have access to is not very large. And that list doesn’t include the largest— Swiss Re, only available through Summit Re.

Analyze Your Reinsurance Coverage

Summit Re uses a proprietary coverage analysis tool, called InSight™, to give CEOs and CFOs the quantitative decision-making data they need to make informed reinsurance decisions. InSight™ is designed to ensure that you obtain the most value out of your reinsurance coverage. Please contact your Summit Re regional vice president for a personalized demonstration of this tool which will allow your plan a better understanding of how to structure coverage to ensure its effectiveness.

A. M. Best Affirms ERC’s Excellent Rating

The following comments are from a GE Insurance Solutions press release regarding a rating review by A.M. Best. “The affirmation of Employers Re'’s financial strength rating by A.M. Best reflects its excellent risk-adjusted capitalization, improving operating performance and its dominant market presence and distribution capabilities. With two years of improving underwriting results bolstered by asset sales and other capital management improvements, as well as significant third-party aggregate stop loss protection, Employers Re has maintained capital in line with an A (Excellent) rating and has exhibited prospective long-term earnings capability stemming from its well diversified business platform. In addition, over the past two years the company has executed underwriting actions, implemented tighter underwriting controls and has maintained its prominence in the worldwide reinsurance markets through its international distribution capabilities.” Fast and Accurate Underwriting

GE Insurance Solutions has developed a web-based rating and underwriting program that allows interactive communication between Summit Re and GE Insurance Solutions. uses a single entry source that helps eliminate errors and yields more efficient underwriting capabilities. Customers benefit from quicker turn-around times on their quotes, as well as more accurate proposals and contracts. *Note: This article was current in 2004, but the URL now houses another site.