At present there are two viable formats for the Transparency in Coverage data: JSON and XML. In this post we’ll dig into what the JSON files look like.
There are also two types of file:
- in-network negotiated rates and
- out-of-network allowed amounts.
In the following sections I’ll explain the significance of each and dissect some sample data.
In-Network Negotiated Rates
In-network negotiated rates refer to the rates that insurance companies negotiate with the healthcare providers (like doctors, hospitals, and clinics) that are part of their network. These providers have agreed to accept this negotiated rate as full payment for services rendered to patients covered by the insurer.
When a patient sees an in-network provider, the provider bills the insurance company directly. The patient may still be responsible for their portion, such as co-pays or deductibles.
There are a few advantages to these rates for the patients:
- patients typically pay less out-of-pocket when they see an in-network provider;
- insurance usually covers a higher percentage of the cost; and
- no surprise bills, as the rate is pre-negotiated.
| Field | Name | Type | Definition | Required |
|---|---|---|---|---|
| reporting_entity_name | Entity Name | String | The legal name of the entity publishing the machine-readable file. | Yes |
| reporting_entity_type | Entity Type | String | The type of entity that is publishing the machine-readable file (a group health plan, health insurance issuer, or a third party with which the plan or issuer has contracted to provide the required information, such as a third-party administrator, a health care claims clearinghouse, or a health insurance issuer that has contracted with a group health plan sponsor). | Yes |
| plan_name | Plan Name | String | The plan name and name of plan sponsor and/or insurance company. | No |
| plan_id_type | Plan Id Type | String | Allowed values: “EIN” and “HIOS” | No |
| plan_id | Plan ID | String | The 10-digit Health Insurance Oversight System (HIOS) identifier, or, if the 10-digit HIOS identifier is not available, the 5-digit HIOS identifier, or if no HIOS identifier is available, the Employer Identification Number (EIN)for each plan or coverage offered by a plan or issuer. | No |
| plan_market_type | Market Type | String | Allowed values: “group” and “individual” | No |
| in_network | In-Network Negotiated Rates | Array | An array of in-network object types | Yes |
| provider_references | Provider References | Array | An array of provider reference object types. | No |
| last_updated_on | Last Updated On | String | The date in which the file was last updated. Date must be in an ISO 8601 format (YYYY-MM-DD) | Yes |
| version | Version | String | The version of the schema for the produced information | Yes |
Sample in-network rate files in JSON format can be found in the CMS Transparency in Coverage repository. Below is a more extensive sample which illustrates some of the features of these files. The data are not real and are for illustrative purposes only!
Don’t worry too much about the details of the data. Scroll down to the discussion and then refer back to the data as required.
{
"reporting_entity_name": "BandAid Brigade Insurance | Patch Perfect Plan",
"reporting_entity_type": "Insurer",
"last_updated_on": "2023-08-01",
"version": "1.0.0",
"provider_references": [
{
"provider_groups": [
{
"npi": [
1225090087
],
"tin": {"type": "ein", "value": "593582520"}
}
],
"provider_group_id": 0
},
{
"provider_groups": [
{
"npi": [
1639508567
],
"tin": {"type": "ein", "value": "272050459"}
}
],
"provider_group_id": 1
},
{
"provider_groups": [
{
"npi": [
1609314343
],
"tin": {"type": "ein", "value": "160743209"}
}
],
"provider_group_id": 2
},
{
"provider_groups": [
{
"npi": [
1215134309,
1043649635,
1851721047,
1285715185
],
"tin": {"type": "ein", "value": "561844651"}
}
],
"provider_group_id": 3
},
{
"provider_groups": [
{
"npi": [
1467430819,
1265410633
],
"tin": {"type": "ein", "value": "581376434"}
}
],
"provider_group_id": 4
},
{
"provider_groups": [
{
"npi": [
1215134309,
1043649635,
1851721047,
1285715185
],
"tin": {"type": "ein", "value": "561844651"}
}
],
"provider_group_id": 5 🗨 This is a duplicate of provider group 3.
},
{
"provider_groups": [
{
"npi": [
1639508567
],
"tin": {"type": "ein", "value": "272050459"}
}
],
"provider_group_id": 6
}
],
"in_network": [
{
"negotiation_arrangement": "ffs",
"name": "100 TABLET in 1 BOTTLE (80725-004-01)",
"billing_code_type": "NDC",
"billing_code_type_version": "2022",
"billing_code": "80725-0004-01",
"description": "Valium - Diazepam",
"negotiated_rates": [
{
"provider_references": [3], 🗨 This links to provider group 3.
"negotiated_prices": [
{
"negotiated_rate": 4.97,
"service_code": [
"11"
],
"negotiated_type": "negotiated",
"expiration_date": "9999-12-31",
"billing_class": "institutional",
"billing_code_modifier": [],
"additional_information": ""
}
]
},
{
"provider_references": [4], 🗨 This links to provider group 4.
"negotiated_prices": [
{
"negotiated_rate": 4.91,
"service_code": [
"11"
],
"negotiated_type": "negotiated",
"expiration_date": "9999-12-31",
"billing_class": "institutional",
"billing_code_modifier": [],
"additional_information": ""
}
]
}
]
}
]
}
The file starts with a few metadata fields:
reporting_entity_name— the name of the publishing entity (normally the name of the insurer and/or insurance plan);reporting_entity_type— the type of the publishing entity (normally an insurer)last_updated_on— the date (in ISO 8601 format) on which the data were last updated; andversion— the schema version in use (determines the layout of the JSON file).
These are followed by two sections:
provider_references— an array of provider groups; andin_network— an array of rates for products and services.
Provider Groups
Each provider group is assigned an identifier (the provider_group_id) which is only unique within the file. These identifiers do not persist between files or between versions of files.
The provider group consists of a collection of one or more providers (might be an individual doctor, a group of doctors in private practice or a facility like a hospital), each of which is identified by its National Provider Identifier (NPI). Either Type 1 and Type 2 NPIs may be used. The Tax Identification Number (TIN) associated with the group is also specified. The TIN is unique to each group that files a tax submission.
💡 An NPI can refer to an individual doctor or a group of doctors. It’s possible for an individual provider to have an NPI, but to be a member of a group which has a different NPI.
It’s worth noting that a provider may be a member of multiple provider groups. And, furthermore, that there might be duplicate provider groups (for example, provider groups 3 and 5).
Rates
Each set of rates is characterised by a collection of fields:
negotiation_arrangement— the type of reimbursement arrangement (ffs→ fee-for-service;bundleorcapitation);name— the name of the product or service;billing_code_type— the type of billing code (most commonlyNDC,HCPCSorCPT);billing_code_type_version— version of the billing code type (if applicable);billing_code— code to identify the product or service; anddescription— a description of the product or service.
The rates are linked to the providers via the provider_references field. In the example above there are two sets of rates. The first set applies to provider group 3 (NPIs 1215134309, 1043649635, 1851721047 and 1285715185), while the second group is for provider group 4 (NPIs 1467430819 and 1265410633).
The negotiated_prices field gives the details of the rates for a specific set of providers. For example, the rates in the example data above pertain to 1 bottle of 100 tablets of Valium. A rate of 4.97 USD has been negotiated by the insurer for the providers in provider group 3. A slightly lower rate or 4.91 USD has been negotiated with the providers in provider group 4.
Each set of negotiated prices includes the following details:
negotiated_rate— the amount either in USD or as a percentage (which applies will depend on thenegotiated_typefield);service_code— the CMS service code;negotiated_type— indicates hownegotiated_rateshould be interpreted (one of"negotiated","derived","fee schedule","percentage"or"per diem");expiration_date— the date on which this negotiated rate expires (most often this field is"9999-12-31", which indicates that there is no expiration date);billing_class— either"professional","institutional"or"both";billing_code_modifier— a modifier for the billing code (only valid for certain billing code types as specified by thebilling_code_typefield);additional_information— extra information (seldom used).
Out-of-Network Allowed Amounts
Out-of-Network Allowed Amounts are the maximum amounts that an insurance company will reimburse for services rendered by out-of-network providers. It’s sometimes called the “usual, customary, and reasonable” (UCR) rate. If the provider’s charge is higher than the allowed amount, the patient is responsible for the difference. An out-of-network provider will typically bill the patient directly for the full amount. The patient then submits a claim to their insurer for reimbursement. The insurer will reimburse based on the allowed amount, and the patient is responsible for the balance.
The patient has the flexibility to choose out-of-network providers. This would be the case if their preferred provider is not in-network. However, the disadvantage for the patient is that they typically pay more out-of-pocket because the insurer covers a smaller percentage of the cost.
| Field | Name | Type | Definition | Required |
|---|---|---|---|---|
| out_of_network | Out Of Network | Array | An array of out-of-network object types | Yes |
Sample out-of-network allowed amounts files in JSON format can be found in the CMS Transparency in Coverage repository. Below is a more extensive sample which illustrate the features of these files. The data are not real and are for illustrative purposes only!
{
"reporting_entity_name": "BandAid Brigade Insurance | Patch Perfect Plan",
"reporting_entity_type": "Insurer",
"last_updated_on": "2023-08-01",
"version": "1.0.0",
"out_of_network": [
{
"name": "1ST HOSPITAL IP/OBS CARE HIGH MDM 75 MINUTES",
"billing_code_type": "CPT",
"billing_code_type_version": "2023",
"billing_code": "99223",
"description": "Initial hospital inpatient or observation care, per day.",
"allowed_amounts": [
{
"tin": {
"type": "npi",
"value": "1114188000.0"
},
"service_code": [
"21"
],
"billing_class": "professional",
"payments": [
{
"allowed_amount": 40.37,
"providers": [
{
"billed_charge": 1098.75,
"npi": [
1114188000
]
}
]
},
{
"allowed_amount": 1968.75,
"providers": [
{
"billed_charge": 2625,
"npi": [
1114188000
]
}
]
}
]
},
{
"tin": {
"type": "npi",
"value": "1164716510.0"
},
"service_code": [
"21"
],
"billing_class": "professional",
"payments": [
{
"allowed_amount": 40.37,
"providers": [
{
"billed_charge": 681.25,
"npi": [
1164716510
]
}
]
}
]
}
]
}
]
}
The file starts with the same metadata fields as for the in-network rates file. This is followed by out_of_network, an array of Allowed Amount objects.
Allowed Amounts
Each set of allowed amounts is characterised by a collection of fields:
name— the name of the product or service;billing_code_type— the type of billing code (most commonlyNDC,HCPCSorCPT);billing_code_type_version— version of the billing code type (if applicable);billing_code— code to identify the product or service; anddescription— a description of the product or service.
In these files there are no provider groups. Each provider is identified individually along with the associated allowed amount.
For example, a provider (NPI 1114188000) has billed patients 1098.75 USD and 2625.00 USD for the service CPT 99223. In the first case the allowed amount was 40.37 USD and the second it was 1968.75. Another provider (NPI 1164716510) billed a patient 681.25 USD for the same service and the allowed amount was 40.37 USD.
Challenges
I pointed out before that there are a number of challenges when working with these data.
Duplication is a pervasive problem, which needs to be addressed if the data are going to be used to compile detailed statistics. In our experience duplication most often occurs when a billing code modifier or service code (or some other distinguishing factor) has been omitted.
The files may contain invalid JSON. For example, sometimes arrays will have missing items. Dealing with such data might be easier using ijson (an iterative JSON parser).
According to the JSON schema the provider_group_id field should contain a number. Most insurers sensibly use integers. Some, however, take this constraint literally and use floating point numbers. Two approaches for dealing with this reliably are to transform this field to a string or use a hash.
Sometimes, instead of containing the details of a provider group, the provider_references items contain references to other files (using a location key). This is allowed by the JSON schema, but can make handling these files more complicated. In this case the other files contain the data that would typically be found in the provider_groups field.
In other cases the provider_references data might be completely missing. Under these circumstances the provider data is included inline in the in_network array.
These edge cases make it challenging to create and maintain a robust and reliable data pipeline. Apart from the intricacies of the file format, the other obstacle is memory management: the files are large and often require large amounts of memory or fancy techniques to process. There are ways to process these large files that are less demanding of memory, but these are also slower, so there is a trade-off between hardware required (hence cost) and speed (which can also impact on cost).
Conclusion
Navigating the intricacies of the Transparency in Coverage data can seem rather daunting, but understanding the structure and significance of the components of the two file types is a step in the right direction. This post provided a deep dive into the JSON format of two primary types of files: in-network negotiated rates and out-of-network allowed amounts. By understanding the nuances of each format, patients and providers can make more informed decisions and better interpret the data.
It’s worth mentioning that aside from understanding the format of these files, perhaps the largest challenge when dealing with these data is being able to handle the enormous size of the files. Although JSON is a convenient and flexible file format for storing this sort of data, reading and analysing large files can be a formidable task and might be overwhelming for the average consumer.