Share on
INTRODUCTION
Hip and knee arthroplasties have experienced a significant increase over the last fifteen years. In OECD countries, the rates of total hip (THA) and total knee (TKA) arthroplasties increased by 22% and 35% between 2009 and 2020 and reached an average of 172 and 119 per 100,000 population, respectively [1, 2]. Moreover, future projections indicate that these numbers are expected to grow in the future [3-5]. In Italy, from 2001 to 2021 the average annual increase rate of THA and TKA was 2.3% and 5.4%, respectively [6].
Hospital Discharge Data (HDD) includes administrative, demographic, and clinical information on all the hospital admissions performed at the national level, enabling large-scale epidemiological studies at the population level. Conceived as a tool to define hospital reimbursement for the admission, HDD does not consider specific device and surgery details, which would allow for an effective surveillance of implants. To fill this gap, in 2006 the Italian Ministry of Health and the Italian National Institute of Health (Istituto Superiore di Sanità, ISS) set up, as a joint effort, the Italian Arthroplasty Registry (Registro Italiano ArtroProtesi, RIAP) with the aim of establishing a national data collection system for monitoring outcomes of procedures and safety of the implanted devices [7]. Waiting for the Regulation that will make data collection mandatory [8], the RIAP data collection is currently on a voluntary basis and covers only about 35% of the national volume [9]. Therefore, HDD might represent a useful tool for carrying out epidemiological studies, due to its national coverage of hospital stay records, which equals 98.9% [10].
In Italy, for arthroplasties, HDD have been used in studies measuring risk adjusted mortality after surgery [11]. However, several authors emphasized the need to assess the reliability of HDD in providing clinical knowledge before using these data for secondary research to perform unbiased statistical and epidemiological population-based analyses [12-17]. Currently available Italian validation studies have targeted other sub-populations of HDD [18-20], or are related to arthroplasty but including data on a smaller scale [21] or addressed the type of surgical procedures only [22, 23].
The present paper aims to evaluate the accuracy of clinical information on the type of arthroplasty procedures and diagnoses reported in the HDD when compared with the data collected by RIAP.
MATERIALS AND METHODS
Data source
RIAP collects information from regions and health structures participating on a voluntary basis. The list of regions and structures with details of the period of their participation available in the Italian Arthroplasty Registry Report [9]. The information collected in registry records includes an HDD part, containing patient demographics and data on hospitalization, clinical procedures and diagnoses, and an additional part, referred to as Minimum Dataset (MDS), containing registry-specific variables that cover knowledge specific to arthroplasties and associated devices. Surgical procedures and diagnoses are filled in both parts, but with different levels of detail depending on the different purposes of the variable collections. Procedure and diagnosis variables in HDD are coded using the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) standard, while the MDS has its own coding system [24, 25]. RIAP participating institutions link the HDD and the MDS parts in each record and upload them on dedicated platforms [26]. Subsequently, data is prepared, integrated and cleansed, to flow into the RIAP Ontology-based data layer (composed by an ontology and a relational database) [27].
Study population
The study population consisted of all records stored in the RIAP database, related to hip and knee arthroplasties performed in years 2007-2021 that successfully passed a syntactic and semantic quality check [28] and that report the codes for type of intervention and cause of intervention listed in Table A1 and Table A2 available online as Supplementary Materials. The syntactic check admitted only those records whose values have been entered correctly (i.e., falling within the correct domain as described by RIAP record sets [24, 25]), while the semantic check ensured the compatibility of diagnoses with procedures and of each procedure type with previous procedures (Figure 1) [27].
Statistical analysis
Analysis on procedures
The ICD-9-CM and the MDS codes for procedures were mapped to two categories and five sub-categories related to hip, and two categories and four sub-categories related to knee. The mapping is shown in Supplementary Table A1 available online. If an ICD-9-CM code of interest appeared in any of the 11 available fields of the HDD part of a record, then the procedure was labelled with the category and sub-category associated with that code. For instance, if the code 81.51 was recorded in any of the fields, then it was associated with the intervention category “Primary” and the sub-category “Total primary”. In the same way, the MDS part was labelled according to the code reported for the type of intervention. For example, if the code recorded in the type of intervention field was equal to “A1” in a record of hip arthroplasty, then the MDS part was associated to the category “Primary” and sub-category “Total primary”. The reporting of procedure types by the HDD was checked against the MDS part, which was assumed to be the gold standard as it is specifically designed for the monitoring and the clinical exploration of the arthroplasty domain in Italy. In other words, the MDS labelling was considered as the true value of the variable, while the HDD labelling was considered its prediction. Hence, the correctness of the HDD in reporting clinical information was checked by looking at sensitivity, specificity, accuracy and kappa statistics derived by comparing true and such deterministically predicted values.
Analysis on diagnoses
The analysis of the diagnoses followed an approach similar to the one used for the procedures. The ICD-9-CM and MDS codes were mapped to the list of categories shown in Supplementary Table A2 available online. If a given ICD-9-CM code appeared in any of the six fields available for diagnoses in a record, then the HDD part of such intervention was labelled with the diagnosis. The same occurs with codes used for the cause of intervention (CAU1 or CAUR, depending on the record being considered as primary or revision in the registry database). Also in this case, the correctness of HDD was evaluated using sensitivity, specificity, accuracy and kappa statistics.
Possible geographical differences in the quality of the reported information were explored by a leave-one-out analysis, where the process was iteratively repeated and the evaluation measures were computed while leaving one geographical area (Italian region) out of the analysis at a time. The consistency of the results was assessed by reporting the resulting range of variation for the considered metrics.
The analysis was performed by using the software R version 4.2.3 (2023-03-15 ucrt) - Shortstop Beagle.
RESULTS
Between 2007 and 2021, a total of 578,419 records passed the RIAP quality checks and were related to hip and knee arthroplasties. From those, 46,433 records were excluded as they were related to spacer substitutions or they did not present the codes of interest in the field “cause of intervention”. The final cohort consisted of 531,986 records (345,671 hip and 186,315 knee).
Table 1 shows the distribution of procedures and diagnoses among the analyzed records. Primary arthroplasties accounted for 94.3% of the analyzed procedures for hip (71.7% were total primary arthroplasties and 22.6% were partial primary arthroplasties) and 95.4% for knee. Revisions accounted for 5.7% of hip records (composed by 1.5% total revisions, 3.4% partial revision, 0.7% removal) and 4.6% of knee records (3.4% total revision, 1.1% partial revision, 0.1% removal).
The most frequent diagnoses for primary hip arthroplasties were osteoarthritis (58.8%) and fracture (29.7%), while aseptic loosening was the most common revision cause (2.6%). In knee, fracture cannot occur as a cause of joint replacement and osteoarthritis occurs in 93.8% of cases. Aseptic loosening was the most frequent diagnosis in knee revisions, associated to 2% of knee arthroplasties.
Table 2 and Table 3 report the validity and accuracy data obtained from the analysis of the HDD codes (ICD9-CM) when compared to the MDS codes related to the procedures and diagnoses of interest, respectively.
All the metrics showed a high value for both main procedure categories (primary and revision) with sensitivity and specificity above 95% and accuracy above 99% (kappa 0.93 and 0.95) for both hip and knee. When considering also sub-categories, sensitivity, specificity and accuracy reached values above 90% for all kinds of procedures, except for the lower sensitivity values in partial revisions (69.9% hip; 65.5% knee) and removals (21.9% hip; 33.5% knee).
The large majority of diagnoses had a level of specificity above 98% and accuracy higher than 97% with the only exception of osteoarthritis, which showed lower values: 89.3% specificity and 93.3% accuracy were observed for hip, while 77.1% specificity and 95.8% accuracy were observed for knee arthroplasties. The sensitivity outcomes for diagnoses varied widely, resulting over 96% for osteoarthritis and fracture (in case of hip), equal to 72.9% and 64.4% for infection for hip and knee, respectively, and lower than 60% for the other revision associated diagnoses for both knee and hip.
The leave-on-out analysis confirmed that the geographical area of transmission does not affect the results, in particular when looking at procedure categories and diagnoses with high prevalence. Sensitivity of primary categories reaches 99.04% - 99.68% for hip and 99.15% - 99.64% for knee, while the values for the categories of revision are in the range 93.24% - 97.72% for hip and 92.58% - 96.51% for knee. On the other hand, the values for revision subcategories had wider range of variation by geographical area, but still low levels for the considered metrics, as observed in the overall analysis. The results for procedures from the leave-one-out analysis are reported in Table B1 available online as Supplementary Material. The sensitivity in reporting osteoarthritis is in the range 96.01% - 96.46% for hip and in the range 93.62%-98.59% for knee. The highest sensitivity in detecting causes of revision is infection for both hip and knee, with ranges of variation equal to 71.75% - 77.08% and 62.4% - 69.63%, respectively. The results for diagnoses from the leave-one-out analysis are reported in Table B2 available online as Supplementary Material.
DISCUSSION
This study, performed on large scale and involving more than five-hundred thousand administrative records, shows that the Italian HDD collects accurate information on arthroplasty procedures and the most frequent related diagnoses. The HDD can discriminate between macro-categories of operated joints and type of intervention, either primary or revision. Moreover, it can distinguish between elective and urgent interventions, as clinical information reported on fractures and osteoarthritis is reliable in over than 95% of records. The small portion of misclassified procedures might be explained by the surgeons’ tendency to use the most frequent ICD9-CM codes to indicate the area of intervention (arthroplasty) instead of providing detailed codes that can discriminate procedures at a deeper level. An example of this can be observed when taking into account the relatively low values of sensitivity in partial revision classification (67.99%) and in total revision classification (92.93%) for hip. This could mean that surgeons correctly report a revision procedure, but choose the code for “Total revision” to identify a “Partial revision”. Similarly, the same can be observed for knee revisions, among which “Partial revision” and “Removal” cases have low sensitivity, as the codes of “Total revision” are mostly used to identify “Revision” in general.
This kind of situation is much more common when it comes to the diagnoses. The HDD does not predict the causes of intervention as it does with the procedures, and sensitivity hardly shows values higher than 50% for the rarest diagnoses, which are not always captured. This result might be partly due to the role of “Osteoarthritis”, for which sensitivity and specificity are respectively over 95% and lower than 90%, as it includes the large majority of cases of elective intervention causes. These elective causes are then often misclassified resulting in a very low sensitivity. The opposite happens for the causes of urgent interventions (i.e., “fractures”), which are correctly classified in almost all records of interest.
These findings partially confirm, on a larger and national scale, the evidence found by Baglio et al. [21] on total hip replacements, who found 96.2% sensitivity of the Lazio Region Hospital Information System in reporting primary hip replacements and 89.3% sensitivity in reporting diagnoses with the main focus on osteoarthritis/arthritis and fractures. Also, Roof et al. [29] evaluated how accurate administrative data is in reporting knee revision, in this case by using ICD-10 coding. They found similar results to this study, as they detected a sensitivity of 98% in identifying that a revision procedure has occurred, but the percentage decreases to 76% when predicting the correct type of revision in terms of the replaced component. However, other studies worldwide have shown different results. For example, Bozic et al. [30] assessed quality in reporting hip and knee revision procedures and related diagnosis by ICD-9-CM in four high-volume total joint arthroplasty centers in the USA. They found that sensitivity in predicting clinical information on procedures by administrative data was over 80% only for patellar component revisions in knee and for femoral and acetabular component revision in hip. Looking at the ability of administrative data to correctly predict diagnoses, Rennert-May et al. [31] found a sensitivity of 85% in their study based on ICD-10 coding and focusing on post-operative surgical site infection. Also, Wilson et al. [32], in their paper about accuracy in reporting diagnoses related to knee arthroplasty by ICD-10 coding, found levels of sensitivity comparable to those found in this work.
Other authors investigated the validity of Italian HDD in detecting diagnoses of interest in different health domains and evaluated its reliability for epidemiological purposes. They found good levels of sensitivity in identifying target diagnoses by testing ICD9-CM codes against external source of information, used as gold standard [33-35]. Moreover, from an international perspective, it is worth mentioning the study by Yamana et al. on validity of administrative data in Japan [36]. Indeed, they found a pattern similar to that described in this study, reporting a good performance in correctly detecting procedures and quite low levels of sensitivity in identifying diagnoses.
This study has several potential limitations. First, the RIAP data is assumed to be the gold standard and to correctly report both procedures and causes of intervention. However, no audit was performed to validate the accuracy of the reported clinical information, even though the dataset concerned has successfully undergone the RIAP quality check process. Second, the RIAP data covers just over 30% of arthroplasty procedures performed in Italy, with a non-homogeneous spatial distribution. This is because participation in the registry is on a voluntary basis and some regions were never covered, while others did not ensure a constant participation over time. As a consequence, this could introduce an information bias because of geographic and time specific coding practices with unpredictable impact on results in terms of magnitude and direction. Also, it could limit the generalizability of the findings of the present study, although it is worth noting that, in the study period, 13 regions, two autonomous provinces and seven single hospitals located in three different regions participated in the registry, and half of them provided information for nine or more years [9]. Nonetheless, the present study did not highlight any systematic patterns in the quality of the reported information by geographical area within the regions in RIAP. Indeed, the good reliability of HDD when used at the macro category level for procedures (primaries and revisions) and for identifying elective and urgent hospitalization was confirmed, because of the high levels of sensitivity for “Osteoarthritis” and “Fracture”, which are consistent by region. On the other hand, detailed sub-categories for procedure and rare diagnoses show wider ranges of variation, but always on a low level of performance. Third, the quality in filling HDD may depend on reimbursement patterns and on Diagnosis Related Group (DRG) codes, which are computed as function of procedures and diagnoses reported in the HDD. A thorough analysis of DRG patterns and changes depending on reported ICD9-CM codes lies in the health economics field and is beyond the aim of this study but might be developed as a further work in collaboration with other bodies of the public National Healthcare System. Fourth, despite the number of codes considered covers all cases of clinical interest with over 200 ICD9-CM codes taken into account, relevant codes of interest may have been excluded unintentionally. Last, the approach to mapping between ICD9-CM and MDS codes is arbitrary and different choices of mapping between codes may lead to different conclusions.
In order to overcome coding and mapping issues, a data driven approach is recommended in the future to select ICD9-CM codes of interest in a sound manner and to build a more reliable mapping. In particular, supervised machine learning approaches may be able to detect associations that escape deterministic observation and arbitrary assumptions. The other mentioned issues could be addressed when the RIAP registry will be further developed at the national level.
In conclusion, the Italian HDD related to arthroplasties and their causes performs well in reporting clinical information when used at the level of macro categories, but caution is required when investigating sub categories of procedures and rare causes of intervention. These findings provide evidence for a careful use of HDD for epidemiological studies on arthroplasties and highlight the need to promote a more accurate coding for HDD. Further, developing a mandatory countrywide arthroplasty registry is crucial to obtain a more accurate picture of arthroplasty, not only to ensure an efficient implant surveillance, but also to allow comparisons and fill research gaps, pursuing the ultimate goal of a better clinical practice to enhance patient safety.
Other Information
Acknowledgements
This study was carried on within the following projects:
“Italian implantable prostheses registry (RIPI): realisation of a platform integrating data flows for orthopaedic prostheses, spinal devices, pacemakers and defibrillators, heart valves”, coordinated by the Italian National Institute of Health and supported by the General Directorate of Medical Devices and the Pharmaceutical Service of the Italian Ministry of Health.
“Italian implantable prostheses registry (RIPI): realization of a platform integrating data flows for orthopedic prostheses, spinal devices, pacemakers and defibrillators, heart valves,” coordinated by the Italian National Institute of Health and supported by the General Directorate of Medical Devices and Pharmaceutical Service of the Italian Ministry of Health and within the “Project ECS 0000024 Rome Technopole – CUP B83C22002820006, NRP Mission 4 Component 2 Investment 1.5, Funded by the European Union – NextGenerationEU”, Partner Italian National Institute of Health, Spoke 2 (Technology transfer, new entrepreneurship, business incubation and acceleration) and Flagship project FP4 (Development, innovation and certification of medical and non-medical devices for health).
The Authors acknowledge Mascia Masciocchi, Attanasio Cornacchia and Alessia Biondi for their management support of the RIPI project.
Authors’ contributions
ECi and MT: concept of the study; ECi: design of the study, data analysis and statistics; SAM, RV and ECa: data management, extraction and processing; ECi, PL, TF, PC, SC and MT: drafting of the manuscript; MT: coordination of the project. All Authors contributed to the interpretation and discussion of the results and content of the manuscript and approved the final version to be published.
Conflict of interest statement
The Authors declare no conflict of interest.
Address for correspondence: Enrico Ciminello, Segreteria Scientifica del Presidente, Istituto Superiore di Sanità, Viale Regina Elena 299, 00161 Rome, Italy. E-mail: enrico.ciminello@iss.it.
Figures and tables
Figure 1. An illustration of RIAP’s quality check process.
| Hip | ||
|---|---|---|
| Procedures | N | % |
| Primary | 326,022 | 94.3 |
| Total primary | 247,717 | 71.7 |
| Partial primary | 78,305 | 22.6 |
| Revision | 19,649 | 5.7 |
| Total revision | 5,343 | 1.5 |
| Partial revision | 11,906 | 3.4 |
| Removal | 2,400 | 0.7 |
| Overall | 345,671 | 100 |
| Diagnoses | N | % |
| Osteoarthritis | 203,125 | 58.8 |
| Reumatoid arthritis | 970 | 0.3 |
| Neoplasy | 806 | 0.2 |
| Necrosis | 10,163 | 2.9 |
| Malformation | 7,894 | 2.3 |
| Fracture | 102,699 | 29.7 |
| Sepsis | 56 | 0.0 |
| Pseudoarthrosis | 309 | 0.1 |
| Aseptic loosening | 8,842 | 2.6 |
| Periprosthetic fracture | 2,373 | 0.7 |
| Dislocation | 2,910 | 0.8 |
| Infection | 1,610 | 0.5 |
| Metallosis | 4 | 0.0 |
| Wear | 1,719 | 0.5 |
| Pain | 1,250 | 0.4 |
| Prosthesis fracture | 313 | 0.1 |
| Ostheolysis | 628 | 0.2 |
| Instability | 453 | 0.09 |
| Rigidity | 187 | 0.04 |
| Overall | 345,671 | 100 |
| Knee | ||
| Procedures | N | % |
| Primary | 177,701 | 95.4 |
| Revision | 8,614 | 4.6 |
| Total revision | 6,427 | 3.4 |
| Partial revision | 1,960 | 1.1 |
| Removal | 227 | 0.1 |
| Overall | 186,315 | 100 |
| Diagnoses | N | % |
| Osteoarthritis | 174,820 | 93.8 |
| Reumatoid arthritis | 1,194 | 0.6 |
| Neoplasy | 97 | 0.1 |
| Necrosis | 1,590 | 0.9 |
| Aseptic loosening | 3,716 | 2.0 |
| Periprosthetic fracture | 151 | 0.1 |
| Dislocation | 182 | 0.1 |
| Infection | 1,766 | 0.9 |
| Wear | 258 | 0.1 |
| Pain | 1,783 | 1.0 |
| Prosthesis fracture | 118 | 0.1 |
| Instability | 453 | 0.2 |
| Rigidity | 187 | 0.1 |
| Overall | 186,315 | 100 |
| Hip | Sensitivity(%) | Specificity(%) | Accuracy(%) | Kappa |
|---|---|---|---|---|
| Primary | 99.58 | 96.39 | 99.4 | 0.94 |
| Total primary | 98.43 | 97.02 | 98.03 | 0.95 |
| Partial primary | 96.56 | 98.85 | 98.33 | 0.95 |
| Revision | 97.22 | 99.56 | 99.43 | 0.95 |
| Total revision | 92.93 | 98.07 | 97.99 | 0.58 |
| Partial revision | 67.99 | 99.56 | 98.48 | 0.75 |
| Removal | 21.92 | 99.89 | 99.35 | 0.32 |
| Knee | Sensitivity (%) | Specificity (%) | Accuracy (%) | Kappa |
| Primary | 99.58 | 95.89 | 99.41 | 0.93 |
| Revision | 96.08 | 99.73 | 99.57 | 0.95 |
| Total revision | 93.43 | 99.35 | 99.14 | 0.88 |
| Partial revision | 65.51 | 99.86 | 99.5 | 0.73 |
| Removal | 33.48 | 99.63 | 99.55 | 0.15 |
| Hip | Sensitivity(%) | Specificity(%) | Accuracy(%) | Kappa |
|---|---|---|---|---|
| Osteoarthritis | 96.12 | 89.3 | 93.31 | 0.86 |
| Reumatoid arthritis | 30.21 | 99.78 | 99.59 | 0.29 |
| Neoplasy | 41.56 | 99.91 | 99.77 | 0.46 |
| Necrosis | 51.57 | 99.22 | 97.82 | 0.57 |
| Malformation | 16.52 | 99.74 | 97.84 | 0.25 |
| Fracture | 97.64 | 97.43 | 97.49 | 0.94 |
| Sepsis | 7.14 | 99.99 | 99.97 | 0.08 |
| Pseudoarthrosis | 62.14 | 99.37 | 99.34 | 0.14 |
| Aseptic loosening | 61.26 | 98.64 | 97.68 | 0.56 |
| Periprosthetic fracture | 54.4 | 99.84 | 99.53 | 0.61 |
| Dislocation | 45.57 | 99.57 | 99.11 | 0.46 |
| Infection | 72.86 | 99.53 | 99.41 | 0.53 |
| Metallosis | 0 | 100 | 100 | 0 |
| Wear | 27.52 | 99.92 | 99.56 | 0.38 |
| Pain | 1.68 | 99.83 | 99.48 | 0.02 |
| Prosthesis fracture | 20.45 | 99.79 | 99.72 | 0.11 |
| Ostheolysis | 21.97 | 99.9 | 99.75 | 0.24 |
| Knee | Sensitivity (%) | Specificity (%) | Accuracy (%) | Kappa |
| Osteoarthritis | 96.99 | 77.15 | 95.76 | 0.67 |
| Reumatoid arthritis | 29.4 | 99.78 | 99.33 | 0.36 |
| Neoplasy | 58.76 | 99.98 | 99.96 | 0.58 |
| Necrosis | 27.92 | 99.87 | 99.26 | 0.39 |
| Aseptic loosening | 27.91 | 99.12 | 97.7 | 0.31 |
| Periprosthetic fracture | 46.36 | 99.98 | 99.94 | 0.54 |
| Dislocation | 6.59 | 98.94 | 98.85 | 0.01 |
| Infection | 64.38 | 99.74 | 99.41 | 0.67 |
| Wear | 14.73 | 99.98 | 99.87 | 0.23 |
| Pain | 20.13 | 99.43 | 98.67 | 0.22 |
| Prosthesis fracture | 21.19 | 99.72 | 99.67 | 0.07 |
| Instability | 10.82 | 99.93 | 99.86 | 0.11 |
| Rigidity | 14.97 | 99.99 | 99.96 | 0.19 |
PDF
