Clinical accuracy of RIFLE and Acute Kidney Injury Network (AKIN) criteria for acute kidney injury in patients undergoing cardiac surgery

1The consensus criteria for acute kidney injury (AKI) developed by the Acute Dialysis Quality Initiative (ADQI) group, first published in 2004 and named with the acronym RIFLE (risk, injury, failure, loss of kidney function, and end-stage renal failure), represent the first concerted effort to overcome the variety of different definitions for AKI. More recently, a modified version was proposed by the Acute Kidney injury Network (AKIN). The two definition systems for AKI differ in three major aspects. (I) In RIFLE the diagnosis is based on changes over a one-week period, while AKIN requires only changes within a designated 48-hour period. (II) Estimated glomerular filtration rate (eGFR) criteria are not included in AKIN. The percentage change from baseline serum creatinine (sCr) is identical in both definitions, aside from the additional criterion of an absolute sCr increment of ≥0.3 mg/dL within the AKIN stage 1 category. (III) Renal replacement therapy (RRT) in RIFLE was not assigned to a given severity class, whereas by definition all patients with RRT are placed in stage 3 by AKIN. Urinary output criteria are identical by both definition sets.

2Our aim was to conduct a detailed comparison of RIFLE and AKIN criteria in a cardiac surgical patient cohort.

3After institutional review board approval (No. 09-001950, study specific informed consent was waived), we retrospectively investigated all patients who underwent cardiac surgical procedures with cardiopulmonary bypass (CPB) at Mayo Clinic Rochester within a three-year period (2005 to 2007). Patient data were recorded in a prospective computerized database. Patients with dialysis prior to surgery or with baseline sCr >3.0 mg/dL were excluded, as were those supported by extracorporeal membrane oxygenation (ECMO) preoperatively, patients undergoing cardiac/lung transplantation, assist device insertion or thoracoabdominal aortic repair. In accordance with Health Insurance Portability and Accountability Act (HIPAA) guidelines, patients who denied access to their medical records for the purpose of research were not considered for analysis. In patients who had more than one cardiac procedure with CPB during the study period at our institution (n = 42) only data related to the first operation were included. Patients who died intraoperatively or within 48 hours postoperatively were excluded (n = 30). In addition to available data each patient`s record was reviewed in order to document the following variables for study purposes: latest preoperative sCr values and sCr at each day postoperatively up to day seven (POD (postoperative day) 1 to 7). For baseline sCr the last value before surgery was recorded. If more than one sCr was measured per day in the postoperative period the highest recorded value was used for study purpose. One patient was excluded due to missing the preoperative sCr. The study cohort consisted of 4,839 patients. Post-hoc three patients were excluded who had RRT planned postoperatively.

4For the definition of AKI the sCr values at baseline and on POD 1 to 7 were used. AKI was defined by the RIFLE criteria using the maximal change in sCr and estimated glomerular filtration rate (eGFR) during the first seven postoperative days compared with baseline values before surgery. eGFR was calculated with the simplified Modification of Diet in Renal Disease (MDRD) formula. Patients were stratified according to the highest RIFLE class R (Risk), I (Injury), or F (Failure) attained by sCr or eGFR criteria. Applying AKIN criteria, the strict definition was used for increments of sCr within a designated 48-hour interval (2). We compared daily sCr value with subsequent levels for the next two days until POD7 to create a moving 48-hour time window (preoperative baseline to POD 1 and 2, POD 1 versus POD 2 and 3, and so forth). The most severe degree of AKI was recorded as final AKIN stage. The RIFLE and AKIN definition criteria utilized are summarized in Table 1. The three thresholds for RIFLE and AKIN are called "classes" and "stages", respectively. All thresholds of RIFLE also included any patient who required RRT during the first seven days postoperatively (POD 1 to 7), whereas by definition these patients were located in stage 3 by AKIN. We did not use urine output criteria in defining AKI.

5Definitions for outcome variables were the following: RRT as outcome variable included need for RRT during the entire postoperative hospital stay or within 30 days postoperatively. Operative mortality was in-hospital mortality or 30-day mortality. Prolonged intubation included those requiring ventilation for >24 hours postoperatively. Length of hospital stay was defined in days from index surgery to discharge.

6Descriptive statistics for categorical variables are reported as frequency and percentage while continuous variables are reported as mean ± standard deviation or median (interquartile range) as appropriate. Categorical variables were compared between RIFLE classes or AKIN stages using Chi-square tests and continuous variables were compared using ANOVA or Kruskal-Wallis test where appropriate. Logistic regression models were used to predict the outcome variables mortality and prolonged intubation using RIFLE or AKIN classifications. Linear regression models were used to predict ICU length of stay and hospital length of stay using RIFLE or AKIN classifications. All tests were two-sided with the alpha level set at 0.05 for statistical significance. Data were analyzed using SAS 9.1 software (SAS Inc., Cary, NC, USA).

7The final study cohort consisted of 4,836 consecutive patients, median age was 67 years (range 18 to 100 years), 34% (n = 1,633) were women. Table 2 presents detailed patient characteristics.

8In 65% of patients maximal sCr values during the first week postoperatively were detected during the first two postoperative days. In the entire cohort 96 (2.0%) patients had postoperative RRT, and 62/96 (65%) had RRT within the first seven days postoperatively.

9Significantly more patients were diagnosed as AKI according to AKIN (n = 1,272, 26.3%) than by RIFLE (n = 915, 18.9%) criteria (P < 0.0001). Distribution of patients as well as agreement and disagreement within the different grades of AKI severity by RIFLE classes and AKIN stages are presented in Table 3.

10Both definitions showed comparable and excellent univariate association to all outcome variables with worse outcome by increased severity of AKI, P < 0.001 for each variable (Table 4). Calculating the predictive ability of both definition systems, RIFLE class as well as AKIN stage were found to be significant predictors of increased mortality, prolonged intubation, prolonged ICU and hospital stay using multivariate analysis (P < 0.001 for all variables, Table 5). This was especially true for the mortality endpoint, where patients had an odds ratio of 4.5 (95% CI 3.6 to 5.6) for one class increase by RIFLE and an odds ratio of 5.3 (95% CI 4.3 to 6.6) for one stage increase by AKIN. Both definition sets of AKI showed good discrimination for the prediction of mortality as evaluated by the areas under the receiver operator characteristic curve (AUC): 0.80 (95% CI 0.75 to 0.85) for RIFLE and 0.82 (95% CI 0.77 to 0.87) for AKIN, respectively (Table 5).

11For the sake of further comparison, an explorative post-hoc multivariate model was constructed including all categories of RIFLE and AKIN. Consistent results were seen for all outcome variables except mortality (P < 0.001 for AKIN and without statistical significance for RIFLE). In this explorative analysis, RIFLE seems to have a lower predictive value for mortality than AKIN.

12Patients who required postoperative RRT (irrespective of staging by RIFLE or AKIN) had very poor outcomes with a mortality of 44.8% (Table 6). We found substantial disagreement between RIFLE and AKIN in classification of patients who required RRT (Table 4). Out of all patients with postoperative RRT, 63 (66%) had RRT within the first seven postoperative days and were consecutively classified in AKIN stage 3, whereas only 18 (19%) of these patients were categorized in the failure class by RIFLE. The other patients were distributed to other RIFLE classes (Table 4).

13It is important to note that whereas 9.6% of patients classified as AKIN stage 1 had no-AKI by RIFLE, only 2.3% of patients in RIFLE class R had no-AKI by AKIN (Table 3). These groups were investigated in detail. Both populations demonstrated intermediate levels of the outcome variables (Table 7). Baseline characteristics of both of these groups showed significant differences compared to the patient groups staged as no-AKI. No patient in AKIN 1/RIFLE no-AKI group showed a sCr increase of ≥0.3 mg/dL within the seven postoperative days compared to preoperative baseline, whereas such an increment was observed in 84/112 (75%) patients in the RIFLE R/AKIN no-AKI group. AKI was over-diagnosed by the AKIN definition.

14Furthermore, separating patients according to initial change in sCr between baseline and POD 1, and focusing occurrence of AKI according to both definition sets, the largest disagreement between RIFLE and AKIN was found in the patient group who showed initial decrease of sCr (Table 8). In contrary, only marginal differences could be detected between final definition and staging in RIFLE or AKIN in the patients who had initial increase of sCr or stable sCr values (Table 8).

15The widespread acceptance of consensus definitions for AKI is reflected in the increased utilization of both RIFLE and AKIN criteria in the literature. In order to progress further, establishment of a uniform definition for AKI applicable in a variety of patient populations is necessary. The aim of our study was to conduct an indepth comparison between both consensus definitions in a large retrospective cohort of patients undergoing cardiac surgery at a single center and to determine the influence of the three modifications made from RIFLE to AKIN. Our data demonstrate that important differences exist between the two classification schemes.

16Existing comparative studies are limited for different reasons. The main focus of comparison is most often the ability of both definition systems to predict outcome. However, this was not the original intention of a consensus definition for AKI. The initial aim was to create a uniform definition to help researchers and ultimately clinicians to classify the extent of renal dysfunction and to improve prophylactic and therapeutic measures. Other limitations include various interpretations of the given criteria, heterogeneous time frames of observation, limitation of comparison to changes in sCr not including GFR thresholds, unknown RRT rates, and finally the lack of sufficient number of patients to determine relevant differences.

17In our cardiac surgical cohort, significantly more patients were diagnosed as AKI by AKIN criteria than by the RIFLE definition set. This reflects one of the intentions of the Acute Kidney Injury Network to increase the sensitivity of AKIN compared to RIFLE.

18Among patients defined as AKI by both definitions only limited disagreement occurred in the staging of severity grade. More interestingly and clinically important, the highest disagreement was found in the patient groups defined as AKI by RIFLE but not by AKIN and vice versa. In this situation either the definition system failed to classify the patients as having AKI or a patient was erroneously labeled with AKI but did not have the condition. It is important to analyze these patients further in detail.

19The largest groups were identified in the lowest severity grades (Table 3). First, patients in both groups had poorer outcome endpoints (versus no-AKI patients), however, mortality rates did not differ significantly (Table 7). Second, baseline characteristics of both subgroups (vs. no-AKI) demonstrated that differences in outcome variables are possibly confounded by clinical factors other than AKI. Finally we determined that the overall differences between patients diagnosed as AKI by RIFLE or AKIN are mainly those who had an initial decrease of sCr from preoperative baseline to POD 1 (Table 8). In this group, post-operative sCr values that were lower than preoperative levels could serve as comparison in the 48-hour moving diagnostic window of AKIN. No patient in the AKIN 1/RIFLE no-AKI group had a sCr increase of ≥0.3 mg/dL above the pre-operative baseline within the entire observation period which is why the diagnosis of AKI is questionable (false-positive). The over-diagnosis of AKI by AKIN (accounting for almost 10% in our study cohort) is clearly caused by the moving 48-hour diagnostic interval and can be avoided only by correction of creatinine for fluid accumulation. This problem highlights the peculiarity of patients where positive fluid balance is present (that is, CPB with hemodilution). A physiological decrease in sCr following cardiac surgery is well understood, and our data demonstrate that this may have an important influence on predicting subsequent development of AKI (Table 8). Since no independent "gold standard" for the definition of AKI is available, we performed in our study the described three-step analysis.

20The other patient group diagnosed as AKI class R by RIFLE but not by AKIN, frequently had an increase of sCr ≥0.3 mg/dL (84/122, 75%) compared to preoperative baseline. AKI could not be detected by AKIN due to the inability to obtain the critical threshold of ≥0.3 mg/dL within a 48-hour window. Thus, the number of patients possibly misdiagnosed with AKI by AKIN is more than four-fold higher (9.6% vs. 2.3%) than by the application of the RIFLE criteria.

21The moving 48-hour diagnostic window was introduced in AKIN in order to overcome the limitation of RIFLE, that a diagnosis of AKI can be difficult when a baseline sCr is unavailable. The initially proposed solution used the revised MDRD formula with a suggested near lower limit of normal GFR (75 mL/minute/m2) to estimate baseline sCr. This has subsequently been proven to perform well only when near-normal baseline kidney function is present. It should be noted that preoperative sCr is available in most if not all patients undergoing cardiac surgery. Additional justification for the creation of a 48-hour diagnostic window was to detect an abrupt increase in sCr.

22Logically, discriminating outcomes between patients with and without AKI may help to determine the validity of a definition/staging system. Several authors have discovered in a variety of patient cohorts that the thresholds of AKI severity defined either with RIFLE or AKIN were strongly associated with adverse patient outcome. We also confirmed these findings in our study. Interpretation of these findings is limited by focusing on renal function because a strong association does not prove a causal relationship. However, there is increasing evidence that the kidney is not simply a passive bystander in multiorgan dysfunction.

23Patients who require postoperative RRT have a very poor outcome (Table 6). The different staging of patients who had RRT within the first seven days after surgery in the two definitions of AKI is very obvious (Table 4). Both classification schemes demonstrated good predictive value for outcome variables, however, the stepwise incremental mortality risk by AKI severity stage is better in AKIN. In this respect the predictive value of RIFLE may increase if all patients with RRT are staged in the highest possible class F, as done in the AKIN definition set. Notably, three patients in our study cohort staged in the highest severity class by AKIN but in RIFLE classified as no-AKI (Table 3) did require postoperative RRT.

24The observation period for the diagnosis of AKI in our study was limited to the first seven days postoperatively. A longer time period might potentially alter our results, however we feel this is unlikely because a) the median postoperative stay of survivors was six days and b) AKI beyond POD 7 is more likely influenced by postoperative factors/complications than by renal injury during index surgery. This is in accordance with the ADQI VI consensus statement where the authors advocate a separation into "early" (within the first seven days) and "late" cardiac surgery-associated AKI.

25We have used sCr and GFR thresholds in calculating RIFLE classes. Besides limited accuracy of eGFR in AKI it has been noted previously that the different thresholds given in RIFLE for sCr increase and eGFR decrease may lead to incongruent definition and staging. We have shown very recently in our study cohort, that eGFR threshold (eGFR is directly dependent on sCr) in RIFLE is more sensitive to classify AKI patients than the sCr criteria. This is in accordance with data recently extracted from a pediatric patient cohort. In our study cohort, patients were classified as having AKI in 9.3% with the sCr criteria versus 18.9% with eGFR criteria, respectively. Thresholds for eGFR change in RIFLE have higher sensitivity to detect patients in class R and I, whereas changes in sCr show better sensitivity for RIFLE class F. As proposed in the original RIFLE publication our patients were allocated to the worst possible RIFLE class they attained by either one or another threshold. Using only the sCr criteria in RIFLE may alter our results considerably.

26We did not use urinary output criteria in our retrospective study. These criteria are identical in RIFLE and AKIN for both amount of urinary output and reference time period. By urinary output criteria both definition sets may diagnose and stage patients in corresponding severity classes which would not considerably influence our comparative study design (except for a few patients with RRT who may be located in a different RIFLE class by urinary output criteria). Nevertheless, the lack of urinary output data in our study has to be considered as a limitation, since there is a potential effect of the use or non-use of the urinary output criterion.

27In our study the strict AKIN criteria were applied in the proscribed 48-hour moving window for diagnosis and staging of AKI. However, in the original AKIN paper the authors stated that "although diagnosis of AKI is based on changes over the course of 48 hours, staging occurs over a slightly longer time frame". Despite this difference in time frames, the likelihood is that it would not alter our results remarkably since the relevant difference between AKIN and RIFLE was not detected in the staging but in the diagnosis of AKI.

28In a later publication, the ADQI group suggested for the use of the AKIN definition in clinical practice that the baseline reference sCr value in the postoperative period should be at least measured more than 24 hours after the start of surgery in order to prevent a diluted serum sample being used as reference. In our study, we did not apply strictly to this recommendation. However, first postoperative sCr values collected for study purpose were the sCr values measured the first day after surgery. The sCr values at ICU admission at the day of surgery were not considered in our study. We could demonstrate that the majority of patients (Table 8) undergoing cardiac surgery present lower sCr at the first postoperative day compared to preoperative baseline. A relevant proportion of patients may also have lower sCr compared to preoperative sCr on the following days. In this respect, the above mentioned 24-hour rule seems to be arbitrary. Nevertheless, it should be acknowledged when AKIN is used as definition criteria for AKI in cardiac surgical patients. In the cardiac surgical setting, when almost all patients have known preoperative sCr values it seems to be worthwhile to use this value as reference baseline throughout the first seven days postoperatively, which is in accordance with the RIFLE definition scheme.

29One important finding of our study is the fact that fluid accumulation has to be addressed for accurate recognition and staging of AKI. In cardiac surgical patients the AKIN definition scheme may potentially lead to over-diagnosis of AKI. This is especially important for epidemiologic studies when sCr values at ICU admission after surgery serve as baseline values. It has been recently demonstrated, however, that dilution of sCr by fluid accumulation in critically-ill patients may in contrast also lead to underestimation of the severity of AKI and correction of sCr for fluid balance can improve recognition and staging.

30Our findings are applicable for the cardiac surgical cohort and the detected differences between the both definition schemes of AKI may differ in other setting.

31In summary, balancing limitations and strengths of both consensus definitions of AKI (in the current versions), we favor the use of RIFLE criteria in patients undergoing cardiac surgery. Modification of RIFLE by staging all patients with acute RRT in the failure class F may improve the predictive value of this classification scheme. AKIN applied in patients undergoing cardiac surgery using the suggested 48-hour diagnostic window without correction of sCr for fluid balance may importantly lead to over-diagnosis of AKI. The quest for a uniform definition for AKI persists in its necessity and relevance.

32• The AKIN definition criteria applied in patients undergoing cardiac surgery using the suggested 48-hour diagnostic window without correction of sCr for fluid balance may lead to over-diagnosis of AKI.

33• Modification of RIFLE by staging all patients with acute RRT in the failure class F may improve the predictive value of this classification scheme.

34• Balancing limitations and strengths of both consensus definitions of AKI (in the current versions), we favor the use of RIFLE criteria in patients undergoing cardiac surgery.

35• Correction of sCr for fluid accumulation has to be addressed for accurate recognition and staging of AKI.

36AKI: acute kidney injury; ADQI: Acute Dialysis Quality Initiative; AKIN: Acute Kidney Injury Network; AUC: areas under the receiver operator characteristic curve; CPB: cardiopulmonary bypass; ECMO: extracorporeal membrane oxygenation; GFR: glomerular filtration rate; HIPAA: Health Insurance Portability and Accountability Act; MDRD: Modification of Diet in Renal Disease; POD: postoperative day; RIFLE: risk, injury, failure, loss of kidney function, and end-stage renal failure; RRT: renal replacement therapy; sCr: serum creatinine.

37The authors declare that they have no competing interests.

38LE, RMS, ZL, and HVS were involved in the conception, design, acquisition of data, analysis and interpretation of data, drafting of the manuscript and revising it critically for important intellectual content and final approval of the version to be published. ETC, RCD, and JAD were involved in acquisition of data, interpretation of data, revising the manuscript critically for important intellectual content and final approval of the version to be published.