Scientia vincere tenebras! Science should help us see in the darkness

Medical professionals receive the unique privilege from society to transgress the physical and mental integrity of the patient, conditional on proof of benefit. This was originally an issue for surgeons (the surgical professions), but over time most medical domains have become interventional with catheters or with powerful drugs, and these interventions seriously disturb several physiological systems, leading to morbidity and mortality. Patients undergoing these procedures often navigate in the darkness of uncertainty, most certainly when this damage to their physical and mental integrity is associated with risk. It is, therefore, the duty of society to evaluate this unique privilege for all doctors. This can be done through the evaluation of the operational systems, the application of international therapeutic guidelines or by the evaluation of the outcomes. This manuscript focuses on the outcome analysis. There are many challenges in the evaluation of medical interventions, and adult cardiac surgery provides us with a few interesting but difficult examples of these challenges, as outlined in this editorial. We will address the dilemma of ‘time zero’ and ‘time end’, the conceptual challenges in proving risk and benefit, data and model considerations including the uncertainty of any scientific observation and the ‘fata morgana’ of seeking one model that fits all. The mission of the interventional cardiologist or of the interventional cardiac surgeon is not simply to bypass a few arteries or to replace/repair a valve. Our mission is not organ-specific. Our mission is holistic: to improve the patient's quality and quantity of life. The correct evaluation of quality should be the optimal balance between early risk and late result; ‘late’ should be defined differently for each pathology since every pathology, treated with interventions, has a different time impact on the quality and quantity of life of the patient. Risk is a possible surrogate for quality but with considerable limitations. In life, we sometimes accept larger costs (risks) for larger benefits (quality), and there may be a danger in avoiding early risks associated with a particular treatment, when in the long-term this treatment is associated with greater benefits compared with a treatment with a low early risk. It is optimal if risk is absent, but some would dare to state that there is no direct correlation between risk and quality. A single sternotomy would certainly carry less risk vs multiple valve repairs and extensive coronary reconstructions; this lesser risk is in most patients no surrogate for better quality of care. Society must understand that the longer the period of observation, the greater will be the correlation between quality and benefit as distinct from the risk of the intervention. If society intends to focus on risk with all its limitations, then again we must clearly define this interval with a complete understanding of the clinical practice and the principles of outcome analysis. In the past ‘time zero’ was often ill defined. Possibilities were: the date of the intervention, the moment the patient entered the interventional area, the start of anaesthesia or maybe the start of the actual intervention. In the past, ‘time end’ of the interval was most frequently the date of discharge from the primary hospital or the closure of the first 30 days after ‘time zero’. Hospital stay is not an event but a biased medical decision influenced, also and correctly so, by social and possibly economical drivers, rarely only by medical drivers. The 30-day interval was created when intensive care was not as efficient as it is today and when most patients with serious events would decease within that interval. This interval, therefore, lacks all scientific validity. It is also irrelevant to the patient and society and they have the right to an exact evaluation of risk. The obvious limitation to starting the observation interval at the moment of the intervention is 3-fold: the exclusion of non-referred or refused patients, the exclusion of patients for whom an alternative interventional approach is proposed and the exclusion of morbidity and mortality on the waiting list. Even in 2013 and in Western European Countries, patients are sometimes waiting several years for cardiac interventions. But it needs to be stated that short waiting intervals do not exclude mortality or morbidity on the waiting list (Table 1). Mortality on the waiting list for isolated primary or repeat CABG at KU. Leuven, Belgium by year of surgery and with the added average waiting time in calendar days for the same list. Mortality on the waiting list for isolated primary or repeat CABG at KU. Leuven, Belgium by year of surgery and with the added average waiting time in calendar days for the same list. Modern Medicine demands that therapeutic decisions be made by multidisciplinary teams [1–3] where all possible treatments are discussed, and therefore, the correct ‘time zero’ should be the moment of the multidisciplinary discussion. It should be assumed that all eligible patients are referred according to the latest guidelines to a multidisciplinary team; this is the basis for good clinical practice and should be independent of the hour of the day or the day of the week. As a consequence, all discussed patients should be included in the same dataset, irrespective of the therapeutic pathway chosen. Today, most datasets are procedure- or department-specific and fail to address the correct ‘time zero and end’. The selection of an appropriate and scientifically correct ‘time end’ should be based on outcome science. Only a scientifically defined interval can have legal consequences. So if the project has the intention to study quality of care, then this interval should be as far as possible in follow-up, possibly 5 or 10 years. Similarly, the higher the early risk, the longer the observation interval should extend. If the project has the intention to only study risk of care, then this interval should be based on the mathematical hazard functions. Using this approach, we will come to an evidence-based disease management [4] driven by multidisciplinary teams. The risk, at each moment in time, after a time zero, is usually a curve that starts high and then gradually diminishes until it reaches a plateau; later on, the risk of having the event will often rise again. The correct ‘time end’ is the moment the risk reaches the lowest plateau. For death after coronary artery bypass grafting (CABG), this interval is minimally 2–3 months [5, 6], for valves it might extend even longer, for transcatheter aortic valve implantation mostly 6 months, for some congenital cardiac pathologies it is 1 year, for stroke after CABG it is 10 days and for infarct after CABG it is 5 days. The correct observation interval will vary by pathology, by comorbidity, by therapy and by studied event. If the observation interval is too short, then a considerable number of events will be missed for the analysis and possible inference building. If one wants to define an interval that engulfs most early risks after cardiac interventions or operations, then a 1-year interval will most certainly be acceptable, but carries the risk that for some events the constant or late hazard is already active. The UK National Audit [7] has identified time ‘zero’ as the induction of anaesthesia and time ‘end’ has been defined as the in-hospital stay in the primary hospital. This excludes all events before or after the index hospitalisation and waiting list and does not give insight into the differential therapy and the possible, sometimes correct sometimes incorrect, exclusion of patients from surgical therapy towards interventional therapy (possibly against international guidelines) or refused all interventional/surgical therapy. The UK National Audit focuses on risk during only part of the hazard interval of risk after a procedure. It is, therefore, by definition no evaluation of risk or of quality of care. The UK National Audit does also not pick up in-hospital mortality in secondary hospitals, rehabilitation units or coma centres even though the patients might never have left a care institute after the primary procedure. The question remains whether we are ever able to prove benefit, even if we can apply the proposed time zero and time end. It all depends on the robustness of our knowledge of the natural history of the disease, and this knowledge is limited and usually outdated for most disease entities. This applies in particular to those patients who are asymptomatic, and who usually have a good quality of life and therefore a lot to lose with an early intervention. But it is also challenging to obtain proof of benefit for the new and rapidly growing group of elderly patients for whom lesser invasive treatment innovations have become available in the past decade. This elderly patient group has a lot to gain with an intervention but at what price? Given their relatively short life expectancy, the weighing of costs and benefits of invasive treatment becomes increasingly important as the main goal of treatment is to provide a better quality and quantity of life. The main challenge in this patient domain is to find the right balance between overtreatment, under-treatment and cost containment, as society's willingness and ability to pay becomes increasingly limited. Twenty-first century registries should be designed with a focus on determining this optimal balance in an era in which new technologies emerge continuously, and provide to patients both potential benefit and potential (still largely unknown) harm. Patients are not comparable with laboratory environments where research is done on animals with the same weight, gender, age and genetic origin. Luckily, the human species is rich in variability and if we want to stay within scientific rigour similar to laboratory environments, we need to correct ‘completely’ for this variability. The prediction of rare events and thereby the possible correction for variability can be based on different methods [8], such as statistical forecasting, expert judgment, decomposed judgment, structured analogies, judgment adjustment of statistical forecasting, Delphi method, prediction markets and scenario planning. The project leaders of the UK National Audit have chosen for this purpose the statistical forecasting method. This method is very dependent upon the reference database; a sparse or inappropriate database makes this method unreliable and in addition misplaced causality is often embedded. Therefore, the database will need to register and correct for the variability that possibly influences the studied outcome. This variability is often corrected with the variables we have at our disposition, but rarely with the variables we should have available mathematically in order to create well-performing correction models. We all know the demographic variability in age and gender; but race, as an example, becomes very difficult to classify today and therefore difficult to include in the modelling. It is well known that Afro-Caribbeans have a tendency towards hypertension, but many patients have difficulty to classify their racial roots. Any variability in whatever domain that can possibly influence risk or quality of care needs to be collected and corrected for: whether that is quality of life, diurnal, circadian, frailty, educational, biological, neurological, hormonal, immunological, hepatic, renal, pulmonary, orthopaedic, muscular … and cardiac. It could very well be that we do not even know which variability is needed. Most of this variability is not collected for practical and economic reasons and this means that we cannot correct for this variability. This is a serious issue. If indeed some specific variability would have an impact on risk and/or quality of care from a scientific perspective, then the multidisciplinary team needs to have that information and in the correct format before the patient is given appropriate differential therapy. If urgency does not give time to collect, then this major limitation needs to be included in the inference building around appropriate differential therapy. There is currently a rise in the number of scoring systems using only administrative data, readily available within hospital structures and sometimes already needed for reimbursement of care. They usually have age, gender, urgency and some ‘history of’ variables. In some countries, a limited number of crude clinical variables are available in administrative data. These systems go for pure pragmatism and reject completely scientific accuracy, even though some of them reach area under the curve (AUC) values in the 0.75 range. They are highly appreciated by Governments since they exclude the often less cooperative medical professional from the system. Some variability is so rare that even if we could register it, it would still not be included in a correcting algorithm. Indeed, the laws of statistics demand sufficient density of risk in a certain domain of risk before that variable can be selected. A typical example is body mass index. This is rarely selected in scoring systems, for the simple reason that body mass index has only a minimal impact if it varies between 20 and 40 and also since medical professionals often exclude cachectic or very obese patients from multidisciplinary therapeutic discussions and from therapies. In the past, it was impossible to perform angiographic studies in very obese patients. One needs sufficient number of patients (density) with the extremes of variability to allow appropriate So we need sufficient number of patients with a body mass index of or and or body mass index before this variable can be included in our modelling. This us to include these patients in our reference or or to find for who have variability in frequently scoring systems, in possible between variables. is the in which the influence of variables on outcome is not could be the of and very high body mass index or and of between variables in a model may to of model The project leaders of the UK National Audit have selected the variable list as their variable list. This list has been discussed in of and excludes correction for a considerable number of This is a major limitation that the scientific and thereby the legal If the inference building is then made a through this of correction for variability. In the UK National of patients are but or a patient to or those to induction of This is a major but does not correct or the issue of patients with in The UK National Audit excludes their is known to be very different and is reasons are very difficult to understand and the that care is given at different that patients with certain risk and that the correcting systems are inappropriate to correct or for the between and care. certain are we that they are able to correct for the variability between patients or patients with a or cardiac are studied in a different issue is the of the surgeon as a This is outdated in therapeutic The UK National Audit the surgeon the for the clinical team the Medical professionals should and are in teams and therefore, be in team intensive care and medical professionals are medical of their right and all an of a therapeutic do not which medical professional is to be in the clinical pathway of patient. Therefore, or cannot and should not have any in their In has that not has for surgeons but rarely for the In the UK National surgeons are vs the medical with are from the which is not for or intensive care with should be under its correct A history of renal, or is a but has so in of variability that most variability can be in a format for many This also quality of life and variability of the A patient with or but on therapy for disease or similar will often be similar to a patient who is on with low or This in and thereby in correction the uncertainty stroke and are events that in time and for as as patients They are events and need to be studied under a The same is for intensive care and hospital The leaders of the UK National Audit have selected the variable list. all of these variables domain variability by using rarely This is indeed an The UK National Audit has the of the into and But are mathematical to the of a variable in to an event. The is not corrected by an of the The UK National Audit studies the events in a This The by the UK National Audit that the risk was not or at its lowest possible for variables with values is an appropriate The that the patient has if the discharge data are is a but most highly for as they will their to discharge data. The of that if a model is not by a variable then the model with the number of variables should be selected. This often known as was in by A single variable such as age under a with an appropriate and an appropriate on its the of a hospital mortality model after cardiac surgery up to the and even higher from the variables should improve the and example is the with only variables. This model as well as the The manuscript identified that the (the correct prediction of the patients having the was only for the and for the for the and for the So the is most certainly not that the is good but that it is This very low should not us in the of low of mortality and average clearly even with and but very low only very low values can be of in the of with a and a of of in the of with a and a of So from a statistical perspective, should have as few as possible from a legal perspective, we need to to the medical professional that we will correct for variability. It is that there is a of variables will improve the but possibly correct for variability. So if we to for practical and mathematical reasons the number of we must dare to state that our correction is not complete and we must a for this therefore, not be for by will fail in correcting for variability. In does not with the of the of an to uncertainty values to the and are an statistical method to cardiac surgery mortality and to of can be to therefore, at their and not lesser demands a of to the with the risk that the or our of knowledge do not the variability. are often using the and the The is as the where is vs So the a evaluation and is not by the with this of analysis. the model at the is or the is The of a scoring will upon the of the event. In cardiac most studied events are in the or range. In the of rare we will need to understand in this is It that most scoring systems for scoring rare events have an in the but obtain these values through a correct prediction of the and a very low correct prediction of the patients the event on So the should for information the the such as the the the prediction of an event at one can good most certainly not when the of a medical institute or professional is at risk. could also to the understanding of the of the The and the in of according to the risk It then simple between the group and event This the information the of the scoring system. The with risk in cardiac surgery is that these do not have gradually risk, but that most risk is in the This the The exclusion of some patients this A and is a of but a and is no proof of good The UK National Audit in the information is given the and the or values or The UK National Audit of in of the in the It is an to that one model fits all. For the of a model appropriate at the time of its in the with time and with the of several cardiac surgery our knowledge of cardiac surgery has and we have come to that cardiac surgery an of procedures with and outcomes. In this and with the that mortality risk is even for the of it is highly that we can the model its and its limited number of variables as the model that fits all. For the of there is a need for in the different treatment domains valve and CABG and that these in the of the treatment domain of and with the patient and patient outcome. In addition to there is to for the of patient information and risk In this it is even important to that the disease and to the treatment to of at the cost of in less The UK National Audit has to go for a single surgery This the but also the for correction for variability. patients and coronary patients have different and of these is in the data list. not these different and by not them to come into specific pragmatism has over scientific and therefore legal The UK National Audit very limited information the of the cardiac procedures Some of these will have an early or late outcome and are most certainly possible for quality of care. For primary CABG a very in the of the left artery has been identified gradually the with no information extensive For valve it is still that many centres do not even reach the The is again a very crude and interesting information would be the of and/or and/or extensive and valve in the early This the scientific to study the possible impact of on a towards early risk and the of of vs late benefit. and fits into this is of data is a balance between the of the and the of have studied the whether patient care data quality of care. They stated that the is and that it is possible that there is a at hospital but that the on and remains This possible at hospital can be through and exclusion of high or higher risk patients. It is sometimes correct to patient care to but inappropriate has been in the of of outcome data even when that the might be The UK National Audit has made an in the analysis of early outcome data at and It is of to any medical professional or any to have the of the National Audit but also to understand the of the This National Audit project has chosen to go for but and have been at each in this and these need to be made as as the medical data. can only that are not as but that appropriate care and is then given to these units after of their with appropriate to variability and inappropriate observation interval. units where the events in secondary or the intervals and those units that fail to or fail to will for the moment the of us medical care, the analysis and the of its according to the in patient care, outcome and in us come to a evidence-based disease management and not within therapeutic