Technology Assessment, Outcome Analysis Research, Comparative Effectiveness, and Evidence-Based Radiation Oncology - Perez & Brady's Principles and Practice of Radiation Oncology (Perez and Bradys Principles and Practice of Radiation Oncology), 6 Ed.

Perez & Brady's Principles and Practice of Radiation Oncology (Perez and Bradys Principles and Practice of Radiation Oncology), 6 Ed.

Chapter 98. Technology Assessment, Outcome Analysis Research, Comparative Effectiveness, and Evidence-Based Radiation Oncology

Carlos A. Perez, Edward C. Halperin, and Yolande Lievens

There is worldwide concern about rising medical costs. The problem is particularly acute in the United States, which has led to multiple efforts to institute health care reforms. Health care expenditures constitute about 17% of the U.S. gross domestic product, compared to 6% to 11% for Canada, Germany, Japan, and the United Kingdom.¹ The cost of administration of U.S. health care is approximately one-fourth of the total national expenditure on health care. Health care administrative costs in the United States exceed those in many other developed countries. In 2002 the administrative cost of the federal Medicare program was 3% of the program’s total budget, whereas the administrative costs of the federal and state Medicaid programs was 6.7% compared to 12.8% of total revenues for private health insurance plans. One reason for the low administrative cost of Medicare is its lack of advertising and marketing expenses. Medicare pays for roughly 1 employee for 10,000 beneficiaries, whereas most large private insurers hire 15 or more employees per 10,000 enrollees. The Kaiser Foundation Health Plan, one of the largest and best- known group-health maintenance organizations (HMOs), has self-reported administrative costs of 4%.²

According to the 2011 report of the Organisation for Economic Co-operation and Development, the average annual growth rate in health care spending between 2000 and 2006 ranged from 1.4% in Germany to 10.7% in Korea. With health expenditure of $6,714 per person per year, the United States largely outspent the other countries surveyed.³

Borger et al.⁴ projected that in 2015 the United States will spend >$4 trillion, about 20% of the gross domestic product, on health care. Over the last 30 years overall inflation in the United States was 106%, whereas health care costs grew by 251%.⁵ It is estimated that >$100 billion are devoted yearly to research, development, and regulatory approval of new technologies, but <$1 billion is spent on assessment of these new technologies.⁶ Concern over the magnitude and continued growth of expenditures for health care in the United States has led to economic pressure on health service providers to contain costs.

Powerful forces affect the assessment of new technologies in radiation therapy. Inventors and developers, medical equipment manufacturers, corporate stock holders, and equipment salespeople seek rapid entry of new technologies into the marketplace at a minimum cost for as rapid a financial return as possible. Radiation oncologists desire new technology either for a real or perceived improvement in health care, career development, competitiveness with other practitioners, and/or personal profit. Patients and family members hope to see an improvement in clinical outcomes. Third-party payers seek cost reductions or stabilization, stable budgetary expenditures, and a plausible explanation of the “medical necessity” of the new technology. Finally, policy makers seek to balance the competing pressures that they feel from medical equipment manufacturers, physicians, and patients and the overall need for cost containment.⁷

Historically, regulatory approval of new drugs and devices was reliant on well-funded developers with a significant revenue stream conducting and funding randomized, prospective clinical trials. Many new technologies in radiation therapy, however, are introduced by small startup companies that are smaller, less well capitalized, and often single-product vendors. They are not capable of funding randomized, prospective trials and they have a powerful financial impetus to bring their product to market as quickly and as profitably as possible.

In 1985 the Blue Cross Blue Shield Association adopted criteria that required “adequate scientific evidence” to determine that technology improved health outcomes. This approach quickly became the norm among public and private payers. In an interesting development, in 2005, the Centers for Medicare and Medicaid Services (CMS) posted on its website draft guidelines describing a new approach to the determination of coverage policies called “Coverage with Evidence Development.”⁷ This proposal argued that Medicare coverage of promising technologies or services could be linked directly to a requirement that the beneficiary be a participant in a trial or registry and that the CMS had statutory or regulatory authority for that requirement. In the United Kingdom there was a similar proposal for funding “only-in-research” programs. Such programs were used for the evaluation of laparoscopic surgery for colorectal cancer, photodynamic therapy for age-related macular degeneration, the National Emphysema Treatment Trial to determine whether lung reduction surgery was superior to pulmonary rehabilitation without surgery, a comparison of balloon angioplasty plus carotid artery stenting versus carotid endarterectomy in patients at high risk for stroke, and the role of positron emission tomography scanning in suspected dementia.⁸

ECONOMIC EVALUATION IN HEALTH CARE

Economic evaluation is routinely becoming an important criterion for evaluation of therapeutic strategies. Unfortunately, consistent criteria are not followed for evaluation of the economics of treatment, which reduces the usefulness and credibility of comparative cost and other financial data/statistics across studies or over time.⁹

Health technology assessment (HTA) plays an essential role in modern health care by supporting evidence-based decision making in policy and practice. HTA involves addressing five questions¹⁰:

1. Can a health care intervention achieve its expected goal when used in optimal circumstances? (Efficacy)

2. Does the intervention do more good than harm when used in routine practice? (Effectiveness)

3. What is the balance between the health outcome obtained and the resources required to deliver the intervention? (Efficiency or Cost-Effectiveness)

4. Is the supply of services matched to locations where they are accessible to persons that need them? (Availability)

5. Who gains and who loses by choosing to allocate resources to one health care program instead of another? (Distribution).

A full economic evaluation typically involves quantitative examination of both costs and outcomes, or consequences, of competing interventions possible, including no intervention. An appropriately performed economic evaluation is incremental, that is, it measures the extra cost incurred to obtain the incremental improvement in outcome. Drummond et al.¹¹ identified different types of full economic evaluation based on the denominator chosen (Table 98.1). The most widely used approach, cost-effectiveness analysis (CEA), compares the incremental costs of a new technology or intervention to the incremental gain or loss of clinical outcome of the new intervention, ideally measured in life-years gained (LYG). Other potential denominators in a CEA are disease-free survival or number of cancers prevented or detected, but the use of such intermediary outcome measures hampers comparison among different CEAs and should therefore be discouraged. The evaluation will result in a ratio of the cost/outcome, for example, cost/life-year gained ($/LYG, €/LYG), called the incremental cost-effectiveness ratio.

TABLE 98.1 LEVELS OF EVIDENCE AND GRADING OF EVIDENCE FOR RECOMMENDATIONS

When the outcome of the evaluated interventions is expected to be the same, the economic evaluation will be limited to a cost comparison called cost-minimization analysis (CMA). Assuming a well-defined comparison of two radiation therapy (RT) approaches and a sufficiently long follow-up, prolongation of life, life-years gained, can be measured, but it can be questioned whether this is the metric that has most relevance to patients. The third type of full economic evaluation, cost–utility analysis (CUA), acknowledges quality of life after anintervention by weighting the LYG with a “utility factor” typically ranging from 0 (death) to 1 (perfect health). For example, if a patient had a utility or preference for a health state of 0.5 and he survived in that state for 1 year, then he would have had a quality-adjusted survival of 6 months. A CUA employs quality-adjusted life years (QALYs) in the denominator, with the result expressed as cost/QALY. Utilities or patient preferences for a certain health outcome can be measured with instruments such as the Health Utilities Index III¹² and EuroQol¹³ or tests such as Time-Trade Off or Standard Gamble.¹⁴ CUAs are helpful in trying to compare nonsimilar health interventions, such as a prostate cancer–screening program with a child immunization program, for example.

The last type of full economic evaluation is cost–benefit analysis, in which both cost and outcome are valued in terms of currency. Where the benefit in monetary units is greater than the cost, the medical intervention is considered worthwhile. Putting a dollar value on the quantity and quality of a life is, however, a highly specialized and controversial area. Full economic evaluation in RT, requiring detailed input on short- and long-term costs and outcomes, can be difficult to perform. One such approach addresses costs and outcomes separately and leads to cost and outcome descriptions, respectively.¹¹ However, there are dangers with separating costs and outcomes, and one has to be cautious that the information generated will not be misused.

The most useful descriptions of outcome of a new treatment strategy, particularly from the patient’s perspective, result from the conduct of clinical trials. Randomized and controlled phase III trials have been categorized as efficacy or effectiveness evaluations.¹¹ It can be argued, however, that such comparisons may indirectly lead to the increased cost of health care as funders come under pressure to broadly adopt new interventions with only marginal clinical benefit and no consideration of the opportunity cost. A pure cost comparison, in contrast, entails the opposite risk. If the tendency to reduce health care spending is strong, this might result in the simple choice of the cheapest approach, which may, in turn, lead to reduced quality, hence inferior outcome. Provided that there is sufficient evidence that the outcome of interest for the experimental treatment or intervention does not differ from the standard treatment, a CMA is undertaken (discussed earlier), that is, a mere cost comparison, with the intervention resulting in the lower cost being the favored one from a health economics’ perspective.

In evaluating medical care, the term efficacy is not interchangeable with effectiveness. The maximum possible reduction in a disease due to the use of medical intervention is properly termed efficacy, which is generally measured with randomized, controlled trials, which will achieve the maximum possible benefit when patients are carefully selected and stratified, randomization arms are properly designed, compliance is high, and trial participants are often free of other diseases or conditions that might interfere with the intervention being evaluated. The benefits established in a randomized, controlled trial are only partially applicable to day-to-day practice in a general population. The value of a medical intervention in the general population is termed effectiveness. The difference between efficacy and effectiveness can be very large, and obtaining a realistic measurement of effectiveness is often difficult.¹⁵

There are three approaches to health economic evaluation for comparing two therapies: (a) cost minimization, in which one assumes or observes no difference in effectiveness, (b) incremental cost-effectiveness, and (c) incremental net benefit. The last can be expressed in units of either effectiveness or costs. When analyzing data from a clinical trial, expressing incremental net benefit in units of cost allows the investigator to examine all the approaches in a single graph, complete with the corresponding statistical inferences.^15a

A computerized literature search of >14,000 articles published since 1994 having cost as a keyword was carried out. Criticisms of these costing studies focus on three areas:

1. Costs are not measured correctly¹⁶^–¹⁸ because most studies use charge or reimbursement dollars as a proxy for cost. This is an incorrect measure because neither value is representative of the resources consumed or foregone in the process of providing a good service; hence, it is not a cost; the former figure is a construct, and the latter is the result of a negotiation between a payer and a provider.

2. Costs are not compiled using a proper perspective and scope.¹⁷^,¹⁹ Most studies only tabulate the cost of the health care provider and not to the patient, the family, or society.

3. There is a lack of consistency in reporting results,¹⁶^,²⁰^,²¹ which prevents interstudy and interinstitutional comparison.²²

As a result of these problems, the data and conclusions from many cost-based studies are suspect.¹⁹

Clearly, a better costing method is required. Russell²³ emphasized that to serve its purpose, a model must produce accurate predictions and low probability for substantial variation in the factors that influence costs and effects. He identified three aspects of modeling: validating effectiveness estimates, modeling costs, and the implications of common statistical forms. Validation procedures similar to those for effectiveness estimates are proposed for costs. Modelers need to pay more attention to ensuring that the events described by a model represent costs and effects. Modelers can also help improve the epidemiologic and clinical research on which cost-effectiveness analyses depend by showing the implications for resource allocation of the statistical forms conventionally used in these fields.

Comparative effectiveness research (CER) aims to improve the quality, effectiveness, and efficiency of health care and to help patients, health care professionals, and purchasers make informed decisions regarding various treatment options, particularly for chronic illnesses or cancer. CER is moving forward, with recently defined priorities and a newly funded Patient-Centered Outcomes Research Institute, which, it is hoped, will survive cost cutting in the U.S. Congress. Economic analyses are most valuable to health policy analysts and health care managers who must allocate resources and establish benefit packages. An economic health care analysis tries to directly relate the incremental cost of an intervention to its potential benefit. The intervention is always evaluated relative to an alternative form of treatment.⁹

Technology assessment, cost–benefit outcome analysis, clinical trials supported by evidence-based medicine, and comparative effectiveness together can enhance the rationale and quality of medical care provided to patients at a competitive cost.

TECHNOLOGY ASSESSMENT

Technological change is the key driver of health care expenditure growth because it is generally associated with increased rather than reduced cost. As the economist Henry Aaron is quoted as saying, “Rapid scientific advance always raises expenditures even as it lowers prices. Those who think otherwise need only to turn their historical eyes to automobiles, airplanes, television, and computers. In each case, massive technological advance drove down the price of services, the total outlay soared.”² A good example in medicine of this phenomenon is laparoscopic cholecystectomy. Although the price of a laparoscopic procedure is less than the price of an open cholecystectomy, after the introduction of the new technology the rates of both types of surgery increased, and the growth in the quantity of services overwhelmed the impact of per-unit reduction in price. Similar examples can be found in the widespread use of magnetic resonance imaging, computer tomography, coronary artery bypass grafting, angioplasty, cardiac intensive care units, neonatal intensive care units, and positron emission tomography and the growth of radiation oncology facilities. Technology may also increase costs by either premature adoption of the technology without adequate evidence that it improves outcome or overuse of technology, regardless of value, to provide a competitive market advantage or to generate revenues.²⁴

Technology assessment is undertaken to determine (a) the appropriateness of adopting a new procedure consistent with the health care organization’s mission and strategic plan; (b) the safety, efficacy, and cost-effectiveness of the new technology; (c) distinctions between appropriate and inappropriate use for the new technology; and (d) quality improvement methods to optimize the use of the new technology.²⁵ Technology assessment is a complex process that involves financial considerations, therapeutic outcome (locoregional tumor control, amelioration of symptoms and survival), treatment-induced morbidity, quality of life, impact on caregivers, family, and coworkers, and lost-opportunity costs.

When new technology emerges and priorities for allocation of increasingly scarce resources are considered, it is imperative to carry out economic analyses and also to assess the positive effects to the population who will benefit from the new technology compared with standard treatment techniques. The U.S. Center for Medicare Management concluded that cost-effectiveness should be considered in approving reimbursement by Medicare for new technology procedures.

Halperin²⁶ pointed out the importance of technology assessment in determining the merits of new devices or modalities and the impact they may have on the cost of health care. In a provocative editorial, Lee²⁷ pointed out that although technology is ubiquitous in radiation oncology, it is our professional responsibility to critically evaluate the merits of new technological developments. Yet, technology assessment can be a thorny and complex subject. Many of the technological advances in radiation oncology have not undergone a strict test of worthiness in formal clinical trials. Simulation, positioning techniques, computed tomography scanning for treatment planning, online imaging, and even intensity-modulated radiation therapy have had multiple dosimetric validations, but no randomized trial has documented their impact on outcome of large numbers of patients. However, lack of this evidence would not lead us to discontinue any of these procedures, which have been shown to improve in some way (i.e., less toxicity) the quality of care provided to patients.

The profusion of new technologies in medicine is related to the number of specialists in an area. Specialists receive income from new technologies and prompt hospitals to invest in them. A hospital that is constructing innovative facilities may attract specialists to the community and improve its revenue. It is also likely that public attitudes are influenced by medical providers and suppliers that advertise new technologies through the mass media.

Many nations have agencies to conduct health technology assessment, and there is an international network of agencies for health technology assessment that can share such data. The United States, however, has no national coordinating body on health technology assessment. Assessments are conducted through organizations such as the Veterans Administration, the Blue Cross Blue Shield Association, and professional organizations like the American College of Radiology, the Medicare Coverage Advisory Committee, and HMOs.

Health care organizations use a variety of models to assess and acquire technology, some of which include outcome. Reliable data for such benchmarking is frequently scarce. Some compare new and emerging technology against existing technology, although reliable outcome data are rare. Most institutions or individuals buy what physicians, advocacy groups, philanthropists, or governing boards want and use manufacturers and physicians as the primary sources of technology information. Cowan and Berkowitz²⁴ pointed out several key mistakes that occur during the diffusion of technology:

1. Technologies are adopted simply because they are new, without assessing their impact on outcomes.

2. Equipment is replaced solely because of its age. To avoid overuse of technology, health care organizations must carefully match the capacity of existing technology with projected utilization in a world of limited resources.

3. Applications of technology expand without adequate justification based on outcome or cost considerations.

4. Numerous vendors are used, and numerous models of products with duplicative specifications are purchased. Reducing the number of brands and models of devices used may provide an opportunity to reduce purchasing and inventory costs, as well as maintenance and repair costs. On the other hand, competitive pressures in the marketplace may provide downward pressure on equipment prices and argue in favor of the existence of multiple vendors.

5. Economies of scale in regionalizing technology decisions are missed because of lack of consolidation of services. This is sometimes related to inadequate systems for patient transport to treatment centers.

6. Duplication of services sometimes adds cost without significant increase in work volume.

A number of studies on the costs of treating cancer patients with radiation therapy²⁸^–³¹ used either relevant costs defined in a number of ways or different proxies for actual costs (billed charges, revenue received, Medicare allowable). Recent reviews illustrate the wide variety of cost information and cost methods used in the literature and call for a consistent method for estimating site- and intent-specific treatment costs.⁹^,¹⁸^,¹⁹^,²²^,³² In estimating costs of health care interventions, several decision tools, including cost analysis, cost-effectiveness, and cost–utility and cost–benefit analysis, seek to define a unit of health outcome per unit input of cost. Before employing any of these decision tools, proper definition and calculation of cost is essential. The literature on the cost-effectiveness of particle therapy for cancer is scarce, studies are often not comparable, and many studies are not performed according to standard health assessment criteria. Evidence on the cost-effectiveness of particle therapy is, therefore, limited. One would hope that adequate reimbursement can be found to support innovative yet costly treatment to learn whether it is truly more efficacious.

TYPES OF COST IN HEALTH CARE

Costs During Treatment

Direct medical costs include the value of all the goods and services consumed in the provision of an intervention or dealing with the side effects of treatment. These costs include the consumption of all capital and human resources such as professional time, diagnostics, treatment, and medications.

Direct non–health care costs include all costs as a consequence of undergoing a treatment but not related to the treatment itself. Examples are the cost of transportation while patients are receiving treatment. Other costs sometimes included are changes in use of informal caregiver time (including the cost of family or other providers to provide health care in the home or continuous nursing care for disabled individuals) and patient time costs (referring to the time patients spend receiving care, including the value of the time consumed in receiving the treatment as well as travel and waiting time).

Indirect medical costs include all costs in the time after the treatment has been terminated and include costs of late toxicity or related diseases. In RT, these costs can differ quite substantially between different approaches if the side effects are very different or if one of the interventions has a higher curative potential. Typically, however, these costs occur many years after termination of the treatment.

Indirect nonmedical costs refer to costs that are important for society, in that they measure loss of productivity due to long-term disability or premature death. Costs to the family or society can be substantial. These include the patient’s travel costs, the patient’s or care provider’s time off from work, and the considerable strain on caregivers. There may also be “downstream” effects. If a caregiver is devoting time to an ill spouse, he or she is taking time away from work or child care responsibilities, for example.

Global cost estimates of the cost of RT are the aggregate costs of the human and capital resources consumed in the delivery of a RT service. They do not contain the fine detail required when comparing, say, one RT protocol with another. However, they can have value as a background to allocating national health care budgets between different classes of medical intervention.

The data show that even in countries with optimal infrastructure, the budget assigned to RT is roughly 5% of the total amount spent on cancer care.³³ In the early 1990s, the European Union estimated the average cost per course of RT at about 3,000€, compared to roughly 7,000€ and 17,000€ for surgical and chemotherapy treatments, respectively. However, in spite of the perception of high capital costs, it is the personnel cost that dominates the cost of the provision of the service (about 70%), with capital investment typically representing <30% of the total RT budget.³^,¹⁴^,³⁴

Reimbursement-Based Costs

Whereas economic evaluation theoretically advocates the use of cost figures that reflect actual resource consumption, this is frequently not achievable, and charges for service are used as a proxy. It should, however, be realized that large discrepancies may exist between the reimbursement of treatments and the costs of resources consumed while delivering a treatment. Due to the fact that reimbursement often lags behind the introduction of novel technologies, in many countries there is a tendency for high-tech treatments to be underreimbursed. The inverse seems to hold for the United States, where the reimbursement is likely to be higher for new technologies, although it decreases with time.

Besides numerical differences between costs and charges, other, more practical and theoretical problems may arise when utilizing reimbursement figures for economic evaluation. This was observed by the Radiation Therapy Oncology Group when performing an economic evaluation based on combining clinical trial data with cost data obtained from Medicare for each patient.³⁵ The first problem was that Medicare-managed care products do not have individual claims because managed care companies receive capitated payments and therefore may not record and/or report individual claims to Medicare. A more accurate estimation of costs would result from using administrative claims data to calculate costs, provided that they contain all claims of care a patient received. Another potential problem is that an underestimation of costs could occur if codes with only a cancer diagnosis were used, thus neglecting costs related to treatment toxicity or indirect medical costs occurring in the further follow-up of the patient. It may also be difficult to generalize the results of an analysis based on Medicare patients older than 65 years to the general population. Finally, it is apparent that obtaining administrative claims data from Medicare can be a costly procedure in itself.³⁶

ACTIVITY-BASED COST ANALYSIS

Activity-based costing (ABC) is a system widely used in manufacturing but rarely in health care. ABC dissects the process of providing patient care into a series of microlevel tasks and assigns costs to these tasks. As a result, ABC directly identifies resources consumed by cost objects (treatments, patients, or health care providers).¹² Tabulating these tasks and resources consumed allows for comparison of cost values across institutions and time in a consistent and credible way. From this, the economic impact of differences in operating policies can be quantified and the comparison of cost to health outcome will be improved.

The traditional cost models that hospitals have used are outdated and ill equipped to provide accurate information on costs of a specific medical procedure or global cost of medical care for a patient from initial diagnosis to death. Cost models break organizations into departments for financial oversight and budgetary control; the focus of these cost models is departmental budgets, and they are not specific for a patient or a procedure, when it is really patients and procedures, not the department, triggering costs. For example, the accounting system of a hospital may be able to provide a fair assessment of the total budget (cost) of its radiation oncology department. However, the same system is of little help in assessing the actual cost of treating a patient with, for instance, prostate cancer using radiation therapy. This lack of applicability of cost information at the patient or the procedure level often leads to faulty decision making because analysis of the whole department does not necessarily translate into data about an individual patient.

FIGURE 98.1. Process flow chart. Activity map and corresponding mean (median) times for definitive irradiation of cancer of prostate. A: Evaluation and management. B: Treatment planning. C: Weekly management

ABC is concerned with measuring resource consumption by an individual patient. There are two general advantages of using ABC: increased accuracy of cost measurements and transparency of cost-incurring sections. The latter advantage—increased accounting transparency—means that there is a more lucid and readily apparent correlation between costs and actions. This allows managers to evaluate each step of the process, perceive inefficiencies in the system, and quantify economic benefits of process improvements.

The cost data of any technology should reflect the facilities and equipment costs (including depreciation and replacement), space (including housekeeping and utilities), and recorded time and effort of all professional staff on hospital payroll treating patients. Specific technical approaches should be identified, for example, in patients undergoing definitive radiation therapy, standard two-dimensional (2D) or three-dimensional (3D) conformal RT (CRT) or intensity-modulated RT (IMRT) or image-guided RT (IGRT). Time and effort observations for multiple tasks must be recorded by the respective staff members as they take place. The numbers of observations provide a representative sample of treatment procedures.

Technical cost is defined as the actual expenditures recorded in hospital accounting records. The total technical cost of a course of therapy includes staff time and effort, compensation including fringe benefits, space, utilities, maintenance, equipment depreciation, supplies, support services, liability cost, and administrative overhead. The ABC method³⁷^–⁴⁰ was used to derive technical (hospital) resource utilization costs for treatment procedures in treating localized prostate cancer.⁹^,³⁰ The costing method consists of four major steps:

1. Creating a process flow map that identifies and defines all of the activities and steps associated with the treatment of patients with standard irradiation or with 3D CRT (Fig. 98.1).

2. Collecting and recording the staff time expended on each activity.

3. Identifying and accumulating all hospital costs from the general accounting ledger system related to the patients treated at a radiation oncology center in a given period.

4. Allocating actual cost to each activity to derive total specific costs of technical resource utilization for a course of 2D or 3D CRT, IMRT, or IGRT.

An example of an ABC cost-calculation model is the one developed in by Lievens et al.³⁴ Because it was felt that current RT protocols, with a shift toward more complex treatment planning and delivery techniques and more thorough quality assurance procedures, were no longer accurately captured within the original model, a novel version has recently been developed.⁴¹ Because evolving technology over the last decade was more complex, it was found that the introduction of IMRT compared to 3D CRT translated into an almost doubling of procedure costs. ABC analysis allowed the authors to identify exactly where these extra costs were incurred. It was shown that the cost increase was the consequence not of the IMRT technique as such but of the combined effect of IMRT using more fields and delivering higher fraction numbers and, very important, the introduction of more, often daily, online imaging for position verification. If, however, techniques such as IMRT or gated radiotherapy delivery make it possible to reduce the number of fractions using simultaneous integrated boosts or accelerated hypofractionation, the resulting costs may decrease while yielding potentially improved treatment outcomes.³⁶

Time and Effort Analysis

An integral phase of ABC is identifying process flow, listing each individual involved in the process and recording the actual amount of time spent to perform an activity by the staff using electronic or paper daily activity logs. Determining the participant’s task time should follow the monopolizing rule: timing begins when a specific patient requires the participant’s attention and stops when he or she is no longer doing so. Whether or not the patient is still present in the activity area is immaterial. For example, for an office visit, the nurse’s time to prepare the patient’s paperwork should be attributed to the patient, even before the patient arrives or after he or she has left the office. The actual activity time may be adjusted upward to recognize some normal unproductive time of activity participants, such as time spent in work breaks. Tables are generated to show the time and effort involved in each of the functions necessary to register and evaluate the patient, perform treatment planning, deliver external-beam irradiation, and supervise the management, as well as in preparation of the appropriate records to document all information, including quality assurance requirements.

Times are collected for each activity in the treatment process beginning with the initial consultation through the first follow-up visit after full treatment. These data points are analyzed and used to generate average times for patients treated with a specific technique along with the standard deviation of times.

RESOURCE UTILIZATION COSTS

Four types of resources should be catalogued and measured for each task:

1. Activity participants (e.g., radiation oncologists, physicists, therapists, dosimetrists, nurses, receptionists)

2. Physical resources (e.g., space, equipment, machines)

3. Supplies (e.g., x-ray films, office supplies)

4. Support services (e.g., administration, centralized information system)

Patient-related expenses include salaries and fringe benefits of the staff, Social Security contributions, retirement annuities, health, life disability and long-term care, dental insurance, tuition and childcare benefits, and disability insurance. Average hourly salary and related costs for each staff member are computed.

There are many supplies, such as office paper and latex gloves, for which the cost is trivial, and it is not economically worthwhile to track their specific use for individual patients. An average cost rate can be computed for such supplies by dividing their total cost by the total number of patients, which is applied to each patient. However, supplies used only by patients with specific diseases or procedures (e.g., contrast material, immobilization devices) need to be recorded for individual patients.

For hospital-based departments, indirect operating costs are classified into general hospital operating expenses and administrative overhead, which includes hospital liability insurance and administrative costs such as shared activities (administrative personnel for purchasing and payroll functions, human resources, public relations, and financial operations). The allocation of indirect operating costs among different departments of the hospital is complex. Models include allocating them as a proportion of personnel costs and many others. The allocation of cost among various units in a hospital can also be the source of pitched battles between different units. In many hospitals allocation of costs is a mechanism of cross-subsidization among different departments (e.g., the surgery department subsidizing primary care). Administrative cost can be determined by computing prorated allocation, dividing the total hospital indirect expenses by the proportion of the radiation oncology center technical budget to the total hospital annual budget.

Technical equipment expenses for treatment of patients are included using American Hospital Association standard depreciation schedules. The cost of depreciation of all equipment involved in the management of these patients, including treatment planning and delivery, should be accounted for on a prorated basis.

COST BENEFIT

If all that matters is minimizing expense, then the lowest-cost treatment (or no treatment) would be preferred. However, cost benefit, which incorporates the incremental impact of a new technology on outcome, and cost utility, which also relates additional costs to impact on survival as well as on the patient’s quality of life and productivity, are important parameters in the assessment of new technology.⁹ The projected cost of treating a patient who initially has control of the local tumor and no distant metastases is about one-third of that for a patient who develops a treatment failure.⁴²

As a concomitant to an increasing proportion of the health insurance market being controlled by managed care, with its cost-cutting pressures, there may be a tendency to de-emphasize the importance of quality of care. However, it is critical, for instance, to document the alleged decreased morbidity of treatment with 3D CRT, IMRT, or protons in comparison with treatment of patients with carcinoma of the prostate⁴³ with conventional external-beam radiation therapy. This benefit, if it is real, is of great financial significance because treatment of major therapy complications may be costly.

Owen et al.⁴⁴ conducted economic studies for two phase III Radiation Therapy Oncology Group studies that compared different irradiation fractionation schedules (1-04 and 90-03, brain metastasis and head and neck cancer, respectively). Expected quantities of current procedural terminology codes and relative value units (RVUs) were modeled. Institutions retrospectively provided procedure codes, quantities, and components, which were converted to RVUs used for Medicare payments. The median and mean RVUs were within the range predicted by the model for all arms of one study and above the predicted range for the other study. The model predicted resource use well for patients who completed treatment per protocol. Actual economic data can be collected for critical cost items. Some institutions experienced difficulty collecting retrospective data, and prospective collection of data is necessary to validate these studies.

We must move from a fee-for-service mentality to a more critical and fiscally responsible health care system delivering high-quality care at a competitive cost. Based on current trends, 70% of hospital revenues will be capitated in a not-too-distant future. A challenge for future health care initiatives will be to strive for the lowest (optimally speaking, most competitive) cost with the highest value for the patient, including decreased morbidity of treatment and enhanced quality of life.⁴⁵

The potential contribution of more complex and expensive procedures to the outcome of cancer therapy needs to be carefully evaluated.⁴⁶ Obviously, a significant portion of the increase in health care costs is linked to new technology; a great deal of this technology is welcome, but cost benefit should be demonstrated. Some technologic advances improve health care and can be justified economically if we document a positive impact on tumor control, patient survival, morbidity of treatment, and quality of life. The efficacy of diagnostic and therapeutic procedures can be related to outcome studies, and costs should be carefully evaluated based on both monetary and health-related considerations. Doubilet et al.⁴⁷ cautioned against the indiscriminate use of “cost-effectiveness” as the criterion on which medical decisions should be made, and thus we must carefully define the endpoints on which these studies are based.

Models can be constructed using a Markov model and available data on the natural history of a tumor, prognostic factors, efficacy and morbidity of a given therapeutic modality (based on outcome analysis), and reimbursement data for treating a specific patient population. When patterns-of-failure data are used, the cost differential can be calculated for a patient treated successfully in contrast to one who fails in a specific anatomic site (locoregional failure, distant metastasis, or combination), incorporating expenditures related to management of the treatment failure. The model can be made more complete (and complex) by adding costs of the morbidity of therapy. With the information and epidemiologic data currently available in the United States, annual costs of care can be computed and compared for various therapeutic modalities. Using Monte Carlo simulation, sensitivity analysis of cost-effectiveness may be performed.

Total cost-of-care projections should be carried out using a Markov-type analysis based on clinical experience, which will provide more accurate cost data for different management options. Providing cost–benefit information to patients (consumers) will be necessary when evaluating diagnostic and treatment options, particularly in carcinoma of the breast and prostate, in which great controversy exists regarding the merits of various modalities.

This information also will be very helpful in contract negotiations with health care organizations and third-party payers and in setting policies for establishing therapy guidelines and justification of health care expenditures.

Cost–utility analysis calculates the value of an intervention as the ratio of its incremental cost divided by its incremental survival benefit, with survival weighted by utilities to produce QALYs. Many studies are at variance with current standards; only 20% of studies took a societal perspective, more than one-third failed to discount both the costs and QALYs, and utilities were often simply estimates from the investigators or other physicians. There remains much room for improvement in the methodologic rigor with which utilities are measured. Considering quality-of-life effects by incorporating utilities into economic studies is particularly important in oncology, when many therapies obtain modest improvements in response or survival at the expense of nontrivial toxicity.

Stinnett and Mullahy⁴⁸ introduced the concept of net health benefit as an alternative to cost-effectiveness ratios for the statistical analysis of patient-level data on the costs and health effects of competing interventions. Net health benefit addresses a number of problems associated with cost-effectiveness ratios by assuming a value for the willingness-to-pay for a unit of effectiveness. Willan⁴⁹ extended the concept of net health benefit to demonstrate that standard statistical procedures can be used for the analysis power and sample size determines cost-effectiveness data. He showed that by varying the value of the willingness to pay, the point estimate and confidence interval for the increment cost-effectiveness ratio can be determined.

The benefit of 3D CRT, IMRT, or IGRT is hypothetically linked to improved local tumor control because of better coverage of the target volume with a specific dose of irradiation, less acute and late morbidity, and possibility of carrying out dose-escalation studies if morbidity is held to an acceptable level, resulting in improved survival.⁴² Several institutions, the Radiation Therapy Oncology Group, and 10 institutions under cooperative agreement with the National Cancer Institute conducted phase I/II dose-escalation studies in carcinoma of the prostate and are carrying out a study on carcinoma of the lung. Depending on the results, the cost benefit of 3D CRT, IMRT, or IGRT must be further evaluated in dose-escalation and in larger multi-institutional phase III studies, comparing it with standard techniques to justify its somewhat higher initial cost. Cases that are most likely to benefit from 3D CRT, IMRT, or IGRT include patients with (a) tumors in sites with complex anatomy, (b) irregular-shaped tumors, (c) tumors adjacent to radiation-sensitive normal structures, (d) tumors that undergo significant changes in volume during treatment, and (e) small-volume or high-dose treatment.

The introduction of protons and heavy ions (hadrons) into the radiation therapy armamentarium has brought additional controversy in technology assessment because of the higher cost of facilities, equipment, and operation, with diverse opinions as to the cost-effectiveness of these innovative modalities. A 2008 review of the literature showed that only 17 publications (out of 777 on particle therapy) dealt with economic aspects, and only 5 of them were on cost-effectiveness.⁵⁰

Many articles have been published on the physical advantages of protons or heavy ions in comparison to photons, resulting in better dose distribution in the target volume with less radiation administered to normal tissues, which may decrease morbidity and eventually the risk of second malignancies.⁵¹^,⁵² However there is a lack of randomized trials documenting superior clinical outcomes with these particles.

Opinions on the need for or ethical considerations to subjecting patients to prospective trials have been expressed.⁵³^,⁵⁴ Perrier et al.⁵⁵ used an ABC approach similar to the examples discussed previously to evaluate the cost of carbon-ion therapy. It allowed an in-depth definition of the cost of treatment preparation and the actual irradiation, depending on various parameters such as actual treatment time and patient setup time. The approach of Peeters et al.,⁵⁶ conversely, was more aggregate, using a spreadsheet model that assumes that treatment costs scale linearly with the number of fractions. This, obviously, is a somewhat simple approximation of reality, which results in less accurate estimates.³⁴

Implementing new technologies in radiation oncology with advanced engineering, such as linear accelerators, cone beams, tomotherapy, treatment planning and delivery techniques, and accessories that enhance the efficiency and accuracy of operation, may decrease overall cost of treatment computed over the lifetime of the patient.

The American Society for Therapeutic Radiology and Oncology maintains the position that new technologies and modifications of existing technologies representing significant paradigmatic shifts in treatment approach should be implemented into clinical practice in such a manner as to ensure safety, efficacy, and, ideally, cost-effectiveness.⁵⁷

EVIDENCE-BASED MEDICINE AND EVIDENCE-BASED ONCOLOGY

There have been significant scientific advances in biologic sciences and health care. At the same time, there has been increased accountability and in some instances attempts to ration services; as a mechanism to accomplish this, arbitrary clinical practice guidelines have been promulgated. A more rational effort to define optimal heath care should be based on identification of innovative approaches, outcome analysis in properly designed clinical trials, and careful assessment of current practice; more important, we have a dire need for solid and credible clinical research and evidence-based decision making in medicine.

The growth in basic and translational research data to guide medical practice has made it critical for clinicians to appraise and use published data for medical decisions. Evidence-based medicine (EBM) is the compelling idea that any recommendation of a specific medical procedure should be supported by empirical evidence for its superior efficacy/toxicity ratio compared with alternative treatment options in a patient with a given presentation.⁴³^,⁵⁸^,⁵⁹

Evidence-based medicine helps correct an imbalance in contemporary medicine in which clinicians are being trained to maintain high standards of critical consciousness in methodologic domains but not in the broader historic and sociocultural domains that surround them.⁶⁰

Evidence-based health care is the conscious use of current best evidence in making decisions about the care of individuals or the delivery of health services. Current best evidence is up-to-date information from relevant valid research about the effects of different forms of health care, the potential for harm from exposure to particular agents, the accuracy of diagnostic tests, and the predictive power of prognostic factors.

Evidence-based clinical practice is an approach to decision making in which the clinician uses the best evidence available, in the context of consultation with the patient, to decide on which option suits the patient best. Evidence-based medicine is a conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.

Evidence-based medicine or evidence-based practice aims to apply the best available evidence gained from the scientific method to clinical decision making. It seeks to assess the strength of evidence of the risks and benefits of treatment or lack of treatment and diagnostic tests. Evidence quality can range from meta-analysis and systematic reviews of double-blind, placebo-clinical trials at the top end down to conventional wisdom at the bottom.

The systematic review of published research is the major method used for evaluating particular treatments. The Cochrane Collaborations are among the best known and respected examples of systematic reviews. Like other collections of systematic reviews, it requires authors to provide a detailed and repeatable plan of literature search and evaluation of the evidence. Once all the evidence is assessed, treatment is categorized as “likely to be beneficial,” “likely to be harmful,” or “evidence did not support either benefit or harm.”

Evidence-based medicine categorizes different types of clinical evidence and rates or grades them according to the strengths of their freedom from the various biases that have beset medical research. The strongest evidence for therapeutic intervention is provided by systematic review of randomized, triple-blind, placebo-controlled trials with allocation concealment and complete follow-up involving a homogeneous patient population and medical condition. In contrast, patient testimonials, case reports, or even expert opinion have little value as proof because of the placebo effect, the biases inherent in observation in reporting cases, and difficulty in ascertaining who is an expert.

Randomized, controlled trials are highly desirable to answer questions about technology assessment, but there are alternatives. Carefully controlled observational studies are good predictors of the outcome of randomized trials as long as enough essential information is collected to ensure that the patients are standardized. It is equally important in an observational series to understand which patients have been excluded.⁶¹

The American College of Radiology (ACR) created a Task Force on Appropriateness Criteria. This task force incorporates attributes for developing exceptional medical practice guidelines that have been used by the Agency for Health Care Research and Quality as designated by the Institute of Medicine. The ACR adopted the AQA’s⁶² definition of appropriateness: “The concept of appropriateness, as applied to health care, balances risk and benefit of treatment, test, or procedure in the context of available resources for an individual patient with specific characteristics. Appropriateness criteria provides guidance to supplement the clinician’s judgment as to whether a patient is a reasonable candidate for the given treatment, test, or procedure.”

The ACR Task Force on Appropriateness Criteria established 20 expert panels. Ten are concerned with diagnostic radiology and 10 with radiation therapy. In this latter group there are panels for bone metastases, brain metastases, breast, gynecologic, head and neck malignancies, Hodgkin disease, and lung, prostate, and rectal/anal malignancies. Each expert panel is chaired by an individual who is deemed by the College to have leadership capabilities and a national recognition of expertise in the area of focus. Volunteer physicians are involved in the criteria development process. Expert panelists go through a process of literature searches, acquisition of scientific articles, drafting of evidence tables, dissemination of materials for the Delphi process, collation of results, conference calls, and documentation processing. For multiple radiation oncology treatments and procedures, there are published ACR appropriateness guidelines that one can consult to garner an opinion about the “consensus view.”⁶²

The Delphi methodology is a technique used to arrive at appropriateness ratings. A series of surveys is conducted to elicit members of the expert panel’s interpretation of the evidence concerning the appropriateness of a therapeutic procedure for specific clinical scenario. The panelist receives a survey with an evidence table and narrative. Each panelist is asked to interpret the available evidence and rank each procedure. The survey is completed by each panelist without consulting fellow panel members. Ratings are done on a scale of 1 through 9, which is further divided into three categories, 1 to 3 is defined as usually not appropriate; 4 to 6 is defined as maybe appropriate; and 7 to 9 is defined as usually appropriate. Each panel member assigns one rating for each radiation therapy procedure or intervention per survey round. Surveys are collected and the results are tabulated, deidentified, and redistributed after each round. A maximum of three rounds is conducted. This modified Delphi technique enables each panelist to express individual interpretations of the evidence and his or her expert opinion without receiving bias from fellow panelists in a simple, standardized, and economical process. A consensus is defined as 80% agreement within a rating category.

Practicing evidence-based medicine requires recognition that, in most encounters with patients, questions arise that should be answered to provide the patient with the best available medical care. Appropriate and relevant clinical questions contain four elements: (a) a patient or problem, (b) an intervention, (c) a comparison intervention (if necessary), and (d) an outcome. Important practical steps in practicing evidence-based medicine include the following:

1. Formulation of the patient’s clinical problem

2. Search of literature for relevant and reliable data

3. Evaluation of validity and usefulness of data

4. Integration of critical evaluation of evidence with clinical judgment

5. Implementation of useful information

6. Assessment of outcome to improve medical practice

Once the right questions have been formulated, the best source for finding most types of best evidence is by searching Medline (National Library of Medicine), PubMed (National Library of Medicine), OVID, the Cochrane Database, or a similar database by computer.

The quality (strength) of evidence is based on a hierarchy, proceeding from the most reliable results of systematic reviews of well-designed prospective clinical trials to results of one or more well-designed meta-analysis studies, results of large retrospective case series, expert opinion, and personal experience.⁶³ Once the best data have been found, the evidence-based medicine approach involves critically appraising the quality of the evidence, determining its magnitude and precision, and applying it to a specific patient.⁶⁴^,⁶⁵

The Cochrane Collaboration was established in 1993 as an international network to help health care providers, policy makers, patients, their advocates, and their caregivers make well-informed decisions about health care by preparing, updating, and promoting the success of Cochrane reviews. More than 4,500 such reviews have been published so far online in the Cochrane Library.⁶⁶

The Cochrane Collaboration traces its origins to a 1972 publication by the British epidemiologist A. Cochrane entitled Effectiveness and Efficiency: Random Reflections on Health Services.⁶⁷ This book called attention to the collective ignorance concerning the effects of health care. In 1979 Cochrane published an essay in which he suggested that “it is surely a great criticism of our profession that we have not organized a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials.”⁶⁸ Responding to Cochrane’s charge, the Cochrane Collaboration undertook such a crucial but enormous undertaking.

There are >28,000 people working within the Cochrane Collaboration in >100 countries. Its central functions are funded by royalties of its publisher, which comes from sales from subscription to the Cochrane Library. The individual entities of the Cochrane Collaboration are funded by a large variety of governmental, institutional, and private funding sources and are bound by organization policy limiting uses of funds from corporate sponsors.

Evidence-based medicine promotes rule-based behavior on the part of physicians in an effort, among other things, to eliminate variations in medical practice. However, professionals do not follow rules per se; they intuit what is right in a situation, including, sometimes, that it is right to defer to a rule.⁶⁹

The practice of evidence-based medicine (careful clinical judgment in evaluating the “best available data”) should be differentiated from the special collection of data regarded as “suitable evidence.” The new collection of best available information has major constraints for the care of individual patients; derived almost exclusively from randomized trials and meta-analyses, the data do not include many types of treatments or patients seen in clinical practice, and the results show comparative efficacy of treatment for an “average” randomized patient, not for pertinent subgroups formed by such cogent clinical features as severity of symptoms, illness, comorbidity, and other clinical nuances. The intention-to-treat analyses in clinical trials reporting do not reflect important postrandomization events leading to altered treatment, and the results seldom provide suitable background data when therapy is given prophylactically rather than remedially. The authoritative aura given to the “collection of data” may lead to abuses that produce inappropriate guidelines or poorly supported doctrinaire dogmas for clinical practice.⁷⁰ There are a variety of mechanisms to apply evidence-based medicine to clinical practice, including formulation of treatment pathways, practice guidelines, appropriateness criteria, consensus conferences, patterns of care, and institutional peer review. A fundamental part of evidence-based medicine is the filtering of evidence through the clinical skill of the clinician. It is not easy to determine which level of evidence is required, and it is estimated that only 20% of medical procedures currently in practice can be justified by evidence-based medical standards. In addition, EBM has a dark companion: what Bentzen and Wasserman⁵⁸ called lack-of-evidence-based medicine (LEBM), a methodology used in the United Kingdom by the National Institute for Clinical Excellence to restrict access to a number of therapies on the National Health Service. Whereas evidence-based medicine is concerned with weighing level I evidence for or against a give intervention, LEBM is concerned with the systematic review of the lack-of-level I evidence on a given topic.

When does a new cancer therapy cease being experimental and become a component of the standard of care? This distinction is subject to powerful influences for for-profit pharmaceutical companies and medical equipment manufacturers wanting to increasing their sales and financial return to shareholders, advocacy groups, politicians, and a public anxiously awaiting the next “cancer breakthrough.” A dispassionate set of systems for technology assessment of cancer therapies is essential to evidence-based oncology practice.

In 1993 in the United States the American Society of Clinical Oncology (ASCO) formed a Health Services Research Committee (HSRC) to perform technology assessments and to develop guidelines for cancer treatment. The Committee is composed of clinical researchers and clinicians from academic centers and private practice and performs technology assessment after hearing from expert advisors. The committee does not develop guidelines; it assembles expert panels to develop guidelines.⁷¹

Among the most important initial tasks of the HSRC was to create an outcome working group to define the outcomes of cancer treatment that should be considered for technology assessment in forming cancer treatment guidelines. Not surprisingly, patients, physicians, researchers, payers, and policy makers all have different ideas about which outcomes of cancer treatment are important. The main purpose of ASCO’s technology assessments and guidelines is to define what constitutes the best cancer treatment, not to inform policy development and payment decisions. The working group on outcomes, therefore, considered outcomes primarily from the perspective of the clinical investigators who produced evidence for the guidelines, individual doctors and patients who have to make decisions about treatment, and health services researchers who evaluate the quality and cost of care produced by the guidelines. Because it is clear that there are health policy implications of ASCO’s technology assessments and guidelines, the working group also considered outcomes that would convince health policy makers that ASCO’s technology assessment was sound and that its guidelines represented worthwhile treatment.²⁵^,⁷¹ For technology assessment the ASCO working group is composed of members of the Health Service Research Committee and selected members. They conduct a literature review and analyze the outcomes-based evidence. Testimony is collected from invited experts and interested parties. After evaluating the information, the working group defines specific questions and provides a report outlining conditions under which the proposed procedures are warranted to assist both physicians and patients in making informed decisions regarding the technologist or procedures.

The working group identifies specific questions to be addressed by the technology assessment, develops a strategy for completing the assessment, and reviews the available literature and evidence. The process may include face-to-face meetings of available working groups and members, circulation of draft forms of the technology assessment, and opportunities for comment. Working groups do not attempt to codify established practice. They review the available evidence and add their best clinical judgment to make final recommendations.

FIGURE 98.2. Conceptual framework of outcome research. Interaction is shown between research topics, endpoints, analytic techniques, and applications in defining outcomes research. Depicted in the upper left corner are the classic clinical trials and analytic techniques that are not outcomes research. In the upper right corner are shown the study topics, endpoints, and analytic techniques that are considered to be outcomes research. Outcomes depicted in the center box may or may not constitute outcomes research, depending on the context. For example, overall survival as measured in a phase III trial is not an outcomes study (efficacy), whereas it is if observed in a large community cohort (effectiveness). Symptoms have both efficacy and outcomes influences. Applications are indicated in italics and may emanate from either clinical trials or outcomes research. See text for further details. (From Lee SJ, Earle CC, Weeks JC. Outcomes research in oncology: history, conceptual framework, and trends in the literature. J Natl Cancer Inst2000;92:195–204.)

Making a treatment selection involves defining the patient’s condition, identifying management options and outcome, collecting and summarizing evidence, and applying value judgments or preferences to arrive at an optimal course of action.⁴⁵ The randomized, controlled clinical trial has become the “gold standard” for evaluating the efficacy of health care intervention. In the 30-year period starting in 1966, >80,000 journal articles from randomized, controlled trials results were registered in the Medline database. The first 5 years of that period contributed <1% of the total number of articles, whereas the last half-decade contributed more than the previous 25 years combined. (For more detailed review of the principles of design in clinical trials, the reader is referred to Bentzen.⁷²) A proposal for structured reporting of randomized clinical trials was described in the consolidation of standards for reporting trials guidelines⁷³ that have been accepted by many medical journals. For instance, the journal Radiotherapy and Oncologyadopted a set of guidelines for reporting clinical research (Table 98.2).⁷² Linking management options to known outcomes in clinical trials or obtained through systematic reviews is becoming a common procedure (Fig. 98.2).

Clinical decisions are complex and based on many sources of information (Fig. 98.3).⁷³ Patient management decisions are always a function of both scientific evidence and individual preferences (physicians, health care providers, patients). A development to improve the basis for clinical decision making is an international movement to improve the reporting of clinical research results, particularly in randomized, controlled trials and method analyses. New data on current molecular biology and genetics investigations have broadened the scope of evidence-based oncology.

Hunt et al.⁷⁴ reported on an analysis of 68 controlled medical trials, 40 of which were published since 1992, to assess the effects of computer-based clinical decision support systems on physician performance and patient outcome. They observed that, in 43 of 65 studies (66%) on physician performance, a benefit was found, including drug-dosing systems, use of diagnostic aids, preventive care assistance, and other medical care procedures. Only 6 of 14 studies assessing patient outcome found a benefit, indicating that the impact of clinical decision support systems on patient outcome has been insufficiently studied. Furthermore, very few reports dealt with cancer-related treatment issues; most reports centered on prevention, activities, and cancer screening.⁷⁵^,⁷⁶

Although oncologists, patients, and national organizations recognize that communication is very important in cancer care, evidence-based data on the subject are scarce. Oncologists should use a patient-centered interview approach, ask patients what level of involvement they want in medical decision making, develop a caring attitude, and consider that patient–physician encounters should provide both cognitive data of patient understanding and feelings.⁷⁷

TABLE 98.2 GUIDELINES FOR REPORTING CLINICAL OUTCOME STUDIES IN THE JOURNAL RADIOTHERAPY AND ONCOLOGY

FIGURE 98.3. Factors that enter into clinical decisions. (From Mulrow CD, Cook DJ, Davidoff F. Systematic reviews; critical links in the great chain of evidence. In: Systematic reviews: synthesis of best evidence for health care decisions. Philadelphia: American College of Physicians, 1998:1–4.)

FIGURE 98.4. Stakeholders in outcomes and effectiveness research. (Adapted from Lewin Group. Outcomes and effectiveness research in the private sector: final report. Fairfax, VA: Lewin Group, 1997; and Mulrow CD, Cook DJ, Davidoff F. Systematic reviews; critical links in the great chain of evidence. In: Systematic reviews: synthesis of best evidence for health care decisions. Philadelphia: American College of Physicians, 1998:1–4.)

QUALITY OF MEDICAL CARE AND EVIDENCE-BASED MEDICINE

Continuous quality improvement emerged from the industrial sector as an effective means of reducing production errors. The quality of health care can be precisely defined and measured with a degree of scientific accuracy comparable to that of most measures in clinical medicine.⁷⁸ Health care quality problems can be classified into three categories:

1. Underuse is the failure to provide a health care service when it would have produced a favorable outcome for a patient, such as not administering irradiation in breast-conserving therapy.

2. Overuse occurs when a health care service is provided under circumstances in which its potential for harm exceeds the possible benefit—for example, prescribing postoperative pelvic irradiation to a patient with stage IA G1 endometrial carcinoma.

3. Misuse takes place when an appropriate service has been selected but a preventable complication occurs and the patient does not receive the full potential benefit of the service; avoidable complications of surgery, medications, or irradiation are important misuse problems.⁷⁹

Other issues influencing the inadequate use of services include geographic variation in the rate of use, training of general practice or specialist physicians, the makeup of the nonphysician health care work force, and the effect of organization of medical services as a determinant of quality.

In health care, continuous quality improvement is most effective when used as an integral part of a scientific approach to improving clinical practice. A potential strength is the ability of motivate good performers to excel and to place emphasis on generating new methods for achieving improvement. Among its limitations are a too-narrow focus on administrative (as opposed to clinical) aspects of care and a lack of attention to problems of overuse or underuse. Several major strategies have been advocated to move the health care delivery system toward improving quality. The challenges are (a) to always provide effective care of those who could benefit from it, (b) to always refrain from providing inappropriate services, and (c) to eliminate all preventable complications.⁸⁰ Skeptics point out that no health care market currently competes on the basis of improving quality, and there is little theoretical basis in economics to predict that this success will occur.⁷⁹ Mendelson et al.⁸¹ noted that there are many parties involved in outcome and effectiveness research (Fig. 98.4) and that payers and providers of health care may in the future use outcome research as a basis for coverage of services, including procedures, devices, or drugs.

OUTCOME RESEARCH

Outcome research focuses on identifying variations in medical procedures and associated health outcomes. Figure 98.2 illustrates the interplay of outcome research and clinical trials with clinical and policy decision making.

The term outcome is frequently used to describe a variety of endpoints or products of health care. Areas of specific focus of outcome analysis include the following:

1. Structure of care (such as features of health care facilities, staffing, equipment)

2. Process of care (such as type of studies performed, services provided)

3. Patient characteristics (gender, age, race, ethnic group)

4. Disease focus (such as location and type of tumor, histology, pathologic features)

5. Treatment-related factors (type of treatment, tumor response, disease-free survival, overall survival, morbidity of therapy)

6. Socioeconomics (such as cost benefit, quality of life, financial impact of the disease, recovery, and rehabilitation)

Meta-analysis is another mechanism to systematically research and collect published/unpublished randomized clinical trial data and quantitatively summarize results to obtain objective assessment of efficacy. The process should be (a) hypothesis driven, (b) protocol based, and (c) reproducible and comprehensive.

However, meta-analysis has some potential shortcomings, such as uncontrolled testing conditions, sample heterogeneity, unrecorded interventions, ambiguous endpoints, lack of independence of determinant factors, synergistic interactions, and sometimes contradictory experimental results.

Efforts have intensified during the last 20 years to promote outcome research in oncology, and there is a critical need to continue to support randomized, controlled studies. Patient and physician acceptance of participation in outcome research studies must be improved because only 10% to 20% of patients with newly diagnosed cancer participate in these important studies.⁸² There is considerable variation in participation of patients in research studies as a function of patient age and disease site. In general, compliance with participation in studies is much higher among pediatric patients than adults.

Among the most commonly used data are those of the Surveillance, Epidemiology, and End Result (SEER) program. SEER is a continuing project of the biometry branch of the U.S. National Cancer Institute. The program draws data from several population-based cancer reporting systems covering approximately 14% of the total population of the United States. When linked to Medicare and other insurance administrative files, it has been extremely valuable in assessing the quality of care of the elderly and other insured populations. Although an excellent data source, SEER has been criticized as being inadequate to represent the diversity of systems of care throughout the country and in general does not have data that may more accurately define patient- or tumor-specific prognostic characteristics or treatment details.

The National Cancer Data Base, a joint project of the American College of Surgeons’ Commission on Cancer and the American Cancer Society, holds information on more than half of all newly diagnosed cases of cancer in the United States and includes many of the demographic, clinical, and health systems data elements necessary to assess quality of care. A serious limitation of the National Cancer Data Base is the absence of complete information on outpatient care; its data have not been widely used to assess quality of care, but it has great potential for doing so.

An effective national system to collect this type of information should be established in the United States to collect data about the following:

1. Demographics of individuals with cancer (e.g., age, ethnic group, socioeconomic status, and insurance or health plan coverage)

2. Type of cancer (stage, histologic type, grade, comorbid conditions)

3. Treatment, including outcome of procedures, such as adjuvant chemotherapy and radiation therapy

4. Specialty training of care providers

5. Site of care delivery (such as community hospital or specialized cancer center)

6. Type of care delivery system (such as managed care, fee for service, government agency)

7. Outcomes (tumor control, survival, complications of treatment, quality of life, satisfaction with services provided)

THE WAYS TO MEASURE OUTCOMES INDICATIVE OF SURVIVAL AND TUMOR RESPONSE

Survival can be measured as overall survival, disease-free survival, progression-free survival, or event-free survival. It is clearly the most important outcome in cancer treatment. It is also clear, however, that survival alone is insufficient as a measure of outcome. The quality of survival and the cost of maintaining and improving survival must also be assessed.⁸³

Making a choice between alternative treatment approaches often involves a tradeoff between length of survival and quality of life. A survival measurement alone might not answer the question of whether gains in survival justify toxicity. Quality-adjusted survival attempts to adjust the absolute length of life to reflect the patient’s quality of life. Quality-adjusted survival provides a framework within which tradeoffs that influence treatment choices can be defined as in decision and cost-effective analysis to assess the effectiveness of alternative therapies.⁷¹^,⁷³

Quality-of-Life Outcome Measures

Quality of life considers the impact of cancer and its treatment on the physical, psychological, and social components of patient’s lives. Cancer-related quality of life is a family of outcomes including the global quality of life as well as physical, psychological, and social dimensions, each one of which is a potentially separate outcome. Because of its subjective nature, the assessment of quality of life generally should include an evaluation by the patient. Quality-of-life measures should be characterized by reliability and validity.⁷³^,⁸¹^,⁸⁴

Toxicity Measures of Outcome

Toxicities are subjective, objective, or both. An example of a subjective toxicity would be a symptom such as nausea that is often not associated with overt signs or laboratory abnormalities. Evaluation of toxicities of this sort rest with the patient’s report. Objective toxicities are measured by physical examination or laboratory tests. The common toxicity criteria use a system for categorizing toxicity from cancer treatment according to its severity. Such systems are widely used in cooperative group trials.

Measures of Cancer Response Outcomes

Measures of tumor response include complete response, response to radiation, and time to progression. Response may also be measured by biomarkers and changes in cancer-induced abnormalities and common blood tests. Some studies demonstrated the positive relationship between response rate and quality of life, but others did not. It is clear that freedom from progression may not be a convincing indication of the benefits of treatment if it does not predict significantly improved survival or cure. Freedom from progression is therefore not an adequate measure of quality of life.⁸²

Comparative effectiveness research must be focused on patient outcomes. One cannot assume that a higher dose of radiation to a target field or a more conformal dose distribution will result in better survival or improved tolerance. Outcomes of significance to the patient such as survival, disease-free survival, quality of life, or specific side effects must be the focal points of research.⁶¹ It is particularly important for radiation oncologists to remember that an improved dose distribution achieved through intensity-modulated radiotherapy, conformal techniques, or particle therapy is not the same as an improved outcome. The presence of an improved outcome as the result of a difference in dose distribution must be proven, not assumed.

Cost Effectiveness as a Cancer Outcome

Cost is not an outcome. It is, instead, what we spend to produce an outcome. Cost-effectiveness, in contrast, is often an important outcome. Cost-effectiveness may be reported in terms of cost per year of life saved or cost per quality adjusted year of life saved. Clinical practice guidelines are generally informed by cost-effectiveness considerations.⁷¹^,⁷³

HOW ONE MIGHT USE OUTCOMES FOR TECHNOLOGY ASSESSMENT AND GUIDELINE DEVELOPMENT

Priority of Outcomes

Patient outcomes are more important than cancer outcomes for technology assessment and development of cancer treatment guidelines. If a new technology or therapy is not ultimately shown to make patients live longer or feel better, then its use cannot be justified. Gathering the necessary data is not easy. Randomized, controlled trials are the gold standard for these comparisons but often fail to accrue enough patients. The reasons for failure to accrue include physicians who refuse to enroll patients because they believe one technology is superior, patients who refuse to be randomized because they have a treatment/technology preference, and the extra work necessary to enroll a patient in a trial.⁶¹

The Need to Use Multiple Outcomes

There is no single outcome that can represent the results of cancer treatment. Each of the various outcomes provides an unique perspective on treatment and has its own limitations. In general, multiple outcomes should be used for technology assessment.

Barriers to Progress in Cancer Outcome Research

Despite growing interest, there are numerous obstacles to overcome in health care outcome research, including the need to apply scientific disciplines that are different from the biomedical sciences (survey research, psychology, health services research, statistics, economics), the need for resources to collect additional data and information beyond that of standard clinical care, and limited coordination and collaboration among researchers working in the field. Other barriers are intrinsic to a specific cancer, including heterogeneity of patients, type of tumor, phases of the disease, and ethnic and socioeconomic characteristics. Furthermore, many institutions do not give a high priority to or adequately fund outcome research because this activity is considered a cost, not a revenue center, and there is a lack of conviction that this effort will reduce health care spending.⁷⁹^,⁸⁵

Data sharing and increased collaboration in data collection and analysis should be promoted among health care providers and the various groups conducting outcome research. Creative approaches to educating providers and patients about outcome studies are needed, as well as mechanisms to ensure that patients and health professionals have ready access to the results of outcome research.

Although oncologists, patients, and national organizations recognize that communication is very important in cancer care, evidence-based data on the subject are scarce. Oncologists should use a patient-centered interview approach, ask patients what level of involvement they want in medical decision-making, develop a caring attitude, and consider that patient–physician encounters should provide both cognitive data of patient understanding and feelings.⁷¹

The Complexity of Randomized, Prospective Clinical Trials

Randomized, prospective, controlled trials are held forth as the gold standard for determining the appropriateness of introducing technology into clinical practice. One must be cognizant, however, of adverse influences upon the generalizability of randomized, controlled trials. If the age, gender, severity of the disease, risk factors, comorbidity, ethnicity, or socioeconomic status of the patients in the randomized, prospective trial are not representative of the general population, it may be inappropriate to generalize the results of the clinical trial to that population. Similarly, if the dose, timing of administration, duration of therapy, quality of care, or comedications of the patients in the trial are not similar to those of the general population, the results may not be applicable.²³

There is a long-standing joke that one does not need a randomized, prospective trial to ascertain the efficacy of a parachute. In medicine, there are long lists of new technologies that were introduced into general practice based on historical controls without randomized, prospective trials. These include, for example, the use of thyroxin for myxedema, insulin for diabetes, vitamin B12 for pernicious anemia, sulfonamides for puerperal sepsis, penicillin for lobar pneumonia, streptomycin for tuberculosis, defibrillation for ventricular fibrillation, and cisplatinum, vinblastine, and bleomycin for disseminated testicular cancer. The analysis of a randomized clinical trial has traditionally been based on a hypothesis that presumes that there is no difference between treatments. The hypothesis is tested by estimating the probability of obtaining a result as extreme as or more extreme than the one observed if the hypothesis is true. The hypothesis may be irrelevant if there had been previous studies demonstrating that the treatment had some benefits. Those who criticize randomized, prospective trials will quickly argue that “no one ever did a randomized trial to prove that a linear accelerator was better than a cobalt machine, that port films were necessary, or that simulation was preferable to a clinical setup.” These arguments are, to some extent, true. One must remember, however, that there is a considerable financial difference between using and not using a port film compared to using a linear accelerator versus a $200 million proton machine.⁷⁶

Nonetheless prospective, randomized clinical trials have come to occupy a prominent role in clinical research in radiation oncology. The most commonly used clinical endpoints are summarized in Table 98.3. Unfortunately, many published reports have unclear definitions or lack some of the fundamental endpoints.

Clinical trials are time consuming and costly. Randomization is the only method that may guarantee that comparisons of treatment outcome are, on average, unbiased. Not all clinical trials yield positive results, and not all new therapies are improvements compared with the standard therapy.⁵⁸^,⁶⁴ As an example, a recent analysis of outcome data from 57 randomized, controlled trials, including a total of 12,734 patients, conducted between 1968 and 2002 by the Radiation Therapy Oncology Group⁶³ showed that overall, experimental and standard radiotherapy were equally successful (odds ratio for survival, 1.01; p = .5); however, treatment-related mortality was significantly greater in the experimental arm of the trials (odds ratio, 1.76; p = .008) A drawback of randomized trials for the evaluation of a new technology in radiation oncology is that to detect small improvements in morbidity from, for example, an alleged improved dose distribution, an enormous number of patients will need to be accrued and followed for a considerable length of time. Radiation oncologists must, therefore, give serious consideration to the development of novel paradigms for health technology assessments without meeting, in some cases, the ideal requirement of randomization.⁶³

Practice Guidelines

Guidelines for health services resulting from valid and appropriate outcome studies have the potential to promote consistency and quality of care. In the United States, clinical practice guidelines have been promulgated by the American Cancer Society, the American College of Radiology, the American Society of Clinical Oncology, the National Cancer Centers Network, and other professional organizations (Table 98.4).⁶¹^,⁸⁴ Sisk⁸⁵ observed that for such guidelines to be successful, it is very important to involve local physicians in the process, to get their views and ultimately their support. The values of practice guidelines in medicine include minimizing inappropriate practice variations, providing reference points for education/practice, improving patient care and outcomes, providing criteria for self-evaluation, setting indicators for external quality review, assisting with service coverage and reimbursement, and decreasing overall cost of medical care. Perceived drawbacks to such guidelines include overregulation and loss of autonomy for both physicians and patients. The conflicting needs and motivations of physicians, patients, third-party carriers, and society bring into play a variety of ethical perspectives. Although contrasting ethical foundations likely will lead to significantly different solutions to the health care resource problem, it is only through educated and reasoned discussion that this problem can be tackled.⁸⁶

Courts in the United States have ruled that guideline developers can be held liable for faulty guidelines and that doctors cannot pass off their liability by claiming that adherence to guidelines corrupted clinical judgment. Protocols and guidelines provide the courts with examples of clinical standards across a wide range of medical practice. However, adherence to guidelines has not automatically been equated with reasonable practice, and the courts seem unlikely to follow the standards enunciated in clinical guidelines without critically evaluating their authority, flexibility, and scope of application.⁸⁷

TABLE 98.3 MOST COMMONLY USED CLINICAL ENDPOINTS IN RADIATION THERAPY TRIALS

TABLE 98.4 AMERICAN COLLEGE OF RADIOLOGY APPROPRIATENESS GUIDELINES LITERATURE EVIDENCE TABLE KEY

The Patterns of Care Study in the United States

The basic concepts of the Patterns of Care Study are based on analysis of structure, process, and outcome.⁸⁸ Initially, all facilities in the United States that provided radiation therapy were identified, and a master list was developed; a statistically determined group of the facilities was selected to reflect the type of practice of radiation oncology throughout university hospitals, community hospitals, and free-standing facilities. Surveys were conducted to define the structure of the facilities, including equipment, staffing, and prevailing standards of practice. More than 1,000 institutions were surveyed in each period of study, and thousands of patients have been included in the database.

Ten diseases were chosen in which curative radiation therapy played a major role. Through extensive discussions with appropriate physicians, a consensus of the best current management was developed for each disease, and decision trees were designed that logically led to the appropriate application of radiation therapy. Process criteria were grouped into three categories, which included workup, treatment, and auxiliary services. Data were collected on individual patients (with appropriate safeguards of privacy), and later the institutions were resurveyed to record the results of treatment, including local/regional recurrence, disease outside the treated field, major complications of therapy, and so forth, leading to publication of several reports correlating process with treatment outcome. An example is a report on the practice of radiotherapy in carcinoma of the uterine cervix published by Eifel et al.⁶⁵

Reports similar to those of the Patterns of Care studies in United States have been published in other countries, such as one by Ringborg et al.³¹ in Sweden.

CONCLUSIONS

Technology assessment, outcome analysis, cost benefit, comparative effectiveness, and clinical trials supported by evidence-based medicine should be strengthened and fostered to enhance the rationale and quality of medical care provided to patients at a competitive cost. Basic and translational laboratory research and properly designed, relevant, and timely prospective clinical trials should be strongly promoted, and patient participation must be increased to acquire more accurate information to develop innovative therapeutic strategies in oncology. Methods for accurate cost accounting of medical care and comparative effectiveness studies need further development. Technology assessment will substantially contribute to better utilization of scarce health care resources and will be invaluable in determining the potential value of innovative therapeutic approaches.

REFERENCES

1. Doyle R. Health care costs. Sci Am 1999;280:36.

2. Bodenheimer T. High and rising health care costs. Part 2: Technologic innovation. Ann Intern Med 2005;142: 932–937.

3. Organisation for Economic Co-operation and Development. OECD health data 2011. Available at: http://www.oecd.org/document/16/o,3343,en_2649_34631_2085200_1_1_1_1,00.html.

4. Borger C, Smith S, Truffer C, et al. Health spending projections through 2015: changes in the horizon. Health Aff (Millwood) 2006;25:w61–w73.

5. Forbes S. Dinosaur U. Forbes February 28, 2011, p. 13.

6. Emanuel EJ, Fuchs VR, Garber AM. Essential elements of a technology and outcome assessment initiative. JAMA 2007;298:1323–1325.

7. Centers for Medicare and Medicaid Services. Coverage with evidence development. Available at: http://www.cms.gov/Medicare/Coverage/Coverage-with-Evidence-Development/index.html.

8. Wallner PE, Konski A. A changing paradigm in the study and adoption of emerging health care technologies: coverage with evidence development. J Am Coll Radiol 2008;5:1125–1129.

9. Hayman J, Weeks J, Mauch P. Economic analyses in health care: an introduction to the methodology with an emphasis on radiation therapy. Int J Radiat Oncol Biol Phys 1996;35:827–841.

10. Detsky AS, Naglie IG. A clinician’s guide to cost-effective analysis. Ann Int Med 1990;113:147–154.

11. Drummond M, Sculpher MJ, Torrance GW, et al. Methods for the economic evaluation of health care programmes. Oxford: Oxford University Press, 2005.

12. Harngren CT, Foster G, Datar SM. Cost accounting. a managerial emphasis, 9th ed. Upper Saddle River, NJ: Prentice-Hall, 1996.

13. EQ-5D. Available at: www.euroqol.org.

14. Perez CA. Methodology of research and practice for the third millennium: evidence-based medicine. Rays 2000;25:285–308.

15. Feeny D, Furlong W, Boyle M, et al. Multi-attribute health status classification systems. Health Utility Index. Pharmacoeconomics 1995;7:490–502.

15a. Meltzer MI. Introduction to health economics for physicians. Lancet 2001;358:993–998.

16. Clancy CM, Kamerow DB. Evidence-based medicine meets cost-effectiveness analysis. JAMA 1996;276:329–330.

17. Neilson AR, Davies HT. Interpreting reported health-care costs. Hosp Med 1998;59:803–806.

18. Siegel JE, Weinstein MC, Russell LB, et al. Recommendations for reporting cost-effectiveness analyses. Panel on Cost-Effectiveness in Health and Medicine. JAMA 1996;276:1339–1341.

19. Russell LB, Gold MR, Siegel JE, et al. The role of cost-effectiveness analysis in health and medicine. Panel on Cost-Effectiveness in Health and Medicine. JAMA 1996;276:1172–1177.

20. Rigby K, Silagy C, Crockett A. Health economic reviews. Are they compiled systematically?Int J Technol Assess Health Care 1996;12:450–459.

21. Smith JM. Effectively costing out options. JAMA 1996;276:1180.

22. Weinstein MC, Siegel JE, Gold MR, et al. Recommendations of the Panel on Cost-effectiveness in Health and Medicine. JAMA 1996;276:1253–1258.

23. Russell LB. Modelling for cost-effectiveness analysis. Stat Med 1999;18:3235–3244.

24. Cowan J, Berkowitz D. Technology assessment at work: part I—principles and a case study. Physician Exec 1996;22:5–6, 8–9.

25. Chlebowski RT, Collyar DE, Somerfield MR, et al. American Society of Clinical Oncology technology assessment on breast cancer risk reduction strategies: tamoxifen and raloxifene. J Clin Oncol 1999;17:1939–1955.

26. Halperin EC. Overpriced technology in radiation oncology. Int J Radiat Oncol Biol Phys 2000;48:917–918.

27. Lee WR. Technology assessment: vigilance required. Int J Radiat Oncol Biol Phys 2008;70:652–653.

28. Cotter GW. Surgery or radiation therapy: a comparative cost analysis for early carcinoma of the prostate and breast. Appl Radiol 1990;19:25–28.

29. Penn CR. Megavoltage irradiation in a district general hospital remote from a main oncology centre: the Torbay experience reviewed. Clin Oncol (R Coll Radiol) 1992;4:108–113.

30. Perez CA, Kobeissi B, Smith BD, et al. Cost accounting in radiation oncology: a computer-based model for reimbursement. Int J Radiat Oncol Biol Phys 1993;25(5):895–906.

31. Ringborg U, Bergqvist D, Brorsson B, et al. The Swedish Council on Technology Assessment in Health Care (SBU) systematic overview of radiotherapy for cancer including a prospective survey of radiotherapy practice in Sweden 2001—summary and conclusions. Acta Oncol 2003;42:357–365.

32. Smith TJ, Hillner BE, Desch CE. Efficacy and cost-effectiveness of cancer treatment: rational allocation of resources based on decision analysis. J Natl Cancer Inst 993;85:1460–1474.

33. Norlund A. Cost of radiotherapy. Acta Oncol 2003;42:411–415.

34. Lievens Y, Van Den Bogaert W, Kesteloot K. Activity-based costing: a practical model for cost calculation in radiotherapy. Int J Radiat Oncol Biol Phys 2003;57:522–535.

35. Konski A, Bhargavan M, Owen J, et al. Feasibility of economic analysis of Radiation Therapy Oncology Group (RTOG) 91-11 using Medicare data. Int J Radiat Oncol Biol Phys 2010;79:436–442.

36. Lievens Y, Dunscombe P, Konski A. Promises and pitfalls of health technology assessment. In: Levit SH, Purdy JA, Perez CA, et al., eds. Technical basis of radiation therapy, 5th ed. New York: Springer, 2012:549–564.

37. Antos J, Brimson JA. Activity-based management for service industries, government entities and nonprofit organizations. New York: Wiley, 1994.

38. Cooper R. Elements of activity-based costing, emerging practices in cost management. New York: Warren, Gorham & Lamont, 1990.

39. Cooper R, Kaplan RS. The design of cost management systems. Upper Saddle River, NJ: Prentice-Hall, 1991.

40. Foster G, Gupta M. Activity accounting: an electronics industry implementation. In: Kaplan RS, ed. Measures for manufacturing excellence. Boston: Harvard Business School Press, 1990:225–268.

41. Van der Werf E, Verstraete J, Lievens Y. The cost of radiotherapy in a decade of technology evolution. Radiother Oncol 2012;102(1):148–153.

42. Perez CA, Kobeissi BJ, Chao KSC, et al. 3-D conformal and intensity modulated radiation therapy: physics and clinical applications. Madison, WI: Advanced Medical Publishing, 2001.

43. Sackett DL, Straus S, Richardson S, et al. Evidence-based medicine: how to practice and teach EBM, 2nd ed. London: Churchill Livingston, 2000.

44. Owen JB, Grigsby PW, Caldwell TM, et al. Can costs be measured and predicted by modeling within a cooperative clinical trials group? Economic methodologic pilot studies of the radiation therapy oncology group (RTOG) studies 90-03 and 91-04. Int J Radiat Oncol Biol Phys 2001;49:633–639.

45. Brook RH, Kamberg CJ, McGlynn EA. Health system reform and quality. JAMA 1996;276:476–480.

46. Fuchs VR, Garber AM. The new technology assessment. N Engl J Med 1990;323:673–677.

47. Doubilet P, Weinstein MC, McNeil BJ. Use and misuse of the term “cost effective” in medicine. N Engl J Med 1986;314:253–256.

48. Stinnett AA, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making 1998;18:S68–S80.

49. Willan AR. Analysis, sample size, and power for estimating incremental net health benefit from clinical trial data. Control Clin Trials 2001;22:228–237.

50. Pijls-Johannesma M, Pommier P, Lievens Y. Cost-effectiveness of particle therapy: current evidence and future needs. Radiother Oncol 2008;89:127–134.

51. Goitien M, Cox JD. Should randomized clinical trials be required for proton radiotherapy? J Clin Oncol 2008;26:175–176.

52. Lodge M, Pijls-Johannesma M, Stirk L, et al. A systematic literature review of the clinical and cost-effectiveness of hadron therapy in cancer. Radiother Oncol 2007;83:110–122.

53. Glatstein E, Glick J, Kaiser L, et al. Should randomized clinical trials be required for proton radiotherapy? An alternative view. J Clin Oncol 2008;26:2438–2439.

54. Suit H, DeLaney T, Goldberg S, et al. Proton vs carbon ion beams in the definitive radiation treatment of patients with cancer. Radiother Oncol 2010;95:3–22.

55. Perrier L, Combs SE, Aurberger T, et al. A decision-making tool for a costly innovative technology. The case for carbon ion radiotherapy. J Econ Med 2007;25:367–380.

56. Peeters A, Grutters JP, Pijls-Johannesma M, et al. How costly is particle therapy? Cost analysis of external beam radiotherapy with carbon-ions, protons and photons. Radiother Oncol 2010; 95:45–53.

57. Buyyounouski MK, Price RA, Harris EER, et al. Stereotactic body radiotherapy for primary management of early stage low to intermediate risk prostate cancer: report of ASTRO Emerging Technology Committee. Int J Radiat Oncol Biol Phys2010;76:1297–1304.

58. Bentzen SM, Wasserman TH. Balancing on a knife’s edge: evidence-based medicine and the marketing of new technology. Int J Radiat Oncol Biol Phys 2008;72:12–13.

59. Bigby M. Evidence-based medicine in a nutshell. A guide to finding and using the best evidence in caring for patients. Arch Dermatol 1998;134:1609–1618.

60. Berkwits M. From practice to research: the case for criticism in an age of evidence. Soc Sci Med 1998;47:1539–1545.

61. Newcomer LN. Finding the answers we need: comparative effectiveness. Pract Radiat Oncol 2011;1:83–84.

62. Leibel SA. ACR appropriateness criteria. Expert Panel on Radiation Oncology. American College of Radiology. Int J Radiat Oncol Biol Phys 1999;43:125–168.

63. Guyatt GH, Sinclair J, Cook DJ, et al. Users’ guides to the medical literature: XVI. How to use a treatment recommendation. Evidence-Based Medicine Working Group and the Cochrane Applicability Methods Working Group. JAMA 1999;281:1836–1843.

64. Bentzen SM Randomized controlled trials in health technology assessment: overkill or overdue? Radiother Oncol 2008;86:142–147.

65. Eifel PJ, Moughan J, Erickson B, et al. Patterns of radiotherapy practice for patients with carcinoma of the uterine cervix: a pattern of care study. Int J Radiat Oncol Biol Phys 2004;60:1144–1153.

66. Cochrane collaboration. Preparing, maintaining, and promoting the accessibility of systemic reviews of the effects of healthcare interventions. Oxford: Cochrane Collaboration, 1999.

67. Cochrane A. Effectiveness and efficiency: random reflection on health services. London: Royal Society of Medicine Press Limited; 2004.

68. Cochrane AL. 1931–1971: A critical review, with particular reference to the medical profession. In: Medicine for the year 2000. London: Office of Health Economics; 1979:1–11.

69. Tanenbaum SJ. Evidence and expertise: the challenge of the outcomes movement to medical professionalism. Acad Med 1999;74:757–763.

70. Feinstein AR, Horwitz RI. Problems in the “evidence” of “evidence-based medicine”. Am J Med 1997;103:529–535.

71. American Society of Clinical Oncology. Outcomes of cancer treatment for technology assessment and cancer treatment guidelines. J Clin Oncol 1996;14:671–670.

72. Bentzen SM. Towards evidence based radiation oncology: improving the design, analysis, and reporting of clinical outcome studies in radiotherapy. Radiother Oncol 1998;46:5–18.

73. Mulrow CD, Cook DJ, Davidoff F. Systematic reviews; critical links in the great chain of evidence. In: Systematic reviews: synthesis of best evidence for health care decisions. Philadelphia: American College of Physicians, 1998:1–4.

74. Hunt DL, Haynes RB, Hanna SE, et al. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA 1998;280:1339–1346.

75. McAlister FA, Straus SE, Guyatt GH, et al. Users’ guides to the medical literature: XX. Integrating research evidence with the care of the individual patient. Evidence-Based Medicine Working Group. JAMA 2000;283:2829–2836.

76. Rawlins M. De testimonio: on the evidence for decisions about the use of therapeutic interventions. Clin Med 8: 579–588, 2008.

77. Back A. Patient-physician communication in oncology: what does the evidence show? Oncology 2006;20:67–74.

78. Pommier P, Lievens Y, Feschet F et al. Simulating demand for innovative radiotherapies: an illustrative model based on carbon ion and proton radiotherapy. Radiother Oncol 2010;96:243–249.

79. Sisk JE. Increased competition and the quality of health care. Milbank Q 1998;76:687–707, 512.

80. Chassin MR, Galvin RW. The urgent need to improve health care quality. Institute of Medicine National Roundtable on Health Care Quality. JAMA 1998;280:1000–1005.

81. Mendelson DN, Goodman CS, Ahn R, et al. Outcomes and effectiveness research in the private sector. Health Aff (Millwood) 1998;17:75–90.

82. Newman L, ed. Medical outcomes and guidelines sourcebook. New York: Faulkner & Gray, 1999.

83. Panek WC. Ethical considerations related to outcome studies-based clinical practice guidelines. J Glaucoma 1999;8:267–272.

84. American Society of Clinical Oncology. Recommended breast cancer surveillance guidelines. J Clin Oncol 1997;15:2149–2156.

85. Sisk JE. How are health care organizations using clinical guidelines? Health Aff (Millwood) 1998;17:91–109.

86. Bentzen S. Radiation oncology health technology assessment—the best is the enemy of the good. Nat Clin Pract Oncol 2008;5:563.

87. Hurwitz B. Clinical guidelines and the law: advice, guidance or regulation? J Eval Clin Pract 1995;1:49–60.

88. Kramer S. An overview of process and outcome data in the patterns of care study. Int J Radiat Oncol Biol Phys 1981;7:795–800.

Page

Contents

If you find an error or have any questions, please email us at admin@doctorlib.org. Thank you!