William C. Shaw
Gunvor Semb
In the modern age, evidence-based care is considered to be an integration of the best research evidence with clinical expertise and patient values. With respect to therapeutic interventions, the strongest evidence is derived from systematic reviews that provide a synthesis of relevant randomized control trials (Sackett et al., 2000).
For cleft lip and palate care providers, however, there are some challenges ahead. The present scientific basis of the discipline is weak since virtually no elements of treatment have been subjected to the rigors of contemporary clinical trial design (Roberts et al., 1991). Thus, highly complex and varied protocols of care are practised by different teams. Generally speaking, choices regarding surgical technique, timing and sequencing, and ancillary procedures, such as orthopedics, orthodontics, and speech therapy, are arrived at following disappointment in the results of former practices rather than firm evidence that the new protocol has succeeded elsewhere. As a consequence, the unsubstantiated testimony of enthusiasts for a particular treatment has done much to shape current practices. Typically, enthusiastic claims are made for a new type of therapy, the procedure is widely adopted, a flow of favorable clinical reports ensues, little or no positive evidence develops to support the desirability of the procedure, and there is a sharp drop in the number of clinical reports, again without evidence to support the change (Spriestersbach et al., 1973).
Judging Evidence of Effectiveness
The general rules of health technology assessment are well established and the quality of treatment comparisons conforms to a widely accepted hierarchy, from anecdotal reports to randomized trials and systematic reviews. This hierarchy relates to the degree of effort made to minimize ever-present sources of research bias, “the usual suspects,” that readily lead to false conclusions:
1. Susceptibility bias. Some patients will inevitably be more susceptible to treatment because their condition is less severe or they inherently possess a better prognosis. Thus, the effectiveness of any technique, applied to a group of cases that are inherently more amenable, will be inflated if compared with another technique applied to a more challenging sample.
2. Proficiency bias. In a similar manner, a more skilled surgeon or clinical team can also flatter or inflate the apparent effectiveness of a technique. If operator A is 10% better than operator B and technique X is 5% better than technique Y, a false conclusion will be reached in a comparison of Y performed by A vs. X performed by B.
3. Follow-up bias. How confident can the consumer of journal or lecture reports be that the whole story has been given? Can we be sure that follow-up was as rigorous for the cases that went badly as for those that went well? Without knowing about all of the cases in which the new technique was tried, reliable conclusions cannot be drawn.
4. Exclusion bias. In reporting the effectiveness of an intervention, it is often tempting to exclude retrospectively cases where the expected progress was not achieved. Typical grounds for retrospective exclusion might be lack of compliance on the part of patient or suspicion that an underlying condition (e.g., an ill-defined syndrome) has prevented the intervention from working. Irregular application of the rules of retrospective exclusion clearly can remove any equivalence that comparison groups may have had.
5. Analysis bias. Given the virtual absence of agreed rating schemes for outcome evaluation, reporting in the cleft literature is inevitably inconsistent. Without objectivity in appraisal, as achieved with blinded, independent panels, comparisons must be unsure.
6. Reporting bias. Clinical researchers, like drug companies, are less likely to report negative findings than positive ones. However, not only are findings more likely to be reported if they are positive, but they are also more readily accepted for publication by journals, more readily accepted for conferences, more often published in English, and more often cited in later publications (Dickersin and Min, 1993; Dickersin et al., 1992; Easterbrook 1991; Egger et al., 1997; Stern and Simes, 1997).
Not surprisingly then, empirical research consistently demonstrates that in studies without randomized control groups an overestimation of effectiveness results (Kunz and Oxman, 1998). Thus, controlled trials of psychiatric interventions found them to be effective only 25% of the time, but in uncontrolled studies of the same medications 75% were positive. Even more dramatically, none of a series of randomized trials portacaval shunt surgery found clear evidence of benefit, but 75% of uncontrolled studies did. Clearly then, uncontrolled studies, which make up the great majority of the current literature of cleft care, must be appraised with great caution. They should be appreciated for the contributions to knowledge they can make, with recognition of their inherent limitations.
Anecdotal Case Reports
Case reports may signal important new developments in surgical practice, but the evidence they contain for a widespread change in practice remains generally unconvincing in the absence of subsequent confirmatory series.
Case Series
Reports of a series of cases treated by the same method provide more substantial evidence of the merits of a particular technique or program of treatment and a general impression of relative efficacy for the professional community. They are of particular value in demonstrating that new procedures can be reliably performed and can have a low risk of serious morbidity. Rather commonly, however, outcome is measured in the short term and the enthusiasm of the reporters may impair true objectivity. Thus, primary bone grafting, first heralded as an important breakthrough in case series reports, was later shown by randomized control trials to be harmful to facial growth (Rehrmann et al., 1970; Jolleys and Robertson, 1972). However, case series of secondary bone grafting using cancellous iliac crest grafts revealed that one aspect of outcome, the patient's dentition, could be reliably restored beyond previously attainable levels (Boyne and Sands, 1972, 1976; Bergland et al., 1986). The immediacy of these benefits ruled against the need for a randomized trial, though potential growth disturbances still deserved consideration (Semb, 1988). Future trials of bone grafting may, however, still be necessary to examine individual aspects of surgical technique or timing or the suitability of alternative graft materials.
Case series rarely provide evidence of the superiority of one technique over others where a choice of broadly similar methods exists and any improvement may be incremental rather than dramatic. This is a major problem in the evaluation of primary surgical repair since this may be achieved with apparently similar success by methods that differ in technique, timing, and sequence. Meaningful comparison of case series reported in the literature is prohibited by methodological inconsistencies in assessment and by the absence of strict and well-defined entry criteria, such as consecutive cases with an equivalent prognosis.
Uncontrolled Comparison Studies
Opportunities for nonexperimental comparisons of therapies or programs of care can arise in several ways: through coexisting therapies at the same center, through replacement of one therapy with another, or by comparison of treatment centers using different therapies. However, any lack of equivalence between the cases prior to treatment or lack of equivalence in the competence of the clinicians will again undermine the conclusions.
Comparison of Coexisting Therapies
When using retrospective material such as case notes or clinical databases, checks can be made on the equivalence of the groups, commonly in terms of gender, age, or cleft subtype. Preferably, cases can be matched pairwise on these characteristics. Alternatively, adjustments can be made in the analysis by stratification or the use of multivariate statistical methods. In either case, doubt will remain that important prognostic factors have been masked, for if two or more therapies were being used concurrently within a single center, selective allocation to treatment must be suspected.
Factors that may have influenced clinical decisionmaking could be unrecorded or unreliably recorded. For example, decisions as to when (at what age) to perform primary surgery may be influenced by unrecorded aspects of the morphology of the cleft, the availability of personnel, the health of the child, or parental attitudes toward the cleft. Should these factors influence outcome, confounding would occur in any study of the effect of age on surgical outcome.
The possibility of confounding in this way is especially likely when treatment was provided 5 or 10 years previously and different staff were involved. Retrospective ascertainment of the details of primary surgery or cleft subtype is difficult, and descriptive terminology may have changed in subtle ways. It may be possible to match or adjust data to remove bias due to gender, age, or cleft subtype; but this gives no guarantee that some other prognostic factor that may affect outcome is not associated with choice of treatment. Also, a critical factor in surgical outcome must be the competence of the surgeon.
Comparison with Historical Controls
Comparisons with historical controls may arise as natural experiments by changes in therapy within a treatment center. Such research is particularly valuable when durable records (e.g., radiographs, study casts, speech recordings, photographs) are obtained in a standardized way for both subjects treated by a previous method (historical controls) and those treated by the new method, allowing simultaneous unbiased evaluation. Data may already exist on two well-documented treatments used in different time periods.
An alternative circumstance in which such studies arise is where data for a group of patients receiving a standard treatment already exist and can be gathered in a similar way when a new treatment is introduced. This design requires only half the number of patients to be gathered prospectively as a randomized clinical trial and is clearly attractive where recruitment of cases is slow. Furthermore, it has been argued that in circumstances of poor outcome it may be unethical to withhold new treatment in order to create a control group (Gehan, 1984). There are nevertheless several biases and possibilities for confounding that generally favor the newly introduced procedure. In practice, changes in technique at a treatment center often come about as a result of changes in personnel, who may have performed differently with respect to the previous method. This leads to bias due to differences in the skill of personnel associated with either treatment method. For example, a new method of treatment is often tested by an experienced and innovative surgeon, who may be expected to achieve better results than the average surgeon. This clearly introduces the confounding effect of operator skill with treatment. Even where there is stability of staff, bias reflecting gradual changes of ability and technique is highly likely and definition or ascertainment of prognosis may change. New methods may also be initially applied with some selectivity to “suitable” cases as experience is gained. Other aspects of clinical management may have been altered with the intention of improving outcome, creating additional possibilities for bias in favor of the innovative procedure. Multivariate methods have been suggested as a way to adjust for these biases, but serial changes in treatment are likely to take place in parallel, resulting in a strong association between treatment variables. This is one reason why historical control design generally unsuited to evaluating primary surgery since other changes in the total program of care are likely to have occurred during the extensive recruitment.
Bias favoring the innovative procedure is a major cause for concern with historical control studies as they may either fail to resolve a controversy or create ethical concerns that preclude further, more rigorous comparisons. Favorable outcomes suggested for a new procedure by historical control studies have been disputed by subsequent randomized controlled trials (Pinsky, 1984; Pollock, 1986). Thus, historical control studies could set in motion an unwarranted cycle of change with no benefit to the patient and, consequently, delay the process of development.
The reduction in recruitment time for a historical control study in which data are gathered prospectively on a new method is also less important when evaluating primary surgery due to the extended follow-up required for each case. If, for example, the proposed follow-up of a trial of two methods primary surgery is 10 years and the recruitment time of patients sufficient for a randomized trial is 4 years, the total duration would be 14 years. The potential saving of time in a partially prospective historical control study would be only 2 years (14%).
Intercenter Comparison
The multicenter approach offers distinct advantages for even the busiest cleft lip and palate treatment centers; the generation of adequate samples within specific cleft subtypes treated by contrasting treatment modalities is extremely difficult. Prospectively planned recall of cases at participating centers allows data on outcome to be collected in a standardized way, and rigorous planning and execution across the centers can ensure consecutive case recruitment and unbiased evaluation (Shaw et al., 1992b).
Provided procedures for entry into the study are equivalent in all participating centers, this strategy is extremely valuable in assessing the outcome of primary surgery together with other major components of the treatment program at respective centers. However, it is difficult, if not impossible, to establish the key beneficial or harmful features of a specific treatment as general scientific conclusion due to the invariably complex and arbitrary mix of surgical techniques, timing, sequence, ancillary procedures, and surgical personnel (Shaw et al., 1992a). For example, if two centers differ in the use of presurgical orthopedics and types of primary lip and palate surgery, there is no way to determine which of these procedures might be responsible for any difference in outcome between centers, nor would a null result allow the conclusion that individual aspects of the treatment program are equivalent. The method is, therefore, better suited to comparative clinical audit and quality assurance than definitive clinical research. Significant disparities in outcome of the overall treatment process provide a basis for speculating as to the possible cause, and intercenter studies should therefore be highly motivating toward the generation of specific hypotheses for subsequent trials.
An ambitious audit of this kind was conducted in the United Kingdom, to determine the standards of cleft care in the country as a whole, and included a comparison with centers in other European countries (Beam et al., 2001). Prospectively planned recall of all 5- and 12-year-olds with unilateral cleft lip and palate and standardized recording of a range of outcomes allowed a series of subgroup comparisons as well as comparison with standards elsewhere. The conclusions of this study formed the basis for government recommendations to reconfigure cleft services in the United Kingdom such that care is now concentrated in the hands of a small number of regional specialist centers (Clinical Standards Advisory Group, 1998).
Randomized Controlled Trials
For the comparison of therapies, there is little doubt that the randomized control trial (RCT) is generally the method of choice both scientifically and ethically. Randomization minimizes conscious or unconscious bias in treatment allocation. Prognostic factors, whether known or unknown to the investigator, tend to be balanced between treatment groups. Since patients are followed prospectively according to a clearly defined protocol, missing data are less likely, and the potential loss to follow-up is reduced. If loss does occur, it may be possible to quantify any induced bias, in contrast with retrospective studies, where the researcher may be unaware of patients lost to follow-up, thus introducing bias into the results.
RCTs, however, can also be performed badly. Notably, if the randomization procedure is not strictly applied (i.e., allocation is not fully concealed from the investigators), bias can enter. As with nonrandomized studies, inadequate concealment is associated with higher odds ratios; i.e., an inflated view of effectiveness emerges (Moher et al., 1998).
The ethical issues concerning RCTs in cleft care are interesting (Berkowitz, 1995; Shaw, 1995), particularly the double standards that are applied in clinical experimentation. History tells us that not all surgical innovations are an enduring success. Discredited techniques, though once fashionable, include gastric freezing for bleeding peptic ulcer, carotid body denervation for bronchial asthma, portacaval shunt to prevent esophageal variceal bleeding, nephropexy for viceroptosis, removal of chronically inflamed appendix, and periarterial sympathectomy (Baum, 1981; Salzman, 1985). One dramatic medical example is the prophylactic use of antiarrhythmic drugs during myocardial infarction: at the peak of their use in the late 1980s, it has been estimated that these drugs caused between 20,000 and 70,000 deaths every year in the United States alone, a yearly total of the same order of magnitude as the total number of Americans who died in the Vietnam War (Moore, 1995).
Where the doctor leads, most patients and parents will follow. Such is the desire to shed handicap and stigma. Yet, innovation per se offers no guarantee. Numerous reports show that new treatments are as likely to be worse as they are to be better than existing alternatives (Chalmers, 1997).
What then should be our ethical position? If we wish to test an innovate procedure in an RCT, we must obtain ethical approval from an appropriate body and fully inform each new patient about any uncertainty and risk prior to obtaining signed consent. Ironically, if we wish to test the same innovation on all of our patients, no such rules currently apply (Chalmers and Lindley, 2000). “Ethical codes that seek to protect patients…regulate the responsible investigator but not the irresponsible adventurer” (Lantos, 1994). In the United States, the National Commission for the Protection of Human Subjects recommended that “medical committees should be responsible for ensuring that major innovations undergo proper scientific evaluation” and be charged with “determining which new treatments need to be evaluated, the proper method of evaluation and how to limit the use…prior to the completion of that evaluation” (Tonelli et al., 1996). As yet, no such body exists in the United States or Europe.
Almost 30 years ago, Spriestersbach and co-workers (1973) identified the need for prospective research to resolve the central problems of cleft management. However, remarkably few RCTs have been performed in cleft lip and palate surgery, despite being the surest means of advancing the discipline in the face of overwhelming uncertainty about the relative efficacy of countless different programs of care around the world. In a review of 25 years of the Cleft Palate Journal, only five controlled clinical trials were identified, with only one involving follow-up after surgery of more tha 4 years (Roberts et al., 1991).
Robertson and Jolleys conducted two small RCTs of primary surgery in the 1960s. In first study, a sample was randomized with respect to alveolar bone grafting at the time of primary surgery in infancy (Robertson and Jolleys, 1968). Follow-up revealed a detrimental effect on facial growth in the grafted group (Robertson and Jolleys, 1983). The second study involved two groups of 20 cases, one group having anterior palate closure delayed until age 5 years. No benefit for dentofacial growth was found in delaying hard palate closure (Robertson and Jolleys, 1974). A follow-up study when the children were 11 years of age reached the same conclusion (Robertson and Jolleys, 1990).
In a quasi-RCT on speech outcome, Marsh et al. (1989) alternated (rather than randomized) palate repair with or without intravelar veloplasty in 51 subjects with a broad range of palatal cleft types. Speech evaluations were made at 2-year follow-up. No difference in outcome was detected, but the procedure including intravelar veloplasty required a significantly longer operating time.
Another RCT on speech outcome and maxillary growth in patients with unilateral complete cleft lip and palate operated on at 6 vs. 12 months of age was undertaken in Mexico (Ysunza et al., 1998). The study groups consisted of 41 subjects operated on at 12 months and 35 subjects operated on at 6 months. There was no statistically significant difference in velopharyngeal insufficiency, maxillary arch development, or soft tissue profile as measured on cephalometric radiographs. However, phonological development was significantly better in patients operated at 6 months, and none of the patients in this group developed compensatory articulation. The authors concluded that cleft palate repair performed at 6 months significantly enhances speech outcome and prevents compensatory articulation disorder.
At the Hospital for Research and Rehabilitation of Craniofacial Anomalies, University of Sao Paulo, Brazil, an RCT comparing velopharyngeal function for speech outcomes in two groups of patients with complete unilateral cleft lip and palate has begun (Williams et al., 1998). The two palatoplasty techniques are the von Langenbeck with intravelar veloplasty and the Furlow procedure. A total of 608 patients will be entered into one of two age categories: patients having surgery before 1 year of age and patients undergoing surgery at approximately 1.5 years of age. This study is designed to determine which of the two surgical procedures is superior at constructing a velum capable of affecting velopharyngeal competence for the development of normal speech.
For patients with velopharyngeal insufficiency, secondary surgery to the pharynx is often recommended. The two most popular techniques are pharyngeal flap and sphincter pharyngoplasty. These techniques are presently being tested in a multisite RCT. Patients are evaluated before and at least twice following surgery by perceptual speech evaluation, video nasopharyngoscopy, nasometry, polysomnographic sleep study, lateral cephalometric radiography, audiometry, and tympanometry. When completed, the study should significantly increase our understanding of both operations and allow an objective comparison between them in terms of speech results, incidence of sleep apnea, other complications, and rate of reoperation, as well as operating time, length of hospital stay, and financial costs (Sloan et al., 1996).
Since 1986, north European teams have been developing a concerted program of multidisciplinary intercenter research on cleft lip and palate, including a comparison of surgical outcome in four Scandinavian centers (Friede et al., 1991; Enemark et al., 1993) and six European centers (Shaw et al., 1992a,b; Mølsted al., 1992, 1993a,b; Mars et al., 1992; Asher-McDade et al., 1992; Morrant and Shaw, 1996; Grunwell et al., 2000).
Following these collaborations, the limitations of intercenter studies became increasingly obvious to these teams. In particular, it became clear that would be impossible to separate out the single elements of the package of care provided in the different centers. It was recognized that outcomes of care reflect surgical skill as well as surgical technique and timing and sequence of surgery as well as other auxiliary procedures, such as presurgical orthopedics.
This experience, therefore, provided a compelling stimulus for starting RCTs on primary surgery of clefts. Ten centers are currently participating in a set of three parallel trials, in which teams test their traditional local protocols against a common protocol (Semb and Shaw, 1998). At the time of this writing, almost half of the proposed sample of 450 infants with unilateral cleft lip and palate had been entered.
Randomized trials of nonsurgical intervention have also been completed. These include the use or nonuse of presurgical orthopedics (Prahl et al., 1996), the use or nonuse of arm splints following surgery (Jigjinni et al., 1993), unrestricted sucking after surgery (Lee, 1999), and feeding methods in infancy (Brine et al., 1994; Shaw et al., 1999).
Such efforts demonstrate the feasibility of RCTs in the cleft field and indicate the probable shape of future progress. Thus, trials of sufficient power are likely to be mounted either through collaborations between funding agencies, clinical scientists, and large, possibly developing world centers or through multicenter trials within collaborative groups with strong geographic or cultural links.
Measuring Treatment Outcome
The ultimate goal of cleft care is restoration of the patient to a “normal” life, unhindered by handicap or disability. However, the measurement of normalcy is a highly complex proposition, and there is certainly no index at present that would allow sufficiently sensitive comparison between alternative treatment protocols. Clinical trials focus more on “proximate” outcomes. These mainly represent different aspects of anatomical form and function in the parts affected by the clefting process, often reflecting the particular interests of individual disciplines and provider groups. In essence, most measures will be an indication of the degree of handicap that persists despite (or as a result of) treatment, such as shortcomings in speech, hearing, and dentofacial development.
Outcome Measures
Proximate outcome measures must satisfy several criteria. The easiest to meet is that the measurement should be reproducible between and within examiners.
The most suitable statistic for comparing the reproducibility of different measurement scales is the intraclass correlation coefficient, also referred to as the reliability coefficient. In a research setting, a measurement may be broken up into two components: the “true,” or error-free, measurement of each case and the error, or “noise,” in the measurement that one would wish to minimize. The intraclass correlation coefficient is a ratio of the variation of the error-free measurement to the total variation including measurement error. As a ratio, it is dimensionless and, hence, independent of the units of measurement whether they be distances, angles, or scale points in a rating scale. Consequently, it allows cross-comparison of reproducibility between different methods of measurement.
In the unlikely circumstance that a measurement scale is applied without any measurement error, the variation of error-free measurement will equal the total variation. In this case, the intraclass correlation coefficient equals 1. However, a scale containing substantial error will have a much smaller intraclass correlation coefficient. In the worst case, where the measurement error is so great that the scale is unable to distinguish between cases, the intraclass correlation coefficient is equal to 0. At the design stage of a study, poor reproducibility of an outcome measure may be offset by an increase in sample size. The size required is increased (relative to that for an entirely reliable scale) by the factor 1/R, where R is the intraclass correlation coefficient. It is generally estimated using analysis of variance. If the scale used is categorical, the weighted kappa statistic may be used (Cohen, 1968). This is equivalent to the intraclass correlation coefficient if squared weights are used (Fleiss and Cohen, 1973).
Another strategy to improve the reproducibility of a measure is to use the total or mean of a set of measurements from a panel of observers working independently. This also reduces any bias that may related to the idiosyncratic perceptions of a particular observer. It is possible to estimate the reliability of such a pooled value using the Spearman Brown formula (Fleiss, 1986). If R is the intraclass correlation coefficient for a single observer, then the intraclass correlation coefficient for a measurement obtained by totaling the scores of m observers is as follows:
More difficult is the requirement of validity that the measure truly represents what it is supposed to represent. For example, do the results of a nasendoscopic examination reflect how well the patient sounds to others? Or does a series of cephalometric measurements actually reflect how well the patient looks to others?
External facial appearance is a crucial outcome for patients since this, after all, is what they and society around them actually see. Cephalometric analysis, with its central place in the thinking of orthodontists, is assumed to be an important outcome in its own right, if only as a surrogate measure (Herson, 1989). It is however, an invalid measure of many aspects external facial appearance.
|
|
|
FIG. 34.1. Comparison of maxillo-mandibular profile measurements for unilateral cleft lip and palate, a-n-b, hard tissue; A-N-B, soft tissue. Data derived from the Eurocleft Study. |
A particular problem in the study of a congenital condition such as cleft lip and palate arises when outcomes are assessed in childhood, though eventual form and function will not be known until adulthood. This is especially so for aspects of facial growth such as maxillary prominence since this feature deteriorates steadily during growth (Semb, 1991). A useful way to identify potential outcome measures that are valid and predictive is to examine longitudinal archives. The relative prominence of the maxilla in patients with complete clefts is an important outcome for evaluating the success of primary surgery. One common method for doing this is to measure angle a-n-b, the relationship of the anterior outlines of the maxilla and mandible to the fronto-nasal suture. However, identification of point A on the maxillary outline is difficult in early childhood because of the position of the unerupted permanent incisors. In the Eurocleft study (Mølsted et al., 1992), soft tissue analysis at age 10 for unilateral cleft lip and palate was broadly consistent with that derived from hard tissues (Fig. 34.1), and if the soft tissue A-N-B angle could be shown to be adequately predictive, it would be a good alternative. Indeed, it has the further advantages of being measurable on photographs, obviating the need for irradiation and reflecting the actual facial outline observable in everyday life.
|
TABLE 34.1. Correlation between Measurement of Maxillo-Mandibular Profile at Age 6 and Subsequently in the Same Cases* |
|||||||||||||||||||
|
|||||||||||||||||||
Data from the Oslo Archive (Semb, 1991) were examined at a number of age points to assess how well early measurement of the A-N-B angle would predict the situation in adulthood. To assess the strength of any linear predictive relationship, r2 was calculated between the soft tissue A-N-B angle at age 6 ± 1 year and measurements at a later stage (Table 34.1). Small groups of 20 to 30 unilateral cleft lip and palate patients from Manchester and Oslo were compared in a number of studies at different ages. In Figure 34.2, the average soft tissue A-N-B angle for each center at ages 6, 9, and 12 years is shown. Though the levels of significance for the differences fall just below the 5% level, the differences between each center at different ages are of similar magnitude, reinforcing the predictive worth of the soft tissue A-N-B angle at age 6 ± 1 year.
Measurement Scales
Development of measurement scales that are both reproducible and valid for cleft outcomes is still at an early stage. Preliminary experience comes from comparisons of dentofacial form and relationships in the Eurocleft study using cephalometric analysis for skeletal form (Mølsted et al., 1992), dental arch relationships (Mars et al., 1992), and nasolabial appearance (Asher-McDade et al., 1991, 1992).
|
|
|
FIG. 34.2. Comparison of Oslo and Manchester samples with unilateral cleft lip and palate for different age groups. Group means with 95% confidence limits. |
Cephalometric measurements, though reproducible, suffer a lack of content validity since they measure three-dimensional structures in a two-dimensional way. Nonetheless, cephalometric relationships can tell a great deal about potential growth inhibition for structures undoubtedly affected by surgical procedures (Semb and Shaw, 1996), and they successfully discriminate between different centers (Mølsted et al., 1992).
To compare dental arch relationships, the Goslon Yardstick, an index designed to systematize subjective perception, was used (Mars et al., 1987). Originally, a large sample of study casts was graded by a panel of orthodontists into a series of five groups containing representative cases ranging from the best (group 1) to the worst (group 5) dental arch relationships. These reference groups were subsequently used to assist in grading new cases. A similar grading system was introduced for use on the study casts of 5-year-old subjects with unilateral cleft lip and palate (Atack et al., 1997). This was extensively used in the recent national enquiry in the United Kingdom described above.
In the Eurocleft study, five observers assessed a sample of 149 study casts using the yardstick and a good level of reliability with an intraclass correlation coefficient of 0.80 was obtained (Table 34.2). The mean of the five measurements was then used as a summary score. Application of the Spearman Brown formula suggests that the reliability of this average score is excellent. From the formula above, the estimated value for the mean of five assessments was 0.95. The mean of the five examiners' scores was sensitive to differences between treatment center (Mars et al., 1992).
In a subsequent study (Morris et al., 1994), an attempt was made to discover whether certain measurements could be made directly without the need of assembling a panel of orthodontists. To relate the subjective assessment of the Goslon Yardstick to objective measurement, overjet, overbite, incisor angulation, and various arch form and crossbite relationships were measured on the same series of study casts using a reflex metrograph. These objective measurements were then used as predictors of the mean Goslon score in a multiple regression analysis. Overjet of the incisor on the unaffected side (all cases were unilateral cleft lip and palate) explained a substantial proportion of the variance (r2 = 0.87). The other measures explained only an additional 3% of the variance.
To compare the nasolabial appearance of patients in the Eurocleft study using photographs, several difficulties were confronted. Technical issues such as film quality, lighting, sharpness of image, facial expression, and background general facial appearance were factors that could influence an observer's opinion. To assess the influence of background appearance, such as hair, eyes, and complexion, a panel of observers was asked to examine independently three frontal views: the nasiolabial area in isolation, the full face, and the surrounding features without the nasolabial part (Asher-McDade et al., 1991). Each view was scored in terms of attractiveness using a visual analogue scale. A strong correlation was found between the full face and surrounding area (r = 0.53, p < 0.001), indicating that the full face is likely to be influenced by surrounding features.
Consequently, a more valid measure would be based on restricting the areas under consideration to those directly affected by the anomaly and its repair. Thus, the Eurocleft examiners were asked to assess a standardized view of the frontal and lateral views of the nasiolabial area. We considered it important to break down the task into four components: (1) nasal form (frontal view), (2) deviation of the nose from the midline, (3) shape of the vermilion border, and (4) profile including the upper lip. Each observer was asked to score each of the components on a 5-point subjective scale, from very good appearance to poor appearance. A total score was also computed by aggregation of the scores of the four components. The reliability each component and the total score ranged from 0.47 for nasal form and vermilion border to 0.36 for symmetry (Table 34.2).
|
TABLE 34.2. Evaluation of Reproducibility for Study Cast (Goslon) and Features of Nasolabial Appearance* |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
We found poor reproducibility compared to that obtained for dental arch relationships, and the reliability of the total score was little better than that for the component scores. In this case, the strategy of splitting the assessment into components appeared to have a limited ability to improve the quality of measurement. Detailed analysis of the data suggested that if one observer scored higher than another for one of the four components, that observer was likely to do so for the other three.
For future work, we will test whether reproducibility is improved by reducing the subjective element, providing reference examples, or “benchmarks” as with the Goslon Yardstick, or by enumeration of specific features for rating. Elsewhere, we have found this to improve reproducibility. For example, rating of dental aesthetics is assisted by providing an illustrated 10-point scale, and orthodontic treatment need on dental health grounds is reproducibly rated when clear diagnostic categories are used (Brook and Shaw, 1989).
Benchmarks must be provided with some caution, however, as the choice of categories for each scale and the subject of each subscale determine the content validity of the total measurement. Thus, there is a danger of imposing the researchers' perceptions of what is important.
An alternative strategy for rating appearance is to rank subjects pairwise against each other. All possible pairs of subjects are compared (Tobiasen, 1989). For each pair, a score of 1 is allocated to the preferred photograph. The score for each case is then its total after comparison with all other cases. One practical difficulty, however, is that the number of comparisons escalates with sample size. The number of possible pairs is equal to n(n - l)/2 for a sample of n cases. Thus, for a sample of 10 subjects, 45 comparisons are needed; for 50 subjects, the number rises to 1225. However, the pairwise technique might be modified by comparing each case against a random or systematic sample of other subjects. For example, the photographs for the complete sample might be arranged in a random sequence and then a score for each case obtained by comparing against the next k subjects. The larger choices for k would improve the reliability of the score for each case but increase the total time to perform the task. A more fundamental limitation is that the scale is meaningful only with regard to the relationship between subjects within the same sample since the comparison is not transferable from one study to another.
The static nature of a still photograph, of course, is a major weakness with respect to validity since the lip in function cannot be judged. Consequently, the use of video recording has been explored (Morrant and Shaw, 1996). An edited sequence for a series of 30 subjects using a number of standardized views of the nasolabial area at rest and function was prepared. A panel of judges then rated the cases using a scoring chart with nine responses for the lip and 10 for the nose.
The nose and lip were assessed separately with eight components and an overall score for lip and nine components and an overall score for the nasal area. For the nasal area, the intraclass correlation coefficient for individual components ranged from 0.40 to 0.27 with 0.52 for the overall score and 0.49 for the sum of the components. For the lip, the intraclass correlation coefficient ranged from 0.39 to 0.10 with 0.28 for the overall score and 0.34 for the sum of components.
While such a dynamic view may be more valid, the interexaminer agreement was generally worse than that achieved from a static image. This may reflect the significantly higher content of information contained in the video format, and by further discussion of the items to be rated and possible provision of improved descriptive categories or illustrated examples, an appearance scale of high validity seems feasible.
Treatment Costs
In most developed countries, the high medical costs of rehabilitating the child with a cleft are borne, or at least supported in part, by the state. Economic pressures around the world now force a re-examination of the true financial costs of treatment, and with declining budgets, clinicians must either be involved in cost control or have arbitrary choices imposed upon them. Surgery is invariably expensive, and successful primary operations that minimize the need for multiple secondary revisions are highly desirable. Furthermore, successful primary repairs are likely to reduce the duration and complexity of ancillary procedures, such as speech therapy, orthodontics, and maxillary osteotomy.
In economic terms, the cheapest care will certainly be that which is provided with a high degree of planning and coordination. Common examples might be combining the placement of drainage tubes with another operation, timing alveolar bone grafting so that natural space closure is facilitated and the duration of subsequent orthodontics is minimized, and recognizing a later need for maxillary osteotomy so that inappropriate early orthodontics is avoided.
Burden of Care
Since the consequences of an orofacial cleft are apparent through every phase of childhood and adolescence, there is seldom a time when the disciplines involved in care could not recommend some or other intervention. The powerful desire of patients and parents to reach the point where the stigma of clefting will be completely eradicated makes it likely that they will accept most proposals. Most patients and parents will willingly comply with protocols of care recommended by all members of their team, no matter how demanding they may be. They have little choice.
So far, the burden of care has received little attention in cleft care, yet the combined total of operations, other treatment episodes, and review appointments for the first 20 years of life, including all of the disciplines that may be involved, can easily exceed 100. Apart from the pain and suffering and disruption of family life and school attendance, the dependent role in which this places the patient may have an adverse effect on the patient's sense of self-determination or locus of control.
Balancing Outcome, Cost, and Burden
Decision analysis (utility theory) is the science of relative utilities or preferences and can be employed to formulate global outcome measures that are more relevant to patients than the short-sighted proximate outcomes that absorb clinicians. Clearly, it behooves the providers of cleft care to seek optimal balance in the development of protocols of care spanning the years from birth to late adolescence.
Undoubtedly, there are intriguing discussions ahead, e.g., the use of early orthopedics and delayed palatal closure. Patients receiving this treatment have to be brought on many additional visits to the treatment center during infancy and childhood and endure the minor risk of impression as well as the greater risk of discomfort associated with the appliances. For the years until the hard palate is closed, they must tolerate a residual cleft or an obturator, while having a clear sense of being an orthodontic patient. How much benefit does this additional treatment have to produce over treatment without early orthopedics to justify its inclusion in care: an average increase in angle s-n-a (maxillary prominence) of 1 degree, a 10% reduction in osteotomy rate in the teens, or a better cosmetic repair of the lip? (If differences do exist, how would they be measured?) Similar questions arise about other elements of care. How often should lip or nasal revision be attempted? When does the law of diminishing returns start to apply? Surgical management of velopharyngeal incompetence is not without some risk, so how bad does the problem have to be to justify the average gain from pharyngoplasty or flap? How much orthodontics should be performed in childhood if the duration and outcome of definitive treatment in the permanent dentition will not be radically altered? These are issues about which patients and parents deserve honest information and the opportunity to have their preferences taken into account.
Conclusion
Regrettably, evidence-based care for children with cleft lip and palate is scarcely available. However, there is some evidence that simple treatment protocols that minimize the burden on the child can produce equivalent or better results than complex ones (Shaw et al., 1992a; Shaw, 1997) and no evidence that the opposite is true. Furthermore, centralization of treatment with therapy provided by high-volume operators provides the best setting for good and comprehensive care and at least allows quality assessment within a reasonable period (Shaw et al., 1996).
It has been said that doctors make choices about treatment in three ways: seduction, induction, or deduction. In the seductive method, the clinician simply adopts what he or she has been taught or encouraged to do by teachers or colleagues, i.e., treatment based on faith. The inductive method includes choices based on clinical experience, i.e., on what seems to work or on theories of what ought to work. For example, do extensive muscle repositioning during primary lip repair because this will encourage growth, do minimal tissue mobilization and disturbance during surgery because more surgery means more scar-induced growth disturbance, do not touch the vomer during lip repair because it is a growth center, or do use a vomerine flap because there is no evidence for growth disturbance and it permits good arch development, minimizes fistulae, and provides a good nasal floor for later bone grafting. Finally, there is the deductive, or hypotheticodeductive, method, in which decisions are made on the unbiased evidence of randomized trials. Initiating multicenter collaborations and protocols for these trials is the challenge that must be grasped by today's clinicians who wish to choose the best treatment for their patients. In time, randomized trials will be aggregated in systematic reviews (Chalmers and Altman, 1995), providing as never before a sound evidence base for provision of cleft care.
Following a recent survey of European cleft centers, the register of 201 teams revealed the use of 194 protocols (Shaw et al., 2000). The start of a new millennium appears to be a timely point for concerted action to leave this morass of clinical uncertainty behind. The unfortunate consumers of cleft care certainly deserve better.
References
Asher-McDade, C, Brattstrom, V, Dahl, E, et al. (1992). A sixcenter international study of treatment outcome in patients with clefts of the lip and palate. Part 4. Assessment of nasolabial appearance. Cleft Palate Craniofac J 29: 409–412.
Asher-McDade, C, Roberts, C, Shaw, WC, Gallagher, C (1991). Development of a method for rating nasolabial appearance in patients with clefts of the lip and palate. Cleft Palate Craniofac J 28: 385–391.
Atack, N, Hathorn, I, Mars, M, Sandy, J (1997). Study models of 5 year old children as predictors of surgical outcome in unilateral cleft lip and palate. Eur J Orthod 19: 165–170.
Baum, M (1981). Scientific empiricism and clinical medicine: a discussion paper. J R Soc Med 74: 504–509.
Beam, D, Mildinhall, S, Murphy, T, et al. (2001). Cleft lip and palate care in the United Kingdom—The Clinical Standards Advisory Group (CSAG) Study. Part 4: outcome comparisons, training, and conclusions. Cleft Palate Craniofacial J 38: 38–43.
Bergland, O, Semb, G, Abyholm, FE (1986). Elimination of the residual alveolar cleft by secondary bone grafting and subsequent orthodontic treatment. Cleft Palate J 23: 175–205.
Berkowitz, S (1995). Ethical issues in the case of surgical repair of cleft palate. Cleft Palate Craniofac J 32: 271–276.
Boyne, PJ, Sands, NR (1972). Secondary bone grafting of residual alveolar and palatal clefts. J Oral Surg 30: 87–92.
Boyne, PJ, Sands, NR (1976). Combined orthodontic-surgical management of residual palatoalveolar cleft defects. Am J Orthod 70: 20–37.
Brine, EA, Rickard, KA, Brady, MS, et al. (1994). Effectiveness of two feeding methods in improving energy intake and growth of infants with cleft palate: a randomized study. J Am Diet Assoc 94: 732–738.
Brook, PH, Shaw, WC (1989). The development of an index of orthodontic treatment priority. Eur J Orthod 11: 309–320.
Chalmers, I (1997). What is the prior probability of a proposed new treatment being superior to established treatments. BMJ 314:74–75.
Chalmers, I, Altman, DC (1995). Systematic Reviews. London: BMJ.
Chalmers, I, Lindley, R (2001). Double standards on informed consent to treatment. In: Informed Consent in Medical Research. Edited by L Doyal and JS Tobias. London: BMJ Publications, Chapter 26.
Clinical Standards Advisory Group (1998).Cleft Lip and/or Palate. London: Department of Health.
Cohen, J (1968). Weighted kappa nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 70: 213–220.
Dickersin, K, Min, YI (1993). NIH clinical trials and publication bias. Online J Curr Clin Trials doc 50.
Dickersin, K, Min, YI, Meinert, CL (1992). Factors influencing publication of research results: follow-up of publications submitted to two institutional review boards. JAMA 267: 374–378.
Easterbrook, PJ, Berlin, JA, Gopalan, R, Matthews, DR (1991). Publication bias in clinical research. Lancet 337: 867–872.
Egger, M, Zellweger-Zahner, T, Schneider, M, et al. (1997). Language bias in randomised controlled trials published in English and German. Lancet 350: 326–329.
Enemark, H, Friede, H, Paulin, G, et al. (1993). Lip and nose morphology in patients with unilateral cleft lip and palate from four Scandinavian centres. Scand J Plast Reconstr Hand Surg 27:41-47.
Fleiss, JL (1986). Design and Analysis of Clinical Experiment. New York: Wiley.
Fleiss, JL, Cohen, J (1973). The equivalence of weighted kappa and the intra-class correlation coefficient as a measure of reliability. J Educ Psychol Measurement 33: 613–619.
Friede, H, Enemark, H, Semb, G, et al. (1991). Craniofacial and occlusal characteristics in unilateral cleft lip and palate patients from four Scandinavian centres. Scand J Plast Reconstr Hand Surg 25: 269–276.
Gehan, EA (1984). The evaluation of therapies: historical control studies. Stat Med 3: 315–324.
Grunwell, P, Brandsted, K, Henningsson, G, et al. (2000). A sixcentre international study of outcome of treatment in patients with clefts of the lip and palate: the results of a cross-linguistic investigation of cleft palate speech. Scand J Plast Reconstr Hand Surg 34: 219–229.
Herson, J (1989). The use of surrogate endpoints in clinical trials (an introduction to a series of four papers). Stat Med 8: 403–404.
Jigjinni, V, Kangesu, T, Sommerlad, BC (1993). Do babies require arm splints after cleft palate repair. Br J Plast Surg 46: 681–685.
Jolleys, A, Robertson, NR (1972). A study of the effects of early bone grafting in complete clefts of the lip and palate—five year study. Br J Plast Surg 25: 229–237.
Kunz, R, Oxman, AD (1998). The unpredictability paradox: review of empirical comparisons of randomised and nonrandomised clinical trials. BMJ 317: 1185–1190.
Lantos, J (1994). Ethical issues—How can we distinguish clinical research from innovative therapy? Am J Pediatr Hematol Oncol 16: 72–75.
Lee, TK (1999). Effect of unrestricted postoperative sucking following cleft repair on early postoperative course. In: Proceedings of the 4th Asian Pacific Cleft Lip and Palate Conference, Fukuoka, Japan. September 28–30.
Mars, M, Asher-McDade, C, Brattstrom, V, et al. (1992). A six-center international study of treatment outcome in patients with clefts of the lip and palate. Part 3. Dental arch relationships. Cleft Palate Craniofac J 29: 405–408.
Mars, M, Flint, DA, Houston, WJB, et al. (1987). The Goslon Yardstick: a new system of assessing dental arch relationships in children with unilateral clefts of the lip and palate. Cleft Palate J 24: 314–322.
Marsh, JL, Grames, LM, Holtman, B (1989). Intravelar veloplasty: a prospective study. Cleft Palate J 26: 46–50.
Moher, D, Pham, B, Jones, A, et al. (1998). Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 352: 609–613.
Mølsted, K, Asher-McDade, C, Brattstrom, V, et al. (1992). A sixcenter international study of treatment outcome in patients with clefts of the lip and palate. Part 2. Craniofacial form and soft tissue profile. Cleft Palate Craniofac J 29: 398–404.
Mølsted, K, Dahl, E, Brattstrom, V, et al. (1993a). A six-center international study of treatment outcomes in children with clefts of the lip and palate: evaluation of maxillary asymmetry. Cleft Palate Craniofac J 30: 22–28.
Mølsted, K, Dahl, E, Skovgaard, LT, et al. (1993b). A multicenter comparison of treatment regimens for unilateral cleft lip and palate using a multiple regression model. Scand J Plast Reconstr Hand Surg 27: 277–284.
Moore, T (1995). Deadly Medicine. New York: Simon and Schuster.
Morrant, DG, Shaw, WC (1996). Use of standardized video recordings to assess cleft surgery outcome. Cleft Palate Craniofac J 33: 134–142.
Morris, T, Roberts, C, Shaw, WC (1994). Incisal overjet as an outcome measure in unilateral cleft lip and palate management. Cleft Palate Craniofac J 31: 142–145.
Pinsky, CM (1984). Experience with historical control studies in cancer immunotherapy. Stat Med 3: 325–329.
Pollock, AV (1986). Historical evolution: methods, attitudes, goals. In: Principle and Practice of Research: Strategies for Surgical Investigators, edited by H Troidl, WO Spitzer, B McPeak, et al. New York: Springer-Verlag, pp. 7–17.
Prahl, C, Kuijpers-Jagtman, AM, Prahl-Andersen, B (1996). A Study into the Effects of Presurgical Orthopaedic Treatment in Complete Unilateral Cleft Lip and Palate Patients. A Three Center Prospective Clinical Trial in Nijmegen, Amsterdam and Rotterdam. Interim Analysis. Nijmegen: Academisch Ziekenhuis.
Rehrmann, AH, Koberg, WR, Koch, H (1970). Long-term postoperative results of primary and secondary bone grafting in complete clefts of lip and palate. Cleft Palate J 7: 206–221.
Roberts, CT, Semb, G, Shaw, WC (1991). Strategies for the advancement of surgical methods in cleft lip and palate. Cleft Palate Craniofac J 28: 141–149.
Robertson, NR, Jolleys, A (1968). Effects of early bone grafting in complete clefts of lip and palate. Plast Reconstr Surg 42: 414–421.
Robertson, NR, Jolleys, A (1974). The timing of hard palate repair. Scand J Plast Reconstr Surg 8: 49–51.
Robertson, NR, Jolleys, A (1983). An 11-year follow-up of the effects of early bone grafting in infants born with complete clefts of the lip and palate. Br J Plast Surg 36: 438–443.
Robertson, NRE, Jolleys, A (1990). A further look at the effects of delaying repair of the hard palate. In: Cleft Lip and Palate. Longterm Results and Future Prospects, edited by AG Huddart and MWJ Ferguson. Manchester: Manchester University Press. Chapter 11, 176–182.
Sackett, DL, Straus, SE, Richardson, WS, et al. (eds) (2000). Evidence Based Medicine. How to Practice and Teach EBM. Edinburgh: Churchill Livingstone.
Salzman, EW (1985). Is surgery worthwhile? Arch Surg 120: 771–776.
Sandy, J, Williams, A, Mildinhall, S, et al. (1998). The Clinical Standards Advisory Group (CSAG) Cleft Lip and Palate Study. Br J Orthod 25: 21–30.
Semb, G (1988). Effect of alveolar bone grafting on maxillary growth in unilateral cleft lip and palate patients. Cleft Palate J 25: 288–295.
Semb, G (1991). A study of facial growth in patients with unilateral cleft lip and palate treated by the Oslo CLP team. Cleft Palate J 28: 1–21.
Semb, G, Shaw, WC (1996). Facial growth in facial clefting disorders. In: Facial Clefts and Craniosynostosis, Principles and Management, edited by TA Turvey, KWL Vig, and RJ Fonseca. Philadelphia: Saunders, pp. 28–56.
Semb, G, Shaw, WC (1998). Facial growth after different methods of surgical intervention in patients with cleft lip and palate. Acta Odontol Scand 56: 352–355.
Shaw, WC (1995). Commentary to “Ethical issues in the case of surgical repair of cleft palate” by S. Berkowitz. Cleft Palate Craniofac J 32: 277–280.
Shaw, WC (1997). The Eurocleft Study—eight year follow-up. Paper presented at the 8th International Congress on Cleft Palate and Related Craniofacial Anomalies, 7-12 September 1997, Singapore.
Shaw, WC, Asher-McDade, C, Brattstrom, V, et al. (1992a). A sixcenter international study of treatment outcome in patients with clefts of the lip and palate. Part 5. General discussion and conclusions. Cleft Palate Craniofac J 29: 413–418.
Shaw, WC, Bannister, RP, Roberts, CT (1999). Assisted feeding is more reliable for infants with clefts—151;a randomized trial. Cleft Palate Craniofac J 36: 262–268.
Shaw, WC, Dahl, E, Asher-McDade, C, et al. (1992b). A six-center international study of treatment outcome in patients with clefts of the lip and palate. Part 1. Principles and study design. Cleft Palate Craniofac J 29: 393–397.
Shaw, WC, Semb, G, Nelson, PA, et al. (2000). The Eurocleft Project 1996-2000. Amsterdam: IOS Press.
Shaw, WC, Williams, AC, Sandy, JR, Devlin, HB (1996). Minimum standards for the management of cleft lip and palate: efforts to close the audit loop. Ann R Coll Surg Engl 78: 110–114.
Sloan, G, Shaw, WC, Downey, SE (1996). Surgical management of velopharyngeal insufficiency: pharyngeal flap and sphincter pharyngoplasty. In: Facial Clefts and Craniosynostosis, Principles and Management, edited by TA Turvey, KWL Vig, and RJ Fonseca. Philadelphia: Saunders, pp. 384–395.
Spriestersbach, DC, Dickson, DR, Fraser, FC, et al. (1973). Clinical research in cleft lip and palate: the state of the art. Cleft Palate J 10: 113–165.
Stern, JM, Simes, RJ (1997). Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ 315: 640–645.
Tobiasen, JM (1989). Scaling facial impairment. Cleft Palate J 26: 249–254.
Tonelli, MB, Benditt, JO, Albert, RK (1996). Lessons from lung volume reduction surgery. Chest 110: 230–238.
Williams, WN, Seagle, MB, Nackashi, AJ, et al. (1998). A methodology report of a randomized prospective clinical trial to assess velopharyngeal function for speech following palatal surgery. Control Clin Trials 19: 297–312.
Ysunza, A, Pamplona, MC, Mendoza, M, et al. (1998). Speech outcome and maxillary growth in patients with unilateral complete cleft lip/palate operated on at 6 versus 12 months of age. Plast Reconstr Surg 102: 675–679.
