Background: The implementation of clinical guidelines in care pathways is being promoted for quality assurance in psychiatry and psychotherapy, as in other medical fields. The achievable benefits are disputed and are generally thought to be small. There have been hardly any studies of the effect of clinical care pathways on the costly inpatient treatment of schizophrenic psychoses.
Methods: We conducted a prospective, controlled, before and after study in 114 patients with schizophrenia to determine whether the implementation of a pathway would improve diagnosis and treatment in conformity with published guidelines, and whether there would be any associated improvement in outcome. The patients’ course was extensively documented with a number of structural, process-related, and outcome-related variables in the years before and after pathway implementation. Moreover, two different intensive methods of pathway implementation were tested. Data were collected from 2003 to 2005. The primary indicators of outcome quality included pharmacotherapy-related variables and assessments of treatment efficacy by the physicians, the nurses, and the patients themselves.
Results: After pathway implementation, some diagnostic tests that had been performed only rarely beforehand were performed much more often. The percentage of over- or undermedicated patients, as defined by the treatment pathway, declined markedly. Surprisingly, however, the patients’ multidimensionally documented psychopathological course and their subjective judgments of their condition were worse after pathway implementation than before on all four scales that were used to assess these variables.
Conclusion: The implementation of a treatment pathway brought about a robust change in process-related variables. The findings of this study furnish no explanation for the observed decline in treatment efficacy.
Meticulous attention to guidelines and their local implementation with the aid of clinical treatment pathways are now being promoted by medical specialty associations, governmental authorities, and health-insurance carriers for the purpose of quality assurance (1). Psychiatric guidelines are now available that meet high standards of quality for both method and content, including those of the German Association for Psychiatry and Psychotherapy (Deutsche Gesellschaft für Psychiatrie, Psychotherapie und Nervenheilkunde, DGPPN) or the National Institute for Health and Clinical Excellence. It is assumed that such guidelines would effectively assure quality if properly applied in routine clinical practice and be used as the primary decision-making aids that they were designed to be (2, 3). As the creation of guidelines uses up considerable resources (4), they must generate a real benefit to patient care in order to be worthwhile.
Nonetheless, the relevant studies that have been carried out to date have not demonstrated any lasting improvement in the quality of care (5). Guidelines have been found to have no more than a weak influence on physicians’ behavior or on the outcome of treatment (6); whatever effect they have is generally transitory and highly dependent on local implementation strategies (7). Thus, one-time measures such as the imparting of information by didactic lectures, the distribution of printed texts, or the use of checklists are largely ineffective, while combined measures for permanent, continuous support—e.g., regular feedback, a local quality circle, or counseling by trained personnel – seem to be more successful. This general rule also holds, in particular, for the inpatient treatment of patients with schizophrenic psychoses, which is very costly (8), highly variable in quality (9), and thus clearly in need of improvement. Only two methodologically adequate studies on this topic have been published to date (10, 11). Vague assertions are still often made in print to the effect that guideline conformity improves various aspects of treatment outcome (9, 10, 12, 13), but the matter is not at all clear and needs to be examined more closely.
We accordingly decided to study whether the implementation of a treatment pathway for schizophrenia actually improves the process variables of management (diagnostics and treatment) as recommended by published guidelines, and also whether this had any effect on the multiple variables by which outcome is measured (psychopathology, subjective condition, medications on discharge). We performed a before and after comparison and tested the effects of two different implementation strategies.
In a prospective before and after study, we documented the course of treatment of 114 schizophrenic patients in detail, in terms of variables relating to management structures, processes, and outcomes, for one year before and one year after the introduction of a clinical treatment pathway. These data were collected in two similar, open, general psychiatric wards, where the treatment pathway was implemented in two different ways:
An overview of the two-year process of data collection and of the patients included in the study is found in Figure 1.
The study was carried out with prior approval of the responsible institutional ethics committee (Charité Universitätsmedizin Berlin, Campus Benjamin Franklin). The data were collected from 2003 to 2005 at a hospital psychiatric service serving the Steglitz-Zehlendorf district of Berlin. In this naturalistic study, random allocation of patients to the two wards of the service was not feasible. Patients were allotted to whichever ward had a free bed on the day they were admitted.
The treatment pathway was created during the first year of the study by an interdisciplinary working group, none of whose members participated in data collection for the study. The schizophrenia treatment guidelines of the DGPPN, the ICD-10 classification of mental disorders, and the evidence-based pharmacotherapy recommendations of the PORT (Schizophrenia Patient Outcomes Research Team [14, 15]) were used as the basis for the treatment pathway. Its specifications for ancillary diagnostic testing, psychotherapeutic modalities, other specialized treatments, and psychosocial counseling were developed and approved by the staff of the psychiatric service. The pathway’s components were not all compulsory to the same extent: they ranged from general outlines of optional offerings, such as exercise therapy, to binding rules for diagnostic assessment and drug treatment. The pathway was made available in computerized form as a flowchart graphic whose elements could be clicked on to show the relevant text document for each (Figure 2), and the individual documents were connected to each other with links.
On each of the two wards, all patients being treated for a main diagnosis of schizophrenic psychosis (ICD-10 code F20.x) were candidates for inclusion in the study. It was an obligate criterion for inclusion that the manifestations of schizophrenic psychosis had to be the most prominent component of the patient’s clinical condition up to the time of discharge. The criteria for exclusion included inadequate German-language skills for responding to questionnaires and short hospitalizations (for seven days or less), which could generally not be assumed to be routine treatments for acute schizophrenic exacerbations. Data were collected on a maximum of one admission per patient; patients admitted to the service multiple times during the study period were only included in the study the first time.
Structural variables—the patients’ sociodemographic data, past medical history, and psychopathology on admission—were documented on established rating scales by the treating physicians (Brief Psychiatric Rating Scale [BPRS]) (16) and nurses (Nurses’ Observation Scale for Inpatient Evaluation [NOSIE]) (17). Rater-training sessions for the physicians and nurses were held every six months over the course of the study. The patients assessed their own conditions with the German-language Frankfurt self-assessment scale for persons with schizophrenia (Frankfurter Befindlichkeits-Skala für schizophren Erkrankte, FBS) (18).
The following process-related variables were recorded at prescribed times, predetermined in relation to the course of the individual patient’s pharmacotherapy (e.g., change of antipsychotic drug for treatment resistance at five weeks, or change to clozapine for treatment resistance at nine weeks):
In this article, we report the findings with regard to diagnostic evaluation and pharmacotherapy.
The outcome variables concerned both pharmacotherapy and the patients’ psychopathology and subjective condition on discharge. An overview of all data collected on the various data collection days over the course of treatment is given in Figure 3.
The following were used as indicators of process quality in conformity with guidelines that were likely to improve patient safety:
As a further indicator of outcome quality beyond these drug-related treatment variables, the percent improvement of BPRS, NOSIE, and FBS scores in comparison to their admission values was recorded.
As the variables were of diverse types, the statistical tests used to assess them were mostly not distribution-based; the particular tests used in each instance are mentioned in parentheses in the Results section, below. All variables were tested for differences between the period before and the period after introduction of the pathway (“pre” vs. “post”), as well as for differences between the two implementation strategies (“active” vs. “passive”). The p-values given here are meant to serve only as exploratory aids to interpretation; no α-error correction was performed. The statistical analysis was carried out and documented with the aid of SPSS 14.0.
114 patients were included in the study. Their sociodemographic characteristics and aspects of their past medical history are shown in Table 1.
For clinical reasons, randomization was not possible. We tested for potential differences, both cross-sectional (Ward A vs. Ward B) and longitudinal (“pre” vs. “post”), between patient groups. The groups did not differ significantly with respect to any of the following variables: present age, age at onset of schizophrenia (unifactorial univariate analysis), sex, highest educational level attained, completion or non-completion of vocational training, living situation, marital/familial status, and the distribution of first-episode diseases (chi square test) and the duration of schizophrenia (Kruskal-Wallis-H).
The summated scores on the BPRS (Kruskal-Wallis-H), NOSIE, and FBS (unifactorial univariate variance analysis) on admission were compared across the four patient groups as an indicator of the severity of disease. No significant intergroup differences between groups were found.
As the patient groups for longitudinal and cross-sectional analysis had the same initial characteristics, any observed differences in treatment processes and outcomes (see below) are likely to be due to real differences in treatment on the two wards and in the two temporal periods in which data were collected.
For the recommended diagnostic evaluation, the pathway specified that a brain imaging study should be obtained for all patients initially presenting with schizophrenia. There were only 19 such patients in the entire study (Table 2); the percentage undergoing brain imaging was marginally higher on Ward B, where the pathway was actively disseminated (86% vs. 75% on Ward A), but the difference was not statistically significant (chi square test, Fisher-Yates).
Women up to age 50 should undergo a pregnancy test before taking antipsychotic medication to minimize the risk of birth defects. On both wards, the introduction of the pathway brought about a marked rise in the frequency of pregnancy testing, and there was also a clear difference between wards. On the ward with active pathway dissemination (Ward B), the frequency of pregnancy testing was significantly higher than before (Fisher-Yates: p = 0.037) and also significantly higher than on the other ward (Ward A), where the pathway was passively implemented (Fisher-Yates: p = 0.021).
According to the treatment pathway, all patients taking antipsychotic medication should have an ECG every two weeks. Pathway implementation resulted in a mean reduction of the interval between ECGs by ten days (Ward A) and by four days (Ward B). Neither these reductions nor the difference between them was statistically significant (Kruskal-Wallis-H).
A further recommendation was for drug monitoring prior to changing neuroleptics at five weeks because of treatment resistance. Before the pathway was introduced, this was done in only 1 of 3 such cases on Ward A and 1 of 6 cases on Ward B. Afterward, the frequency of drug monitoring in this situation increased on both wards, but the increase was statistically significant only on the ward with active dissemination (from 15% to 69%, p = 0.004 by the chi square test).
The results regarding quality indicators for pharmacotherapy over the course of treatment are presented exemplarily for Day 35 and as average daily doses over the entire period of hospitalization.
No significant intergroup differences were found in the percentage of patients taking only one neuroleptic drug, which fluctuated in the range of 39% to 71% over all of the days on which this parameter was determined. The two wards did not differ from each other in this respect either before or after introduction of the pathway, nor was pathway introduction associated with any significant before-and-after change (Kruskal-Wallis-H).
After pathway introduction, on Day 35 of treatment, 50% of patients were found to be underdosed on the ward where the pathway was passively disseminated, but only 7% on the ward where it was actively disseminated; this difference was statistically significant (p = 0.041, Fisher-Yates). All other intergroup changes and differences with respect to under- or overdosage were insignificant.
The average daily dose over the entire course of treatment went down on both wards after pathway introduction. This reduction was statistically significant among patients treated for the initial presentation of schizophrenia on Ward A (passive dissemination), from 886 to 450 CPZ equivalents (p = 0.008, Mann-Whitney-U). These figures, however, are derived from only 2 patients, so the finding, while statistically significant, cannot be used to draw any firm conclusion.
Over the entire course of the study, anticholinergic drugs were used only in rare, exceptional cases to treat extrapyramidal motor side effects.
Antipsychotic monotherapy was achievable on discharge from the hospital in about 50% of patients both before and after introduction of the pathway. The percentage of patients who were underdosed (as specified by the pathway) at the time of discharge went down significantly after pathway introduction on the ward with active dissemination (p = 0.046, chi square test). None of the other quality indicators for pharmacotherapy were significantly changed by the introduction of the pathway (see also “Process variables,” above; add-on use of clozapine for treatment resistance for more than nine weeks, need for anticholinergic medication).
Pathway introduction was found to be associated with significant changes in the patients’ scores for psychopathology on the rating scales that were used (Figure 4): the mean improvement in psychopathology on the BPRS scale, rated by physicians, was about 30% before the pathway was introduced, but only about 15% afterward (p = 0.002, t-test). Thus, the introduction of the pathway actually lowered the efficacy of treatment. This paradoxical effect was observed and was statistically significant with either active or passive dissemination. Similarly, the improvement in psychopathology over the course of treatment was found to be less after pathway introduction when rated by the nursing staff on the NOSIE scale, although this worsening was not statistically significant. As for the patients’ self-assessments with the FBS, treatment efficacy was found to be lower after pathway introduction on Ward A (passive dissemination) and nearly unaltered on Ward B (active dissemination), but these differences did not reach statistical significance.
Findings implying lower treatment efficacy, some of which were statistically significant, were also obtained on the Clinical Global Impressions scale (CGI, Kolmogoroff-Smirnov test) and the SF36 dimension “mental well-being” (p = 0.025, Mann-Whitney-U).
In summary, some types of diagnostic test for the purpose of patient safety were carried out relatively rarely before introduction of the pathway and much more frequently thereafter—mainly when the treating physicians were notified that these measures were indicated, as specified by the pathway. There was no such direct notification in other studies of clinical pathways (6, 9, 13, 21), which perhaps explains why the present study reveals a more lasting effect on physicians’ behavior than is generally documented in published reports (5).
The guideline-specified quality indicators for pharmacotherapy did not all change in the desirable direction after the clinical pathway was introduced. At any rate, the underdosing of patients during hospitalization and on discharge became significantly less common with personal notification of the treating physicians, while the overdosing of patients initially presenting with schizophrenia became significantly less common even with so-called passive pathway implementation.
On the other hand, multidimensional assessments of treatment-induced changes in the patients’ psychopathology and subjective condition, as evaluated by the physicians, the nurses, and the patients themselves, showed that the treatment outcomes were actually worse after pathway introduction than they had been before. The worsening was found to be statistically significant on some of the rating scales that were used to assess clinical efficacy. This surprising result cannot be directly explained by any other findings of the study.
We considered whether structural changes might provide an explanation for the worse outcomes. At the same time that the pathway was introduced, Ward A moved from one building to another, and the senior attending psychiatrist in charge of the ward was replaced by a new one. Yet the new facilities were neither more nor less comfortable than the old ones and had roughly the same degree of space and building infrastructure, and the old and new senior attending psychiatrist were equally experienced and had both been trained in the same clinic. Furthermore, worse treatment outcomes were seen not just on Ward A, but on Ward B as well, where no comparable changes had occurred. All of the other structural variables that we examined, e.g., patient characteristics, length of stay, or size of medical and nursing staff, remained constant in both wards over the period of the study. It should be noted, too, that similar paradoxical effects have been reported for the guideline-oriented treatment of depression (5, 22).
Because of methodological limitations, the conclusions of this exploratory pilot study are neither entirely firm nor immediately generalizable (small sample size, lack of power analysis, lack of randomization, study carried out in a single center with “pathway ambitions,” lack of α-error adjustment for multiple statistical testing). Some degree of uncertainty remains about the comparability of patient groups and about the possible confounding effect of secular trends over the course of this before and after study. In future studies, changes might be more accurately detected by spacing the “pre” and “post” phases of data collection more widely apart, to avoid contamination of the results by any momentary changes that might taking place around the time of pathway introduction. The use of blinded raters would be a further methodological gain.
As the mere knowledge of guidelines has been found to have little or no influence on therapeutic behavior (6), conformity checking with personal feedback to the treating physicians might seem to be a good way to insure that guidelines were more consistently followed. This can be done economically only through the use of electronic hospital information systems, and only if the diagnostic and therapeutic steps to be undertaken are laid down as compulsory rules and their implementation (or otherwise) is electronically documented, automatically assessed, and fed back to the user in timely fashion. Yet, even if this is done, electronic feedback without further consequences can easily be clicked away and ignored. At present, therefore, the best way to assure guideline implementation seems to be a continuous discussion of the need for conformity—and for deviations from conformity in certain well-defined cases—with respected leaders of the medical team.
In our experience, the creation of a clinical pathway by physicians and nurses working together can foster discussion among all caregivers on the service and improve the professional abilities of everyone involved. We think that the findings relating to outcome probably have little to do with the particular contents of the guideline that was used, as the quality criteria first had to be agreed upon by a consensus among the staff of the psychiatric service. These criteria are regularly re-evaluated at specified intervals to keep them up to date.
To insure that laboriously generated guidelines pay off in a benefit to the patients treated under them, some of the invested resources should be devoted to the creation of standardized ordering forms and other support measures for implementation. The implementation of evidence-based quality criteria in this way might, in the future, become a requirement of audits and certification procedures for clinical services. Before this is done, however, more research will be needed to find out why, in this study, improved implementation paradoxically worsened what it had been meant to improve above all, namely, the patients’ clinical outcome.
The authors thank Mr. René Berton for his inestimable help with data management and Professor Mackert for his kind support.
Conflict of interest statement
Bruno Steinacher has received lecture honoraria from Novartis and reimbursement of medical meeting participation fees from Lilly. The other two authors state that no conflict of interest exists.
Manuscript submitted on 22 March 2012, revised version accepted on 29 August 2012.
Translated from the original German by Ethan Taub, M.D.
Dr. med. Dr. phil. Bruno Steinacher, MBA
Vivantes Wenckebach-Klinikum, Klinik für Psychiatrie, Psychotherapie und Psychosomatik/Gerontopsychiatrie
Wenckebachstr. 23, 12099 Berlin, Germany
|1.||Dick B, Sitter H, Blau E, Lind N, Wege-Heuser E, Kopp I: Behandlungspfade in Psychiatrie und Psychotherapie. Nervenarzt 2006; 77: 12–22. CrossRef MEDLINE|
|2.||Burgers JS, Grol R, Klazinga NS, Mäkelä M, Zaat J: Towards evidence-based clinical practice: an international survey of 18 clinical guideline programs. Int J Qual Health Care 2003; 15: 31–45. CrossRef MEDLINE|
|3.||Gaebel W, Schwarz M, Janssen B: Qualitätsmanagement in der Psychiatrie. In: Gaebel W, Müller-Spahn F (eds.): Diagnostik und Therapie psychischer Störungen. Stuttgart: Kohlhammer 2002; 1257–73.|
|4.||Wobrock T, Schneider F, Falkai P: Leitlinienintentionen der DGPPN. Nervenarzt 2010; 81: 1041–8. CrossRef MEDLINE|
|5.||Weinmann S, Koesters M, Becker T: Effects of implementation of psychiatric guidelines on provider performance and patient outcome: systematic review. Acta Psychiatr Scand 2007; 115: 420–33. CrossRef MEDLINE|
|6.||Karbach U, Schubert I, Hagemeister J, Ernstmann N, Pfaff H, Höpp HW: Physicians' knowledge of and compliance with guidelines: An exploratory study in cardiovascular diseases. Dtsch Arztebl Int 2011; 108(5): 61–9. VOLLTEXT|
|7.||Grimshaw JM, Thomas RE, MacLennan G, et al.: Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004; 8: 1–84. MEDLINE|
|8.||Clade H: Fallbeispiel Schizophrenie: Hohe soziale Kosten. Dtsch Arztebl 2003; 100: 1598–9. VOLLTEXT|
|9.||Janssen B, Weinmann S, Berger M, et al.: Leitlinienkonformität und Behandlungsergebnisse in der stationären Schizophreniebehandlung – ein Klinikvergleich. Nervenarzt 2005; 76: 315–26. CrossRef MEDLINE|
|10.||Miller AL, Crismon ML, Rush AJ, et al.: The Texas medication algorithm project: clinical results for schizophrenia. Schizophr Bull 2004; 30: 627–47. CrossRef MEDLINE|
|11.||Weinmann S, Hoerger S, Erath M, Kilian R, Gaebel W, Becker T: Implementation of a schizophrenia practice guideline: clinical results. J Clin Psychiatry 2008; 29: 1299–306. CrossRef MEDLINE|
|12.||Godemann F, Blittersdorf K, Poschenrieder M, Klimitz H, Hauth I, Gutzmann H: Leilinienkonformität in der Behandlung schizophrener Patienten: Einführung eines IT-gestützten Behandlungspfades. Nervenarzt 2010; 81: 584–93. CrossRef MEDLINE|
|13.||Owen RR, Thrush CR, Kirchner JE, Fischer EP, Booth BM: Performance measurement for schizophrenia: adherence to guidelines for antipsychotic dose. Int J Qual Health Care 2000; 12: 475–82. CrossRef MEDLINE|
|14.||Lehman AF, Kreyenbuhl J, Buchanan RW, et al.: The schizophrenia patient outcomes research team (PORT): updated treatment recommendations 2003. Schizophr Bull 2004; 30: 193–217. CrossRef MEDLINE|
|15.||Lehman AF, Steinwachs DM: Translating research into practice: the schizophrenia patient outcomes research team (PORT) treatment recommendations. Schizophr Bull 1998; 24: 1–10. CrossRef MEDLINE|
|16.||Overall JE, Gorham DR: BPRS Brief Psychiatric Rating Scale. In: Guy W, Bonato RR (eds.): ECDEU Assessment Battery Rev. Ed. Rockville 1976; 157–69.|
|17.||Honigfeld G, Gillis RD, Klett CJ: NOSIE Nurses’ Observation Scale for Inpatient Evaluation. In: Guy W, Bonato RR (eds.): ECDEU Assessment Battery Rev. Ed. Rockville 1976; 265–73.|
|18.||Süllwold L, Herrlich J: Frankfurter Befindlichkeitsskala (FBS) für schizophren Erkrankte. Berlin: Springer 1987. CrossRef MEDLINE|
|19.||Baldessarini RJ, Tarazi FI: Drugs and the treatment of psychiatric disorders: antipsychotic and antimanic agents. In: Hardman JG, Limbird LE, Gilman AG (eds.): Goodman and Gilman’s the pharmacologic basis of therapeutics. New York: McGraw-Hill 2001; 485–520.|
|20.||Woods SW: Chlorpromazine equivalent doses for the newer atypical antipsychotics. J Clin Psychiatry 2002; 64: 663–7. CrossRef MEDLINE|
|21.||Chuang WC, Crismon ML: Evaluation of a schizophrenia medication algorithm in a state hospital. Am J Health Syst Pharm 2003; 60: 1459–67. MEDLINE|
|22.||Linden M, Schotte K: A randomized controlled clinical trial comparing “guideline exposed” and “guideline naive” physicians in respect to dosage selection and treatment outcome with Doxepin in depressive disorders. Pharmacopsychiatry 2007; 40: 77–81. CrossRef MEDLINE|