Discussion 5: Internet-Based Psychological Interventions

User Generated

N ZpTengu

Humanities

Description

With so many individuals using the Internet today, the psychology of the individual, the family, the community, and the society has changed. Some might argue that the traditional psychological theories cannot be applied and therefore psychology as an entire field needs to be revamped to incorporate the impact and the application of internet technology. As a student completing a bachelor’s degree in psychology, your future career will undoubtedly incorporate the use of the Internet in very new ways. Read the following article: Internet interventions- In review, in use, and into the future..pdf

Citation: Ritterband, L. M., Gonder-Frederick, L., Cox, D. J., Clifton, A. D., West, R. W., & Borowitz, S. M. (2003). Internet interventions: In review, in use, and into the future. Professional Psychology: Research and Practice, 34(5), 527-534. doi:http://dx.doi.org/10.1037/0735-7028.34.5.527


Identify a problem area or psychological issue that could benefit from the use of an internet intervention. Now, using what you learned from the article, propose an internet intervention using the following guidelines:

  1. What problem area would the internet intervention address?
  2. Provide a general description of the proposed internet intervention.
  3. How would the effectiveness of the proposed intervention be tested?
  4. What are any legal and ethical issues related to the proposed intervention? Include the specific APA ethical standard(s) in your response.


Unformatted Attachment Preview

Professional Psychology: Research and Practice 2003, Vol. 34, No. 5, 527–534 Copyright 2003 by the American Psychological Association, Inc. 0735-7028/03/$12.00 DOI: 10.1037/0735-7028.34.5.527 Internet Interventions: In Review, In Use, and Into the Future Lee M. Ritterband, Linda A. Gonder-Frederick, Daniel J. Cox, Allan D. Clifton, Rebecca W. West, and Stephen M. Borowitz This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. University of Virginia Health System The provision of health care over the Internet is a rapidly evolving and potentially beneficial means of delivering treatment otherwise unsought or unobtainable. Internet interventions are typically behavioral treatments operationalized and transformed for Web delivery with the goal of symptom improvement. The literature on the feasibility and utility of Internet interventions is limited, and there are even fewer outcome study findings. This article reviews empirically tested Internet interventions and provides an overview of the issues in developing and/or using them in clinical practice. Future directions and implications are also addressed. Although Internet interventions will not likely replace face-to-face care, there is little doubt that they will grow in importance as a powerful component of successful psychobehavioral treatment. information (see Crutsinger, 2000; Employment Policy Foundation, 2001; Pew Research Center, 2002; Rabasca, 2000). This new field of Internet interventions is only going to grow and expand. The benefits are vast, as there is the potential for greater numbers of people to receive more services than ever before. However, to meet this potential, Internet interventions, like any other treatment, must first demonstrate feasibility and efficacy through rigorous scientific testing. In this article we present a critical examination of the current state of the available literature focused on the development and testing of these types of interventions. In addition, issues that should be addressed when constructing Internet interventions are enumerated, followed by a discussion of possible future directions and implications for research and clinical practice. With the advancement of computer technologies over the past 20 years, a flood of new ways to communicate, provide, and deliver psychological treatments has emerged. The Internet has the ability to reach people all over the world and provide highly specialized psychological interventions otherwise not sought or obtainable. There is a new but growing literature on the use of the Internet as a means of delivering treatment. These Internet treatments are typically focused on behavioral issues, with the goal of instituting behavior change and subsequent symptom improvement. They are usually self-paced, interactive, and tailored to the user, and they make use of the multimedia format offered by the Internet. Individuals with computer and Internet access who use these treatments may also overcome many of the barriers to obtaining traditional care because they can seek such treatments at any time, any place, and often at significantly reduced cost. Clearly, people are using computers and the Internet in greater frequency and with the specific purpose of obtaining mental health Researchers are beginning to apply the Web as a way to have an impact on patient behaviors by reducing negative physical and LEE M. RITTERBAND received his PhD in clinical psychology and doctoral minor in computer science from the University of South Florida in 1998. He is an assistant professor at the Center for Behavioral Medicine Research at the University of Virginia Health System in Charlottesville, VA. His research interests focus on the development of Internet treatment interventions for various health psychology issues. LINDA A. GONDER-FREDERICK received her PhD in health psychology from the University of Virginia in 1985 and her PhD in clinical psychology from the University of Virginia in 1994. She is an associate professor in the Department of Psychiatric Medicine at the University of Virginia and clinical director of the Behavioral Medicine Center. Her research interests include behavioral and psychological aspects of Type 1 diabetes in adults and children, including the development of psychobehavioral interventions. DANIEL J. COX received his PhD in clinical psychology from the University of Louisville in 1977. He holds a diplomat in clinical psychology and is a professor in the Departments of Psychiatric Medicine and Internal Medicine at the University of Virginia. He is also the director of the Center for Behavioral Medicine Research. His primary areas of research involve behavioral medicine aspects of Type 1 diabetes, pediatric encopresis, and attention-deficit/hyperactivity disorder. ALLAN D. CLIFTON received his MA in clinical psychology in 1999 from the University of Virginia, where he is currently a doctoral candidate. He will be attending clinical internship at Western Psychiatric Institute and Clinic in Pittsburgh, PA. His research interests focus on interpersonal aspects of adaptive and maladaptive personality traits. REBECCA W. WEST received her JD from the University of Richmond School of Law in 1983. She is the executive director of the Piedmont Liability Trust as well as an assistant professor of general medicine at the University of Virginia School of Medicine, where she lectures on law and medicine. She has lectured and published often on legal issues in medicine. STEPHEN M. BOROWITZ received his MD from Rush Medical College in Chicago in 1980. He is a professor of pediatrics and health evaluation sciences as well as the assistant chief information officer at the University of Virginia Health System. His research interests focus on childhood constipation and encopresis and the use of information technology in the delivery of health care and health education. CORRESPONDENCE CONCERNING THIS ARTICLE should be addressed to Lee M. Ritterband, University of Virginia Health System, Department of Psychiatric Medicine, Center for Behavioral Medicine Research, P.O. Box 800223, Charlottesville, Virginia 22908. E-mail: leer@virginia.edu Internet Education and Interventions 527 528 RITTERBAND ET AL. psychological symptoms. Patient health-related information Web sites, Web-based treatment interventions (WBTIs), and hybrid treatment interventions (HTIs) have all been created for this purpose. People are using the information they find on the Internet to become better informed (Pew Research Center, 2002), although the link between such improved knowledge/awareness and actual behavioral change has not been demonstrated. Nonetheless, most of the available studies examining online interventions have shown that this treatment approach is, at the very least, feasible. This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Health-Related Information Web Sites for Patients Over 100,000 static health information Web sites have been constructed that deliver basic information on various problems and disorders (Kolata, 2000). Huge amounts of money have been invested into some of the more well-known and widely used patient information sites (e.g., WebMD.com, drkoop.com, Mayohealth.org, FDA.gov, and NIH.gov). A large number of people report that information from the Web has affected their health-care-related decisions (Pew Research Center, 2002). The provision of health-related information has the potential to change the health care market by making consumers more knowledgeable. Although major investments have been made to provide health information on the Internet, little research has been conducted as to the efficacy of health-related information on the Web. Since 1995, the Children’s Medical Center at the University of Virginia has maintained a series of Web-based patient education information sites about common pediatric problems directed at parents and children (Borowitz & Borowitz, 2000; Borowitz & Ritterband, 2001). Between January 1, 1998, and April 30, 2000, these tutorials received more than 650,000 successful page requests from more than 100,000 distinct hosts, and over 4,000 completed feedback forms were received. More than 90% of respondents found that the information helped them understand why children develop some of their health problems and felt that the information helped them care for a child suffering from a particular health problem. More than 85% of respondents rated this as a “very good” way to teach people about health problems. These results further support the fact that many people are searching the Internet for information about common healthrelated problems. In addition, information sites can provide parents and families with useful and accurate information and help address questions and concerns. However, although these patient education/health-related Web sites can be very helpful, the ultimate achievement is to deliver comprehensive, personalized, engaging, and empirically validated treatments that could be quickly and easily distributed over the Internet. Web-Based Treatment Interventions (WBTIs) Although WBTIs are a few years away from being a truly comprehensive treatment option, mainly due to issues of bandwidth, a thorough literature search revealed that some researchers are beginning to test the feasibility and effectiveness of delivering this form of treatment intervention over the Internet. Identified studies for inclusion in this review are empirically tested, randomly assigned Internet interventions in which a treatment had been developed and operationalized specifically for Web delivery. Tested Internet interventions (see Table 1) include those for smoking cessation (Schneider, Walter, & O’Donnell, 1990), weight loss (Tate, Wing, & Winett, 2001; Winett et al., 1999), headaches (Ström, Pettersson, & Andersson, 2000), body image (Celio et al., 2000; Winzelberg et al., 2000), posttraumatic stress and pathological grief (Lange, van de Ven, Schrieken, & Emmelkamp, 2001), physical activity (McKay, King, Eakin, Seeley, & Glasgow, 2001), panic disorder (Klein & Richards, 2001), tinnitus (Andersson, Strömgren, Ström, & Lyttkens, 2002), diabetes management (McKay, Glasgow, Feil, Boles, & Barrera, 2002), and pediatric encopresis (Ritterband et al., 2003). These studies all focus on behavioral medicine/health psychology issues, which seem to be more adaptable to Internet interventions (Childress & Asamen, 1998) because of the availability of highly structured treatment approaches to many problems. Generally, these studies provide support for the notion that Internet interventions can be feasible and effective. These studies also demonstrate that some behaviorally related psychological treatments can be operationalized, transformed, and transported to the user via the Internet. To test efficacy, most of these studies used some form of a nontreatment control group rather than a face-to-face treatment group for the identified disorder. This decision makes sense given that the first step in verifying the applicability of this form of intervention is to make sure that it works rather than to hold it up to the “gold standard” of face-to-face treatments. Subsequent generations of clinical trials will need to incorporate face-to-face treatment interventions as one of the experimental conditions. However, it should not be necessary for Internet interventions to prove more effective than face-to-face treatments but rather to provide close to equivalent benefits and outcome results. As the many advantages of Internet interventions, especially accessibility, become increasingly clear, people who might not otherwise have obtained treatment may do so. These and future studies in this area are not meant to imply that face-to-face treatment should be replaced; rather, they are meant to provide an alternative or adjunctive component to already well-established and highly effective interventions. Web-based treatment interventions offer an opportunity for psychologists to provide specific behavioral treatments, tailored to individuals who prefer or need to seek help from their own homes. The technology is now available and will be more readily accessible with high-speed bandwidth, which should be widely accessible in the near future. Although the current sparse literature examining this area may have limitations, these studies represent the pioneering efforts to develop what will likely become a major force in the delivery of psychological treatments. However, many of these interventions do not take advantage of the full capabilities of the Internet. Currently, Internet interventions tend to be limited in their graphical elements and other potentially engaging factors (i.e., audio, animation, interactivity) and can certainly be improved with new development software for Web applications. There is a significant need for more diverse and comprehensive interventions, but the time and effort to operationalize an intervention is considerable. Although these new interventions are developed, current applications must overcome the difficulties of delivering their often large-file-sized programs over phone lines. A compromise that has been created is a hybrid approach. INTERNET INTERVENTIONS 529 Table 1 Empirically Tested Internet Interventions Target behaviors/ symptoms Subjects (N) Schneider, Walter, & O’Donnell, 1990 Smoking cessation 1,158 adults Internet intervention with smoking diaries vs. outline of behavioral strategies. Trends of greater cessation at 1, 3, and 6 months for Internet group. Winett et al., 1999 Improve teenage girls’ health behaviors 180 high school females Semester of Internet intervention focusing on weight loss, healthy eating, and health behaviors vs. standard high school health class with similar content. Internet group had significant improvements in many health-related behaviors. Celio et al., 2000 Body image satisfaction and eating attitudes 76 women Psychoeducational Internet program vs. same content delivered in a class vs. wait-list control. Internet group had fewer weight and shape concerns and better eating attitudes at posttreatment. Risk factors for eating disorders reduced at 4-month follow-up. Ström, Pettersson, & Andersson, 2000 Headaches 45 adults Internet intervention focused on applied relaxation and problem-solving techniques vs. wait-list control. After 6-week program, Internet intervention group experienced significantly fewer and less severe headaches. Winzelberg et al., 2000 Body image satisfaction 60 women Psychoeducational Internet program vs. control condition. After 8-week program and at 3-month follow-up, Internet group had greater satisfaction with their body and less desire for “thinness.” Klein & Richards, 2001 Panic 22 adults Internet intervention containing psychoeducational components and techniques to reduce panic vs. selfmonitoring control group. Internet subjects had reduced panic-related symptoms. Lange, van de Ven, Schrieken, & Emmelkamp, 2001 Posttraumatic stress and pathological grief 25 college students Internet intervention utilizing 5 weeks of writing assignments vs. wait-list control. 80% of experimental group demonstrated clinically significant improvement in symptoms of trauma and general psychopathology posttreatment. McKay, King, Eakin, Seeley, & Glasgow, 2001 Physical activity 78 patients with Type 2 diabetes Internet physical activity intervention vs. information-only control. Both groups increased activity levels, but no significant differences between groups. Those in Internet group who more often used program did significantly better. Tate, Wing, & Winett, 2001 Weight loss 91 adults Behavior weight loss Internet program with e-mail consultation vs. education about weight loss and access to information-based Web sites. Internet behavior therapy led to significantly more weight loss at 3 and 6 months. Andersson, Strömgren, Ström, & Lyttkens, 2002 Tinnitus 117 adults Internet cognitive–behavioral intervention for tinnitus vs. wait-list control. After 6-week program, Internet group experienced significantly greater improvements in several measures of anxiety, depression, and other tinnitusrelated distress vs. controls. McKay, Glasgow, Feil, Boles, & Barrera, 2002 Diabetes selfmanagement 133 patients with Type 2 diabetes Internet-based diabetes support program on diet, mental health, and physiologic outcomes vs. information-only control. Both groups improved, but no significant differences between groups. Ritterband et al., 2003 Pediatric encopresis 24 children Psychoeducational and interactive behavioral Internet intervention vs. no intervention. Significant improvements for children in Internet condition on no. of accidents, no. of bowel movements in the toilet/week, and increased trips to the bathroom without a parental prompt. This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Study a All studies used random assignment as part of their design. Designa Results RITTERBAND ET AL. 530 This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Hybrid Treatment Interventions (HTIs) HTIs address the problem of limited bandwidth by providing highly engaging interventions without losing the critical Internet connection. All the large files are stored either on a CD-ROM or a hard disk drive. The computer still connects to the Internet to transfer small bits of information back and forth from the utilized server. In this way, all the benefits of the Web can still be maintained, including the ability to update information, exchange information, collect data, monitor user activity in real time, provide feedback, and prompt behavior change, without losing the ability to have large data files, such as extensive graphic and audio files. The disadvantage is that the large files must be furnished to the user in order to receive the intervention. This typically means providing a CD-ROM to the user through regular mail or in person. An example of an HTI is the “U-CAN-POOP-TOO” Web site developed by our research group at the University of Virginia and supported by the National Institutes of Health. The goal in designing this program was to deliver a child-focused, entertaining, and engaging site that would provide all the necessary behavioral and medical components of Enhanced Toilet Training (Cox, Borowitz, Kovatchev, & Ling, 1998) to successfully treat pediatric encopresis. This HTI encompasses hundreds of pages of content, with numerous illustrations, interactive components, animated tutorials, and reinforcing quizzes. In order to validate this intervention, we placed a computer in the homes of half the subjects and provided Internet access to the Web site. The large graphic and animation files were put on the hard drive in order to reduce long wait times when progressing through the program. All subjects continued to receive care from their primary care physician. Children who received the Web intervention in addition to care by their primary care physician decreased their accidents by 93%, increased their number of bowel movements (BMs) in the toilet each week by 152%, and increased their trips to the bathroom without a parental prompt by 109%. The control children decreased their accidents by 31%, but they also decreased their number of BMs in the toilet each week by 16% and their trips to the bathroom without a parental prompt by 37% (Ritterband et al., 2003). These results are equivalent to those found in tests of a face-to-face behavioral intervention for pediatric encopresis (Cox et al., 1998). This hybrid example allowed for the assessment of (a) the feasibility of this type of treatment intervention, (b) the usability/ acceptability of such a system to patients, and (c) the effectiveness of such a system for delivering the necessary information, promoting behavior change, and ameliorating symptoms. Further studies of this kind are necessary to establish the validity of this type of intervention. Eventually, as more high-speed connections become available, the HTI model can be discarded, and the WBTI model will be the standard. Developing and Using Internet Interventions Developing Internet interventions is an arduous, sometimes tedious, and always time-intensive process. It necessitates an interdisciplinary approach, requiring a team of diverse professionals, including clinicians and other health care providers to provide content; computer and Web programmers to build essential applications; Web designers to create the Web structure; Web graphic artists to create still and animated images; database developers to integrate a mechanism to store and retrieve data; health informatics evaluators to evaluate user interface issues and outcomes; and behaviorists to incorporate behavior change concepts into the system. Other potential members of the team may include business/financial advisors to ensure proper marketing, management, and sales; videographers to create video; audio engineers to integrate Web audio; psychometricians to certify appropriate scale integration; tech support personnel to provide user support; costanalysis specialists to determine savings; linguists/translators to provide readability testing and translation; disability experts to oversee usability issues; and health educators to make certain the content is structured in such a way that the majority of users will find it helpful. Obviously, only a small number of psychologists would be likely to want or to have the resources to be able to create an Internet intervention. However, many clinicians may want to utilize this form of treatment within their own clinical practice. The most plausible reason for this is to supplement skills that could be addressed with an adjunctive Internet intervention. For example, a clinician who feels comfortable treating depression and anxiety may not have the training to implement the behavioral components for treating insomnia, which commonly co-occurs with these disorders. An Internet intervention for insomnia with demonstrated effectiveness could very easily be used as a component of faceto-face treatment, allowing the clinician to target this specific issue. A brief summary of the steps involved in developing Internet interventions may be helpful in conveying the underlying process of this type of approach. It may also instill an appreciation for the efforts that go into creating these programs. Also, a greater understanding of these systems will help clinicians recognize how to best integrate them into their own practice. A number of factors must be considered when creating an Internet intervention (see Figure 1). First, the disorder must be identified and the treatment should be translatable (Step 1). This means that the intervention is structured such that it can be delivered using the Internet. Typically, the treatment is highly structured and can be at least semi-self-guided. It is also important to determine the effectiveness of the intervention to be transformed (Step 2). An effective face-to-face intervention is the gold standard by which an Internet intervention will ultimately be compared. Once the treatment is identified and determined to be effective, it must be operationalized completely (Step 3). To operationalize a treatment, one must identify all critical aspects of the intervention, including specific treatment techniques and procedures. There are numerous legal and ethical issues that must be considered when developing an Internet intervention (Step 4). These include issues of privacy, confidentiality, data validity, potential misuse of Internet interventions by professionals, equality of Internet access, and credentialing issues (see Humphreys, Winzelberg, & Klaw, 2000; Jerome et al., 2000; Koocher & Morray, 2000; Sampson, Kolodinsky, & Greeno, 1997; Winker et al., 2000; Winzelberg et al., 2000). Examining these issues in depth is beyond the scope of this article, but each issue should be carefully considered before creating and/or utilizing an Internet intervention in a clinical practice. Many of these same ethical and legal issues need to be considered when incorporating an Internet intervention as an adjunctive component of treatment. This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. INTERNET INTERVENTIONS Figure 1. 531 Development of Internet interventions. The multimedia aspect of the Internet should be used in creating the intervention (Step 5). Without using Internet components such as audio, graphics, animation, and video, the intervention is little more than a self-help book that can be read online. These elements help make the intervention engaging and should increase motivation to use and complete the treatment program. Interactivity is another key component that will likely enrich the intervention and keep users connected. Personalization allows the user to receive more individually tailored content, helping to focus the intervention specifically to the user (Step 6). Personalization may be as simple as using the name of the user when presenting certain content or as sophisticated as the system identifying specific treatment areas the individual user may need to address. It is critical that users receive some feedback regarding their progress within the treatment (Step 7). These feedback loops may allow users to track specific elements of the treatment and may even provide information to be used with their clinician if the system is an adjunctive component of treatment. Several technical issues must also be addressed when constructing the Internet program (Step 8). It needs to be determined whether a fully Web-based or hybrid treatment intervention would be most appropriate. All issues of Internet delivery, including type of browser or self-contained application, type of database, and programming languages to use, must be resolved. Also, issues of cross-platform compatibility (e.g., making sure the program will work on a PC and a Macintosh), as well as hardware issues (providing minimum requirements, such as amount of machine memory and hard drive space needed) and software issues (identifying necessary browser, plug-ins, and software incompatibility issues), should be considered. Most of these issues are usually decided on the basis of the needs of the target audience. Finally, once the program is developed, testing of the application must be conducted (Step 9). There are several steps to testing the Internet intervention. Early tests may include focus groups, in which individuals are invited to view the program and provide feedback regarding its use. The intervention will also need to be pilot tested with a small group of patients in order to determine issues of feasibility, usability, and possibly early determinations of efficacy. Finally, a large clinical trial will need to be conducted in order to demonstrate effectiveness. Revisions will likely need to be incorporated into the program on the basis of users’ feedback, so a loop between Step 9 and Step 8 will occur. Future Directions and Implications Computers and the Internet have become important tools used in the field of psychology (for other reviews, see Barak, 1999; Laszlo, Esterman, & Zabko, 1999; Smith & Senior, 2001; Stamm, 1998). However, many clinical and research issues need to be addressed to further solidify as well as broaden their place in the discipline. Clinical Directions It is likely that there are many other Internet interventions currently in various stages of development and evaluation. Some are computer treatments that are in the process of being transformed to Internet interventions, including computer-based health information and support systems for patients with life-threatening illnesses (such as for patients who are HIV positive and for those with breast cancer; Gustafson et al., 1999, 2001), phobias (Kenwright, Liness, & Marks, 2001), fears of public speaking (Botella et al., 2000), and marital therapy (Jerome et al., 2000). Additional treatments need to be developed and examined for their efficacy in order to establish Internet interventions as a viable treatment alternative and to support generalizability. Some issues, such as This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 532 RITTERBAND ET AL. sexual dysfunctions, may be particularly suited to an Internet intervention, because many people may be uncomfortable seeking face-to-face help for such concerns. Internet interventions for other issues, such as insomnia, may provide help to those who do not believe their problem warrants a doctor’s visit but who would follow a treatment plan on their own. Internet interventions can help reduce many of the traditional barriers inherent in the current mental health care deliverance model, including unavailability of skilled professionals, long lag time for dissemination of information, length of treatment, costs and inconvenience of treatment, and unwillingness to seek treatment. With the use of the Internet, professionals or patients can gather information whenever they wish, and the treatments can be presented in great detail through the use of the written word as well as through such visuals as pictures, movies, and animated graphics to enhance the understanding of a disorder. Patients are likely to feel empowered by being able to digest the information at their own pace and to better use it to enhance treatment efficacy. Also, the cost of obtaining access to this information, already minimal, continues to decline every day. Finally, this mode of treatment deliverance may be much more appealing to some patients, increasing their willingness to participate and follow the recommendations. Even though Internet interventions may help overcome many barriers to mental health care treatment, some critical issues still need to be addressed and resolved, including problems of selfassessment and diagnosis, dissemination of information and treatments, establishment of a financial model, and compliance. There are potential significant negative consequences to individuals assessing and diagnosing themselves, including misdiagnosis and improper treatment selection. Internet interventions are not meant to replace face-to-face treatment but rather to provide an alternative for individuals who might otherwise choose not to receive treatment (e.g., because of embarrassment) or who might be unable to obtain treatment (e.g., because of location) or to find appropriate treatment (e.g., because no provider is available). However, it is difficult to disseminate information about these interventions and make individuals aware of them. New ways of broadcasting this type of treatment are needed so that people may know and take advantage of them. A financial model must be established so that individuals can purchase treatments, insurance companies pay for treatments, and providers charge for development and usage of treatments. Without some financial framework, these interventions will not survive regardless of how effective they are found to be. However, before individuals should be charged for these services, research must be conducted to prove their effectiveness. Compliance is a problem for traditional approaches and for Internet interventions. Although people reported enjoying the “UCAN-POOP-TOO” program, it was still difficult to maintain compliance. Subjects were instructed to access the program at least once a week; however, some subjects needed numerous phone calls to remind and encourage them to return to the site and complete the program. In the body image studies conducted at Stanford University (Winzelberg et al., 2000), subjects typically completed less than two thirds of the program, and compliance progressively declined each week of the intervention. Suggestions have been made for improving compliance, including stressing the importance of following through with treatment recommendations at the beginning of the program, utilizing various “motivational components” built into the program, and providing reinforcement for cooperation and program completion. Research Directions It is vital that methodologically sound clinical studies be conducted to ensure the efficacy of Internet interventions. A few early studies show promising outcomes; however, although it appears that the feasibility of this form of treatment approach is possible, the development of additional Internet treatments and adequate clinical trials are necessary to establish Internet interventions as a viable treatment option. Many of the areas identified in the Clinical Directions section of this article need to be examined by means of scientific measures. Issues of assessment and diagnosis, cost-effectiveness, and compliance all need to be explored. Compliance, in particular, is an area in which Internet interventions may make a significant difference. Patient compliance is said to increase when patients’ satisfaction with the communication of the health-related information increases (Ley, 1982). Research is necessary to determine whether computer and Internet components, such as audio, video, graphics/animation, and interaction, will improve compliance. It may be that through the use of Internet interventions, treatment programs can be made to be more engaging, thus increasing an individual’s motivation to continue working. Internet interventions and other computer treatments can include personalized and pertinent information, quizzes, case reports, and games to make the programs more enticing, but the impact of all of these needs to be assessed. Internet treatment programs can also utilize the power of e-mail (e.g., individuals could be sent an e-mail with a hyperlinked Web address to access the Web program more easily). This may also serve as a behavioral prompt that might improve compliance. However, it is also possible that compliance may be even more problematic with computer/Internet treatment programs because of limited supervision, but no research as of yet has focused on this important question. There is also a need for more basic research examining components of the Internet and, specifically, the Web and its ability to elicit behavioral changes. Some researchers have questioned whether changing various elements within a computer program, such as adding color, sound, and video, would improve or hinder outcome (Sproull, Subramani, Kiesler, & Walker, 1996). Similar inquiries need to be addressed with respect to the Web. These basic research questions will help shape a model or framework for developing WBTIs. Although it might be tempting to use business models already developed for creating engaging and “sticky” Web sites (i.e., those Web sites that keep users at the site and draw them back for future visits), it is important to note that these models do not address the most critical component of Internet interventions: behavior change. A theoretical model for this type of approach is necessary so that additional Internet interventions can be developed. Future Possibilities In addition to the developments already presented, the future promises even greater technologies and applications for psychology. Web-based treatment interventions, as well as enhanced This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. INTERNET INTERVENTIONS videoconferencing and other bandwidth-intensive approaches, will likely flourish as increased and improved Internet access becomes more prevalent. Wireless technology is also rapidly improving, and consumer products with wireless capabilities are becoming more available. Some users already have cellular phones and hand-held computers with wireless Internet access. These products will become even more widely available and easier to use over the coming years. These wireless products will allow individuals to track and share information with their health care professions in real time, ease data input, reduce data-entry errors, and, we hope, improve compliance. In addition to wireless technologies, older technologies such as the telephone will be better able to interface with some of the newer technologies, including the Internet (see Ritterband et al., 2001). For example, Internet applications can be created to call patients on a regular basis to track progress and symptom relief and prompt for recommended behaviors. This information can be stored on a Web server database that can be accessed by the individual and the health care professional. Patient care and patients’ perceptions of care will likely improve with this type of technology. Gaming consoles and future personal digital assistants already have Internet capabilities, and treatment-based software could be created for use on these platforms. Children may be more interested in receiving treatments if they are presented in an engaging and entertaining way. Hand-held computers can be carried anywhere, which could improve the collection of symptom information and provide reminders of treatment components to improve compliance. This information could be wirelessly transmitted to the user’s health care professional, who could help expedite treatment goals. Implications It is unlikely that Internet interventions will replace face-to-face psychotherapy; however, this technology may be helpful in the treatment of some psychological problems that might otherwise go untreated. It is also possible that such interventions may enhance traditional therapy as an adjunctive component. These new possibilities usher in a whole new platform from which mental health, and health care in general, can be conducted. Clinicians will have to be trained to understand, create, and use these forms of treatment. In time, advances in technology will escalate and push health care to use its power to improve care to the benefit of all. The following are some of the implications of the issues and the research presented in this article: 1. Additional clinical treatment interventions need to be operationalized and transformed into Internet interventions for greater patient consumption. 2. These interventions must be empirically validated through well-designed clinical research studies. 3. Studies of all components of Internet interventions are necessary to establish and improve feasibility, usability, and efficacy. 4. Psychologists need to accept that technology is changing the world. This means that multidisciplinary teams will likely include a rather unusual set of people to provide new treatment interventions. 5. New definitions of treatment must be created that embrace 533 the use of new technologies and encompass the use of the Internet in their deliverance. 6. Costs to create and test Internet interventions are significant and should not be underestimated when considering the development of a new program. 7. Psychologists need to understand and accept that there are numerous ethical and legal issues relevant to providing new ways to deliver health care and that it takes time before these issues can be appropriately managed. References Andersson, G., Strömgren, T., Ström, L., & Lyttkens, L. (2002). Randomized controlled trial of Internet-based cognitive behavior therapy for distress associated with tinnitus. Psychosomatic Medicine, 64, 810 – 816. Barak, A. (1999). Psychological applications on the Internet: A discipline on the threshold of a new millennium. Applied and Preventive Psychology, 8, 231–246. Borowitz, K. C., & Borowitz, S. M. (2000, May). Using the World Wide Web to teach people about cerebral palsy. Paper presented at the annual meeting of the Society for Pediatric Research, Boston, MA. Borowitz, S. M., & Ritterband, L. M. (2001). Using the Internet to teach parents and children about constipation and encopresis. Medical Informatics and the Internet in Medicine, 26, 283–295. Botella, C., Banos, R., Guillen, V., Perpina, C., Alcaniz, M., & Pons, A. (2000). Telepsychology: Public speaking fear treatment on the Internet. Cyberpsychology & Behavior, 3, 959 –968. Celio, A., Winzelberg, A. J., Wilfley, D., Eppstein-Herald, D., Springer, E., Dev, P., & Taylor, C. B. (2000). Reducing risk factors for eating disorders: Comparison of an Internet- and a classroom-delivered psychoeducational program. Journal of Consulting and Clinical Psychology, 68, 650 – 657. Childress, C. A., & Asamen, J. K. (1998). The emerging relationship of psychology and the Internet: Proposed guidelines for conducting Internet intervention research. Ethics & Behavior, 8, 19 –35. Cox, D. J., Borowitz, S. M., Kovatchev, B., & Ling, W. (1998). Contributions of behavior therapy and biofeedback to laxative therapy in the treatment of pediatric encopresis. Annals of Behavioral Medicine, 20, 70 –76. Crutsinger, M. (2000, October 17). Computers in half of U.S. homes. The Washington Post, p. E17. Employment Policy Foundation. (2001, July 11). Computer ownership and Internet access: Opportunities for workforce development and job flexibility. Technology Forecast. Retrieved from http://www.epf.org/ research/newsletters/2001/tf20010111.pdf Gustafson, D. H., Hawkins, R., Boberg, E., Pingree, S., Serlin, R., Graziano, F., & Chan, C. L. (1999). Impact of a patient-centered, computerbased health information/support system. American Journal of Preventive Medicine, 16, 1–9. Gustafson, D. H., Hawkins, R., Pingree, S., McTavish, F., Arora, N. K., Mendenhall, J., et al. (2001). Effect of computer support on younger women with breast cancer. Journal of General Internal Medicine, 16, 435– 445. Humphreys, K., Winzelberg, A. J., & Klaw, E. (2000). Psychologists’ ethical responsibilities in Internet-based groups: Issues, strategies, and a call for dialogue. Professional Psychology: Research and Practice, 31, 493– 496. Jerome, L. W., DeLeon, P. H., James, L. C., Folen, R., Earles, J., & Gedney, J. J. (2000). The coming age of telecommunications in psychological research and practice. American Psychologist, 55, 407– 421. Kenwright, M., Liness, S., & Marks, I. (2001). Reducing demands on clinicians by offering computer-aided self-help for phobia/panic. British Journal of Psychiatry, 179, 456 – 459. This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 534 RITTERBAND ET AL. Klein, B., & Richards, J. C. (2001). A brief Internet-based treatment for panic disorder. Behavioural and Cognitive Psychotherapy, 29, 113–117. Kolata, G. (2000, March 6). Web research transforms visit to the doctor. New York Times. Retrieved from http://archives.nytimes.com Koocher, G. P., & Morray, E. (2000). Regulation of telepsychology: A survey of state attorneys general. Professional Psychology: Research and Practice, 31, 503–508. Lange, A., Ven, J.-P. van de, Schrieken, B., & Emmelkamp, P. (2001). Interapy. Treatment of posttraumatic stress through the Internet: A controlled trial. Journal of Behavior Therapy and Experimental Psychiatry, 32, 73–90. Laszlo, J. V., Esterman, G., & Zabko, S. (1999). Therapy over the Internet? Theory, research, and finances. CyberPsychology & Behavior, 2, 293– 307. Ley, P. (1982). Satisfaction, compliance and communication. British Journal of Clinical Psychology, 21, 241–254. McKay, G. H., Glasgow, R. E., Feil, E. G., Boles, S. M., & Barrera, M. (2002). Internet-based diabetes self-management and support: Initial outcomes from the Diabetes Network Project. Rehabilitation Psychology, 47, 31– 48. McKay, G. H., King, D., Eakin, E. G., Seeley, J. R., & Glasgow, R. E. (2001). The Diabetes Network Internet-based physical activity intervention. Diabetes Care, 24, 1328 –1334. Pew Research Center. (2002, May 22). Vital decisions: How Internet users decide what information to trust when they or their loved ones are sick. Pew Internet & American Life. Retrieved May 30, 2002, from http:// www.pewinternet.org/reports/toc.asp?Report⫽59 Rabasca, L. (2000, April). Self-help sites: A blessing or a bane? Monitor on Psychology, 31(4), 28 –30. Ritterband, L. M., Cox, D. C., Kovatchev, B. P., Borowitz, S. M., Ling, W. D., & Marshall, B. (2001). A telecommunication monitoring system for clinical and research practice. Professional Psychology: Research and Practice, 32, 636 – 641. Ritterband, L. M., Cox, D. C., Walker, L., Kovatchev, B., McKnight, L., Patel, K., et al. (2003). A Web-based treatment intervention as adjunctive therapy for pediatric encopresis. Journal of Consulting and Clinical Psychology, 71, 910 –917. Sampson, J. P., Kolodinsky, R. W., & Greeno, B. P. (1997). Counseling and the information highway: Future possibilities and potential problems. Journal of Counseling & Development, 75, 203–212. Schneider, S. J., Walter, R., & O’Donnell, R. (1990). Computerized communication as a medium for behavioral smoking cessation treatment: Controlled evaluation. Computers in Human Behavior, 6, 141–151. Smith, M. A., & Senior, C. (2001). The Internet and clinical psychology: A general review of the implications. Clinical Psychology Review, 21, 129 –136. Sproull, L., Subramani, M., Kiesler, S., & Walker, J. H. (1996). When the interface is a face. Human-Computer Interaction, 11, 97–124. Stamm, B. H. (1998). Clinical applications of telehealth in mental health care. Professional Psychology: Research and Practice, 29, 536 –542. Ström, L., Pettersson, R., & Andersson, G. (2000). A controlled trial of self-help treatment of recurrent headache conducted via the Internet. Journal of Consulting and Clinical Psychology, 68, 722–727. Tate, D. F., Wing, R. R., & Winett, R. A. (2001). Using Internet technology to deliver a behavioral weight loss program. Journal of the American Medical Association, 285, 1172–1177. Winett, R. A., Roodman, A. A., Winett, S. G., Bajzek, W., Rovniak, L. S., & Whiteley, J. A. (1999). The effects of the Eat4Life Internet-based health behavior program on the nutrition and activity of high school girls. Journal of Gender, Culture, and Health, 4, 239 –254. Winker, M. A., Flanagin, A., Chi-Lum, B., White, J., Andrews, K., & Kennett, R. L. (2000). Guidelines for medical and health information sites on the Internet: Principles governing AMA Web sites. Journal of the American Medical Association, 283, 1600 –1606. Winzelberg, A. J., Eppstein, D., Eldredge, K. L., Wilfley, D., Dasmahapatra, R., Dev, P., & Taylor, C. B. (2000). Effectiveness of an Internet-based program for reducing risk factors for eating disorders. Journal of Consulting and Clinical Psychology, 68, 346 –350. Received June 11, 2002 Revision received May 15, 2003 Accepted May 22, 2003 䡲 Professional Psychology: Research and Practice 2003, Vol. 34, No. 5, 527–534 Copyright 2003 by the American Psychological Association, Inc. 0735-7028/03/$12.00 DOI: 10.1037/0735-7028.34.5.527 Internet Interventions: In Review, In Use, and Into the Future Lee M. Ritterband, Linda A. Gonder-Frederick, Daniel J. Cox, Allan D. Clifton, Rebecca W. West, and Stephen M. Borowitz This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. University of Virginia Health System The provision of health care over the Internet is a rapidly evolving and potentially beneficial means of delivering treatment otherwise unsought or unobtainable. Internet interventions are typically behavioral treatments operationalized and transformed for Web delivery with the goal of symptom improvement. The literature on the feasibility and utility of Internet interventions is limited, and there are even fewer outcome study findings. This article reviews empirically tested Internet interventions and provides an overview of the issues in developing and/or using them in clinical practice. Future directions and implications are also addressed. Although Internet interventions will not likely replace face-to-face care, there is little doubt that they will grow in importance as a powerful component of successful psychobehavioral treatment. information (see Crutsinger, 2000; Employment Policy Foundation, 2001; Pew Research Center, 2002; Rabasca, 2000). This new field of Internet interventions is only going to grow and expand. The benefits are vast, as there is the potential for greater numbers of people to receive more services than ever before. However, to meet this potential, Internet interventions, like any other treatment, must first demonstrate feasibility and efficacy through rigorous scientific testing. In this article we present a critical examination of the current state of the available literature focused on the development and testing of these types of interventions. In addition, issues that should be addressed when constructing Internet interventions are enumerated, followed by a discussion of possible future directions and implications for research and clinical practice. With the advancement of computer technologies over the past 20 years, a flood of new ways to communicate, provide, and deliver psychological treatments has emerged. The Internet has the ability to reach people all over the world and provide highly specialized psychological interventions otherwise not sought or obtainable. There is a new but growing literature on the use of the Internet as a means of delivering treatment. These Internet treatments are typically focused on behavioral issues, with the goal of instituting behavior change and subsequent symptom improvement. They are usually self-paced, interactive, and tailored to the user, and they make use of the multimedia format offered by the Internet. Individuals with computer and Internet access who use these treatments may also overcome many of the barriers to obtaining traditional care because they can seek such treatments at any time, any place, and often at significantly reduced cost. Clearly, people are using computers and the Internet in greater frequency and with the specific purpose of obtaining mental health Researchers are beginning to apply the Web as a way to have an impact on patient behaviors by reducing negative physical and LEE M. RITTERBAND received his PhD in clinical psychology and doctoral minor in computer science from the University of South Florida in 1998. He is an assistant professor at the Center for Behavioral Medicine Research at the University of Virginia Health System in Charlottesville, VA. His research interests focus on the development of Internet treatment interventions for various health psychology issues. LINDA A. GONDER-FREDERICK received her PhD in health psychology from the University of Virginia in 1985 and her PhD in clinical psychology from the University of Virginia in 1994. She is an associate professor in the Department of Psychiatric Medicine at the University of Virginia and clinical director of the Behavioral Medicine Center. Her research interests include behavioral and psychological aspects of Type 1 diabetes in adults and children, including the development of psychobehavioral interventions. DANIEL J. COX received his PhD in clinical psychology from the University of Louisville in 1977. He holds a diplomat in clinical psychology and is a professor in the Departments of Psychiatric Medicine and Internal Medicine at the University of Virginia. He is also the director of the Center for Behavioral Medicine Research. His primary areas of research involve behavioral medicine aspects of Type 1 diabetes, pediatric encopresis, and attention-deficit/hyperactivity disorder. ALLAN D. CLIFTON received his MA in clinical psychology in 1999 from the University of Virginia, where he is currently a doctoral candidate. He will be attending clinical internship at Western Psychiatric Institute and Clinic in Pittsburgh, PA. His research interests focus on interpersonal aspects of adaptive and maladaptive personality traits. REBECCA W. WEST received her JD from the University of Richmond School of Law in 1983. She is the executive director of the Piedmont Liability Trust as well as an assistant professor of general medicine at the University of Virginia School of Medicine, where she lectures on law and medicine. She has lectured and published often on legal issues in medicine. STEPHEN M. BOROWITZ received his MD from Rush Medical College in Chicago in 1980. He is a professor of pediatrics and health evaluation sciences as well as the assistant chief information officer at the University of Virginia Health System. His research interests focus on childhood constipation and encopresis and the use of information technology in the delivery of health care and health education. CORRESPONDENCE CONCERNING THIS ARTICLE should be addressed to Lee M. Ritterband, University of Virginia Health System, Department of Psychiatric Medicine, Center for Behavioral Medicine Research, P.O. Box 800223, Charlottesville, Virginia 22908. E-mail: leer@virginia.edu Internet Education and Interventions 527 528 RITTERBAND ET AL. psychological symptoms. Patient health-related information Web sites, Web-based treatment interventions (WBTIs), and hybrid treatment interventions (HTIs) have all been created for this purpose. People are using the information they find on the Internet to become better informed (Pew Research Center, 2002), although the link between such improved knowledge/awareness and actual behavioral change has not been demonstrated. Nonetheless, most of the available studies examining online interventions have shown that this treatment approach is, at the very least, feasible. This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Health-Related Information Web Sites for Patients Over 100,000 static health information Web sites have been constructed that deliver basic information on various problems and disorders (Kolata, 2000). Huge amounts of money have been invested into some of the more well-known and widely used patient information sites (e.g., WebMD.com, drkoop.com, Mayohealth.org, FDA.gov, and NIH.gov). A large number of people report that information from the Web has affected their health-care-related decisions (Pew Research Center, 2002). The provision of health-related information has the potential to change the health care market by making consumers more knowledgeable. Although major investments have been made to provide health information on the Internet, little research has been conducted as to the efficacy of health-related information on the Web. Since 1995, the Children’s Medical Center at the University of Virginia has maintained a series of Web-based patient education information sites about common pediatric problems directed at parents and children (Borowitz & Borowitz, 2000; Borowitz & Ritterband, 2001). Between January 1, 1998, and April 30, 2000, these tutorials received more than 650,000 successful page requests from more than 100,000 distinct hosts, and over 4,000 completed feedback forms were received. More than 90% of respondents found that the information helped them understand why children develop some of their health problems and felt that the information helped them care for a child suffering from a particular health problem. More than 85% of respondents rated this as a “very good” way to teach people about health problems. These results further support the fact that many people are searching the Internet for information about common healthrelated problems. In addition, information sites can provide parents and families with useful and accurate information and help address questions and concerns. However, although these patient education/health-related Web sites can be very helpful, the ultimate achievement is to deliver comprehensive, personalized, engaging, and empirically validated treatments that could be quickly and easily distributed over the Internet. Web-Based Treatment Interventions (WBTIs) Although WBTIs are a few years away from being a truly comprehensive treatment option, mainly due to issues of bandwidth, a thorough literature search revealed that some researchers are beginning to test the feasibility and effectiveness of delivering this form of treatment intervention over the Internet. Identified studies for inclusion in this review are empirically tested, randomly assigned Internet interventions in which a treatment had been developed and operationalized specifically for Web delivery. Tested Internet interventions (see Table 1) include those for smoking cessation (Schneider, Walter, & O’Donnell, 1990), weight loss (Tate, Wing, & Winett, 2001; Winett et al., 1999), headaches (Ström, Pettersson, & Andersson, 2000), body image (Celio et al., 2000; Winzelberg et al., 2000), posttraumatic stress and pathological grief (Lange, van de Ven, Schrieken, & Emmelkamp, 2001), physical activity (McKay, King, Eakin, Seeley, & Glasgow, 2001), panic disorder (Klein & Richards, 2001), tinnitus (Andersson, Strömgren, Ström, & Lyttkens, 2002), diabetes management (McKay, Glasgow, Feil, Boles, & Barrera, 2002), and pediatric encopresis (Ritterband et al., 2003). These studies all focus on behavioral medicine/health psychology issues, which seem to be more adaptable to Internet interventions (Childress & Asamen, 1998) because of the availability of highly structured treatment approaches to many problems. Generally, these studies provide support for the notion that Internet interventions can be feasible and effective. These studies also demonstrate that some behaviorally related psychological treatments can be operationalized, transformed, and transported to the user via the Internet. To test efficacy, most of these studies used some form of a nontreatment control group rather than a face-to-face treatment group for the identified disorder. This decision makes sense given that the first step in verifying the applicability of this form of intervention is to make sure that it works rather than to hold it up to the “gold standard” of face-to-face treatments. Subsequent generations of clinical trials will need to incorporate face-to-face treatment interventions as one of the experimental conditions. However, it should not be necessary for Internet interventions to prove more effective than face-to-face treatments but rather to provide close to equivalent benefits and outcome results. As the many advantages of Internet interventions, especially accessibility, become increasingly clear, people who might not otherwise have obtained treatment may do so. These and future studies in this area are not meant to imply that face-to-face treatment should be replaced; rather, they are meant to provide an alternative or adjunctive component to already well-established and highly effective interventions. Web-based treatment interventions offer an opportunity for psychologists to provide specific behavioral treatments, tailored to individuals who prefer or need to seek help from their own homes. The technology is now available and will be more readily accessible with high-speed bandwidth, which should be widely accessible in the near future. Although the current sparse literature examining this area may have limitations, these studies represent the pioneering efforts to develop what will likely become a major force in the delivery of psychological treatments. However, many of these interventions do not take advantage of the full capabilities of the Internet. Currently, Internet interventions tend to be limited in their graphical elements and other potentially engaging factors (i.e., audio, animation, interactivity) and can certainly be improved with new development software for Web applications. There is a significant need for more diverse and comprehensive interventions, but the time and effort to operationalize an intervention is considerable. Although these new interventions are developed, current applications must overcome the difficulties of delivering their often large-file-sized programs over phone lines. A compromise that has been created is a hybrid approach. INTERNET INTERVENTIONS 529 Table 1 Empirically Tested Internet Interventions Target behaviors/ symptoms Subjects (N) Schneider, Walter, & O’Donnell, 1990 Smoking cessation 1,158 adults Internet intervention with smoking diaries vs. outline of behavioral strategies. Trends of greater cessation at 1, 3, and 6 months for Internet group. Winett et al., 1999 Improve teenage girls’ health behaviors 180 high school females Semester of Internet intervention focusing on weight loss, healthy eating, and health behaviors vs. standard high school health class with similar content. Internet group had significant improvements in many health-related behaviors. Celio et al., 2000 Body image satisfaction and eating attitudes 76 women Psychoeducational Internet program vs. same content delivered in a class vs. wait-list control. Internet group had fewer weight and shape concerns and better eating attitudes at posttreatment. Risk factors for eating disorders reduced at 4-month follow-up. Ström, Pettersson, & Andersson, 2000 Headaches 45 adults Internet intervention focused on applied relaxation and problem-solving techniques vs. wait-list control. After 6-week program, Internet intervention group experienced significantly fewer and less severe headaches. Winzelberg et al., 2000 Body image satisfaction 60 women Psychoeducational Internet program vs. control condition. After 8-week program and at 3-month follow-up, Internet group had greater satisfaction with their body and less desire for “thinness.” Klein & Richards, 2001 Panic 22 adults Internet intervention containing psychoeducational components and techniques to reduce panic vs. selfmonitoring control group. Internet subjects had reduced panic-related symptoms. Lange, van de Ven, Schrieken, & Emmelkamp, 2001 Posttraumatic stress and pathological grief 25 college students Internet intervention utilizing 5 weeks of writing assignments vs. wait-list control. 80% of experimental group demonstrated clinically significant improvement in symptoms of trauma and general psychopathology posttreatment. McKay, King, Eakin, Seeley, & Glasgow, 2001 Physical activity 78 patients with Type 2 diabetes Internet physical activity intervention vs. information-only control. Both groups increased activity levels, but no significant differences between groups. Those in Internet group who more often used program did significantly better. Tate, Wing, & Winett, 2001 Weight loss 91 adults Behavior weight loss Internet program with e-mail consultation vs. education about weight loss and access to information-based Web sites. Internet behavior therapy led to significantly more weight loss at 3 and 6 months. Andersson, Strömgren, Ström, & Lyttkens, 2002 Tinnitus 117 adults Internet cognitive–behavioral intervention for tinnitus vs. wait-list control. After 6-week program, Internet group experienced significantly greater improvements in several measures of anxiety, depression, and other tinnitusrelated distress vs. controls. McKay, Glasgow, Feil, Boles, & Barrera, 2002 Diabetes selfmanagement 133 patients with Type 2 diabetes Internet-based diabetes support program on diet, mental health, and physiologic outcomes vs. information-only control. Both groups improved, but no significant differences between groups. Ritterband et al., 2003 Pediatric encopresis 24 children Psychoeducational and interactive behavioral Internet intervention vs. no intervention. Significant improvements for children in Internet condition on no. of accidents, no. of bowel movements in the toilet/week, and increased trips to the bathroom without a parental prompt. This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Study a All studies used random assignment as part of their design. Designa Results RITTERBAND ET AL. 530 This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Hybrid Treatment Interventions (HTIs) HTIs address the problem of limited bandwidth by providing highly engaging interventions without losing the critical Internet connection. All the large files are stored either on a CD-ROM or a hard disk drive. The computer still connects to the Internet to transfer small bits of information back and forth from the utilized server. In this way, all the benefits of the Web can still be maintained, including the ability to update information, exchange information, collect data, monitor user activity in real time, provide feedback, and prompt behavior change, without losing the ability to have large data files, such as extensive graphic and audio files. The disadvantage is that the large files must be furnished to the user in order to receive the intervention. This typically means providing a CD-ROM to the user through regular mail or in person. An example of an HTI is the “U-CAN-POOP-TOO” Web site developed by our research group at the University of Virginia and supported by the National Institutes of Health. The goal in designing this program was to deliver a child-focused, entertaining, and engaging site that would provide all the necessary behavioral and medical components of Enhanced Toilet Training (Cox, Borowitz, Kovatchev, & Ling, 1998) to successfully treat pediatric encopresis. This HTI encompasses hundreds of pages of content, with numerous illustrations, interactive components, animated tutorials, and reinforcing quizzes. In order to validate this intervention, we placed a computer in the homes of half the subjects and provided Internet access to the Web site. The large graphic and animation files were put on the hard drive in order to reduce long wait times when progressing through the program. All subjects continued to receive care from their primary care physician. Children who received the Web intervention in addition to care by their primary care physician decreased their accidents by 93%, increased their number of bowel movements (BMs) in the toilet each week by 152%, and increased their trips to the bathroom without a parental prompt by 109%. The control children decreased their accidents by 31%, but they also decreased their number of BMs in the toilet each week by 16% and their trips to the bathroom without a parental prompt by 37% (Ritterband et al., 2003). These results are equivalent to those found in tests of a face-to-face behavioral intervention for pediatric encopresis (Cox et al., 1998). This hybrid example allowed for the assessment of (a) the feasibility of this type of treatment intervention, (b) the usability/ acceptability of such a system to patients, and (c) the effectiveness of such a system for delivering the necessary information, promoting behavior change, and ameliorating symptoms. Further studies of this kind are necessary to establish the validity of this type of intervention. Eventually, as more high-speed connections become available, the HTI model can be discarded, and the WBTI model will be the standard. Developing and Using Internet Interventions Developing Internet interventions is an arduous, sometimes tedious, and always time-intensive process. It necessitates an interdisciplinary approach, requiring a team of diverse professionals, including clinicians and other health care providers to provide content; computer and Web programmers to build essential applications; Web designers to create the Web structure; Web graphic artists to create still and animated images; database developers to integrate a mechanism to store and retrieve data; health informatics evaluators to evaluate user interface issues and outcomes; and behaviorists to incorporate behavior change concepts into the system. Other potential members of the team may include business/financial advisors to ensure proper marketing, management, and sales; videographers to create video; audio engineers to integrate Web audio; psychometricians to certify appropriate scale integration; tech support personnel to provide user support; costanalysis specialists to determine savings; linguists/translators to provide readability testing and translation; disability experts to oversee usability issues; and health educators to make certain the content is structured in such a way that the majority of users will find it helpful. Obviously, only a small number of psychologists would be likely to want or to have the resources to be able to create an Internet intervention. However, many clinicians may want to utilize this form of treatment within their own clinical practice. The most plausible reason for this is to supplement skills that could be addressed with an adjunctive Internet intervention. For example, a clinician who feels comfortable treating depression and anxiety may not have the training to implement the behavioral components for treating insomnia, which commonly co-occurs with these disorders. An Internet intervention for insomnia with demonstrated effectiveness could very easily be used as a component of faceto-face treatment, allowing the clinician to target this specific issue. A brief summary of the steps involved in developing Internet interventions may be helpful in conveying the underlying process of this type of approach. It may also instill an appreciation for the efforts that go into creating these programs. Also, a greater understanding of these systems will help clinicians recognize how to best integrate them into their own practice. A number of factors must be considered when creating an Internet intervention (see Figure 1). First, the disorder must be identified and the treatment should be translatable (Step 1). This means that the intervention is structured such that it can be delivered using the Internet. Typically, the treatment is highly structured and can be at least semi-self-guided. It is also important to determine the effectiveness of the intervention to be transformed (Step 2). An effective face-to-face intervention is the gold standard by which an Internet intervention will ultimately be compared. Once the treatment is identified and determined to be effective, it must be operationalized completely (Step 3). To operationalize a treatment, one must identify all critical aspects of the intervention, including specific treatment techniques and procedures. There are numerous legal and ethical issues that must be considered when developing an Internet intervention (Step 4). These include issues of privacy, confidentiality, data validity, potential misuse of Internet interventions by professionals, equality of Internet access, and credentialing issues (see Humphreys, Winzelberg, & Klaw, 2000; Jerome et al., 2000; Koocher & Morray, 2000; Sampson, Kolodinsky, & Greeno, 1997; Winker et al., 2000; Winzelberg et al., 2000). Examining these issues in depth is beyond the scope of this article, but each issue should be carefully considered before creating and/or utilizing an Internet intervention in a clinical practice. Many of these same ethical and legal issues need to be considered when incorporating an Internet intervention as an adjunctive component of treatment. This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. INTERNET INTERVENTIONS Figure 1. 531 Development of Internet interventions. The multimedia aspect of the Internet should be used in creating the intervention (Step 5). Without using Internet components such as audio, graphics, animation, and video, the intervention is little more than a self-help book that can be read online. These elements help make the intervention engaging and should increase motivation to use and complete the treatment program. Interactivity is another key component that will likely enrich the intervention and keep users connected. Personalization allows the user to receive more individually tailored content, helping to focus the intervention specifically to the user (Step 6). Personalization may be as simple as using the name of the user when presenting certain content or as sophisticated as the system identifying specific treatment areas the individual user may need to address. It is critical that users receive some feedback regarding their progress within the treatment (Step 7). These feedback loops may allow users to track specific elements of the treatment and may even provide information to be used with their clinician if the system is an adjunctive component of treatment. Several technical issues must also be addressed when constructing the Internet program (Step 8). It needs to be determined whether a fully Web-based or hybrid treatment intervention would be most appropriate. All issues of Internet delivery, including type of browser or self-contained application, type of database, and programming languages to use, must be resolved. Also, issues of cross-platform compatibility (e.g., making sure the program will work on a PC and a Macintosh), as well as hardware issues (providing minimum requirements, such as amount of machine memory and hard drive space needed) and software issues (identifying necessary browser, plug-ins, and software incompatibility issues), should be considered. Most of these issues are usually decided on the basis of the needs of the target audience. Finally, once the program is developed, testing of the application must be conducted (Step 9). There are several steps to testing the Internet intervention. Early tests may include focus groups, in which individuals are invited to view the program and provide feedback regarding its use. The intervention will also need to be pilot tested with a small group of patients in order to determine issues of feasibility, usability, and possibly early determinations of efficacy. Finally, a large clinical trial will need to be conducted in order to demonstrate effectiveness. Revisions will likely need to be incorporated into the program on the basis of users’ feedback, so a loop between Step 9 and Step 8 will occur. Future Directions and Implications Computers and the Internet have become important tools used in the field of psychology (for other reviews, see Barak, 1999; Laszlo, Esterman, & Zabko, 1999; Smith & Senior, 2001; Stamm, 1998). However, many clinical and research issues need to be addressed to further solidify as well as broaden their place in the discipline. Clinical Directions It is likely that there are many other Internet interventions currently in various stages of development and evaluation. Some are computer treatments that are in the process of being transformed to Internet interventions, including computer-based health information and support systems for patients with life-threatening illnesses (such as for patients who are HIV positive and for those with breast cancer; Gustafson et al., 1999, 2001), phobias (Kenwright, Liness, & Marks, 2001), fears of public speaking (Botella et al., 2000), and marital therapy (Jerome et al., 2000). Additional treatments need to be developed and examined for their efficacy in order to establish Internet interventions as a viable treatment alternative and to support generalizability. Some issues, such as This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 532 RITTERBAND ET AL. sexual dysfunctions, may be particularly suited to an Internet intervention, because many people may be uncomfortable seeking face-to-face help for such concerns. Internet interventions for other issues, such as insomnia, may provide help to those who do not believe their problem warrants a doctor’s visit but who would follow a treatment plan on their own. Internet interventions can help reduce many of the traditional barriers inherent in the current mental health care deliverance model, including unavailability of skilled professionals, long lag time for dissemination of information, length of treatment, costs and inconvenience of treatment, and unwillingness to seek treatment. With the use of the Internet, professionals or patients can gather information whenever they wish, and the treatments can be presented in great detail through the use of the written word as well as through such visuals as pictures, movies, and animated graphics to enhance the understanding of a disorder. Patients are likely to feel empowered by being able to digest the information at their own pace and to better use it to enhance treatment efficacy. Also, the cost of obtaining access to this information, already minimal, continues to decline every day. Finally, this mode of treatment deliverance may be much more appealing to some patients, increasing their willingness to participate and follow the recommendations. Even though Internet interventions may help overcome many barriers to mental health care treatment, some critical issues still need to be addressed and resolved, including problems of selfassessment and diagnosis, dissemination of information and treatments, establishment of a financial model, and compliance. There are potential significant negative consequences to individuals assessing and diagnosing themselves, including misdiagnosis and improper treatment selection. Internet interventions are not meant to replace face-to-face treatment but rather to provide an alternative for individuals who might otherwise choose not to receive treatment (e.g., because of embarrassment) or who might be unable to obtain treatment (e.g., because of location) or to find appropriate treatment (e.g., because no provider is available). However, it is difficult to disseminate information about these interventions and make individuals aware of them. New ways of broadcasting this type of treatment are needed so that people may know and take advantage of them. A financial model must be established so that individuals can purchase treatments, insurance companies pay for treatments, and providers charge for development and usage of treatments. Without some financial framework, these interventions will not survive regardless of how effective they are found to be. However, before individuals should be charged for these services, research must be conducted to prove their effectiveness. Compliance is a problem for traditional approaches and for Internet interventions. Although people reported enjoying the “UCAN-POOP-TOO” program, it was still difficult to maintain compliance. Subjects were instructed to access the program at least once a week; however, some subjects needed numerous phone calls to remind and encourage them to return to the site and complete the program. In the body image studies conducted at Stanford University (Winzelberg et al., 2000), subjects typically completed less than two thirds of the program, and compliance progressively declined each week of the intervention. Suggestions have been made for improving compliance, including stressing the importance of following through with treatment recommendations at the beginning of the program, utilizing various “motivational components” built into the program, and providing reinforcement for cooperation and program completion. Research Directions It is vital that methodologically sound clinical studies be conducted to ensure the efficacy of Internet interventions. A few early studies show promising outcomes; however, although it appears that the feasibility of this form of treatment approach is possible, the development of additional Internet treatments and adequate clinical trials are necessary to establish Internet interventions as a viable treatment option. Many of the areas identified in the Clinical Directions section of this article need to be examined by means of scientific measures. Issues of assessment and diagnosis, cost-effectiveness, and compliance all need to be explored. Compliance, in particular, is an area in which Internet interventions may make a significant difference. Patient compliance is said to increase when patients’ satisfaction with the communication of the health-related information increases (Ley, 1982). Research is necessary to determine whether computer and Internet components, such as audio, video, graphics/animation, and interaction, will improve compliance. It may be that through the use of Internet interventions, treatment programs can be made to be more engaging, thus increasing an individual’s motivation to continue working. Internet interventions and other computer treatments can include personalized and pertinent information, quizzes, case reports, and games to make the programs more enticing, but the impact of all of these needs to be assessed. Internet treatment programs can also utilize the power of e-mail (e.g., individuals could be sent an e-mail with a hyperlinked Web address to access the Web program more easily). This may also serve as a behavioral prompt that might improve compliance. However, it is also possible that compliance may be even more problematic with computer/Internet treatment programs because of limited supervision, but no research as of yet has focused on this important question. There is also a need for more basic research examining components of the Internet and, specifically, the Web and its ability to elicit behavioral changes. Some researchers have questioned whether changing various elements within a computer program, such as adding color, sound, and video, would improve or hinder outcome (Sproull, Subramani, Kiesler, & Walker, 1996). Similar inquiries need to be addressed with respect to the Web. These basic research questions will help shape a model or framework for developing WBTIs. Although it might be tempting to use business models already developed for creating engaging and “sticky” Web sites (i.e., those Web sites that keep users at the site and draw them back for future visits), it is important to note that these models do not address the most critical component of Internet interventions: behavior change. A theoretical model for this type of approach is necessary so that additional Internet interventions can be developed. Future Possibilities In addition to the developments already presented, the future promises even greater technologies and applications for psychology. Web-based treatment interventions, as well as enhanced This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. INTERNET INTERVENTIONS videoconferencing and other bandwidth-intensive approaches, will likely flourish as increased and improved Internet access becomes more prevalent. Wireless technology is also rapidly improving, and consumer products with wireless capabilities are becoming more available. Some users already have cellular phones and hand-held computers with wireless Internet access. These products will become even more widely available and easier to use over the coming years. These wireless products will allow individuals to track and share information with their health care professions in real time, ease data input, reduce data-entry errors, and, we hope, improve compliance. In addition to wireless technologies, older technologies such as the telephone will be better able to interface with some of the newer technologies, including the Internet (see Ritterband et al., 2001). For example, Internet applications can be created to call patients on a regular basis to track progress and symptom relief and prompt for recommended behaviors. This information can be stored on a Web server database that can be accessed by the individual and the health care professional. Patient care and patients’ perceptions of care will likely improve with this type of technology. Gaming consoles and future personal digital assistants already have Internet capabilities, and treatment-based software could be created for use on these platforms. Children may be more interested in receiving treatments if they are presented in an engaging and entertaining way. Hand-held computers can be carried anywhere, which could improve the collection of symptom information and provide reminders of treatment components to improve compliance. This information could be wirelessly transmitted to the user’s health care professional, who could help expedite treatment goals. Implications It is unlikely that Internet interventions will replace face-to-face psychotherapy; however, this technology may be helpful in the treatment of some psychological problems that might otherwise go untreated. It is also possible that such interventions may enhance traditional therapy as an adjunctive component. These new possibilities usher in a whole new platform from which mental health, and health care in general, can be conducted. Clinicians will have to be trained to understand, create, and use these forms of treatment. In time, advances in technology will escalate and push health care to use its power to improve care to the benefit of all. The following are some of the implications of the issues and the research presented in this article: 1. Additional clinical treatment interventions need to be operationalized and transformed into Internet interventions for greater patient consumption. 2. These interventions must be empirically validated through well-designed clinical research studies. 3. Studies of all components of Internet interventions are necessary to establish and improve feasibility, usability, and efficacy. 4. Psychologists need to accept that technology is changing the world. This means that multidisciplinary teams will likely include a rather unusual set of people to provide new treatment interventions. 5. New definitions of treatment must be created that embrace 533 the use of new technologies and encompass the use of the Internet in their deliverance. 6. Costs to create and test Internet interventions are significant and should not be underestimated when considering the development of a new program. 7. Psychologists need to understand and accept that there are numerous ethical and legal issues relevant to providing new ways to deliver health care and that it takes time before these issues can be appropriately managed. References Andersson, G., Strömgren, T., Ström, L., & Lyttkens, L. (2002). Randomized controlled trial of Internet-based cognitive behavior therapy for distress associated with tinnitus. Psychosomatic Medicine, 64, 810 – 816. Barak, A. (1999). Psychological applications on the Internet: A discipline on the threshold of a new millennium. Applied and Preventive Psychology, 8, 231–246. Borowitz, K. C., & Borowitz, S. M. (2000, May). Using the World Wide Web to teach people about cerebral palsy. Paper presented at the annual meeting of the Society for Pediatric Research, Boston, MA. Borowitz, S. M., & Ritterband, L. M. (2001). Using the Internet to teach parents and children about constipation and encopresis. Medical Informatics and the Internet in Medicine, 26, 283–295. Botella, C., Banos, R., Guillen, V., Perpina, C., Alcaniz, M., & Pons, A. (2000). Telepsychology: Public speaking fear treatment on the Internet. Cyberpsychology & Behavior, 3, 959 –968. Celio, A., Winzelberg, A. J., Wilfley, D., Eppstein-Herald, D., Springer, E., Dev, P., & Taylor, C. B. (2000). Reducing risk factors for eating disorders: Comparison of an Internet- and a classroom-delivered psychoeducational program. Journal of Consulting and Clinical Psychology, 68, 650 – 657. Childress, C. A., & Asamen, J. K. (1998). The emerging relationship of psychology and the Internet: Proposed guidelines for conducting Internet intervention research. Ethics & Behavior, 8, 19 –35. Cox, D. J., Borowitz, S. M., Kovatchev, B., & Ling, W. (1998). Contributions of behavior therapy and biofeedback to laxative therapy in the treatment of pediatric encopresis. Annals of Behavioral Medicine, 20, 70 –76. Crutsinger, M. (2000, October 17). Computers in half of U.S. homes. The Washington Post, p. E17. Employment Policy Foundation. (2001, July 11). Computer ownership and Internet access: Opportunities for workforce development and job flexibility. Technology Forecast. Retrieved from http://www.epf.org/ research/newsletters/2001/tf20010111.pdf Gustafson, D. H., Hawkins, R., Boberg, E., Pingree, S., Serlin, R., Graziano, F., & Chan, C. L. (1999). Impact of a patient-centered, computerbased health information/support system. American Journal of Preventive Medicine, 16, 1–9. Gustafson, D. H., Hawkins, R., Pingree, S., McTavish, F., Arora, N. K., Mendenhall, J., et al. (2001). Effect of computer support on younger women with breast cancer. Journal of General Internal Medicine, 16, 435– 445. Humphreys, K., Winzelberg, A. J., & Klaw, E. (2000). Psychologists’ ethical responsibilities in Internet-based groups: Issues, strategies, and a call for dialogue. Professional Psychology: Research and Practice, 31, 493– 496. Jerome, L. W., DeLeon, P. H., James, L. C., Folen, R., Earles, J., & Gedney, J. J. (2000). The coming age of telecommunications in psychological research and practice. American Psychologist, 55, 407– 421. Kenwright, M., Liness, S., & Marks, I. (2001). Reducing demands on clinicians by offering computer-aided self-help for phobia/panic. British Journal of Psychiatry, 179, 456 – 459. This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. 534 RITTERBAND ET AL. Klein, B., & Richards, J. C. (2001). A brief Internet-based treatment for panic disorder. Behavioural and Cognitive Psychotherapy, 29, 113–117. Kolata, G. (2000, March 6). Web research transforms visit to the doctor. New York Times. Retrieved from http://archives.nytimes.com Koocher, G. P., & Morray, E. (2000). Regulation of telepsychology: A survey of state attorneys general. Professional Psychology: Research and Practice, 31, 503–508. Lange, A., Ven, J.-P. van de, Schrieken, B., & Emmelkamp, P. (2001). Interapy. Treatment of posttraumatic stress through the Internet: A controlled trial. Journal of Behavior Therapy and Experimental Psychiatry, 32, 73–90. Laszlo, J. V., Esterman, G., & Zabko, S. (1999). Therapy over the Internet? Theory, research, and finances. CyberPsychology & Behavior, 2, 293– 307. Ley, P. (1982). Satisfaction, compliance and communication. British Journal of Clinical Psychology, 21, 241–254. McKay, G. H., Glasgow, R. E., Feil, E. G., Boles, S. M., & Barrera, M. (2002). Internet-based diabetes self-management and support: Initial outcomes from the Diabetes Network Project. Rehabilitation Psychology, 47, 31– 48. McKay, G. H., King, D., Eakin, E. G., Seeley, J. R., & Glasgow, R. E. (2001). The Diabetes Network Internet-based physical activity intervention. Diabetes Care, 24, 1328 –1334. Pew Research Center. (2002, May 22). Vital decisions: How Internet users decide what information to trust when they or their loved ones are sick. Pew Internet & American Life. Retrieved May 30, 2002, from http:// www.pewinternet.org/reports/toc.asp?Report⫽59 Rabasca, L. (2000, April). Self-help sites: A blessing or a bane? Monitor on Psychology, 31(4), 28 –30. Ritterband, L. M., Cox, D. C., Kovatchev, B. P., Borowitz, S. M., Ling, W. D., & Marshall, B. (2001). A telecommunication monitoring system for clinical and research practice. Professional Psychology: Research and Practice, 32, 636 – 641. Ritterband, L. M., Cox, D. C., Walker, L., Kovatchev, B., McKnight, L., Patel, K., et al. (2003). A Web-based treatment intervention as adjunctive therapy for pediatric encopresis. Journal of Consulting and Clinical Psychology, 71, 910 –917. Sampson, J. P., Kolodinsky, R. W., & Greeno, B. P. (1997). Counseling and the information highway: Future possibilities and potential problems. Journal of Counseling & Development, 75, 203–212. Schneider, S. J., Walter, R., & O’Donnell, R. (1990). Computerized communication as a medium for behavioral smoking cessation treatment: Controlled evaluation. Computers in Human Behavior, 6, 141–151. Smith, M. A., & Senior, C. (2001). The Internet and clinical psychology: A general review of the implications. Clinical Psychology Review, 21, 129 –136. Sproull, L., Subramani, M., Kiesler, S., & Walker, J. H. (1996). When the interface is a face. Human-Computer Interaction, 11, 97–124. Stamm, B. H. (1998). Clinical applications of telehealth in mental health care. Professional Psychology: Research and Practice, 29, 536 –542. Ström, L., Pettersson, R., & Andersson, G. (2000). A controlled trial of self-help treatment of recurrent headache conducted via the Internet. Journal of Consulting and Clinical Psychology, 68, 722–727. Tate, D. F., Wing, R. R., & Winett, R. A. (2001). Using Internet technology to deliver a behavioral weight loss program. Journal of the American Medical Association, 285, 1172–1177. Winett, R. A., Roodman, A. A., Winett, S. G., Bajzek, W., Rovniak, L. S., & Whiteley, J. A. (1999). The effects of the Eat4Life Internet-based health behavior program on the nutrition and activity of high school girls. Journal of Gender, Culture, and Health, 4, 239 –254. Winker, M. A., Flanagin, A., Chi-Lum, B., White, J., Andrews, K., & Kennett, R. L. (2000). Guidelines for medical and health information sites on the Internet: Principles governing AMA Web sites. Journal of the American Medical Association, 283, 1600 –1606. Winzelberg, A. J., Eppstein, D., Eldredge, K. L., Wilfley, D., Dasmahapatra, R., Dev, P., & Taylor, C. B. (2000). Effectiveness of an Internet-based program for reducing risk factors for eating disorders. Journal of Consulting and Clinical Psychology, 68, 346 –350. Received June 11, 2002 Revision received May 15, 2003 Accepted May 22, 2003 䡲
Purchase answer to see full attachment
User generated content is uploaded by users for the purposes of learning and should be used following Studypool's honor code & terms of service.

Explanation & Answer


Anonymous
Really helped me to better understand my coursework. Super recommended.

Studypool
4.7
Trustpilot
4.5
Sitejabber
4.4

Related Tags