Journal of Adolescence 36 (2013) 1153–1163
Contents lists available at ScienceDirect
Journal of Adolescence journal homepage: www.elsevier.com/locate/jado
Testing the Index of Problematic Online Experiences (I-POE) with a national sample of adolescents Kimberly J. Mitchell a, *, Lisa M. Jones a, Melissa Wells b a b
Crimes against Children Research Center, University of New Hampshire, 10 West Edge Drive, Suite 106, Durham, NH 03824, USA Department of Social Work, University of New Hampshire, 314 Pettee Hall, Durham, NH, USA
a b s t r a c t Keywords: Internet Technology Problem behavior Mental health Adolescence Assessment Problematic internet use
This article assesses the utility of the Index of Problematic Online Experiences (I-POE) in a national sample of adolescents in the United States. The study was based on a crosssectional national telephone survey of 1560 Internet users, ages 10 through 17. Data were collected between August, 2010 and January, 2011. The I-POE is an 18-item binary response index which can be used to assess problematic internet use across multiple behaviors and activities. Exploratory and confirmatory factor analysis supported a revised index with two factors: a 9-item “excessive use” scale and a 9-item “online social and communication problems” scale among this population. The I-POE showed favorable psychometric properties including adequate internal consistency for the overall scale and for the two subscales. Scores correlate with offline emotional and behavioral difficulties and the I-POE could have value for use as a part of broad mental health assessment procedures in clinical or school settings. Ó 2013 The Foundation for Professionals in Services for Adolescents. Published by Elsevier Ltd. All rights reserved.
For today’s youth, technology has become an integral part of their lives. Ninety-five percent of teens in the United States, ages 12 through 17 use the internet (Lenhart et al., 2011), much of it to communicate with their friends. Eighty percent of teens who are online use social media sites like Facebook or MySpace. Seventy-five percent of teens use text messaging (Lenhart, 2012) with a median of 60 texts sent and received per day. Texting has become a primary mode of daily communication between teens and their friends and family. With this rapid rise in the use of internet technology by youth, a variety of concerns have developed including, for example, overuse and the use of technology to further problematic communication and relationships with known peers or those met online (Mitchell, Becker-Blease, & Finkelhor, 2005). While most youth use technology responsibly and report positive interactions, some have problems interacting in this environment. Mental health professionals report that overuse, relationship problems, and other problematic online experiences are showing up as both primary and secondary presenting problems (Mitchell & Wells, 2007). However, because new technology is so pervasive in adolescent culture, it can sometimes be difficult for parents, and health professionals to distinguish between normative and problematic technology use by youth. For example, it can be difficult to determine how much is “too much” use. Adults can also find it difficult to monitor who their children are talking to online and what their conversations are about. There has been increasing study of problematic internet use (PIU) over the past decade – of both youth and adults alike and throughout the United States, Asia and many European countries. Study of PIU, or “use of the internet that creates psychological, social, school and/or work difficulties in a person’s life.” (Beard & Wolf, 2001,
* Corresponding author. Tel.: þ1 603 862 4533. E-mail addresses: [email protected] (K.J. Mitchell), [email protected] (L.M. Jones), [email protected] (M. Wells). 0140-1971/$ – see front matter Ó 2013 The Foundation for Professionals in Services for Adolescents. Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.adolescence.2013.09.004
1154
K.J. Mitchell et al. / Journal of Adolescence 36 (2013) 1153–1163
page 378) has been complicated by differences in conceptualization. Young based her initial diagnostic criteria for “internet addiction” on substance dependence due to similarities with tolerance levels and withdrawal (Young, 1996); other researchers based their conceptualizations on behavioral addiction not involving a chemical, such as gambling, sex, and gaming (Griffiths, 1997, 2001; Lopez-Fernandez, Freixa-Blanxart, & Honrubia-Serrano, 2013); still others have based their perspectives of PIU as being more similar to an impulse control disorder (Shapira et al., 2003). Three of the most widely used scales to assess PIU are Young’s Diagnostic Questionnaire (Young, 1998a), Young’s Internet Addiction Test (Young, 1998b), and the Chen Internet Addiction Scale (Chen, Weng, Su, Wu, & Yang, 2003). Other instruments include the Generalized Problematic Internet Use Scale 2 (Caplan, 2010), the Problematic Internet Entertainment Use Scale for Adolescents (Lopez-Fernandez et al., 2013); and the Online Cognition Scale (Davis, Flett, & Besser, 2002); all of which focus on the measurement of overuse, excessive use or addiction and the consequences or functional impairment of such usage. However, these instruments are limited for use with adolescents in the United States for two important reasons. First, most of the research on PIU in the United States has focused on adults (existing studies that measure PIU among adolescents are predominately from Asia and Europe (Lopez-Fernandez et al., 2013; Moreno, Jelenchick, Cox, Young, & Christakis, 2011)). No measures, to our knowledge, were specifically designed for use with adolescents in the Unites States (Moreno et al., 2011) and nationally representative data on Unites States adolescents is needed to help establish norms for determining when technology use is and is not problematic for this age group. Additionally, given that internet use and problematic experiences change significantly from pre-adolescence to late adolescence (Jones, Mitchell, & Finkelhor, 2012; Livingstone, 2009), there is a critical need for research to look separately at younger and older youth. Second, most of the pre-existing instruments focus on overuse and addiction. Given concerns about problems with online harassment and cyberbullying (Hinduja & Patchin, 2008; Jones, Mitchell, & Finkelhor, 2013; Mishna, Cook, Gadalla, Daciuk, & Solomon, 2010; Oblad, 2012; Schneider, O’Donnell, Stueve, & Coulter, 2012; Ybarra, Diener-West, & Leaf, 2007; Ybarra, Espelage, & Mitchell, 2007; Ybarra & Mitchell, 2004a, 2004b), particularly among adolescents, we believe the conceptualization of PIU needs to be broadened to include communication or behavior problems online, perhaps even rare but very serious events that result in adverse outcomes. The current study presents new data on a tool for identifying and better understanding PIU patterns aimed at addressing these gaps in the measurement literature: the Index of Problematic Online Experiences (I-POE) (Mitchell, Sabina, Finkelhor, & Wells, 2009). A reliable and validated tool for measuring youth problems online against national norms could be a valuable part of a standard clinical assessment in behavioral or mental health settings. It would allow clinicians to incorporate such information into therapeutic interventions and responses. Additionally, research using the I-POE to measure PIU for youth could also improve our understanding of how such problems fit into larger patterns of problematic emotional and behavioral functioning. There is an increasing understanding that serious online problems may be indicative of a broader pattern of problem behaviors and/or underlying emotional issues for youth, and vice versa. For example, previous research has determined that youth who encounter repeated difficulties online, such as unwanted sexual advances and online harassment, report elevated rates of victimization and behavioral problems offline (Ybarra & Mitchell, 2004a). However, it is not yet clear the degree that problematic internet experiences and behavior by youth are extensions of offline problems, or if there are subsets of youth for whom new technology provides an environment or opportunity for problems to occur when they might not have otherwise. The I-POE was developed to assess an overall pattern of internet problems, rather than examining any one specific type of problem behavior (Mitchell et al., 2009). The initial domains for the I-POE were constructed using Davis’ model of generalized problematic internet use (Davis, 2001), and drawing from a study of online problematic experiences identified as presenting or secondary issues in mental health settings, such as overuse, use of pornography, or exploitation (Wells & Mitchell, 2007a, 2007b; Wells, Mitchell, Finkelhor, & Becker Blease, 2006). The original I-POE domains included: 1) overuse, 2) problems with family or friends, 3) problems with daily obligations, 4) problems related to interactions with people online, 5) upset or concern about one’s own internet use, and 6) online behavior concerns. The first study tested the I-POE on a sample of college students in Northern New England and found initial support for construct validity: the I-POE scores were correlated with a variety of relevant constructs: depression, anger/irritability, tension-reduction behavior, sexual concerns, dysfunctional sexual behavior, amount of internet use, and permissive attitudes toward engaging in sexual activities (Mitchell, et al. 2009). In the current article we expand beyond that initial study with a larger, nationally representative sample of adolescents, ages 10 through 17. Given that new technology use and problems often begin and expand in adolescence, we were interested in determining the validity of the I-POE and establishing national norms with this population. An additional aim of the current study was to re-evaluate the structure of the I-POE to determine whether fewer factors were supported by the data. Some of the originally developed subscales seemed to overlap, were determined by only a few items, and it was unclear how useful the six subscales might be in assessments. Below, we report on a confirmatory factor analysis to evaluate whether the prior 6-factor solution found among college students in a Northern New England university applies among a national sample of adolescents in the United States. We then use exploratory factor analysis and, again confirmatory factor analysis to identify and validate a new factor structure with this population. Given research which suggests relationships between PIU and poor well-being, including depression (Caplan, 2010; Kim et al., 2006; Liu, Desai, Krishnan-Sarin, Cavallo, & Potenza, 2011), delinquency and aggressive behavior (Liu et al., 2011), and substance use (Frangos, Frangos, & Sotiropoulos, 2011; Liu et al., 2011), as well as indicators of specific types of problematic experiences online (Liu et al., 2011; Mitchell et al., 2009), we examine the I-POE’s reliability and construct validity by examining its relationship with established clinical markers (depression, delinquency, substance use), as well as high internet use, online victimization, online perpetration and online sexual behavior. Finally, we explore sex and age differences for total I-POE scores as well as for individual sub-indices.
K.J. Mitchell et al. / Journal of Adolescence 36 (2013) 1153–1163
1155
Method Study design The I-POE measure was administered to the current sample of youth via the 3rd Youth Internet Safety Survey (YISS-3). The data collection for YISS-3 took place between August, 2010 and January, 2011. YISS-3 was conducted via telephone surveys with a national sample of 1560 youth internet users, ages 10–17, and their parents. A sample size of 1500 was pre-determined based upon a maximum expected sampling error of 2.5% at the 5% significance level. Human subject participation in YISS-3 was reviewed and approved by the University of New Hampshire Institutional Review Board and conformed to the rules mandated for research projects funded by the U.S. Department of Justice. Sample characteristics The sample for YISS-3 was drawn from a national sample of households with telephones developed by random digit dialing with the goal of completing 1500 surveys. Eligible respondents were youth, ages 10–17, who had used the internet at least once a month for the past six months from any location, and a caregiver in each household that self-identified as the one most knowledgeable about the youth’s internet practices. A broad definition of ‘internet use’ was used to ensure a wide range of internet use behaviors and to include youth with and without home internet access. Table 1 provides details of the overall sample characteristics as well as a breakdown of demographic characteristics and our key online and offline indicators (described below) by age (10–12 years old versus 13–17 years old) and I-POE endorsement (none or any). Procedures Upon reaching a household, interviewers spoke with an adult and determined whether there was an eligible child in the household. In households with eligible children, interviewers asked to speak with the adult who was most familiar with that child’s internet use and after receiving informed consent, asked a series of questions about internet use. At the close of the parent survey, the interviewer asked for permission to interview the child. Parents were informed by interviewers that the youth interview would be confidential, would include questions about “sexual material your child may have seen on the Table 1 Demographic and clinical characteristics of all youth and by any I-POE endorsement. Characteristic
Demographic characteristics Age at time of survey 10–12 13–15 16–17 Sex Boy Girl Race/ethnicity White Black Hispanic or Latino (any race) Youth lives with both biological parents Parent a college graduate Low household income in 2009 (
Contents lists available at ScienceDirect
Journal of Adolescence journal homepage: www.elsevier.com/locate/jado
Testing the Index of Problematic Online Experiences (I-POE) with a national sample of adolescents Kimberly J. Mitchell a, *, Lisa M. Jones a, Melissa Wells b a b
Crimes against Children Research Center, University of New Hampshire, 10 West Edge Drive, Suite 106, Durham, NH 03824, USA Department of Social Work, University of New Hampshire, 314 Pettee Hall, Durham, NH, USA
a b s t r a c t Keywords: Internet Technology Problem behavior Mental health Adolescence Assessment Problematic internet use
This article assesses the utility of the Index of Problematic Online Experiences (I-POE) in a national sample of adolescents in the United States. The study was based on a crosssectional national telephone survey of 1560 Internet users, ages 10 through 17. Data were collected between August, 2010 and January, 2011. The I-POE is an 18-item binary response index which can be used to assess problematic internet use across multiple behaviors and activities. Exploratory and confirmatory factor analysis supported a revised index with two factors: a 9-item “excessive use” scale and a 9-item “online social and communication problems” scale among this population. The I-POE showed favorable psychometric properties including adequate internal consistency for the overall scale and for the two subscales. Scores correlate with offline emotional and behavioral difficulties and the I-POE could have value for use as a part of broad mental health assessment procedures in clinical or school settings. Ó 2013 The Foundation for Professionals in Services for Adolescents. Published by Elsevier Ltd. All rights reserved.
For today’s youth, technology has become an integral part of their lives. Ninety-five percent of teens in the United States, ages 12 through 17 use the internet (Lenhart et al., 2011), much of it to communicate with their friends. Eighty percent of teens who are online use social media sites like Facebook or MySpace. Seventy-five percent of teens use text messaging (Lenhart, 2012) with a median of 60 texts sent and received per day. Texting has become a primary mode of daily communication between teens and their friends and family. With this rapid rise in the use of internet technology by youth, a variety of concerns have developed including, for example, overuse and the use of technology to further problematic communication and relationships with known peers or those met online (Mitchell, Becker-Blease, & Finkelhor, 2005). While most youth use technology responsibly and report positive interactions, some have problems interacting in this environment. Mental health professionals report that overuse, relationship problems, and other problematic online experiences are showing up as both primary and secondary presenting problems (Mitchell & Wells, 2007). However, because new technology is so pervasive in adolescent culture, it can sometimes be difficult for parents, and health professionals to distinguish between normative and problematic technology use by youth. For example, it can be difficult to determine how much is “too much” use. Adults can also find it difficult to monitor who their children are talking to online and what their conversations are about. There has been increasing study of problematic internet use (PIU) over the past decade – of both youth and adults alike and throughout the United States, Asia and many European countries. Study of PIU, or “use of the internet that creates psychological, social, school and/or work difficulties in a person’s life.” (Beard & Wolf, 2001,
* Corresponding author. Tel.: þ1 603 862 4533. E-mail addresses: [email protected] (K.J. Mitchell), [email protected] (L.M. Jones), [email protected] (M. Wells). 0140-1971/$ – see front matter Ó 2013 The Foundation for Professionals in Services for Adolescents. Published by Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.adolescence.2013.09.004
1154
K.J. Mitchell et al. / Journal of Adolescence 36 (2013) 1153–1163
page 378) has been complicated by differences in conceptualization. Young based her initial diagnostic criteria for “internet addiction” on substance dependence due to similarities with tolerance levels and withdrawal (Young, 1996); other researchers based their conceptualizations on behavioral addiction not involving a chemical, such as gambling, sex, and gaming (Griffiths, 1997, 2001; Lopez-Fernandez, Freixa-Blanxart, & Honrubia-Serrano, 2013); still others have based their perspectives of PIU as being more similar to an impulse control disorder (Shapira et al., 2003). Three of the most widely used scales to assess PIU are Young’s Diagnostic Questionnaire (Young, 1998a), Young’s Internet Addiction Test (Young, 1998b), and the Chen Internet Addiction Scale (Chen, Weng, Su, Wu, & Yang, 2003). Other instruments include the Generalized Problematic Internet Use Scale 2 (Caplan, 2010), the Problematic Internet Entertainment Use Scale for Adolescents (Lopez-Fernandez et al., 2013); and the Online Cognition Scale (Davis, Flett, & Besser, 2002); all of which focus on the measurement of overuse, excessive use or addiction and the consequences or functional impairment of such usage. However, these instruments are limited for use with adolescents in the United States for two important reasons. First, most of the research on PIU in the United States has focused on adults (existing studies that measure PIU among adolescents are predominately from Asia and Europe (Lopez-Fernandez et al., 2013; Moreno, Jelenchick, Cox, Young, & Christakis, 2011)). No measures, to our knowledge, were specifically designed for use with adolescents in the Unites States (Moreno et al., 2011) and nationally representative data on Unites States adolescents is needed to help establish norms for determining when technology use is and is not problematic for this age group. Additionally, given that internet use and problematic experiences change significantly from pre-adolescence to late adolescence (Jones, Mitchell, & Finkelhor, 2012; Livingstone, 2009), there is a critical need for research to look separately at younger and older youth. Second, most of the pre-existing instruments focus on overuse and addiction. Given concerns about problems with online harassment and cyberbullying (Hinduja & Patchin, 2008; Jones, Mitchell, & Finkelhor, 2013; Mishna, Cook, Gadalla, Daciuk, & Solomon, 2010; Oblad, 2012; Schneider, O’Donnell, Stueve, & Coulter, 2012; Ybarra, Diener-West, & Leaf, 2007; Ybarra, Espelage, & Mitchell, 2007; Ybarra & Mitchell, 2004a, 2004b), particularly among adolescents, we believe the conceptualization of PIU needs to be broadened to include communication or behavior problems online, perhaps even rare but very serious events that result in adverse outcomes. The current study presents new data on a tool for identifying and better understanding PIU patterns aimed at addressing these gaps in the measurement literature: the Index of Problematic Online Experiences (I-POE) (Mitchell, Sabina, Finkelhor, & Wells, 2009). A reliable and validated tool for measuring youth problems online against national norms could be a valuable part of a standard clinical assessment in behavioral or mental health settings. It would allow clinicians to incorporate such information into therapeutic interventions and responses. Additionally, research using the I-POE to measure PIU for youth could also improve our understanding of how such problems fit into larger patterns of problematic emotional and behavioral functioning. There is an increasing understanding that serious online problems may be indicative of a broader pattern of problem behaviors and/or underlying emotional issues for youth, and vice versa. For example, previous research has determined that youth who encounter repeated difficulties online, such as unwanted sexual advances and online harassment, report elevated rates of victimization and behavioral problems offline (Ybarra & Mitchell, 2004a). However, it is not yet clear the degree that problematic internet experiences and behavior by youth are extensions of offline problems, or if there are subsets of youth for whom new technology provides an environment or opportunity for problems to occur when they might not have otherwise. The I-POE was developed to assess an overall pattern of internet problems, rather than examining any one specific type of problem behavior (Mitchell et al., 2009). The initial domains for the I-POE were constructed using Davis’ model of generalized problematic internet use (Davis, 2001), and drawing from a study of online problematic experiences identified as presenting or secondary issues in mental health settings, such as overuse, use of pornography, or exploitation (Wells & Mitchell, 2007a, 2007b; Wells, Mitchell, Finkelhor, & Becker Blease, 2006). The original I-POE domains included: 1) overuse, 2) problems with family or friends, 3) problems with daily obligations, 4) problems related to interactions with people online, 5) upset or concern about one’s own internet use, and 6) online behavior concerns. The first study tested the I-POE on a sample of college students in Northern New England and found initial support for construct validity: the I-POE scores were correlated with a variety of relevant constructs: depression, anger/irritability, tension-reduction behavior, sexual concerns, dysfunctional sexual behavior, amount of internet use, and permissive attitudes toward engaging in sexual activities (Mitchell, et al. 2009). In the current article we expand beyond that initial study with a larger, nationally representative sample of adolescents, ages 10 through 17. Given that new technology use and problems often begin and expand in adolescence, we were interested in determining the validity of the I-POE and establishing national norms with this population. An additional aim of the current study was to re-evaluate the structure of the I-POE to determine whether fewer factors were supported by the data. Some of the originally developed subscales seemed to overlap, were determined by only a few items, and it was unclear how useful the six subscales might be in assessments. Below, we report on a confirmatory factor analysis to evaluate whether the prior 6-factor solution found among college students in a Northern New England university applies among a national sample of adolescents in the United States. We then use exploratory factor analysis and, again confirmatory factor analysis to identify and validate a new factor structure with this population. Given research which suggests relationships between PIU and poor well-being, including depression (Caplan, 2010; Kim et al., 2006; Liu, Desai, Krishnan-Sarin, Cavallo, & Potenza, 2011), delinquency and aggressive behavior (Liu et al., 2011), and substance use (Frangos, Frangos, & Sotiropoulos, 2011; Liu et al., 2011), as well as indicators of specific types of problematic experiences online (Liu et al., 2011; Mitchell et al., 2009), we examine the I-POE’s reliability and construct validity by examining its relationship with established clinical markers (depression, delinquency, substance use), as well as high internet use, online victimization, online perpetration and online sexual behavior. Finally, we explore sex and age differences for total I-POE scores as well as for individual sub-indices.
K.J. Mitchell et al. / Journal of Adolescence 36 (2013) 1153–1163
1155
Method Study design The I-POE measure was administered to the current sample of youth via the 3rd Youth Internet Safety Survey (YISS-3). The data collection for YISS-3 took place between August, 2010 and January, 2011. YISS-3 was conducted via telephone surveys with a national sample of 1560 youth internet users, ages 10–17, and their parents. A sample size of 1500 was pre-determined based upon a maximum expected sampling error of 2.5% at the 5% significance level. Human subject participation in YISS-3 was reviewed and approved by the University of New Hampshire Institutional Review Board and conformed to the rules mandated for research projects funded by the U.S. Department of Justice. Sample characteristics The sample for YISS-3 was drawn from a national sample of households with telephones developed by random digit dialing with the goal of completing 1500 surveys. Eligible respondents were youth, ages 10–17, who had used the internet at least once a month for the past six months from any location, and a caregiver in each household that self-identified as the one most knowledgeable about the youth’s internet practices. A broad definition of ‘internet use’ was used to ensure a wide range of internet use behaviors and to include youth with and without home internet access. Table 1 provides details of the overall sample characteristics as well as a breakdown of demographic characteristics and our key online and offline indicators (described below) by age (10–12 years old versus 13–17 years old) and I-POE endorsement (none or any). Procedures Upon reaching a household, interviewers spoke with an adult and determined whether there was an eligible child in the household. In households with eligible children, interviewers asked to speak with the adult who was most familiar with that child’s internet use and after receiving informed consent, asked a series of questions about internet use. At the close of the parent survey, the interviewer asked for permission to interview the child. Parents were informed by interviewers that the youth interview would be confidential, would include questions about “sexual material your child may have seen on the Table 1 Demographic and clinical characteristics of all youth and by any I-POE endorsement. Characteristic
Demographic characteristics Age at time of survey 10–12 13–15 16–17 Sex Boy Girl Race/ethnicity White Black Hispanic or Latino (any race) Youth lives with both biological parents Parent a college graduate Low household income in 2009 (
