NECC 2007 Research Paper Discussion – Greenhow 21st Century Teaching Online: Strategies for Planning, Implementation and Assessment Christine Greenhow University of Minnesota greenhow@umn.edu Purpose The purpose of this study was to outline practical approaches to planning, implementing, and assessing 21st century teaching online and discuss implications of these approaches for K-16 educators in view of emerging Web 2.0. technologies. Lessons from “early adopters” about the kinds of institutional supports they felt were necessary can help educators trying to emulate similar practices. This paper presents findings from a yearlong case study of the online teaching processes of a unique group of experienced technology-using educators at a Midwestern university. Classes studied used teaching and collaboration tools which are increasingly common in K-12 and higher education contexts, including asynchronous threaded discussion, chat, videoconferencing software, online course management systems (i.e., WebCt/Blackboard) and online survey tools. However, instructors were unique in their orientation toward learner-centered rather than lecture-oriented teaching and content delivery (Bransford, Brown & Cocking, 1999). Socio-cultural activity theoretical perspectives and Activity-oriented Design Methods (AODM) were used to investigate three questions: (1) How do instructors go about planning to teach online, (2) How do instructors use online tools to achieve their teaching and student learning goals, and (3) How do instructors assess learning and teaching when using online course elements. Perspectives Access to online teaching and learning has exploded in education. Today, over 93% of K-12 instructional classrooms offer high-speed Internet (NCES, 2005). In higher education, a majority of colleges and universities offer online courses and distance learning programs (Fletcher, 2004). However, to keep pace with economic and educational advances in other countries, US students and educators need to take better advantage of existing technology infrastructures to develop 21st century skills and knowledge among all students (Partnership for 21st Century Skills, 2003). This knowledge includes: * Competency in core subjects; * Learning skills (e.g., the capacity for flexible decision-making; the ability to analyze, manage, create and communicate information in a variety of forms and contexts; the ability to demonstrate teamwork and leadership, monitor one’s own understanding, and practice personal and social responsibility); * An orientation toward lifelong learning, or the capacity to use information and communication technologies to develop one’s learning day-to-day and across the lifespan; * Situational skills, or the ability to connect one’s actual experiences to governmental, economic, and civic trends and stay up on current events; * High quality assessments that measure students’ performance of the elements of the 21st century education, including non-linear, ongoing, classroom-based assessments for the improvement of teaching and learning. Development of students’ 21st Century Skills is facilitated by 21st Century Teaching. Such teaching emphasizes interdisciplinary thinking; active problem-solving; reflective evidence-based reasoning; theory-practice connections; ongoing feedback and assessment; flexible roles in collaborative settings; students’ autonomy tempered with inter-dependence and teamwork; diverse perspectives; reciprocity and social responsibility. However, educators’ processes for planning, implementing and assessing teaching online that align with 21st Century skills is a subject understudied in the literature. Research on distance education via the Internet has traditionally focused on student attitudes, student satisfaction and the design of online materials and interfaces rather than the pedagogical intentions of instructors or their actual processes of implementation (Menchaca, 2006; Sloan Consortia, 2005). Earlier studies of teaching online have tended to focus on content delivery and traditional lecture-oriented methods rather than study in-depth the kinds of interactive and student-centered methods needed to foster students’ learning in Web-based settings today (Dede, 1999; Jonassen, 2000). Therefore, to develop a more comprehensive and timely model of teaching online (i.e., planning, implementation and assessment) relevant to the needs of today’s institutions, we used principles of Activity Theory to guide data collection and analysis in both hybrid and fully online settings. Activity theory (Engeström, 1993,1999; Nardi,1996) provides a particularly powerful lens through which to study the complex social practices that arise within online course contexts. This is mainly due to its focus on the relationships that exist among those involved in an activity. This is enhanced by the historical development of the activity, the issues surrounding the development and use of tools to support the activity, and the explicit and tacit perspectives and practices of participants in the context in which the activity is carried out (Mwanza, 2002). This framework acknowledges that online teaching is a highly situated and social endeavor, involving the interaction of multiple constituencies over time for various aims, and embedded in local culture and resource networks. In such an environment, opportunities to demonstrate thinking and participating, or cognitive and social presence, are important to learning and teaching success (Garrison, Anderson, & Archer, 2001; Bransford, Brown & Cocking, 1999). Feedback, especially from multiple perspectives, is essential for guiding the development of the instructor’s competency (Eraut, 1994, p. 97; Argyris & Schon, 1974). In addition, students’ participation in the design and critique of educational activities becomes critical to the learning of the course-related community (e.g., administrators, educational technologists, instructional designers) as well as to students’ own learning (Bransford, Brown & Cocking, 1999; Engeström, 1991; Lee, Chan, & Aalst, 2006). Finally, while many studies have looked at teaching online from a single perspective (i.e., student’s) or investigated the use of a particular tool, trends in education, including the increasing diversity of students, ubiquitous access to the Internet, and unbundling of instructors’ roles with a greater number of temporary and part-time instructional staff, have each contributed to the need for deeper understanding of the online teaching process, especially as the coordination of various constituencies involved in instructional design and implementation (Paulson, 2002; Murnane & Levy, 2006). Therefore, this study also sought to capture the voices of students and departmental or interdepartmental collaborators who helped shape, implement or assess the online teaching components of the course. Method In the fall of 2004, the Office of Information Technology at the University of Minnesota funded a multi-year research project designed to identify practices that illuminate how course design teams used WebCT to develop engaging e-learning courses. The goals were to: * investigate the design and implementation activities that characterized e-Learning, * develop grounded theory of best practices regarding collaborative knowledge building among course participants, and * inform course design practices by sharing the insights gained. To investigate (1) how do course teams go about planning to teach online?, (2) how do participants use online tools to achieve their teaching and student learning goals?, and (3) how do course teams assess learning and teaching when using online course elements, data were collected regarding online teaching practices (i.e., the Activity) in five courses. In this paper, we profile one case in the series to illuminate the research method before turning to a synthesis of findings from three of the five cases. To identify these courses and recruit this sample, a letter was distributed via email or face-to-face to leaders and consultants who are in regular contact with a wide range of instructors and staff. Promising subjects were then screened for several criteria via an initial survey administered over the phone. Participants were selected from a variety of disciplines, representing different course types (e.g., hybrid and fully online) who employed technology-enhanced strategies to address particular problems in teaching (e.g., student engagement, community building, etc). Targeting a range of content areas and course types allowed a more complete and nuanced picture of the dimensions of online teaching on campus. Participants were also selected for their degree of technology-using experience, including: experience using online teaching tools over several semesters; use of the Internet daily (i.e. for gathering information, communicating, posting information to the Web, providing instruction for their students, etc.); and use of course management tools for more than just delivering documents but for other potentially innovative teaching strategies (e.g., fostering student-student interactivity or active learning projects). The identification of experienced technology-using instructors in technology-accessible settings was essential because educators who have difficulty overcoming lack of access to adequate technology, what Ertmer calls a “first order barrier” (1999), and novice users (Becker & Ravitz, 1999) are least likely to demonstrate inspirational uses of the technology which others might emulate (Anderson & Ronnkvist, 1998). Finally, candidates selected for this study were those who reported a “learner-centered” and “community-centered” orientation to teaching most relevant for today’s students and defined as: 1) Course organization that allowed for teacher-student and student-student interaction, 2) Orientation toward using various methods, 3) Instructor expressed positive attitude toward students and subject matter, 4) Instructor expressed interest in providing students with opportunities to be self-directed, tap prior knowledge, question, connect to reality, consider multiple perspectives, reflect, self assess, or collaborate, and 5) Instructor believed in his or her capacity for teaching and learning and perceived WebCT as a potentially valuable resource (Bransford, Brown & Cocking, 1999; Becker & Ravitz, 1999; Jonassen, 2000; Wiske, 1998). Data were collected from three complementary sources: self-report data as obtained through semi-structured interviews; observations of teaching online as obtained through Web logs; and students’ perceptions as obtained through focus groups. Activity-oriented Design Methods (Mwanza, 2002) were used to study the complex activities surrounding online instruction. For instance, an hour-long semi-structured interview format adapted from Mwanza (2002) was used to focus interviews around key components of the course activity but still allow inductive idea generation and design flexibility to probe for participants’ understandings (Seidman, 1998). Two separate interviews were conducted with instructors (and their assistants where applicable): one at the beginning of the course to investigate their initial conceptions and designs and a second, after the course was over, to gather details of their experience and reflection on meaning with respect to planning, implementation and assessment (Seidman, 1998, p. 11-12). Interview questions targeted the eight components of an activity system (objectives, actors, values or espoused principles, tools or practices used, division of labor, community, and desired outcomes). In addition, Web logs, or records of course participants’ learning and interactions online, were collected, including threaded discussion messages and examples of students’ work. These helped to confirm or disconfirm what instructors reported in interviews and served to contextualize the teaching practices they described. Finally, hour-long focus groups with students were conducted mid-way through the course to test emerging themes and understand the students’ perspective after they had a chance to experience the course and become proficient in using the online tools. These data sources served to triangulate findings during analysis. Using Activity-Oriented Design Methods (AODM) (Mwanza, 2002) to interpret, model, and describe the online teaching practices of course teams and course participants involved a six-stage process, where each stage leads to increasingly focused and refined observations. The six stages are as follows: Stage 1. Interpret the situation being examined in terms of Activity Theory Stage 2. Model the situation being examined Stage 3. Decompose the situation Stage 4. Generate research questions Stage 5. Conduct a detailed investigation Stage 6. Interpret and communicate findings AODM provides four methodological tools to support data-gathering, analysis, and representing insights during the six stages of analysis. The following descriptions are provided to briefly orient the reader to the tools that guide the six stages of analysis. Table 1 The Eight-Step-Model Identify the: - Questions to Ask Step 1 Activity of interest What sort of activity am I interested in? Step 2 Object - ive Why is the activity taking place? Step 3 Subjects Who is involved in carrying out the activity? Step 4 Tools By what means are the subjects performing this activity? Step 5 Rules and Regulations Are there cultural norms, rules or regulations governing the performance of the activity? Step 6 Division of labour Who is responsible for what and how are roles organized? Step 7 Community What is the environment in which this activity is carried out? Step 8 Outcome What is the desired Outcome from carrying out this activity? The first tool is the eight-step-model, which is used to facilitate interpreting the data in terms of activity theory concepts; it provides a series of eight questions, each of which corresponds to a specific activity theory concept (see Table 1). The second tool is an activity notation system, which facilitates decomposition of the main activity system into sub-activities. The activity notation aids in analyzing the interrelationships and tensions (i.e., contradictions) among the actors/doers, mediating artifacts, and shared object (See first column in Table 2 for an illustration). The third tool is generating sub-activity-oriented research questions, which is a technique to formulate research questions that are used to conduct a detailed investigation of an activity system. The technique is used to focus and support further data gathering and analysis (See second column in Table 2). The fourth tool, mapping operational processes, is a technique which integrates information produced using tools two, three and four; it makes the links between them more explicit for the researcher and represents the combined insights gained during each stage of analysis (see the second row in Table 2 for an example). In particular, operational mapping is used to identify contradictions for each sub-activity and, also, identify instances in which a contradiction is related to multiple sub-activities. Table 2 A format used for operational mapping to represent and communicate findings Sub-Activity Triangle focused on Questions generated from case study Identified area of contradiction Subject-Tool-Object How do the guidelines for small-group threaded discussions (Tool) affect a student’s (Subject) development of practical and conceptual knowledge of statistical research methods (Object)? Some discussion questions and tasks lead to a single correct answer. In applying stages one and two of AODM, the analyst interprets and then models the situation in terms of activity theory. In Stage 1, the eight-step model shown above is used to facilitate the process of interpreting the data in terms of the eight activity theory concepts (see Table 1, middle column). Questions corresponding to each concept (e.g., Why is the activity taking place?) help to scaffold the researcher’s interpretive process. For example, to complete the first step in table 1, identifying the activity of interest, we translated our research agenda into activity theoretical terms (i.e., “we are interested in course design activity and teaching and learning activity”). Although these steps may seem obvious, they may be especially helpful for novice activity theorists trying to conceptualize and characterize the data as well as for experienced researchers trying to make explicit a process that has become intuitive. Figure 1. Expanded Activity Theory Triangle Tools Subjects Object-ive Transformation Process Outcome Rules & Conventions Community Division of Labor. Results To date, data has been coded along the following dimensions: instructors’ and students’ objectives, rules/values, tools or practices, division of labor, community influences, desired outcomes, and essential actions, in accordance with the activity theory framework. Below, I will draw from one of the five cases studied to suggest planning, implementation, assessment and support insights. These preliminary findings are organized according to the following general categories: (1) principles for learning and teaching that guided design and implementation; (2) guidelines and structures that facilitated constructive and content-focused instructor-student and/or student-student interaction online;(3) instructor roles vs. students’ roles vs. community roles; (4) embedded and implemented feedback loops; (5) external collaborations and supports that assisted instructors in enacting their pedagogical intentions online; and (6) instructor’s strategies for managing time and workload and disseminating what works in the online teaching process. Discussion of these themes with roundtable participants will include how they support the development of 21st Century skills. Technology Integration Online Course Overview. In this case of an educational technology integration course, the course team included the instructor, a part-time teaching assistant (TA) and two instructional technology support (ITS) staff. The course was part of a larger grant-funded certificate program and therefore, the TA assisted not only in creating materials for this course but also for other courses in the program. Instructional support staff similarly assisted in performing the up-front technical work of creating this and the other online course sites using WebCt/Vista. The instructor, Julie, was a tenure-track faculty member in a college of education who taught courses on learning technologies to undergraduate and graduate students. Julie had been teaching at the university level for ten years and had over twelve years of experience in designing, developing and teaching in technology-enhanced settings. At the time of this study, she had been teaching online courses for three years and served as a co-principal investigator and project director for the certificate program. Julie, unlike many instructors, did not share teaching responsibilities with a graduate teaching assistant or instructional designer; however, she sought input from the graduate students, instructional technologists and others associated with the grant to develop the course materials and online environment. She also shared ideas and resources with the co-principal investigator, who taught other courses in the certificate program. Ultimately, she coordinated the work of the various course team members to ensure that the technology integration course was well organized, sequenced, and navigable prior to implementation. Eleven students enrolled in the technology integration course. These included masters and doctoral students and students completing a continuing education program. Participants came primarily from educational organizations where they were district or school level administrators, curriculum specialists or teachers. The educational technology integration course was a continuation of a face-to-face course on technology integration that participants completed as a cohort during the previous summer. Course Design. Figure 2 represents the activity system for the technology integration course. Figure 2. Model of Technology Integration Activity Artifacts, Tools, & Practices * Discussion Forum Guidelines (DFG) for constructing online threaded discussion * Organizing theme-based activities (3) * Focusing questions * Replacement/Amplification/Transformation (RAT) framework * Weekly checklist of to-dos Technological * Email * Threaded discussion * CMS Learning modules Subjects * Individual student * Group member Object-ive Develop technology integration knowledge -----> Outcome * Theoretical understanding * Ability to apply knowledge to transform teaching/learning in organizations Values, Rules, & Conventions * Students need to apply technology integration frameworks by investigating their organizations; site-based learning (core value) * Cohort/Community of learners model * Expectations for students’ original postings distributed over a two-week period & grounded in experience * National standards for Technology Integration Community * Instructor * TA * Instructional technologists * Grant staff * Co-principal investigator Division of Labor * Clear role delineation among course team members * Some coordination with co-director * Instructor as course facilitator and discussion moderator * Student roles and responsibilities (e.g., peer reviewer, group member) The object-ive of the course activity was to develop students’ technology integration knowledge. As depicted in Figure 2, the course design emphasized students’ developing their ability to apply theoretical frameworks for technology integration to transform the settings in which they worked. The course emphasized site-based learning where students were to gather feedback on their emerging practices from colleagues in their organizations. The course also emphasized a community of learners’ model, intending for students to proceed through the course (and certificate program) as a cohort. The intention was for students to continuously share their ideas, reflect on experiences applying new practices within their respective institutions, and create a common repository for their work in the online space. Course designers hoped that the online structure would help forge shared values, goals, and practices that might sustain the community beyond their participation in the online program. Desired outcomes for this course were that students understand assessment frameworks for clarifying characteristics of technology integration; apply and share these frameworks with colleagues; identify, analyze, and assess technology-supported lessons in their organizations, and consider the contextual and personal factors that impact the diffusion of technology integration in an educational institution. To achieve these outcomes, the course was organized around three overarching questions or “themes” which included a series of readings, a site-based activity, and a threaded discussion around the following questions: 1. What about technology integration? 2. What role do technology integration frameworks play in understanding effective technology integration? 3. How do you facilitate the diffusion of technology integration in your classroom, building, or district? For instance, to examine the second theme (i.e., role of technology integration frameworks in understanding effective technology integration), students were asked to read different frameworks for technology integration and then, apply one framework to critique technology-integrated lesson plans in their school settings. Students selected the lesson plans to critique and applied the Replacement/Amplification/Transformation (RAT) framework (Hughes, 2006), to categorize and evaluate them. Students were to include a written reflection about why they had characterized their lesson plans as such and post their work in the online course space for public comment. Over a two-week period, students were also required to comment on two of their peers’ categorizations and revisit the course themes and central questions in the threaded discussion forum. Students were asked to continue revisiting these same questions throughout the course and discuss them as a group. The instructor explained what she felt as the value of collaborative dialogue to students’ development of technology integration knowledge: Generally, to build knowledge they [students] have to connect new knowledge with experiences and beliefs they already have…So ways I can do that. One is I can give them knowledge…through…a presentation…or a reading…that on it’s own usually isn’t sufficient for everyone to build deep knowledge. So then you move into discussion phases, where they can ask questions; they can talk out ideas; they can read and interact with their peers who can lend interpretations to them, clarify things or provide even more connections than they might have seen. So generally that is how the knowledge building process goes [emphasis added]. To facilitate this group knowledge-building process through focused discussion around three overarching themes, Julie developed Discussion Forum Guidelines (Tool). These were subject line headings she asked students to use when responding to a discussion thread to help guide the content of the discussion and get them to reflect on the type of contribution they were making to the group (e.g., reflection, expansive question, substantive insight, collegial challenge, personal realization/transformation, connection, etc.). For example, a student who titled his discussion comment “Personal Realization/Transformation” would post a comment explaining shifts in his perspective on the topic. A student who titled his post “Connection” would direct readers to another post with similar ideas or make a comment that connected another person’s stated ideas while explicitly acknowledging the connection. As evidenced in interviews and focus groups, the instructor and students felt that the course discussions were “really, really rich” and that the use of “the little subject category things…makes [students] think” about the knowledge-building process and their contribution to it. Outcomes. Table 3 provides a summary of findings from a detailed analysis of threaded discussions during a themed learning module in the course. The information in the first column indicates the sub-activity components; the second column indicates the targeted areas of inquiry; and the third column lists contradictions that became apparent in looking at the influence of various tools, rules, and roles on the cohort’s development of technology integration knowledge. Table 3. Operational mapping of technology integration course sub-activities Sub-Activity Triangle focused on Questions generated from case study Identified area of contradiction Subject-Tool-Object How do the guidelines for the discussion forum (DFG) (Tool) affect students’ (Subject) development of technology integration knowledge (Object)? Some students experienced these apriori categories as a barrier to expressing their ideas Subject-Rules-Object How does the rule that students are to contribute original experience-based discussion posts over an extended period (Rule) affect a student’s (Subject) developing technology integration knowledge (Object)? Students who required more workload management and/or strategies for handling the overwhelming # of posts did not contribute regularly or withdrew Subject-Div. Labor-Object How does the instructor’s role in moderating the themed discussion forum (Div of Labor) affect student’s (Subject) development of technology integration knowledge (Object)? Instructor was inconsistent in ability to moderate; students’ reading/responding to moderator’s prompts was not assured Community-Tool-Object How does the theme-based structure (Tool) support the course team’s (Community) effort to encourage development of technology integration knowledge (Object)? None Community-Rules-Object How does the course team (Community) emphasizing a cohort /community of learners model (Rule) support development of technology integration knowledge (Object)? Not all students moved along in a cohort; some fell behind, needing to retake the course & resulting in students less likely to participate Community-Div. Labor-Object How does delegating responsibility to students for categorizing their knowledge contributions (Div of Labor) support the course team’s (Community) effort to develop technology integration knowledge (Object)? Students were not always able to reach the levels of insight or connection to reach desired outcomes. Insights. In the technology integration course, the course team consisting of the instructor, teaching assistant and instructional support staff coordinated efforts with the project co-director and cooperated to ensure that specific tasks were done to create the course and fulfill grant and certificate program obligations. Team members provided information to each other in the development phase, had clearly defined roles, and made decisions largely independently of one another. Analysis of the technology integration course archives and transcripts suggested that students in many ways attained the desired outcomes. For instance, discussion postings demonstrated that students were indeed incorporating Discussion Forum Guidelines to categorize their contribution to the discussion. Also, students commented in focus groups that this tool helped the group develop a common system for organizing and recognizing the contribution of its members. Students believed they were learning from the shared experiences of the cohort, developing professional networks, and deepening in their understanding of common issues and barriers for diffusing technology integration in their organizations and how to apply strategies to resolve them, which was an explicit course goal. However, both individuals and the group as a whole demonstrated difficulty in surfacing and synthesizing key insights. In several instances, the instructor, in taking on the role of discussion moderator, helped to scaffold individual thinking and bring implied insights to the group’s attention. For example, in considering the question: what assumptions about knowledge and learning do technology and/or technology integration make?, students discussed their experiences in using various technologies, trying to integrate technology, reflecting on readings, etc. Their responses indicated tacit assumptions underlying technology or technology integration, which the instructor, as moderator helped them make more explicit: Table 4. Discussion Forum Excerpts Subject: Insight Author: Julie So, Kathy, it seems like you are saying that any assumptions about knowledge and learning that technology makes is in the hands of the users and instructional designers (teachers) who define how technology is used in the classroom. The technology, itself, does not have inherent assumptions regarding learning/knowledge. Sound right? Subject: Re:Insight Author: Kathy I am going to need time to dwell and reflect on this a little more. I will get back to you with an answer soon… Subject: Expansive Question Author: Julie Mike - say more about the SmartBoard. Do you think it, on its own, makes assumptions about learning and knowledge? … Subject: Affirmation Author: Julie Okay - great you have identified some assumptions built into technology or technology use (generally): one assumption is - increased motivation for learning. Another assumption is - increased retention (of knowledge learned through using it, I assume [emphasis added]. These are global assumptions- you can also consider specific technologies and what they might be assuming or people are assuming about them. What else? However, in analyzing contradictions within and between the sub-activities, tensions surfaced between the course design, which emphasized a cohort model, and the instructor’s ability as the sole facilitator to assist students in cohering, progressing at the same pace, and engaging their peers in substantive dialogue. For instance, several students experienced challenges to managing the workload for this course and staying connected to other participants. A few students were repeating the course and therefore, did not have the same connection to their classmates as did those from the same cohort which had convened during the face-to-face summer sessions. These students tended to fall behind and contribute less consistently and substantively to the threaded discussions. Moreover, the instructor was not able to monitor and moderate each of the three online discussions with the same intensity. As her workload fluctuated during the semester, so did the frequency and depth of her comments to individual students and synthesis of the group’s insights. Unlike face-to-face courses which are scheduled and unavoidable, online instructors are left to their own devices in protecting regular “teaching time.” Of the total number of postings, the instructor contributed about 10% of the total. She was conflicted about her ability to keep up with the course and mentioned that distributing responsibility to a teaching assistant might help improve the quality of feedback she could provide to keep all students engaged. In addition, students seemed less inclined to reach higher levels of thinking to build knowledge of technology integration over time without explicit scaffolding from the instructor. For instance, the most frequent types of responses posted in the discussion forum included: Affirmations, Reiterations, Expansive questions, and Reflections versus Substantive insights, Collegial challenges, Summary, Connection or some of the other synthesizing and evaluative commentary that might advance group cognition (Hughes, 2005). Finally, although the majority of students and the instructor felt that desired outcomes were achieved, contradictions we identified indicated that the course suffered from inconsistent pacing and performance on the part of some students and the instructor; lack of clear expectations for student-instructor interaction (e.g., students did not recognize the need to respond to instructor’s questions in the threaded discussion although the instructor expected their response); lack of modeling and under-emphasis of higher order discussion responses, and perception on the part of some students that the Discussion Forum Guidelines were barriers to self-expression. Such insights provide opportunity for further refinements and reflection. Insights Related to Planning, Implementation, Assessment & Supports Much has been written about the importance of fostering students’ interaction, collaboration and social presence to improve active learning and retention in online courses (Hiltz, et.al., 2000; Haythornthwaite, Kazmer, & Robins, 2000); however, few studies have actually examined the intentions of course teams as they are manifested in course designs and as they relate to students’ demonstration of online interaction and collaboration. In each of the cases we examined, an intention of the course team was to create an environment that facilitated students’ ability to engage in substantive collaborative dialogue with peers, where emerging knowledge and reflections on applying newly learned concepts in authentic settings could be shared, critiqued, and co-evolved. Our goal, therefore, was to identify and describe practices that the course team used to facilitate (or inhibit) such content-focused collaborative dialogue, or collaborative knowledge-building, once students were actually using the course site. Course Team Collaboration. Increasingly, institutions moving to e-Learning formats are recognizing the importance of an interdisciplinary, interdepartmental team approach to online course design and development. Although we might have expected to discover a common model for collaboration across the course teams we studied, in reality, the three teams functioned somewhat differently. To help us understand and characterize the kinds of collaboration we were seeing, we turned to Frey, Lohmeier, Lee & Tollefson’s (2006) review of stage models of collaboration (Hogue, 1993; Bailey & Koney, 2000; and Gajda, 2004) and Gajda’s (2006) model of strategic alliances. Synthesizing prior studies, Frey et. al. (2006) developed a levels of collaboration survey, which has been used to measure collaborative practices along a continuum. The survey defines five levels of collaboration and their characteristics, ranging from no interaction (level 0) to loose collaboration or networking (level 1), cooperation (level 2), coordination (level 3), coalition (level 4), and collaboration (level 5). Levels are distinguished by the degree to which ideas and resources are exchanged, communication is frequent and prioritized, and decision-making is shared or involves consensus. Similarly, Gajda’s strategic alliance formative assessment rubric (2006) distinguishes between five levels of integration among team members (i.e., networking, cooperating, partnering, merging, and unifying) which are characterized according to several criterion (purpose, strategies/tasks, leadership/decision-making, interpersonal/communication). Teams with the highest degree of integration, what Gajda terms unification, relinquish autonomy to acquire a single purpose/structure; engage in permanent reorganization of tasks; create unified, central leadership; and communicate clearly, frequently, formally and informally. Synthesizing these models, we characterized the collaborative practices of the course teams along a continuum (see Table 7) with the technology integration course exhibiting the lowest level of collaboration and the statistics course exhibiting the highest level. Table 5. Characterization of Levels of Course Team’s Collaboration Networking Cooperation Coordination Coalition/ Partnering Collaboration/ Merging Coadunation/ Unifying Technology Integration Adult Education Statistics Although a full discussion of these differences is outside the scope of this paper, it is interesting to note that the two online courses which seemed to demonstrate the most success in carrying out their objectives were the technology integration course and the statistics course, two courses at opposite ends of the collaboration continuum. Therefore, rather than recommend one model to which course teams should aspire, it is perhaps more useful to emphasize that in all cases, design and/or implementation involved access to expertise beyond that of the course instructor; specification of shared goals and priorities among team members; division of labor; clear communication; and leadership but that the degree of shared decision-making, consensus building, and overlapping practices varied. These findings also suggest that the degree to which students should be considered as part of the course team and given responsibility for facilitating course objectives are issues that ought to be revisited throughout design and implementation phases. For instance, in the technology integration course, students might have benefited further if the instructor, when recognizing her own limitations, had modeled how to moderate online discussions and subsequently, delegated responsibility for such moderation to a few capable students. In contrast, students in the adult education course might have benefited if the instructor and teaching assistant had adjusted their nonintervention and assignment elaboration strategies to reduce the time and energy students spent figuring out procedures and expectations. If they had taken a more active role initially in helping students to be independent learners, students may have demonstrated more substantial critique and reflection. Course Structure and Organization. Recently, researchers have suggested the importance of considering media multiplicity and distributed cognition (Dede, 2004) when designing e-Learning courses that leverage Internet capabilities and support how people learn. These cases suggest that e-Learning environments which are multi-functional, media and resource-rich, thematically organized and provide opportunities for both self-directed learning and extended collaborative dialogue around authentic practice opportunities seem those most likely to prove successful in fostering course objectives. For instance, online features which seemed to facilitate collaborative knowledge building in these courses included a repository for students to share their work, guided threaded discussion forums, a public calendar, and synchronous meeting capabilities. Features which seemed to facilitate students’ independence and self-pacing included thematic units, or learning modules, online tutorials, assignment tools which helped students manage their time on task, online grade books through which students could monitor their progress, and periodic feedback via email to individuals about their individual and group performance. All courses teams encountered the challenge of balancing students’ independent work while fostering a sense of community. Our focus groups with students suggest that uncovering students’ perceptions of online learning (e.g., as self-paced or cohort-based) might be an important step in helping course teams understand and clarify students’ expectations at the outset. Ideally, online course environments should support and showcase both individual and collective intelligence. Scaffolds for Collaborative Knowledge-building Online. These cases suggest several strategies for fostering collaborative knowledge building online. First, course teams should provide clear guidelines, specifying assignment procedures and expectations for participation in small group activities at the beginning of the course. As one instructor put it, “You have to be super organized at the outset” in creating an online course structure and materials. Because students do not have ready access to on-demand help and are just beginning to develop their sense of connection to others in the course, minimizing their frustrations and confusion in the first several weeks is essential to retaining and engaging them. Many of the students we interviewed suggested that online courses should build-in an orientation to course site resources, including an explanation of what purpose different tools serve, who to contact with technical questions, and how to actually use the tools to fulfill course objectives. Second, courses designs should enable students and instructors to participate in substantial, content-focused interactions which draw on each individual’s knowledge and experience. Course teams should ensure that the instructor has clearly communicated expectations for what a significant contribution to the group’s experience entails. In the case of the technology integration course, the definition of a significant knowledge-generating contribution was explained and modeled at the beginning of the course and scaffolded over time as the instructor pushed individuals to deepen their thinking and gave them individualized feedback on the quality of their discussion posts. Third, course teams should carefully consider their division of labor and distribution of expertise. For instance, although there is no required “teaching time” online as there is in scheduled face-to-face classes, instructors, in consultation with the course team, should ideally plan times to be present within the online space and consider how this planned teaching time online might be used to the greatest advantage and efficiency. In all three cases, there was no consideration of time allocation or how to leverage the expertise of the different team members so that each member’s time was used most effectively. Instructors did not have clear strategies for how, how well, or how often they interacted with students online or which kinds of interactions should involve which people. For instance, instructors often fielded questions from students that were both technical and content-related rather than establishing clear expectations for whom students should contact in different situations. Fielding such a range of questions took time and distracted instructors from providing their content-related expertise and feedback on the quality of students’ ideas. In addition, instructors did not have a strategy for tracking which students they had interacted with and therefore, it was possible for some students to reach the end of the course with very little mentoring. In the case of the technology integration course, it was clear that the when the instructor brought her expertise about technology integration to bear on the group’s discussion, she helped students make key insights explicit and pushed their thinking to elevate the quality of the overall dialogue. Fourth, to foster collaborative knowledge building online, students should be supported in assuming positions of authority in the course, such as moderating discussions, peer coaching, and contributing resources to the collective. Moderation, for instance, is essential to evolving group consensus around key concepts or surfacing important insights. Instructors we interviewed tended, in many cases, to moderate intensively early in the course and trail off as the course progressed without putting additional supports in place. Demonstrating how to moderate effectively and then, redistributing authority to students with the instructor in a coaching role, would improve the coherence and consistency of students’ experiences and help develop a knowledge-building community over time. Challenging, open-ended, meaningful practice opportunities. These cases suggest that course design teams should focus on building e-Learning around challenging, open-ended tasks that resemble the kinds of problem-solving and collaboration required of professionals in the field. For instance, designing a group research project and applying statistical procedures or critically evaluating the quality of actual technology integration lesson plans and discussing these critiques with colleagues are just two examples of course activities students engaged in that mimicked actual professional practices. Finally, the team approach to instructional design and implementation is a relatively new phenomenon in K-12 and higher education. This report provides course design teams with a suite of tools they can use to consider how to think about the design process and the elements to which they should attend. The AODM method also provides a means of conducting both macro- and micro-level analysis of course activities that may be useful to guide iterative refinements. Finally, the themes presented here are not often discussed in the e-Learning literature. They are offered, therefore, to stimulate and expand the conversation. Roundtable Discussion How do these kinds of planning, implementation, assessment and support practices relate to 21st Century Skills for students and 21st Century Skills for teachers? * Competency in core subjects; * Learning skills (e.g., the capacity for flexible decision-making; the ability to analyze, manage, create and communicate information in a variety of forms and contexts; the ability to demonstrate teamwork and leadership, monitor one’s own understanding, and practice personal and social responsibility); * An orientation toward lifelong learning, or the capacity to use information and communication technologies to develop one’s learning day-to-day and across the lifespan; * Situational skills, or the ability to connect one’s actual experiences to governmental, economic, and civic trends and stay up on current events; * High quality assessments that measure students’ performance of the elements of the 21st century education, including non-linear, ongoing, classroom-based assessments for the improvement of teaching and learning. References Anderson, R. E. & Ronnkvist, A. (1998). The presence of computers in American schools. Teaching, learning and computing survey, Report #2. Center for Research on Information Technology and Organizations. University of Minnesota & University of California, Irvine. Available [Online] at http://www.crito.uci.edu/tlc/. Argyris, C. & Schon, D.A. (1974). Theory in practice: Increasing professional effectiveness. San Fransisco: Jossey-Bass. Becker, H. J. (1999). Internet use by teachers: Conditions of professional use and teacher-directed student use. Teaching, learning, and computing: 1998 national survey. Report #1. Irvine, CA: Center for Research on Information Technology and Organizations, University of California, Irvine. Available [Online] at: http://www.crito.uci.edu/ Becker, H.J. & Ravitz, J. (1999). The influence of computer and Internet use on teachers’ pedagogical practices and perception. Journal of Research on Computing in Education 31 (4), 356-84. Bransford, J.D., Brown, A.L. & Cocking, R.R. (1999). How people learn: Brain, mind, experience and school. Washington, D.C.: Committee on Developments in the Science of Learning. Borko, H., & Putnam, R. T. (1996). Learning to teach. In D. C. Berliner & R. C. Calfee (Eds.), The handbook of educational psychology (pp. 673-708). New York: Macmillan Publishing. Dede, C. (1999). The role of emerging technologies for knowledge mobilization, dissemination, and use in education. Paper commissioned by the Office of Educational Research and Improvement, U.S. Department of Education. Available [Online] at http://www.virtual.gmu.edu/EDIT895/knowlmob.htm. Dede, C., Korte, S., Nelson, R., Valdez, G., & Ward, D.J., (September 2005). Transforming learning for the 21st century: An economic imperative. Naperville, IL: Learning Point Associates. Engeström, Y. (1993). Developmental studies of work as a testbench of activity theory: The case of primary care medical practice. In S. Chaiklin and J. Lave (Eds.), Understanding practice (pp. 64-103). Cambridge: Cambridge University Press. Engeström, Y. (1999a). Activity theory and individual and social transformation. In Y. Engeström, R. Miettinen, & R. Punamaki (Eds.), Perspectives on activity theory (pp. 19-38). Cambridge: Cambridge University Press. Eraut, M. (1994). Developing professional knowledge and competence. London: RutledgeFalmer. Ertmer, P. (1999). Addressing first-and second- order barriers to change: Strategies for technology integration. Educational Technology Research and Development 47(4), 47-61. Jonassen, D.H. (2000). Computers in the classroom: Mindtools for critical thinking. Englewood Cliffs, NJ: Prentice-Hall. Lee, E.Y. C., Chan, C. K. K., & Aalst, J. V. (2006). Student assessing their own collaborative knowledge building. International Journal of Computer-supported Collaborative Learning, 1 (1), 57-87. Levy, F. & Murnane, R. J. (2004). The new division of labor: How computers are creating the next job market. Princeton, NJ: Princeton University Press. Mwanza, D. (2002). Conceptualising work activity for CAL systems design. Journal of Computer Assisted Learning, 18 (1), 84-92. Nardi, B.A. (1996). Context and consciousness: Activity theory and human- computer interaction. Cambridge, MA: The MIT Press. National Center for Educational Statistics. (2005). Internet access in U.S. public schools and classrooms: 1994-2003. NCES Number: 2005015. Retrieved from http://nces.ed.gov/Surveys/frss/publications/ on July 31, 2005. Oblinger, D. (2003). Boomers, gen-xers, and millenials: Understanding the new students. Educause Review (July/August), 37-47. Partnership for 21st Century Skills. 2003. "Learning for the 21st Century." Washington, D.C. Online: www.21stcenturyskills.org. Paulson, K. (2002). Reconfiguring faculty roles for virtual settings. Journal of Higher Education, 73 (1), 123-140. Pew Internet and American Life Project Report (August 2002). The Digital Disconnect: The widening gap between Internet-savvy students and their schools. Retrieved August 1, 2005, from http://www.pewinternet.org/PPF/r/67/report_display.asp Seidman, I. (1998). Interviewing as qualitative research: A guide for researchers in education and social sciences. 2nd edition. New York: Teachers College Press, Columbia University. Wiske, M.S. (1998). What is Teaching for Understanding? (pp. 61-87). In M.S., Wiske, (Ed.). Teaching for Understanding: Linking research with practice. San Francisco: Jossey-Bass. Yin, R.K. (1994). Case study research: Design and methods. (Applied Social Research Methods Series, Vol. 5). 2nd edition. Thousand Oaks, CA: Sage. Zhao, Y., Pugh, K., Sheldon, S., & Byers, J.L. (2002). Conditions for classroom technology innovations. Teachers College Record 104 (3), 482-515. 21st Century Teaching Online & Activity-oriented Design Methods Christine Greenhow (greenhow@umn.edu) – University of Minnesota NECC 07 Discussion-Paper