Project+3

=__One-to-One Evaluation__=

**Participation by subject matter expert**

 * James Gates
 * James Gates was chosen to participate as the subject matter expert because of his experience with teaching computer applications, software, and hardware. He is familiar with designing instruction to share information about new technologies available to all teachers, specifically web 2.0 tools. He has extensive experience with developing and implementing Moodle into his instruction. He will be able to accurately and effectively assess the instructional process, assessment, and materials.

Materials Provided to Subject Matter Expert

 * 1) Instructional Analysis
 * 2) Performance Objectives
 * 3) Content
 * 4) Assessments (Pre-Test) (Post-Test)

Procedure
The subject matter expert will be provided with the necessary materials to effectively assess the instruction in the following ways.
 * 1) Are the performance objectives met by the instruction?
 * 2) Is there congruence between objectives, instruction, and assessments?
 * 3) Does the instruction appear feasible for use with the available learners, resources, and setting?
 * 4) Are all aspects of the instruction correct/accurate?
 * 5) Are the following items presented in a clear and understandable way?
 * Pre-Test Directions
 * Instructional Delivery
 * Post-Test Directions/expectations
 * 1) Are the following items appropriate for the intended learners?
 * Entry Skills
 * Vocabulary
 * Instructional Sequence
 * Assessments
 * Learner Participation Activities
 * Engagement/Motivation

Participation by Learners from the Target Population

 * There will be three main types of learners expected to be encountered. A one-to-one evaluation will be presented to one participant from each of the following examples:
 * Those very familiar with technology and use it regularly in their classrooms (utilizes technology on a daily/weekly basis)
 * Those somewhat familiar with technology and have used it occasionally in their classrooms (Implements 1-5 presentations per year - in computer lab or monthly/bimonthly basis)
 * Those unfamiliar with technology and its uses in a classroom (Does not use technology)
 * There is no arrangement necessary for the learners to participate. Learners do not need to be grouped to be successful. Sit next to participant.
 * Discuss the process with each individual participant.
 * Administration and review of pre-test
 * Review of content delivery and materials
 * Administer and review Pre-test
 * Can the participant read the directions and complete the pre-test with ease?
 * Does the participant have the prerequisite skills?
 * Take note of any behaviors and/or actions of participant
 * Take note of any changes needed
 * Review of content delivery and materials
 * Participant should have a separate piece of paper to take note of:
 * Difficulties
 * Likes
 * Dislikes
 * Understanding problems
 * Unclear delivery, content, examples, or situation
 * If participant comes across a difficulty attempt a verbal prompt to provide clarity. Take note of unclear portion to make necessary changes.
 * If verbal prompt does not help, provide a more elaborate explanation, possibly changing presentation order. Take note of explanation necessary to provide clarity in order to make necessary changes.
 * Observe the participants behaviors. Does he/she appear bored? engaged? What is causing the distraction? Take note of behaviors to reevaluate information "chunks", is more practice needed to break up instruction?
 * Take note of any changes or additions throughout the one-to-one. Make sure notes are in rough form in order to not distract the learner.

**Outcomes**

 * Instructional analysis is done well according to the technology coach. He did not see any omitted skills.
 * Entry skills are noted as appropriate. He believe majority of learners will have these skills because of prior trainings provided by the school district. Also, all teachers have been provided with a district-issued laptop and are required to use it at school for email access and grade reporting.
 * Objectives and outcomes were appropriate and clear according to the technology coach. He recommended more review and and posting of the objectives throughout the instruction to ensure learners know what is expected.
 * The information presented was appropriate. There were very few extra visuals; showing of the instructor's screen is key to success of instructing a larger group, according to the technology coach. The demonstration type lesson works well with this type of instruction. He recommended use of a "zoom" tool, so users can see more detail of specific areas of the screen.
 * Assessment items and assessment directions were clear for the technology coach. None of the items were complicated, which he thought was good.
 * No wording was found to be unclear or faulty.

=__**Small-Group Evaluation**__=

Learner Characteristics
>>
 * 9 participants of a heterogeneous selection
 * Participants ability level will range:
 * Those very familiar with technology and use it regularly in their classrooms (utilizes technology on a daily/weekly basis)
 * Those somewhat familiar with technology and have used it occasionally in their classrooms (Implements 1-5 presentations per year using computer lab or monthly/bimonthly basis)
 * Those unfamiliar with technology and its uses in a classroom (Does not use technology)
 * Participants ability level will be determined on a self-evaluation by the learners
 * Participants must display the following entry skills
 * Ability to use web browser
 * Ability to complete an online form
 * Have file on computer (Ex. handout for class)
 * Have 5 (five) sample questions for quiz
 * Participants are chosen with a range of comfort levels using technology. This is necessary for the designer to determine if the instruction is effective for all levels and types of participants who may encounter the instruction. All participants will be current teachers, but will have varying levels of exposure to and experience with technology.

Environment

 * District computer lab
 * No grouping necessary
 * One computer (laptop/desktop) per participant
 * Multimedia projector and screen connected to instructor's computer

Time

 * Instruction will occur in sessions; two (2), two hour sessions will take place during after school hours in two successive days.

Materials

 * Multimedia Projector connected to instructor's computer and screen
 * Computer (laptop or desktop) - one per participant
 * Participant/Learner self-assessment checklist (one per participant)

Procedure

 * Inform participants they will be providing feedback on the following instruction and materials presented.
 * Instructor will need to take notes throughout the following instruction of any changes, omissions, and/or additions needed.
 * Administer Pre-Test
 * During this have participants take notes of any unclear directions or test items on a separate piece of paper. Also, note any suggestions they may have during the pre-test
 * Observe participants and take note as they attempt the pre-test. Note any changes to the materials. Evaluator should look for the following pieces in the participants to aide assessment of instructional delivery, instructional materials, and clarity of content.
 * Are the directions easily followed?
 * Are the participants able to complete the form?
 * Administer Pre-Instructional Activities
 * During this have participants take notes on unclear statements or examples. Also, have them note any suggestions or changes they could recommend
 * Observe participants during the pre-instructional activities and take note of their actions. Note any changes needed. Evaluator should look for the following pieces in the participants to aide assessment of instructional delivery, instructional materials, and clarity of content.
 * Does this engage the participants?
 * Can you observe a connection made?
 * Watch facial expressions - any confused faces?
 * State Objectives
 * Have participants take note on a separate piece of paper if any objective is unclear or confusing. Also, have participants note any suggestions, recommendations, or changes they think are needed.
 * Observe participants and take note of behaviors. Note any changes necessary. Evaluator should look for the following pieces in the participants to aide assessment of instructional delivery, instructional materials, and clarity of content.
 * Watch facial expressions - any confused looks?
 * Do the participants grasp what they are supposed to produce?
 * Recall Prerequisite Skills
 * Have participants take note of unclear examples or connections. Have them note any suggestions.
 * Observe participants and take note of behaviors. Note any changes necessary. Evaluator should look for the following pieces in the participants to aide assessment of instructional delivery, instructional materials, and clarity of content.
 * Are they able to connect what they know to what they will be learning?
 * Deliver Content
 * Tell participants their feedback on instructional materials is very valuable in this portion.
 * Have participants pay particular attention to the content delivery. Remind the participants to take note of any unclear directions, unclear examples, unclear vocabulary used, unclear explanations, or any unclear delivery of material
 * Remind participants to continue through the content delivery until the end. Instruction will not cease for discussion, please take note of all suggestions and/or recommendations.
 * Observe the participants throughout the content delivery. Take note of their behaviors and actions. Take note of any changes needed. Evaluator should look for the following pieces in the participants to aide assessment of instructional delivery, instructional materials, and clarity of content.
 * Are participants able to follow directions/demonstration?
 * Do many participants "fall behind" or "get ahead"?
 * Are there particular portions that are more confusing than others?
 * Do participants seemed disengaged or "bored"?
 * Attitudinal Questionnaire
 * Administer the following questionnaire to participants during the session (via email)

media type="custom" key="7690733"


 * Discussion of notes
 * Notes taken by participants during instruction will be shared and discussed
 * Take note of feedback from participants for possible changes
 * Notes taken by instructor during instruction will be shared and discussed
 * Look for participant discussion and/or feedback based on observations. Did they agree or disagree?

Outcomes

 * Participants liked how the steps were sequencial and followed exactly what they saw on their screens
 * Participants noted creating the quiz was initially difficult, but the step-by-step directions helped
 * Participants liked seeing instructors screen with zoom feature
 * Participants stated objectives were clear and understandable, but wanted to see them posted to remind them of the "end task" throughout the instruction
 * Participants noted easiness of pre-test. Directions were simple and straightforward; nothing complicated.
 * Participants liked the "direct" and "authentic" nature of post-test.
 * Participants would have liked more opportunities to practice with assistance if needed.
 * Participants expressed interest in video tutorial

=__**Revisions Implemented**__=
 * Objectives will be posted throughout entire instruction
 * Implementation of "zoom" feature to ensure details are seen by all learners
 * More practice opportunities/times included at the end of each instructional cluster
 * Revisit and make cluster #4 (creating a quiz) more detailed with more explanation
 * Explore video tutorial avenue