Monthly Archives: February 2015

ET 505 Week 6 assignment

1. Summarize chapter 4

This week I chose to do an alternative assessment, and made my first infographic.

2. Update on evaluation report

Here is my update on the project…

History

I have used online grading tools for several years with my students.  All students have a sign on, and are able to see individual scores and their average.  The online grading has evolved into a class website that I have used for about three years.  The website that I used last year included content.   Last year, my students told me (in a survey and also during formative assessments-conversations) that what they are looking for in a website is: access to grades, weekly agenda, and assignments. They emphatically told me that they did not use the website to learn content because they were able to get that information from the class activities and other websites that we use.

Current Program

From that feedback, this school year I began using the school provided Aspen X2 by Follett to provide both grades and  a class website that includes weekly agenda (in class & homework), links needed for assignments, links to worksheets and class notes.

Goals

The goals that I developed are:

  1. increase student ownership for their work,
  2. increase student interaction with the course content,
  3. give students the ability to prepare for class,
  4. give students the ability to review the material from prior lessons,
  5. give students access to information when absent from class.

The weekly ET505 assignments & my Evaluation Project

Each week I gain some new insight into where I am going with the evaluation.  Last week looking at the Program Cycle in the text (chapter 3) I found very helpful.  It made me realize that the program (website) was started based on a student survey (a very basic needs assessment) that asked students what they wanted in a class website.  From the results, the current website was created.  Doing last week’s assignment made me realize that  MY goals for website are what I would like my students to do, the needs assessment did not reflect the goals at all.  The needs assessment asked something completely different.  IN retrospect, I should have been trying to find out not only what information they want in a class website, but how could a class website increase their ownership, interaction w/ content, etc.

This week’s reading I found helpful as I try to discern the goals from the activities, and lastly measurement tools.  Another item from chapter four that I found very helpful was turning the goals into questions as a starting point for the evaluation.  For example, the goal is to increase student ownership for their work.  The Evaluation question is; Has the website increased the students’ ownership for their work.  Another example: goal give students access to information when absent from class.  The evaluation question would be: Do students who are absent from class access the information?

In this week’s discussion, a classmate (Nora) suggested that the first two goals need to be more specific.  Rather than the goal being students will take ownership for their work, the goal should further clarified such as what is meant by increased ownership, and what is meant by increased interaction with content.  For me, increased ownership means checking grades to make sure all assignments are complete (rather than my reminding students), checking the website to know what is happening each day in class, etc.  Increased interaction with course content means checking the website to know what we are doing in class, reviewing what we did that day to see if they still have questions, etc.

Current Evaluation Plan

The current plan is to develop a series of questions (Likert 1-5 scale) to survey students.

I can also see how often students sign on to website, so I plan to collect that data as well.

I still need to develop the questions and administer the survey.

Questions

My biggest concern is that the goals are goals that I created based on my vision for the class rather than what the survey revealed.  Nora’s comments helped, but I am unsure if I should change the goals to be more specific, OR do I leave the goals as is, and develop questions to evaluate the specifics of the goals.  The other option is update the goals to reflect the original survey.  I am unsure how to proceed.   I am interested to see if any of the information provided is helping students achieve the goals or will it point me in other directions?  Will I find some other unintended outcomes?   (One I know for sure is that the website is helping to keep me more organized!)  Will it reveal that the website isn’t enough to achieve those goals, but rather it is a combination of website & course design?  Another thing that I am interested in finding out is do the students like having the website, what do they like about it, what don’t they like about it, how can the content and layout be improved to help students.

Still lots of questions as I proceed, but I think it is starting to make a little more sense.

ET505_week6_assignment

ET 505 Week 6 Discussion

This week’s discussion topic.

Use this forum to talk about your Evaluation Report-course project. Start with a brief summary of your project. What are you planning to do? Give us an update. Have the recent weekly assignments and/or these discussions been helpful in honing your thoughts about your project? Or are you stumped as to what to do? Ask for feedback on your proposed project. Get ideas on possible topics and evaluation methodologies from Dr. T and your classmates. The more you participate in this forum, the more you (and your project) will benefit so don’t hold back. This is the perfect time to give and receive peer feedback in a collaborative manner.

(This forum is similar to part of this week’s assignment so you can post about your project here before you submit your assignment, if you prefer. Sort of a “two-fer.”)

(And don’t overlook the “Evaluation Proposal-Response to RFP”  project that’s due during week 8. If you have any questions about it, ask them there. We’ll have a discussion devoted to it next week.)

My initial post follows:

History

I have used online grading tools for several years with my students, and students have been able to access grades.  The online grading has evolved into a class website that I have used for about three years.  The website that I used last year included content.   Last year, my students told me that what they are looking for in a website is access to grades, weekly agenda, and assignments. They emphatically told me that they did not use the website to learn content because they were able to get that information from the class activities and other websites that we use.

Current Program

From that feedback, this school year I began using the school provided Aspen X2 by Follett to provide both grades and  a class website that includes weekly agenda (in class & homework), links needed for assignments, links to worksheets and class notes.

Goals

The goals that I developed are:

  1. increase student ownership for their work,
  2. increase their interaction with the course content,
  3. give students the ability to prepare for class,
  4. review the material from prior lessons,
  5. have access to information when absent from class.

The weekly assignments & my Evaluation Project

Each week I gain some new insight into where I am going with the evaluation.  Last week looking at the Program Cycle in the text (chapter 3) I found very helpful.  It made me realize that the program (website) was started based on a student survey (a very basic needs assessment) that asked students what they wanted in a class website.  From the results, the current website was created.  Doing last week assignment’ made me realize that  MY goals for website are what I would like my students to do, the needs assessment did not reflect the goals at all.  The needs assessment asked something completely different.  IN retrospect, I should have been trying to find out not only what information they want in a class website, but how could a class website increase their ownership, interaction w/ content, etc.

This week’s reading I found helpful as I try to discern the goals from the activities, and lastly measurement tools.  Another item from chapter four that I found very helpful was turning the goals into questions as a starting point for the evaluation.  For example, the goal is to increase student ownership for their work.  The Evaluation question is; Has the website increased the students’ ownership for their work.  Another example: goal give students access to information when absent from class.  The evaluation question would be: Do students who are absent from class access the information?

Curent Evaluation Plan

To survey students after developing a series of questions to evaluate the goals.  I plan to use a scale (1-5) for the questions.

I can also see how often students sign on to website, so I plan to collect that data as well.

I still need to develop the questions and administer the survey.

Questions

My biggest concern is that the goals are goals that I created w/o much planning, and the website I think is doing something different.  I assume that I should still evaluate the website against the original goals.  I am interested to see if any of the information provided is helping students achieve the goals or will it point me in other directions?  Will I find some other unintended outcomes?   (One I know for sure is that the website is helping to keep me more organized!)  Will it reveal that the website isn’t enough to achieve those goals, but rather it is a combination of website & course design?

I guess my question is do I keep forging ahead w/ the original plan?  I think so, but would love your input!

Last question is regarding the “Evaluation Proposal-Response to RFP”  I have just begun to look at this, and am worried that I am behind.  Will the next two weeks be enough time to complete this project?

ET 505 Week 5 assignment

This week’s assignment has two parts:

1. Apply Gap Analysis to the program/project you detailed in exercises in previous assignments. Specifically discuss each part of the three parts of Gap Analysis relative either to the program you have chosen to evaluate or to your actual evaluation of the program. Specify which. That is, discuss in terms of the three “legs” (A, B, C) of Gap Analysis’s three-legged stool. Be specific as to what goes with each leg (i.e., describe each of the three legs specifically as applied to the program/project you intend to evaluate or to the evaluation itself). (Include short description of program being evaluated. 200-400 words or alternative, 4 possible points)

Leg A =
Leg B =
Leg C =

2.  Explain Figure 3.1 Program Cycle (p. 51) in terms of the program that you have chosen to evaluate (not the evaluation itself) as detailed in previous assignments (or a revised version or a new evaluation topic). (150-300 words or alternative, 4 possible points)

Here is my submitted assignment.

1. Apply Gap Analysis to the program/project you detailed in exercises in previous assignments.

Leg A = Where we are

Student surveys from prior years suggest that students want to have the following information on the class website: grades, homework information, links to class notes. Currently the teacher has a class website where students can access grades, see the daily agenda (for in class & homework), obtain class worksheets and notes, and weblinks for specific projects. Students have a secure sign on for the website.

Leg B = Bridge between A&B, how to get there

The website has information that students requested (grades, class notes, homework).  The teacher needs to determine if this information is helping to achieve the larger goals of student responsibility, preparing for class, accessing when absent, etc.  To bridge the gap, the teacher needs to determine ways to ensure that students:  interact with the website,  learn how to access information, have incentive for referring to website.

Leg C = Goals

Website Goals:

  1. increase student ownership for their work,
  2. increase student interaction with the course content,
  3. give students the ability to prepare for class,
  4. give students the ability to review material from prior classes,
  5. give students access to information when absent from class.
  6. give students access to grades

MHS Academic goals: One of the MHS academic core values is “the MHS student will use technology effectively.”

2.  Explain Figure 3.1 Program Cycle (p. 51) in terms of the program that you have chosen to evaluate

Organizational Goals: A core value of MHS is “The MHS student will use technology effectively.” Goals of the website include: increase student ownership for their work, increase student interaction with the course content, give students the ability to both prepare for class and review the material from prior lessons, and access information when absent from class

Needs Assessment: The teacher thought having a class website would help students be more organized, and take more responsibility for learning, but did not do a  needs evaluation prior to creating the website.  As a result, the evaluation will have to look at the goals of the project, and develop the appropriate questions to determine if goals are being met.

Program Planning: From prior years surveys of students regarding courses and course websites, the teacher determined that students most wanted the following in a class website: grades, homework assignments, and access to documents such as notes and weblinks. The website was created with the following content: grades, weekly plan for classwork and homework, class notes, and weblinks.

Implementation and Formative Evaluation:The teacher created the website during the first trimester of the school year.  Based on informal formative student feedback (from the first trimester), the website evolved for the second trimester.  One big adjustment is that the teacher updates the website as plans change to accurately reflect what was done in class each day, so that students have access to up to date content for both classwork and homework.  Other changes included adding more course documents such as worksheets and handouts.

Summative Evaluation:This will be the final report done as part of this class and will determine if the goals are being met.  The evaluator will determine the appropriate survey tools.

et505_week5_assignment

Week 5 Discussion

This week’s discussion centered around the program planning cycle.

planning cycle

Program Cycle from Chapter 3 in B&D text. Graphic courtesy of Megan Jacobson, former 505 student.

Our mission for the week: Discuss the graphic above (from p. 51) and/or Table 3.1 – Program Planning Cycle and Evaluation Activities (pp. 62-3). Pros/cons. Something you disagree with? Where does “monitoring” occur? Anything missing? Bring in any prior experience you might have with such a planning cycle.

Will this model fit with the program/project you discussed in the chapter 1 and the subsequent assignments? Other aspects of chapter three you’d like to discuss? Other planning cycles you’ve used?

Here is my initial response:

At first thought, I didn’t think I had any experience w/ the evaluation process, but realized that the process of NEASC recertification that our high school did the last two years followed the cycle (to some degree).  I was unaware of the cycle, and was only involved in small piece, but I see how that process fit the cycle mow that I am learning about evaluation.

First, as a staff, we developed our core mission statement (organizational goals)

We then broke into groups by, and each group evaluated our school against one set of  the NEASC standards.  From there, all the groups reported their findings and the coordinators created a needs analysis. (needs assessment)

Next the official NEASC visiting committee came to our school, reviewed our report and gathered additional evidence.  Using our formative assessments and their findings, they created a final report w/ both commendations and recommendations for the school.

The process is still ongoing as we look for ways to implement the NEASC suggestions.

I can see how our administrators had an evaluation plan, although at the time I had no idea.  I saw it more as a task list to accomplish.  I see the process overall as beneficial, but I can see how the evaluation piece can be left out of the planning process, which will only hinder the process as it proceeds.

Will this model fit with the program/project you discussed in the chapter 1 and the subsequent assignments? Other aspects of chapter three you’d like to discuss? Other planning cycles you’ve used?

Applying this model to the teacher website (my own) I plan to evaluate will work for portions.

Organizational Goals: One of the core values of MHS is “The MHS student will use technology effectively.” I see evaluating a teacher website as part of this as the teacher can model effective use of technology, and can encourage students to utilize the tools.

Needs Assessment: A formal needs assessment was not done prior to the creation of the website.  The teacher (me) thought having a class website would help students be more organized, and take more responsibility for learning, but did not think of formal needs or the evaluation process when creating the website.

Program Planning: From prior years surveys and websites, the teacher determined that students wanted to access grades, see homework assignments, and access documents such as notes.  From these older surveys, the teacher created a website that included a weekly agenda (classwork & homework), current grades, and links to notes and websites.

Implementation and Formative Evaluation: The teacher created the website during the first trimester of the school year.  Based on student feedback, the website evolved for the second trimester.  One big adjustment is that the teacher updates the website as plans change to reflect accurately what was done in class each day.   Other changes included adding more course documents such as worksheets and handouts.

Summative Evaluation: This will be the final report done as part of this class and will determine if the goals are being met.

The goals are:

1.      increase student ownership for their work,

2.      increase their interaction with the course content,

3.      give students the ability to prepare for class,

4.      review the material from prior lessons,

5.      have access to information when absent from class

Thanks for reading this.

CC

As usual, my peers gave me some excellent feedback, and I feel like I am beginning to get a better idea on how to focus on the objectives for evaluation.

Week 4 Assignment

This week, we had two tasks.
1. Create a “shell” of our evaluation project
2. Create a table based on Exhibit 1.1 Evaluation Design Format

Below is the link.

ET505_week4_Eval_Report_Outline_and_table1_1.docw4x

The discussion for the week was to submit a post that describes our Evaluation Report providing as much insight and detail as possible.  Here is the link to my discussion.

here is

Ed Tech 505 Week 3 assignment

I am beginning to learn about the importance of evaluations.  

“Evaluation enables accountability” (Boulmetis & Dutwin, 2011, p. 38). According to Dr. Thompson, “Three simple, but powerful words. Make this your mantra for the rest of the semester, and everything else will fall into place.”

I am beginning to see that evaluation offers accountability to all stakeholders.  Based on the evaluation, decisions are made regarding the value of a program and how to update, change, or even cancel a program based on the evaluation. I know I have plenty to learn, but each week, I get more information to try to connect the pieces of evaluation into a coherent and meaningful process.  At this point I plan to evaluate my class website against the following objectives: increasing student ownership for their work, increasing their interaction with the course content, giving students the ability to prepare for class by reviewing the material from prior lessons, and accessing information when absent from class.

Week 3 Assignment: Read Chapter 2, participate in discussion, complete the following exercise:

a) Boulmetis & Dutwin use three guiding questions to lead us in resolving the question of why to evaluate:
• What are the benefits and limitations of an evaluation?
• What factors ensure that an evaluation will be successful?
• How might one use evaluation results?

Answer the three questions above as they apply to the following scenario.

Rich Kids, Poor Kids
The Maricopa Community College District (MCCD) partnered with Cashflow Technologies, Inc., to design and deliver a course in financial literacy. The course is based on the products – books, games, and videos – created and sold by the company. Components in the course include training sessions for future course instructors and seminars at which attendees play the Cashflow game. MCCD commissioned an evaluation of the program to verify its value to students and the community. In addition, Cashflow Technologies is exploring the possibility of marketing the educational program to educational institutions throughout the nation so an educational evaluation of the program should lend it credibility. Cashflow Technologies has been producing the materials for three years. The course was offered for the first time during the fall 2011 semester. No training sessions or seminars have been performed.

The program will be evaluated by students in Educational Technology at Boise State University. The evaluators have access to sales information for the products, student performance data from the course, and contact information for the students and instructors. The budget for the evaluation is $6,000.

b) Look back at the assignment you did last week. You described a potential program for your evaluation project. Now having read chapter two, you can answer these questions in reference to that evaluation:

1. Would the program you detailed in Chapter One benefit from an evaluation? If so, how?
2. What are the inherent limitations in the evaluation of the program? Note: We’re talking about potential limitations of the evaluation, not the program being evaluated.
3. How might you use the results to benefit the organization, community, schools, or yourself?

week3_assignment

a

1.The benefits of evaluating the Financial Literacy Program include: determining if the program is achieving its goals of adding value for the students that participate in the course, determining if the training program is effective for the course instructors, and last determining if the program is viable “as is” for marketing to other schools (or does it need to adjusted?).  Another benefit of the evaluations is that the evaluators are not directly involved with the program, and thus do not have a personal interest which increases the objectivity, both real and perceived, of the evaluation. Limitations of the evaluation include: the people doing the evaluating are students and are learning how to evaluate, it is imperative for the credibility of the evaluation that the evaluators are seen as qualified, so hopefully, the student evaluators work closely with their expert instructors as they perform the evaluation.  Another benefit and potential limitation will be determining the impact, in other words,  the long term effects of the program on the students in the program to determine if the course has changed behaviors.

2. The factors that ensure success are making sure that the all involved (students, instructors, and Cashflow Technologies) see that the evaluation is objective.  Ii is crucial that all decision makers be prepared for information that may or may not support their intended plans for the program. All involved should be prepared for alternative suggestions.

3. The evaluation results may corroborate what Cashflow Technologies is trying to determine, that is the program is successful, and has potential to be marketed nationally to other schools.  On the other hand, the evaluation results may determine that the program needs to be improved.  Improvements may be needed in teacher training or course materials and curriculum may need to be improved before CashFlow Technologies can begin to market the program to other schools.

b The program to be evaluated is a teacher’s class website (my own) to determine the effectiveness of website at: increasing student ownership for their work, increasing their interaction with the course content, giving students the ability to prepare for class, review the material from prior lessons, and accessing information when absent from class.

1. The website program would benefit from an evaluation to determine if it is meeting the objectives set forth by the teacher. Another potential benefit is that the evaluation might uncover other ways students are looking to interact with course content, and the website can be improved based on the feedback and in turn might generate new objectives for having a class website (and a future evaluation!). The evaluation will hopefully show which objectives are being met and which are not.  As a consequence, the teacher might change the objectives and the website for the future.

2. Limitations of the evaluation are the credentials and objectivity of the evaluator.  The evaluator is a student, a novice completing her first evaluation, and thus the evaluation could be missing some key components (hopefully not!). Another potential limitation is that the evaluator is evaluating her own program, which could limit her ability to be objective.  The evaluator is interested in learning how to improve the website, and hopefully her willingness to be open to both positive and negative feedback will increase her ability to be objective.  Another potential limitation is receiving high percentage of student feedback on surveys, etc. The more feedback, the better in this evaluator’s opinion!

3. The results of the evaluation will be used first and foremost to improve this teacher’s website for her students.  Secondarily, the evaluation has the potential to help her peers feel more comfortable creating a class website that is based on direct student feedback to help teachers create meaningful websites.  Last, there is the potential to create professional development for teachers interested in creating a website using the school’s X2 Aspen software to create a website.

ET 505 week 2 assignment

Week 2 Assignment

Read Chapter 2, post to discussion board, and complete exercise 1 on p. 22. Caroline Cooney ET505 Sp 2015_week2

1. Over the past several years, I have begun to utilize an on-line grading website in order to create a class website where students can access both grades and content. Over the past few years, I have tried Engrade (easy grading, but website is not easy to navigate and set up) and My Haiku (great website functionality, and fairly easy grading).  However, this year, I decided to try the program that our school supports, Aspen X2.  I tried it because there has been some pushback regarding using “outside” websites with our students.  All students (and parents) have an X2 sign on, and students must access the system to choose classes.  The functionality for both grading and websites are fine once a teacher gets it set up.  Neither the grading, nor the website  functionalities are intuitive for set up.  The website that I have for my students is simple.  It gives the weekly agenda (with links to webpages and documents) including classwork and homework.  Students may also see their individual assignment grades, as well as their current average.  Assignments may be set up so that electronic submission is available. The objectives of the website are to: increase student ownership for their work, increase their interaction with the course content, give students the ability to prepare for class, review the material from prior lessons, have access to information when absent from class.

2. The website should be evaluated to determine if the students are accessing the website, how and why they are accessing the website.  The evaluation should also help determine how to make the website better for students in terms of what kind of information would help to make them more successful in class.

3. I subscribe more to the first definition where evaluation is completed to determine    “whether and to what degree objectives have been or are being achieved”.  I think I tend toward this definition as in my teaching, my curriculum is very objectives based as in “the student will be able to…”  I think that for me, success depends on the success or failure of the objective.  If the objective is not being met, then the second definition comes into play where the evaluation is used to make a decision, such as continue with the program (or curriculum) or not.