I am beginning to learn about the importance of evaluations.
“Evaluation enables accountability” (Boulmetis & Dutwin, 2011, p. 38). According to Dr. Thompson, “Three simple, but powerful words. Make this your mantra for the rest of the semester, and everything else will fall into place.”
I am beginning to see that evaluation offers accountability to all stakeholders. Based on the evaluation, decisions are made regarding the value of a program and how to update, change, or even cancel a program based on the evaluation. I know I have plenty to learn, but each week, I get more information to try to connect the pieces of evaluation into a coherent and meaningful process. At this point I plan to evaluate my class website against the following objectives: increasing student ownership for their work, increasing their interaction with the course content, giving students the ability to prepare for class by reviewing the material from prior lessons, and accessing information when absent from class.
Week 3 Assignment: Read Chapter 2, participate in discussion, complete the following exercise:
a) Boulmetis & Dutwin use three guiding questions to lead us in resolving the question of why to evaluate:
• What are the benefits and limitations of an evaluation?
• What factors ensure that an evaluation will be successful?
• How might one use evaluation results?
Answer the three questions above as they apply to the following scenario.
Rich Kids, Poor Kids
The Maricopa Community College District (MCCD) partnered with Cashflow Technologies, Inc., to design and deliver a course in financial literacy. The course is based on the products – books, games, and videos – created and sold by the company. Components in the course include training sessions for future course instructors and seminars at which attendees play the Cashflow game. MCCD commissioned an evaluation of the program to verify its value to students and the community. In addition, Cashflow Technologies is exploring the possibility of marketing the educational program to educational institutions throughout the nation so an educational evaluation of the program should lend it credibility. Cashflow Technologies has been producing the materials for three years. The course was offered for the first time during the fall 2011 semester. No training sessions or seminars have been performed.
The program will be evaluated by students in Educational Technology at Boise State University. The evaluators have access to sales information for the products, student performance data from the course, and contact information for the students and instructors. The budget for the evaluation is $6,000.
b) Look back at the assignment you did last week. You described a potential program for your evaluation project. Now having read chapter two, you can answer these questions in reference to that evaluation:
1. Would the program you detailed in Chapter One benefit from an evaluation? If so, how?
2. What are the inherent limitations in the evaluation of the program? Note: We’re talking about potential limitations of the evaluation, not the program being evaluated.
3. How might you use the results to benefit the organization, community, schools, or yourself?
1.The benefits of evaluating the Financial Literacy Program include: determining if the program is achieving its goals of adding value for the students that participate in the course, determining if the training program is effective for the course instructors, and last determining if the program is viable “as is” for marketing to other schools (or does it need to adjusted?). Another benefit of the evaluations is that the evaluators are not directly involved with the program, and thus do not have a personal interest which increases the objectivity, both real and perceived, of the evaluation. Limitations of the evaluation include: the people doing the evaluating are students and are learning how to evaluate, it is imperative for the credibility of the evaluation that the evaluators are seen as qualified, so hopefully, the student evaluators work closely with their expert instructors as they perform the evaluation. Another benefit and potential limitation will be determining the impact, in other words, the long term effects of the program on the students in the program to determine if the course has changed behaviors.
2. The factors that ensure success are making sure that the all involved (students, instructors, and Cashflow Technologies) see that the evaluation is objective. Ii is crucial that all decision makers be prepared for information that may or may not support their intended plans for the program. All involved should be prepared for alternative suggestions.
3. The evaluation results may corroborate what Cashflow Technologies is trying to determine, that is the program is successful, and has potential to be marketed nationally to other schools. On the other hand, the evaluation results may determine that the program needs to be improved. Improvements may be needed in teacher training or course materials and curriculum may need to be improved before CashFlow Technologies can begin to market the program to other schools.
b The program to be evaluated is a teacher’s class website (my own) to determine the effectiveness of website at: increasing student ownership for their work, increasing their interaction with the course content, giving students the ability to prepare for class, review the material from prior lessons, and accessing information when absent from class.
1. The website program would benefit from an evaluation to determine if it is meeting the objectives set forth by the teacher. Another potential benefit is that the evaluation might uncover other ways students are looking to interact with course content, and the website can be improved based on the feedback and in turn might generate new objectives for having a class website (and a future evaluation!). The evaluation will hopefully show which objectives are being met and which are not. As a consequence, the teacher might change the objectives and the website for the future.
2. Limitations of the evaluation are the credentials and objectivity of the evaluator. The evaluator is a student, a novice completing her first evaluation, and thus the evaluation could be missing some key components (hopefully not!). Another potential limitation is that the evaluator is evaluating her own program, which could limit her ability to be objective. The evaluator is interested in learning how to improve the website, and hopefully her willingness to be open to both positive and negative feedback will increase her ability to be objective. Another potential limitation is receiving high percentage of student feedback on surveys, etc. The more feedback, the better in this evaluator’s opinion!
3. The results of the evaluation will be used first and foremost to improve this teacher’s website for her students. Secondarily, the evaluation has the potential to help her peers feel more comfortable creating a class website that is based on direct student feedback to help teachers create meaningful websites. Last, there is the potential to create professional development for teachers interested in creating a website using the school’s X2 Aspen software to create a website.