NOTETAKING GUIDES: CURRICULUM EVALUATION

Evaluation

• The process or group of processes that people perform in order to gather data that will enable them to decide whether to accept, change, or eliminate something.
• Concerned with "relative values" and "statements of worth".
• Determining whether the expected or the planned has occurred or is occurring in relation to the intended.
• Did the curriculum or training program, as designed, developed, and implemented produce the desired results.

Comprehensive Evaluation of a Curriculum

• The curriculum or curriculum document.
• How might it be judged?
• Ritz Model = Document validation

Validity & Curriculum Evaluation

• Do the instruments do what they are supposed to do!

How best to evaluate curriculum?

• Data based judgements
• Student achievement
• Expert analysis (survey/checklist)
• What is the theoretical rationale for the curriculum?
• Is the curriculum a coherent whole?
• How were the curriculum purposes (goals) devised? Is the content consistent with the purpose (aim)?
• Observe it in action (student performance)?

Product Evaluation

• Student evaluation.
• How might it be done?
• What provides the bulk of the data on which product evaluations are usually made?

Student Evaluation

• Objective Based (goals and objectives)
• Domains of Learning
• Cognitive - knowledge assessment
• Psychomotor - skill assessments
• Attitudes - values
• Multiple Activities and Assessments

The Five Value Questions

• Intrinsic Value --
• the goodness and appropriateness of the curriculum.
• Does the curriculum incorporate the best thinking to date on what is known of the content and the presentation of the content?
• Instrumental Value --
• What use is the curriculum, and who is the intended audience? Does the curriculum address the goals and objectives?
• Comparative Value --
• Is the new program better that the one it replaced?
• Idealization Value --
• How can the curriculum be improved for optimal benefit?
• Decision Value --
• Should the new program be retained, modified, or discarded?

Scientific vs. Humanistic Evaluation

• Scientific Evaluation --
• More focused on quantifiable data gathering
• Uses tests results of experimental subjects
• Analyzes data statistically
• Humanistic Evaluation --
• More focused on qualifiable data gathering
• Relies on impressions of what is observed
• Engages in actual incidents that are observed
• Today, most evaluators use both types of evaluation.

What can be said about intelligence tests?

• Extensive caution and prudence needs to be exercised in drawing inferences from these measuring tests?

Standards in Testing

• Absolute maximum standard
• Absolute minimum standard
• Relative standard
• Multi-standard

Formative and Summative Evaluation

• Formative -- (part)
• Collecting data on those activities undertaken to improve an existing program during the development and early piloting of a course or program.
• Summative -- (whole)
• Collecting data on an existing program after it has been developed and implemented.
• Focuses on the effectiveness of the course or program.
• Terminal judgement.

Phases of Evaluation

1. Focusing on the curricular phenomena to be evaluated (what).
2. Collecting the information.
3. Organizing the information.
4. Analyzing the information.
5. Reporting the information.
6. Recycling the information (continual update).

Alternative Evaluation

• Portfolio - Compilation of learner's work over time.

Norm-References vs Competency-Based Evaluation

• Norm-Referenced - Comparing student's performance against others.
• Competency-Based - What a student can or cannot do.

Evaluation vs Grading

• The evaluation of student learning is far too complex an enterprise to be reduced to a single grade.
• More grades = better evaluation.

Goals and Roles of Evaluation

Goals - What it is supposed to do, i.e., determine the progress of learners achieving the aims, goals, and objectives of the curriculum.
Roles - What it is used to do, e.g., honor role, clubs, punishment, promotion, rewards, etc.

Does training add value to corporate strategy?

• Much does not because:
• Programs are not linked specifically to strategies, challenges, or problems in the organizations.
• Programs designed to create awareness and understanding, but not competence.

Evaluation of Corporate Training

• Programs focus on individuals rather than operating units.
• Participants attend programs for reasons other than personnel or organizational needs.
• Programs fail to help participants confront reality.

Focus of Training Evaluation

• Program Evaluation -- How efficient and effective is the training or education program?
• Individual competence -- How well did the participant learn?
• Program Value -- Does the training provide a good return on cost or investment?

Training Evaluation

• Training evaluation takes place during and after course implementation. It is based on data that may come from:
• Internal Review
• Trainees' Observations
• Instructor Observations
• Trainees' Course Performance
• Training Completer's On-The-Job Performance
• Improvement to the course may be made after analysis of the data gathered.

Calculating Costs of Training

• Steps to Calculating Costs -- To determine the costs of a training project, a trainer needs to calculate:
• All development and personnel costs
• All materials costs
• All delivery costs
• Total expenditures

Benefits of Training

• To determine the worth of the course a boss wants to know the benefits for implementing the course. Possible benefits you might consider are:
• Time
• Materials
• Equipment
• Personnel Turnover
• Safety
• Intangible Benefits

Goal of Evaluation

• Learner improvement.
• Methods
• Document validation
• Product assessment