Formative Evaluation: Our Process

For the purposes of our course, we will be conducting a formative evaluation within OER Commons using the Achieve evaluation rubrics Links to an external site. discussed in Modules 2 and 4. For an overview of the evaluation process using these rubrics, please view the video (2:41 minutes) below.

Achieve Evaluation Criteria

As noted in the video above, there are eight aspects of the instruction that are evaluated within the Achieve evaluation. Please open the Achieve OER Rubrics and the Achieve Guide to using the open educational resources rubrics. Let's review the focus of each rubric section as reprinted from the Achieve Guide.

Rubric I: Degree of Alignment to Standards*

This rubric is applied to learning objects that have suggested alignments to standards. It is used to rate the degree to which an individual object actually aligns to each proposed standard. Note, the rubric was designed specifically for the Common Core State Standards (CCSS), but can be used with any set of standards. [As we have addressed elsewhere, the College and Career Readiness Standards (CCRS) that we are using for our adult education resources were developed from the CCSS.] Before the rubric can be applied, the assumption is that a user has proposed an alignment between the object and the selected standard(s).

Rubric II: Quality of Explanation of Subject Matter*

This rubric is applied to objects designed to explain subject matter. It is used to rate how thoroughly the subject matter is explained or otherwise revealed in the object. Teachers might use this object with a whole class, a small group or an individual student. Students might use the object to self-tutor. For objects that are primarily intended for teacher use, the rubric is applied to the explanation of the subject matter not to the planning instructions for the teacher.

Rubric III: Utility of Materials Designed to Support Teaching*

This rubric is applied to objects designed to support teachers in planning or presenting subject matter. The primary user would be a teacher. This rubric evaluates the potential utility of an object at the intended grade level for the majority of instructors.

Rubric IV: Quality of Assessment*

This rubric is applied to those objects designed to determine what a student knows before, during or after a topic is taught. When many assessment items are included in one object, as is often the case, the rubric is applied to the entire set.

Rubric V: Quality of Technological Interactivity*

This rubric is applied to objects designed with a technology-based interactive component. It is used to rate the degree and quality of the interactivity of that component. “Interactivity” is used broadly to mean that the object responds to the user; in other words, it behaves differently based on what the user does. This is not a rating for technology in general, but for technological interactivity. The rubric does not apply to interaction between students, but rather to how the technology responds to the individual user.

Rubric VI: Quality of Instructional Tasks & Practice Exercises*

This rubric is applied to objects that contain exercises designed to provide an opportunity to practice and strengthen specific skills and knowledge. The purpose of these exercises is to deepen understanding of subject matter and to routinize 10 foundational skills and procedures. Sets of practice exercises are treated as a single object, with the rubric applied to an entire group.

Rubric VII: Opportunities for Deeper Learning*

This rubric is applied to objects designed to engage learners in at least one of the following deeper learning skills, which can be applied across all content areas:

Think critically and solve complex problems.
Work collaboratively.
Communicate effectively.
Learn how to learn.
Reason abstractly.
Construct viable arguments and critique the reasoning of others.
Apply discrete knowledge and skills to real-world situations.
Construct, use or analyze models.

Rubric VIII: Assurance of Accessibility*

This rubric is used to ensure that materials are accessible to all students, including students identified as blind, visually impaired or print disabled, and those students who may qualify under the Chafee Amendment to the U.S. 1931 Act to Provide Books to the Adult Blind as Amended. It was developed to assess compliance with U.S. standards and requirements, but could be adapted to accommodate differences in other sets of requirements internationally. Accessibility is critically important for all learners and should be considered in the design of all online materials. Identification of certain characteristics will assist in determining if materials will be fully accessible for all students. Assurance that materials are compliant with the standards, recommendations and guidelines specified assists educators in the selection and use of accessible versions of materials that can be used with all students, including those with different kinds of challenges and assistive devices.

* Source: Reprinted from the Achieve Guide Links to an external site.