Making Sense of Feedback
Weighing Feedback
To this point in your design project, you have written a draft Design Proposal, self-assessed your proposal, created a prototype in Open Author, and received feedback during the formative evaluation. Now, you must decide what to do with feedback you have received, make adjustments, and turn in your final deliverable. The following highlights three of the most common scenarios you could be facing right now.
Conflicting Options
In reviewing the evaluation feedback, you may have noticed conflicting opinions. It is common when multiple people evaluate the same lesson to have different opinions on the same aspect of the instruction. One reviewer may have noted an area as weak and in need of improvement, while another noted the strengths. In addition, you may disagree with some of the feedback that was offered. In his book Planning and Conducting Formative Evaluation, Tesmer (1993) suggests using the second opinion approach. This involves seeking the opinion from another (third) person to talk through the areas of contention. To do this, you could ask a peer, or reach out to subject matter expert, and ask how they would address the discrepancy without indicating your perspective.
Too Many Revisions
Formative evaluation may also yield a lot of recommendations. At times, it may seem better to start over from scratch. However, tackling all of the recommendations (or starting over) could be difficult, or impractical if the time and cost associated with the changes are too great. If you are faced with this situation, Tessmer (1993, p. 65) recommends focusing on the impact revisions that you feel will make the "most impact on improving the performance of the learners who complete the instruction". For example, recommendations addressing minor aesthetic factors could have little influence on the learners ability to utilize the instruction, and could take a back seat to revisions associated with content accuracy. One approach is to write out all of the noted revisions, and rank them based on perceived impact. Typically, this approach yields a much smaller roster of "needed" versus "nice to have" revisions.
Not Enough Feedback
In contrast to these first two scenarios, you may find yourself with not as much feedback as you had hoped, or feedback that doesn't give you much guidance and direction. This is common in the real world, as well. It takes a lot of time to offer someone detailed feedback, and sometimes people don't have the time to evaluate your work to the level of detail you want. In addition, some reviewers have a hard time offering a critical assessment, and don't want to come across as harsh. Other times, a reviewer may not appreciate the value of their feedback within the formative evaluation process, and rush through the review. In this case, you must weigh the cost (spending more time) and benefit (more feedback) of soliciting more opinions. Tessmer (1993) recommends that if you decide to go back to request additional feedback, spend time upfront preparing the reviewer for the evaluation by being organized in what feedback you want and outlining your specific questions and areas of concern.