Lesson 3: Measuring Creativity

Can creativity be measured?

Creativity is so important to life that we want to capture it – and for psychologists that means operationally defining it in some way to study it. For educators that means assessing the creative potential of students. If you are cringing a bit you have good reason. There have been many attempts to measure creativity and they have all been imperfect. Yet, as stated earlier there is a bigger down side to not even trying to explore the mechanisms, motivations, and perspectives of creativity. We will never know anything about this important aspect of humanity if we don’t try. Likewise, if we don’t acknowledge the gaps in our ability to grasp and measure this concept, we will never improve on our understanding. Keeping in mind that researchers generally acknowledge the huge gaps in our knowledge, lets explore how science and education have approached the study of creativity.

Some attempts toward measuring creativity

There are a several available tests of creativity (Thys, Sabbe, & De Hert, 2014). but we will discuss just a few of the most well known. These 4 tests were selected for their diversity – to give you a sampling of the different approaches to the study of creativity and for their popularity among researchers.

  1. Remote Associates Test (RAT)
  2. Torrance Tests (ex: Abbreviated Torrance Test for Adults, ATTA)
  3. Consensual Assessment Technique, (CAT)
  4. Creative Personality Scale Creative (CPS)

Remote Associates Test (RAT)

The Remote Associates Test (RAT) was developed by Sarnoff and Martha Mednick (1967), who believed that the basis of all thinking was chains of semantic associations (such as TABLE-CHAIR) and that creative individuals possessed the ability to make more distant associations along this conceptual chain (such as TABLE-CHEESE, Perkins, 1981). Click here Links to an external site. for a more complete summary of the theory. The RAT was designed to test the ability to make these more distant associations. The following are examples from the RAT:

Read the following word triads and come up with one word that fits with all three:

Railroad ........ Girl ........ Class

Surprise ........ Line........ Birthday

Wheel ........ Electric ........ High

Out ........ Dog ........ Cat

The standard answers to the above items are: Working, party, chair or wire, and house.

Click here Links to an external site. to see a version of the RAT. 

The main problem with the RAT is that it really measures convergent thinking and probably reveals more of a propensity for verbal fluency than creativity, so it lacks some construct validity (Kaufman, 2009).

Torrance Tests

Torrance Tests were developed by E.P. Torrance rely heavily on Guildford’s structure of Intellect theory and divergent thinking (see section above).  Torrance test are still frequently used in education to test for giftedness. In a Torrance test, you are typically asked to generate many responses to a situation within a set period of time. Here are some examples of questions on a Torrance Test:

  • Unusual (or Alternative) uses: The examinee must list all the interesting and unusual uses for a common object such as a cardboard box.
  • Asking Questions: The examinee must write out all the questions he or she can think of based on a drawing of a scene.
  • Product Improvement: The examinee must list ways to change a product to fulfill a purpose, for example, so that children would have more fun playing with it.
  • Circles: The examinee expands empty circles into different drawings and titles them.

You may wonder how such tests would be scored. Say we gave a person the Unusual uses version of the Torrance test, asking what other uses might there be for a brick. Lets say this person came up with the following 5 responses in the allotted 3 minutes:

-use as a paperweight

-use as a weapon against possible intruders

-use it to threaten your big brother when being a jerk

-paint a face on it and if you ever get stuck on a desert island it can be your “Wilson” only name it “Brock the Brick”

-Use it to improve your cooking.

The four aspects included in a score on the Torrance Test are:

(1) Fluency, which is the ability to generate multiple different responses. This is scored simply by counting the number of relevant responses. I would give the above person a “4” since the last response doesn’t seem relevant.

(2) Originality, which is the ability to generate responses that are unusual or uncommon. This is scored by giving the prompt to a large group of people and establishing a list of the most common responses. Responses not on that list are counted as original. I think probably only the 4th response would be counted as original – so this person would receive a score of 1.

(3) Elaboration, which is the ability to complete ideas with rich details. To score elaboration, you count the number or details per response. For example, the 4th response would get an elaboration score of 4: 1 for painting a face on the brick, 1 for using it as a friend, 1 for using it when stuck on a desert island, and 1 for naming it Brock.

(4) Flexibility, which is the ability to respond to the same object in different ways. For flexibility, you would count the number of different categories within the response. For flexibility, responses 2 and 3 would likely only count as 1category because they are in the same category of “weapon.” So the score would be 3 (you only count the relevant responses and so the “cooking” response wouldn’t count).

Consensual Assessment Technique (CAT)

Amabile (1982) hypothesized that “a product or response is creative to the extent that appropriate observers independently agree it is creative” (p. 1001).

She constructed a way of assessing creativity: 1) ask a sample of people to create something: it could be a soliloquy or a story or a drawing or to improve a product in a creative way. Importantly, the CAT could be used on a wide range creative products so long as a tangible product has been created. 2) Ask experts to independently rate the products. The experts compare the creative products in the sample to each other instead of an “ideal;” in other words the products are not being compared to the best in the field – like to Shakespeare or Picasso. Previous research has shown that experts nearly always show high agreement with each other (e.g., Amabile, 1983, 1996; Hennessey & Amabile, 1999). Of course, this technique is best suited to evaluate a particular creative product, not the person or process. Reliability tends to be high among expert raters, less for novices and poor between experts and novices, suggesting that the experts may truly be the best source for judging creativity (Kaufman, 2009; Kaufman & Baer, 2012). The main problem with the CAT, of course, is who qualifies as an expert? However, this question has been addressed in many publications since the test has been in use and is generally found to have good reliability and validity (Kaufman, 2009).

Creative Personality Scale (CPS)

There are many self-report scales of creativity. While self-report techniques have obvious drawbacks, they are widespread and easy to use. One self-report technique is the Creative Personality Scale (CPS, Gough, 1979; Gough & Heilbrun, 1983), a list of 30 adjectives; 18 of which are indicative of creativity, 12 of which are counter-indicative or creativity. Some adjectives indicating creativity are: clever, individualistic, and unconventional. Adjectives suggesting low creativity include: cautious, conservative, and mannerly. Though the CPS has good to moderate reliability and validity (Gough, 1979), self-report techniques should generally be approached with caution and the CPS specifically may be subject to biases (Luescher et al., 2016).

Now that you have some idea of what creativity is, how it has been defined by researchers, and how it may be measured, let’s look at some of the big questions concerning creativity: it’s relationship with intelligence, whether creativity represents one skill or many, and how/if it can be improved.

Take CPS here Links to an external site.

Take 2 minutes to take some notes: List the 4 tests you just learned about. Think about which of the 4 perspectives has been adopted by each test. What are the advantages and drawbacks of each?