Measurements are not the same as the attributes being measured. When you measure a thing, you assign a number to that object, preferably with some method in mind. But do the numbers actually represent the objects in a correct way? When you measure the temperature in a room. Suppose it is 20 degrees Celsius. Is this twice as hot as in a room of 10 degrees?Take a look at the following amusing example. Two villages, Sprintel and Fastel hold an annual running competition. The ten fastest runners from each village run each a distance of one thousand meters. The time it takes for each runner is measured. Runner 1 of Sprintel needs 116 seconds and the one of Fastel only 105 seconds. The results of the 20 runners are shown in the table below.
If you look at the two left columns, the result is clear-cut. Sprintel needed on average less seconds (120 s) than Fastel (122.7 s). So, Sprintel has won! But according to a psychologist from Fastel, speed is usually measured in kilometers per hour, not in seconds per kilometer. Therefore, an independent jury converted all the scores with the formula km/hour = 3600 x (km/seconds). This is done in columns 3 and 4 of the table. There you can see that the first runner of Fastel has an average of 34.2857 km/h and that the total average of Fastel is greater (thus faster) than Sprintel. So, Fastel has won!
But the two villages cannot win both, do they? So, what is the correct method of measurement?
We made the same mistake as in the following riddle. A train travels 200 km. The first 100 km it rides at 40 km/hour. The second 100 km, it speeds up to 60 km/hour. How long does the train need for this trajectory?
The following argument is not correct but illustrates the same mistake as in the Sprintel-Fastel problem. For the first 100 km, the train runs at 40 km/hr and for the second 100 km at 60 km/hr. So, on average it rides at 50 km/hr and therefore it will take 4 hours to finish the tour.
Now, think twice. If the little train runs the first 100 km at 40 km/hr, then it will need 2 hours and 30 minutes to complete this trajectory (40 + 40 + 20 km). The second 100 km will be done in 1 hour and 40 minutes (60 + 40 km). So, the train need 4 hours and 10 minutes in total; not 4 hours! The error we made in the first calculation was averaging the speed. This is only allowed when the train travels equal time periods, not equal distances. If the riddle was that the train drives 1 hour at 40 km/hr and another hour at 60 km/hr, then it is allowed to calculate the average speed. The same holds true in the Sprintel-Fastel problem. If the runners had run equal periods of time, then it was allowed to calculate the average speed. So, our first method was the correct one and Sprintel has indeed won the game.
So, why bother? Psychologists however measure a lot of things: intelligence, personality, skills, attitudes, … But, how certain are we that the numbers actual represent the measured attributes? And what kind of arithmetic operation are we allowed to perform on those numbers? For example, if we aggregate the subtests of an intelligence testbattery, do we make the same error as above? Measurement theory helps to make those decisions.
Another frequently made error by inexperienced researchers is the following.
Sprintel and Fastel have found another way to decide who has won. They count the medals at the games. As you can see, Sprintel won respectively 60%, 40% and 30% of all medals in running, jumping and cycling. They won, on average , 43% percent of the medals while Fastel won 57%. So, Fastel has won more medals than Sprintel and is the winner! But, is it?
When you look at the actual medals, Sprintel has won in total 23 medals and Fastel one less (22). The real winner is thus Sprintel. You are not allowed to calculate the average of percentages, unless the numbers on which the percentages are derived are equal. Sprintel, in fact, has won because it has 60% (= 18 medals) of the running medals, of which there were 30 in total. This is far more than the 70% (7 medals) of the cycling medals of Fastel.
Of course, it is also not very good practice to calculate percentages on numbers less than 100 because with this transformation you suggest a precision which is not there. When you have only two observations, it is obvious that you can have only the values 0%, 50% or 100% and not all the values in between.
REFERENCES
The post Measurement theory: some pitfalls appeared first on Cogpsy.info.