Title:

Did Microsoft commission a fair test of the geographical knowledge of British children?

Author:

Hamish Chalmers, 22 Leckford Road, Oxford, UK, OX2 6HX This email address is being protected from spambots. You need JavaScript enabled to view it.

Objective:

Microsoft commissioned a test of the geographical knowledge of British school children, using multiple-choice questions. The results were presented in the National Press as showing gross deficiencies in children’s geographical knowledge, and government curriculum advisers said they found the extent of the children’s ignorance “disturbing”. I judged some of the methods used in Microsoft’s test likely to have misled the children. The current study was carried out to assess whether the design of the multiple-choice questions influenced the proportion of children giving correct answers.

Design:

Randomised, parallel group, controlled trial. Independent samples design. Within each of eight junior school classes, I used coin tosses to allocate each child on the class roll to one or other of the two comparison groups.

Setting:

Paulton Junior School (ages 7 years to 11 years old), a rural school in Bath & North East Somerset, England, 1997

Population:

All 224 pupils present on the days of the experiment participated.

Intervention:

As in the Microsoft study, participants were asked to complete a set of three multiple-choice questions: ‘What language is spoken in: Mexico, Paris and Tokyo?’ The control group were given the answer options that were used in the Microsoft study, which included the plausible non-languages Mexican, Parisian, and Tokyan. The answer options available to the experimental group replaced these non-languages with real languages.

Within each stratum, control and experimental groups were tested separately but concurrently. Participants were given the questions in written and oral form by class teachers who read standardised instructions to them. When all the children in the group had completed the test, the survey forms were collected and the children were told the correct answers to the questions.

Data Collected:

Answers to the three questions were collected using a standard questionnaire, which differed only in respect of one answer option per question. The proportion of participants in each group answering the questions correctly was compared using Chi2 test.

Results:

For each of the three questions, children presented with the ‘Microsoft’ answer options were far more likely to give incorrect answers than those answer options in which non-languages had been replaced with real languages.

Place

Experimental Group (n=113)
No. (%)

'Microsoft' Group (n=111)
No. (%)

Statistical Significance

Mexico

37 (33)

6 (5)

0.001

Paris

77 (68)

57 (49)

0.01

Tokyo

48 (42)

12 (11)

0.001

Conclusion:

The design of the Microsoft study greatly influenced the proportion of children giving correct answers, and resulted in gross misrepresentation of children’s geographical knowledge. A clue to the possible motives to the gross misrepresentation becomes clear in the closing statement of Microsoft’s press release which states: “The Microsoft Geographical Survey reveals some surprising results about children’s knowledge. Geography can be a fascinating subject, and, at Microsoft, we’ve tried to bring it to life with Microsoft Encarta 97 World Atlas – the richest world atlas on CD ROM – for your computer.”

A more detailed report of this study is available from the author.

Research

Register now for CEM Assessments

We use cookies to improve our website and your experience when using it. Cookies used for the essential operation of the site have already been set. To find out more about the cookies we use and how to delete them, see our Privacy Notice.

I accept cookies from this site

SIGN UP FOR CEM UPDATES

Read our Privacy Notice (opens in new window)

Find out what our CEM Updates include (opens in new window).

BOOK A DEMO / WEBINAR

Read our Privacy Notice (opens in new window)

Find out what our CEM Updates include (opens in new window).