File(s) not publicly available

Mixing interview and questionnaire methods : practical problems in aligning data

journal contribution
posted on 06.12.2017, 00:00 authored by Lois HarrisLois Harris, G Brown
Structured questionnaires and semi-structured interviews are often used in mixed method studies to generate confirmatory results despite differences in methods of data collection, analysis, and interpretation. A review of 19 questionnaire-interview comparison studies found that consensus and consistency statistics were generally weak between methods. Problems in aligning data from the two different methods are illustrated in a questionnaire-interview study of teacher conceptions of assessment. Poor alignment appeared attributable to: differences in data collection procedures, the complexity and instability of the construct being investigated, difficulties in making data comparable, lack of variability in participant responses, greater sensitivity to context and seemingly emotive responses within the interview, possible misinterpretation of some questionnaire prompts, and greater control of content exposure in the questionnaire. Results indicated that if 'confirmatory' results are being sought, researchers must create tightly aligned and structured instruments; present the construct in a simple, concrete, and highly contextualised manner; collect the two types of data with a minimal time gap; and estimate agreement between methods using consistency statistics. However, the cost of confirmation through strong alignment may lead to the loss of rich complementary data obtained through allowing each method to be analysed in its own right.

Funding

Category 1 - Australian Competitive Grants (this includes ARC, NHMRC)

History

Volume

15

Issue

1

Start Page

1

End Page

19

Number of Pages

19

ISSN

1531-7714

Location

United States

Publisher

University of Maryland

Language

en-aus

Peer Reviewed

Yes

Open Access

No

External Author Affiliations

Hong Kong Institute of Education; TBA Research Institute; University of Auckland;

Era Eligible

Yes

Journal

Practical assessment, research & evaluation.