Chapter One
 Chapter Two
 Chapter Three
 Chapter Four
 Chapter Five

   

Chapter Three - Boxes

Box 3.1 : The elements of research design

The elements of research design

  • A clear statement of the problem/need that has given rise to the research;
  • Constraints on the research (e.g. access, time, people, politics);
  • The general aims and purposes of the research;
  • The intend outcomes of the research: what the research will do and what is the ‘deliverable’ outcome;
  • How to operationalize research aims and purposes;
  • Generating research questions (specific, concrete questions to which concrete answers can be given) and hypotheses (if appropriate);
  • The foci of the research;
  • Identifying and setting in order the priorities for the research;
  • Approaching the research design;
  • Focusing the research;
  • Research methodology (approaches and research styles, e.g.: survey; experimental; ethnographic/naturalistic; longitudinal; cross-sectional; historical; correlational; ex post facto);
  • Ethical issues and ownership of the research (e.g.: informed consent; overt and covert research; anonymity; confidentiality; non‑traceability; non‑maleficence; beneficence; right to refuse/withdraw; respondent validation; research subjects; social responsibility; honesty and deception);
  • Politics of the research: who is the researcher; researching one’s own institution; power and interests; advantage; insider and outsider research;
  • Audiences of the research;
  • Instrumentation, e.g.: questionnaires; interviews; observation; tests; field notes; accounts; documents; personal constructs; role‑play;
  • Sampling: size/access/representativeness; type: probability: random, systematic, stratified, cluster, stage, multi‑phase; non‑probability: convenience, quota, purposive, dimensional, snowball;
  • Piloting: technical matters: clarity, layout and appearance, timing, length, threat, ease/difficulty, intrusiveness; questions: validity, elimination of ambiguities, types of questions (e.g. multiple choice, open‑ended, closed), response categories, identifying redundancies; pre‑piloting: generating categories, grouping and classification;
  • Time frames and sequence (what will happen, when and with whom);
  • Resources required;
  • Reliability and validity:

validity: construct; content; concurrent; face; ecological; internal; external;

reliability: consistency (replicability); equivalence (inter‑rater, equivalent forms), predictability; precision; accuracy; honesty; authenticity; richness; dependability; depth; overcoming Hawthorne and halo effects; triangulation: time; space; theoretical; investigator; instruments;

  • Data analysis;
  • Verifying and validating the data;
  • Reporting and writing up the research.

 


Box 3.2 Elements of research styles

Model

Purposes

Foci

Key terms

Characteristics

Survey

Gathering large scale data in order to make generalizations

 

Generating statistically manipulable data

 

Gathering context-free data

Opinions

Scores

Outcomes

Conditions

Ratings

Measuring

Testing

Representativeness

Generalizability

Describes and explains

 

Represents wide population

 

Gathers numerical data

 

Much use of questionnaires and assessment/test data

Experiment

Comparing under controlled conditions

 

Making generalizations about efficacy

 

Objective measurement of treatment

 

Establishing causality

Initial states, intervention and outcomes

 

Randomized controlled trials

Pre-test and post-test

 

Identification, isolation and control of key variables

 

Generalizations

 

Comparing

 

Causality

Control and experimental groups

 

Treats situations like a laboratory

 

Causes due to experimental intervention

 

Does not judge worth

 

Simplistic

Ethnography

Portrayal of events in subjects’ terms

 

Subjective and reporting of multiple perspectives

 

Description, understanding and explanation of a specific situation

Perceptions and views of participants

 

Issues as they emerge over time

Subjectivity

 

Honesty, authenticity

 

Non-generalizable

 

Multiple perspectives

 

Exploration and rich reporting of a specific context

 

Emergent issues

Context specific

 

Formative and emergent

 

Responsive to emerging features

 

Allows room for judgements and multiple perspectives

 

Wide data base gathered over a long period of time

 

Time consuming to process data

 

Action research

To plan, implement, review and evaluate an intervention designed to improve practice/ solve local problem

 

To empower participants through research involvement and ideology critique

 

To develop reflective practice

 

To promote equality democracy

 

To link practice and research

 

To promote collaborative research

Everyday practices

 

Outcomes of interventions

 

 

 

 

 

 

 

Participant empowerment

 

 

 

 

Reflective practice

 

 

Social democracy and equality

 

Decision making

Action

Improvement

Reflection

Monitoring

Evaluation

Intervention

Problem-solving

Empowering

Planning

Reviewing

Context-specific

 

Participants as researchers

 

Reflection on practice

 

Interventionist – leading to solution of ‘real’ problems and meeting ‘real’ needs

 

Empowering for participants

 

Collaborative

 

Promoting praxis and equality

 

Stakeholder research

Case study

To portray, analyze and interpret the uniqueness of real individuals and situations through accessible accounts

 

To catch the complexity and situatedness of behaviour

 

To contribute to action and intervention

 

To present and represent reality – to give a sense of ‘being there’

Individuals and local situations

 

Unique instances

 

A single case

 

Bounded phenomena and systems:

  • individual
  • group
  • roles
  • organizations
  • community

Individuality, uniqueness

 

In-depth analysis and portrayal

 

Interpretive and inferential analysis

 

Subjective

Descriptive

Analytical

Understanding specific situations

 

Sincerity

Complexity

Particularity

In-depth, detailed data from wide data source

 

Participant and non-participant observation

 

Non-interventionist

 

Empathic

 

Holistic treatment of phenomena

 

What can be learned from the particular case

 

 

Testing and assessment

To measure achievement and potential

 

To diagnose strengths and weaknesses

 

To assess performance and abilities

Academic and non-academic, cognitive, affective and psychomotor domains – low-order to high order

 

Performance, achievement, potential, abilities

 

Personality characteristics

Reliability

Validity

Criterion- referencing

Norm-referencing

Domain- referencing

Item-response

Formative

Summative

Diagnostic

Standardization

Moderation

Materials designed to provide scores that can be aggregated

 

Enables individuals and groups to be compared

 

In-depth diagnosis

 

Measures performance


Box 3.3 A matrix for planning research

ORIENTING DECISIONS

Question

Sub-issues and problems

Decisions

1. Who wants the research?

Is the research going to be useful?

Who might wish to use the research?

Are the data going to be public?

What if different people want different things from the research?

Can people refuse to participate?

Find out the controls over the research which can be exercised by respondents.

What are the scope and audiences of the research.

Determine the reporting mechanisms.

2. Who will receive the research?

Will participants be able to veto the release of parts of the research to specified audiences?

Will participants be able to give the research to whomsoever they wish?

Will participants be told to whom the research will go?

Determine the proposed internal and external audiences of the research.

Determine the controls over the research which can be exercised by the participants.

Determine the rights of the participants and the researcher to control the release of the research.

3. What powers do the recipients of the research have?

What use will be made of the research?

How might the research be used for or against the participants?

What might happen if the data fall into the ‘wrong’ hands?

Will participants know in advance what use will and will not be made of the research?

Determine the rights of recipients to do what they wish with the research.

Determine the respondents’ rights to protection as a result of the research.

 

4. What are the time scales of the research?

Is there enough time to do all the research?

How to decide what to be done within the time scale?

Determine the time scales and timing of the research.

5. What are the purposes of the research?

What are the formal and hidden agendas here?

Whose purposes are being served by the research?

Who decides the purposes of the research?

How will different purposes be served in the research?

Determine all the possible uses of the research.

Determine the powers of the respondents to control the uses made of the research.

Decide on the form of reporting and the intended and possible audiences of the research.

6. What are the research questions?

Who decides what the questions will be?

Do participants have rights to refuse to answer or take part?

Can participants add their own questions?

Determine the participants’ rights and powers to participate in the planning, form and conduct of the research.

Decide the balance of all interests in the research.

7. What must be the focus in order to answer the research questions?

Is sufficient time available to focus on all the necessary aspects of the research?

How will the priority foci be decided?

Who decides the foci?

Determine all the aspects of the research, prioritize them, and agree on the minimum necessary areas of the research.

Determine decision making powers on the research.

8. What costs are there – human, material, physical, administrative, temporal?

What support is available for the researcher?

What materials are necessary?

Cost out the research.

9. Who owns the research?

Who controls the release of the report?

What protections can be given to participants?

Will participants be identified and identifiable/traceable?

Who has the ultimate decision on what data are included?

Determine who controls the release of the report.

Decide the rights and powers of the researcher.

Decide the rights of veto.

Decide how to protect those who may be identified/identifiable in the research.

10. At what point does the ownership pass from the respondent to the researcher and from the researcher to the recipients?

Who decides the ownership of the research?

Can participants refuse to answer certain parts if they wish, or, if they have the option not to take part, must they opt out of everything?

Can the researcher edit out certain responses?

Determine the ownership of the research at all stages of its progress.

Decide the options available to the participants.

Decide the rights of different parties in the research, e.g. respondents, researcher, recipients.


RESEARCH DESIGN AND METHODOLOGY

Question

Sub-issues and problems

Decisions

11. What are the specific purposes of the research?

How do these purposes derive from the overall aims of the research?

Will some areas of the broad aims be covered, or will the specific research purposes have to be selective?

What priorities are there?

Decide the specific research purposes and write them as concrete questions.

12. How are the general research purposes and aims operationalized into specific research questions?

Do the specific research questions together cover all the research purposes?

Are the research questions sufficiently concrete as to suggest the kinds of answers and data required and the appropriate instrumentation and sampling?

How to balance adequate coverage of research purposes with the risk of producing an unwieldy list of sub-questions?

Ensure that each main research purpose is translated into specific, concrete questions that, together, address the scope of the original research questions.

Ensure that the questions are sufficiently specific as to suggest the most appropriate data types, kinds of answers required, sampling, and instrumentation.

Decide how to ensure that any selectivity still represents the main fields of the research questions.

13. What are the specific research questions?

Do the specific research questions demonstrate construct and content validity?

Ensure that the coverage and operationalization of the specific questions addresses content and construct validity respectively.

14. What needs to be the focus of the research in order to answer the research questions?

How may foci are necessary?

Are the foci clearly identifiable and operationalizable?

Decide the number of foci of the research questions.

Ensure that the foci are clear and can be operationalized.

15. What is the main methodology of the research?

How many methodologies are necessary?

Are several methodologies compatible with each other?

Will a single focus/research question require more than one methodology (e.g. for triangulation and concurrent validity)?

Decide the number, type and purposes of the methodologies to be used.

Decide whether one or more methodologies is necessary to gain answers to specific research questions.

Ensure that the most appropriate form of methodology is employed.

16. How will validity and reliability be addressed?

Will there be the opportunity for cross-checking?

Will the depth and breadth required for content validity be feasible within the constraints of the research (e.g. time constraints, instrumentation)?

In what senses are the research questions valid (e.g. construct validity)?

Are the questions fair?

How does the researcher know if people are telling the truth?

What kinds of validity and reliability are to be addressed?

How will the researcher take back the research to respondents for them to check that the interpretations are fair and acceptable?

How will data be gathered consistently over time?

How to ensure that each respondent is given the same opportunity to respond?

Determine the process of respondent validation of the data.

Decide a necessary minimum of topics to be covered.

Subject the plans to scrutiny by critical friends (‘jury’ validity).

Pilot the research.

Build in cross-checks on data.

Address the appropriate forms of reliability and validity.

Decide the questions to be asked and the methods used to ask them.

Determine the balance of open and closed questions.

17. How will reflexivity be addressed?

How will reflexivity be recognized?

Is reflexivity a problem?

How can reflexivity be included in the research?

Determine the need to address reflexivity and to make this public.

Determine how to address reflexivity in the research.

18. What kinds of data are required?

Does the research need words, numbers or both?

Does the research need opinions, facts or both?

Does the research seek to compare responses and results or simply to illuminate an issue?

Determine the most appropriate types of data for the foci and research questions.

Balance objective and subjective data.

Determine the purposes of collecting different types of data and the ways in which they can be processed.

19. From whom will data be acquired (i.e. sampling)?

Will there be adequate time to go to all the relevant parties?

What kind of sample is required (e.g. probability/non-probability/ random/stratified etc.)?

How to achieve a representative sample (if required)?

Determine the minimum and maximum sample.

Decide on the criteria for sampling.

Decide the kind of sample required.

Decide the degree of representativeness of the sample.

Decide how to follow up and not to follow up on the data gathered.

20. Where else will data be available?

What documents and other written sources of data can be used?

How to access and use confidential material?

What will be the positive or negative effects on individuals of using certain documents?

Determine the necessary / desirable / possible documentary sources.

Decide access and publication rights and protection of sensitive data.

21. How will the data be gathered (i.e. instrumentation)?

What methods of data gathering are available and appropriate to yield data to answer the research questions?

What methods of data gathering will be used?

How to construct interview schedules/questionnaires/tests/

observation schedules?

What will be the effects of observing participants?

How many methods should be used (e.g. to ensure reliability and validity)?

Is it necessary or desirable to use more than one method of data collection on the same issue?

Will many methods yield more reliable data?

Will some methods be unsuitable for some people or for some issues?

Determine the most appropriate data collection instruments to gather data to answer the research questions.

Pilot the instruments and refine them subsequently.

Decide the strengths and weaknesses of different data collection instruments in the short and long term.

Decide which methods are most suitable for which issues.

Decide which issues will require more than one data collection instrument.

Decide whether the same data collection methods will be used with all the participants.

22. Who will undertake the research?

 

Can different people plan and carry out different parts of the research?

Decide who will carry out the data collection, processing and reporting.

 

 

 


DATA ANALYSIS

Question

Sub-issues and problems

Decisions

23. How will the data be analyzed?

Are the data to be processed numerically or verbally?

What computer packages are available to assist data processing and analysis?

What statistical tests will be needed?

How to perform a content analysis of word data?

How to summarize and present word data?

How to process all the different responses to open-ended questions?

Will the data be presented person by person, issue by issue, aggregated to groups, or a combination of these?

Does the research seek to make generalizations?

Who will process the data?

Clarify the legitimate and illegitimate methods of data processing and analysis of quantitative and qualitative data.

Decide which methods of data processing and analysis are most appropriate for which types of data and for which research questions.

Check that the data processing and analysis will serve the research purposes.

Determine the data protection issues if data are to be processed by ‘outsiders’ or particular ‘insiders’.

24. How to verify and validate the data and their interpretation?

What opportunities will there be for respondents to check the researcher’s interpretation?

At what stages of the research is validation necessary?

What will happen if respondents disagree with the researcher’s interpretation?

Determine the process of respondent validation during the research.

Decide the reporting of multiple perspectives and interpretations.

Decide respondents’ rights to have their views expressed or to veto reporting.


PRESENTING AND REPORTING THE RESULTS

Question

Sub-issues and problems

Decisions

25. How to write up and report the research?

Who will write the report and for whom?

How detailed must the report be?

What must the report contain?

What channels of dissemination of the research are to be used?

Ensure that the most appropriate form of reporting is used for the audiences.

Keep the report as short, clear and complete as possible.

Provide summaries if possible/fair.

Ensure that the report enables fair critique and evaluation to be undertaken.

26. When to write up and report the research (e.g. ongoing or summative)?

How many times are appropriate for reporting?

For whom are interim reports compiled?

Which reports are public?

Decide the most appropriate timing, purposes and audiences of the reporting.

Decide the status of the reporting (e.g. formal, informal, public, private).

27. How to present the results in tabular and/or written-out form?

 

How to ensure that everyone will understand the language or the statistics?

How to respect the confidentiality of the participants?

How to report multiple perspectives?

Decide the most appropriate form of reporting.

Decide whether to provide a glossary of terms.

Decide the format(s) of the reports.

Decide the number and timing of the reports.

Decide the protection of the individual’s rights, balancing this with the public’s rights to know.

28. How to present the results in non-verbal forms?

 

Will different parties require different reports?

How to respect the confidentiality of the participants?

How to report multiple perspectives?

Decide the most appropriate form of reporting.

Decide the number and timing of the reports.

Ensure that a written record is kept of oral reports.

Decide the protection of the individual’s rights, balancing this with the public’s rights to know.

29. To whom to report (the necessary and possible audiences of the research)?

Do all participants receive a report?

What will be the effects of not reporting to stakeholders?

Identify the stakeholders.

Determine the least and most material to be made available to the stakeholders.

30. How frequently to report?

 

Is it necessary to provide interim reports?

If interim reports are provided, how might this affect the future reports or the course of the research?

Decide on the timing and frequency of the reporting.

Determine the formative and summative nature of the reports.


Box 3.4 A planning sequence for research

What are the specific purposes of the research?

What are the research questions?

What needs to be the focus of the research in order to answer the research questions?

What is the main methodology of the research?

How will validity and reliability be addressed?

How will reflexivity be addressed?

What kinds of data are required?

From whom will data be acquired (sampling)?

Where else will data be available?

How will the data be gathered (instrumentation)?

Who will undertake the research?

How will the data be processed and analyzed?

How to verify and validate the data and their interpretation?

How to write up and report the research?

How to present the results in written and non-verbal forms?

To whom to report?

When to report?

Who wants the research?

Who will receive the research?

What powers do the recipients of the research have?

What are the time scales of the research?

What costs are there – human, physical, material administrative, temporal?

Who owns the research?

At what point does the ownership pass from the respondent to the researcher and from the researcher to the recipients?

What are the powers of the researcher?

What are the main foci of the research?

What are the ethics of the research?

 

What are the purposes of the research?

Down Arrow Callout: Stage One    Identify the purposes of the research    Down Arrow Callout: Stage Two    Identify and give priority to the constraints under which the research will take place  Down Arrow Callout: Stage Three    Plan the possibilities for the research within these constraints
 

Box 3.5 A planning matrix for research

Time

Sample

Stage 1

(start)

Stage 2

(3 months)

Stage 3

(6 months)

Stage 4

(9 months)

Stage 5

(12 months)

Principal/

Headteacher

Documents

Interview

Questionnaire 1

Interview

Documents

Questionnaire 2

Interview

Documents

Interview

Questionnaire 3

Teacher group 1

Questionnaire 1

 

Questionnaire 2

 

Questionnaire 3

Teacher group 2

Questionnaire 1

 

Questionnaire 2

 

Questionnaire 3

Teacher group 3

Questionnaire 1

 

Questionnaire 2

 

Questionnaire 3

Students

 

 

Questionnaire 2

 

Interview

Parents

Questionnaire 1

 

Questionnaire 2

 

Questionnaire 3

University teacher educators

Interview

Documents

 

 

 

Interview

Documents

 

Box 3.6 Understanding the levels of organizational culture

LEVELS OF CULTURE

 

 

INSTRUMENTS

EASY TO UNCOVER

 

TANGIBLE

 

SUPERFICIAL

NON-PARTICIPANT OBSERVER

ARTIFACTS

 

OBSERVATIONAL DATA

 

DOCUMENTARY DATA

 

Qualitative data

 

ENACTED VALUES

(BEHAVIOURS)

 

SURVEY QUESTIONNAIRES AND NUMERICAL MEASURES

 

Qualitative data

 

 

 

 

 

UNDERLYING ASSUMPTIONS

QUALITATIVE & ETHNOGRAPHIC DATA

 

INTERVIEWS (GROUP AND INDIVIDUAL)

 

 

 

 

 

 

 

 

HARD TO UNCOVER

 

 

INTANGIBLE

 

 

DEEP

 

 

PARTICIPANT OBSERVER

 

FACE-TO-FACE

 

 

 

 

 
Copyright © 2006 Taylor & Francis Group, an informa business