Research Questions ACG: Day 1

Session Content Finding your research focus Types of research questions A process to develop your research questions Using your research questions throughout your research.

Finding your Research Focus Sharpest focus Influential context Also in the picture

Finding your Research Questions What topic do you want to study? Why? What has brought you to it? What in particular do you want to find out? Do you have any hunches? Why is it important to find out? Why is this relevant for your discipline and field? What will it add to what you / we already know? So: What are your key research questions? (and component questions embedded)

Types of research questions Definitional – what do we mean by…? where and how is it defined? Contextual – what is the context for it – eg legal and policy, social, professional contexts; whats the research context Descriptive - what is going on? where, with whom, how much of it? how is it experienced? whose accounts/perspectives do we have? are there gaps? Explanatory - what helps to explain what is happening? what factors underlie it? what helps to make it happen? (what causes it?); what concepts and theories help us to understand why it is happening? Evaluative - how useful/effective is it? does it work? over what period of time? how well and for whom? how can we tell? Strategic - what are the implications? (why) does it matter? does it help us to recommend plans for change or development; should it affect practice or policy priorities? how and why?

A process to develop your research questions What Topic do you want to study? Why do you want to study this? What in particular do you want to find out? Any hunches? Why is it important to find out?

A process to develop your research questions Main question Component questions Further questions

Using your research questions throughout your research. Allow yourself time and space to develop your question and its component questions; be flexible and open, but let yourself develop a framework Use your research question/s to decide what is the approach to take Use them to guide what literature (methodological and substantive) you search for, what you read and how you read it – ie what is it saying that addresses my particular question/s? Use your research question/s to guide whom you research, what you ask them or explore about them and how you go about it Use them to reflect on how well your methodology and method have worked Later on: Use your question/s to draw together/synthesise what you find – develop themes and critical argument It often makes sense to structure your writing around your component research questions (eg one section/ chapter per question)

Capturing Data ACG: Day 1

What is data?

Capturing Data Documents Photographs Observations Conversations – interviews Audio recording Video recording

Limitations Effects of being observed.

Ethics: Permission, Informed Consent, Trust ACG: Day 1

What is trust? What words do you associate with trust in the context of evaluating the action research projects? Between the researcher and the respondents. Between the researcher and the research commissioners

What are research ethics? Set of principles about how researchers and research organizations should conduct themselves. It covers: how you deal with research participants, other researchers and colleagues, the users of your research society in general.

Six key principles of ethical research identified by the ESRC. Principle 1 Research should be designed, reviewed and undertaken to ensure integrity, quality and transparency. Integrity Quality Transparency Examples

Six key principles of ethical research identified by the ESRC. Principle 2 Participants must normally be informed fully about the purpose, methods and possible uses of the research. What participation entails and what risks, if any, are involved. Informed consent Risks

Six key principles of ethical research identified by the ESRC. Principle 3 Confidentiality and the anonymity of respondents must be respected. Confidentiality Anonymity

Six key principles of ethical research identified by the ESRC. Principle 4 Participants must take part voluntarily, free from any coercion. Voluntary Coercion Examples

Six key principles of ethical research identified by the ESRC. Principle 5 Harm to research participants and researchers must be avoided. Harm Examples

Six key principles of ethical research identified by the ESRC. Principle 6 The independence of research must be clear, and any conflicts of interest or partiality must be explicit. Independence Conflicts of interest Partiality

Data Collection & Analysis: Interviews ACG: Day 1

What is an interview? A fixed series of questions for data collection? A conversation with a purpose? (open and trustworthy) As a social action? (position of participants is addressed openly) A social action to find meanings, motivations, reasons?

Types of Interview Formal Structured Individual One off Public Informal Unstructured Group Sequential Private

Approaches to Interview

STRUCTURED INTERVIEWS : PROS AND CONS Advantages: Pre-set agenda – answers to the questions youre looking for Time/resource efficient Consistency – the same questions asked of all – eliminates interviewer idiosyncrasy Comparability - aids categorisation and thematisation within existing frameworks and codes for ease of analysis Less obtrusive? Disadvantages Pre-set agenda – only get answers to the questions youre looking for…. Formality, inflexibility Less responsive to subject Potentially leading Less partnership/user voice? Ensuring clarity of meaning trickier Less facilitative of discussion

Whats involved in semi-structured interviewing? The interviewer prepares a list of questions and topics to be discussed. The order of the questions and topics is undefined, dependant on the flow of the discussion. It is best to start with a topic that is not sensitive and is important to the respondent. Thus, an informal, friendly atmosphere can be created, facilitating a natural flow of ideas and opinions. The researcher acts as a moderator, guiding the respondent from one topic to another. Conducting such interviews requires a skilled moderator. Potentially more time-consuming, but more flexible and responsive to the individual and what emerges (Who, 2013)

Planning an Interview Developing an interview schedule. Refer to Research Questions. Start with generalised questions and move to more specific questions. Decide on questions which are descriptive to start, analytical towards the end.

A Structure of an Interview Schedule. Interview title, date and location. Script for Ethics and Informed Consent. Permission to record interview. List of Main Questions (open and related to Research Questions). Introduction: First question should put respondent at ease. Main: Move from descriptive to analytical. Sub-questions and prompts for each main question. Closure: Summary and Thanks.

Writing Effective Questions Phrasing of the questions – not ambiguous nor complex, nor long Move from the particular to the general – descriptive to analytical responses Closed (What, how) to Open (Why do you think?). Balance of closed/open/probes (merits and flaws) Probes can allow for qualification /expansion of view as closed questions may close down the discussion Dont be leading! – some questions may suggest answers Appropriate language focus for audience (including age, competence, culture, knowledge of the topic) Logic and clarity (self-explanatory, unambiguous)

Practical Considerations Location, access, timing, confidentiality, seating, ensuring no interruptions, communication, exit Do you send your interview questions/schedule to your respondents beforehand? Group interviews Being reflexive during the interview Importance of getting a triangulation of views from a diverse group.

Doing the interview Aim to have the respondent speak more than the interviewer. Encourage the respondent to give more information: Tell me more about that. How did you feel about that? Why did you decide to do that? Can you give me an example? Allow pauses for thinking time. Recap what the respondent says to check your understanding. You said X earlier? Does that link with Y? So, to summarise….. Is that correct?

Recording the Interview Written notes taken during the interview/or immediately afterwards. Audio-taping the interview and transcribing afterwards (1 hour of tape = 3 hours transcribing) Video recording Sending the written transcription or your notes back to the respondent Organising a second meeting to analyse the interview notes/recording together as part of the whole process

Analysing an Interview Reading entire text in one go with research question (s) as the driver (deductive) Noting overarching themes or patterns at the end Second, close read to create more defined categories New categories emerge from the date (inductive) Correlation and comparison of interview data when sequential, over time, or between respondents

FOCUS GROUPS The aim is for the group participants to generate material through their interaction, although stimulated and guided by the researcher who is a moderator rather than interviewer (Stewart and Shamdasani, 1990). Advantages Opportunity for different stakeholders to meet together, for insights to be generated and complex thoughts debated (Krueger, 1994). Through this synergistic group process new insights based on attitudes, opinions and experiences may be produced which the individual might not achieve alone (Asbury, 1995). Direct evidence can be obtained of similarities and differences in respondents views (Morgan, 1997).

Making an Interview Schedule Workshop ACG: Day 1

Research Questions Have the Action Research projects impacted on teaching and learning? If so, how? What is the evidence? Has doing AR impacted more widely on practice over time? If so, how? What is the evidence? What if any, How do they (TRC, Teacher, Pupil, Head teacher) see the action research process to date impacting upon their future professional life in the short, medium and long term?

Data Analysis: Interviews with Adults ACG DAY 2

Data analysis and interpretation Research design – logic of the research Researcher interests Empirical experiences of the researcher Theoretical framework RESEARCHER IDENTITY

The generic within data analysis and interpretation Analysis Certain texts are selected and others put aside; Data reduction and organization; Observations coded, grouped or themed Interpretation Through theoretical constructs; Contextual parameters; Informed by the research questions

Ways of Analysing Interviews

Analysis of qualitative data Does not involve statistical techniques – because it is not making the same sort of claims as quantitative data about generalisability, certainty etc May be part of a systematic approach but does not have to be (structured/ semi-structured/unstructured approaches) Can involve counting for descriptive or comparative purposes and to assist with the identification of themes Qualitative researchers do not make large claims based on their data – to add to the robustness of the study different perspectives and sources are often triangulated

Objectivity is often associated with a quantitative approach and subjectivity with a qualitative approach (but note that these words carry implicit messages…) Places a great deal of emphasis on reflexivity and a discussion of the researchers position within the research –is therefore more explicitly self-consciously situated. Our closeness to the processes on which we are reflecting can be seen as both an asset and a liability (Gewirtz et al. (2009), p.568)

46 Kanizsa triangle

What you bring to the analysis is important – and includes not only reflexivity about what you bring to the research but also an understanding of context Context works across a range of levels, from the very specific to the more general (eg, school/area/national/policy etc) Knowledge of what is already known is another identify and synthesise the key issues from the literature an important part of the process Issues of definition are important: think back to Andys example of social class… 47

Critical and emancipatory approaches Critical social research does not take the apparent social structure, social process or accepted history for granted. It tries to dig beneath the surface of appearances. It asks how social systems really work, how ideology or history conceals the processes which oppress and control people… it questions the nature of prevailing knowledge and directs attention to the processes and institutions which legitimate knowledge. (Harvey, 1990, p.16)

What is already known Concepts How it can be known and understood epistemology The nature of what is to be understood ontology Positionality andTransparency Reflexivity Thinking about the links between methodological approach and data analysis from a qualitative perspective

CODING Categories meaningful / representative? Saturation of categories reached? Re-code when you reach that point Any false polarisation or creation of opinions? Mutually exclusive categories or fuzzy categories? Consistency / planned variation Format appropriate for analysis? Summary / composite codings Manual or electronic coding? e.g. Nvivo

methodological coherence across the whole: words and tone When framing research questions: – To what extent is X affected by Y? – How does X affect Y? When writing about your data: – Viewed from this perspective X can be seen to be… – It is essential that we pay more attention to X as the most influential factor in …. Make up some further examples of your to reflect different methodological approaches

Triangulation: validity and reliability. ACG DAY 3

Validity Qualitative research: Concerns honesty, richness, depth and scope. Never perfect Not an absolute, but researchers strive to maximise it.

Validity Data comes from the natural context Context bound and thick description Data are socially situated and socially and culturally saturated. The researcher is part of the researched Acknowledges that interpretation is part of the situation. Holism The researcher is the key instrument Data are descriptive Data are analysed inductively Data are presented in terms of the respondents Seeing and reporting the situation through the eyes of the respondents Respondent validation. Catching meaning and interpretation are essential Cohen, Manion & Morrison (2000) p.106

Two Types of Validity Internal Validity Seeks to demonstrate that the explanation of a particular event, issue of set of data which a piece of research provides can actually be sustained by the data. The findings must accurately describe the phenomena being researched. External Validity The degree to which results can be generalised to the wider population, cases and situations.

Reliability Problematic in qualitative research In interviews minimise bias by being aware of: The attitudes, opinions, and expectations of the interviewer; The tendency of the interviewer to see the respondent in her own image; A tendency for the interviewer to seek answers to his preconceived notions; Misinterpretation on the part of the interviewer of what the respondent is saying; Misunderstandings on the part of the respondent of what is being asked.

Triangulation …the use of two or methods of data collection in some aspect of human behaviour….Triangulation is a powerful way of demonstrating concurrent validity, particularly in qualitative research.

Triangulation Sources and methods: Documents Observations TRC perspective Teacher perspective Head teacher perspective Pupil perspective Outsider perspective Similarities between perspectives. Differences between perspectives.

Forming Conclusions ACG DAY 3

Writing an Evaluation Report ACG DAY 3

Purpose of an Evaluation Report Share findings Discuss findings Provide a summary Make recommendations to stakeholders

Outline of an Evaluation Report

Point, Evidence, Analysis

Language and writing

Our Iqs for TRCs Tell me/Remind me about the significant developments in the ARPs in this school since we met in Nov project Has your perception of AR changed over time? If so, how? What evidence do you have of this? Has the perception of AR across your school changed over time? If so, how? What evidence do you have of this? From your experience of AR as a TRC, what essential qualities (attitudes, behaviours, practices) are important for improving teaching and learning? Teachers, other professionals, students, headteachers, senior leaders, whole school, between schools. Have you identified any evidence of sustained change (or for future sustained change) as the result of the AR in your school (and beyond)? If so what evidence? Or what would you look for?