Research Approaches, Methods and Design. #2

This post relates to the article on:

Practical Assessment Research and Evaluation – A Guide to Writing the Dissertation.

Randolph (2009) Practical Assessment Research and Evaluation – A Guide to Writing

the Dissertation. In Practical Assessment, Research & Evaluation. Volume 14,

Number 13, June 2009

Conducting a literature review is a mean sof demonstrating an authors knowledge about a particularly field of study including vocabulary theories, key variables and phenomena and its methods and history.

Gall (1996) finds that Literature Reviews plays a part in:

• Delimiting the research problem
• Seek new lines of enquiry
• Avoiding fruitless approach
• Gaining Methodogical insights
• Identifying Recommendations for Future Research.
• Seeking Support for ground theory.

Literature Reviews also provide opportunities to distinguish betwen what has been done and what needs to be done. Without establishing what the previous research has been undertaken, it is impossible to establish how future research will enhance that research undertaken previous.

Literature Reviews are a means of analysing and synthesizing previous study into an area and more than just a bibliography of previous research. Literature Reviews provide the opportunity to summarise the literature and drawing conclusions on this.

Cooper’s Taxomony of a Literature Review:

Literature Reviews can be classified according to these categories:

• Focus.
• Goal
• Perspective
• Coverage
• Organization
• Audience.

Focus:

Two main outcomes exist:  Outcome option may deal with theories related to phenomonem being investigated and introduce the practice. Methodolgical reviews help identify the methodological weakenesses and strengths of a certain body of research.

Goal

The goal of many literatrure review is to integrate and generlize findings across:

• Units
• Treatments
• Outcomes
• Settings.

Literature Reviews can be either Methodogical Based or Outcomes Oriented –The technique of metaanlysis is used in Literature Reviews  with the goal of integrating  quantitative outcomes across studies.  Critical Analysis as a technique can be used to explain an argument within a field.

Perspective

There are two perspectives that a literature review can take:

• Qualititative Primary Research – Where the author takes on the perspective to reveal their own pre-existing review and biases which can impact the review.
• Quantitative Primary Research: - Where the authors can attempt to take a neutral perspective and present the findings as fact.

Coverage

Cooper proposes 4 possible scenarios:

1. Exhaustive Review- The author promises to locate and consider every piece of research on a certain topic.
2.  Exhaustive Review – Selection Criteria – where you define the population in such a way that is bounded (an example could be just using journal articles.
3. Representative Sample – Review sample of articles and make inferences about, random sampling.
4. Purposeful Sample: - Examines only the central protocol. In this scenario, the goal is to convince the reader that the sample is pivotal to their understanding.

Organization

Literature Reviews can take various forms. The most common include:

• Historical format

 Working chronologically = common when looking a progression of research or theories over a period of time.

• Conceptual format

 Organized relating to concepts.

• Methodogical format.

Generally organized with introduction, method results and discussion.

It is possible to include a combination of a variety of these in order to be most effective (eg mix and match the formats)

Audience

The primary audience for most literature reviews are generally a supervisor.

The secondary audience are generally peers.

It is important to remember to avoid writing the review in a non academic format.

How to write a literature review.

Cooper (1984) Describes the stages of conducting a literature review:

1. Problem Formulation.
2. Data collection.
3. Data Evaluation.
4. Analysis and Interpretation.
5. Public Presentation.

Problem Formulation

Formulating the question is the first step after deciding on the format of the review. There are two types of questions:

Literature Review Question – Which are those which can only be answered through reviewing the secondary research.

Empirical Research Question -  those which can only be answered through reviewing primary research.

Questions can be:

• Critical Analysis.
• Integrate Research Outcomes.
• Focus on past themes.

Data Collection

Collection can be either exhaustive, semi exhaustive representative or pivotal set of relevant articles.  The reviewer is advised to describe their data collection method. Using reference lists of the research being reviewed is a source of finding research.

The data collection process can stop when the point of saturation (eg all the articles meeting the criteria) is reached.

Careful identification of the types of data which is to be included / excluded at the commencement of the research review will assist with this process. The exclusion process will continue throughout the collection process.

Data Evaluation

At this stage the reviewer begins to extract and evaluate those which meet the criteria for inclusion.  The reviewer needs to come up with a strategy for extracting data, of which the data is determined by the focus and goal of the review.

Data Analysis / Evaluation

The reviewer at this stage attempts to mae sense of the data which ha sbeen extracted.  Depending on the type of data extracted, a quantitative, qualitative or mixed method synthesis will be performed.

Public Presentation

At this stage, the reviewer needs to determine which information is most important and what will be presented in the review and what will not.

My personal overview of the article.

This article provided a really good description of the stages involved in the planning and structure of the Literature Review.  The article will assist in the development of my literature review, particularly with the importance of planning the structure and what should / should not be excluded

Synthesising Knowledge: Learning /Technology Research Cycle and Future Trends.

 This post looks at the following article:

Fraenkel, J.R. & Wallen, N.E. (2006) How to Design and Evaluate Research in Education, Chapter 24 - Preparing Research Proposals and Reports. 6^th edition, Boston: McGraw-Hill.

           

Preparing a Research Proposals and Reports.

Research Proposal- completed before the study.

Research Report – completed after the study.

The structure of the Research Proposal can take the following format:

• Purpose of the Study.
• Justification of the Study
• The Research Question and Hypothesis including Variables.
• Definition of Terms.

Research Proposal.

The proposal shows the intention of the research and also makes clear the justification as to why the research to be undertaken is important / needed.  The proposal provides the author with the opportunity to identify problems and state the hypothesis to be addressed and the variables and terms which are relevant.  The proposal also provides the opportunity to determine the design of the procedures to analyse data which are to be included in the proposal as well as how that data will be analysed once collected.

Purpose of the Study.

In this section of the report the reviewer needs to succinctly state, what they propose to investigate in the study. Any assumptions should be outlined in this stage of the report.

Justification for the Study.

In the justification section the researcher must make clear why the topic / subject are important and why it is vital to be investigated further. The justification section also provides the reviewer with the opportunity to present the argument into the ‘worth’ of the study.  A good justification should also include any specific implications that follow if relationships are identified.

Research Question.

This is where the particular question to be investigated should be stated. Most reviewers favour a hypothesis for this  research strategy to be pursued.

Definitions

All key terms that will be referred to or assumed knowledge, should be included as a definition in this section of the proposal. The inclusion of any terms in this section will minimise the difficulties in data analysis / research when undertaking the report.

Literature Review.

This section of the report is quite lengthy; it provides a partial summary of a previous research into the area relating to the hypothesis.  The reviewer here is trying to show that they are familiar with the research and opinion of the topic and that they have an understanding of this research.

Sample / Populations.

This section is where in detail the reviewer describes how the subjects of the study will be obtained the sample and whether a generalized sample (random sample) will be included or if a more specific group – a convenience group is required (e.g. a specific demographic, age, gender etc)  then the specifics should also be included in this section.

Data Analysis

In this section – the reviewer should indicate how the data is to be collected and how it will be organized as well as what data will be included or excluded in the study.

My personal observations of the article.

Fraenkels’ article provides a strong basis for the topic areas that need to be included when preparing a Research Proposal. The article contains Questions to ask at various stages of the proposal and report process, these allow for the reviewer to track where the progress is at various stages of the process.

Developing Learning Technology Research Proposal.

The following overview is found in an article published by Deakin University on Research Design.  The article is;

Author Unknown Topic 3. Research Design n.d., published by Deakin University , accessed 14 October. <www.deakin.edu.au/buslaw/.../research/docs/research-design.pdf >.

Research Design.

Good research design is an important part of successful research.

Research is a process, one in which requires a series of actions.

A Research Model is a way of proceeding through the research process.

The Research Model includes:

1. Define research objectives – Decide specifically what questions need to be answered.

2. Do preliminary research – collect the info that that will help define objectives more clearly. Literature Reviews can assist in determining what else is out there in the field.

3. Design the formal research – Develop data collection instruments.

4. Do fieldwork – collect the data.

5. Analyse the data –analyse, interpret and report findings.

One beneficial way of identifying a problem is to use a topic that the researcher has an in-depth or personal understanding of.

Identifying the Broad Problem

Whilst a topic may start coming from a researchers personal knowledge of a topic, this may lead to a problem solving focus of the research, which could itself turn to be an advantage.  This doesn’t mean that researchers shouldn’t stop at including topics in which you don’t have an intimate knowledge of.

Hasty adoption of an idea for a topic may not turn out to address the real issues. Undertaking situational analysis will assist in precisely identify issues.

Problem Statements.

Once you’ve identified the problem, there’s a number of ways of articulating focus of the search. It is important to continually keep reshaping your question.

Types of Variables

A variable is anything that can take on differing or varying values.  The values differ at various times for the same object or person, or the values can differ at the same time for different objects or persons (Sekaran 1992, p 64)

The dependant variable is the variable of primary interest to the researcher.  The researcher’s goal is to explain or predict the variability in the dependent variable. In other words, it is the main variable that lends itself as a viable issue for investigation. Through analysis of the dependent variable (i.e. what variables influence it) it is possible to find answers or solutions to the problem at hand. (Sekaran 1992 pp 65)

Independent Variable is often called an Experimental variable,

The changes in dependent variable are sometimes referred to as a criterion.

Preliminary Data Gathering.

Gathering preliminary data is a way of defining your thoughts.

The hypothesis

Hypothesis is an educated guess. A hypothesis is a proposition. a theoretical statement to account for the facts. Hypotheses are generally given the benefit of the doubt. .

Research Paradigm.

This is theoretical framework which underpins the research process.  There are two fundamental research paradigms.

1. Positivistic
2. Phenomenological

Your preliminary data collecting will help you deciding which mode you are in.

Positivistic methods of collecting data are collected systematically and methodically (e.g. a survey)  

Phenomenological are personal observations and case specific.

My personal thoughts on the article.

This article whilst not in a formally published journal as such, I found to be very beneficial. the article really broke down the components in an easy to understand and apply format. The article really assisted in reinforcing my undertanding of the importance of design in the research process.

Data Collection and Analysis Technique.

 This overview relates to the following article:

Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. Journal of the Learning Sciences, 6(3), 271-315.

Quantifying Qualitative Analyses of Verbal Data

It has become increasingly needed in educational and cognitive science research to collect and analyse messy data – Messy data can be seen as those items such as verbal explanations, observations, videotaping and gestures[1].  This data is required to give a complete understanding of a skill or learning something other than just numerical or qualitative data.

Verbal data and observational data are two forms of data which can be used to in this situation.  Both of these data analysis forms have been used for some time.

Verbal Analysis is a methodology for quantifying the subjective or qualitative coding of verbal utterances (what a subject said).  The verbal analysis method is embedded in research to understand cognition, and the kind of knowledge one gains from learning. Verbal analysis can include more than just verbal data, and can be adapted to include various non cognitive issues (social motivational and behavioural) by the use of observational and video data.

The goal of verbal analysis is to capture the representation of knowledge that a learner has learnt and how that changes with the acquisition of learning.  Using verbal analysis to uncover what a learner knows the analysis of verbal utterances’ (what the learner said) is needed.  Analysing the utterances not just I the words, but the other elements which can underlie the words such as pointing and expressions and the impact that this can have on the acquisition of learning.   

The most frequent and systematic use of verbal data is in the context of protocol analysis. The differences between verbal analysis and protocol analysis are:

• Instruction
• The goal or focus.
• The analysis
• The validation
• The conclusion.

The first difference focuses on the way verbal data is collected.

Protocol Analysis – uses think aloud protocols (Ericsson and Simon 1984, 1993)[2] where subjects instructed to verbalise the information they attended to while solving a problem.

Talking is more than pointing at numbers. Think aloud analysis discourages any description of what subjects are doing. These descriptions, explanations and justifications are known as Explaining.

My personal thoughts on the article.

This article provided an overview of the difference methods of both qualitative and quantitative methods of research and the use of the verbal and observational analysis and how the use of multiple methods of data (e.g. video, and visual as well as the spoken word)  

---

[1] Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. Journal of the Learning Sciences, 6(3), 271-315.

[2]  Ericsson, K. A., & Simon H. (1984). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.

]

Research Approaches, Methods and Design.

This post relates to the following article:

Hubner S, et al (2008):  Writing Learning Journals: Instructional support to overcome learner strategy deficits in Learning and Instruction 20 (2010) 18-29    

Writing Learning Journals: Instructional support to overcome learner strategy deficits.

Writing is a way of reinforcing learning it supports cognitive and metacognitive strategies.  Learning journals are becoming increasingly used in education to promote learning and are in many cases a core component of majority of online courses.

Learning journals are a writing task which is generally used as a follow up to traditional course work. Learning journals can be completed in both an online capacity or in a traditional face to face component.  Research into the use of the learning journals has found that learners who use learning journals have a greater knowledge in the area being written about than those who do not. Learning journals are intended to induce productive, cognitive and metacognitive strategies through self regulated learning. (Hubner S,2008)

On a cognitive level, journals allow learners to organize and elaborate on learning materials. Journals also allow for learners to implement elaboration strategies as they get to generate examples of application of the learning and using analogies to that relate to the material they have learned.

Developing appropriate scaffolding structures to be used in these learning journals remained a key component throughout the article. The study undertaken involved using learners from differing levels of study e.g. kindergarten to university level structures, each with differing degrees and types of scaffolding. The results found that at each of the levels of learning a different type of scaffolding was required.  For example the use of prompts – that is those questions or hints which  promote productive learning behaviour, remained a key factor in the university based student group, whilst different techniques were needed at the kindergarten level.

My personal reflection on the article.

This article brought up the valuable point that, you write differently if your writing for yourself than what you do if you are for writing for a supervisor or a peer. Learning journals or blog writing remains a key component of most online courses, for example this blog here itself can be classified as a learning journal that is one in where reflection of learning occurs. Reflection can also be gained by commenting or posting on others blogs and posts, which itself reinforces that component of scaffolding (peer to peer).

Selecting and Framing Research Questions and Design

This post relates to the following article:

Boote, D. N., & Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Researcher, 34(6), 3-15.

Scholars before researchers: On the centrality of the dissertation literature review in research preparation

A substantive literature is an integral part of undertaking substantive and thorough research.

“Good” research is good because it advances our collective understanding. A good researcher needs to understand what has happened before them and the strengths and weaknesses of existing students and what this might mean.  Researchers cannot perform significant research until they understand the literature in the field.  (Boote 2005)

Literature Reviews should accomplish several important objectives:

• It sets out the broad context of the study.
• Clearly demonstrates what is and what is not included.
• Justifies your decisions for what is to be included and not included.

Literature Review enables the author to distinguish what has been learnt and what remains to be learned.

Good Literature Reviews must be both methodogical and sophisticated.  – A good synthetic literature review has 3 key characteristics:

• Clarifies and perhaps resolves a problem rather than ‘glossing’ over a problem.
• Progressive shift that yields a new perspective on the literature and predictive power than is offered in existing perspectives.
• Finally satisfies the formal criteria of a goal theory.

Boote and Beille, have created a table for analysing literature reviews, which there are 5 categories and 12 scoring system.  These include:.

1. Coverage.	How well the author has justified the criteria for inclusion / exclusion.
2. Synthesis	To gauge how well the author summarised, analysed and synthesised the selected literature
3. Methodology	Measures how the author identified main methodological and research techniques that have been used and the advantages and disadvantages.
4. Significance	How well the dissertation rationalised the practical and scholarly significance in the research problem.
5. Rhetoric	How the literature review was written, was it clear, coherent structure that supports the review?

This article provided a good basis for the structure of a literature review and particularly what needs to be included in the actual report.

My personal thoughts on the article.

The article also reiterates the need for having a clear understanding of the research currently undertaken in your area before you commence any further research.  Boote and Beille also speak under their heading of ‘significance’ about the importance of ensuring the balance between scholarly application and practical application. This is where in the past I have found my greatest problems. I have found that as a professional educator, I tend to apply what I am learning in a practical sense, without applying the same scholarly application to that practice e.g. the research behind the practical application. This last point is where I need to go.

Boote and Beille provided a good infrastructure to start the literature review. This article was clearly laid out and whilst I think it targeted a more specific scope of student (e.g. someone e about to do a dissertation) the key reiteration of understanding what research is currently available as to where the current research is needed.

Conceptual Foundations, Research Issues and Research Perspectives # 3

 This blog relates to the following article:

.

Palloff R.M, Pratt K. (2005) Online Learning Communities Revisited presented at 21^st Annual Conference Teaching and learning Retrieved 14-08-2010 http://www.uwex.edu/disted/conference/

Online learning Communities Revisited.

Online Communication is a major part in Online Learning. Whether in the traditional face to face capacity or online, the social aspect of learning remains key to its success. The social aspect of learning and creation of social identity in online communities is driving online technology. 

The definition of the role that online communities play in online learning have not changed drastically over time. It is through the use of reflective practice and learning that differentiates online collaborative learning with online communication. 

The social aspect of learning remains fundamental in the research of the satisfaction levels of staff completing online courses.  Students link social presence to student satisfaction. The use of collaboration enhances the social presence, enhances learning and breaks down feelings of isolations.  The collaboration then allows for students the opportunity to expand their knowledge. (Palloff 2005)

For learning to be effective, participants in learning communities need to do far more than just ‘lurk’ or just checking into their learning community in order to support the development of the community.  It is through the development of this community, and the reflection that comes through contributing that learning occurs.

My personal overview of the readings.

Our course is fully online, well at least it is for me, and isolation is definitely one of the key contributors on my challenges for learning. Whilst online course provide me with many benefits like flexibility of time and being able to work from home or work, Isolation is definitely something which makes completing the course and assignments much harder.  This article reiterated that need for interaction amongst peers to minimise this impact of isolation.