Concept of Variance, Partitioning of variance, Source of error variance. Control techniques

 Concept of Variance, Partitioning of variance, Source of error variance. Control techniques

Research in common parlance referred to as search for knowledge. It can also be defined as a scientific and systematic search for pertinent information on a specific topic. In fact, research is an art of scientific information. Redman and Mory defines research as a “systematized effort to gain new knowledge.”Some people consider research as a movement from the known to the unknown. According to Clifford Woody research comprises defining and redefining problems, formulating hypothesis or suggesting solution; collecting, organizing and evaluating data; making deductions and reaching conclusions; and at last carefully testing the conclusion to determine whether they fit the formulating hypothesis. Research is, thus, an original contribution to the existing stock of knowledge making for its advancement. It is the persuit of truth with the help of study, observation, comparison and experiment. In short, the search for knowledge through objective and systematic method of finding solution to a problem is research. As such the term ‘research’ refers to the systematic method consisting of enunciating the problem, formulating a hypothesis, collecting the facts or data, analyzing the facts and reaching certain conclusions either in the form of solutions(s) towards the concerned problem or in certain generalizations for some theoretical formulation.

The purpose of research is to discover answers to questions through the application of scientific procedures. The main aim of research is to find out the truth which is hidden and which has not been discovered as yet.

Every science has goals. In physics, the goals are concerned with learning how the

physical world works. In astronomy, the goals are to chart the universe and understand both

how it came to be and what it is becoming.

 The goals of psychologist conducting basic research are to describe, explain, predict

and control behavior.

 Description: What is Happening?

 Explanation: Why is it Happening?

 Prediction: When Will it Happen Again?

 Control: How can it be Changed?

RESEARCH DESIGN

MEANING OF RESEARCH DESIGN

Research design is the plan, structure and strategy of investigation conceived so as to obtain answers to research questions and to control variance. The plan is the overall scheme or the program of the research. It includes an outline of what the investigator will do from writing the hypothesis and their operational implications to the final analysis of data. Structure of the research is outline of the research design, and the scheme is the paradigm of operation of the variable. Strategy includes the methods to be used to gather and analyze the data. In other words, strategy implies how the research objective will be reached and how the problems encountered in the research will be tackled. (Kerlinger, 2007)

 A traditional research design is a blueprint or detailed plan as to how a research study is to be completed. That is, how it would operationalise variables so that they can be measured, how to select a sample of interest to the research topic, how to collect data to be used as a basis for testing hypothesis, and how to analyze the results. (Thyer, 1993)

FUNCTION OF A RESEARCH DESIGN

The function of a research design is:

 To provide answer to research question and 

 To enable the researcher to answer question as validly, accurately and as economically as possible.

CONCEPT OF VARIANCE

Variance is a measure of the dispersion or spread of a set of scores. It describes the extent to which the scores differ from each other. Variance and variation, though used by synonymously, are not identical terms. Variation is a more general term which includes variance as one of the statistical methods of representing methods. The main technical function of research design is to control variance. Research design is a set of instructions to the investigator together analyze data in certain ways. Therefore, research design acts as control mechanism and enables the researcher to control unwanted variances. Variance control is a central theme of research design.

PARTITIONING OF VARIANCE

The researcher is directly concerned with three types of variance namely experimental variance, extraneous variance and error variance. Main functions of research design are to maximize the effect of systematic variance, control extraneous variance and minimize error variance. A discussion of these variances is presented below.

SYSTEMATIC VARIANCE: By constructing an efficient research design the investigator attempts to maximize the variance of the variable of substantive research hypotheses. Systematic variance is the variability in the dependent measure due to the manipulation of the experimental variable by the experimenter. An important task of the experimenter is to maximize this variance. This objective is achieved by making the level of the experimental variable as unlike as possible. Suppose an experimenter is interested in studying the effect of intensity of light on visual acuity. The experimenter decides to study the effect by manipulating three levels of light intensity, i.e. 10ml, 15ml, 20ml. as the differences between any two levels of the experimental variable is not substantial, and there is little chance of separating its effect from the total variance. Thus, in order to maximize systematic variances, it is desirable to make the experimental conditions (levels) as different as possible. In this experiment it would be appropriate, then to modify the levels of light intensity to 10ml, 20ml, 30ml so that the difference between any two levels is substantial.

EXTRANEOUS VARIANCE

Extraneous variance is produced by the extraneous variables or the relevant variables. An experimenter always tries to control the relevant variables and thus, also wants to eliminate the variances produced by these variables. For elimination of extraneous variance it is essential that the extraneous variables be properly controlled. There are four ways to control the extraneous variances.

1. Randomization: It is considered to be the most effective way to control the variability due to all possible extraneous sources. It is a procedure for equating groups with respect to secondary variable. Randomization means random selection of the experimental units from the larger population. Random assignment means that every experimental unit has an equal chance of being placed in any of the treatment conditions or group. In using randomization method some problems may be encountered. It is possible to select a random sample from a population, but then assignment of experimental units to groups may get biased. Random assignment of subjects is critical to internal validity. If subjects are not assigned randomly, confounding may occur.

 Randomized group design and randomized block design are the examples of research design in which randomization is used to control the extraneous variable.

2. Elimination: this procedure is the easiest way to controlling the unwanted extraneous variable through elimination of variable. Suppose, the sex of the subject as unwanted secondary variable, is found to influence the variable in an experiment. Therefore the variable of sex has to be controlled. The researcher may decide to take either all males and all females in an experiment and thus, controlled through elimination the variability due to the sex variable.

 By using elimination for controlling the extraneous variables, researcher looses the power of generalization. If the researcher selects the subject from a restricted range then the researcher can generalize the results within restricted range and not outside it. Elimination procedure is used in non-experimental design.

3. Matching: is also a non experimental design procedure, is used to control the extraneous source of variance. In case of controlling organismic and background variable matching is used in this procedure, the relevant variable are equated or held constant across all conditions of experiments. Suppose if the researcher finds that the variable of intelligence is highly correlated with the dependent variable, it is better to control the variance through matching on the variable of intelligence. However as amethod of control matching limits the availability of subjects. If the researcher decides to match subjects on two or three variables he may not find enough subjects for the experiment. Besides this the method of matching biases the principles of randomization.

4. Statistical Control: in this approach, no attempt is made to restrain the influence of

secondary variables. In this technique, one or more concomitant secondary variables (covariates) are measured and the dependent variable is statistically adjusted to remove the effect of the uncontrolled sources of variation. Analysis of covariances is one such technique. It is used to remove statistically the possible amount of variation in the concomitant secondary variable.

ERROR VARIANCE

The third function of a research design is to minimize the error variance. The error variance is defined as those variance or viabilities in the measures, which occurs as a function of the factors not controllable by the experimenter. Such factors may be related to the individual differences among the subjects themselves such as to their attitude, motivation, need, ability etc. They may be related to what is commonly called the errors of measurements such as the differences in trials, differences in conditions of experiment, temporary emotional state of the subject, fatigability etc.

 Statistical control can be applied to minimize such error variance. For example, repeated measures design can be used to minimize the experimental error. By this technique the variability due to the individual differences is taken out from the total variability, and thus, the error variance is reduced. Analysis of covariance is also a technique to reduce the error variance. Further, error variance can be controlled by increasing the reliability of measurement by giving clear and unambiguously instructions and by using a reliable measuring instrument etc.

SOURCES OF ERROR VARIANCE

 Poorly designed measurement instruments ( instrumental error)

 Error emanating from study subjects ( e.g., response error)

 Contextual factors that reduce a sound/ accurate measurement instrument’s capacity to measure accurately

 

CONTROL TECHNIQUES

Controlling variance of confounding variables:

 It can produce undesirable variation in the study’s dependent variables, and cause misleading or weird results

 In Experimental Settings :

1. Conducting the experiment in a controlled environment ( e.g., laboratory), where we can hold values of potential confounding variables constant

2. Subject selection (e.g., matching subjects in experiments)

3. Random assignments of subjects ( variation of confounding variables are evenly distributed between the experimental and control group)

 In Survey Research:

1. Sample selection (e.g., including only subjects with appropriate characteristics – using male college graduates as subjects will control for potential confounding effects of gender and education)

2. Statistical Control

CONCLUSION

There are several research designs and the researcher must decide in advance of collection and analysis of data as to which design would prove to be more appropriate for his research project.

An Effective research design is a function of - 

1) Adequate variability in values of research variables 

2) Precise and accurate measurement 

3) Identifying and controlling the effects of confounding variables and 

4) Appropriate subject selection

REFERENCE

1. Kerlinger, F N (1986). Foundations of Behavioural Research. New York: Holt Rinehart and Winston.

2. Thyer, B.A. (1993) ‘ Single-systems Research Design’ in R.M. Grinnell (ed), Social Work, Research and Evaluation ( 4th ed), Itasca Illionois:Peacock

3. Kothari, C.R. (2004). Research Methodology: Methods and Techniques. New Delhi: New Age International Publishers.

Small N Designs

Small N Designs

Single Subject Research Design (SSRD) involves studying a single individual or system by taking repeated measurements of 1 or more dependent variables and systematically applying & sometimes, withdrawing or varying the independent variable.

Small N Designs

• Designs that involve a very small number of participants

• Rather than reporting measures of central tendency, focus is placed on observations of individual scores/behaviors

Practical Reasons for doing Small N Research

• Procedures are costly / time consuming

• Procedures are invasive

• Procedures require intensive training

• Participants are rare / hard to find

Single-Subject Designs

• Goal: Behavior of a single participant must be shown to change as a result of the treatment

• Three Components

1) Target behavior must be operationally defined

2) Baseline level of responding must be established

3) Conduct treatment and monitor behavior 

• A-B Designs (baseline → treatment)

• Withdrawal Designs

– A-B-A Design

– A-B-A-B Design

Flood et al. (2002)

Study of ADHD Treatment Efficacy

• Participants: 3 10-year olds previously diagnosed with ADHD, not on meds

• IV: Treatment (single level)

– Ss paired with non-ADHD peer who praises Ss for “on task” behavior and prompts Ss after “off task” behavior

• DV: “Off task” behavior

– Looking away from assigned task for 3 seconds

Multiple Baseline Design

• Three varieties of Multiple Baseline Designs

• Baselines are established for:

(1) 1 type of behavior in 2+ individuals in 1 setting

(2) 2+ types of behaviors in 1 individual in 1 setting

(3) 1 behavior in 1 individual across 2+ settings

Wagaman et al. (1993)

• Multiple Baselines Across Subjects

– (1) 1 behavior in 2+ individuals in 1 setting 

• Participants: 8 school children

• IV: “Regulated breathing” treatment

• DV: Stuttering

Criticisms of Small N Designs

• Low external validity

• How do we handle this criticism?

– Evaluate generalizabilty within the design

– Replicate and extend the study 

Ethical Considerations

• Withdrawing a treatment that has changed behavior for the good

• Treatments can be controversial

• Case of special needs children

– Benefits of treatment

TYPES OF EXPERIMENTAL DESIGN

 TYPES OF EXPERIMENTAL DESIGN 

Three types of experimental designs 

A. BETWEEN-SUBJECTS DESIGN: - Different groups of subjects are randomly assigned to the levels of the independent variable.

B. WITHIN-SUBJECTS DESIGNS: - Only one group of subjects; subjects receive all levels of the independent variable at different times.

C. SINGLE-SUBJECT DESIGNS: - Use the same method of varying the level of the independent variable used by the with-in subjects designs. - Focus on change of individual subjects under the different treatment conditions.

WITHIN-SUBJECTS DESIGNS (repeated measures) 

The comparison of treatment effects involves looking at changes in performance within each subject across treatments. - Expose a single group of subject to all the treatments

TYPES OF WITHIN-SUBJECTS DESIGNS (family of designs)

1. SINGLE-FACTOR TWO-LEVEL DESIGN: 2 levels of a single independent variable. All subjects receive both levels of the variable. - If dependent variable not strongly affect by subject- related variable design will be less effective.

2. SINGLE-FACTOR MULTILEVEL DESIGNS: - More than two levels of the independent variable - Single group of subject is exposed to three or more levels of a single independent variable.

3. MULTIFACTOR WITHIN-SUBJECTS DESIGNS: Includes two or more independent variables

Factorial Designs: Each subject is exposed to every combination of levels of all the factors (independent variable) - Have main effects and interactions.

4. OTHER WITHIN-SUBJECTS DESIGNS: Nonfactorial design

5. MULTIVARIATE WITHIN-SUBJECTS DESIGNS: Use of more than one dependent variable

WHEN TO USE A WITHIN-SUBJECTS DESIGN:

1. SUBJECT VARIABLES CORRELATED WITH THE DEPENDENT VARIABLE: - Use when subject differences contribute heavily to variation in the dependent variable.

2. ECONOMIZING ON SUBJECTS: - Use when # of subjects is limited and carryover absent or minimized.

3. ASSESSING THE EFFECTS OF INCREASED EXPOSURE ON BEHAVIOR: - Measured a number of trials, passage of time, etc. - Looking at changes as a function of earlier exposure

A. ADVANTAGES: - Close to matched groups. Within provides the ultimate in matching of characteristics. - Tends to be more powerful than equivalent between-subject design. - Increase power may allow for use of less subjects. 

B. DISADVANTAGES: - Amount of time in the experiment - Carryover effects

C. SOURCES OF CARRYOVER EFFECTS: Potential sources for carryover effects 

1. Learning: if learn a task; second performance is likely to be better if similar.

2. Fatigue: may lead to deterioration in later performance.

3. Habituation: reduction in responsiveness for repeated exposure.

4. Sensitization: exposure to one stimulus can cause subjects to respond more strongly to another stimulus.

5. Contrast: exposure to one condition may alter responses in other conditions

6. Adaptation (adjustments): may lead to different results due to adaptive changes (e.g., drug tolerance) 

D. DEALING WITH CARRYOVER EFFECTS: Three ways to deal with carryover effects. 

1. COUNTERBALANCING: assign various treatments of the experiment in a different order for different subjects.

a) COMPLETE COUNTERBALANCING: every possible ordering of treatments assign at least 1 subject to each ordering.

1) Every treatment follows every other treatment equally often.

2) Every treatment appears equally often in each position. - Minimum number of subjects is equal to the number of different ordering of the treatments. - k treatments have k! (k factorial). 3 x 2 x 1 = 6;

For four treatments: 4 x 3 x 2 x 1 = 24

b) PARTIAL COUNTERBALANCING: include only some possible treatment orders. Orders retained are chosen randomly from total set. Latin Square Design: partially-counterbalanced design - Number of treatment order equivalent to number of treatments. - Each item appears exactly once in each column and row.

2. TAKING STEPS TO MINIMIZE CARRYOVER: if minimize the error variance – increases power - Pre-train subject - Allow breaks

3. MAKING TREATMENT ORDER AN INDEPENDENT VARIABLE: Order of treatment as a second independent variable.

BETWEEN-SUBJECTS DESIGNS (Family of Designs)

SINGLE-FACTOR RANDOMIZED GROUPS DESIGNS

Randomized Group Design: randomly assign subjects to level of independent; variable to form “groups” of subjects

1) THE RANDOMIZED TWO-GROUP DESIGN: 

+ Randomly assign subjects to two groups + Expose each to different levels of the independent variable. 

ADVANTAGES: simple to carry out 

1. Economical in terms of time and material. 

2. No Pre-testing of subjects in necessary 

DISADVANTAGES: 

1. Provides limited amount of information about effects of independent variable. 

2. Sensitivity:

2) THE RANDOMIZED MULTIGROUP DESIGN 

1. Single-Factor Parametric Design: assess independent variable at more than two levels.

2. Single-Factor Nonparametric Design: Nonparametric Design - nominal scale

3. Multiple Control Group Design: - Include multiple control groups when a single control group is not adequate to rule out alternative explanations of your results.

Small-N designs

Single Subject Research Design (SSRD) involves studying a single individual or system by taking repeated measurements of 1 or more dependent variables and systematically applying & sometimes, withdrawing or varying the independent variable.

Pseudo-experiment

It is a research design in which someone tests a claim about a variable (e.g., a product, a charm, a clinical treatment) by exposing people to the variable of interest and noting that these people think, feel, or behave as expected. People often treat these as if they were true experiments.

CONTROL TECHNIQUES

 CONTROL TECHNIQUES

Controlling variance of confounding variables:

In Experimental Settings :

1. Conducting the experiment in a controlled environment ( e.g.,laboratory),

2. Subject selection (e.g., matching subjects in experiments)

3. Random assignments of subjects ( variation of confounding variables are evenly distributed between the experimental and control group )

? In Survey Research:

1. Sample selection (e.g., including only subjects with appropriate characteristics – using male college graduates as subjects will control for potential confounding effects of gender and education)

2. Statistical Control

Moderating

In each relationship there is one Independent Variable (IV) & one Dependent Variable (DV)

Four day work week (IV) will lead to higher productivity (DV).

Moderating variable is a second independent variable that has significant effect on the originally stated IV–DV relationship.

Four day work week (IV) will lead to higher productivity (DV), especially among young workers (MV)

Some of the points you can also refer from this links:

https://books.google.co.in/books?id=h9KY83C0K5AC&pg=PA89&lpg=PA89&dq=Control+techniques-+Elimination,+Constancy&source=bl&ots=06kP2QDHfC&sig=ACfU3U0wNy8ID0gAxVye8ejOXY7dAAFgMQ&hl=en&sa=X&ved=2ahUKEwjq-YOGju7mAhVu4jgGHemKCHE4ChDoATACegQIBxAB#v=onepage&q=Control%20techniques-%20Elimination%2C%20Constancy&f=false

Randomization

It is a statistical control technique designed to ensure that extraneous variable will not systematically bias the study results; 

It is done by equating groups of participants-ensuring that every member has an equal chance of being assigned to any group.

Eg. Randomized group design and randomized block design

Randomization: an important method of controlling extraneous variables is randomization. It is considered to be the most effective way to control the variability due to all possible extraneous sources. If through randomization has been achieved then the treatment groups in the experiment could be considered statistically equal in all possible ways. Randomization is a powerful method of controlling variable. In other words it is a procedure for equating groups with respect to secondary variable. Randomization means random selection of the experimental units from the larger population. Random assignment means that every experimental unit has an equal chance of being placed in any of the treatment conditions or groups. In using randomization method some problems may be encountered. It is possible to select a random sample from a population, but then assignment of experimental units to groups may get biased. Random assignment of subjects is critical to internal validity. If subjects are not assigned randomly, confounding may occur.

Randomized group design and randomized block design are the examples of research design in which randomization is used to control the extraneous variable.

Statistical control:

Statistically controlling for extraneous variables is an option for removing the influence of the variable on the study design.

Evaluators need to collect data on the extraneous variables, as well as the independent and dependent variables for analysis. 

Evaluators can use a range of statistical techniques for control including:

General Linear Model

Analysis of covariance

Blocking

blocking is the arranging of experimental units in groups (blocks) that are similar to one another.

Blocking reduces unexplained variability.

Blocking used for nuisance factors that can be controlled. (reduce exptl error)

Examples

Male and Female:

Elevation --high or low

Intervention

Randomized block design

The general rule is:

“Block what you can; randomize what you cannot.”

Matching

It is used to control the extraneous source of variance.

Matching is a statistical technique which is used to evaluate the effect of a treatment by comparing the treated and the non-treated units in an experiment.

Matching enables a comparison of outcomes among treated and non-treated units to estimate the effect of the treatment reducing bias due to confounding.

Variance, Partitioning of variance, Sources of error variance

Important concepts- Variance, Partitioning of variance, Sources of error variance.

Kerlinger (1986) conceptualized experimental design as variance control.

Variance

Variance is the measure of the dispersion or spread of a set of scores. It describes the extent to which the scores differ from each other.

See the attachment file for practicing how to find out variance....

--measures of dispersion (spread of scores)

--statistical methods of data representation.

--Research design --variance is used for to control variance

--Researcher is analyze data to control unwanted variance.

--So, Variance control is the main theme in research design.

PRACTITIONING OF VARIANCE

--A researcher is used with 3 types of variance.

1.Experimental variance

2.Extraneous variance

3.Error variance

The main function of research design is MAXMINCON principle.

MAXIMIZE the effect of systematic variance

MINIMIZE error variance

CONTROL extraneous variance

1.Experimental variance (Systematic Variance)

--a researcher is maximize the variance of the variables related to the hypothesis.

--ie., variability in dependent variable measure to manipulate IV.

--So the experimenter MAXIMIZE variance

eg: Effects of intensity of light on visual acuity.

-manipulating three levels of light intensity i.e., 10ml, 15ml, 20ml.

2.Extraneous variance

--it is extraneous variables or relevant variables that   the experimenter tries to control relevant variables and eliminate variance produced by these variables.

-- so it requires proper CONTROL of extraneous variables.

--controlling techniques

-Randomization: random selection of exp. and con. group from population..equal chance of being represented.  eg, Randomized group design and randomized block design

-Elimination: avoiding secondary variable influence

-Matching: relavant variables

-Statistical Control: control secondary variable influence ---using analysis of covariance.

3. Error Variance:

--minimize error variance.

--is used when factors are not controlled by experimenter

-sources: attitude, motivation, need, ability etc...... differences in trials, conditions of experiment....emotional state of subject.... fatigability etc.

--statistical control is used minimize error variance

eg. repeated measures design (reduce experimental error)

    Analysis of covariance technique (to increase reliability_

SOURCES OF ERROR VARIANCE

? Poorly designed measurement instruments ( instrumental error)

? Error emanating from study subjects ( e.g., response error)

? Contextual factors that reduce a sound/ accurate measurement instrument’s

capacity to measure accurately

Experimental Research Design

 Experimental Research Design

Research Design- Introduction

RESEARCH DESIGN

 DEFINITION

 According to William Zikmund, “Research design is defined as a master plan specifying the methods and procedures for collection and analyzing the needed information.”

 According to Kerlinger, “Research design is the plan, structure, and strategy of investigation conceived so as to obtain answer to research questions and to control variance.”

 

IMPORTANCE OF RESEARCH DESIGN

 It reduces inaccuracy.

 Helps to get maximum efficiency and reliability.

 Eliminates bias and marginal errors.

 Minimizes wastage of time.

 Helpful for collecting research materials.

 Gives an idea regarding the type of resources required in terms of money, manpower, time, and efforts.

 Guides the research in the right direction

A research design may be regarded as the blueprint of those procedures which are adapted by the researcher for testing the relationship between the dependent variable and the independent variable. There are several kinds of experimental designs and the selection of any one is based upon the purpose of the research, types of variables to be controlled and manipulated as well as upon the conditions under which the experiment is to be conducted. 

The main purpose of experimental design is to help the researcher in manipulating the independent variables freely and to provide maximum control of the extraneous variables so that it may be said with all certainty that the experimental change is due to only the manipulation of the experimental variable. 

The main function of a research design is to explain how you will find answers to your research questions. The research design sets out the logic of your inquiry. It includes the study design and the logistical arrangements that you propose to undertake as well as the measurement procedures. It includes also the sampling strategy, the frame of analysis and the time frame. For any investigation the selection of an appropriate research design is crucial to enable the researcher to arrive at valid findings, comparisons and conclusions. 

According to Thyer (1993) a traditional research design is a blueprint or detailed plan for how to conduct a research study and how to complete the same. Planning such a research design involves, (i) operationalising variables so that they can be measured, (ii) selecting a sample of interest to study, (iii) collecting data to be used as a basis for testing hypothesis, and (iv) analysing the results.

According to Matheson (1970) a research design is a basic plan for research, including the assignment of subjects to the levels of the independent variable and the manipulation of the independent variable.

According to Kerlinger (1986) research design is the plan, structure, and strategy of investigation conceived so as to obtain answers to research questions and to control variance.

The definition of Kerlinger reveals three important components, which are 

(i) research design is a plan 

(ii) research design is the structure 

(iii) research design is the strategy. 

Let us see what these are:

i) Research Design is the Plan: The plan is the overall scheme or program of the research. It includes an outline of what the investigator will do from writing the hypotheses and their operational implications to the final analysis of data.

ii) Research Design is the Structure: The structure of the research is more specific. It is the outline, the scheme, the paradigm, of the operation of the variables. When we draw diagrams that outline the variables and their relation and juxtaposition, we build structural schemes for accomplishing operational research purposes.

iii) Research Design is the Strategy: Strategy as used here is also more specific than plan. It includes the methods to be used to gather and analyse the data. In other words strategy implies how the research objectives will be reached and how the problems encountered in the research will be tackled.