LEAD 549 Midterm Quiz Diyang Yu Example and describe quasi-experimental design in the real world
LEAD 549 Midterm Quiz Diyang Yu
Example and describe quasi-experimental design in the real world, organizational setting.
Example: a researcher designed a study of whether metacognition training program could help high school students to improve the ability of math problem solving.
Participants: the research subjects are come from the high school students and same class. Totally 50 participants joined into study.
At the beginning, the researcher sent out a math test and graded. Then, the researcher separates these participants into two groups, and marked experimental group’s average grade as O11, comparison group’s average grade as O12.
For the experimental group, they were asked to take the metacognition training program (marked as X) and combined with the math question practices, while the rest of the students, which as comparison group, only did the math question practices.
After one week, researcher called all the participants to take another test, which was different from the first test, but they examined the material that includes what participants study in this experiment week. The experimental group’s average grades were marked as O21, the comparison group’s average grades were marked as O22.
Comparing experimental group’s grade O11 and O12, also comparing the comparison group’s grade O21 and O22. The conclusion was conducted depending on the percentage increases in average score of each group.
The study results show that the students who take the metacognition training program performed better than the students who did not take the training program.
What is the dependent variable?
The dependent variable in this study is “the math test grades”. Because it changes followed by the changes of independent variables.
What is independent variable?
The independent variable in this study is “taking the metacognition training program”. The independent variable can be controlled by researcher because he has rights to assign it to either two groups.
Can causality be inferred? Why or why not?
Yes, but not accurate. There are some concerns of this research but the most important one is because the participants were not randomly assigned into each group, it could impact the results.
Describe and explain 4 criteria for causation and pro/cons of the following research designs with respects to these 4 criteria:
Aims to identify the relationship between two variables:
An association of the dependent variables and independent variables happens when a change in one variable must accompanied by a change in another.
There will be a correlation between variable X and variable Y, when X change, Y must change.
The experiment has control group and random assignment between groups.
Random assignment to ensure the equivalent in all respects at the outset of the experiment. Random assignment helps that neither group’s aggressiveness nor any of their other characteristics could influence the experiment results.
Program manager controls the intervention variables.
The intervening variables used to explain the causal links between other variables. It cannot be observed because it is hypothetical.
For example: there is an association between working hard in study and having good grades. But just because someone is work hard does not enough that will lead to academic success. So other hypothetical variables are used to explain the phenomenon. Such as IQ level.
Avoid the threats to internal validity:
Threats to internal validity challenge whether the data collected can support the assumption that the experiment worked. To ensure the experiment are credible, the must try to avoid any alternative explanation for the observed changes seen after experiment.
One-shot case study
It is the primitive research design.
One-shot case study is all we know that what happened and what we observed;
an intervention occurs, and an observation is madePros: good for identifying problems for future research
Cons: no control for internal validity.
Pre-experiment designs for descripting research. The pre-experimental design often used to initiate exploratory research to see whether need to implement full scale research.
Pre-experiments can be an effective way to discern whether a further experiment is worth to explore.
pre-experiment does not have control group (single group is studied but cannot make comparison with an equivalent non-treatment group).
Pre-experiment design are threatened by lack of validity because it can be uncertain if any of effects caused by hypotheses.
Study the variations in a specific outcome because the researcher controls the variables.
Researcher can have control variables
Experimental research can conduct results that relevant with consistency.
Randomly assign variables
Research could create unrealistic situation because the variables can be controlled that can lead results toward a favorable result.
when quasi-experiment used by true experimental design, it does not have time and logistical limitations.
Threats to validity can be identified and addressed in order to reduce the impact.
The researcher can control the variation of experimental research while still maintaining the validity.
The experiment lacks randomization and threats the internal validity, the conclusion is not stable, and the data analyses may not be meaningful.
Explain the difference between nominal, ordinal, and continuous variables and give an example for each.
The most basic variables. It has two criteria: exhaustiveness and mutual exclusivity. It cannot be mathematically quantified.
Example: to measure the gender, the researcher might ask participants to identify if they are male, female or others. These three attributes exhaust the possibilities for gender, and normally a person cannot identify more than one.
Ordinal variables can be ranked in order. It also has two criteria: exhaustiveness and mutual exclusivity. The distance between those attributes cannot be mathematically calculate. But we can measure one attribute is more or less than another attribute.
Example: to measure whether people preferred to drink iced soda or eat ice-cream in hot weather, we can say that people are more likely to drink soda than eat ice cream, but we cannot say exactly how much more.
Continuous variables can be ranked in order. It also has two criteria: exhaustiveness and mutual exclusivity. The distance between attributes can be mathematically calculate, which is equal, and attributes have a true zero point. The comparison between two attributes can be measured.
Examples: to measure the age of people who eat in restaurant.
Define and give an example of reliability and validity
Reliability is about consistency. When a measure is reliable, the same measure is applied to the same subject and the result will be the same each time.
When a person tries to weigh himself on a scale. A reliable scale will show the same reading over and over, no matter how many times he weighs.
Validity is about whether a test measures accurately what it claims to measure.
The GMAT test use quantitative section to measure people’s ability to reason quantitatively and interpret data, but not use this section to measure people’s ability to read and comprehend written material because Quantitative section has no validity as a measure of verbal ability.
Define and give an example of each of the following four sampling methods:
Is probability sampling method. Researcher picks clusters randomly and selects elements from each cluster randomly.
When a researcher wants to measure the satisfaction rates of university cafeteria in San Diego. First the researcher will be looking for a list of all universities in San Diego, then the researcher could draw a random sample of universities. At the end, the researcher could draw a random sample of school cafeterias from within the libraries he initially picked.
Convenience or Availability
Is nonprobability sampling method. Used by both qualitative and quantitative researchers.
The researcher gathers data from people or place to which the researcher has most convenient to get.
The researcher can measure the student’s satisfaction rates of school cafeteria by taking a brief interview of students who is eating on campus cafeteria.
The benefit of this sampling method is convenience.
Purposive or Judgmental
Is nonprobability sampling method.
the researcher looks for the participants who cover the full range of perspectives that the researcher wants to examine.
Researchers are using judgement to pick the participants. Such as the researcher would like to pick participants who has meal plan in order to measure the cafeteria satisfaction rates, rather than pick participants who lives outside the campus.
The purpose is to find someone who has specific experience related with study direction.
A researcher might to know one or two people who would like to participant in researcher’s study, and then the initial participants help to identify more participants who can join into the study.
It is useful when the participants are hard to find. Such as the researcher wants to study on the status quo of the people who infected with HIV in college campus. The researcher would not be likely to find many participants at the beginning, but after the researcher might know someone who HIV positive, and then be referred by interviewee to another potential participants.
What are the 4 types of measurement validity and how can they be assessed?
When something appears to be valid, face validity occurs. This type of measurement depends much on the judgement of observer, but insufficient and requires another solid validity to make the conclusion.
How to assess:
The measure often starts out with face validity.
Example: math test is not valid to measure drawing skills
Content validity needs to adequately reflects the full range of material, which is relevant for the concept of measures.
How to assess:
For example, when a teacher designing the final test, the test is validity if the final test will represent all the subjects actually taught to students, rather than asking the last two chapters.
Concurrent validity is a measure of a particular test correlates with a previously validated measure.
How to assess:
The test is for the same, or very closely related and allow researcher to validate new methods.
For example, an instrument has concurrent validity when it measures the students’ study hours correlates highly with the test score.
Predictive validity is a measure can predicts future outcome.
How to assess:
For example: the university use high-school GPA as one reference to decide which students to accept in order to ensure the students are smart and dedicated. All this process is based on the assumption that the student who has high GPA is more likely to achieve high grades than those who has low GPA.