Quantitative Research Operations Flashcards

Question 1

What is the difference between a descriptive, correlational, and experimental study?

Accepted Answer

A descriptive study documents the characteristics of a population or phenomenon without trying to explain relationships. A correlational study measures how two or more variables move together without manipulating either one. An experimental study assigns conditions deliberately and observes outcomes, which allows stronger claims about cause and effect when randomization and control are sound.

Question 2

Why does a researcher write a hypothesis before collecting any data?

Accepted Answer

A hypothesis is a testable statement about the expected relationship between variables, written before data collection so that the analysis plan is fixed in advance. This discipline limits the temptation to search for patterns after seeing results and to report whichever findings look best.

Question 3

What is the purpose of a power analysis in study planning?

Accepted Answer

A power analysis estimates the sample size needed to detect an effect of a given size with a specified probability, given a chosen significance level. Its purpose is to prevent studies that are too small to find real effects, which waste resources, and to avoid recruiting far more participants than the question requires.

Question 4

What does internal validity mean, and why does it matter?

Accepted Answer

Internal validity is the degree to which a study supports a causal conclusion about the relationship it tests, free from confounding explanations. It matters because a result that cannot be attributed to the intended variable is of limited use, even when the numbers look precise. Threats include selection bias, history, maturation, and attrition.

Question 5

What is a confounding variable, and how can a design reduce its influence?

Accepted Answer

A confounding variable is an outside factor related to both the predictor and the outcome, creating a misleading association between them. Designs reduce its influence through random assignment, matching, restriction of the sample, or statistical adjustment, although random assignment is usually the most powerful safeguard in experiments.

Question 6

What is the difference between a cross-sectional and a longitudinal design?

Accepted Answer

A cross-sectional design collects data from a population at one point in time, offering a snapshot of prevalence or association. A longitudinal design follows the same participants or units across multiple time points, allowing researchers to observe change within individuals and to examine the order in which events occur.

Question 7

Why is a pilot study useful before a full quantitative project begins?

Accepted Answer

A pilot study is a small trial run of the procedures, instruments, and logistics planned for the main study. It is useful because it reveals confusing survey items, unrealistic recruitment rates, equipment problems, and timing issues while changes are still cheap. Pilot effect estimates are unstable, so they should not be treated as final.

Question 8

What role does a preregistered analysis plan play in quantitative research?

Accepted Answer

A preregistered analysis plan is a time-stamped record of hypotheses, variables, exclusion rules, and statistical tests written before data are examined. It helps separate confirmatory findings from exploratory ones, making it harder for flexible decisions to inflate apparent support for a claim. Exploratory analyses remain valuable when clearly labeled as such.

Question 9

What is the difference between a population and a sample?

Accepted Answer

The population is the complete group about which a researcher wants to draw conclusions, while the sample is the subset actually measured. Inference moves from the sample back to the population, so the sample must be selected carefully enough to represent the population's relevant characteristics.

Question 10

How does simple random sampling differ from convenience sampling?

Accepted Answer

Simple random sampling gives every member of the defined population a known, equal chance of selection, which supports generalization when the sampling frame is accurate. Convenience sampling selects whoever is easiest to reach, so the probability of selection is unknown and results may not represent the broader population.

Question 11

What is a sampling frame, and why can errors in it bias results?

Accepted Answer

A sampling frame is the list or mechanism from which sample members are drawn, such as a roster, directory, or database. Errors in the frame bias results when some population members are missing, duplicated, or ineligible, because the sample then reflects the frame rather than the target population.

Question 12

What is nonresponse bias, and how can researchers assess it?

Accepted Answer

Nonresponse bias occurs when people who do not respond differ systematically from those who do, so the observed estimates misrepresent the population. Researchers assess it by comparing respondents and nonrespondents on available characteristics, following up with a sample of nonrespondents, or examining whether estimates change across early and late responders.

Question 13

What is a codebook, and what should it contain?

Accepted Answer

A codebook is a document that defines every variable in a dataset, including its name, label, type, allowed values, coding scheme, and missing-data conventions. It should also record the source question, the units, and how any derived variables were computed, so anyone can interpret the data correctly long after collection.

Question 14

Why should data collection procedures be standardized across interviewers or sites?

Accepted Answer

Standardization ensures that every participant experiences the same wording, instructions, timing, and conditions, so differences in responses reflect real differences rather than differences in how data were gathered. Without it, interviewer effects and site variation can masquerade as substantive findings, and the study becomes difficult to replicate or combine with others.

Question 15

What is a response rate, and why do journals and reviewers care about it?

Accepted Answer

The response rate is the proportion of sampled units that provide usable data, calculated by a stated formula. Reviewers care about it because low rates raise the risk of nonresponse bias, though a high rate does not guarantee representativeness, so reports should describe who responded and how the sample compares with the target population.

Question 16

What is reliability, and how is it different from validity?

Accepted Answer

Reliability is the consistency of a measurement, meaning that repeated administrations under similar conditions yield similar scores. Validity concerns whether the instrument measures the construct it claims to measure. A measure can be highly reliable yet invalid, for example a bathroom scale that consistently reads five pounds too heavy.

Question 17

What does Cronbach's alpha estimate, and what are its limits?

Accepted Answer

Cronbach's alpha estimates internal consistency, the extent to which items in a multi-item scale vary together, and is often used as a rough index of reliability. It depends on item count and average inter-item correlation, so a high value does not prove the scale is unidimensional or valid.

Question 18

What is content validity, and how do researchers establish it?

Accepted Answer

Content validity is the extent to which the items of an instrument adequately sample the full range of the construct being measured. Researchers establish it through systematic review by subject-matter experts, clear conceptual definitions, and mapping each item to a domain of the construct, often before any statistical testing begins.

Question 19

Why is translating an instrument into another language more involved than simple word-for-word conversion?

Accepted Answer

Translation must preserve meaning, not just vocabulary, because idioms, concepts, and response scales may not carry over directly between cultures. Good practice includes forward translation, back translation, reconciliation by a committee, and cognitive pretesting with members of the target group, followed by checks that the translated measure behaves similarly to the original.

Question 20

What is measurement error, and how does random error differ from systematic error?

Accepted Answer

Measurement error is the difference between an observed value and the true value of what is being measured. Random error fluctuates unpredictably across observations and mainly reduces precision, while systematic error pushes results consistently in one direction and threatens accuracy. Random error tends to average out with larger samples, but systematic error does not.

Question 21

What is a Likert scale, and what are common issues with analyzing its items?

Accepted Answer

A Likert scale presents a series of statements and asks respondents to indicate agreement along an ordered set of categories, such as strongly disagree through strongly agree. Common issues include treating ordinal responses as interval data without justification, combining items without checking their coherence, and ignoring reverse-worded items that must be recoded before scoring.

Question 22

What do cognitive testing and pilot testing of a survey item involve?

Accepted Answer

Cognitive testing examines how respondents understand, recall, judge, and answer a question by asking them to think aloud or explain their interpretation. Pilot testing checks the whole instrument under realistic conditions for timing, skip logic, and completion rates. Both steps reveal ambiguous wording that designers often miss.

Question 23

What is a data management plan, and what should it cover?

Accepted Answer

A data management plan is a written document describing how data will be collected, organized, stored, documented, shared, and eventually preserved or destroyed. It should cover file naming conventions, access permissions, backup procedures, version control, consent restrictions on sharing, and responsibilities for each stage, so that the team works consistently.

Question 24

Why should raw data never be overwritten during cleaning?

Accepted Answer

Raw data should be preserved unchanged because every cleaning decision must be traceable and reversible if errors are later discovered. Cleaning is done through scripts or logged steps applied to a copy, so anyone can reproduce the analysis dataset from the original file. Overwriting the source removes the audit trail and hides errors.

Question 25

What are common methods for handling missing data in quantitative datasets?

Accepted Answer

Common approaches include listwise deletion, which removes cases with any missing value, pairwise deletion, and single imputation such as mean substitution, though these can distort estimates. Multiple imputation generates several plausible completed datasets and pools results, and it generally performs better when data are missing at random. The chosen method should be justified and reported.

Question 26

How should researchers identify and decide how to handle outliers in a dataset?

Accepted Answer

Researchers should first distinguish between values that are impossible or due to entry errors, which may be corrected or removed with documentation, and values that are extreme but plausible, which usually stay in the data. Identification methods include visual plots, z-scores, and interquartile range rules, but any exclusion should be justified before results are seen.

Question 27

What is data provenance, and why does it matter for reproducibility?

Accepted Answer

Data provenance is the documented history of a dataset, recording where it came from, who collected it, how it was transformed, and which versions were used. It matters for reproducibility because another analyst can only recreate published results if they can trace each step from the original source to the final analysis file.

Question 28

How can double data entry or validation rules reduce errors?

Accepted Answer

Double data entry has two people key the same records independently, and a comparison program flags every disagreement for resolution against the source document. Validation rules work at the point of entry by restricting values to allowed ranges, formats, and logical relationships, such as a birth date that cannot come after an enrollment date.

Question 29

What is version control for analysis code, and why use it?

Accepted Answer

Version control is a system that records each change to files over time, allowing authors to compare revisions, recover earlier states, and see who changed what. Using it for analysis code reduces the risk that a published table came from an outdated script, and it supports collaboration by making merges and disagreements visible.

Question 30

Why is documenting every analytic decision important in quantitative operations?

Accepted Answer

Documenting analytic decisions creates a record of what was done, why, and in what order, which allows reviewers and future researchers to judge whether conclusions depend on arbitrary choices. Decisions such as outlier rules, variable transformations, model specifications, and subgroup selections often change results, so hiding them makes findings look more robust than they are.

Question 31

What is statistical power, and how does sample size affect it?

Accepted Answer

Statistical power is the probability that a test will detect an effect when a true effect exists. Larger samples generally increase power because estimates become more precise, while smaller effects, higher variability, and stricter significance thresholds all reduce it. Low power raises the chance of missing real effects.

Question 32

What is the difference between statistical significance and practical significance?

Accepted Answer

Statistical significance indicates that an observed result would be unlikely if no true effect existed, at a chosen threshold. Practical significance asks whether the size of the effect matters in real-world terms. A tiny effect can reach significance in a very large sample, so effect sizes and confidence intervals should be reported alongside p-values.

Question 33

What should a results section include so that readers can evaluate the findings?

Accepted Answer

A results section should report sample characteristics, response rates, descriptive statistics, the estimated effects with confidence intervals, and the tests used, along with any deviations from the preregistered plan. Readers need enough detail to judge precision and to recompute key values, so tables should display the numbers that support each claim, not just asterisks.

Question 34

What ethical obligations apply to quantitative researchers who collect data from people?

Accepted Answer

Researchers must obtain informed consent that explains the purpose, procedures, risks, and the voluntary nature of participation. They must protect confidentiality through secure storage and de-identification, minimize harm, and have the study reviewed by an appropriate ethics board before data collection begins. Participants may withdraw without penalty.

Question 35

What is p-hacking, and what forms does it commonly take?

Accepted Answer

P-hacking refers to flexible analysis choices, such as adding covariates, dropping cases, or testing many outcomes until a p-value falls below a threshold. It inflates false positive rates because each extra attempt raises the chance that a spurious result looks significant. Researchers reduce it by reporting every test conducted, not only those that succeed.

Question 36

What is a confidence interval, and how should it be interpreted?

Accepted Answer

A confidence interval is a range of values constructed by a procedure that, across repeated samples, would contain the true parameter at a stated rate. For any single interval, the true value is either inside or outside it, so the interval is best read as conveying precision and a plausible range.

Question 37

Why must statistical tests be matched to the level of measurement and distribution of the data?

Accepted Answer

Test assumptions describe the conditions under which a test produces valid p-values and interval estimates, such as independence, appropriate scale, and approximate distribution of residuals. Matching measurement level and data structure to the test prevents misleading conclusions, and when assumptions are doubtful, researchers should consider robust or nonparametric alternatives and report the choice.

Question 38

How should operational timelines and budgets be monitored during a quantitative project?

Accepted Answer

Operational monitoring tracks milestones such as recruitment targets, instrument completion, data receipt, and cleaning progress against a documented schedule and budget. Teams usually hold regular status reviews, flag slippage early, and record decisions about scope changes, so that delays are addressed before they threaten data quality or the reporting deadline.

Question 39

What is the difference between exploratory and confirmatory analysis?

Accepted Answer

Exploratory analysis examines data to generate patterns, hypotheses, and questions for future study, often without a fixed plan. Confirmatory analysis tests specific, prespecified hypotheses using a planned model and decision rule. Results from exploration should be reported as hypothesis generating and checked in new data before being treated as established.

Question 40

What is a sensitivity analysis, and why do researchers run one?

Accepted Answer

A sensitivity analysis repeats the main analysis under alternative reasonable assumptions, such as different outlier rules, missing-data methods, or model specifications, to see whether conclusions hold. If results change substantially across defensible choices, the finding is fragile and should be described with that caveat. Stable results across choices increase confidence in the conclusion.

Quantitative Research Operations Flashcards

1Research Design Fundamentals

2Sampling and Data Collection

3Measurement and Instrument Quality

4Data Management and Cleaning

5Analysis, Reporting and Ethics

Learning Resources