Data Mining Operations Flashcards

Question 1

What is data mining in the context of data operations?

Accepted Answer

Data mining is the process of discovering useful, non-trivial patterns, relationships, or predictions in large collections of data using statistical, machine learning, and database methods. It is one step within a broader knowledge discovery process, and it depends on careful preparation and evaluation to produce results that hold up outside the original dataset.

Question 2

What does the acronym CRISP-DM stand for and what does it describe?

Accepted Answer

CRISP-DM stands for Cross-Industry Standard Process for Data Mining, a widely used framework that describes a data mining project as a cycle of six phases. The phases are business understanding, data understanding, data preparation, modeling, evaluation, and deployment, and the process often loops back to earlier phases as new insights emerge.

Question 3

What is the difference between a feature and a target variable?

Accepted Answer

A feature is an input attribute that describes each record, while a target variable is the outcome the model tries to predict or explain. In a dataset of loan applications, income and credit history would be features, and whether the loan defaulted would be the target.

Question 4

What is a record, and how does it relate to attributes in a dataset?

Accepted Answer

A record is a single observation in a dataset, such as one customer, transaction, or sensor reading, and each record is described by a fixed set of attributes. Attributes are the columns that hold the values for those descriptive properties, often shown with one row per record.

Question 5

What is supervised learning, and how does it differ from unsupervised learning?

Accepted Answer

Supervised learning uses labeled examples, where each record includes a known target value, so the model learns a mapping from inputs to that target. Unsupervised learning works with unlabeled data and looks for structure, such as groups or associations, without a predefined outcome to match.

Question 6

Why is it important to separate training data from test data?

Accepted Answer

Separating training data from test data lets you estimate how well a model will perform on new cases it has never seen. If the same records are used for both fitting and testing, the results tend to be overly optimistic, because the model may have memorized noise rather than learned patterns that generalize.

Question 7

What is the purpose of a data dictionary in a mining project?

Accepted Answer

A data dictionary documents each field in a dataset, including its meaning, data type, allowed values, units, and source system. It keeps the team from misreading columns, such as confusing a monthly total with an annual one, and it helps new analysts understand the data quickly.

Question 8

What is a data mining pipeline, and why is it useful to automate one?

Accepted Answer

A data mining pipeline is a sequence of connected steps, such as ingesting data, cleaning it, transforming features, training a model, and scoring new records. Automating the pipeline makes results reproducible, reduces manual errors, and lets the same process rerun whenever new data arrives.

Question 9

What are missing values, and why do they need attention before modeling?

Accepted Answer

Missing values are fields that contain no recorded entry, either because the data was never collected or because it was lost during transfer. Many algorithms cannot process blanks directly, and ignoring them can bias results, so analysts must decide whether to remove, impute, or flag them.

Question 10

What is data normalization, and when is it needed?

Accepted Answer

Normalization rescales numeric attributes to a common range, such as zero to one, so that variables measured on very different scales contribute fairly. It matters most for distance-based methods like k-nearest neighbors and clustering, where a large-valued attribute could otherwise dominate the calculation.

Question 11

What is the difference between data cleaning and data transformation?

Accepted Answer

Data cleaning corrects or removes errors, duplicates, and inconsistencies so the data accurately reflects what it measures. Data transformation reshapes clean data into a form the analysis needs, such as encoding categories, aggregating records, or deriving new features. Cleaning fixes what is wrong, while transformation changes how correct data is represented.

Question 12

How do outliers affect a dataset, and how can they be detected?

Accepted Answer

Outliers are values that sit far from the typical range of an attribute, and they can distort averages, skew models, and inflate error measures. Common detection methods include z-scores, interquartile range fences, and visual checks with box plots or scatter plots, though a flagged value should be investigated before it is removed.

Question 13

How should categorical attributes be encoded for most mining algorithms?

Accepted Answer

Categorical attributes are usually converted to numeric form, most often with one-hot encoding, which creates a binary column for each category. Ordinal categories with a natural order may instead receive integer codes, but arbitrary integer codes for nominal categories can mislead algorithms into assuming a ranking that does not exist.

Question 14

What is data leakage, and why does it produce misleading results?

Accepted Answer

Data leakage occurs when information that would not be available at prediction time sneaks into the training process, often through features or preprocessing steps. Models then appear far more accurate during testing than they will be in real use, because they were effectively given part of the answer.

Question 15

Why should preprocessing steps be fit only on training data?

Accepted Answer

Preprocessing steps such as computing means, scaling ranges, or choosing imputation values should be fit on the training data alone, then applied unchanged to the test data. Fitting them on the full dataset lets information from the test records influence training, which is a common form of data leakage.

Question 16

What role does sampling play when a dataset is too large to process fully?

Accepted Answer

Sampling selects a representative subset of records so that analysts can explore data, test methods, and build models within practical time and memory limits. A random sample preserves overall patterns when it is drawn carefully, while stratified sampling keeps the proportions of important groups, such as rare outcomes, intact.

Question 17

What is classification, and what kind of output does it produce?

Accepted Answer

Classification assigns each record to one of a set of predefined categories based on its attributes. The output is a discrete class label, such as spam or not spam, and the model is learned from records whose correct labels are already known.

Question 18

How does regression differ from classification in predictive modeling?

Accepted Answer

Regression predicts a continuous numeric value, such as a house price or monthly energy use, while classification predicts a category. Both are supervised methods that learn from labeled examples, so the main difference lies in the type of target variable being estimated.

Question 19

What is clustering, and what does a good cluster look like?

Accepted Answer

Clustering groups records so that items within the same group are similar to each other and different from items in other groups, without using predefined labels. A good cluster is compact, with members close to a typical point, and well separated from neighboring clusters.

Question 20

What is k-means clustering, and what must the analyst choose in advance?

Accepted Answer

K-means clustering partitions records into k groups by repeatedly assigning each record to its nearest centroid and then recalculating each centroid as the mean of its members. The analyst must choose k, the number of clusters, in advance, and results can depend on the starting centroids.

Question 21

What is association rule mining, and what do support and confidence measure?

Accepted Answer

Association rule mining discovers rules of the form if itemset A occurs, then itemset B tends to occur, such as customers who buy bread often buy butter. Support measures how frequently the combined items appear across all transactions, while confidence measures how often B appears when A is present.

Question 22

What does a decision tree show, and why are its results easy to explain?

Accepted Answer

A decision tree shows a sequence of attribute tests that branch from a root node down to leaf nodes, where each leaf gives a predicted class or value. Its results are easy to explain because a person can follow one path from the top to a prediction and see each condition applied.

Question 23

What is anomaly detection, and how does it differ from classification?

Accepted Answer

Anomaly detection identifies records that deviate markedly from normal behavior, such as fraudulent card transactions or failing equipment readings. Unlike standard classification, it often works when examples of the unusual class are rare or poorly labeled, so it typically focuses on modeling what normal looks like.

Question 24

How do ensemble methods such as random forests improve prediction?

Accepted Answer

Ensemble methods combine many models so their individual errors partly cancel out, which usually produces more accurate and stable predictions than a single model. A random forest builds many decision trees on random samples of records and attributes, then combines their votes or averages.

Question 25

What is cross-validation, and why is it preferred over a single split?

Accepted Answer

Cross-validation divides the data into several folds, trains the model on all but one fold, and tests it on the held-out fold, repeating until each fold has served as the test set. It is preferred over a single split because every record is used for testing once, giving a stable estimate of performance.

Question 26

What does a confusion matrix show for a binary classifier?

Accepted Answer

A confusion matrix counts true positives, true negatives, false positives, and false negatives, showing how predicted labels compare with actual labels. Metrics such as accuracy, precision, recall, and specificity are all calculated from these four counts. Reading the matrix reveals which kinds of mistakes a model makes, which a single accuracy number can hide.

Question 27

Why can accuracy be a misleading metric on imbalanced data?

Accepted Answer

When one class is very common, a model can achieve high accuracy by predicting that common class almost every time, even though it misses the rare cases that matter. For example, a fraud model that never flags fraud could still appear highly accurate if fraud is rare.

Question 28

What is the difference between precision and recall?

Accepted Answer

Precision is the share of positive predictions that are actually correct, while recall is the share of actual positives that the model successfully finds. High precision means few false alarms, and high recall means few missed cases, so the right balance depends on the costs of each error.

Question 29

What does the area under the ROC curve measure?

Accepted Answer

The area under the ROC curve summarizes how well a classifier ranks positive cases above negative cases across all possible decision thresholds. A value of one indicates perfect ranking, while a value of one half corresponds to random guessing. This makes it useful for comparing models before a threshold is chosen.

Question 30

What is overfitting, and how can it be detected during model development?

Accepted Answer

Overfitting happens when a model learns noise and idiosyncrasies of the training data, so it performs well on training records but poorly on new data. It can be detected by comparing training and validation error, where a widening gap signals that the model is memorizing rather than generalizing.

Question 31

What is the bias-variance tradeoff in model complexity?

Accepted Answer

The bias-variance tradeoff describes how simple models tend to underfit, with high bias, while very flexible models tend to overfit, with high variance. Good generalization usually requires a complexity level that balances these two sources of error rather than minimizing only one.

Question 32

Why is a holdout test set kept untouched until the final evaluation?

Accepted Answer

A holdout test set is kept untouched so that it gives an unbiased estimate of performance on unseen data. If analysts repeatedly adjust models after viewing its results, the test set effectively becomes part of training, and the reported accuracy will be overly optimistic.

Question 33

What does deployment mean for a data mining model?

Accepted Answer

Deployment means putting a validated model into an operational environment where it scores new records or supports decisions in real time or on a schedule. It also includes integrating outputs with existing systems, such as a customer relationship management tool, so that people can act on predictions.

Question 34

What is model drift, and why does it require monitoring after deployment?

Accepted Answer

Model drift is the decline in a deployed model's accuracy that occurs when the patterns in incoming data change over time. Because customer behavior, markets, and sensors all shift, teams must monitor input distributions and performance metrics and retrain the model when quality falls below an agreed threshold.

Question 35

Why does interpretability matter when a model influences decisions about people?

Accepted Answer

Interpretability matters because people affected by a decision often deserve an understandable reason for it, and reviewers need to check whether the model relies on inappropriate factors. Explanations also help domain experts spot errors, and they support accountability when outcomes are challenged.

Question 36

How can sensitive attributes lead to unfair outcomes in data mining?

Accepted Answer

Sensitive attributes such as race, sex, or age can influence predictions directly, and proxies such as postal code can carry the same information even when those attributes are removed. Models trained on historical decisions may therefore reproduce past bias, so fairness checks compare error rates and outcomes across groups.

Question 37

What is privacy-preserving data mining, and what is one common technique?

Accepted Answer

Privacy-preserving data mining aims to extract useful patterns while limiting the exposure of individual records. One common technique is anonymization, which removes direct identifiers, though it must be combined with other safeguards because combinations of remaining attributes can still reveal identities in practice.

Question 38

What is the difference between a data mining result and an actionable insight?

Accepted Answer

A data mining result is a statistical finding, such as a pattern, score, or cluster, while an actionable insight is a finding that a decision maker can use to change something. A result becomes an insight only after it is validated, interpreted in business context, and linked to a specific action.

Question 39

Why should a mining project document its assumptions and limitations?

Accepted Answer

Documenting assumptions and limitations tells future readers when a model is appropriate and when it may fail, such as when populations or data collection methods change. It also makes results reproducible and helps stakeholders weigh the findings against risks before they act on them.

Question 40

How should a team communicate mining results to nontechnical stakeholders?

Accepted Answer

Teams should lead with the business question and the main finding, then explain the evidence in plain language, using visuals and concrete examples rather than technical jargon. They should also state uncertainty clearly, including how confident the results are and what would change the recommendation.

Data Mining Operations Flashcards

1Core Workflow and Vocabulary

2Data Preparation and Cleaning

3Pattern Discovery Methods

4Model Evaluation and Validation

5Applied Deployment and Responsible Use

Learning Resources