Machine Learning Engineering Flashcards

Question 1

What is the main difference between a machine learning model and a machine learning system?

Accepted Answer

A model is the learned function that maps inputs to predictions, while a system is everything around it that makes the model useful in practice. The system includes data ingestion, feature preparation, serving infrastructure, logging, monitoring, and retraining processes. Engineers design these parts so predictions arrive reliably and on time.

Question 2

Why does reproducibility matter so much when building machine learning systems?

Accepted Answer

Reproducibility means that anyone can rerun the same code on the same data and obtain the same model and results. It matters because teams must debug failures, compare experiments fairly, and pass audits or reviews with confidence. Achieving it requires versioned code, versioned data, recorded parameters, and fixed random seeds where possible.

Question 3

What is the difference between training and inference in a machine learning workflow?

Accepted Answer

Training is the process of fitting model parameters to historical data by minimizing a loss function. Inference is the process of using the trained model to produce predictions on new inputs. Training is usually compute intensive and runs occasionally, while inference often must respond quickly and runs continuously.

Question 4

What is a feature in supervised learning and why does it matter?

Accepted Answer

A feature is an individual measurable property of the data that the model uses as input to make a prediction. Features matter because model quality depends heavily on how well they capture the signal relevant to the target. Good features can let a simple model outperform a complex model fed with poorly chosen inputs.

Question 5

What is overfitting, and how does it differ from underfitting?

Accepted Answer

Overfitting occurs when a model learns noise and quirks of the training data so closely that it performs poorly on new data. Underfitting occurs when a model is too simple to capture the real pattern, so it performs poorly on training data too. Comparing training and validation scores helps reveal which problem is present.

Question 6

What is a baseline model, and why should engineers build one first?

Accepted Answer

A baseline model is a simple predictor, such as one that always outputs the most common class or a basic linear model. Engineers build one first to establish a reference point that more complex approaches must beat to justify their cost. A baseline also reveals whether the data contains a usable signal at all.

Question 7

What does it mean to version data in a machine learning project?

Accepted Answer

Versioning data means recording immutable snapshots or identifiable states of a dataset so that each model can be traced to the data it was trained on. This matters because datasets change through new collection, corrections, and filtering, and silent changes can alter results. Teams track versions with hashes, timestamps, or data version control tools.

Question 8

What is the purpose of a model card or similar documentation?

Accepted Answer

A model card is a structured document describing a model's intended use, training data, evaluation results, and known limitations. Its purpose is to help engineers, reviewers, and downstream users decide whether the model fits their needs and to prevent misuse outside its scope. Documentation like this turns tacit knowledge into shared, reviewable information.

Question 9

What is a data pipeline in machine learning, and what stages does it usually include?

Accepted Answer

A data pipeline is an automated sequence of steps that moves raw data from its sources to a form a model can use. Typical stages include ingestion, validation, cleaning, transformation, feature computation, and storage of the prepared output. Well-designed pipelines run the same logic for training and serving so the two stay consistent.

Question 10

What is data leakage, and why does it make model evaluation misleading?

Accepted Answer

Data leakage occurs when information that would not be available at prediction time influences training, often through features, preprocessing steps, or improperly split data. It makes evaluation misleading because the model appears accurate in testing but fails in production. Preventing it requires fitting transformations only on training data and splitting data before any aggregation.

Question 11

Why should data be split into training, validation, and test sets?

Accepted Answer

Separate sets let engineers fit the model, tune its settings, and estimate final performance without reusing the same examples for each purpose. The training set fits parameters, the validation set guides hyperparameter choices, and the test set gives an unbiased estimate after decisions are final. Reusing the test set for tuning inflates results.

Question 12

What is feature scaling, and which kinds of models usually need it?

Accepted Answer

Feature scaling transforms numeric features onto a comparable range, such as by standardizing them to zero mean and unit variance or by min-max normalization. Models that rely on distances or gradient-based optimization, such as k-nearest neighbors, support vector machines, and neural networks, usually need it. Tree-based models are generally less sensitive to scale.

Question 13

How do one-hot encoding and ordinal encoding differ for categorical features?

Accepted Answer

One-hot encoding creates a separate binary column for each category, so no artificial order is implied between categories. Ordinal encoding maps categories to integers, which is appropriate only when the categories have a meaningful order. Using ordinal codes for unordered categories can mislead models into treating some values as closer or larger than others.

Question 14

What is a feature store and what problem does it address?

Accepted Answer

A feature store is a shared system for defining, computing, storing, and serving features so that teams reuse them consistently. It addresses the problem of training-serving skew, where features are calculated differently in offline training and online prediction. It also reduces duplicated engineering work across projects that need the same inputs.

Question 15

What is training-serving skew, and how can teams reduce it?

Accepted Answer

Training-serving skew is a mismatch between the data or feature logic used during model training and the logic applied when the model makes live predictions. It often causes a model to perform well offline but poorly in production. Teams reduce it by sharing feature code between paths, validating input distributions, and logging serving features.

Question 16

Why do data quality checks matter before model training begins?

Accepted Answer

Data quality checks catch problems such as missing values, duplicate records, schema changes, and out-of-range numbers before they corrupt a model. Training on flawed data can produce a model that looks acceptable in metrics yet behaves poorly on real inputs. Automated checks that fail early are cheaper than debugging a bad model after deployment.

Question 17

What is a loss function, and what role does it play during training?

Accepted Answer

A loss function is a formula that measures how far a model's predictions are from the desired outputs for a set of examples. During training, an optimizer adjusts the model parameters to reduce the average loss across the training data. The choice of loss function should match the task, such as classification or regression.

Question 18

What is gradient descent and what does the learning rate control?

Accepted Answer

Gradient descent is an iterative optimization method that updates parameters in the direction that most reduces the loss, guided by the gradient. The learning rate controls the size of each update step. A rate that is too large can overshoot or diverge, while one that is too small can make training slow or stall.

Question 19

What is the practical difference between precision and recall in classification tasks?

Accepted Answer

Precision is the share of positive predictions that are correct, while recall is the share of actual positives that the model finds. Precision matters most when false alarms are costly, and recall matters most when missing a true case is costly. Both metrics depend on the decision threshold, so they should be reported together.

Question 20

Why is accuracy alone often a poor metric for imbalanced datasets?

Accepted Answer

Accuracy measures the fraction of all predictions that are correct, so a model can score highly by always predicting the majority class. In a dataset where one class is rare, that trivial model may miss every important case. Precision, recall, and F1 score reveal performance on the minority class.

Question 21

What is cross-validation, and why might an engineer use it?

Accepted Answer

Cross-validation splits the training data into several folds, trains on all but one fold, and evaluates on the held-out fold, rotating until every fold has been used for evaluation. Engineers use it to get a more stable estimate of performance. The averaged score helps compare models more reliably than a single split would.

Question 22

What is hyperparameter tuning, and how does it differ from parameter learning?

Accepted Answer

Hyperparameter tuning is the search for settings, such as learning rate, tree depth, or regularization strength, that are chosen before training begins. Parameter learning is the process by which training adjusts the model's internal weights to fit the data. Tuning typically evaluates many configurations on validation data and keeps the one that generalizes best.

Question 23

What is regularization, and how does it help a model generalize?

Accepted Answer

Regularization adds a penalty to the training objective that discourages overly large or complex parameter values. This constrains how closely the model can fit noise in the training data, which often improves performance on unseen examples. Common forms include L1 and L2 penalties, dropout in neural networks, and early stopping.

Question 24

Why should an engineer test a model on data that resembles production conditions?

Accepted Answer

A model can perform well on a standard test set yet fail when inputs shift in timing, population, or collection method. Evaluating on data that resembles production, including recent or out-of-distribution examples, reveals these gaps before release. They give a realistic picture of how the model will behave once users depend on it.

Question 25

What does it mean to deploy a machine learning model?

Accepted Answer

Deploying a model means making it available in an environment where it can receive real inputs and return predictions to users or other systems. Deployment includes packaging the model with its dependencies, exposing it through an interface, and configuring the infrastructure that runs it. A model left in a notebook has not been deployed.

Question 26

What is the difference between batch prediction and online prediction?

Accepted Answer

Batch prediction runs the model over a large set of stored inputs on a schedule, and results are saved for later use. Online prediction returns a result for a single request in real time, usually through an API. Batch suits tasks where delay is acceptable, while online suits tasks needing immediate responses.

Question 27

What is model serialization, and why is it needed for deployment?

Accepted Answer

Model serialization converts a trained model's parameters and structure into a stored file format that can be saved, transferred, and loaded later. It is needed because the serving environment often runs separately from the training environment and must reconstruct the same model. Store serialized artifacts with the code and data that produced them.

Question 28

Why do teams often use containers when deploying machine learning models?

Accepted Answer

Containers package the model with its exact libraries, system dependencies, and runtime settings into a portable unit. This reduces the common problem of code that works on one machine but fails on another. Containers also make it easier to scale identical copies of a service and roll back to a previous version when needed.

Question 29

What is a REST API, and how is it used to serve predictions?

Accepted Answer

A REST API is an interface where clients send HTTP requests to defined endpoints and receive responses, often formatted as JSON. To serve predictions, a service exposes an endpoint that accepts input features in the request body, runs the model, and returns the predicted value. Clients can integrate predictions without knowing model internals.

Question 30

What is a canary deployment, and why do teams use it for new models?

Accepted Answer

A canary deployment sends a small share of live traffic to a new model version while the existing version continues serving most requests. Teams use it to compare behavior and error rates in production before a full rollout. If the new version performs poorly, traffic can be shifted back quickly, limiting user impact.

Question 31

What is latency in model serving, and why does it matter to users?

Accepted Answer

Latency is the elapsed time between a request arriving at a serving system and the response being returned. It matters because slow predictions can degrade user experience, break time-sensitive workflows, or exceed service agreements. Engineers manage latency through model size choices, batching, caching, hardware selection, and efficient request handling.

Question 32

Why might a team choose to use a model registry?

Accepted Answer

A model registry is a central catalog that stores trained model versions along with their metadata, metrics, lineage, and approval status. Teams use it to know which version is in production, to promote or roll back models in a controlled way, and to keep an audit trail.

Question 33

What is model drift, and what are its main types?

Accepted Answer

Model drift is a change in the relationship between inputs and outcomes that causes a deployed model to become less accurate over time. Data drift refers to changes in the distribution of input features, while concept drift refers to changes in how inputs relate to the target. Both can occur together.

Question 34

Which metrics should teams monitor once a machine learning model is in production?

Accepted Answer

Teams should monitor system health, such as latency, error rates, and throughput, alongside input feature distributions and prediction distributions. When ground truth becomes available, they should also track model quality metrics such as accuracy, precision, or calibration. Comparing these signals over time helps separate infrastructure failures from genuine model degradation.

Question 35

Why is ground truth often delayed in production, and why does that matter?

Accepted Answer

Ground truth is often delayed because true outcomes may only be known days or weeks later. This delay means teams cannot directly measure accuracy in real time and must rely on proxy signals. Proxy monitoring can miss problems, so delayed labels should be collected and compared against predictions as they arrive.

Question 36

What is retraining, and what triggers a team to retrain a model?

Accepted Answer

Retraining means fitting a model again on updated data, usually to restore performance lost to drift or to incorporate new patterns. Common triggers include measured drops in quality, significant data distribution shifts, scheduled intervals, or changes in business requirements. Retraining should go through the same validation and approval steps as the original release.

Question 37

What does the term technical debt mean in machine learning engineering?

Accepted Answer

Technical debt is the future cost created by shortcuts or design choices that make a system harder to change, test, or maintain. In machine learning systems it often arises from entangled features, hidden dependencies on data sources, undocumented preprocessing, and pipelines glued together with fragile scripts. Paying it down keeps future model updates manageable.

Question 38

What is an alert threshold, and how should teams set one for model monitoring?

Accepted Answer

An alert threshold is a boundary on a monitored metric that triggers a notification or action when crossed. Teams should set thresholds using historical variation and business impact, not arbitrary round numbers, so that alerts are meaningful. Overly sensitive thresholds create alert fatigue, while overly loose ones miss real degradation.

Question 39

Why should teams keep logs of predictions and inputs from a production model?

Accepted Answer

Logged predictions and inputs provide the evidence needed to diagnose incidents, measure drift, and reproduce problems seen in live traffic. They also create a dataset for later evaluation once ground truth arrives. Logging should respect privacy rules, so teams should minimize sensitive fields and control who can access the records.

Question 40

What is a retirement plan for a machine learning model?

Accepted Answer

A retirement plan defines when and how a model is taken out of service, including criteria for decommissioning, notice to dependent teams, and handling of archived artifacts. Without one, outdated models can keep running silently and affect decisions long after they stop being reliable. The plan should cover transitions too.

Machine Learning Engineering Flashcards

1Foundations of ML Engineering

2Data Pipelines and Feature Engineering

3Model Training and Evaluation

4Deployment and Model Serving

5Monitoring and Ongoing Maintenance

Learning Resources