vaayaa Inclusive Education
Technical risk assessment

Data Science and Business Analytics Risk Assessment: Data Quality, Privacy and Model Validity Risks

A worked example of how an analytics team finds, scores and controls risks in data quality, privacy, reproducibility, model validity and third-party data use, before results shape business decisions.

Updated November 2026 8 hazards assessed Review every 6 months Highest risk: Critical to High
See the Data Science and Business Analytics course

Executive summary

This assessment shows how an analytics team can find, rate and control the risks that determine whether its findings can be trusted. Eight risks were chosen because they are plausible in everyday data science and business analytics work. Sampling bias, weak methodology and personal data in analysis carry the highest natural scores. Privacy and fairness risks stay material after controls, so they keep senior attention. Each risk is scored before controls and again after existing and added controls, and no residual score is ever higher than its inherent score. The first actions are a written sampling plan, version control for analysis code, tagging of personal fields and automated row count checks. The team is a teaching example, so every score illustrates method, not a real organization.

Focus area
Data science and business analytics
Setting
Analytics team supporting business decisions within a mid-sized organization that holds customer and operational data
Method
Five by five likelihood and impact risk register with inherent and residual scoring

Scope

This assessment covers an analytics team that collects, cleans, models and reports on business and customer data, from data sourcing and sampling through modelling, dashboards and decisions. It looks at risks to data quality, privacy, reproducibility, model validity and third-party data use. It does not cover any specific organization, legal jurisdiction or software product. Jurisdiction, data volumes, tools and sector are assumptions made for teaching.

Context

The team has two analysts, two data engineers, a data steward and a head of analytics who reports to a business director. Source systems feed a central data warehouse that holds customer records, transactions and operational logs. Analysts work in shared notebooks, publish dashboards to managers, and produce a quarterly forecast model used to set budgets. An external dataset is licensed for market benchmarking. Each risk is scored before controls to show its natural level, then again after the existing controls and the additional controls that should be added.

Objectives, assets and stakeholders

The objective is to produce reliable, reproducible and lawful analysis that supports good business decisions while protecting the people whose data is used. The assets in scope are the source data and warehouse, analysis code and notebooks, models and forecasts, dashboards and reports, and the licensed external datasets. Stakeholders include the analysts and engineers who do the work, the head of analytics, the data protection officer, business managers who act on findings, customers whose records are held, and external data providers. Each risk has one owner role, and the head of analytics reports the register to the business director each quarter.

Risk matrix: before and after controls

Each cell shows hazard numbers placed by likelihood and impact. Compare the two grids to see how controls move each hazard.

Before controls

Inherent risk

Risk before controls: rows are likelihood from almost certain to rare, columns are impact from insignificant to catastrophic. Cells list hazard numbers.
1
Insignificant		 2
Minor		 3
Moderate		 4
Major		 5
Catastrophic
5 Almost certain
4 Likely Likelihood 4, impact 4, score 16, high: H01 Likelihood 4, impact 5, score 20, critical: H03
3 Possible Likelihood 3, impact 3, score 9, medium: H06 Likelihood 3, impact 4, score 12, high: H02 H04 H08 Likelihood 3, impact 5, score 15, high: H05
2 Unlikely Likelihood 2, impact 4, score 8, medium: H07
1 Rare

After controls

Residual risk

Risk after controls: rows are likelihood from almost certain to rare, columns are impact from insignificant to catastrophic. Cells list hazard numbers.
1
Insignificant		 2
Minor		 3
Moderate		 4
Major		 5
Catastrophic
5 Almost certain
4 Likely
3 Possible Likelihood 3, impact 3, score 9, medium: H01 Likelihood 3, impact 4, score 12, high: H03
2 Unlikely Likelihood 2, impact 2, score 4, low: H06 Likelihood 2, impact 3, score 6, medium: H04 H08 Likelihood 2, impact 4, score 8, medium: H02 H05
1 Rare Likelihood 1, impact 4, score 4, low: H07
  • Low 1 to 4
  • Medium 5 to 9
  • High 10 to 16
  • Critical 17 to 25

Hazard register

A summary first, then the detail for each hazard.

Summary of hazards with scores before and after controls
No.HazardBeforeAfterOwner
H01 Sampling bias and poor data quality 16 High 9 Medium Lead data steward
H02 Weak methodology and irreproducible results 12 High 8 Medium Analytics lead
H03 Privacy breach from personal data in analysis 20 Critical 12 High Data protection officer
H04 Consent gaps and failed retention or deletion 12 High 6 Medium Records and retention manager
H05 Model bias and unfair decision outcomes 15 High 8 Medium Head of analytics
H06 Pipeline failure and silent reporting errors 9 Medium 4 Low Data engineering lead
H07 Third-party data licence breach 8 Medium 4 Low Supplier contract manager
H08 Unmonitored model drift and misleading business decisions 12 High 6 Medium Analytics lead
H01

Sampling bias and poor data quality

Technology and data Before 16 After 9

Samples leave out key groups, source fields are missing or mis-coded, and duplicate records are merged wrongly, so the data does not represent the population the analysis claims to describe.

Who or what is affected: Analysts, business decision makers and customers whose records are in the data

Causes

Convenience sampling, unmanaged source changes and no agreed definitions for key fields.

Consequences

Biased estimates, wrong targeting decisions and loss of trust in analytics outputs.

Existing controls

  • Analysts profile key fields for nulls and ranges before modelling.
  • Source owners sign off a data dictionary for the main tables.

Control gaps

Sample design is rarely documented, and source changes are not announced to the analytics team.

Additional controls

  • Prevent: Write a sampling plan that states the target population, frame and exclusions before analysis starts.
  • Detect: Run automated checks on volume, missingness and category shifts against the previous load.
Likelihood 4 x impact 4 before; 3 x 3 after Owner: Lead data steward Target: Within 45 days Early warning: Share of records with missing key fields rising above a set threshold between loads.
H02

Weak methodology and irreproducible results

Operational Before 12 After 8

Analyses run from unversioned notebooks, random seeds and parameters are not recorded, and results cannot be rerun, so findings cannot be checked or trusted by colleagues or decision makers.

Who or what is affected: Analysts, peer reviewers and managers relying on reports

Causes

Ad hoc workflows, time pressure and no shared code standard across the team.

Consequences

Wrong findings used for decisions, wasted rework and failed peer checks.

Existing controls

  • Analysts keep a shared folder for final reports.
  • A senior analyst reads major reports before release.

Control gaps

Reviews focus on charts and wording, not on whether the code and data can reproduce the figures.

Additional controls

  • Prevent: Keep all analysis code in version control with fixed environments and recorded random seeds.
  • Detect: Rerun a sample of reports from raw data each quarter and record any differences.
Likelihood 3 x impact 4 before; 2 x 4 after Owner: Analytics lead Target: Within 60 days Early warning: A report that cannot be rerun from raw data to produce the same outputs.
H03

Privacy breach from personal data in analysis

Legal and compliance Before 20 After 12

Customer records with names, contact details or sensitive attributes are copied into notebooks, exports or shared files, where people who do not need them can see them.

Who or what is affected: Customers whose data is held, analysts and the data protection officer

Causes

Analysts working in local copies, no masking of fields and broad read access to the warehouse.

Consequences

Privacy complaints, regulator inquiries, customer harm and fines where applicable.

Existing controls

  • Warehouse access requires a named login.
  • Customer names are removed from the standard reporting views.

Control gaps

Personal fields are not tagged, so the team cannot tell which extracts contain them.

Additional controls

  • Prevent: Tag personal fields in the data catalogue and pseudonymize them before any extract leaves the warehouse.
  • Respond: Isolate any exported file found outside approved storage and record the incident for review.
Likelihood 4 x impact 5 before; 3 x 4 after Owner: Data protection officer Target: Within 30 days Early warning: Personal data found in a notebook, export or shared folder during a routine scan.
H04

Consent gaps and failed retention or deletion

Legal and compliance Before 12 After 6

Data is used for purposes customers did not agree to, or is kept after its need has ended, and deletion requests are not carried through to copies, backups and models built from the data.

Who or what is affected: Customers who provided data, and marketing and service teams that use outputs

Causes

Purpose limits not recorded with each dataset and no named owner for deletion across copies.

Consequences

Unlawful use claims where applicable, complaints, forced deletion of models and reputational harm.

Existing controls

  • Consent flags are stored with customer records.
  • Datasets older than a set age are reviewed once a year.

Control gaps

Deletion requests are handled in the main database only, not in copies, extracts or feature stores.

Additional controls

  • Prevent: Record the permitted purpose for every dataset and block use outside it in the access process.
  • Recover: Build a deletion routine that covers copies, extracts and feature stores, and test it each quarter.
Likelihood 3 x impact 4 before; 2 x 3 after Owner: Records and retention manager Target: Within 90 days Early warning: Deletion requests still open past the agreed response time or found in a copy.
H05

Model bias and unfair decision outcomes

Reputational Before 15 After 8

A predictive model treats groups differently because of skewed training data or proxy variables, and its scores are then used to set prices, offers or service levels for individuals.

Who or what is affected: Customers in affected groups, the business and any oversight bodies

Causes

Historical data reflects past unequal decisions, and proxy variables are included without review.

Consequences

Unfair outcomes, complaints, withdrawal of the model and loss of public trust.

Existing controls

  • Models are reviewed by a second analyst before release.
  • Protected or sensitive attributes are excluded from model inputs.

Control gaps

Fairness is not tested across groups before a model goes live, and ethical review is informal.

Additional controls

  • Prevent: Hold an ethical review with a named reviewer for any model that affects individuals before release.
  • Detect: Compare error rates and outcomes across groups each month and record gaps above the agreed limit.
Likelihood 3 x impact 5 before; 2 x 4 after Owner: Head of analytics Target: Within 60 days Early warning: Gap in error rates between groups that is above the agreed limit in monthly checks.
H06

Pipeline failure and silent reporting errors

Technology and data Before 9 After 4

A scheduled job fails, a join duplicates rows or a source schema change goes unnoticed, so dashboards show stale or wrong figures without any visible error message.

Who or what is affected: Managers reading dashboards, finance and operations teams

Causes

Brittle scripts, no monitoring of row counts and unannounced changes in source systems.

Consequences

Wrong figures used in meetings, delayed decisions and rework.

Existing controls

  • Scheduled jobs send a failure email to the data engineering inbox.
  • Dashboards show the date of the last successful refresh.

Control gaps

Job alerts go to one shared inbox and nobody checks row counts after each load.

Additional controls

  • Detect: Add row count and total checks between source and target after each load, with alerts.
  • Respond: Hold dashboard publication when a load fails and notify the named owner at once.
Likelihood 3 x impact 3 before; 2 x 2 after Owner: Data engineering lead Target: Within 30 days Early warning: Row count change between source and target outside the agreed range after a load.
H07

Third-party data licence breach

Supplier Before 8 After 4

An external dataset is used beyond its licence terms, such as resale, sharing with clients or merging with restricted fields, which can end access to the data and create contract claims.

Who or what is affected: Analysts, the licensing team and the external data provider

Causes

Licence terms are not summarized for analysts and renewals are handled by procurement alone.

Consequences

Contract breach, loss of access to the data and legal claims.

Existing controls

  • Licences are stored in a contracts folder.
  • Procurement signs each new data agreement.

Control gaps

Terms are filed but not linked to the datasets or projects that use them.

Additional controls

  • Prevent: Record licence limits in the data catalogue and check them before any new project uses the dataset.
  • Detect: Review project outputs against licence terms at each release and log any conflict found.
Likelihood 2 x impact 4 before; 1 x 4 after Owner: Supplier contract manager Target: Within 60 days Early warning: A new project that uses a licensed dataset without a recorded permitted use.
H08

Unmonitored model drift and misleading business decisions

Strategic Before 12 After 6

A model that was accurate at launch loses accuracy as customer behaviour changes, but nobody tracks its performance, so forecasts and recommendations keep guiding spending and planning.

Who or what is affected: Managers, budget holders and customers affected by the decisions

Causes

No planned revalidation, reliance on a single accuracy figure and changing market conditions.

Consequences

Poor spending, lost revenue and wrong strategic choices.

Existing controls

  • Models are validated on a holdout sample before release.
  • Forecast results are discussed in the monthly review.

Control gaps

Validation is done once before launch and results are not compared with actual outcomes afterwards.

Additional controls

  • Prevent: Set a revalidation schedule and a retirement rule for each model, approved by the analytics lead.
  • Detect: Track forecast error against actual results each month and flag drift above the agreed limit.
Likelihood 3 x impact 4 before; 2 x 3 after Owner: Analytics lead Target: Within 90 days Early warning: Forecast error above the agreed limit for two consecutive months.

Rating method and evaluation

Each risk is rated on two five-point scales. Likelihood runs from Rare to Almost certain, and impact runs from Insignificant to Severe. Multiplying the two gives a score from 1 to 25, which falls into one of four bands: Low from 1 to 4, Medium from 5 to 9, High from 10 to 16 and Critical from 17 to 25. The score is a guide for ordering work, not a precise measurement. Inherent risk is the score before any controls. Residual risk is the score after the existing and added controls, and it must never be higher than the inherent score. Impact is judged on harm to individuals, wrong decisions, financial loss, legal exposure and loss of trust. Privacy and fairness risks are not rated down only because no harm has yet been seen. Anything still High needs senior attention and a dated plan, anything Medium needs routine controls with a named owner, and anything Low is accepted and watched. Analysts and reviewers should challenge every rating, and each rating is reviewed whenever an indicator reaches its threshold.

Likelihood, impact and rating bands

Likelihood

  • 5 Almost certain: Expected to happen in most circumstances.
  • 4 Likely: Will probably happen in many circumstances.
  • 3 Possible: Could happen at some time.
  • 2 Unlikely: Not expected, but it could happen.
  • 1 Rare: Only in exceptional circumstances.

Impact

  • 1 Insignificant: No lasting effect. Handled within routine work.
  • 2 Minor: Small effect that is quickly corrected.
  • 3 Moderate: Noticeable effect that needs managed effort to correct.
  • 4 Major: Serious effect on people, delivery or reputation.
  • 5 Catastrophic: Severe or lasting effect that threatens the whole activity.

Bands (likelihood x impact)

  • Low (1 to 4): Accept and monitor.
  • Medium (5 to 9): Manage with routine controls.
  • High (10 to 16): Senior attention and a treatment plan.
  • Critical (17 to 25): Act first. Do not proceed until reduced.

Monitoring, escalation and review

Review this assessment every 6 months, and sooner when an indicator below reaches its threshold.

Early-warning indicators and escalation thresholds
Indicator to watchEscalate when
Share of records with missing key fields in each scheduled load.Escalate to the lead data steward when the share rises above the agreed threshold, and to the head of analytics if it stays high for two loads.
Personal data found outside approved storage during routine scans.Escalate to the data protection officer at once, and isolate the file until it is reviewed and cleared.
Error rate gap between groups in monthly model checks.Escalate to the head of analytics when the gap exceeds the agreed limit, and pause the model pending ethical review.
Forecast error against actual results for each live model.Escalate to the analytics lead after two consecutive months above the limit, and to the business director if decisions have already changed.
Deletion requests open past the agreed response time.Escalate to the records and retention manager when any request is overdue, and to the data protection officer after the agreed deadline passes.

Contingency arrangements

If a serious event occurs, such as a personal data exposure, a model found to treat groups unfairly or a published figure found to be wrong, the head of analytics pauses the affected outputs and preserves the relevant code, data and logs. The data protection officer assesses the privacy impact and the data steward confirms which datasets and reports are affected. Business owners are told which decisions relied on the figures. Affected records are corrected or deleted, and any required notices are made through the appropriate channels in the relevant jurisdiction.

Prioritized action plan

  1. Write and approve a sampling plan template for every new analysis, owner lead data steward, within 45 days.
  2. Tag personal fields in the catalogue and block unmasked extracts, owner data protection officer, within 30 days.
  3. Move all analysis code into version control with recorded environments, owner analytics lead, within 60 days.
  4. Add row count and total checks to every scheduled load, owner data engineering lead, within 30 days.
  5. Set up an ethical review and a revalidation calendar for every model that affects individuals, owner head of analytics, within 60 days.

Conclusion

Trustworthy analytics depends on a few steady habits: documented sampling, version-controlled code, tagged personal data, clear permitted purposes, automated pipeline checks and scheduled revalidation of models. The highest scores sit with sampling bias, privacy exposure and fairness, and each has actions that bring the residual score down. Treat the register as a living record, review it on the stated cycle, and seek qualified legal, privacy and research ethics advice for any real project.

Assumptions and limitations

Assumptions

  • The team draws on a central warehouse fed by several source systems, with a mix of personal and aggregated business data.
  • Jurisdiction is not specified, so privacy, consent and licensing duties are described in general terms only.
  • Some analysis code lives in notebooks shared between analysts and is not yet under full version control.

Limitations

  • The scores are teaching examples and do not describe any real team, dataset or model.
  • Privacy, research ethics and data licensing rules vary and change, so specialist review is needed.
  • This is educational content and does not replace a formal site-specific assessment or licensed legal advice.

Related frameworks to study

  • Model risk management principles covering validation, monitoring and governance
  • Privacy by design and data minimization
  • Reproducibility checklists for analysis code and data lineage
  • Three lines model of risk ownership
  • Hierarchy of controls

Technical Risk Assessment

China, India, United States, Indonesia, Pakistan, Nigeria, Brazil, Bangladesh, Russia, Japan, Mexico, Philippines, Vietnam, Egypt, Turkey, Germany, United Kingdom, France, Italy, Canada, South Korea, Australia, Spain.

For Learning Reference Only: This assessment demonstrates how risk assessment principles are applied in a practical setting, providing a useful reference for learning and understanding. Actual assessments may be adapted to suit specific site conditions and circumstances.

Keep learning

Study the subject behind this assessment.

Data Science and Business Analytics course Read the best practices for this course

More Technical risk assessments

Hourly Active: 45,552 Learners