vaayaa Inclusive Education
Technical risk assessment

Business Intelligence Development Risk Assessment: Pipeline, Data Quality and Access Controls

A worked example of how a business intelligence development team finds, scores and controls the risks that affect pipelines, data quality, privacy, licensed data and the trust placed in dashboards and reports.

Updated November 2026 8 hazards assessed Review every 6 months Highest risk: Critical to Medium
See the Business Intelligence Development course

Executive summary

This assessment shows how a business intelligence development team can find, rate and control the risks that threaten the accuracy of its reports, the privacy of the people in its data and the trust placed in its dashboards. Eight hazards were chosen because they are plausible in everyday pipeline, modeling and publishing work. Pipeline failure carries the highest inherent score because it is likely and its effects reach every dashboard. Data quality and dashboard logic errors follow closely because they are common and hard to trace. Privacy exposure and licensing breaches are less frequent but cause serious harm. Each risk is scored before controls and again after existing and added controls. Most fall to Medium or Low once the added controls are in place, and the first actions are a metric dictionary, automated data quality tests, role based access reviews and a tested restore of the warehouse. Every score illustrates method, not a real organization.

Focus area
Business intelligence development and analytics reporting
Setting
Analytics team building data pipelines, warehouse models and dashboards for a mid-sized organization
Method
Five by five likelihood and impact risk register with inherent and residual scoring

Scope

This assessment covers the work of a small business intelligence development team, from source extraction and pipeline build to data modeling, dashboard publishing and ongoing maintenance. It looks at risks to data accuracy, privacy, licensed data, access, retention and decisions made from reports. It does not cover legal advice on privacy or licensing, or any real organization's systems. Jurisdiction, data sources, tools and headcount are assumptions made for teaching.

Context

The team has a lead developer, two data engineers, an analyst who writes dashboards and a data steward who approves definitions. Source data comes from an operational transaction system, a customer relationship tool and a licensed market dataset supplied by an outside vendor. Transformations run in scheduled jobs into a central warehouse, and dashboards are published to business users on a shared platform. Each hazard is scored before controls to show its natural level, then again after existing controls and the additional controls that should be added.

Objectives, assets and stakeholders

The objective is to deliver timely, accurate and reproducible reporting, protect personal and licensed data, meet retention and deletion duties, and keep the confidence of business users and data owners. The assets in scope are the source extracts, the warehouse and its models, the transformation code, the dashboards and their definitions, the licensed third-party dataset and the personal data held for analysis. Stakeholders include the development team, the data steward, business users and decision makers, customers whose data is processed, the data vendor, the privacy lead and senior leaders who rely on the figures. Every risk has one owner role, and the data steward keeps the register current and reports to the governance group each quarter.

Risk matrix: before and after controls

Each cell shows hazard numbers placed by likelihood and impact. Compare the two grids to see how controls move each hazard.

Before controls

Inherent risk

Risk before controls: rows are likelihood from almost certain to rare, columns are impact from insignificant to catastrophic. Cells list hazard numbers.
1
Insignificant		 2
Minor		 3
Moderate		 4
Major		 5
Catastrophic
5 Almost certain
4 Likely Likelihood 4, impact 3, score 12, high: H02 Likelihood 4, impact 5, score 20, critical: H01
3 Possible Likelihood 3, impact 3, score 9, medium: H07 H08 Likelihood 3, impact 4, score 12, high: H03 H05
2 Unlikely Likelihood 2, impact 4, score 8, medium: H06 Likelihood 2, impact 5, score 10, high: H04
1 Rare

After controls

Residual risk

Risk after controls: rows are likelihood from almost certain to rare, columns are impact from insignificant to catastrophic. Cells list hazard numbers.
1
Insignificant		 2
Minor		 3
Moderate		 4
Major		 5
Catastrophic
5 Almost certain
4 Likely
3 Possible Likelihood 3, impact 2, score 6, medium: H02
2 Unlikely Likelihood 2, impact 2, score 4, low: H07 Likelihood 2, impact 3, score 6, medium: H01 H03 H05 H08 Likelihood 2, impact 4, score 8, medium: H04
1 Rare Likelihood 1, impact 4, score 4, low: H06
  • Low 1 to 4
  • Medium 5 to 9
  • High 10 to 16
  • Critical 17 to 25

Hazard register

A summary first, then the detail for each hazard.

Summary of hazards with scores before and after controls
No.HazardBeforeAfterOwner
H01 Pipeline failure and stale reporting 20 Critical 6 Medium Lead developer
H02 Source data quality and inconsistent definitions 12 High 6 Medium Data steward
H03 Misleading dashboard logic and metric errors 12 High 6 Medium Analytics manager
H04 Personal data exposure through over-broad access 10 High 8 Medium Privacy lead
H05 Retention and deletion failures 12 High 6 Medium Data engineering lead
H06 Third-party data licensing breach 8 Medium 4 Low Procurement lead
H07 Sampling bias and model bias in predictive features 9 Medium 4 Low Senior analyst
H08 Irreproducible analysis and uncontrolled change 9 Medium 6 Medium Analytics manager
H01

Pipeline failure and stale reporting

Operational Before 20 After 6

A scheduled load fails silently, runs late or loads partial data, so dashboards show old or incomplete figures that users treat as current.

Who or what is affected: Business users, decision makers and the development team

Causes

Source system changes, network outages, unhandled errors and jobs that depend on one another without checks.

Consequences

Wrong operational decisions, missed deadlines and loss of confidence in the reporting platform.

Existing controls

  • Jobs send an email alert when they fail.
  • A daily row count is compared with the previous day.

Control gaps

Alerts go to one mailbox, and nobody confirms that partial loads are caught.

Additional controls

  • Prevent: Use dependency checks so downstream models do not run when a source is missing.
  • Detect: Show a last refreshed time on every dashboard and alert on jobs that overrun.
  • Recover: Keep a tested restore point and a documented rerun procedure for failed loads.
Likelihood 4 x impact 5 before; 2 x 3 after Owner: Lead developer Target: Within 30 days Early warning: Scheduled loads that finish later than their agreed window.
H02

Source data quality and inconsistent definitions

Technology and data Before 12 After 6

Source records contain duplicates, missing keys, mismatched codes and inconsistent definitions across systems, so totals are reported with errors that are hard to trace.

Who or what is affected: Analysts, business users and customers whose records are counted

Causes

Several source systems, manual entry and definitions agreed informally between teams.

Consequences

Conflicting totals, rework, disputed reports and poor decisions about customers and products.

Existing controls

  • Required fields are enforced in the source system.
  • The data steward reviews definitions when requested.

Control gaps

Tests cover row counts but not the meaning of key fields, so semantic errors pass unnoticed.

Additional controls

  • Prevent: Agree one metric dictionary and map each source code to a single standard value.
  • Detect: Run automated tests for duplicates, nulls and out of range values on every load.
Likelihood 4 x impact 3 before; 3 x 2 after Owner: Data steward Target: Within 45 days Early warning: Two dashboards that show different totals for the same measure on the same date.
H03

Misleading dashboard logic and metric errors

Operational Before 12 After 6

Filters, joins or calculated fields are built incorrectly, so a chart shows a plausible but wrong figure, such as double counted revenue or the wrong time period.

Who or what is affected: Managers, analysts and the teams that act on the figures

Causes

Complex joins, unclear grain of tables and pressure to release quickly.

Consequences

Wrong targets, misallocated budgets and corrections that damage trust in the team.

Existing controls

  • An analyst reviews each new dashboard before release.
  • Calculated fields are written in a shared, named library.

Control gaps

Reviews check layout more than logic, and totals are not reconciled to source.

Additional controls

  • Prevent: Require a second developer to review join logic and calculated fields before publication.
  • Detect: Reconcile headline totals each month against the source system.
  • Respond: Keep a list of published reports so any faulty one can be withdrawn quickly.
Likelihood 3 x impact 4 before; 2 x 3 after Owner: Analytics manager Target: Within 60 days Early warning: A dashboard figure that changes sharply after a minor filter or model edit.
H04

Personal data exposure through over-broad access

Legal and compliance Before 10 After 8

Analysts and developers can see identifiable customer records they do not need, or shared extracts are sent to personal inboxes, so personal data leaves controlled systems.

Who or what is affected: Customers, staff whose data is held and the privacy lead

Causes

Broad default roles, copies of data for testing and convenience sharing.

Consequences

Harm to individuals, complaints, regulatory attention and loss of customer trust.

Existing controls

  • Warehouse logins are individual and use single sign on.
  • Staff are trained not to email raw extracts.

Control gaps

Roles were granted at the schema level, and no one reviews who reads personal fields.

Additional controls

  • Prevent: Mask identifiers in development and analysis environments and grant column level access by role.
  • Detect: Log queries against personal fields and review unusual access each month.
  • Respond: Revoke access and inform the privacy lead when an extract is found outside the platform.
Likelihood 2 x impact 5 before; 2 x 4 after Owner: Privacy lead Target: Within 30 days Early warning: A user querying personal fields outside their assigned reporting area.
H05

Retention and deletion failures

Legal and compliance Before 12 After 6

Personal records are kept in the warehouse, backups and extracts longer than the stated purpose allows, and deletion requests are not carried through every copy.

Who or what is affected: Customers who asked for deletion, staff and the privacy lead

Causes

No purge logic in early pipelines, copies made for one-off analyses and backups kept for long periods.

Consequences

Unlawful retention, failed deletion requests, complaints and fines where relevant.

Existing controls

  • A retention period is written in the data catalogue.
  • Deletion requests are logged by the customer service team.

Control gaps

Deletion is handled manually, and derived tables are not in the purge scope.

Additional controls

  • Prevent: Build automated purge jobs that follow the retention schedule for each table.
  • Recover: Confirm deletion in backups and derived extracts within the agreed cycle.
Likelihood 3 x impact 4 before; 2 x 3 after Owner: Data engineering lead Target: Within 60 days Early warning: Personal records held past their scheduled retention date.
H06

Third-party data licensing breach

Supplier Before 8 After 4

A licensed market dataset is loaded into reports or shared with clients in ways the vendor agreement does not permit, or the licence lapses while dashboards still depend on it.

Who or what is affected: The data vendor, the organization and clients receiving outputs

Causes

Unclear licence terms, reuse of data for new products and renewal dates that are missed.

Consequences

Contract claims, forced removal of reports, costs and damage to the vendor relationship.

Existing controls

  • The licence agreement is stored with the contracts team.
  • Access to the licensed dataset is restricted to the analytics group.

Control gaps

Licence terms are not translated into rules the developers can check.

Additional controls

  • Prevent: Record each permitted use in a register and check new reports against it before release.
  • Detect: Review dataset use and renewal dates each quarter.
  • Respond: Remove dependent outputs promptly if a use is found outside the licence.
Likelihood 2 x impact 4 before; 1 x 4 after Owner: Procurement lead Target: Within 60 days Early warning: A licensed dataset used for a purpose not listed in the vendor agreement.
H07

Sampling bias and model bias in predictive features

Reputational Before 9 After 4

A predictive or segmentation model is trained on a sample that under represents some customer groups, so forecasts and rankings are systematically less accurate for them and decisions drift unfairly.

Who or what is affected: Customers in under represented groups, sales and service teams and the analytics team

Causes

Historic data that reflects past practices, small samples for some groups and features that stand in for sensitive attributes.

Consequences

Unfair treatment of customers, poor forecasts, reputational harm and complaints.

Existing controls

  • Model outputs are reviewed by an analyst before use.
  • Training data is documented with its source and time window.

Control gaps

Only overall accuracy is reported, so gaps between groups are not visible.

Additional controls

  • Prevent: Check sample coverage across key segments before training and document any gaps.
  • Detect: Compare error rates across segments each time the model is refreshed.
Likelihood 3 x impact 3 before; 2 x 2 after Owner: Senior analyst Target: Within 90 days Early warning: A gap in forecast error between two customer segments that widens after a refresh.
H08

Irreproducible analysis and uncontrolled change

Operational Before 9 After 6

Transformation code and report logic are edited directly in production without version control or testing, so earlier results cannot be rebuilt and a change silently alters past figures.

Who or what is affected: Developers, analysts, auditors and decision makers who rely on repeated reports

Causes

Urgent fixes made by hand, no separate test copy and undocumented assumptions.

Consequences

Results that cannot be explained, repeated disputes and wasted time reconciling past reports.

Existing controls

  • Code is stored in a shared repository with named authors.
  • Major releases are announced to users in advance.

Control gaps

Hotfixes bypass review, and no one checks whether older reports still reproduce.

Additional controls

  • Prevent: Require peer review and a test environment before any change reaches production.
  • Detect: Compare key outputs before and after each release and record any difference.
  • Recover: Keep previous versions of code and reports so they can be reinstated quickly.
Likelihood 3 x impact 3 before; 2 x 3 after Owner: Analytics manager Target: Within 45 days Early warning: A production change made without a linked review record.

Rating method and evaluation

Each risk is rated on two five-point scales. Likelihood runs from Rare to Almost certain, and impact runs from Insignificant to Catastrophic. Multiplying the two gives a score from 1 to 25, placed in one of four bands: Low from 1 to 4, Medium from 5 to 9, High from 10 to 16 and Critical from 17 to 25. The score orders the work and is not a measurement. Inherent risk is the score before any controls. Residual risk is the score after existing and added controls, and it must never be higher than the inherent score. Impact is judged on wrong decisions, time to correct, harm to individuals, legal exposure and loss of trust, and a risk is never rated down only because it has not yet happened. Critical and High risks need senior attention and a dated plan. Medium risks need routine controls and a named owner. Low risks are accepted and watched. Ratings are challenged in two ways. Developers and analysts review likelihood, since they see failures first, and the data steward reviews impact, since they know who relies on each figure. Any rating can be reopened when an indicator reaches its threshold or when a report is found to be wrong.

Likelihood, impact and rating bands

Likelihood

  • 5 Almost certain: Expected to happen in most circumstances.
  • 4 Likely: Will probably happen in many circumstances.
  • 3 Possible: Could happen at some time.
  • 2 Unlikely: Not expected, but it could happen.
  • 1 Rare: Only in exceptional circumstances.

Impact

  • 1 Insignificant: No lasting effect. Handled within routine work.
  • 2 Minor: Small effect that is quickly corrected.
  • 3 Moderate: Noticeable effect that needs managed effort to correct.
  • 4 Major: Serious effect on people, delivery or reputation.
  • 5 Catastrophic: Severe or lasting effect that threatens the whole activity.

Bands (likelihood x impact)

  • Low (1 to 4): Accept and monitor.
  • Medium (5 to 9): Manage with routine controls.
  • High (10 to 16): Senior attention and a treatment plan.
  • Critical (17 to 25): Act first. Do not proceed until reduced.

Monitoring, escalation and review

Review this assessment every 6 months, and sooner when an indicator below reaches its threshold.

Early-warning indicators and escalation thresholds
Indicator to watchEscalate when
Scheduled loads that fail or finish later than their agreed window.Escalate to the lead developer at once for any failed load, and to the head of analytics if a dashboard is stale for more than one business day.
Data quality test failures or rows rejected at the warehouse load.Escalate to the data steward when failures exceed the agreed threshold in a week, and hold publication until the cause is recorded.
Users with access to personal data who do not need it for their role.Escalate to the privacy lead at once, and remove the right within five working days.
Personal records held past their scheduled retention date.Escalate to the privacy lead when any such records are found, and to the governance group if the backlog is not cleared within 30 days.
Licensed dataset used for a purpose not listed in the vendor agreement.Escalate to the procurement lead and legal adviser at once, and pause the use until the position is confirmed.

Contingency arrangements

If a serious event occurs, such as a confirmed exposure of personal data, a licence breach or a dashboard published with wrong figures, the lead developer withdraws the affected report and pauses the related load. The data steward informs business owners which figures to avoid, and the privacy lead assesses whether affected people must be told. Data engineers restore the warehouse from the latest tested backup and rerun the affected jobs. Vendor contact and any legal notice go through the head of operations as required in the relevant jurisdiction.

Prioritized action plan

  1. Publish a metric dictionary with a named steward for each definition, owner data steward, within 30 days.
  2. Add automated data quality tests to every scheduled load and stop publication on failure, owner lead developer, within 45 days.
  3. Review dashboard and warehouse access by role and remove unused rights, owner analytics manager, within 30 days.
  4. Create a retention and deletion schedule for personal data in the warehouse, owner privacy lead, within 60 days.
  5. Check vendor licence terms against actual use and record each permitted purpose, owner procurement lead, within 60 days.

Conclusion

Trustworthy business intelligence rests on a few steady habits: documented definitions, automated checks on every load, role based access, dated retention rules, tested restores and a clear owner for each dataset. The highest scores sit with pipeline failure and data quality, and each has actions that bring the residual score down. Treat the register as a living record, review it on the stated cycle, and seek qualified advice for any real privacy, licensing or legal question.

Assumptions and limitations

Assumptions

  • The organization operates in one jurisdiction that is not named here, so privacy duties are described in general terms only.
  • Source systems are managed by other teams, and the team has read access only through agreed interfaces.
  • Dashboards inform management decisions but are not used alone for regulated or safety-critical actions.
  • The team uses version control, scheduled jobs and a standard reporting platform, with no unusual tooling.

Limitations

  • The scores are examples for learning and do not describe any real organization or data platform.
  • Privacy, licensing and records rules differ by country and contract and change over time, so specialist review is needed.
  • This is educational content and is not legal, data protection or procurement advice.

Related frameworks to study

  • Data governance operating model with defined owners and stewards
  • Three lines model of risk ownership
  • Hierarchy of controls
  • Plan, do, check, act improvement cycle

Technical Risk Assessment

China, India, United States, Indonesia, Pakistan, Nigeria, Brazil, Bangladesh, Russia, Japan, Mexico, Philippines, Vietnam, Egypt, Turkey, Germany, United Kingdom, France, Italy, Canada, South Korea, Australia, Spain.

For Learning Reference Only: This assessment demonstrates how risk assessment principles are applied in a practical setting, providing a useful reference for learning and understanding. Actual assessments may be adapted to suit specific site conditions and circumstances.

Keep learning

Study the subject behind this assessment.

Business Intelligence Development course Read the best practices for this course

More Technical risk assessments

Hourly Active: 35,661 Learners