vaayaa Inclusive Education
Technical risk assessment

Data Engineering Solutions Risk Assessment: Access, Pipeline Integrity and Privacy Controls

A worked example of how a data engineering team finds, scores and controls the risks that threaten pipelines, data stores and the people whose information they hold, from access misuse and misconfiguration to outages and privacy duties.

Updated November 2026 8 hazards assessed Review every 6 months Highest risk: Critical to Medium
See the Data Engineering Solutions course

Executive summary

This assessment shows how a data engineering team can find, rate and control the risks that threaten the data it moves and stores. Eight hazards were chosen because they are plausible in everyday pipeline and platform work. Compromised credentials and access misuse carry the highest inherent score because they can expose large volumes of personal data. Misconfigured storage, phishing on engineer laptops and insider misuse also score High before controls. Each hazard is scored before controls and again after existing and proposed controls. Most fall to Medium or Low once the additional controls are in place. The first actions are removing long-lived keys, blocking public storage access, phishing resistant sign in and automated checks on data contracts. The organization in this example is hypothetical, so every score illustrates method, not a real platform.

Focus area
Data engineering, pipelines and data platforms
Setting
Data platform team that ingests, transforms and serves analytics and operational data for a mid-sized organization
Method
Five by five likelihood and impact risk register

Scope

This assessment covers a data engineering function that builds and runs ingestion pipelines, transformation jobs, storage platforms and data products used by analysts and business applications. It looks at risks to confidentiality, integrity and availability of data, to the people described by that data, and to the legal and contractual duties that apply. It does not cover legal advice or any real organization. Jurisdiction, data types, cloud provider and team size are assumptions made for teaching.

Context

The team uses managed cloud storage, a workflow scheduler, a distributed processing engine and a central catalogue. Source data comes from customer relationship, billing and product systems, plus files from third parties. Engineers work from laptops with shared code repositories, and a small operations group handles on-call support. Each risk is scored before controls to show its natural level, then again after existing controls and the additional controls proposed here.

Objectives, assets and stakeholders

The objective is to deliver accurate, timely and protected data to users, while meeting privacy commitments and keeping the platform available. The assets in scope are source and processed datasets, storage and processing platforms, the workflow scheduler, code repositories, backups, and the credentials that control them. Stakeholders include data engineers, analysts, the operations group, the data protection lead, business data owners, customers whose information is held, third-party providers and regulators where duties apply. Clear roles matter: each hazard has one owner role, the platform lead maintains the register, and the data protection lead reviews it with the business owners each quarter.

Risk matrix: before and after controls

Each cell shows hazard numbers placed by likelihood and impact. Compare the two grids to see how controls move each hazard.

Before controls

Inherent risk

Risk before controls: rows are likelihood from almost certain to rare, columns are impact from insignificant to catastrophic. Cells list hazard numbers.
1
Insignificant		 2
Minor		 3
Moderate		 4
Major		 5
Catastrophic
5 Almost certain Likelihood 5, impact 4, score 20, critical: H01
4 Likely Likelihood 4, impact 3, score 12, high: H03 H05 Likelihood 4, impact 4, score 16, high: H02
3 Possible Likelihood 3, impact 4, score 12, high: H06 H07 Likelihood 3, impact 5, score 15, high: H04 H08
2 Unlikely
1 Rare

After controls

Residual risk

Risk after controls: rows are likelihood from almost certain to rare, columns are impact from insignificant to catastrophic. Cells list hazard numbers.
1
Insignificant		 2
Minor		 3
Moderate		 4
Major		 5
Catastrophic
5 Almost certain
4 Likely
3 Possible
2 Unlikely Likelihood 2, impact 2, score 4, low: H05 H07 Likelihood 2, impact 3, score 6, medium: H02 H03 H06 H08 Likelihood 2, impact 4, score 8, medium: H01 H04
1 Rare
  • Low 1 to 4
  • Medium 5 to 9
  • High 10 to 16
  • Critical 17 to 25

Hazard register

A summary first, then the detail for each hazard.

Summary of hazards with scores before and after controls
No.HazardBeforeAfterOwner
H01 Compromised credentials and unauthorized access to data stores 20 Critical 8 Medium Identity and access manager
H02 Misconfigured storage exposing datasets publicly 16 High 6 Medium Platform engineering lead
H03 Phishing and malware on engineer workstations 12 High 6 Medium Security operations lead
H04 Insider misuse of sensitive personal data 15 High 8 Medium Data protection lead
H05 Data integrity failures from schema drift and unchecked transformations 12 High 4 Low Data quality lead
H06 Third-party and open source supply chain exposure 12 High 6 Medium Supplier and vendor manager
H07 Pipeline outage and failed recovery of critical data products 12 High 4 Low Operations and reliability lead
H08 Privacy breach from over retention and unapproved data transfers 15 High 6 Medium Privacy and compliance officer
H01

Compromised credentials and unauthorized access to data stores

Technology and data Before 20 After 8

Service account keys or engineer passwords are stolen, reused or left in code, allowing an attacker or unauthorized user to read or change large datasets in storage and processing platforms without being noticed.

Who or what is affected: Customers whose records are stored, data engineers and the business owners of data products

Causes

Keys stored in code repositories, shared service accounts and missing multi-factor sign in for some roles.

Consequences

Large scale data disclosure, regulator notification duties, loss of customer trust and costly clean up.

Existing controls

  • Engineers sign in through single sign on with individual accounts.
  • Storage buckets are private by default and access is granted per project.

Control gaps

Service account keys are not inventoried, and access logs are not reviewed for reads outside normal hours.

Additional controls

  • Prevent: Remove long-lived keys from code, use short-lived workload identities and rotate any key that is found.
  • Detect: Alert on unusual access volume, new keys and sign ins from unfamiliar locations.
  • Respond: Disable affected identities within one hour of a confirmed compromise and review all reads from them.
Likelihood 5 x impact 4 before; 2 x 4 after Owner: Identity and access manager Target: Within 30 days Early warning: A secret or key pattern detected in a code commit, or a service account reading far more rows than its normal daily volume.
H02

Misconfigured storage exposing datasets publicly

Technology and data Before 16 After 6

A storage bucket, table or export link is set to allow public or broad access during testing or troubleshooting and is left that way, so sensitive extracts can be read by anyone who finds them.

Who or what is affected: Data subjects, analysts, and the owners of the affected data products

Causes

Manual changes to production, copy of templates with loose settings and pressure to unblock a deadline.

Consequences

Unauthorized disclosure, breach reporting duties and reputational damage.

Existing controls

  • Infrastructure changes are made through version controlled templates.
  • A scheduled scan lists storage with public access once a week.

Control gaps

Weekly scans leave a window of several days, and exceptions have no expiry date.

Additional controls

  • Prevent: Block public access at the account level and require approval for any exception with an expiry date.
  • Detect: Run continuous configuration checks and alert on any change that widens access.
  • Recover: Revoke links, rotate affected secrets and record which objects were exposed.
Likelihood 4 x impact 4 before; 2 x 3 after Owner: Platform engineering lead Target: Within 30 days Early warning: Any storage object or table that changes to public or link sharing without a linked change request.
H03

Phishing and malware on engineer workstations

Technology and data Before 12 After 6

An engineer opens a malicious attachment, approves a fake sign in prompt or runs a compromised package, giving an attacker a foothold to alter pipeline code, scheduler jobs or stored data.

Who or what is affected: Engineers, pipeline owners and downstream data consumers

Causes

Targeted phishing, unpatched software and engineers with broad local rights.

Consequences

Tampered pipelines, stolen credentials, lateral movement and delayed releases.

Existing controls

  • Company laptops run endpoint protection with automatic updates.
  • Staff complete annual security awareness training.

Control gaps

Endpoint alerts are not linked to repository and scheduler activity, so changes are hard to trace.

Additional controls

  • Prevent: Require phishing resistant multi-factor sign in and restrict administrator rights on laptops.
  • Detect: Monitor endpoints for unusual processes and changes to pipeline code shortly after a sign in.
  • Respond: Isolate the device, reset its credentials and review commits made since the suspected time.
Likelihood 4 x impact 3 before; 2 x 3 after Owner: Security operations lead Target: Within 60 days Early warning: A repeated sign in prompt that the user did not start, or a new scheduled job created outside the normal release path.
H04

Insider misuse of sensitive personal data

People Before 15 After 8

A staff member or contractor with read access copies, exports or shares personal records for private use, or looks up records without a business need, and the activity is not noticed for some time.

Who or what is affected: Data subjects, colleagues and the organization's reputation

Causes

Broad read access, weak offboarding and limited oversight of admin queries.

Consequences

Privacy harm to individuals, disciplinary and legal action, and loss of trust.

Existing controls

  • Access to personal data requires a documented business need.
  • Staff sign a confidentiality agreement when they join.

Control gaps

Access reviews happen once a year and do not check which fields were actually read.

Additional controls

  • Prevent: Mask personal fields by default and grant unmasked access only for named tasks with a time limit.
  • Detect: Flag bulk exports and reads of records that no current task requires, and review them monthly.
  • Respond: Suspend access during an investigation and follow the documented employment process.
Likelihood 3 x impact 5 before; 2 x 4 after Owner: Data protection lead Target: Within 60 days Early warning: A user who exports more personal records in a week than their role has exported in the previous quarter.
H05

Data integrity failures from schema drift and unchecked transformations

Operational Before 12 After 4

A source system changes a field type or meaning, or a transformation silently drops or duplicates rows, so downstream reports and applications show wrong figures without any error being raised.

Who or what is affected: Analysts, business decision makers and customers receiving data products

Causes

Unannounced source changes, weak typing in transformations and missing reconciliation checks.

Consequences

Wrong decisions, rework, loss of confidence in data products and contract penalties.

Existing controls

  • Transformations are stored in version control and peer reviewed.
  • Row counts are logged for each daily load.

Control gaps

Row counts are logged but not compared with expected ranges, so silent changes pass.

Additional controls

  • Prevent: Enforce data contracts with each source that define types, keys and allowed ranges.
  • Detect: Add automated tests for null rates, duplicates and totals against the source each run.
  • Recover: Hold publication of affected tables and reprocess from the last validated snapshot.
Likelihood 4 x impact 3 before; 2 x 2 after Owner: Data quality lead Target: Within 45 days Early warning: A daily load where the row count or total moves outside an agreed band compared with the source.
H06

Third-party and open source supply chain exposure

Supplier Before 12 After 6

A managed service, third-party data feed or open source library is compromised, changes behavior without notice or stops supplying data, and this flows into the platform before the team can react.

Who or what is affected: Data engineers, data subjects in shared datasets and partner organizations

Causes

Reliance on a few providers, unvetted open source packages and feeds with no change notice.

Consequences

Compromised data, outages, unexpected schema changes and contract disputes.

Existing controls

  • Third-party providers are listed in a supplier register with a named contact.
  • Dependencies are pinned to specific versions in the build.

Control gaps

Supplier security questionnaires are not refreshed, and feed format changes are not monitored.

Additional controls

  • Prevent: Scan dependencies before adoption and approve new libraries through a short review.
  • Detect: Subscribe to security advisories for critical components and check hashes of inbound files.
  • Respond: Pause the affected feed, roll back to a pinned version and notify dependent teams.
Likelihood 3 x impact 4 before; 2 x 3 after Owner: Supplier and vendor manager Target: Within 90 days Early warning: A dependency with a new advisory or a third-party file whose checksum or column layout differs from the last delivery.
H07

Pipeline outage and failed recovery of critical data products

Operational Before 12 After 4

A scheduler failure, cloud region problem or failed deployment stops daily loads, and the restore process takes longer than expected because backups are untested or runbooks are out of date.

Who or what is affected: Operations staff, dashboard users and customers relying on timely data

Causes

Single scheduler instance, untested restores and deployments made without a rollback path.

Consequences

Missed reporting deadlines, customer service impact and manual rework.

Existing controls

  • Pipelines retry failed tasks up to three times.
  • Backups of core tables are taken nightly.

Control gaps

Restores have never been timed end to end, and runbooks do not name a decision owner.

Additional controls

  • Prevent: Deploy pipelines through staged environments with automated rollback on failure.
  • Respond: Follow a written runbook that names who decides to switch to the standby process.
  • Recover: Test a full restore of one critical table each quarter and record the time taken.
Likelihood 3 x impact 4 before; 2 x 2 after Owner: Operations and reliability lead Target: Within 60 days Early warning: Two consecutive missed loads of a critical data product or a restore test that exceeds the agreed recovery time.
H08

Privacy breach from over retention and unapproved data transfers

Legal and compliance Before 15 After 6

Personal data is kept longer than its purpose requires, copied into development environments or moved to a location without an approved transfer basis, so the organization breaches its privacy commitments.

Who or what is affected: Data subjects, the legal and compliance team and the business owners of the data

Causes

Retention rules that are not enforced by code, ad hoc copies made for testing and unclear transfer rules.

Consequences

Regulatory inquiry, fines where applicable, mandatory notifications and customer harm.

Existing controls

  • A retention schedule lists the main datasets and their review dates.
  • Production data is not copied to development by default.

Control gaps

The retention schedule is not linked to the platform, so expired data is not removed automatically.

Additional controls

  • Prevent: Automate deletion of expired records and block copies of personal data into non-production stores.
  • Detect: Report datasets with no owner or no retention date each month.
  • Respond: Log each transfer exception and suspend it until a privacy review confirms the basis.
Likelihood 3 x impact 5 before; 2 x 3 after Owner: Privacy and compliance officer Target: Within 90 days Early warning: A dataset with personal fields older than its retention date or a new copy of production data in a test environment.

Rating method and evaluation

Each hazard is rated on two five-point scales. Likelihood runs from Rare to Almost certain, and impact runs from Insignificant to Catastrophic. Multiplying the two gives a score from 1 to 25, which falls into one of four bands: Low from 1 to 4, Medium from 5 to 9, High from 10 to 16 and Critical from 17 to 25. The score guides the order of work and is not a precise measurement. Inherent risk is the score before any controls. Residual risk is the score after existing and additional controls are in place. Impact considers the number of people affected, the sensitivity of the data, the duration of disruption, legal exposure and harm to trust. A risk is never rated down only because it has not yet happened. Anything still High needs senior attention and a dated plan. Anything Medium needs routine controls with a named owner. Anything Low is accepted and watched. Engineers and data owners should challenge every rating, and each rating is reviewed whenever a warning indicator reaches its threshold or a material change is made to the platform.

Likelihood, impact and rating bands

Likelihood

  • 5 Almost certain: Expected to happen in most circumstances.
  • 4 Likely: Will probably happen in many circumstances.
  • 3 Possible: Could happen at some time.
  • 2 Unlikely: Not expected, but it could happen.
  • 1 Rare: Only in exceptional circumstances.

Impact

  • 1 Insignificant: No lasting effect. Handled within routine work.
  • 2 Minor: Small effect that is quickly corrected.
  • 3 Moderate: Noticeable effect that needs managed effort to correct.
  • 4 Major: Serious effect on people, delivery or reputation.
  • 5 Catastrophic: Severe or lasting effect that threatens the whole activity.

Bands (likelihood x impact)

  • Low (1 to 4): Accept and monitor.
  • Medium (5 to 9): Manage with routine controls.
  • High (10 to 16): Senior attention and a treatment plan.
  • Critical (17 to 25): Act first. Do not proceed until reduced.

Monitoring, escalation and review

Review this assessment every 6 months, and sooner when an indicator below reaches its threshold.

Early-warning indicators and escalation thresholds
Indicator to watchEscalate when
Secrets or keys detected in code commits or in build logs.Escalate to the identity and access manager at once, revoke the key within one hour, and inform the security operations lead if any use is found.
Storage objects or tables with public or link sharing enabled.Escalate to the platform engineering lead immediately, and to the data protection lead if personal data is involved.
Daily loads with row counts outside the agreed band compared with the source.Escalate to the data quality lead on the first breach, and hold publication if the breach persists after one rerun.
Personal records exported or read beyond the normal range for a user role.Escalate to the data protection lead within one working day, and suspend access pending review if misuse is suspected.
Restore tests that exceed the agreed recovery time for a critical data product.Escalate to the operations and reliability lead for a corrective plan within 10 days, and to senior management if the target is missed twice.

Contingency arrangements

If a serious event occurs, such as a confirmed credential theft, an exposed bucket or a corrupted critical table, the operations lead opens the incident process and the security lead isolates affected identities and systems. Publication of affected data products stops. The platform team restores from the latest validated backup and records the time taken. The privacy officer assesses notification duties with qualified advice, and business owners are told what they need to know early and plainly.

Prioritized action plan

  1. Remove long-lived keys from code and enforce short-lived workload identities, owner identity and access manager, within 30 days.
  2. Block public access on storage accounts and require expiring exceptions, owner platform engineering lead, within 30 days.
  3. Move engineer sign in to phishing resistant multi-factor and restrict local administrator rights, owner security operations lead, within 60 days.
  4. Automate deletion of expired personal data and block copies into test environments, owner privacy and compliance officer, within 90 days.
  5. Publish data contracts for each source and add automated reconciliation tests, owner data quality lead, within 45 days.

Conclusion

Reliable data engineering depends on a few steady habits: short-lived credentials, least privilege access, tested restores, enforced data contracts, deletion on schedule and a clear owner for every dataset. The highest scores sit with access compromise, misconfiguration and insider misuse, and each has actions that bring the residual score down. Treat the register as a living record, review it on the stated cycle, and seek qualified specialist advice for any real privacy, security or legal question.

Assumptions and limitations

Assumptions

  • The organization processes personal data and is subject to privacy duties, but the specific jurisdiction is not named, so legal obligations are described in general terms only.
  • Infrastructure is managed in a public cloud with role-based access, and engineers use company-issued laptops with central device management.
  • Third-party data feeds and open source libraries are used, and contracts with the main providers exist but are not reviewed here in detail.

Limitations

  • The scores are examples for learning and do not describe any real platform, team or incident.
  • Security, privacy and data protection rules differ by country and change over time, so specialist review is needed.
  • This is educational content and does not replace a formal site-specific assessment or advice from a licensed security or legal professional.

Related frameworks to study

  • CIA triad of confidentiality, integrity and availability
  • Least privilege principle
  • Defense in depth layered controls
  • Three lines model of risk ownership
  • Data lifecycle model from collection to deletion

Technical Risk Assessment

China, India, United States, Indonesia, Pakistan, Nigeria, Brazil, Bangladesh, Russia, Japan, Mexico, Philippines, Vietnam, Egypt, Turkey, Germany, United Kingdom, France, Italy, Canada, South Korea, Australia, Spain.

For Learning Reference Only: This assessment demonstrates how risk assessment principles are applied in a practical setting, providing a useful reference for learning and understanding. Actual assessments may be adapted to suit specific site conditions and circumstances.

Keep learning

Study the subject behind this assessment.

Data Engineering Solutions course Read the best practices for this course

More Technical risk assessments

Hourly Active: 45,552 Learners