Data Analytics Applications Case Studies
Technical5 Case StudiesUpdated Sep 2026
Data analytics applications sit between raw records and the decisions people make, so failures tend to surface in pipelines, dashboards, access rules, and the habits of the teams that maintain them. The five cases below involve fictional organizations and cover a production incident, a reporting conflict, a suspected compromise, an architecture choice under load, and the evaluation of an automated scoring tool. Each case weighs the same four concerns: system requirements, performance evidence, access controls, and maintainability. The decisions are ordinary, and the trade-offs are real.
Case Study 1
Stale promotional prices after a Thursday release
Tallowmere Grocers Cooperative, a regional grocery cooperative running a shared inventory and pricing platform for forty member stores.
The situation
Tallowmere's pricing service began returning stale promotional prices about forty minutes after a Thursday release. The release bundled a schema change to the product cache with a new batch job that refreshed promotions every fifteen minutes. Store managers started calling the help line when checkout totals disagreed with shelf tags. The analytics team owned the nightly aggregation that fed the margin dashboard, and their first alert came from a sudden drop in rows written to the promotions table. Query latency on the reporting database had climbed to nearly eleven seconds at the median, up from under one second on the previous release. Nobody had a clear picture of which change was responsible.
The challenge
The on-call engineer, Marisol, had to restore prices for thirty-one stores without freezing the release history, which the audit group needed intact. Two people had admin rights on the cache and the batch scheduler, and the shared service account was used by three vendors. Rolling back the whole release would also undo a fix for a tax rounding bug. Leadership wanted service back before the weekend promotion window opened in the next eight hours.
Analysis
Teams grounded in Data Analytics Applications usually begin by separating what the system was required to do from what it was observed doing, because the release notes describe intent and the telemetry describes behavior. Marisol pulled query plans, cache hit ratios, and job run logs for the two hours on either side of the deploy, then compared them against the service's documented freshness requirement of no more than fifteen minutes of drift. That comparison narrowed the problem to the batch job, which was reading from a cache key that the schema change had renamed. The evidence was specific enough to act on before the cause was fully understood.
Access was the second thread. The shared service account had write permission on the promotions table, which meant the batch job, two vendor integrations, and any manual script ran under the same identity. Logs could not say which actor issued the faulty writes. Marisol asked for a separate service identity per job and a short list of who could change scheduler definitions. Maintainability mattered as much: the batch job had no owner listed in the repository, and its configuration lived in a spreadsheet that the store operations team edited by hand. Nobody could safely change it without knowing who else depended on it.
- Release goes liveSchema change and new batch job deployed together.
- Stale prices appearStore managers report totals that disagree with shelf tags.
- Latency alert firesMedian reporting queries climb past eleven seconds.
- Batch job rolled backPrior cache key restored while schema change stays.
- Service confirmed restoredMissed windows rerun; promotions table matches source.
Actions taken
- Rolled the promotions job back to the prior cache key while keeping the schema change in place, then reran the missed windows for each store in order of sales volume.
- Created a dedicated service identity for the batch job and each vendor integration, with write access limited to the tables each one actually touched. Revoked the shared account.
- Moved the job definition and its configuration into the repository with a named owner, a change review step, and a README listing downstream consumers before any further edits.
- Added alerts on promotion row counts, cache miss rates, and freshness lag, with thresholds written into the runbook and tested against the last two incidents.
Results
Service was restored in just under three hours, and the weekend promotion window opened with correct prices in every store. Median query latency returned to under one second before the end of the same afternoon. The audit group received its release history intact, and the review traced the faulty writes to the batch job without relying on guesswork. In the following quarter, the freshness alert fired during a vendor outage, and the on-call engineer resolved it before any store reported a problem. The spreadsheet configuration is still being migrated into the repository, which is slower than anyone wanted.
Questions for discussion
- Which evidence would have revealed the schema rename before the release went out, and who should have been responsible for checking it?
- How should a team decide whether a shared service account is worth keeping for a vendor integration that cannot yet use its own identity?
- What belongs in a runbook for a scheduled job that has no named owner and no documented downstream consumers?
Case Study 2
Conflicting on-time figures in a board packet
Ostrander Transit Authority, a regional bus and rail operator that publishes ridership and on-time figures to its board each month.
The situation
Two reports reached the board in the same week with different on-time figures for route 14 in the previous month. The operations dashboard showed 91 percent of trips within schedule tolerance. The board packet, assembled by the finance analysts from a separate extract, showed 84 percent. Both teams believed their numbers came from the same automatic vehicle location feed. Riders had already begun asking why published schedules did not match what they experienced at stops. A council member asked for the figure to be explained before the next meeting, and nobody could say with confidence which number was right. The gap was roughly seven points, large enough to change how the route was described in the service review.
The challenge
Dariusz, the data steward, had to decide which figure could go into the board packet without a full rebuild of the pipeline. The finance extract applied a timestamp filter in local time, while the operations feed stored UTC, and a daylight time change had shifted some trips across the cutoff. Neither team had documentation for the filter, and the transformation code was owned by a contractor whose agreement had ended. The board meeting was in the second week, and the review could not wait for a full audit.
Analysis
Analysts facing reporting disputes often treat a number as a lineage problem before it becomes a judgment call. Dariusz mapped each figure back through its transformations, recording the source table, the filter, the join keys, and the time zone assumption at every step. Row counts were the first check. The operations feed contained 12,400 trip records for the month, while the finance extract held 11,870, and the gap clustered in the final two days before the cutoff. Once the clustering was visible, the timestamp logic became the obvious suspect, and it could be tested against a sample of 300 trips with known arrival times from the vehicle logs.
The second problem was access. The contractor had left a shared login on the reporting database, and the finance analysts could rewrite the view that defined on-time status without any review step. Dariusz proposed that the view be owned by the data team, that finance get read-only access to published tables, and that every change to the filter logic carry a version tag and a named reviewer. Maintainability depended on documentation that did not exist yet. A short data dictionary now defines the on-time metric, including the time zone, tolerance window, and cancelled trip rule, and it sits beside the code.
Actions taken
- Rebuilt the on-time view with explicit local time conversion and a tolerance window, then reran the month for both routes and the full network to check the totals.
- Added row count reconciliation between the vehicle feed, the staging tables, and the published view, with a daily check that blocks publication when counts diverge by more than half a percent.
- Removed the shared login, moved finance to read-only access on published tables, and assigned the data team ownership of every view that defines a board metric.
- Put filter logic under version control with a named reviewer for each change, and attached the data dictionary to every board packet so readers see the definition used.
Results
The corrected figure for route 14 was 88 percent, which sat between the two earlier numbers and reflected the time zone error at the cutoff. The board received a short note explaining the discrepancy and the corrected series for the previous six months. Two later reports drew on the same published view without a single reconciliation query from the analysts. Since then the daily check has held back one publication for a schema change upstream, and the owners fixed it the same morning. The team still has to retire two legacy extracts that nobody has used in months.
Questions for discussion
- Which controls should decide whether a dashboard number is allowed to disagree with a published one, and who should hold that decision?
- When a contractor has left and the code has no owner, what is the minimum documentation needed before anyone changes a metric?
- How would you set the threshold for a reconciliation check so that it catches real errors without blocking routine publication?
Case Study 3
Suspicious export from a service account
Bellwether Credit Union, a member-owned lender whose analytics group builds risk models used in underwriting.
The situation
A fraud analyst noticed an unusual pattern in the nightly export logs. A service account used by the member risk dashboard had downloaded roughly 48,000 member records over a single weekend, far outside its normal range of a few hundred rows per day. The download came from a hosting provider's address, not the credit union's network. The analyst raised the alert with the security lead, Okonkwo, who confirmed that the account's password had been rotated only once since it was created. No one knew whether the access came from a leaked credential, a misconfigured integration, or an insider testing something they should not. The dashboard had been built for a pilot and never went through a formal access review.
The challenge
Bellwether was a member-owned lender, so any notice to members had to be accurate and timed carefully, and the regulator expected a documented response. The security team could not tell whether the export had reached anyone outside the organization. Logs for the dashboard server were kept only seven days, and the first alert came five days after the export. The response lead had to contain the account without shutting down a risk model that underwriting used every morning, and every step had to be authorized and recorded.
Analysis
Good responders scope a compromise from the edges inward, starting with what the affected identity could reach rather than what the attacker is known to have done. Okonkwo listed every table, view, and export path the service account had permission to touch, then compared that list with the query history for the previous thirty days. The account could read a raw identity table the dashboard never used, which explained the large export. Before containing anything, the team exported the relevant logs, hashed them, and stored copies in a location the dashboard owners could not modify. Evidence that disappears during a containment step cannot be reconstructed later.
Access control was the part that needed the most work. The service account carried a password that had changed only once since creation, sat in a configuration file on the dashboard server, and had no expiry. Okonkwo proposed moving it to a managed identity with a short token lifetime, limiting its read rights to the two tables the model actually used, and requiring a named approver for any new grant. Maintainability was the quieter concern. The dashboard had no owner in the access register, and its integration documentation described a process that no longer existed. Until someone owned it, every future fix would depend on memory and luck.
- Scope the exposure Map what the identity could reach and what it actually touched.
- Preserve evidence Hash and copy logs to storage the affected owners cannot change.
- Contain the risk Disable or narrow the credential while keeping the model running.
- Document and notify Record each approved step and send member notices within counsel's window.
Actions taken
- Disabled the service account, preserved its logs and configuration snapshot under a hold, and routed dashboard users to a read-only copy of the morning risk file.
- Pulled every export made by the account in the past thirty days, matched each file hash against the access log, and listed the recipients for the legal and compliance review.
- Issued a new managed identity with a short token lifetime and read rights limited to two tables, after written approval from the security lead and the model owner.
- Drafted the member notice with counsel and the regulator liaison, set a review date, and recorded each authorized step in the incident log with the approver's name.
Results
Containment took about four hours from the first alert, and the dashboard returned to service the next morning under its new identity. The review traced the large export to a contractor's test script that reused the production credential, which turned a suspected intrusion into a serious process failure. Affected members received notice inside the window counsel had set, and the regulator accepted the written timeline without a follow-up request. The team has since removed shared credentials from three other integrations and is still working through the rest of the access register.
Questions for discussion
- What evidence should be captured before a containment step, and who should be allowed to hold it?
- How should a team decide whether a shared credential can be replaced in place or must be retired during an incident?
- Which parts of a response could be written down in advance so that the next one moves faster without skipping approvals?
Case Study 4
Estimate latency and cost across three designs
Lindqvist Freight Exchange, a marketplace that matches shippers with independent carriers and publishes live rate estimates.
The situation
Rate estimates on the Lindqvist marketplace had grown slow enough that shippers were abandoning quotes before booking. The estimate service priced 2,300 active lanes from fuel indices, carrier capacity, and lane history held in one relational database. Peak hours brought bursts of several thousand requests per minute, and median response time had moved from under a second to more than six seconds in one quarter. Finance flagged the database cluster as the largest cloud cost, much of it read replicas added during earlier incidents. The platform group had three proposals and one budget cycle to choose. The customer team wanted fast estimates, but they also wanted the data kept inside the existing security boundary.
The challenge
Ruth, the platform lead, had to choose between moving the estimate workload to a managed cache with precomputed lane prices, adding a streaming pipeline that updated prices as fuel and capacity changed, or splitting the lanes into regional services. Each option carried different cost and data handling exposure. Precomputed prices would be fast but might be stale by up to fifteen minutes during volatile fuel swings. Regional splitting would reduce blast radius but multiply the number of deployments the three-person operations team had to maintain.
Analysis
Architecture work starts with the workload shape, since a design that looks cheap on average traffic can fail on peak requests. Ruth pulled a week of request logs and separated them into three classes: shippers checking a single lane before booking, bulk requests from integration partners, and internal recalculations triggered by fuel index updates. The peak load was dominated by the first class, which needed sub-second answers but tolerated small price differences. Existing latency evidence showed the database spending most of its time on joins across lane history, not on the fuel lookup, which pointed toward precomputation for the common path and a narrower live path for exceptions.
Access rules shaped the options too. Lane history contained rate agreements with carriers, and some limited which shippers could see them. A regional split would copy that data into several services, each with its own access list, which multiplied the audit surface. A precomputed cache held the same prices but could be read by any service with the right key, so it needed the same row-level restrictions as the source. Maintainability decided the rest. The operations team had three people, and each new deployment meant another set of alerts, secrets, and rollback steps to keep current. The streaming option was the most capable and the hardest to keep correct.
Actions taken
- Built a benchmark that replayed a week of real request traffic against the precomputed cache design and the streaming design, recording p95 latency and database cost for each.
- Set a freshness requirement of fifteen minutes for fuel-driven prices and a five-second ceiling for single-lane requests, then tested each option against both limits before scoring.
- Applied row-level access rules to the cache and the regional replicas through the same policy engine as the source, and ran an access review on every service identity before traffic moved.
- Chose the precomputed cache for the common path, kept a narrow live query for stale-sensitive exceptions, and wrote a runbook with one owner and a rollback that finished within ten minutes.
Results
The precomputed path brought median response time under a second and p95 time under five seconds during the next peak. Database spending for the estimate workload fell by roughly a third once the read replicas were retired. The access review found two service identities with broader rights than they needed, and both were narrowed before launch. The streaming design remains on the roadmap for fuel-sensitive lanes, but the team decided it was not worth the operating cost for the current volume.
Questions for discussion
- What volume would have to arrive before the streaming design became the cheaper option to run?
- How should a team decide how stale a price can be before the business stops accepting it?
- When two designs both pass the performance test, which operational risks should decide between them?
Case Study 5
Screening tool pilot with hidden bias risk
Corrigan Hill Staffing, a placement firm that screens applicants for contract roles across manufacturing and logistics clients.
The situation
Corrigan Hill's recruiting group was asked to adopt a vendor tool that ranked incoming applications and flagged candidates for recruiter review. The vendor claimed the tool would cut screening time by more than half and reduce the backlog that had grown in the previous season. The recruiting group handled about 4,200 applications a month across eleven client accounts. During a short pilot, the tool ranked applicants who listed employment gaps lower, and its scores correlated with postal codes in ways that were hard to explain. Recruiters liked the speed. Compliance staff wanted proof that the scores did not function as a proxy for protected characteristics, and the client contracts required a documented human decision for every rejection.
The challenge
Hiram, the operations director, needed to settle two things before the next client review: whether the pilot should keep running, and what evidence would justify expanding it. The vendor offered a dashboard of accuracy numbers, but the test set was drawn from their own benchmark and had no client-specific roles. Recruiters were already using the scores informally to sort queues. The pilot could not be stopped cleanly without a week of rework on the intake forms, and a client audit was scheduled in the next six weeks.
Analysis
Evaluation has to begin with the decision the tool will influence, because accuracy on a benchmark says little about who gets screened out in practice. Hiram's team defined the metrics before looking at the vendor numbers: agreement between the tool's shortlist and a recruiter's blind shortlist on the same 300 applications, false rejection rate for candidates later hired by clients, and score differences across groups that the firm was obliged to monitor. Time to decision was tracked as well. The benchmark had none of these measures, so the pilot had to produce its own evidence, with a holdout of applications that recruiters reviewed without seeing any score.
Access and maintainability formed the other half. The scores were generated by a model the vendor hosted, which meant applicant data left the firm's boundary, and the contract allowed the vendor to use anonymized records for training. Compliance wanted that clause removed and retention shortened. Maintainability raised a harder question. The vendor could update the model without notice, so a score that was acceptable in one month might shift in the next, and the firm had no change log to explain why a shortlist looked different. The team asked for version pinning, a notice period before any model change, and a way to export scores so analysts could recompute them.
Actions taken
- Ran a blind comparison on 300 applications, with recruiters scoring first and the tool second, and recorded every disagreement with the reason given by each reviewer.
- Wrote a release gate requiring score gaps across protected groups to stay within a threshold the compliance team approved, and blocked expansion if any monitored gap widened.
- Required a recruiter to record a human decision for every rejection, with the score visible only after the reviewer had entered an initial view, and logged each override.
- Pinned the vendor model version, removed training rights from the contract, and set a thirty-day notice requirement before any model change reached live applicant queues.
Results
The pilot was extended to two client accounts, not the full eleven. Time to first decision fell by about 40 percent in those accounts, but the blind comparison showed the tool disagreed with recruiters on a share of strong candidates with short employment gaps. Those cases now go to a manual queue. Compliance accepted the expansion plan on the condition that gap-related scoring was reviewed each month. The team has not yet decided whether the tool earns a place in the process, and that question will go back to the operations director at the end of the quarter.
Questions for discussion
- What should count as enough blind evidence before a screening tool is allowed to influence which applicants reach a recruiter?
- How would you separate a real signal in applicant data from a proxy for something the firm cannot legally use?
- When a vendor controls the model, what contract terms and internal records would let the firm explain a shortlist months later?
Learning Resources
Technical CASE STUDIES
China, India, United States, Indonesia, Pakistan, Nigeria, Brazil, Bangladesh, Russia, Japan, Mexico, Philippines, Vietnam, Egypt, Turkey, Germany, United Kingdom, France, Italy, Canada, South Korea, Australia, Spain.