Software Architecture and Engineering Best Practices: Evidence on Adoption, Security and Operating Trade-offs

Summary

This report covers software architecture and engineering practice across the global technology and digital services sector, with U.S. breach-cost figures reported separately ¹². The source pack contains no verified market-size or spending data for architecture services, tools or skills, so the market is described qualitatively. Security is the best-quantified area. The latest IBM study puts the global average breach cost at USD 4.44 million, down from USD 4.88 million, while the U.S. average rose to a record USD 10.22 million ¹². AI use in engineering is widespread, but trust is limited ⁷. Claims about architecture patterns come mostly from vendors and practitioners. The main risks are weak AI access controls, software supply chain failures and cost claims based on single workloads ¹⁴⁸. Forecasts point to fast growth in embedded AI agents, but evidence that vendor-built agents will last is weak ¹¹¹².

Key findings

1. 1

   Breach costs are high and diverge by geography

   The latest IBM study reports a global average breach cost of USD 4.44 million, while the U.S. average reached USD 10.22 million. These are observed averages from different samples and should be compared with care.

   Confidence: High
2. 2

   Most organizations hit by AI-related incidents lacked AI access controls

   97% of breached organizations with an AI-related security incident lacked proper AI access controls. This is an observed finding from a large industry study, so it supports a governance priority for architects.

   Confidence: High
3. 3

   Broken access control ranks first among web application risks

   The current OWASP Top 10 ranks Broken Access Control first, Security Misconfiguration second and Software Supply Chain Failures third. OWASP describes the list as an awareness document, not a binding standard.

   Confidence: High
4. 4

   Developers use AI widely but many do not trust its output

   A secondary summary of the DORA survey reports that 90% of respondents use AI at work and 30% report little or no trust in AI-generated code. The primary report was not retrieved, so these figures need checking.

   Confidence: Medium
5. 5

   Architecture cost gains rest on a single company-reported case

   Amazon reported a 90% infrastructure cost reduction after moving one Prime Video monitoring workload from a distributed serverless design to a single ECS task. The figure is company-reported, covers one workload and has not been independently verified.

   Confidence: Qualitative
6. 6

   Vendor-built agent projects face a forecast risk of abandonment

   Gartner forecasts that 70% of enterprises will abandon agentic AI built through vendor forward-deployed engineering. This is a forecast, not a measured outcome.

   Confidence: Medium

1Industry definition and scope

For this report, the field covers the design, documentation, governance and delivery practices that determine how software systems are structured and built. It includes architecture patterns such as monolithic, microservice, event-driven and serverless designs, secure design and coding practice, software supply chain controls, and the delivery practices used to measure engineering performance ⁴⁶⁸. The pack does not provide an official taxonomy for the field, so these boundaries are inferred from the sources.

The evidence is global, except that the IBM breach-cost study reports a separate U.S. average ¹². The sources cover the latest available study periods: the latest IBM breach study, the current DORA survey, the current OWASP Top 10 and recent Gartner publications. Samples and definitions differ between studies, so their figures should not be combined into one measure.

In this report, 'vendor claims' means statements by firms that sell products or services, and 'secondary' means news or blog summaries of primary research. Several sources are vendor blogs or secondary summaries, so they are labeled as such throughout.

Scope was set by the topics covered by the source pack. No external data was added.

2Market size, structure and value chain

The source pack contains no independently verified market-size, revenue or growth figures for software architecture consulting, architecture tooling or architecture skills. Vendor market-size figures were excluded. The market therefore cannot be sized from these sources, and this report does not estimate it.

The value chain can be described by function. Architecture advice and design comes from internal teams, consultancies and vendor engineers. Cloud platforms supply the infrastructure, and UpCloud, a cloud provider, publishes guidance on standard infrastructure and exit paths ¹⁴. AI coding tools support development ¹³. Security and supply chain tooling and documentation tools add further layers ⁴⁵¹⁵.

Gartner describes a model in which vendors embed forward-deployed engineers to build agentic AI for customers ¹¹. Gartner also forecasts that 40% of enterprise applications will include task-specific AI agents by the end of the forecast window, up from less than 5% today ¹². Its best-case projection has agentic AI driving about 30% of enterprise application software revenue ¹². These are forecasts, not observed revenue.

The pack gives no market shares, so concentration cannot be assessed. Our interpretation is that value may shift toward firms that control integration, governance and cost management. This is an inference, not a verified finding.

• Enterprise applications forecast to include task-specific AI agents 40percent of enterprise applications Forecast Global, End of the forecast window stated by source [12]

Only forecasts are available for AI agent integration. No observed market values were found in the pack.

3Customers, demand and adoption

The main customers are enterprise engineering leaders, security teams and individual developers. Security teams face the highest cost exposure. The IBM study reports that global average detection and escalation costs fell almost 10% to USD 1.47 million, making them the largest cost driver for the fourth consecutive period ². Organizations identified and contained breaches in a mean of 241 days, the lowest in nine years ¹.

AI adoption is growing faster than trust. Secondary summaries of the DORA survey report that 90% of respondents use AI at work, while 30% report little or no trust in AI-generated code ⁷. Gartner reports that only 17% of organizations have deployed AI agents, although more than 60% expect to do so within two years ¹⁰. A Gartner-related secondary source reports that 90% of engineering leaders say AI coding agents have improved team output, with a net average productivity gain of 19.3% as reported ¹³. That figure was not checked on the Gartner document.

Buying criteria are not measured in the pack. Our reading, which is interpretation, is that security, access control and trust in AI output are likely to weigh heavily in purchase decisions, because the sources repeatedly highlight these risks ¹⁶⁷. Barriers include limited trust, unclear governance of AI systems and the cost of recovering from incidents ¹¹⁰.

The table mixes primary and secondary figures, so each row states its evidence status. Figures from different surveys are not directly comparable.

4Competitive landscape and key players

The pack names no market shares and no firm-level revenues, so competitive intensity cannot be quantified. The comparison below is by provider type. Several sources are vendor material, so their claims are presented as claims.

Cloud platform providers offer managed infrastructure. UpCloud's guidance recommends keeping an exit path from hyperscaler lock-in by running core workloads on standard infrastructure where possible ¹⁴. This is vendor guidance, and the pack contains no independent evidence on lock-in costs. Services-led vendors use forward-deployed engineers. Gartner forecasts that 70% of enterprises will abandon agentic AI built this way and that fewer than 20% of these engagements will turn recurring needs into core product capabilities ¹¹. These are forecasts from a research firm, not measured outcomes.

Documentation and tooling vendors make claims about the state of architecture records. DevPulse cites an industrial survey of 147 participants that found architecture documentation was often outdated and inconsistent ¹⁵. The survey is undated and its sampling is not stated, so it is indicative only. Research and reporting firms such as IBM with the Ponemon Institute provide breach benchmarks ¹, which are independent of product sales but still reflect their own methods.

Business models differ. Subscription platforms sell infrastructure and tools, while services models sell engineering time. The pack provides no comparable total-cost-of-ownership figures for either model, nor for open-source versus proprietary stacks, so a cost comparison is not possible. Competitive pressure is likely to come from the question of whether tools improve engineering discipline. DORA states that AI amplifies existing organizational strengths and weaknesses rather than acting as a silver bullet ⁶, which suggests that differentiation may come from engineering practice rather than from tool choice. This is our interpretation.

Comparisons rely on vendor, research and secondary sources. The pack did not include a neutral benchmark of providers.

5Technology, operations and workforce

Architecture pattern choice is contested in the sources. UpCloud advises starting simpler and letting production pain, rather than anticipated scale, drive architectural complexity ¹⁴. The pack contains no independent uptime, latency, throughput or defect-rate benchmarks comparing monolithic, microservice and event-driven designs, so the advice cannot be tested against measured performance.

Delivery performance is measured by DORA. A secondary summary reports that the current DORA survey replaced four performance levels with seven team archetypes and used nearly 5,000 responses ⁷. The primary report was not retrieved, and the effects of AI on change failure rate and throughput are described only in secondary summaries ⁶. Productivity claims should therefore be tested in an organization's own delivery data.

Security practice is shaped by OWASP. The current list ranks Broken Access Control first, Security Misconfiguration second and Software Supply Chain Failures third, while Insecure Design and Mishandling of Exceptional Conditions are separate categories ⁴. Xmart reports that 50% of community survey respondents ranked supply chain failures first, but the same source notes limited presence in datasets and 11 associated CVEs, so the figure does not show that half of applications have the problem ⁵.

Workforce evidence is thin. No official skills-demand statistics or job-posting data for software architects were retrieved. Gartner identifies agentic AI governance, agentic AI security and FinOps for agentic AI as emerging profiles ¹⁰, which suggests a growing skills need. This is an inference from market signals, not a measured shortage.

No operational cost or productivity benchmark for architecture styles was found in the pack.

6Regulation, standards, risk and sustainability

The pack contains no binding laws, regulator rules or formal requirements on software architecture. It does not identify a jurisdiction for binding obligations, so none are stated here. Organizations should check the rules that apply in the jurisdictions where they operate. This section is not legal advice.

OWASP Top 10 is a standard awareness document reflecting broad consensus on critical web application risks ⁴. It is voluntary guidance, not a certification or binding requirement. Privacy, environmental and social sustainability data were not in the pack, so these areas are not assessed.

Risk evidence focuses on breaches and AI. Insider attacks produced the highest average breach costs among initial threat vectors, at USD 4.92 million ³. Among organizations, 13% reported an AI-model incident that resulted in a breach ³. The IBM study finds that 97% of breached organizations with an AI-related security incident lacked proper AI access controls ¹. Trust and documentation quality add operational risk ⁷¹⁵.

• Global average cost of a data breach 4.44USD million per breach Observed Global, Latest available IBM study period [1]
• Breached organizations with AI-related incidents lacking proper AI access controls 97percent of affected organizations Observed Global, Latest available IBM study period [1]

The chart joins three observed figures from the same IBM study series but different reporting cuts.

7Challenges, opportunities and case examples

The main constraints are trust in AI output, access control for AI systems, and the cost of architecture choices. Opportunities include governance tooling, cost management and better documentation, all of which the sources connect to risk reduction ¹⁷¹⁰¹⁵. The pack gives no verified figures for the cost of these measures, so the trade-off cannot be quantified.

Amazon Prime Video offers the most detailed architecture case. The team reported that the distributed serverless design hit a hard scaling limit at around 5% of expected load in the design phase ⁸. Moving the workload into a single Amazon ECS task reduced cloud infrastructure costs by 90%, according to the company ⁸. Commentary notes that the service remained a microservices application, with one service refactored like a monolith ⁹. The team also cited high AWS Step Functions state transition costs and frequent S3 calls ⁹. This shows a trade-off for one workload, not a general rule.

Vendor claims should be separated from demonstrated capability. The 90% saving is company-reported and covers a single workload. Gartner's forecasts about vendor-built agents are predictions ¹¹. The productivity figure of 19.3% is secondary and not verified ¹³. Demonstrated evidence is strongest for security incident costs and for documented OWASP risks ¹⁴. Our interpretation is that architecture cost and productivity claims should be tested in the organization's own workloads before adoption.

• Reduction in cloud infrastructure cost for one Prime Video monitoring workload 90percent (company-reported) Observed Single Prime Video monitoring workload, Reported in the latest available case period [8]

Case evidence is limited to one company and one workload. Comparable cases were limited in the pack.

8Outlook and strategic implications

In the near term, AI agent use is expected to rise. Gartner reports that 17% of organizations have deployed AI agents, while more than 60% expect to do so within two years ¹⁰. Governance, security and cost management are expected to become more important, as the emerging Hype Cycle profiles suggest ¹⁰. This near-term view is based on survey expectations, not on measured outcomes.

In the medium term, Gartner forecasts that 40% of enterprise applications will include task-specific agents ¹². The forecast that 70% of enterprises will abandon vendor-built agentic AI ¹¹ signals risk for organizations that rely on outside builders. These forecasts are uncertain, and a secondary failure-rate prediction could not be verified.

Practical implications follow from the evidence. Stakeholders should prioritize AI access controls ¹, supply chain controls ⁴ and documentation quality ¹⁵. They should start with simpler architectures and adjust them as production evidence builds ¹⁴. Cost claims for architecture changes should be tested against their own data ⁸⁹. These implications are our interpretation of the sources, not verified findings.

Forecasts from Gartner were not checked against full paywalled reports.

9Conclusion

Security and AI governance are the best-evidenced priorities in this field, while architecture performance and market size remain unmeasured in the pack. The most defensible takeaways are that AI access control and supply chain security carry measurable risk ¹⁴, that AI adoption outpaces trust ⁷, and that architecture cost claims need workload-level testing ⁸.

How this was built

Sources

1. 1

   Cost of a Data Breach Report (IBM Cost of a Data Breach Report, latest edition)

   IBM with Ponemon Institute Company report

   ibm.com
2. 2

   Cost of a Data Breach Report coverage (Cyberscoop)

   CyberScoop News report

   cyberscoop.com
3. 3

   Ten Key Insights from IBM's Cost of a Data Breach Report

   Baker Donelson News report

   bakerdonelson.com
4. 4

   OWASP Top 10 (current edition)

   OWASP Foundation Standard

   owasp.org
5. 5

   OWASP Top 10 current edition and Software Supply Chain Failures

   Xmart Solutions (vendor blog) Vendor report

   xmartsolutions.com.br
6. 6

   State of AI-assisted Software Development (current DORA Report)

   DORA (Google Cloud DevOps Research and Assessment) Industry association

   dora.dev
7. 7

   DevOps Statistics: DORA Benchmarks and Adoption

   Eltexsoft (vendor blog citing DORA) Vendor report

   eltexsoft.com
8. 8

   Amazon Prime Video case: moving a monitoring service from distributed serverless to a monolith

   The Stack (technology news reporting on Prime Video engineering blog) News report

   thestack.technology
9. 9

   Prime Video monolith case: context and caveats

   Simform newsletter (secondary commentary) News report

   simformnewsletter.substack.com
10. 10

    Current Hype Cycle for Agentic AI

    Gartner Market research

    gartner.com
11. 11

    Gartner Predicts 70% of Enterprises Will Abandon Agentic AI Built by Vendor Forward-Deployed Engineering

    Gartner Market research

    gartner.com
12. 12

    Gartner Predicts 40% of Enterprise Apps Will Feature Task-Specific AI Agents

    Gartner Market research

    gartner.com
13. 13

    Gartner's Current AI Coding Agent Report: What It Actually Means for Your Build Decision

    Zetrix (knowledge-base article citing Gartner Magic Quadrant) Market research

    zetrixweb.com
14. 14

    Modern Software Architecture Patterns That Scale in Current Practice

    UpCloud (cloud provider blog) Vendor report

    upcloud.com
15. 15

    Software Architecture Document Guide and Template

    DevPulse (vendor blog) Vendor report

    devpulse.com