Safety Norms: Software Quality Assurance and Test Automation

1Purpose, Scope, and Safety Principles

This document sets norms for people who plan, build, run, and review software testing, including manual testers, automation engineers, quality leads, release managers, and anyone who operates test environments or test pipelines. Its purpose is to prevent harm while still allowing teams to find defects early and report accurate release information. In this field, harm usually means exposed personal data, unintended changes to live systems, disruption of shared services, leaked credentials, and release decisions based on misleading results.

The norms cover test design, test data, automation code, continuous integration pipelines, device labs, performance and security testing, defect tracking, and test reporting. Where local approved procedures, the organization's information security policy, or privacy and contractual obligations are stricter, those take precedence.

• MANDATORY: Test only systems for which you hold written authorization from the system owner.
• MANDATORY: Never run automation, load, or security tests against production unless an approved change or test window exists and the release manager has signed off.
• MANDATORY: Use synthetic or approved masked data. Real personal data may enter a test environment only through the data steward approval process.
• RECOMMENDED: Design tests so that failure is safe, meaning a broken test cannot delete, send, or charge anything real.

2Roles, Competence, and Accountability

Each test environment, test suite, and dataset must have a named owner. The test lead owns the test plan and scope. The environment owner controls access, configuration, and backups. The data steward approves any use of production-derived data. The security contact reviews security test scope and findings. The release manager decides whether known defects or unverified risks may proceed to release.

Competence comes before access. Before receiving production-like credentials, a tester or automation engineer must complete onboarding covering environment rules, data handling, secret management, and the tools in use. Engineers whose tests call external APIs, change state, or run at volume need supervised experience with those systems first. Authorization is given in writing and limited to a defined environment, purpose, and time period set by the environment owner.

• MANDATORY: Each role holder must confirm in writing that they have read these norms and the applicable local procedures.
• MANDATORY: Shared administrator accounts are prohibited for test execution. Each person uses an individual account.
• RECOMMENDED: Pair new automation engineers with an experienced reviewer for their first changes that touch shared environments.

3Course-Specific Hazards and Risk Assessment

The hazards in this field are mostly digital and operational, but they can cause serious harm. Common examples include automation pointed at a live endpoint because a configuration value was wrong, copied production records containing personal or financial information, credentials embedded in scripts or pipeline logs, load tests that degrade a shared service, destructive test steps such as bulk deletion or mass email, and security scanning of systems outside the approved scope. Flaky tests, unreviewed third-party plugins, and test devices with damaged batteries also create risk.

Assess risk before starting new tools, environments, data sources, performance tests, security tests, and any test that writes, sends, pays, or deletes. Rate each hazard by likelihood and severity using four levels: low, medium, high, and critical. High and critical risks require documented controls and approval from the environment owner before work begins. Record every assessment in the risk register and repeat it when scope changes.

• Critical: test traffic or writes reaching production systems or real customers.
• High: real personal, financial, or health data appearing in test artifacts.
• High: secrets leaking through source control, logs, screenshots, or test reports.
• Medium: load or stress tests affecting shared services without notice or a stop condition.
• Medium: unvetted libraries, browser extensions, or plugins in the automation toolchain.
• Low to medium: flaky tests that hide real defects or trigger repeated false alarms.

4Preventive Controls and Safe Work Practices

Separation is the first control. Test environments must use hostnames, accounts, and credentials distinct from production. Automation frameworks should read the target environment from an explicit setting and refuse to run when that setting is missing or matches a production naming pattern. Network egress from test environments should be restricted so test code cannot reach live third-party services unless the owner has approved the connection. Use vendor sandboxes for payment, mail, and messaging services wherever the provider offers them.

Test data follows the minimum necessary principle. Synthetic data is the default. Where production-derived data is essential, the data steward approves a masking method, retention period, and access list before any copy is made. Test accounts receive only the rights each scenario needs. Destructive tests create their own records with identifiable prefixes and clean them up through tested teardown code. Tests must be safe to rerun, so they should not depend on leftover state from earlier runs.

Automation code is production-quality code. Secrets are retrieved from an approved secrets manager at runtime and never committed to source control. Secret scanning is required in the commit or pipeline stage. Dependencies are pinned, reviewed, and sourced from approved repositories. Test code receives the same peer review as application code when it touches credentials, environment selection, data generation, or external services.

• MANDATORY: Stop automation if the target environment is missing, unknown, or production-like.
• MANDATORY: Do not paste real customer records, tokens, or session cookies into tickets, chat, or screenshots.
• MANDATORY: Performance and load tests require an approved window, a rate limit, a named operator, and a stop condition the operator can trigger immediately.
• MANDATORY: Security testing requires written scope from the system owner and review by the security contact. Do not test third-party systems unless the provider has authorized it in writing.
• RECOMMENDED: Run a read-only or dry-run mode before any test that writes data for the first time.
• RECOMMENDED: Quarantine flaky tests with a named owner and a fixed review interval rather than deleting them or ignoring failures.
• MANDATORY: The release manager reviews open defects, skipped tests, and quarantined tests before sign-off. Any waiver must be written, specific, and time limited.

5Equipment, Environment, Information, and Protective Safeguards

Use managed workstations with current endpoint protection, full disk encryption, automatic screen locking, and approved remote access tools. Test devices in mobile and hardware labs need controlled physical access, asset tags, and a usage log. Lithium battery devices should be charged only with approved chargers, never left charging unattended, and removed from service if swelling, heat, or leakage is observed. Report such devices to the lab owner and follow the local hazardous material procedure rather than attempting repairs.

Access control uses individual identities, multi-factor authentication for test and build systems, and role-based permissions. Access to production-like data, secrets, and device labs is reviewed at an interval set by the environment owner. Administrative access to test infrastructure should be granted just in time where the platform supports it. Execution, credential, and data access logs are retained under local rules and protected from alteration.

Privacy and information handling are jurisdiction-dependent. Data protection laws, contracts, and internal classification rules determine what may be copied, where it may be stored, and how long it may be kept. Verify current requirements with the privacy office or qualified counsel. Screenshots, logs, and defect attachments must be scanned for personal data and redacted before leaving the team. Any test environment holding personal data takes the same classification as its source.

6Incident Reporting, Emergency Response, and Escalation

Report any suspected incident immediately, even when the cause is unclear. Incidents include a test that changed production data, a credential exposed in a repository or log, personal data found in a test artifact, a run that caused a service outage, and a security weakness found during testing. A near miss is an event that could have caused harm but did not, such as a configuration error caught before a run. Log near misses, because they reveal weak controls.

Immediate steps are the same for most incidents. Stop the running test or pipeline. Revoke or rotate any exposed credential through the approved process. Do not delete logs, screenshots, or data that may serve as evidence. Notify the environment owner and security contact through the incident channel. The incident owner decides whether privacy, legal, or customer notification is needed. Those obligations come from law and contract and vary by jurisdiction.

Physical emergencies override digital procedures. For battery fire, smoke, electrical shock, or injury, move people away from the hazard, call local emergency services, and follow the posted lab procedure. Do not fight a lithium battery fire with improvised methods or enter a hazard area without training. The lab owner must complete a written review before the area reopens.

• MANDATORY: Report within the time frame set by the internal incident policy. Never delay reporting to investigate first.
• MANDATORY: Do not use an exposed credential to test whether it still works.
• RECOMMENDED: Hold a blameless review after resolution, focused on control gaps rather than individual fault.

7Records, Training, Inspection, and Review

Keep records that allow another person to reconstruct what was tested, where, with which data, and who approved it. Minimum records include the test plan, scope and authorization letters, environment inventory, test data register, risk assessments, automation run logs, defect records, waivers, incident reports, and training completion. Store records in an approved system with access control and retention rules that match local obligations.

Training is required before access and must be refreshed when tools, data rules, or risk levels change. Core topics are environment separation, safe test data handling, secrets management, authorized security testing, incident reporting, and release gate procedures. The quality governance owner approves training content.

Inspection keeps controls working. Audit automation regularly for hardcoded hostnames, unmasked data, unreviewed dependencies, and leftover test accounts. The governance group reviews metrics such as escaped defects, flaky test rate, incident count, near-miss count, and overdue waivers. Use findings to tighten controls and retire unsafe practices.

• RECOMMENDED: Run secret scanning and test data audits at an interval set by the governance owner.
• MANDATORY: Retire any test account or credential no longer in use within the period defined by the access policy.

8Verification, Limits, and Conclusion

This guidance is educational and does not replace formal training, approved organizational procedures, supervision by qualified staff, or advice from legal, privacy, or security professionals. It does not establish legal requirements. Recognized testing and security frameworks, data protection rules, export controls, and vendor documentation change over time and vary by jurisdiction, so verify current official versions before relying on them.

Safe testing depends on discipline. When in doubt, stop the test, record the question, and ask the test lead or environment owner before continuing. A passing test that ran against the wrong system or used real data is a process failure, not a success.

Implementation checklist

• Confirm written authorization and the named target environment before every test run.
• Verify the target is not production and that automation refuses unknown targets.
• Use synthetic or approved masked data and record its source in the data register.
• Load secrets from the approved vault only and confirm secret scanning passed.
• Check that each test account is individual and carries only the rights the scenario needs.
• Obtain security contact approval before any security scan or penetration-style test.
• Set a stop condition and a named operator before any performance or load test.
• Redact personal data from logs, screenshots, and tickets before sharing.
• Report every incident and near miss within the internal window and preserve evidence.
• Review waivers, quarantined tests, and access grants at the defined review interval.