Safety Norms: AI Solution Development

1Purpose, Scope, and Safety Principles

These norms set baseline expectations for anyone who frames, collects, labels, trains, evaluates, integrates, deploys, or retires an AI solution. They apply to employees, contractors, students, researchers, and vendors working on internal tools, customer-facing assistants, decision support systems, and automated classifiers. They cover the full lifecycle, from the first problem statement to the final decommissioning of a model and its data.

Two labels are used throughout. A mandatory rule is written with the word must and reflects organizational policy or a legal obligation the team cannot waive. A recommendation is written with should and reflects good practice that may be adapted if the team records its reasons and the model owner approves the change.

Four principles guide every decision. People come before performance, so a model that improves average accuracy but harms a group of users needs review before release. Data is treated as a hazard as well as an asset. No automated output may affect a person's rights, health, finances, employment, or access to services without a defined human check. Problems are reported early and never hidden to protect a schedule.

2Roles, Competence, and Accountability

Every AI project names one accountable model owner who has authority to stop, delay, or roll back the solution. This person owns the risk assessment, the release decision, and the monitoring plan. Tasks may be shared, but accountability cannot be delegated. Small teams may hold several roles, yet no one should approve their own work when an independent reviewer is available.

Competence is verified before access, not assumed. Engineers must complete role-based training on privacy, secure coding, and the organization's AI policy before receiving access to production data or model endpoints. Staff handling sensitive data, such as health, financial, biometric, or children's information, need additional authorization from the data steward and privacy lead. Electrical work, hardware installation, and clinical or legal validation must be performed only by qualified professionals.

• Executive sponsor: funds the project, accepts residual risk beyond team authority, and provides resources for safety reviews.
• Model owner: signs the risk assessment, approves release, and decides on rollback.
• Data steward: grants and reviews dataset access and confirms lawful basis and retention rules.
• Privacy and security leads: review data flows, threat models, and access controls before launch.
• Domain experts: confirm that outputs are correct and suitable for the intended use context.
• Operations on-call lead: monitors dashboards, executes runbooks, and escalates incidents.
• Legal counsel: confirms intellectual property, consumer protection, and sector obligations, which vary by jurisdiction.

3Course-Specific Hazards and Risk Assessment

AI solution development involves hazards that differ from ordinary software work. Training data may contain personal information, copyrighted material, or content collected without valid permission. Models can memorize and reproduce sensitive records. Outputs may be fluent but wrong, a failure commonly called 'hallucination', and users may trust them more than they should. Adversaries can manipulate inputs through prompt injection, poison training data, or extract model behavior through repeated queries. Automated decisions can also encode historical bias and treat groups unfairly.

Physical and environmental hazards are present as well. Training servers and GPU workstations draw high electrical loads, generate heat, and may use liquid or forced-air cooling. Equipment racks are heavy and can tip. Battery-powered laptops used for field data collection can overheat if damaged or charged improperly. Long screen sessions cause eye strain and musculoskeletal problems. Large training jobs consume significant energy, which should be measured and justified.

Each hazard is scored for likelihood and severity on a simple low, medium, and high scale. Any use case affecting safety, legal status, health, finances, employment, housing, education, or essential services is high risk by default. Activities the organization has prohibited, such as covert monitoring or social scoring, are stop items and must not proceed. A written risk assessment, signed by the model owner, must exist before personal data is collected or any model is shown to real users. The assessment is updated whenever data sources, architecture, or intended use changes.

• Personal data exposure: high severity. Control with minimization, pseudonymization, and access logging.
• Biased outputs: high severity for decision support. Control with subgroup evaluation and documented acceptance thresholds.
• Hallucinated or unsafe content: medium to high. Control with grounding, output filters, and clear user disclosure.
• Prompt injection and data poisoning: high for systems with tool access. Control with input isolation and data provenance checks.
• Secrets leakage: high. Control with a secrets manager and repository scanning.
• Electrical and thermal hazards: high in server rooms. Control with qualified maintenance and monitored cooling.
• Ergonomic strain: medium. Control with adjustable workstations and scheduled breaks.

4Preventive Controls and Safe Development Practices

Controls should be layered so that a single failure does not cause harm. Data controls come first. Collect only the fields needed for the stated purpose, document the source and license for each dataset, and remove or mask direct identifiers before data reaches experimental notebooks. Production data must not be copied to laptops, personal cloud storage, or unapproved tools. If the lawful basis or consent scope is unclear, collection stops until the data steward and legal counsel confirm it. Confirmation rules vary by jurisdiction and must be verified locally.

Model controls require evidence before release. Each model is evaluated against a documented test set that reflects the real population and the high-risk scenarios identified in the risk assessment. Teams test accuracy, robustness to noisy or adversarial inputs, subgroup performance, privacy leakage, and harmful content generation. Any failed threshold blocks release unless the model owner and an independent reviewer approve a documented exception. Red-team exercises should run before launch for conversational or agentic systems and again after major changes.

Operational safeguards keep the system safe after release. Deploy changes through staging, use feature flags or canary releases, and keep a tested rollback path. Systems that take actions, such as sending messages, changing records, or moving funds, must require human approval for high-risk steps. Users must receive clear notice that they are interacting with AI and how to reach a human reviewer, where applicable rules require it. Monitor drift, error rates, complaint trends, and unusual query patterns, and set alert thresholds before launch rather than after an incident.

• Must: keep code, data versions, and model artifacts under version control with reviewer sign-off.
• Must: scan repositories and pipelines for credentials, and rotate any exposed secret immediately.
• Must: restrict production endpoints to authenticated, least-privilege roles.
• Must: block release when a required safety test fails unless a documented exception is approved.
• Should: publish a model card and dataset sheet describing purpose, limits, and known failure modes.
• Should: include a domain expert in acceptance testing for medical, legal, financial, or safety content.
• Should: version prompts, retrieval sources, and system instructions and review them like code.

5Equipment, Environment, Information, and Protective Safeguards

Workstations and servers need basic physical safeguards. Maintain electrical equipment according to manufacturer instructions and local electrical rules, and allow only qualified electricians to work on power distribution. Server racks should be anchored and loaded according to the manufacturer's guidance. Staff must not lift heavy hardware alone, and they must follow the power-off or lockout procedure set by the facility owner before touching live equipment. Cooling and power alarms must route to on-call staff, and temperature thresholds must be set with the facility team. Specific limits depend on the equipment and must be confirmed with its manufacturer.

Personal protective equipment is usually limited for office and laboratory-style development work. Field and data center tasks may require closed-toe footwear, task-rated gloves, hearing protection in high-noise rooms, and eye protection during cable or hardware work. Workstations should have adjustable chairs, monitors at eye level, and scheduled breaks. Staff who report pain, numbness, or eye strain should be referred to occupational health services. Ergonomic assessments are recommended for teams that spend long hours at keyboards.

Information safeguards are mandatory. Use multi-factor authentication for all development and cloud accounts. Encrypt data at rest and in transit. Classify datasets as public, internal, confidential, or restricted, and apply access rules to each class. Set retention periods and delete data when the approved purpose ends, including backups and derived embeddings where policy applies. Mask personal data in prompt and output logs unless the privacy notice allows retention. Tracking compute hours and energy use is recommended, and unnecessary training runs should be avoided.

6Incident Reporting, Emergency Response, and Escalation

Anyone who observes an AI incident must report it to the on-call lead and the model owner as soon as safely possible, and no later than the deadline in the local incident policy. Reportable events include privacy exposure, suspected data breaches, harmful or discriminatory outputs reaching users, unauthorized model changes, security compromise, unexpected automated actions, and near misses where harm was avoided only by chance. Good-faith reporters must be protected from blame. Concealing an incident is a policy violation.

Containment comes first. The on-call lead may disable an endpoint, switch to a fallback, revoke credentials, or roll back to the last approved version, and this authority must be written into runbooks. Preserve logs, prompts, outputs, and model versions for investigation. The privacy lead and legal counsel decide whether individuals, customers, or regulators must be notified. These obligations and timelines are jurisdiction-dependent and must be verified with qualified advisers.

For physical emergencies, call local emergency services first. Fire, electrical shock, burns, and crush injuries require evacuation or first aid by trained responders. Nobody should repair energized equipment, enter a smoking or flooded server room without training and authorization, or move an injured person unless trained to do so. Post-incident reviews must be blameless, identify root causes, and assign corrective actions to named owners with agreed completion steps.

7Records, Training, Inspection, and Review

Records prove that controls were applied and allow independent review. Maintain a risk register, data inventory, dataset sheets, model cards, test reports, release approvals, access logs, training completion records, and incident files. Keep records for the period required by policy and law, which varies by jurisdiction and sector. Records should be searchable and linked to the model version they describe.

Training must be role-based and refreshed when rules change or a significant incident occurs. Every project should include a short onboarding module covering the policy, data classes, prohibited uses, and the escalation path. Teams should run incident drills, such as a simulated prompt injection leak, at a defined interval and record the findings and fixes.

Monitoring and review close the loop. The model owner sets a review interval for production metrics, complaint trends, and incident logs. The risk assessment must be repeated when the use case, data, or user population changes. Access reviews remove unused accounts promptly. These norms should be reviewed after each significant incident, after relevant legal changes, and at a regular interval. Approved changes are communicated to all staff and affected contractors.

Implementation checklist

• Confirm the use case, risk level, and any prohibited elements before collecting data.
• Document the lawful basis, license, and retention period for every dataset.
• Assign a named model owner with authority to stop or roll back the system.
• Verify role training and access approval before anyone touches production data.
• Run subgroup, robustness, privacy, and harmful-output tests and record the results.
• Require human approval for high-risk automated actions and publish clear user notices.
• Keep secrets out of code and enforce multi-factor authentication on all accounts.
• Restrict server room work to qualified staff and follow lockout and alarm procedures.
• Report incidents and near misses within the policy deadline and preserve evidence.
• Review the risk assessment, access list, and monitoring metrics at the set interval.