Generative AI Applications

01Purpose and scope

• This manual supports the course Generative AI Applications. It is written for engineers, analysts, product owners, and subject matter experts who build or maintain software that calls a generative model to draft text, summarize records, answer questions over internal documents, classify requests, or produce structured output for another system.
• The scope covers the working life of one application: choosing a task that suits a generative model, designing the prompts and data connections, testing behavior against expectations, deploying with controls, and keeping the application useful as models, data, and users change. It does not cover training foundation models from scratch, which is a separate discipline with different cost and infrastructure concerns.
• Practice varies by context. A internal drafting tool used by twelve analysts carries different risks than a customer-facing assistant that can change account settings. The same workflow applies to both, but the depth of testing, the review gates, and the amount of human oversight should scale with the consequences of a wrong answer. Where an application touches medicine, law, finance, tax, or safety, this guidance does not replace qualified professional judgment, approved procedures, or required authorization from the responsible organization.

02Core principles

• Start with the task, not the model. A generative model is a general tool that produces plausible continuations of its input. It is well suited to tasks where a reviewer can quickly judge the output, such as turning meeting notes into an action list, and poorly suited to tasks that require exact recall of facts with no source to check against. Write down the task in one sentence, the input it receives, the output it must produce, and who uses that output.
• Treat the model output as a draft unless the workflow has evidence that it is reliable for that specific task. Plausible wording is not the same as correct content. Design the application so that users can see sources, flag errors, and reach a person when the answer matters.
• Separate the parts you control from the parts you do not. You control the instructions, the retrieved context, the output format, the logging, and the fallback behavior. You do not control the model's internal behavior, and the provider may change a model version under you. Pin versions where the platform allows it and record which version produced each result.
• Prefer simple architectures. A single well-written prompt with a clear output schema often beats a chain of six agents. Each added component adds latency, cost, failure modes, and something to test. Add complexity only when a measured gap justifies it.
• Measure before and after every change. Changes to prompts, retrieval settings, or model versions can improve one class of input while breaking another. Without a fixed evaluation set, teams tend to judge changes by the last few examples they happened to try.

03Planning and preparation

• Begin with a short use case statement. Name the users, the task, the source material, the expected output, the people affected by errors, and the action taken on the output. If the action is irreversible, such as sending an email to a customer or updating a record, plan for a human approval step before that action runs. Stakeholders usually include the business owner, the engineering lead, a data steward for any source material, a security reviewer, a privacy or legal contact, and at least a few end users who will test early versions.
• Inventory the data. For applications that answer questions from documents, list the document sources, owners, update frequency, access rules, and personal or confidential content. Decide which content may be sent to an external model service and which must stay in an approved environment. This decision often determines the architecture more than any other, so resolve it before building.
• Build an evaluation set before building the application. Collect fifty to a few hundred realistic inputs drawn from logs, past tickets, or subject matter expert examples. For each one, write the expected answer or the criteria a correct answer must meet. Include hard cases, ambiguous inputs, and inputs that should be refused or escalated. This set becomes the reference for every later decision, so keep it versioned and protected from casual edits.
• Define success criteria in measurable terms before coding starts. Examples include the percentage of answers that a reviewer rates as factually supported by the cited source, the rate of responses that contain unsupported claims, median response time, and the share of cases correctly routed to a human. Set thresholds that the business accepts, and record who signed them off.
• Choose tools deliberately. Most teams compare a hosted model service with a self-hosted open model, and the trade-off usually involves data handling, cost per request, latency, control over versions, and the staff needed to operate the model. Orchestration frameworks can speed up early work but add abstraction layers, so keep the prompt and evaluation logic in code you can read and test.

04Recommended workflow

1. 1
   : Write the task specification. Describe the input format, the output format as a schema where possible, the allowed sources, the tone, and the explicit rules for what the application must not do, such as giving a legal conclusion or quoting a figure that is absent from the source. Keep the specification in the repository next to the code.
2. 2
   : Draft the system instructions and a small set of examples. Use plain language and state the reasons behind each rule, since the model handles reasons better than bare commands. Provide two or three examples that show a correct output for a typical input and one showing a refusal or escalation. Avoid stuffing the prompt with every edge case; move edge cases into the evaluation set instead.
3. 3
   : Connect the context. For retrieval-based designs, chunk the source documents at sensible boundaries such as section headings, store them with metadata like owner, date of last revision, and access group, and retrieve the top candidates for each query. Check retrieval separately from generation. If the right passage is not retrieved, no prompt change will fix the answer.
4. 4
   : Enforce the output format. Request structured output such as JSON with named fields, and validate it in code before anything downstream reads it. Reject or retry responses that fail validation, and log the failure. Free text is fine for a human reader, but automated steps need predictable fields.
5. 5
   : Run the evaluation set and score the results. Use a mix of automated checks, such as schema validity, required citations present, and banned phrases absent, and human review for factual support and usefulness. Automated judging by another model can speed up scoring, but spot check its judgments against human ratings before trusting it, because it tends to share the same blind spots as the model it grades.
6. 6
   : Iterate on one variable at a time. Change the instructions, the retrieval depth, or the model version in separate runs. Record the scores for each run in a simple table. When a change improves the average but worsens a critical category, such as questions about dosage or contract termination, treat it as a regression until the team decides otherwise.
7. 7
   : Pilot with real users under monitoring. Start with a limited group, collect feedback through a visible control such as a thumbs-up or a flag button, review a sample of conversations each week, and widen access only when the agreed thresholds hold. Keep a rollback path to the previous version or to the manual process.

05Quality checks, risk controls and compliance

• Quality checks should run at three points: before release, on every change, and continuously in production. Before release, the full evaluation set runs and a reviewer examines a random sample of outputs and all failures in the critical categories. On every change, a smaller regression set runs automatically. In production, sample logs are reviewed on a schedule and user flags are triaged within a set time.
• Guard against the common failure modes. Unsupported claims occur when the model states something absent from the retrieved context. Mitigate this by instructing the model to answer only from supplied sources, requiring citations to specific passages, and checking that cited passages exist and contain the claimed content. Prompt injection occurs when text inside a retrieved document or user input tries to override instructions. Treat retrieved text as data, separate it clearly in the prompt, limit what tools the model can call, and require confirmation for any action with side effects. Leakage of personal or confidential data can happen through logs, prompts sent to external services, or outputs shown to the wrong user, so apply the same access rules that govern the source systems.
• Risk controls should match the impact. For low-stakes internal drafting, a disclosure label and a feedback channel may be enough. For applications that influence decisions about people, money, health, or legal status, add mandatory human review, documented limits on use, and a record of each decision the application informed. Some uses may be prohibited or subject to specific legal requirements depending on where the application operates. Rules differ by jurisdiction and change over time, so check the current text of the applicable law or regulation with your legal or compliance team, and refer to official government sources rather than summaries.
• The NIST AI Risk Management Framework, published by the United States National Institute of Standards and Technology, is a voluntary framework organized around four functions: govern, map, measure, and manage. Many teams use it as a checklist vocabulary for documenting risks and controls. It is not a certification, and using it does not establish legal compliance on its own.
• Privacy and security reviews should happen before pilot use. Confirm the data retention terms of any external service, whether inputs are used for model improvement under your agreement, where data is stored, and who can access logs. Keep API keys in a secrets manager, restrict network paths, and log access to sensitive documents.

06Collaboration and documentation

• Keep one owner accountable for each application and a named reviewer for each change to instructions or data sources. Prompt text is production logic, so store it in version control with change descriptions, and run the evaluation set in the same pull request review that approves code changes. Product owners should review evaluation results and risk statements, while engineers own the implementation and the monitoring.
• Write a short model card or system card for each application. Include the purpose, intended users, out-of-scope uses, the model and version, the data sources and their owners, the evaluation results with the date-free version identifiers used in your repository, known limitations, the human review requirements, and the contact for reporting problems. Update the card when anything material changes. Auditors and new team members rely on it more than on memory.
• Document decisions, not only outcomes. When the team chooses a smaller model to reduce cost, record the quality difference observed, the accepted trade-off, and who approved it. When a category of question is excluded from scope, record the reason and how users are redirected.
• Coordinate with the teams that own the source systems. Retrieval quality depends on document owners keeping content current, and an application that quotes an outdated policy creates a real problem. Agree on a process for removing or correcting documents and for notifying the application team when a source changes.

07Measures and continuous improvement

• Track a small set of indicators that reflect both quality and use. Useful measures include the share of sampled answers rated fully supported by the cited sources, the rate of unsupported or contradicted claims in reviewed samples, the rate of escalations to humans and whether those escalations were appropriate, user-reported problems per hundred sessions, latency at the 95th percentile, and cost per completed task. Do not treat a single aggregate score as the whole story. Break results down by user group, document type, and question category.
• Review the evaluation set itself on a regular schedule. Add new failure cases from production, retire examples that no longer reflect real use, and check that the labels still reflect current policy. An evaluation set that stops changing gradually stops measuring what matters.
• Use production findings to decide where to invest. If most errors come from retrieval, improve chunking, metadata filters, or the document library before rewriting prompts. If most errors come from formatting, tighten the output schema and validation. If users ignore citations, test whether the citation display is clear enough.
• Plan for model and provider change. Model versions are deprecated, behavior shifts between releases, and pricing changes. Keep the evaluation set portable so that a new model can be compared quickly, and schedule a comparison before any forced migration.

08Common mistakes and how to prevent them

• Teams often skip the evaluation set and judge quality by demonstrations. A few impressive examples do not show how the application behaves on the ordinary range of inputs. Build the set first, even if it is small, and grow it from real failures.
• Teams often blame the model when retrieval is the problem. Log the retrieved passages for every answer and check whether the needed information was present. Many apparent hallucinations are retrieval misses combined with a prompt that allows the model to fill gaps.
• Teams often overload the prompt. Long instructions with contradictory rules produce inconsistent behavior. Keep the prompt focused, remove rules that the evaluation set shows are ineffective, and move special handling into code where it can be tested directly.
• Teams often launch without a human path. Users who receive a wrong answer with no way to report it lose trust quickly. Provide a visible flag, a named contact for problems, and an escalation route for questions outside scope.
• Teams often let the application grow beyond its approved purpose. A drafting assistant becomes a decision tool without new review. Tie each new use to a documented scope change and a new risk assessment, and make the change visible in the model card.
• Teams often forget logs are sensitive. Full prompts and outputs can contain personal or confidential content. Set retention periods, restrict access, and redact where the business allows it.

09Practical checklist

• Write a one sentence task specification that names the input, the output, the users, and the action taken on each output.
• Confirm which data may be sent to each model service and record the approval from the data owner and security reviewer.
• Build and version an evaluation set of realistic inputs with expected answers or acceptance criteria, including refusal and escalation cases.
• Agree on measurable release thresholds and record who approved them before development starts.
• Validate every structured output in code and log validation failures.
• Log retrieved passages and model version with each response so answers can be traced.
• Run the full evaluation set before pilot and the regression set before every change to prompts, retrieval, or model version.
• Review a weekly sample of production conversations and triage user flags within the agreed time.
• Keep an up to date model card, a change log for instructions, and a named owner for the application.
• Check the current applicable laws and internal policies with the responsible compliance contact before expanding scope.

★Learning sources

• Micro Degree: Generative AI Applications
• Micro Degree: Data Analytics Applications
• Micro Degree: Azure Cloud Fundamentals
• Micro Degree: Google Cloud Platform Basics