Chapter 9 of 13
GenAI Risk Scenarios: Mapping Generative Models and Use Cases
From copilots and chatbots to content generators and code assistants, GenAI use cases differ in who they affect and how they can fail. This module applies the Map function through the lens of the Generative AI Profile to structure GenAI-specific risk scenarios.
From AI RMF to GenAI Mapping: Why Scenarios Matter
From AI RMF to GenAI
The NIST AI RMF and its Playbook help turn high-level outcomes into concrete actions. The Generative AI Profile (NIST AI 600-1, 2024) extends this with GenAI-specific risks like hallucinations and synthetic media.
Why Mapping Is Different for GenAI
For generative models, mapping must go beyond traditional ML: prompts are open-ended, outputs can be long and multimodal, and one base model can appear as a chatbot, copilot, or API.
What You Will Practice
You will learn to recognize GenAI archetypes, map use cases to Generative AI Profile scenarios, and adapt intake and inventory templates to capture GenAI-specific details.
Step 1: Recognize GenAI System Archetypes and Deployment Patterns
GenAI Archetypes
The Generative AI Profile groups risks by archetypes. Start mapping by asking: is this a user-facing assistant, an embedded feature, an API-based model, or a content-generation pipeline?
User-Facing Assistants
These include chatbots and copilots where users prompt the model directly. Risks center on misleading advice, harmful content, and user overreliance on outputs.
Embedded and API-Based
Embedded features hide GenAI inside a larger product, while API-based models expose GenAI as a service. These patterns raise interaction, dependency, and misuse risks.
Why Archetypes Matter
Identifying the archetype narrows which Generative AI Profile scenario clusters are most relevant and helps you focus your risk analysis.
Step 2: Example – Classifying a GenAI Use Case by Archetype
Scenario: LMS Writing Helper
A university adds an AI writing helper to its LMS. Students highlight text and click “Improve with AI” to get clearer wording and grammar suggestions.
Archetype and Pattern
This is mainly an embedded GenAI feature inside the LMS, with a user-facing assistant element. The model is accessed via a cloud API using prompts built from student text.
Who Is Affected?
Students use it directly; instructors, academic integrity staff, and IT are indirectly affected by how it changes writing and grading practices.
Risk Flags from the Profile
Key risks: academic integrity and overreliance, data leakage to third parties, and potential bias in how writing is “improved” across different dialects or styles.
Step 3: GenAI-Specific Mapping – Inputs, Outputs, and Context of Use
GenAI Inputs: Prompt Channels
Map who can send prompts, through which channels (chat UI, API, IDE), how constrained those prompts are, and whether they contain sensitive data like personal information or exam content.
GenAI Outputs: Modalities and Flow
Record what the model outputs (text, images, code, JSON), whether outputs are shown directly or filtered, and whether they feed into other systems or workflows.
Context of Use Matters
Specify the domain, decision criticality, user expertise, and environment. The same hallucination risk looks very different in a casual brainstorm app versus a medical or legal assistant.
Why This Detail?
The Generative AI Profile scenarios are context-dependent. Capturing these details in intake and inventory templates lets you select the right scenarios and risk tier.
Step 4: Thought Exercise – Mapping Inputs, Outputs, and Context
Activity: Imagine your university wants to roll out a GenAI course-planning chatbot for students. The chatbot helps students draft a semester plan (courses, workload, extracurriculars) based on their major and interests.
Without worrying about the exact model, answer these mapping questions in your own words (you can jot notes on paper or in a text editor):
- Inputs (prompt channels)
- How do students interact with it? (mobile app, LMS, website?)
- What kind of prompts will they send? (free-text questions, form-based inputs?)
- Will they share sensitive data (GPA, mental health concerns, financial constraints)?
- Outputs (modalities and controls)
- What does the chatbot output? (course lists, explanations, schedules?)
- Are outputs just suggestions, or can they auto-populate official registration forms?
- Is there any human review before high-impact changes (like overloading a schedule)?
- Context of use
- What is the decision criticality? Could bad advice delay graduation or harm well-being?
- Are users mostly first-year students, international students, or others who may be more vulnerable?
After you answer, compare your notes to this checklist:
- You mentioned at least one prompt channel and whether prompts are free-form.
- You identified output types and whether they are advisory or drive actions.
- You described the stakes (what happens if the chatbot is wrong) and the user population.
If any of these are missing, add them. These details will later help you pick relevant Generative AI Profile scenarios (for example, student-facing advisory assistants, planning tools, and vulnerable users).
Step 5: Applying the Generative AI Profile Scenarios to a Use Case
From Map to Scenarios
After mapping archetype, inputs, outputs, and context, use the Generative AI Profile to find scenario clusters that match your system’s type and domain.
Three-Step Process
1) Start from archetype. 2) Filter by context and stakes. 3) Write 3–5 concrete “what could go wrong” statements for each relevant scenario cluster.
Example: LMS Writing Helper
Possible scenarios: hallucinated citations, loss of authentic student voice, data leakage to providers, and biased suggestions that penalize certain dialects.
Why This Matters
These concrete scenarios connect your system map to the Generative AI Profile and prepare you to select AI RMF Playbook actions to mitigate the risks.
Step 6: GenAI-Aware Intake Template (Structured Example)
Here is a simplified, GenAI-aware intake form schema you might use in JSON-like pseudocode to capture mapping details. You do not need to code this, but reading it helps you see what information should be collected.
```json
{
"system_name": "LMS AI Writing Helper",
"owner": "Center for Teaching and Learning",
"genai_archetypes": [
"embedded_feature",
"userfacingassistant"
],
"modelprovidertype": "thirdpartyapi",
"modelnameor_family": "(e.g., GPT-4 class, Llama 3)",
"prompt_channels": {
"interfaces": [
"LMStexthighlight_button"
],
"prompttype": "constrainedfree_text",
"containssensitivedata": true,
"examples": [
"Student paragraph with personal reflections",
"Course-specific essay drafts"
]
},
"outputs": {
"modalities": ["text"],
"primary_functions": [
"rewriteforclarity",
"grammarandspelling"
],
"post_processing": [
"toxicity_filter",
"length_limit"
],
"humanreviewrequired": false
},
"contextofuse": {
"domain": "higher_education",
"usergroups": ["undergraduatestudents", "graduate_students"],
"decision_criticality": "medium",
"environment": "university_LMS",
"special_considerations": [
"academic_integrity",
"nonnativespeakers"
]
},
"model_customization": {
"fine_tuned": false,
"retrieval_augmented": true,
"ragdatasources": [
"writingcenterguidelines",
"universitystyleguide"
]
}
}
```
Step 7: Quick Check – Mapping GenAI-Specific Details
Answer this question to check your understanding of GenAI-specific mapping.
Which pair of fields is MOST important to add when adapting a traditional AI intake form for a generative **chatbot** use case?
- Model accuracy metric and confusion matrix
- Prompt channels (who can ask what, through which interface) and output modalities (text, image, code, etc.)
- Database schema and feature engineering steps
- Server CPU type and data center location
Show Answer
Answer: B) Prompt channels (who can ask what, through which interface) and output modalities (text, image, code, etc.)
For GenAI mapping, you must capture how prompts enter the system (channels, users, constraints) and what kinds of outputs are produced (modalities, how they are shown or used). Traditional metrics, schemas, or hardware details may matter later, but they are not the most GenAI-specific additions to an intake form.
Step 8: Key Term Review – GenAI Mapping Concepts
Flip through these flashcards to review important terms for mapping GenAI risk scenarios.
- GenAI System Archetype
- A high-level pattern describing how a generative AI system is deployed and used (for example, user-facing assistant, embedded feature, API-based model, content-generation pipeline). It guides which risk scenarios from the Generative AI Profile are most relevant.
- Prompt Channel
- The path through which prompts reach a generative model, including the interface (chat, API, plugin), who can send them, how constrained they are, and whether they may contain sensitive data.
- Output Modality
- The form of the model’s output (text, image, audio, video, code, structured JSON, etc.). Different modalities create different risk profiles (for example, deepfake images vs. misleading text).
- Context of Use
- The real-world setting in which a GenAI system operates: domain, user population, decision criticality, and environment. The same model can pose very different risks in different contexts.
- Retrieval-Augmented Generation (RAG)
- A design where the model retrieves documents or data from external sources and uses them to condition its responses. Mapping must record what sources are used and how reliable they are.
- Generative AI Profile (NIST AI 600-1)
- A NIST document (first released in 2024) that extends the AI RMF with GenAI-specific risk scenarios and outcomes, organized by system archetypes and contexts of use.
Key Terms
- NIST AI RMF
- The NIST Artificial Intelligence Risk Management Framework, a voluntary framework (first released in 2023) for managing AI risks across functions: Govern, Map, Measure, and Manage.
- Prompt Channel
- The interface and pathway through which prompts or inputs reach a generative model, including who can send them and how structured or free-form they are.
- Output Modality
- The type or format of content generated by the model (for example, text, images, audio, video, code, or structured data).
- AI System Inventory
- An organization-wide catalog of AI systems, including generative systems, that records key attributes (purpose, data, risks, controls) for oversight and risk management.
- Decision Criticality
- A description of how high the stakes are if an AI output is wrong, ranging from low (entertainment) to high (healthcare, safety, legal, or financial decisions).
- GenAI System Archetype
- A category describing how a generative AI system is deployed and interacts with users or other systems, such as user-facing assistants, embedded features, or API-based models.
- Intake Form / Intake Template
- A structured questionnaire used when proposing or registering a new AI system, capturing details needed to assess risk and decide on governance requirements.
- Retrieval-Augmented Generation (RAG)
- An architecture where a generative model retrieves external documents or data to ground its responses, improving relevance but adding dependency on data sources.
- Generative AI Profile (NIST AI 600-1)
- A NIST profile that extends the AI RMF with generative AI-specific risks, scenarios, and outcomes, published in 2024 and used to tailor risk management to GenAI systems.