Measure function (in NIST AI RMF)

The function that defines, conducts, and documents evaluations of AI systems to generate evidence about performance, risks, and trustworthiness, supporting informed risk decisions and continuous improvement.

Evaluations that check how an AI system performs under variations, noise, distribution shifts, or stress conditions, revealing whether it generalizes beyond the training data and behaves reliably in realistic scenarios.

Bias and fairness assessment

Systematic measurement of model performance across groups (e.g., by race, gender, age) using metrics like false positive/negative rates, demographic parity, equal opportunity, and calibration to detect and mitigate unfair disparities.

Privacy impact analysis (e.g., DPIA)

A structured assessment of how an AI system collects, uses, and stores personal data, the risks to individuals' privacy, and the safeguards in place. Often required by regulations like the GDPR for high-risk processing.

Red-teaming (for AI systems)

A structured process where internal or external teams actively try to make an AI system fail or misbehave (e.g., jailbreak an LLM, extract data, bypass safety filters) to discover vulnerabilities before adversaries or the public do.

Changes over time in the distribution of input data that can cause AI model performance to degrade if not detected and addressed.

The degree to which an AI system maintains acceptable performance when faced with noise, distribution shifts, adversarial inputs, or other realistic variations from its training conditions.

A property of probabilistic predictions where predicted probabilities match observed frequencies (e.g., among people predicted to have a 0.7 risk, about 70% actually experience the event).

The US National Institute of Standards and Technology's AI Risk Management Framework (version 1.0 released in 2023), a voluntary framework that defines functions (Map, Measure, Manage, Govern) to help organizations manage AI risks.

Measure: Evaluating AI System Performance, Risk, and Trustworthiness — Operationalizing the NIST AI RMF and Generative AI Profile in Practice

Q: An online proctoring AI flags students as 'suspicious' during remote exams. Which metric is MOST important to examine to reduce the risk of unfairly accusing students, especially from marginalized groups?

False positive rate, broken down by demographic group. False positive rate by group directly reflects how often students are wrongly flagged as suspicious. High or uneven false positives can create serious fairness and reputational harms. Overall accuracy can hide group disparities, and training loss or parameter count do not directly relate to this risk.

1. What the Measure Function Is (and Why It Matters)

Measure in the AI RMF

In the NIST AI RMF (2023), Measure turns AI risk questions into concrete tests, metrics, and evidence. It connects your Map and Govern work to real evaluations of safety, fairness, security, and reliability.

Key Measure Outcomes

Measure covers four outcomes: M1 valid and reliable measurement, M2 robustness and generalization, M3 trustworthiness characteristics, and M4 continuous monitoring and improvement over time.

Why It Matters Now

As of 2026, the EU AI Act and US guidance all stress testing, documentation, and monitoring. Measure is how organizations create the evidence needed for risk decisions like deploy, restrict, redesign, or retire.

2. From Mapped Use Case to Measurement Questions

Start From the Use Case

Use your Map outputs. Example: a university admissions scoring model. You know its purpose, who it affects, and potential harms like unfair rejection, privacy risks, and over-reliance on scores.

Turn Context into Questions

Translate the mapped context into questions: How accurate is it overall? Does accuracy differ by group? What is the false negative rate for each group? Can small input changes flip decisions?

Rule of Thumb

If you cannot write a clear performance or risk question, you are not ready to pick a metric. Measurement must be driven by context and plausible harms, not just generic benchmarks.

3. Connecting Measure Subcategories to Activities

M1: Valid Measurement

M1 focuses on valid, reliable measurement. Activities include defining realistic evaluation datasets, picking appropriate metrics for the task, and documenting assumptions and limitations of each metric.

M2: Robustness

M2 covers robustness and generalization. You run robustness tests by adding noise or shifts, and stress tests for heavy load or partial outages to see how the system behaves under strain.

M3: Trustworthiness

M3 includes bias and fairness checks, privacy evaluations, security red-teaming, and explainability studies. These go beyond accuracy to measure how safe, fair, and transparent the system is.

M4: Monitoring

M4 is about continuous monitoring and improvement. Define drift and incident metrics, set thresholds that trigger review or rollback, and feed monitoring data back into governance and mapping.

4. Thought Exercise: Pick Metrics for a Real Use Case

Apply what you have learned to a different scenario.

Scenario: A hospital uses an AI model to predict which patients are at high risk of being readmitted within 30 days, to offer extra support.

Task: For each category below, jot down 1–2 metrics or tests you would use.

Performance

What metric(s) would you track to see if the model is good at identifying high-risk patients?
Hint: Think about false negatives (missed high-risk patients) vs false positives.

Bias and fairness

How would you check whether predictions are equitable across groups (e.g., age, gender, race, insurance type)?
Hint: Consider comparing error rates or calibration across groups.

Robustness

How would you test whether the model still works when hospital workflows change (e.g., new documentation system, new patient population)?

Privacy and security

What would you measure to ensure patient data is protected in model training and deployment?

Write your answers in a notebook or text editor before moving on.

When you are ready, move to the next step and compare your ideas with an example solution.

5. Example Answers: Hospital Readmission Model

Performance Metrics

For hospital readmission: use AUROC to see ranking quality, recall to avoid missing high-risk patients, and precision to avoid over-flagging. High recall is especially important to prevent harm.

Fairness Checks

Compare false negative rates and calibration across groups like race, gender, and age. If one group has much higher false negatives, they may be unfairly denied extra support.

Robustness and Privacy

Test temporal and site robustness, and simulate missing data. For privacy, ensure compliant data handling, evaluate membership inference risk if exposed, and monitor for suspicious query patterns.

6. Simple Evaluation Pipeline (Python Example)

You do not need to be a software engineer to understand evaluation pipelines, but seeing a simple example helps.

Below is a small Python-style pseudocode snippet showing how you might structure an evaluation script for a classification model. It includes performance, fairness, and basic logging.

```python

import pandas as pd

from sklearn.metrics import accuracyscore, rocaucscore, recallscore

1. Load data and model (details omitted)

data = pd.readcsv("evaldata.csv")

X = data[feature_cols]

y_true = data["label"]

protected = data["gender"] # example protected attribute

model = loadmodel("models/admissionsmodel.pkl")

ypredproba = model.predict_proba(X)[:, 1]

ypred = (ypred_proba >= 0.5).astype(int)

2. Overall performance metrics

metrics = {}

metrics["accuracy"] = accuracyscore(ytrue, y_pred)

metrics["rocauc"] = rocaucscore(ytrue, ypredproba)

metrics["recall"] = recallscore(ytrue, y_pred)

3. Group fairness metrics (example: recall by gender)

for group in protected.unique():

mask = protected == group

if mask.sum() < 30:

continue # skip tiny groups to avoid unstable stats

grouprecall = recallscore(ytrue[mask], ypred[mask])

metrics[f"recallgender{group}"] = group_recall

4. Save metrics for traceability

import json

from datetime import datetime

result = {

"timestamp": datetime.utcnow().isoformat(),

"model_version": "v1.3.2",

"dataset": "admissionseval2026Q2",

"metrics": metrics,

}

with open("logs/evalresults2026Q2.json", "w") as f:

json.dump(result, f, indent=2)

print("Evaluation complete. Key metrics:")

for k, v in metrics.items():

print(f"{k}: {v:.3f}")

```

Key ideas illustrated:

Centralized metrics dict: easy to extend with robustness or privacy metrics.
Group metrics: quick way to start fairness checks.
Traceability: every run logs timestamp, model version, and dataset name, supporting audit and continuous monitoring.

In real systems, this would be integrated into CI/CD pipelines and monitored dashboards.

7. Quick Check: Metrics and Misuse

Test your understanding of how to choose metrics aligned with risk.

An online proctoring AI flags students as 'suspicious' during remote exams. Which metric is MOST important to examine to reduce the risk of unfairly accusing students, especially from marginalized groups?

Overall accuracy across all students
False positive rate, broken down by demographic group
Model training loss after each epoch
Number of model parameters

Show Answer

Answer: B) False positive rate, broken down by demographic group

False positive rate by group directly reflects how often students are wrongly flagged as suspicious. High or uneven false positives can create serious fairness and reputational harms. Overall accuracy can hide group disparities, and training loss or parameter count do not directly relate to this risk.

8. Flashcards: Key Measure Concepts

Flip through these cards to reinforce key terms from the Measure function.

Measure function (in NIST AI RMF): The function that defines, conducts, and documents evaluations of AI systems to generate evidence about performance, risks, and trustworthiness, supporting informed risk decisions and continuous improvement.
Robustness testing: Evaluations that check how an AI system performs under variations, noise, distribution shifts, or stress conditions, revealing whether it generalizes beyond the training data and behaves reliably in realistic scenarios.
Bias and fairness assessment: Systematic measurement of model performance across groups (e.g., by race, gender, age) using metrics like false positive/negative rates, demographic parity, equal opportunity, and calibration to detect and mitigate unfair disparities.
Privacy impact analysis (e.g., DPIA): A structured assessment of how an AI system collects, uses, and stores personal data, the risks to individuals' privacy, and the safeguards in place. Often required by regulations like the GDPR for high-risk processing.
Red-teaming (for AI systems): A structured process where internal or external teams actively try to make an AI system fail or misbehave (e.g., jailbreak an LLM, extract data, bypass safety filters) to discover vulnerabilities before adversaries or the public do.
Evidence traceability: The ability to trace evaluation results back to specific model versions, datasets, methods, and dates, enabling audits, reproducibility, and accountability for AI risk decisions.
Continuous monitoring: Ongoing collection and review of metrics and incidents from a deployed AI system (e.g., performance, drift, complaints, safety events) to detect emerging risks and trigger updates, retraining, or rollback.

Key Terms

Data drift: Changes over time in the distribution of input data that can cause AI model performance to degrade if not detected and addressed.
Robustness: The degree to which an AI system maintains acceptable performance when faced with noise, distribution shifts, adversarial inputs, or other realistic variations from its training conditions.
Calibration: A property of probabilistic predictions where predicted probabilities match observed frequencies (e.g., among people predicted to have a 0.7 risk, about 70% actually experience the event).
NIST AI RMF: The US National Institute of Standards and Technology's AI Risk Management Framework (version 1.0 released in 2023), a voluntary framework that defines functions (Map, Measure, Manage, Govern) to help organizations manage AI risks.
Red-teaming: A testing practice where specialists intentionally try to cause failures or exploit vulnerabilities in an AI system to discover weaknesses before they can be abused or cause harm.
Traceability: The ability to track how and why an AI system and its evaluations were produced, including links between datasets, model versions, code, metrics, and decisions.
Generalization: An AI model's ability to perform well on new, unseen data drawn from the same or a similar distribution as the training data.
Fairness metric: A quantitative measure used to evaluate whether an AI system's outputs differ systematically across groups in ways that may be unfair, such as differences in false positive rates or selection rates.
Measure function: One of the four NIST AI RMF functions, focused on defining, conducting, and documenting evaluations of AI systems to generate evidence about performance, risks, and trustworthiness.
Differential privacy: A mathematical framework for limiting the privacy risk to individuals in a dataset by adding carefully calibrated noise to computations, controlled by a parameter often called epsilon.