M — Mission decision
Define the fictional user, analyst, source, service, privacy, recovery, risk, or leadership decision the measure supports.
High School AdvancedModule A6Lesson 8 of 10Quality, Coverage, Workload, Source Health, Recovery, and Risk
A6.8 Dashboards and Metrics
Learn how fictional defenders design dashboards and metrics that support real decisions about alert quality, source health, queue aging, triage, escalation, case closure, coverage, workload, privacy, recovery, residual risk, and improvement.
Lesson Progress
Dashboards and Metrics
High School Advanced • A6: SIEM and Alert Triage Concepts • Lesson 8 of 10
80% complete
Readiness Check
Before You Start
0/6 ready
Professional Hook
A Better Number Can Represent a Worse Outcome
A fictional program celebrates three improvements: alert volume is down, average closure time is faster, and Unknown cases are nearly gone. A closer review shows that broad suppression hid a changed destination, cases close when alerts stop, and analysts avoid Unknown because it lowers their score. The numbers improved while detection coverage, case quality, and evidence honesty weakened.
Weak conclusion
“Fewer alerts and faster closure prove the SIEM program is improving.”
Strong conclusion
“Volume and speed improved, but grouping regression, reopen rate, source health, coverage, closure quality, and state corrections show whether the mission outcome actually improved.”
Metrics are tools for decisions. When they become targets without balance, they can reward the wrong behavior.
Exactly Five Learning Objectives
What You Will Be Able to Do
Objective 1
Explain how fictional SIEM dashboards and metrics should support defender decisions rather than reward alert volume, rapid closure, broad suppression, or unsupported certainty.
Objective 2
Distinguish fictional volume, timeliness, source-health, alert-quality, triage, escalation, case-quality, coverage, workload, privacy, recovery, and residual-risk metrics.
Objective 3
Design fictional metric definitions with purpose, population, numerator, denominator, grain, time range, source health, owner, threshold, limitation, review trigger, and decision use.
Objective 4
Evaluate fictional dashboards for misleading averages, hidden backlog, duplicate inflation, blind periods, stale context, denominator problems, privacy risk, and metric gaming.
Objective 5
Create a portfolio-ready fictional Dashboards and Metrics Package containing dashboard layouts, metric dictionaries, evidence models, quality gates, owner matrices, validation cases, leadership summaries, residual risks, and review triggers.
Why This Matters
Dashboards Shape Attention, Incentives, and Resource Decisions
Fictional dashboards influence which sources are repaired, which alerts are tuned, which cases are reviewed, which owners receive resources, which services appear at risk, and which program changes are considered successful. Weak measures can hide Blind periods, duplicate inflation, stale context, queue starvation, incomplete recovery, privacy problems, or residual risk.
Decision-centered
Every fictional panel should support a bounded analyst, owner, quality, privacy, recovery, risk, or leadership decision.
Evidence-aware
Every fictional metric should show population, source health, scope, transformation, limitations, and uncertainty.
Behavior-aware
Balanced fictional measures should discourage broad suppression, premature closure, forced certainty, and unhealthy rankings.
Core Framework
The M-E-A-S-U-R-E Method
E — Exact definition
Document fictional population, numerator, denominator, grain, time range, exclusions, fields, and transformations.
A — Assess source health
Show fictional freshness, completeness, schema, parser, queue, duplicates, conflicts, blind periods, recovery, and confidence.
S — Segment the view
Break fictional results by priority, service, source, rule, state, owner role, complexity, time, and mission context.
U — Use balanced measures
Pair fictional speed with quality, volume with uniqueness, suppression with coverage, closure with reopening, and workload with complexity.
R — Review behavior and risk
Check fictional gaming, privacy, fairness, stale context, denominator drift, scope changes, thresholds, and residual risk.
E — Evolve or retire
Assign fictional owners, review dates, change history, tests, thresholds, actions, debt, replacement coverage, and retirement.
Decision-ready metric statement
This fictional metric measures median and 90th-percentile alert-to-first-review time for High-priority alerts, excludes synthetic validation alerts, shows source-health and owner-delay context, and triggers review when either threshold exceeds the documented response expectation.
Advanced Vocabulary
Terms for Dashboards and Metrics
Dashboard
A fictional visual summary that combines selected measures, states, trends, exceptions, and context to support a bounded defensive decision.
Metric
A fictional defined measure with purpose, population, numerator, denominator, time range, source, owner, limitation, and decision use.
Measure
A fictional observed value such as a count, duration, percentage, rate, ratio, trend, state, distribution, or age.
Indicator
A fictional measure interpreted as a signal about quality, risk, workload, source health, coverage, or performance.
KPI
A fictional key performance indicator selected because it supports an important mission or program decision.
KRI
A fictional key risk indicator selected because it signals meaningful exposure, uncertainty, deterioration, or residual risk.
Numerator
The fictional counted or measured events that appear above the division line in a rate or percentage.
Denominator
The fictional full relevant population used to interpret the numerator.
Population
The fictional identities, devices, services, alerts, cases, sources, owners, time periods, or records included in a metric.
Metric grain
The fictional level at which a measure is calculated, such as alert, rule, case, service, source, identity category, owner, or day.
Aggregation
A fictional combination of multiple values into a count, average, median, percentile, rate, distribution, or trend.
Average
A fictional arithmetic mean that may hide extreme values, uneven populations, or skewed distributions.
Median
A fictional middle value that may better represent typical performance when extremes exist.
Percentile
A fictional value below which a defined percentage of observations fall.
Rate
A fictional measure comparing counted events to a relevant population or time period.
Trend
A fictional pattern of change over time that may reflect real improvement, seasonality, source changes, scope changes, or measurement drift.
Baseline
A fictional documented reference used to compare current values with expected historical, peer, service, or operating-state patterns.
Threshold
A fictional documented boundary that triggers review, escalation, communication, or another decision.
Leading indicator
A fictional measure that may signal future quality or risk before the final outcome occurs.
Lagging indicator
A fictional measure that describes a result after activity or decisions have already occurred.
Coverage metric
A fictional measure of which identities, services, sources, behaviors, periods, or states are included, excluded, blind, degraded, or untested.
Quality metric
A fictional measure of usefulness, evidence completeness, source health, false positives, false negatives, state accuracy, closure quality, or documentation quality.
Workload metric
A fictional measure of analyst effort, queue age, owner delay, duplicate work, evidence requests, reopenings, or handoffs.
Metric gaming
A fictional behavior in which people optimize the measured number while weakening the real mission outcome.
Instructional Section 1
Compare Twelve Metric Families
Alert volume
Measure fictional raw alerts, grouped alerts, unique conditions, expected alerts, duplicates, replay, and suppression.
Strong questions
How many distinct work items exist? Which rules, sources, services, and identity categories contribute volume? Which records are duplicate, expected, or replayed?
Weak use
Treating lower alert count as proof of better detection quality.
Decision use
Identify workload, duplication, source recovery, grouping needs, and sudden volume changes.
Timeliness
Measure fictional event-to-collection, collection-to-processing, processing-to-alert, alert-to-review, owner-response, action, validation, and closure time.
Strong questions
Where does delay occur, and which delays reduce evidence quality, response opportunity, user protection, or recovery options?
Weak use
Rewarding fast closure without measuring decision quality or reopen rate.
Decision use
Improve collection, queue, triage, ownership, escalation, recovery, and closure timing.
Source health
Measure fictional freshness, completeness, schema, parser, queue, clock, coverage, conflicts, blind periods, recovery, and replay.
Strong questions
Which conclusions and detections are affected by Conditional, Degraded, Blind, Conflicting, or Recovering evidence?
Weak use
Reporting source connectivity without measuring usable evidence quality.
Decision use
Prioritize restoration, alternate evidence, historical reassessment, and source-quality improvements.
Alert quality
Measure fictional usefulness, expected alerts, false positives, known false negatives, missing context, source-health handling, and alert-contract completeness.
Strong questions
Does each alert help answer its defender question, and which evidence or context is usually missing?
Weak use
Using analyst closure labels as the only quality truth.
Decision use
Improve rule logic, source requirements, context, alert presentation, testing, and documentation.
Triage quality
Measure fictional question quality, evidence-layer accuracy, source-health visibility, request precision, state accuracy, and owner response.
Strong questions
Do analysts ask bounded questions, preserve uncertainty, and request only decision-relevant evidence?
Weak use
Measuring only time to first review.
Decision use
Improve runbooks, training, evidence requests, state definitions, and owner workflows.
Escalation quality
Measure fictional trigger precision, delayed escalation, premature escalation, handoff completeness, acceptance, aging, and de-escalation.
Strong questions
Did the right question reach the right owner at the right time with complete context?
Weak use
Rewarding more escalations as proof of stronger security.
Decision use
Improve thresholds, recipients, deadlines, handoffs, leadership paths, and de-escalation.
Case quality
Measure fictional note neutrality, evidence traceability, chronology, decision logs, action-validation separation, owner deadlines, closure, and reopening.
Strong questions
Can another reviewer reconstruct the case and defend each decision?
Weak use
Measuring case count and closure speed alone.
Decision use
Improve case templates, note quality, validation, residual risk, and lifecycle.
Coverage
Measure fictional identity, device, service, destination, source, behavior, time, environment, state, and test coverage.
Strong questions
Which mission-relevant areas are monitored, unmonitored, degraded, blind, excluded, or untested?
Weak use
Reporting number of rules without showing what they cover.
Decision use
Prioritize new coverage, source investment, testing, residual risk, and leadership decisions.
Workload and capacity
Measure fictional queue age, analyst effort, evidence hunting, duplicate work, owner delay, reopenings, handoffs, and specialist demand.
Strong questions
Which work consumes time without improving decisions, and where is capacity insufficient?
Weak use
Comparing analysts by closed-case count without complexity or quality context.
Decision use
Improve staffing, automation, ownership, training, evidence access, and prioritization.
Privacy and governance
Measure fictional field purpose, access, sharing, retention, unnecessary evidence, corrections, ownership, and review status.
Strong questions
Are dashboards and case metrics using only necessary, current, appropriately shared information?
Weak use
Displaying identity-level performance without a bounded need.
Decision use
Reduce exposure, clarify access, update retention, review purpose, and improve governance.
Recovery and resilience
Measure fictional restoration, backlog, replay, duplicates, session state, source health, service validation, historical reassessment, and rollback.
Strong questions
Has trustworthy mission and evidence recovery occurred, or only connectivity restoration?
Weak use
Marking recovery complete when a source reconnects.
Decision use
Improve recovery criteria, validation, rollback, ownership, and historical review.
Residual risk and debt
Measure fictional unresolved coverage, stale context, missing tests, owner gaps, source risk, case debt, escalation debt, and documentation debt.
Strong questions
Which known limitations remain, who owns them, and when will they be reviewed?
Weak use
Counting debt without mission impact, age, or owner context.
Decision use
Support prioritization, resource allocation, risk acceptance, and roadmap planning.
Instructional Section 2
Write a Twelve-Field Metric Definition
| Field | Requirement | Fictional example |
|---|---|---|
| Metric name | Use a fictional clear name that describes the measure without implying an unsupported outcome. | Median Alert-to-First-Review Time |
| Decision purpose | State which fictional decision the metric supports. | Identify queue or staffing delay that may reduce response opportunity. |
| Population | Define fictional alerts, cases, sources, services, owners, identities, or periods included and excluded. | All High-priority fictional alerts opened during the month, excluding synthetic validation alerts. |
| Numerator or measured value | Define exactly what is counted or measured. | Minutes between alert creation and first documented analyst review. |
| Denominator | Define the relevant fictional population for rates or percentages. | All eligible High-priority alerts with valid timestamps. |
| Grain | Define whether the metric is calculated per alert, rule, case, service, source, owner, or time period. | Per alert, summarized monthly by median and 90th percentile. |
| Time range and update cadence | Define the fictional measurement period and refresh schedule. | Rolling thirty days, refreshed daily. |
| Data sources and provenance | List fictional source categories, fields, transformations, health, and owners. | Alert record, case intake note, state-transition log, and source-health record. |
| Threshold or interpretation | Explain which fictional values trigger review and why. | Review when median exceeds fifteen minutes or the 90th percentile exceeds forty-five minutes. |
| Limitations | Document fictional missing timestamps, reopened cases, complexity, source delay, duplicates, and scope changes. | Fast review does not prove good triage or correct priority. |
| Owner | Assign the fictional role responsible for definition, data quality, review, and action. | Queue-quality owner with analyst-lead review. |
| Review trigger | Define which fictional source, scope, workflow, policy, or mission changes require reassessment. | New case tool, changed priority model, source outage, or major staffing change. |
Instructional Section 3
Design Ten Dashboard Panels
Mission impact overview
Decision question
Which fictional users, services, identities, suppliers, privacy outcomes, evidence capabilities, or recovery functions are currently affected?
Fictional measures
Active-impact cases, critical-service cases, privileged-authority cases, broad source Blind periods, recovery exceptions, and residual-risk count.
Risk if weak
Leadership may see workload without understanding mission consequence.
Primary owner
Program owner and service-risk owner.
Alert volume and uniqueness
Decision question
How much fictional work is unique, duplicate, expected, grouped, suppressed, replayed, or newly changed?
Fictional measures
Raw alerts, grouped alerts, unique conditions, expected alerts, duplicates, replay records, and grouping break events.
Risk if weak
Raw volume may exaggerate workload or hide important new scope.
Primary owner
Detection-quality owner.
Source-health coverage
Decision question
Which fictional sources, fields, populations, periods, and detections are Healthy, Conditional, Degraded, Blind, Conflicting, or Recovering?
Fictional measures
Health-state duration, affected rules, affected services, blind minutes, recovery backlog, schema defects, and unresolved conflicts.
Risk if weak
Quiet alerts may look healthy during missing evidence.
Primary owner
Source-quality owner.
Queue and priority
Decision question
Which fictional alerts and cases are aging, blocked, misprioritized, or missing owners?
Fictional measures
Queue age by priority, first-review delay, owner-response delay, aging overrides, starvation, reassignment, and priority debt.
Risk if weak
Averages may hide a small number of severely delayed cases.
Primary owner
Queue-quality owner.
Triage quality
Decision question
Are fictional analysts using neutral questions, complete evidence, source-health context, and correct states?
Fictional measures
Question completeness, evidence-layer errors, source-health visibility, request precision, state corrections, and Conditional/Unknown use.
Risk if weak
Low Unknown count may indicate forced certainty rather than strong evidence.
Primary owner
Triage-quality owner.
Escalation and ownership
Decision question
Do fictional escalations reach the correct owner with complete handoffs and timely acceptance?
Fictional measures
Escalations by type and level, delayed escalation, premature escalation, handoff completeness, owner acceptance, aging, and de-escalation.
Risk if weak
More escalations may reflect weak triage or unclear ownership.
Primary owner
Escalation-governance owner.
Case quality and closure
Decision question
Can fictional cases be reconstructed, validated, closed, and reopened correctly?
Fictional measures
Evidence-ledger completeness, decision-log completeness, action-validation separation, closure checklist completion, reopen rate, and case debt.
Risk if weak
Fast closure may hide weak evidence, residual risk, or premature resolution.
Primary owner
Case-quality owner.
Coverage and residual risk
Decision question
Which fictional mission-relevant areas remain unmonitored, degraded, untested, undocumented, or accepted as residual risk?
Fictional measures
Coverage by service, identity category, behavior, source, period, health state, test status, debt age, owner, and risk acceptance.
Risk if weak
High rule count may hide important coverage gaps.
Primary owner
Coverage and risk owner.
Workload and capacity
Decision question
Where does fictional analyst or owner effort concentrate, and which work creates little decision value?
Fictional measures
Case complexity, evidence requests, duplicate work, reopenings, owner delays, specialist demand, queue age, and workload distribution.
Risk if weak
Comparing people by case count may punish careful work or complex assignments.
Primary owner
Operations and workforce owner.
Privacy and governance
Decision question
Are fictional dashboard fields, case notes, metrics, sharing, and retention purpose-limited and current?
Fictional measures
Unnecessary fields, broad access, overdue retention, privacy exceptions, owner gaps, review dates, and public-safe validation.
Risk if weak
Dashboards can expose identity-level data without a bounded need.
Primary owner
Privacy and governance owner.
Instructional Section 4
Recognize Ten Misleading Metric Patterns
Average hides the tail
Fictional example
Fictional mean first-review time is eight minutes, but several High-priority alerts waited more than an hour.
Why misleading
A few severe delays may be hidden by many fast reviews.
Correction
Use median, percentile, distribution, priority segment, and longest-delay review.
Raw count hides duplicates
Fictional example
Fictional alert volume rises by 200%, but most records are recovery replay duplicates.
Why misleading
The measure confuses delivery repetition with unique analyst work.
Correction
Show raw alerts, unique conditions, grouped work items, duplicates, replay, and break conditions.
Lower volume appears successful
Fictional example
Fictional alerts fall after broad suppression, but one meaningful destination change is no longer visible.
Why misleading
Noise reduction may have weakened coverage.
Correction
Pair volume with misses, coverage, grouping regression, source health, and quality review.
Closure speed rewards weak cases
Fictional example
Fictional average closure time improves because cases close when alerts stop.
Why misleading
Source recovery, validation, residual risk, and reopen criteria may remain incomplete.
Correction
Measure closure quality, reopen rate, validation completeness, residual risk, and source-health reconciliation.
No alerts appears healthy
Fictional example
Fictional alert volume drops to zero while a required source is Blind.
Why misleading
Missing evidence may appear as normal quiet activity.
Correction
Show source-health state, blind duration, affected detections, and Unknown coverage.
Rule count appears to equal coverage
Fictional example
Fictional program reports one hundred rules but does not show which services, identities, behaviors, or blind periods are covered.
Why misleading
Many rules can overlap while important mission areas remain uncovered.
Correction
Measure coverage by mission question, population, source, state, and test status.
Percentages use unstable denominators
Fictional example
Fictional false-positive rate falls because unreviewed and Unknown cases are removed from the denominator.
Why misleading
The result changes because the population changed, not necessarily because quality improved.
Correction
Define reviewed, unreviewed, Unknown, Source-Degraded, Expected, reopened, and excluded populations clearly.
Individual rankings distort behavior
Fictional example
Fictional analysts are ranked by closed-case count.
Why misleading
Complexity, quality, collaboration, source gaps, and owner delay are ignored.
Correction
Measure team outcomes, case complexity, quality, evidence, workload, and improvement rather than personal scoreboards.
Trend ignores scope change
Fictional example
Fictional alert rate rises after adding a new service and source.
Why misleading
The program may have expanded coverage rather than become noisier.
Correction
Annotate source, scope, population, rule, policy, and workflow changes.
Metric becomes a target
Fictional example
Fictional team closes cases faster to meet a target, increasing reopenings.
Why misleading
The measured number improves while the mission outcome weakens.
Correction
Use balanced measures, quality gates, review samples, and anti-gaming checks.
Instructional Section 5
Measure Six Source-Health Dimensions
Freshness compliance
Percentage of fictional source intervals meeting documented freshness requirements.
Denominator or population
All expected source intervals for the included population and time range.
Caution
Fresh data may still be incomplete, semantically wrong, or poorly mapped.
Decision
Investigate delay patterns and affected detections.
Completeness rate
Percentage of fictional expected records or required fields received for the documented population.
Denominator or population
Expected records or fields based on service, source, and operating-state assumptions.
Caution
The expected population may itself be uncertain.
Decision
Review source gaps, denominator assumptions, and alternate evidence.
Blind minutes
Total fictional minutes in which required evidence was unavailable for a defined population.
Denominator or population
Reported directly and as a percentage of expected monitoring time.
Caution
A short Blind period can still be high impact if time-sensitive.
Decision
Prioritize restoration and historical reassessment.
Schema and parser defect rate
Percentage of fictional records or intervals affected by unsupported schema, parser, or field-mapping behavior.
Denominator or population
All processed records or intervals for the source version.
Caution
Low defect rate may hide defects in rare critical fields.
Decision
Review field-level impact, regression, and rollout controls.
Conflict rate
Percentage of fictional related source relationships that disagree beyond documented tolerance.
Denominator or population
All relationships eligible for comparison.
Caution
Some source differences are expected and meaningful rather than errors.
Decision
Review source authority, timing, semantics, and owner reconciliation.
Recovery reconciliation
Percentage of fictional backlog, replay, duplicate, schema, timing, and historical obligations reconciled after recovery.
Denominator or population
All documented recovery obligations for the affected source period.
Caution
Connectivity restoration can make this metric look better before validation is complete.
Decision
Keep source Recovering until obligations pass.
Instructional Section 6
Define Eight Alert and Case Quality Metrics
Alert usefulness rate
Percentage of fictional reviewed alerts that helped answer the documented defender question.
Numerator
Reviewed alerts rated useful for a bounded decision.
Denominator
All reviewed alerts in scope, including Expected, Conditional, Unknown, and Source-Degraded.
Limitation
Analyst ratings may vary and require calibration.
Alert-contract completeness
Percentage of fictional alerts containing observation, evidence, source health, context, confidence, severity, priority, alternatives, owners, and non-proof statements.
Numerator
Alerts passing all required contract fields.
Denominator
All alerts in the reviewed rule population.
Limitation
Complete fields may still contain stale or incorrect values.
Expected-alert accuracy
Percentage of fictional Expected alerts matching current approval, scope, owner, purpose, time, service, destination, and source health.
Numerator
Validated Expected alerts with current matching context.
Denominator
All alerts labeled Expected during the period.
Limitation
Validation may depend on delayed or stale owner evidence.
False-positive rate
Percentage of fictional reviewed alerts whose risky interpretation was unsupported after evidence review.
Numerator
Reviewed alerts labeled false positive under current definitions.
Denominator
All reviewed alerts eligible for that outcome.
Limitation
Unknown, Source-Degraded, and unreviewed alerts should not be forced into the denominator.
Known false-negative count
Count of fictional meaningful conditions discovered through testing, owner reports, recovery, or later evidence that were not alerted.
Numerator
Known missed conditions.
Denominator
Reported as a count plus affected coverage because the full unknown population is unavailable.
Limitation
Unknown misses cannot be measured completely.
Triage-state correction rate
Percentage of fictional cases whose state changed after quality review because the earlier state did not match evidence.
Numerator
Cases with documented state correction.
Denominator
All cases sampled for quality review.
Limitation
A higher rate may reflect better review or weaker initial triage.
Closure-quality pass rate
Percentage of fictional closed cases meeting evidence, source health, authorization, scope, impact, validation, owner, residual-risk, and reopen requirements.
Numerator
Closed cases passing all closure gates.
Denominator
All closed cases in the reviewed sample.
Limitation
Passing sampled cases does not prove all cases are high quality.
Reopen rate
Percentage of fictional closed cases reopened because of new evidence, failed validation, changed scope, repeated behavior, or source recovery.
Numerator
Cases reopened within the defined review period.
Denominator
All eligible closed cases.
Limitation
A low reopen rate may mean strong closure or weak detection of reopen triggers.
Instructional Section 7
Measure Six Workload and Capacity Dimensions
Queue age by priority
Show fictional unresolved alert and case age separately for High, Medium, and Low priority.
Strong view
Median, 90th percentile, oldest cases, owner status, source health, and active-impact segment.
Weak view
One average across all priorities.
Decision use
Identify starvation, blocked work, owner delay, and staffing need.
Evidence-request burden
Measure fictional requests per case, response delay, repeated requests, missing fields, and owner effort.
Strong view
Requests by purpose, owner, case complexity, source health, and decision value.
Weak view
Total request count only.
Decision use
Improve evidence access, runbooks, source context, ownership, and automation.
Duplicate work
Measure fictional repeated alerts, repeated evidence review, duplicate cases, replay, and unaccepted handoffs.
Strong view
Duplicate cause, affected rule or source, hours, owners, and corrective action.
Weak view
Analyst case count without uniqueness.
Decision use
Improve grouping, case linking, recovery, routing, and ownership.
Owner-response delay
Measure fictional time between bounded request, receiving-owner acceptance, response, and validation.
Strong view
By owner role, urgency, case type, response quality, alternate path, and missed deadline.
Weak view
Average response time without complexity or deadline context.
Decision use
Improve ownership, backups, escalation, service expectations, and communication.
Rework rate
Measure fictional repeated triage, corrected notes, reopened cases, failed validation, and repeated owner requests.
Strong view
Rework reason, mission impact, source health, owner, and preventable cause.
Weak view
Treating all rework as analyst failure.
Decision use
Improve alerts, sources, runbooks, training, case templates, and closure.
Specialist demand
Measure fictional technical, identity, service, supplier, privacy, recovery, and leadership escalations.
Strong view
Demand by bounded question, level, urgency, handoff quality, response, and outcome.
Weak view
Total escalation count.
Decision use
Improve specialist capacity, training, ownership, evidence access, and thresholds.
Instructional Section 8
Validate Twelve Dashboard Scenarios
| Case | Type | Fictional input | Expected result | Quality protected |
|---|---|---|---|---|
| DASH-T01 | Duplicate inflation | Recovery replay triples raw alert volume without increasing unique conditions. | Separate raw alerts, unique conditions, grouped work items, duplicates, replay, and affected rules. | Accurate workload and trend interpretation. |
| DASH-T02 | Blind source | Alert volume falls to zero while a required source is Blind. | Highlight Blind state, affected detections, blind minutes, Unknown coverage, and reassessment. | False-normal interpretation. |
| DASH-T03 | Average hides delay | Mean first-review time is eight minutes, but several High-priority alerts waited over one hour. | Show median, percentiles, distribution, oldest cases, and priority segments. | Tail-risk visibility. |
| DASH-T04 | Denominator change | False-positive rate drops after unreviewed and Unknown cases are removed. | Fail validation until reviewed, unreviewed, Unknown, Source-Degraded, Expected, and excluded populations are documented. | Rate integrity. |
| DASH-T05 | Broad suppression | Alert volume decreases, but grouping regression reveals a hidden new destination. | Keep noise metric Conditional and show regression failure, coverage risk, and rollback status. | Balanced quality. |
| DASH-T06 | Fast closure | Closure time improves while reopen rate and incomplete source reconciliation rise. | Flag the tradeoff and require closure-quality review. | Anti-gaming behavior. |
| DASH-T07 | Coverage expansion | Alert rate increases after a new service and source enter scope. | Add scope-change annotation and normalized population view. | Fair trend interpretation. |
| DASH-T08 | Source recovery | Source reconnects, but backlog, replay, duplicates, schema, and historical gaps remain. | Keep source Recovering and show reconciliation obligations. | Recovery quality. |
| DASH-T09 | Individual ranking | Analysts are ranked by cases closed. | Fail privacy and governance review; shift to team-level quality, complexity, workload, and improvement. | Fairness and healthy behavior. |
| DASH-T10 | Stale criticality | Priority dashboard uses a service-criticality catalog not reviewed for nine months. | Mark metric Conditional and trigger owner review. | Context freshness. |
| DASH-T11 | Missing owner | Leadership dashboard includes residual-risk count with no definition or owner. | Fail governance validation until definition, population, owner, limitation, and decision use are documented. | Accountability. |
| DASH-T12 | Public portfolio | Student uses sanitized real dashboard screenshots. | Fail portfolio validation; every organization, source, metric, alert, owner, date, and value must be invented. | Confidentiality and safety. |
Instructional Section 9
Govern Ten Metric Lifecycle Domains
Purpose
Governance question
Which fictional decision should the dashboard or metric support?
Fictional evidence
Mission question, stakeholder need, meeting decision, owner workflow, and expected action.
Failure if ignored
The dashboard becomes decoration or encourages unbounded monitoring.
Definition
Governance question
Are fictional population, numerator, denominator, grain, time range, exclusions, and transformations explicit?
Fictional evidence
Metric dictionary, field definitions, examples, tests, and owner review.
Failure if ignored
Different viewers interpret the same number differently.
Source health
Governance question
Can fictional source delay, Blind periods, schema changes, duplicates, conflicts, and recovery affect the measure?
Fictional evidence
Source-health states, quality tests, affected periods, and confidence rules.
Failure if ignored
Missing evidence becomes false improvement.
Ownership
Governance question
Who owns fictional metric definition, source quality, dashboard review, threshold response, and retirement?
Fictional evidence
Owner matrix, review dates, escalation path, change log, and debt register.
Failure if ignored
Metrics remain stale or trigger no action.
Interpretation
Governance question
Which fictional conclusions are supported, and which are not?
Fictional evidence
Decision notes, limitations, examples, counterexamples, and review guidance.
Failure if ignored
A single number becomes a broad performance judgment.
Thresholds
Governance question
Which fictional values trigger review, escalation, communication, or action, and why?
Fictional evidence
Baseline, percentiles, service expectations, risk tolerance, owner input, and validation.
Failure if ignored
Thresholds become arbitrary or copied across different populations.
Privacy and fairness
Governance question
Does the fictional dashboard expose unnecessary identity, owner, team, or case-level information?
Fictional evidence
Purpose map, access roles, aggregation, sharing, retention, and fairness review.
Failure if ignored
Metrics create surveillance or unhealthy personal rankings.
Anti-gaming
Governance question
Could fictional users improve the metric while weakening the mission outcome?
Fictional evidence
Balanced measures, counter-metrics, quality samples, reopenings, misses, and behavior review.
Failure if ignored
Teams optimize the number rather than the real result.
Change management
Governance question
How do fictional source, scope, rule, workflow, staffing, policy, or mission changes affect trends?
Fictional evidence
Annotations, version history, change dates, normalized views, and review triggers.
Failure if ignored
Trends appear to improve or worsen because the measurement changed.
Retirement
Governance question
When should the fictional metric or panel be replaced, merged, archived, or removed?
Fictional evidence
Decision use, duplication, owner review, stale context, low action value, and replacement coverage.
Failure if ignored
Dashboards accumulate stale measures and conflicting definitions.
Fictional Dashboard Architecture
Northbridge Decision-to-Metric Model
This conceptual architecture is completely invented and intentionally non-operational. It teaches metric design without real dashboards, products, alerts, source names, identities, services, owners, screenshots, values, suppliers, or internal priorities.
Mission inputs
Users, services, identity, privacy, recovery, risk
Evidence inputs
Alerts, cases, sources, timing, health, coverage
Workflow inputs
Queue, triage, escalation, decisions, actions
Governance inputs
Owners, thresholds, privacy, change, retirement
Fictional Metric Core
Purpose
Decision, audience, mission, expected action
Definition
Population, numerator, denominator, grain, time
Evidence
Sources, fields, transformations, health, limits
Balance
Speed, quality, volume, uniqueness, coverage, risk
Segmentation
Priority, service, source, state, owner role
Validation
Blind, duplicate, average, denominator, scope tests
Governance
Owner, threshold, privacy, anti-gaming, change
Lifecycle
Review, debt, replacement, retirement, residual risk
Analyst output
Queue, quality, evidence, workload decisions
Owner output
Source, service, detection, privacy, recovery actions
Leadership output
Impact, trends, resources, debt, milestones
Portfolio boundary
Fully fictional, privacy-safe, non-operational
Fake Dashboard
Fake Northbridge SIEM Quality Dashboard
Fictional alert uniqueness, source health, queue aging, triage quality, closure quality, coverage, recovery, privacy, and residual risk for training only.
High-priority alerts beyond review deadline
4
Median review time is healthy, but the 90th percentile and oldest-case view reveal delayed tail risk.
Detections affected by Blind or Recovering sources
7
Three critical-service detections require historical reassessment after source recovery.
Open fictional dashboard and metric debt
12
Definitions, denominators, source health, owners, privacy, thresholds, change annotations, anti-gaming checks, validation, residual risk, review dates, and retirement remain open.
Fake SOC Alert
Dashboard Interpretation Requires Immediate Review
Source: Fake Northbridge Metrics Governance Console • Time: 5:24 PM
The fictional dashboard reports lower alert volume and faster closure as improvements. However, a required source was Blind, broad grouping failed regression, reopen rate doubled, four High-priority alerts exceeded deadline, and the false-positive denominator excludes Unknown and Source-Degraded cases.
Defensive recommendation: Mark the fictional dashboard Conditional. Correct source-health context, uniqueness, grouping regression, closure-quality measures, tail latency, denominator definitions, owner assignments, privacy review, and leadership interpretation before use.
Fake Log Panel
Fake Dashboard Review Timeline
training-log-viewer.log
09:00 DASHBOARD id='METRICS-ST-06' 09:02 METRIC raw-alerts='down-28-percent' 09:03 METRIC unique-conditions='down-4-percent' 09:04 SOURCE network='blind' 09:05 COVERAGE affected-rules='7' 09:06 METRIC closure-time='improved' 09:07 METRIC reopen-rate='doubled' 09:08 METRIC review-median='7-minutes' 09:09 METRIC review-p90='52-minutes' 09:10 QUEUE high-overdue='4' 09:11 GROUPING regression='failed' 09:12 DENOMINATOR unknown='excluded' 09:13 DENOMINATOR source-degraded='excluded' 09:14 PRIVACY analyst-ranking='present' 09:15 OWNER residual-risk='missing' 09:16 STATUS dashboard='conditional' 17:24 ALERT issue='interpretation-review'
Training note: this is fake data for defensive analysis practice only.
Fictional Evidence Matrix
What Dashboard Evidence Supports—and What It Does Not Prove
METRIC-E01
Fictional alert-volume dashboard
Observation
Raw alert volume increased by 180%, but unique conditions increased by only 8% and replay markers explain most of the difference.
Supports
Duplicate or replay delivery is a major volume driver.
Does not prove
The dashboard does not prove all repeated alerts are unnecessary.
Dashboard use
Review uniqueness, grouping, recovery, and workload separately.
METRIC-E02
Fictional source-health panel
Observation
Network source is Blind for forty minutes across three service zones, while alert volume for related rules is near zero.
Supports
The quiet period is not trustworthy as normal activity.
Does not prove
The panel does not prove harmful activity occurred.
Dashboard use
Show Unknown coverage, affected rules, restoration, and reassessment.
METRIC-E03
Fictional queue dashboard
Observation
Median first-review time is seven minutes, but the 90th percentile is fifty-two minutes and four High-priority alerts exceed deadline.
Supports
The typical case is fast, but a meaningful delayed tail exists.
Does not prove
The panel does not identify every cause of delay.
Dashboard use
Review priority, ownership, staffing, source health, and blocked cases.
METRIC-E04
Fictional closure dashboard
Observation
Average closure time improved by 30%, while reopen rate doubled and closure-quality pass rate fell.
Supports
Faster closure may be weakening case quality.
Does not prove
The panel does not prove every fast closure is poor.
Dashboard use
Review closure definitions, validation, residual risk, and incentives.
METRIC-E05
Fictional coverage panel
Observation
Rule count increased, but two critical services remain Blind during recovery periods and one privileged identity category is untested.
Supports
Rule count does not equal mission coverage.
Does not prove
The panel does not show every unrecognized coverage gap.
Dashboard use
Prioritize mission-based coverage and testing.
METRIC-E06
Fictional triage-quality panel
Observation
Unknown and Conditional states fell sharply after a reporting target was introduced, while state-correction rate increased.
Supports
Analysts may be forcing certainty to satisfy the target.
Does not prove
The panel does not prove intentional gaming.
Dashboard use
Review definitions, incentives, coaching, and quality samples.
METRIC-E07
Fictional privacy panel
Observation
An analyst leaderboard displays identity-level case counts and closure times without a bounded operational need.
Supports
The dashboard exceeds purpose and fairness boundaries.
Does not prove
The finding does not prevent all workload measurement.
Dashboard use
Move to team-level, role-level, complexity-aware quality and capacity views.
METRIC-E08
Fictional metric dictionary
Observation
Residual-risk count has no population, owner, denominator, aging rule, or decision use.
Supports
The metric is not decision-ready or governable.
Does not prove
The count may still identify real unresolved work.
Dashboard use
Complete definition and ownership before leadership use.
Analyze the Evidence
Which Dashboard Decision Is Best Supported?
Raw alert volume fell by 28%, but unique conditions fell by only 4%.
A required network source was Blind and affected seven detections.
Broad grouping failed a regression case involving a new destination.
Average closure time improved while reopen rate doubled.
Median first-review time is seven minutes, but four High-priority alerts exceeded deadline.
False-positive rate excludes Unknown and Source-Degraded cases.
An analyst leaderboard displays personal case counts and closure times.
Residual-risk count has no definition or owner.
Which fictional conclusion best represents the Northbridge metrics review?
Common Mistakes
Avoid Ten Dashboard and Metric Errors
Alert volume is treated as security performance
Fictional observation
A fictional dashboard celebrates lower alert count without reviewing source health, coverage, suppression, or misses.
Decision impact
Coverage loss may appear as improvement.
Professional correction
Pair volume with uniqueness, expected alerts, source health, coverage, regressions, and known misses.
Fast closure is rewarded alone
Fictional observation
A fictional team target reduces average closure time while reopenings increase.
Decision impact
Cases may close before evidence, validation, residual risk, and reopen criteria are complete.
Professional correction
Balance timeliness with closure quality, reopen rate, source reconciliation, and residual risk.
Average hides important cases
Fictional observation
A fictional average first-review time looks healthy despite several severely delayed High-priority alerts.
Decision impact
Tail risk and starvation remain hidden.
Professional correction
Use median, percentiles, distribution, priority segments, and oldest-case review.
The denominator is unclear
Fictional observation
A fictional false-positive rate excludes Unknown, Source-Degraded, and unreviewed cases without documentation.
Decision impact
The percentage may be misleading or impossible to compare.
Professional correction
Define the complete population, exclusions, eligibility, and missing outcomes.
No alerts is treated as no risk
Fictional observation
A fictional panel shows zero alerts during a Blind source period.
Decision impact
Missing evidence becomes false normality.
Professional correction
Show source health, blind duration, affected detections, Unknown coverage, and reassessment.
Rule count is treated as coverage
Fictional observation
A fictional program reports many rules without mission, service, identity, behavior, source, or test coverage.
Decision impact
Overlapping rules may hide critical gaps.
Professional correction
Measure coverage by defender question, population, source, state, and validation.
Personal leaderboards are used
Fictional observation
A fictional dashboard ranks analysts by cases closed and average closure time.
Decision impact
People may avoid complex cases, close early, or compete instead of sharing evidence.
Professional correction
Use team-level, complexity-aware, quality-focused, privacy-reviewed measures.
Trends ignore scope changes
Fictional observation
A fictional alert-rate increase is called deterioration after new services and sources enter scope.
Decision impact
Coverage expansion may be mislabeled as noise growth.
Professional correction
Annotate changes and use normalized populations.
Metrics have no owner or action
Fictional observation
A fictional dashboard displays residual-risk count without a definition, owner, threshold, or next step.
Decision impact
The number creates concern but no accountable decision.
Professional correction
Assign purpose, definition, owner, threshold, limitation, review trigger, and decision use.
Real dashboards enter the portfolio
Fictional observation
A fictional project uses sanitized real screenshots, alert counts, source names, owners, or service metrics.
Decision impact
Sensitive systems, priorities, people, suppliers, and defensive capabilities may still be exposed.
Professional correction
Invent every organization, source, metric, alert, owner, date, value, decision, and outcome.
Safe Fictional Practice Lab
Build the Northbridge Dashboards and Metrics Package
Use only the supplied fictional information on this page. Do not access, copy, sanitize, upload, inspect, measure, compare, publish, review, or modify any real dashboard, metric, SIEM, alert, case, source, account, service, supplier, organization, team, or person.
1
Define dashboard audiences
Choose fictional analyst, source-owner, service-owner, detection-owner, privacy, recovery, risk, and leadership decisions.
Required output
Audience-to-decision map.
Quality check
Every panel has a bounded user and decision purpose.
2
Select metric families
Choose fictional alert volume, timeliness, source health, alert quality, triage, escalation, case quality, coverage, workload, privacy, recovery, and residual risk.
Required output
Metric-family inventory.
Quality check
The set balances speed, quality, coverage, risk, workload, and governance.
3
Write metric definitions
Document fictional name, purpose, population, numerator, denominator, grain, time range, sources, threshold, limitation, owner, and review trigger.
Required output
Metric dictionary.
Quality check
Another reviewer can reproduce and interpret each measure.
4
Design dashboard panels
Create fictional mission, volume, source-health, queue, triage, escalation, case, coverage, workload, privacy, recovery, and residual-risk panels.
Required output
Dashboard wireframe.
Quality check
Each panel shows context, state, trend, exceptions, and decision use.
5
Add source-health context
Show fictional Healthy, Conditional, Degraded, Blind, Conflicting, and Recovering states with affected metrics and periods.
Required output
Metric-confidence layer.
Quality check
Missing evidence cannot appear as improvement.
6
Add balanced views
Pair fictional volume with uniqueness, speed with quality, closure with reopening, suppression with coverage, and workload with complexity.
Required output
Balanced-metric matrix.
Quality check
No single metric can reward harmful behavior by itself.
7
Test misleading patterns
Evaluate fictional duplicates, blind periods, averages, denominator changes, scope expansion, suppression, fast closure, personal rankings, and stale context.
Required output
Dashboard validation matrix.
Quality check
Expected interpretations are documented before review.
8
Define governance
Assign fictional purpose, definition, source, threshold, privacy, anti-gaming, change, review, and retirement owners.
Required output
Metric-governance matrix.
Quality check
Every measure has an accountable lifecycle.
9
Create leadership communication
Summarize fictional mission impact, trend, uncertainty, source health, workload, debt, decisions, owners, milestones, and residual risk.
Required output
Leadership dashboard brief.
Quality check
The summary explains what decisions are needed, not only what numbers changed.
10
Prepare the portfolio package
Combine the fictional audiences, metric families, dictionary, panels, balanced views, tests, governance, leadership brief, residual risks, and reflection.
Required output
Public-safe Dashboards and Metrics Package.
Quality check
Every organization, source, metric, alert, owner, date, value, decision, and outcome is invented.
Scenario Decision Lab
Alert Volume Falls during a Blind Source Period
A fictional dashboard reports a major improvement because related alert volume falls to zero. The required network source was Blind for forty minutes across three critical service zones.
Scenario Decision Lab
Closure Time Improves while Reopen Rate Doubles
A fictional dashboard shows a 30% improvement in case-closure time. Quality review shows that cases often close when alerts stop, source reconciliation remains incomplete, and reopen rate has doubled.
Advanced Challenge
Defend a SIEM Dashboard before a Review Board
Fictional Northbridge presents a leadership dashboard with lower alert volume, faster closure, fewer Unknown states, more escalations, and higher rule count. The dashboard does not show unique conditions, source Blind periods, coverage, denominator changes, reopenings, closure quality, grouping regression, workload complexity, privacy, or residual-risk ownership.
Defend the decisions
Explain which fictional analyst, source, service, privacy, recovery, risk, and leadership decisions each panel supports.
Defend the definitions
Explain fictional populations, numerators, denominators, grain, time range, exclusions, transformations, and limitations.
Defend source health
Explain fictional Healthy, Conditional, Degraded, Blind, Conflicting, and Recovering effects on metrics.
Defend balance
Explain fictional volume versus uniqueness, speed versus quality, suppression versus coverage, and closure versus reopening.
Defend behavior and privacy
Explain fictional anti-gaming, team-level views, complexity, access, retention, fairness, and purpose limitation.
Defend lifecycle
Explain fictional owners, thresholds, review triggers, change annotations, debt, residual risk, replacement, and retirement.
Challenge output
Produce a fictional dashboard charter, audience map, metric-family inventory, metric dictionary, dashboard wireframe, balanced-metric matrix, source-health layer, misleading-pattern review, validation matrix, privacy review, anti-gaming review, owner matrix, change history, metric-debt register, residual-risk statement, leadership brief, and public portfolio boundary.
Defender Habits
Dashboards and Metrics Checklist
Check Your Understanding
A6.8 Mini Quiz: Dashboards and Metrics
Choose your answers first. Explanations appear only after submission.
1. What is the strongest purpose of a fictional SIEM dashboard?
2. Why is raw alert volume alone weak?
3. A fictional average first-review time is eight minutes, but several High-priority alerts waited over one hour. What is the best improvement?
4. A fictional source is Blind and related alert volume drops to zero. What should the dashboard show?
5. Why must a fictional metric define its denominator?
6. Which fictional metric set is most balanced?
7. Which public portfolio approach is safest?
Portfolio Prompt
Portfolio Prompt
Create a fully fictional Dashboards and Metrics Package for the Northbridge Student-Support Cooperative. Include mission, stakeholders, dashboard audiences, analyst decisions, source-owner decisions, service-owner decisions, detection-owner decisions, privacy decisions, recovery decisions, risk decisions, leadership decisions, alert-volume metrics, raw-alert counts, unique-condition counts, grouped-work-item counts, expected-alert counts, duplicate counts, replay counts, timeliness metrics, event-to-collection time, collection-to-processing time, processing-to-alert time, alert-to-first-review time, owner-response time, action time, validation time, closure time, source-health metrics, freshness, completeness, schema quality, parser quality, queue health, clock health, conflict rate, blind minutes, recovery reconciliation, alert-quality metrics, usefulness, alert-contract completeness, expected-alert accuracy, false-positive rate, known false-negative count, triage-quality metrics, question quality, evidence-layer accuracy, source-health visibility, request precision, state accuracy, escalation metrics, trigger precision, delayed escalation, premature escalation, handoff completeness, acceptance, aging, de-escalation, case-quality metrics, note neutrality, evidence traceability, chronology accuracy, decision-log completeness, action-validation separation, closure quality, reopen rate, coverage metrics, identity coverage, device coverage, service coverage, destination coverage, behavior coverage, source coverage, time coverage, environment coverage, source-health coverage, test coverage, workload metrics, queue age, evidence-request burden, duplicate work, owner-response delay, rework rate, specialist demand, privacy metrics, unnecessary fields, access, sharing, retention, exceptions, review dates, recovery metrics, backlog, replay, duplicates, sessions, source health, service validation, residual-risk metrics, coverage debt, detection debt, priority debt, escalation debt, case debt, documentation debt, metric names, decision purpose, populations, numerators, denominators, grain, time ranges, update cadence, data sources, provenance, transformations, thresholds, interpretations, limitations, owners, review triggers, mission-impact panel, volume panel, source-health panel, queue panel, triage panel, escalation panel, case-quality panel, coverage panel, workload panel, privacy panel, recovery panel, residual-risk panel, average-versus-median review, percentiles, distributions, duplicate-inflation review, blind-source review, denominator review, scope-change annotations, suppression-regression review, fast-closure review, individual-ranking review, stale-criticality review, metric-owner review, privacy review, anti-gaming review, change management, metric retirement, validation cases, expected outcomes, observed outcomes, defects, corrective actions, quality gates, metric debt, owner matrix, leadership summary, reflection, and a statement that every organization, source, metric, alert, owner, date, value, decision, and outcome is invented.
Begin with fictional decisions and audiences before selecting measures.
Define every population, numerator, denominator, time range, source-health rule, owner, limitation, and review trigger.
Balance fictional speed, volume, quality, coverage, workload, privacy, recovery, and residual risk.
Test for blind periods, duplicates, denominator changes, scope expansion, grouping regressions, premature closure, and metric gaming.
Keep the entire artifact completely fictional, defensive, non-operational, privacy-safe, evidence-aware, maintainable, and suitable for a public learning portfolio.
Confidence / Readiness Reflection
Are You Ready for Reducing Noise and Improving Quality?
Before moving to A6.9, rate your readiness from 1 to 5 for dashboard purpose, metric families, populations, numerators, denominators, grain, time ranges, source health, distributions, coverage, workload, privacy, anti-gaming, governance, residual risk, and complete fictionalization.
I can explain why a better-looking fictional metric may represent a worse mission outcome.
I can define a fictional metric so another reviewer can reproduce it.
I can identify misleading averages, denominators, raw counts, and scope changes.
I can make source-health and coverage limitations visible.
I can balance fictional speed, quality, workload, coverage, recovery, and residual risk.
I can design privacy-aware and anti-gaming dashboard controls.
I can assign fictional metric ownership, thresholds, review triggers, change history, debt, and retirement.
I can produce a safe fictional dashboard package without copying real screenshots, values, sources, or priorities.
Record one fictional dashboard decision, one metric, its population, numerator, denominator, source-health limitation, owner, and one question you will carry into A6.9.
Key Takeaways
What You Should Remember
1.Fictional dashboards should support bounded decisions rather than prove success or reward activity.
2.Every fictional metric needs purpose, population, numerator, denominator, grain, time range, sources, limitations, owner, threshold, and review trigger.
3.Raw alert volume does not show uniqueness, usefulness, coverage, source health, or why the count changed.
4.Averages can hide severely delayed cases; medians, percentiles, distributions, segments, and oldest-case views provide stronger context.
5.Blind or Degraded sources can make quiet dashboards look healthy even when evidence is missing.
6.Balanced measures should pair speed with quality, suppression with coverage, closure with reopening, and workload with complexity.
7.False-positive, closure, and quality rates require clear denominators that preserve Unknown, Source-Degraded, unreviewed, Expected, and excluded populations.
8.Personal leaderboards, stale context, unowned metrics, and target-driven gaming can weaken privacy, fairness, collaboration, and mission outcomes.
9.Dashboard governance includes purpose, definition, source health, ownership, interpretation, thresholds, privacy, anti-gaming, change management, and retirement.
10.Every CyberShield dashboard artifact must remain fully fictional, authorized, defensive, non-operational, privacy-safe, and incapable of exposing real systems or people.
Navigation
Continue Module A6
Next, learn how fictional defenders reduce alert noise and improve quality through root-cause analysis, source repair, context, grouping, deduplication, thresholds, suppression, testing, rollback, ownership, coverage review, and residual-risk tracking.