System Risk Assessment
Impact Assessment and Risk Assessment are two critical evaluations for AI systems in healthcare.
Impact Assessment identifies who will be affected by the AI system - doctors, nurses, IT staff, patients, and others - and analyzes how the system’s design, implementation, and operation will change their workflows, responsibilities, and decision-making processes.
Risk Assessment systematically evaluates what could go wrong, identifying potential technical, clinical, operational, and regulatory risks, assessing their severity and likelihood, and determining who would be harmed and how severely.
In Pacific AI, Risk Assessment is an AI-assisted workflow that automatically analyzes uploaded system documentation (architecture, design, quality assurance, operational procedures, policies) to generate structured risk reports. These reports enable Risk Managers and Compliance Officers to understand the system’s safety profile, prioritize mitigation strategies, and make informed deployment decisions.
Both assessments incorporate role-based analysis - considering how risks manifest differently for various stakeholders - and support periodic reassessment throughout the system lifecycle to ensure evaluations remain current as the AI system evolves.
Initiate Risk Assessment
Section titled “Initiate Risk Assessment”Navigate to System Risk within a specific system to start the guided risk assessment process. The workflow walks users through document selection, architecture review, impact analysis, and risk identification.
Select Relevant Documents
Section titled “Select Relevant Documents”Select one or more Architecture and Design documents that will serve as the basis for analysis.
Click Start Assessment to initiate AI-powered risk evaluation using the selected documents.
Architecture Information
Section titled “Architecture Information”The system analyzes the selected documents and automatically generates an Architecture Information snapshot. This snapshot provides an initial view of the system architecture based on the available documentation.
The Risk Manager can review and validate the generated information by:
- Reviewing the generated architecture summary
- Editing or refining architectural details
- Adding missing information
- Removing inaccurate or irrelevant content
If additional information is required, the Risk Manager can either provide or request clarification in several ways:
- Click the Clarification Required icon next to a section and enter a clarification request directly.
- Generate a Clarification Link and share it with team members to collect the required information.
- Download all clarification questions as a Markdown (
.md) file, update the file with the responses, and upload it as a system document.
Once reviewed and validated, the Architecture Information snapshot serves as the foundation for subsequent risk assessments and analysis activities.
Impact Assessment
Section titled “Impact Assessment”Assess how the system affects different stakeholder roles.
Default roles include Doctor, Nurse, Administrator, Technician, Patient, Radiologist, DataScientist and others. Custom roles can also be added if required.
The system generates an AI-based impact summary for each selected role. The Risk Manager reviews and edits the assessment before proceeding.
Risk Registry
Section titled “Risk Registry”Based on the Architecture Information and Impact Assessment, the system proposes risks aligned with Responsible AI principles.
Risk categories include:
- Fairness
- Accountability
- Transparency and Explainability
- Reliability and Robustness
- Safety
- Privacy and Security
- Accessibility
Users may use Auto Discovery to generate risks across all categories, or Guided Definition to focus on specific categories. They can also accept, reject, or edit proposed risks, and attach supporting evidence or additional explanation.
Summary and Risk Level
Section titled “Summary and Risk Level”After completing all steps, the system generates a consolidated summary including:
- Architecture Information
- Impact Assessment
- Risk Registry
- Overall Risk Level
Reports can be reviewed online or downloaded for recordkeeping and audits.
Submit for Review
Section titled “Submit for Review”Once finalized, the Risk Manager selects Submit for Review under the Actions button at the top of the page. The assessment status changes from Draft to In Review.
Compliance Officer Review
Section titled “Compliance Officer Review”The Compliance Officer may take one of the following actions:
Approve
Section titled “Approve”Approve the assessment and complete required governance fields:
- Lifecycle Stage
- Last Checkpoint
- Next Checkpoint
- Availability (Not Released, Internal Use, External Use, Global)
- Regulatory Approval (Not Required, Required, Pending, Acquired)
The system automatically reflects the calculated risk level.
Reject
Section titled “Reject”Reject the assessment and provide a reason or feedback for revision.
Return to Draft
Section titled “Return to Draft”Return the assessment to Draft status for updates. A comment is required to clarify the requested changes.
Delta Risk Assessment
Section titled “Delta Risk Assessment”A Delta Risk Assessment automatically updates an existing Risk Assessment based on changes to system documentation or organizational policies.
Instead of regenerating the complete Risk Registry, a Delta SRA analyzes the relevant changes and updates the existing assessment. This allows manually reviewed risks, edits, and mitigations to be preserved while new or modified risks are identified.
For system document changes, Pacific AI compares the new document version with the previous version and analyzes only the relevant changed sections.
When an organizational policy changes, Pacific AI identifies systems with completed Risk Assessments that may be affected and queues them for reassessment. Policy-driven reassessments run as background jobs so they do not interfere with other ongoing AI evaluations.
Delta SRA Processing
Section titled “Delta SRA Processing”Delta SRA processing runs in the background. The system displays the current processing status so users can monitor the assessment.
The status indicates whether the Delta SRA is:
- Queued
- In Progress
- Completed
- Failed
When processing is complete, the identified risk changes are applied to the appropriate system and recorded in the risk change history.
Review Generated Risk Changes
Section titled “Review Generated Risk Changes”After a Delta SRA completes, Risk Managers can review the generated changes as part of the System Risk Assessment review process. The Risk Manager can review newly identified risks and changes to existing risk levels, make any necessary updates, and review the supporting information before finalizing the assessment.
The generated changes do not bypass the normal SRA governance workflow. After review, the Risk Manager can submit the assessment for review, and the Compliance Officer can Approve, Reject, or Return to Draft the assessment according to the standard review and approval process.
This ensures that automatically identified risk changes remain subject to human review and organizational governance before the assessment is finalized.
Risk Change History
Section titled “Risk Change History”Pacific AI maintains a history of changes made to risks to provide traceability for Risk Managers and Governance Officers. The history records both Manual changes made by users and Automatic changes generated by Pacific AI during SRA processing.
Each change can include:
- Source: Manual or Automatic
- User or System: The user who made the change or the system process that generated it
- Date and Time: When the change occurred
- Reason: The event that triggered the change, such as a document being added or deleted, a document version being updated, a policy being deprecated, or a policy being activated
- Source Document: The document or document version associated with the change, when applicable
This history provides a clear record of how the Risk Registry has changed over time and supports governance reviews and audits.
SRA Change Summary
Section titled “SRA Change Summary”After a full Risk Assessment or Delta Risk Assessment completes, Pacific AI generates a plain-language summary of the changes made during the run. The summary provides a high-level explanation of what changed and why, allowing Risk Managers and Governance Officers to review the outcome without examining each individual risk change.
Depending on the assessment, the summary may include:
- Documents or policies that changed
- Architecture sections that changed
- Impact Assessment sections that changed
- Newly identified risks
- Risks whose risk levels changed
- Reasons for new or changed risks
The summary is stored for the relevant system and is available only to users who have access to that system’s System Risk Assessment.
System Risk Assessment provides structured documentation, traceable risk management, and ongoing oversight of AI systems. Incremental assessments, risk change history, and change summaries help organizations keep risk assessments current while preserving human review and governance decisions throughout the system lifecycle.