Tools For Risk

Management
Week 7

Lecturer: Barney Baldwin

Announcements (cont’d)

• Individual Project
• BEACON - We will be meeting tomorrow at 12 noon.
• ARCHER – you should be working in Archer to implement the risk assessment
• Today we will
• Review Change Capability from Last week
• Cover Enterprise Change Processes
• Drill into the group project

2
Quiz 2
Individual Project
Individual Project

• Final Artifacts due April 3

• Archer – submitted in Canvas
• Presentation of Risk Assessment, suitable for CRO
• 8-12 slides
• Company background and statement of the risk
• Approach to assessing the risk (including the questions)
• Presentation of the results
• Quantitative (with charts from Archer)
• Qualitative (describe the outcome)
• Mitigation strategy (how should the firm respond?)
• Beacon
• Review in office hours, submit code and results in Canvas

5
Group Project
Group Project Announcements

• All groups should be formed, risk problem selected

• Sprints are in progress
• All teams must join office hours 11 AM Wednesday March 26!
• LMK if you can’t make it and we will set up another time
• DRAFT documents due March 27
• Scrum template
• DRAFT High Level Requirements
• DRAFT Tasks and Plan
• DRAFT Presentation (if available)

7
OPTIONAL Peer Reviews for Group Project

• Submit grade for each fellow student in your group

• (M) Met expectations (should be most common)
• (E) Exceeded expectations
• (B) Below expectations

• Email CONFIDENTIALLY to lecturer (only). I may follow up with

questions or clarification
• Only required if peer grade is “E” or “B”
• Accounts for up to 25% of group exercise grade

8
Change Capability
(review)
 Change Capability, ROI, and the Value/Cost curve
• Change Capability is the ability to
efficiently implement changes to business
processes
• Return on Investment (ROI) in the context
of a systems change is:
Increase
value / cost Change
Value
of the change, over a time period Capability

• Quantifying value can be challenging for

technology and risk projects Change 1
• Appropriate management of tools will Change 2
maximize Return on Investment – which
is the derivative of the value/cost curve
Cost

10
How to Improve Change Capability

• Measure complexity, and account for complexity in change projects

• Adopt standards for technology and clear processes for adoption of new
technology
• Establish robust change management practices
• Version control is absolutely essential!!!
• Automate all repetitive tasks for delivery
• Continuous Integration / Continuous Delivery (CI/CD)
• DevOps is the practice of automating development/change
• All steps between code change and production release are automated

• Simplify and standardize enterprise data

• Adopt Agile practices where possible

11
Planning and Change
Management
Budgets, Planning, and Change Management

• Risk Tools require enterprise budget

• Extensive planning for implementation
• Change Management (CM) is the processes for implementing
technology changes

13
Tools Planning - Technology Budgets

• Budget and planning

• Complex process involving senior management
• Most firms do this annually
• Limited ability to adapt, respond to business changes
• More progressive firms plan quarterly or monthly
• Technology Spend Governance
• Progressive firms will fund new initiatives and ongoing value streams
• Budget for improving change capability
• Funding teams and platforms (e.g. Agile) allows top-level planning and budget
• Teams, with stakeholder involvement & product owner, prioritize changes
• Big initiatives or changes in team size require quarterly escalation

14
Technology Change Management

• Change Management (CM) is the processes that firms use to reduce risk
from changes
• Coordinate changes across systems, avoid collisions
• Ensure systems are tested, processes followed to mitigate risk
• At some firms, the process is so heavy-weight that business users go to
great lengths to avoid it
• The change process includes:
• Development and testing
• Production Changes
• Configuration and data changes

15
Change Management – Software Lifecycle

• Changes migrate from:

• Development, to
• Test environments, to
• Production (Live) environment(s)
• Key practices to improve these processes
• Manage environments carefully
• Automate software builds
• Automate software testing
• Ensure realistic data is available in test environments
• Mask sensitive data
• Automate migration from between environments
• DevOps: the practice of for automating the software lifecycle

16
Change Management – Data and system configuration

• Data changes and system configuration

• System or department-specific activity
• Usually managed by the business teams, sometimes with IT support

• Any time somebody uses the system some data changes

• Changes that affect the system broadly must be managed carefully
• New types of financial products or deals
• New user activities
• These should be tested, almost always in a test environment

• Configuration can grow complex and unmanageable

• Governance is needed
• “Power users” or ”citizen developers’

17
Best Practices for Change Management (1)

Understand the organizations’ risk tolerance and plan accordingly

• Critical systems, regulatory issues need much more care
• Department-specific and local changes are less critical
Use data to adapt the change management approach
• Measure changes and incidents
• Analyze the data and focus resources on the riskiest changes.

Simplify the processes as much as possible

• Minimize approvals
• Leverage tools to automate change processes

Adapted from Atlassian [link] 18

Best Practices for Change Management (2)

Establish appropriate, lightweight governance

• Risk-based approach to change assessment
• Work towards high frequency of changes
Use tools to automate and tune processes
• Simple checks and balances built into processes
• Automate testing, release, back-out
• Standardize dev/test/production environments
Make smaller changes to build success
• Release small changes frequently
• Easier to identify issues, build velocity

Adapted from Atlassian [link] 19

Best Practices for Change Management (3)

Industry frameworks and approaches are guidelines, not rules

• ITIL (IT Information Library) is a set of guidelines for change and service management – but is not
universal
• Likewise, Agile, Waterfall, DevOps and other practices are not always the best approach
• Tailor the approach to the problem
• Focus on the team’s culture and capability
Prioritize Collaboration
• Change management involves many teams
• Value interactions over processes and tools (Agile manifesto)

Adapted from Atlassian [link] 20

Best Practices for Change Management (4)

Stress the systems to ensure reliability

• Build resilience into the change processes to reduce risk
• Chaos engineering involves breaking or shutting down components at random
• Netflix Chaos Monkey
• Resilience engineering involves deliberately addressing all possible system stressors –
eg. Very high users, system demand, etc.
• Systems should fail gracefully, if at all
Standardize the Tools used for change management
• Repeatable processes and established tools
• Automate the change processes across all platforms

Adapted from Atlassian [link] 21

Project & Tech Risk: Basis
for Group Project
Types of Technology Risk

IT Risk Sub-discipline Key Risk

Information and cybersecurity Leakage of confidential customer and internal data, fraudulent
transactions, phishing, ransomware
Resilience and disaster Recurring or prolonged interruptions of IT services supporting
recovery processes that are critical for customers or bank
Vendor and third-party Vendors or third parties not delivering reliable and secure service
management
Project and change IT projects not delivering on schedule and within budget, or not at
management adequate quality
Architecture, development, Systems not being designed to deliver long term affordable, reliable and
and testing maintainable service to enterprise
Data quality and governance Legal/regulatory or transaction-settlement issues as a result of
inaccurate, inconsistent, or missing data
IT compliance Noncompliance of IT systems and process with regulations

Adapted from McKinsey [link] 23

Why do so many projects fail?

• PwC 2014 study of 10,000 projects found that only 2.5% of

companies delivered their projects on-time and on-budget
• Almost every industry study finds failure rates of >50%
• Why? Most common cited reasons:
• Poor project management
• Inadequate stakeholder engagement
• Overly ambitious objectives
• Many project teams believe the initiatives are “doomed from the start”
• People & Process Issues ➔ Low Change Capability

24
Will HypoBank succeed with the implementation?

• What can go wrong with the project?

• Consider
• People
• Process
• Technology
• Data
• Change Capability
• Your job is to ensure that this is successful

25
Discussion – How to reduce ERM tool project risk

• Consider the T4RM framework

• People/Process/Tech/Data/Change Capability

• What questions would you ask to identify project risks and

solutions?

26
T4RM Framework for a Successful Implementation
(People)
• For an ERM Tool Implementation, all aspects of the Framework
should be considered
• People/Organization
• Do we have the right teams?
• Are there accountable staff for all activities?
• Is our tech team ready?
• Is our vendor capable?
• Is hiring needed?
• Will roles change?
• Is a reorganization needed?

27
T4RM Framework for a Successful Implementation
(Process)
• Process
• How will the risk management process change?
• Consider measurement, aggregation, presentation
• What new processes are needed?
• What processes will go away?
• Are there opportunities to improve existing processes?

28
T4RM Framework for a Successful Implementation
(Technology)
• Technology
• Is the platform aligned with our tech standards?
• Do we need to adapt or change our tech policies?
• What tech processes will be needed to operate and support the new platform?
• Do we have appropriate test and production environments?
• What security considerations are needed to protect the data?
• What is are the requirements for system resilience?
• What are the performance requirements for the new system?
• How will the system be tested?

29
T4RM Framework for a Successful Implementation (Data)

• Data
• What data will the new tool require?
• What new data will be produced?
• How will the data be transformed in and out of the system?
• Can we eliminate any legacy data repositories, files, processes?
• What data cleansing is needed in advance of the implementation?
• What data mapping is needed to and from the new system?
• Upstream (data consumption) and Downstream (data outputs)
• What reports are needed?

30
T4RM Framework for a Successful Implementation
(Change Capability)
• Change Capability
• How will changes to the system be governed?
• What kinds of changes are needed for the initial implementation?
• What kinds of changes will likely be needed in the future?
• Will Agile teams be leveraged for some parts of the delivery?

31
Sample Project Plan – Gantt Chart (in “templates” folder)

32
Implementing an
ERM Tool
Revisiting HypoBank’s organization

• Board and senior management sets strategy

• Risk Management team is responsible for:
• Risk Control Policies
• Risk Tools *
• Risk Data
• Front Office teams (sales / trading)
• Front office Tools *
• Front office Data
• Technology team
• All technology platforms and Tools *

* Why is Tool responsibility shared? 34

Enterprise Technology Processes

• Virtually every large firm has processes for implementing major

software platforms
• These processes are designed to:
• Mitigate project risk
• Enforce business and tech standards
• Architecture compliance
• Security reviews
• Business continuity planning (BCP)
• Increase the likelihood of a successful implementation
• These steps will vary from firm to firm
• We will discuss best practices and common pitfalls
35
Organizational Stakeholders

• Governance for Risk Tool implementations

• Sponsoring Organization (Risk Management, CRO)
• Technology Teams (CIO, CTO, Tech management)
• Finance Team
• Business stakeholders, depending on the project
• Change Governance function, if applicable
• Enterprise Project Management

36
Implementing a Risk Tool - Budget

• Budget Approval
• Presentation by business sponsors
• Annual or quarterly review
• Senior management (CRO, CTO, CFO), Board of Directors approval
• Vendor (aka Third-Party) Risk Assessment
• Find appropriate vendors
• Industry assessment
• RFI (request for information) process
• Comprehensive questionnaire for the shortlisted vendors
• Early Cost Estimates
• Select a Vendor
• Sometimes with Proof of concept (POC) implementation

37
Implementing a Risk Tool – Vendor Engagement

• Vendor Selection
• Find appropriate vendors
• Industry assessment
• RFI (request for information) process
• Comprehensive questionnaire for the shortlisted vendors
• Early Cost Estimates
• Select a Vendor
• Sometimes with Proof of concept (POC) implementation

• Vendor (aka Third-Party) Risk Assessment

• Client and industry perception of vendor
• Viability (size, track record, etc)
• Assess vendor practices

38
Implementing a Risk Tool – Requirements and Contract

• Select the final vendor

• Usually involves a quantitative scorecard
• Contract negotiations continue during this process
• Finalize the requirements for initial implementation
• In Agile terms – Minimum Viable Product (MVP)
• Generally a subset of the desired functionality
• Requirements document is created by stakeholders
• Shared with vendor
• Usually some iteration to finalize the requirements
• Contract Negotiation ongoing

39
Implementing a Risk Tool – Execute the Contract

• Execute the contract

• Usually, CRO or CTO for >$1mm
• Multiple approvals
• Contract Payment Terms
• Initiation
• Milestones
• Final Payment

40
Implementing a Risk Tool – Business Process Changes

• Most ERM tools will require significant changes to business

processes
• Improvements to Risk Management processes (e.g. Risk modelling, Risk Reporting)
• Organizational accountability is established
• Often a reorganization is required
• Examples Critical
• Market Risk problem: for
• New business processes to support IMM Success
• Op Risk Problem: !!!
• Centralized Issue management Team
• Consistent Taxonomy for Op Risk Management

41
Implementing a Risk Tool – High Level Architecture and
Implementation Plan
• High-level System Architecture
• On-premise, private, or public cloud
• Data interfaces
• Batch processes Critical
• Business Continuity for
• Change processes & environments Success
• High-level project plan !!!
• Stages of implementation
• Sequencing business process migration
• Team roles, Vendor and contractor engagement
• Test Plan

42
Implementing a Risk Tool – Technology Risk
• Security Reviews
• User roles, Access controls
• Data Encryption
• At Rest
• In Flight
• Penetration Testing
• Data controls in production, test, and dev
environments
• Functional Reviews
• Performance testing
• Availability
• Vendor reviews
• Project Risk Reviews
43
Implementing a Risk Tool - Development

• Some large firms have the capability to develop risk tools from
scratch.
• Pros: Totally custom solution, tailored to the business
• Cons: Extremely costly and risky, particularly these days
• Most choose to use vendor platforms
• Most implementations will require some in-house development
• Integration with other systems (data mapping, feeds)
• Risk Model integration using vendor APIs
• Other extensions
• Development will follow the firm’s processes

44
Implementing a Risk Tool – Technology Environments

• Set up infrastructure for environments

• Development
• Test (usually multiple environments)
• Production (including Business Continuity environment) Critical
• Automate deployments dev->test->production! for
Success
• Establish data management practices and
!!!
tools
• Data migration from prior systems into new tool
• Data masking for lower environments
• Data backups, archiving
• Automate data processes!

45
Implementing a Risk Tool – Staging releases

• Large systems will require multiple releases, even in a pure

waterfall environment
• Staging by business process and/or features
• In steps, migrate business processes and add features with subsequent releases
• Waterfall and Agile methods are often combined
• Waterfall for major releases
• First release
• Major upgrades
• Agile for smaller changes
• Model changes, Configuration changes, reports

46
Implementing a Risk Tool – Testing Major Releases

• A comprehensive test plan is critical to success

• Developers and power users perform “unit” tests.
• Release for integration testing
• Also called “QA” (quality assurance) or “SIT” (system
integration testing) environment
• Tests conducted with all upstream and downstream
systems
• User acceptance testing is in a dedicated environment
(“UAT”)
• Often, a separate testing (“QA”, quality assurance) team is
engaged to conduct tests
• Automation of tests is critical for change capability

47
Implementing a Risk Tool – Major release process

• Software release (“drop”) from vendor into dev environment

Dev
• Functional configuration of screens, reports
• Development to extend and integrate into client environment
• System feeds (in and out) integrated SIT
• Custom models integrated
• Release to System Integration Environment UAT
• Test system interactions
• Release to user testing (UAT) environment
• Iterate with bug fixes flowing from dev->test->UAT Prod
• Sign off when critical bugs are fixed
• Release to production

48
Implementing a Risk Tool – Go-live

• A new risk tool is likely to require several major releases

• Usually done over a weekend
• Each will require
• Sign-off on critical bugs
• User Training completed
• Change Management Process
• Software Deployment from UAT environment to Production
• Configuration of new production services
• “Smoke” Testing (health check)
• Careful monitoring in the morning

49
Implementing a Risk Tool – Staging and Migration

• It is generally safer to have multiple small migrations rather

than one large one
• For example, our group project requires us to migrate multiple
business units to a new platform
• We should start with either the easiest or the most critical business
• Each migration will require Dev/Test/Release cycles
• Data migration will be required, both for testing and for go-live, from the legacy
platform

50
Mid-term course
evaluation!
Next Week…
Next Module (Week 8)

• Focus on Risk Reporting

• Read:
• Agarwal, R., & Kallapur, S. (2021,.June 9). Four Ways to Improve Risk Reporting.
(part of HBP Coursepack).

53