A maturing

DevSecOps
landscape
2021 Global Survey results
• 4300 respondents
• Dramatic advances in release/deployment
frequencies, automation, security
• Developers, operations and security pros
in their own words
Table of contents

01 INTRODUCTION 20 OPERATIONS
Operations
02 OVERVIEW Still so many tools
The starting point Working with development
Software development today Looking to the future
Where the tools rank
The role of the DevOps platform 24 KEEPING THE DEVOPS MOMENTUM

09 DEVELOPERS
Devs and DevOps
Developer daily life
Security
Looking to the future

16 SECURITY
Security and DevSecOps
Roles are changing
Shifting left
Who’s in charge?
About the bugs
Looking to the future
 1

Introduction

60% of developers are releasing code 2x faster Anecdotally, we heard about the hard work that
For the fourth year in a row, we went into these results: mindset shifts, tough
than before, thanks to DevOps – up 25% from
asked DevOps teams to tell the truth discussions, and detailed analysis. DevOps isn’t
(pre-pandemic) 2021.
about their practices and processes, easy (even for us) but it seems a focus on outcomes
their challenges and their careers. using real data can help teams of all sizes in all
72% of security pros rated their organizations’ countries move forward.
With a global pandemic swirling, we security efforts as “good” or “strong” – up
were surprised when nearly 4,300 13% over 2021. As always, we’ll remind you this is our survey, so
people took time to do just that this don’t be surprised if respondents mention us or
56% of ops teams members said they are use our products (roughly 50% of survey takers are
past February. GitLab customers). Also, 43% of our survey takers
“fully” or mostly automated – up 10% from
2021. have been “doing” DevOps for between three
We were even more surprised by the results. years and five years or more, so they’re seasoned
This year, for the first time ever, DevOps became practitioners with what are often aspirational
serious. It’s somber. It’s grown up. It’s happening. Almost 25% of respondents claimed to have
results. Your results may be different, and that’s ok.
We didn’t ask a single question about Covid, but the full test automation – up 13% from 2021.
answers seem shaped by that lingering experience. Let’s get started.
It’s as if, in the face of calamity, teams everywhere 75% of teams are either using AI/ML or bots for
decided to focus on what mattered most, whether test/code review, or they’re planning to – up
that was automation, or testing, or embracing
41% from 2021.
cutting-edge technologies.

In 2021, teams are poised to step out of the Last year dev, sec, and ops said they needed
DevOps “culture” battle and into the real work better communication and collaboration
of technology implementation and (surprisingly) skills for their future careers. This year, after
upbeat results. an intense period of enforced soft skills, their
priorities have shifted dramatically to AI/ML
(devs), subject matter expertise (sec), and
advanced programming (ops).
 2

Overview

2021 DevSecOps Survey top findings

DevOps = broader technology base

CI/CD still matters, but DevOps platforms are on the rise as
is AI/ML.

The trouble with testing

For the third year in a row, respondents pointed to testing as the
primary reason for release delays.

Want better code quality?

Then it’s DevOps hands down. Other benefits include improved
time to market and better planning.

Getting serious
Last year, teams talked about Kubernetes and microservices; this
year, they’re using them, or planning to soon.

Not just dev and ops

A DevOps platform offers something for everyone – literally. A full
23% said everyone in their company uses the DevOps platform.
 3

The starting point GENDER

Here’s a closer look at the nearly 4300 (4294) people who completed the survey
14.95% Female
between February and early March 2021.
81.86% Male
ROLE
0.93% Non-binary/third gender
41.67% Software Developer / Software Engineer
0.51% Prefer to self describe
8.59% DevOps Engineer
1.75% Prefer not to say
6.87% Development/Engineering Leadership
INDUSTRY
7.88% Software Architect

40.04% Computer Hardware / Services / Software / SaaS

4.56% Other

6.3% Banking / Financial Services

2.52% DevOps Leadership

4.78% Other
2.81% Technology Executive - CIO / CTO/VP

18.05% Education
4.11% Project Manager

5.29% Business Services / Consulting

1.66% Site Reliability Engineer

3.4% Telecommunications
2.57% Systems Administrator

2.19% Media & Entertainment

1.91% Systems Engineer / Network Engineer

1.91% Healthcare
2.23% Product Manager

2.24% Government
1.56% Quality Assurance

1.14% Operations Leadership

 4

REGION NUMBER OF EMPLOYEES

20.28% Europe and Russia 25.37% 1 - 10 people

10.78% North America 21.34% 11 - 100 people

50% Asia 5.11% 101 - 500 people

2.28% South America 8.14% 501 - 1,000 people

0.77% Australia and New Zealand 12.31% 1,001 - 10,000 people

1.82% Africa 11.08% 10,000+ people

0.98% Middle East 5.92% Don’t know

 5

Software development today Other technologies mentioned include:

In 2021, a majority of survey respondents (35.9%) told us their teams develop

Infrastructure as code (IaC)
software using DevOps or DevSecOps, followed closely by Agile/Scrum at
31.78%. That’s a big jump for “self-identified” DevOps usage in a year: in 2020 GitOps
only 27% of teams described their process as DevOps or DevSecOps. Just over
Kubernetes
10% said their teams use Waterfall (up from under 8% in 2020) and, identical
to last year, 5% of teams are “creative” and describe their process as Water/ NetDevOps (networking at scale)
Scrum/Fall.
Platform “bootstrapping” using CI/CD
MOST PRACTICED DEVELOPMENT METHODOLOGIES: Extensive use of SREs

35.9% DevOps/DevSecOps Unix

31.78% Agile/Scrum For the second year in a row, respondents said devs are the most likely to
benefit from a DevOps practice (36%), followed by ops (22%), security (16%),
13.02% Kanban with QA and the business side coming in at 13%.

10.02% Waterfall The top three reasons to choose DevOps? Code quality, faster time to
market, and security. Other clear benefits from a DevOps practice: improved
5.01% Water/Scrum/Fall
communication/collaboration and happier developers, both of which rated
4.20% Lean
much higher in 2021 than in 2020.

Just over 30% of respondents said their DevOps practices are between one and Almost 59% of survey respondents said their teams deploy multiple times a
three years old. Almost 27% have had DevOps in place for a year or less, while day, once a day, or once every few days, a percentage nearly identical to last
nearly 23% have been doing it for five or more years. About 20% are in the year’s and one that likely tracks with the over 61% of survey takers who work
DevOps “sweet spot” of between three and five years, meaning they’ve known at companies with 500 employees or less. All told, 28% deploy continuously
success and are comfortable with the processes and routines. (multiple times a day), while 15% deploy once a week, 10% once a month, and
under 7% once every few months.
What do today’s DevOps implementations look like? CI/CD was the most likely
to be part of the process, followed by DevSecOps, test automation, and a
DevOps platform. In 2020, just 4% of respondents used AI/ML in DevOps; this
year, 11.5% reported they do.
 6

Not surprisingly, the vast majority of survey respondents participate in open Testing remains tough
source projects – over 69% this year up from 63% last year. More than 29% said For the third year in a row, a majority of survey takers resoundingly pointed to
they contributed to Gitlab, while 14% are involved with Kubernetes, and 13% testing as the area most likely to cause delays. The other bottlenecks include
with VS Code. Nearly 19%, though, said they are involved in “other” projects, planning, code development, and code review, again reflecting what we’ve
many of them smaller and lesser-known (a trend we saw last year as well). seen in our 2019 and 2020 surveys.

Anecdotally, our respondents had a lot to say about these areas:

“Covid** (mentioned a number of times).”

“Testing delays everything.” “Testing can be both slow in writing and running.”

“Communication between business and development is hard.”

“Security is not integrated at all into the development process :(.”

“We have a strict code review process and it often takes several days for
the reviewer to respond to requests for review.”

“Finding someone for code review can be hard (1 day average). After that
“Many people find it a chore to review code.” business tests take time to be complete (2-4 days on average).”

“Testing is not yet fully automated in the deployment cycle; hoping to

improve that with our move from BitBucket + Jenkins/drone to GitLab.”

“A majority of our delays are created by multiple teams being involved,

under different management umbrellas, without one cohesive vision at
the top layer.”

“Operation failures due to rushed or ignored planning.” “We are currently using the same technology stack that was chosen five
years ago, so it is not up to date with modern practices yet like TDD,
version control, and hexagonal architecture.”

“Developers are sometimes unaware they have to do code reviews. They

aren't sure how to perform them and if they are effective. Sometimes
they are skipped so the process can go through.”
 7

Threading the testing needle is a challenge, but there are some small signs of Where the tools rank
forward momentum. Almost 25% of teams report full test automation (more
Almost 85% of survey takers use Git for source control (down from 92% last
than double what was reported last year) and 28% of respondents say they’re
year), while almost 4% use Team Foundation Server, and 2% use CVS. Just 5%
at least half-way there. Roughly 34% of survey takers said developers test some
of respondents said they don’t use any source control.
of their own code (up from 31% last year) and 32% said automated testing
happens as code is written, a big jump from 25% in 2020.
GitLab is the tool of choice for CI/builds (34%), followed by Jenkins (21%),
GitHub Actions (14%), and BitBucket (8%).
But 25% of teams either have no test automation or are just starting to think
about it, and 9% admit their teams haven’t shifted testing far enough left.
Just over 37% of survey takers said they “partially” use microservices,
while 34% fully use them (up from 26% last year), and 28% don’t use them
Unsurprisingly, frustration with the lack of automated testing is clear:
at all. Some respondents said they were planning to or are investigating
microservices, while one said, “We’re planning to move to them in the next year
“Automated testing is ignored ‘due to time constraints.'"
or two.”
“Testing? That's an interesting idea.”
But when it comes to Kubernetes, it’s definitely a “What a difference a year
“We intended to do TDD but it usually ends up being after the fact.”
makes” situation: In 2020, only 38% of our survey takers used K8s, as it’s
“I try to write my code with TDD when it's possible; it's complicated when known. This year, 46% use Kubernetes, while 37% do not (down from 50% last
writing React components, or when changing a function that is not tested year).
with many side effects and many inputs and the tech lead forbids (me) to
refactor it at the moment .... ='(.” DevOps teams not yet on board are much closer to actually implementing
Kubernetes this year than last year:

The strongest light at the end of the testing tunnel may be found in the use of “Not yet, but this is very much in-plan for our desired approach.”
artificial intelligence/machine learning. In 2020, just 16% of survey respondents
said they had “bots” testing their code or an AI/ML tool in place for test; this “Planned for 2021.”
year the percentage was just over 41%. All told, 25% of respondents use bots to
“Yes, but not for every workload. ECS, serverless are also used.”
test their code, 16% use AI/ML to review code before a human sees it, and 34%
are exploring the idea of AI/Ml but haven’t done anything about it yet. Exactly “We've tried but now it's just not needed for small production
one-quarter of respondents aren’t using AI/ML in test. environments.”

“We are moving to K8s this year. Woot.”

 8

Low code/no code development tools are also being taken more seriously this The role of the DevOps platform
year. Last year 75% of respondents told us they don’t use them; this year, 41%
This year we asked survey respondents for the first time about DevOps
use low code/no tools, while 59% do not.
platform usage. Slightly over 70% said their teams use a DevOps platform (but
we left it up to survey takers to actually define what a DevOps platform means
HAS YOUR ORGANIZATION ADOPTED MICROSERVICES?
to them). The top benefits of a DevOps platform? Better DevOps, improved
collaboration, easier automation, and visibility/traceability were named as the
37% Partially
biggest benefits. Respondents also offered other ways a DevOps platform was
helping their teams:
28% No

34% Yes
“More ownership of everything to do with the product.”
1% Other

DOES YOUR ORGANIZATION USE KUBERNETES?

37% No “Reduced mean time to recovery (MTTR),

quicker time to market, reduced lead time for
46% Yes
fixes, and fewer change failures.”
14% I don't know

2% Other

“Reliability, repeatability, consistency, productivity.”

DOES YOUR ORGANIZATION USE A LOW CODE OR NO CODE TOOL?

49% Yes

51% No Not surprisingly, the group most likely to use a DevOps platform is the DevOps
team (43%), but 23% said “everyone” in their company uses the platform.
 9

Developers

Development top findings

DevOps = faster releases

Almost 60% of devs are releasing code 2x faster, thanks to DevOps.

Investing in the process

DevOps teams didn’t see small incremental tech changes in 2021
- they added the big guns: SCM, CI/CD, a DevOps platform, and
automated testing.

What’s missing?
More testing of all types and more (and different) code reviews.

Role changing continues

Devs continue to take on jobs that used to be the sole
responsibility of ops.

Future facing
A full 30% of devs think an understanding of AI/ML will be
critical to the next step in their career.
 10

Devs and DevOps WHAT CHANGES HAVE YOU MADE TO YOUR SOFTWARE
DEVELOPMENT PROCESS?
If you feel the need for speed (when it comes to code release), DevOps
continues to be the right choice. Just over 84% of devs told us they’re
21.02% Source code management
releasing code faster than before. About 57% said code is being released
twice as fast (a big jump from last year’s 35%), and 19% said code goes out 17.74% Continuous integration
the door 10x faster.
13.59% Continuous delivery
Why is code being released more quickly? We asked developers what’s
changed in their process. Just over 21% of survey respondents said they’ve 11.65% DevOps platform
added source code management to their DevOps practice (up from 15%
last year), while almost 18% added CI and 13% added CD. Nearly 12% said 10.38% Automated testing

adding a DevOps platform has sped up the process, while just over 10%
5.29% Toolchain integration
have added automated testing.
4.12% Planning tools/methodology

3.95% Cloud Native

3.52% Serverless

2.89% Requirements Management

2.7% Automated security testing/shift left security

2.38% Release Orchestration

0.6% Other
 11

DevOps teams are adding new processes, but there are also some mindset
shifts going on. We asked developers to take a deep dive into what’s really
made it possible for them to release code faster.
“We divide and conquer: Splitting the code into more
modules has helped decrease debug time, improved
Many said the process was a holistic one: stability, and allow a mix and match approach."

“By capturing more errors and bad practices in the development phase
we are able to deploy more often and securely to production.”
“We changed our release plans so versions developed concurrently
“Our team adopted microservices on a new project and then fully
depend less on each other.”
embraced continuous delivery. To get to continuous delivery, we need
to assure quality, so we have automated tests built-in. Investing in these “We evaluated the team and did value stream mapping and finalized
areas allowed our team to deploy 2000 times to production over a year, the desired state. In most of the cases we found the team needs an
where in the past we would deploy maybe 6 times.” automated pipeline for faster delivery and immediate feedback so that
they can act fast rather than later. We also moved security left so that
developers can fix security issues fast. We also made sure developers are
“We are releasing code globally instead of into specific doing code review in a collaborative way though pull requests.”
locations with automated deployments. Principally,
cutting commit-to-live time (by removing batching)
encouraged smaller changes incurring less overhead
(due to removing a coping strategy of increasing scope).”
 12

But automation was also critical:

“We have full automation from dev to production.” “To be honest, continuous delivery helped us the most to
speed up our delivery with our clients.”
“I use automatic semantic versioning to speed up releases.”

“We automate everything possible, to be able to test our product ‘like in

real life’ without any downside. This increases confidence and simplifies
tests for everything.”

“We got rid of manual deployments.” “We automated the building process, code linting, and
“Massively expanding our CI capabilities has decreased deploy time testing of key parts of our project. We put all of these into
significantly.”

“Integration testing has been a big plus in how confident we are to

release automatically and deliver a version. We are now able to deliver
any day.”
“We are not relying on developers to have
“It helps that devs don't need to keep track of test running; they just need
to push and pipeline will check there code before merge to master.” remembered to create and run tests for their
code before deploying.”

“Automation in every possible way.”

 13

All that automation has translated into a huge list of things devs told us
they no longer have to do, including:

“Security and dependency management.”

“Having to manually create backups and do manual “Raw coding.”

testing and deployment.” “Run lint. (It’s auto run when using git commit.)”

“Building containers.”

“Relying on code review to have caught all the test scenarios. We now use
a coverage scanning tool to tell us if we’ve got it all.”

“Waiting.”
“I don’t have to keep written cheat sheets.”
“Extended periods of indecision.”

“Tests on local environments are no longer required.”

“Tedious security red tape reviews, weeks setting up a server, delaying

for change management windows.”

“The dev teams no longer perform production deployments.”

“Testing of back-end code by hand.”
“No more blocking on every dev push.”

“Communicate on individual changes and instead use GitLab as a

platform to collaborate and communicate more effectively.”
 14

What aren’t devs working on that they would like to be? Developer daily life
It’s a long list, starting with way more testing and code review:
In a trend that we saw starting in 2020, developer roles continue to shift, taking
on more responsibility for what were traditionally ops roles. Nearly 26% said
Shift left security Switch to a GitOps operation
they instrument the code they’ve written for production monitoring (up from
model
Dynamic testing, e.g., RAM, just 18% last year), while 38% define and/or create the infrastructure their app
memory leaks, CPU Better planning, writing more runs on. About 13% monitor and respond to that infrastructure.
detailed requirements, involving
More code review (x100) Nearly 45% of survey respondents said they review code weekly, and 22% do
stakeholders more
More testing, more automation it bi-weekly (up from 14% last year). But anecdotally, some developers tell a
Have improved approval different tale about code review on their teams, ranging from not doing it at all,
on commits
processes with stakeholders to conducting code reviews on every single merge request/ticket/pull. Many
Microservices told us they review code daily, or even multiple times a day. Not surprisingly,
AI testing
nearly 60% of developers said code reviews were “very valuable” when it
More open source
DevSecOps platform comes to security and code quality. Code reviews are most likely to be done via
Performance optimizations and an online “chat” service and devs said they far preferred to review code in an
Integrate AI/ML for writing code
accessibility IDE vs. a browser.
TDD, BDD, testing against mocks,
More static analysis earlier to
CI/CD Devs are spending time on code review, but a majority of them aren’t spending
shorten the feedback loop ahead
much time on toolchain integration and maintenance – 41% said they spend
of pull requests Each time a bug is fixed we
less than 10% of their time doing so monthly, while 20% said they spend
should add a regression test
More automated tests. Code between 11% and 20% of their time on those tasks.
systematically
coverage analysis. Intelligent
test subset selection, so More machine learning and pair Who sets dev’s priorities? This year 43% of devs said they set their own
branches fail faster programming priorities (a big change from 2020 when only 24% said this), while 38% said
product managers, and 19% said the business side. When prioritizing work
Reusing code/logic already Tackle very old legacy code
and features, cost of development is the most important priority to developers
deployed for new projects
“Think twice, code once” (43%), followed by developer workload (34%), and product roadmap (31%).
 15

Security Looking to the future

A full 39% of developers feel fully responsible for security in their organizations In a striking change from 2020, 30% of developers told us an understanding of
(up from 28% last year), while 32% said they shared the burden with other AI/ML is the most important skill for their future careers; last year it was 22%
teams. And the optimism about security improving is reflected by developers and was second to soft skills. Soft skills, like communication and collaboration,
too: 75% said they feel their organizations make it possible for them to avoid are still important and they, along with cutting edge programming languages,
breaches. were both cited by 18% of respondents, followed by GitOps at 14%, and IoT/
blockchain at 11%.
How do developers describe their teams efforts to keep things secure?
Anecdotally, survey takers also said they wanted to know more about cloud/
“AI” cloud native, cross-platform development, low code, data science, Python, and
cryptography. But this quote sums it up well:
“Security red and blue teams auditing running applications,
using quality tools and checking dependencies.”
“I see substantial growth in developers as responsible for their platforms.
“Through code reviews and automated testing.” This is the DevOps notion writ large and fast - there is substantial work to
string the world of microservices together, for instance.”
“Risk analysis in the design phase and (you can) avoid the problems
with security issues.”

“Mostly through vulnerability scanning of libraries, container

images and hosts.”

“It’s all up to the developer!” “We run static code analysis in our CI/CD pipeline.
We should do more than that.”

“Code review; monitoring and implementing best practices for

security; frequent backups; strong passwords.” “Some expensive tools are all we need.”
 16

Security

Security top findings

DevSecOps is real
A full 72% of security pros rated their organizations’ security efforts as either
“strong” or “good.” Wishful thinking it might be, but it’s a significant increase
in optimism from a group not necessarily known to be upbeat.

The shift left continues

DevSecOps teams are running more DAST, SAST, container, and
dependency scans than ever before.

Sec and dev are friendlier

But there is still confusion over who “owns” security and the
finger-pointing game is strong.

Facing the future

Security pros feel they need subject matter expertise to excel in their future
careers, but nearly the same percentage said soft skills would be most
important. Also of interest: advanced programming and AI/ML
 17

Security and DevSecOps Roles are changing

Could 2021 be the year that DevSecOps becomes a reality? Perhaps. An Perhaps one reason security pros are so bullish about their teams’ efforts is
unprecedented 72% of security pros reported their organizations’ security the fact that security roles continue to evolve. Nearly 28% said they are now
efforts were either “good” or “strong.” That’s a significant change from last part of a cross-functional team (identical to last year’s results), while 26% are
year when only 59% said the same thing. The largest year over year increase now more focused on compliance (up nearly 4% from 2020) and 24% are more
was in the “strong” category – last year only 19.95% of respondents considered involved in daily tasks/more hands on (down slightly from last year). About 20%
their security posture in that light compared to nearly 33% in 2021. said they didn’t see any changes in their roles, a percentage nearly identical to
last year’s survey.

HOW WOULD YOU RATE YOUR ORGANIZATION’S IN YOUR EXPERIENCE HOW IS THE SECURITY
SECURITY EFFORTS? ROLE CHANGING?

39.52% Good 27.61% I am increasingly part of a cross-functional team focused on security

32.51% Strong 24.42% I am more involved in the day to day/more hands on

20.25% Fair 25.89% I am more compliance-focused

6.37% Poor 19.99% My role is not changing

1.35% Other

“I think security, in most cases, is not a single person's

specialization. Security must be a practice of every member
of the team from the frontend developer to the system
administrator (also non tech roles).”
 18

Shifting left security team members said the same thing and a previously unheard of 37%
said devs actually find between one-quarter and one-half of all bugs.
Another positive sign: Security is also continuing to shift left and at a faster
pace than we’ve seen before. Over 70% of security pros report their teams have
And while 83% of security pros agreed at some level that finding bugs is a
shifted left (i.e., moved security earlier in the development process), up from
developer performance metric, nearly the same percentage (81%) complained it
65% last year.
was difficult to get devs to make bug fixes a priority. In the end, 77% of security
pros agreed at some level that bugs are mostly found by them (and not devs)
Dig in deeper though, and some curious dichotomies surface. Scanning has
after code is merged in a test environment.
certainly increased: Today 53% of developers run SAST scans (a dramatic jump
from last year’s less than 40%) and 44% run DAST scans (up a lot from last year’s
27%). And well over 50% of security pros report their devs scan containers, run
Who’s in charge?
dependency scans, and do license compliance checks. The question of security “ownership” remains a tricky one in nearly every
organization, and that’s particularly true when it comes to the security team.
But while there are more scans run, most results aren’t easily available to Almost 31% told us they (security) were fully responsible for it, but almost 28%
developers. In fact, just 23% of teams put SAST lite scanners in a web IDE, said everyone was responsible. That response was eerily similar to last year’s,
and only 20% pull scan results into a web pipeline report for devs. DAST, and underscores the need for clarity on this subject.
dependency, and container scans fare worse: Only 16% make DAST and
dependency scans easily available and 14% do the same for container scans. IN YOUR ORGANIZATION, WHICH GROUP IS PRIMARILY
These results show the barest improvement over 2020; last year fewer than 19% RESPONSIBLE FOR SECURITY?
of companies put SAST results in a report for devs and less than 14% did so for
DAST. 30.73% Security

For a security shift left to work, devs have to be able to get access to results 27.88% All of the above

while in their IDEs, so this remains a work-in-progress area for sure.

20.91% Developers

Another work-in-progress is the sometimes contentious relationship between 12.26% Operations

security teams and developers. This year’s survey showed the finger pointing
remains in full force, but, surprisingly, at much lower percentages than we’ve 6.88% None of the above
seen in the past. Last year a whopping 93% of security pros said developers
caught 25% or less of the available bugs to be found in existing code (meaning
three-quarters of the bugs were left for sec to find later). This year, just 45% of
 19

About the bugs Looking to the future

Security testing remains a sticking point for team members. It’s happening When it comes to what will help them most in their future careers, security pros
too late in the process (over 42%), and nearly the same percentage said it was were nearly evenly split: 22.95% said subject matter expertise while 22.91%
a struggle to unpack, process, and fix vulnerabilities. Almost 37% said it was said soft skills like communication and collaboration. (In 2020, soft skills were
tough to track the status of the bug fixes, and 33% said it was hard to prioritize the hands down winner; this year’s change may be a reflection of pandemic-
the remediations. Finally, 32% said it was difficult to just find someone to fix enforced collaboration improvements, leaving room for something else to
the problems. focus on.) Almost 21% said advanced programming, while nearly the same
percentage said AI/ML.
When bugs are found, severity level is most important (59%), followed by
category (51%), number of vulnerabilities solved (41%), and time elapsed since
found (37%).
“Security knowledge, pen testing, bug bounties, a good
Modern application development strategies -- including microservices and knowledge of Linux, the technologies used and the
containers -- are increasingly popular, but only roughly half of security pros runtime (containers, servers, etc.).”
reported having processes in place to monitor and protect them. Teams using
monitoring tools most often mentioned Prometheus, Grafana, and AWS Watch.

“Developers have their own observability stack.” “Cloud and serverless architecture.”

“There really isn’t [a process] in place.” “Cloud skills.”

“Our flagship acts like a closed system that accepts very little user input “Malware analysis and threat intelligence, penetration testing.”
and because it runs on top of App Engine the containers are practically
self-securing.” “Automation and DevOps”

The security outlook is a bit brighter when it comes to cloud native and
serverless, however. Last year 64% of respondents said their organizations
had nothing in place to secure cloud native and serverless, but this year 53% of
teams have built it in.
 20

Operations

Operations top findings

DevOps = change
Over 62% report new and different responsibilities because of DevOps.

In the cloud, and not

More than half of ops pros primarily manage cloud services, but almost the
same percentage are firmly planted on the ground and focused on hardware
and infrastructure.

Automation gets real

Almost 19% of ops teams report full automation, while 37% are “mostly”
automated.

In the future...
Ops pros think programming will be the most important skill they can have,
a big change from 2020’s focus on soft skills.
 21

Operations
“I’m a DevOps coach."
The struggle can be real in operations, placed as it is as gatekeeper to past,
present, and future technologies and methodologies. Not surprisingly, roles “I’m a platform engineer.”
continue to evolve rapidly in operations, and ops pros told us DevOps is the
“I’m a Jack of all trades...a lil bit of everything I can get myself into.”
reason.
“DevOps, SRE monitor and make sure the platform works.”
What do those changes look like?

Today, 49% of ops pros see their role as primarily managing hardware and
infrastructure (a big jump from last year’s 42%), while 56% say their first
“Everything from provisioning servers to managing priority is managing cloud services (a four point increase from last year).
people. Most of the stuff in between is building automation
platforms to do the day-to-day work.” They’re also spending more time on compliance than they were in 2020 – last
year over 55% said they spent very little time (10% or less) dealing with audit
and compliance issues, but this year just 36% reported spending so little time.
In fact, 29% now say they spend about one-quarter of their time dealing with
audits and compliance issues.

“I plan the company roadmap for software

development, manage the entire developer team,
Still so many tools
and come up with R&D efforts.” And it can’t be operations without a lot of tools. Nearly 50% of ops pros said
their teams use between two and five monitoring tools (down sharply from
65% last year), while 28% don’t use any monitoring tools at all. All told, 72% of
ops teams use tools that make it easy to feed developers real time data (and
just over 40% of those said it was because their organization used a DevOps
platform).
“Maintain the tools of the DevOps toolchain in
operational condition and continue to improve Logging (capturing and viewing application logs) is the most important
the platform and practices.” monitoring category, followed by metrics.
 22

The top choice for logging is Elasticsearch (28%) followed by Splunk (21%) WHICH TOOLS DO YOU USE FOR APP METRICS (TRACES)?
and Datadog (19%). (Last year, Elasticsearch was used by 38% of ops teams,
while 14% used Splunk and 12% chose Datadog.) When it comes to tracking DataDog
14.51% in 2020 21.02% in 2021
application metrics, 21% use Datadog and nearly 14% use New Relic. Nearly
20%, however, don’t use any tool to keep track of metrics. Prometheus is the None
29.12% in 2020 19.46% in 2021
tool of choice for capturing time-series metrics for the second year in a row:
New Relic
30% of ops teams use it, followed by Datadog at nearly 23%. 13.68% in 2020 13.79% in 2021
Jaeger
WHICH TOOLS DO YOU USE TO VIEW/CAPTURE LOGS? 11.73% in 2020 11.6% in 2021
Zipkin
Elasticsearch 4.94% in 2020 10.67% in 2021
(Elk Stack) 37.51% in 2020 28% in 2021
AppDynamics
Other 6.38% in 2020 10.15% in 2021
7.52% in 2020 NA in 2021
Dynatrace
Splunk 5.56% in 2020 7.86% in 2021
13.64% in 2020 20.58% in 2021
Other
Datadog 14.09% in 2020 5.46% in 2021
11.71% in 2020 19.3% in 2021
None
12.44% in 2020 11.92% in 2021 WHICH TOOLS DO YOU USE TO CAPTURE TIME-SERIES METRICS?
Sumologic
2.95% in 2020 6.83% in 2021 Prometheus
35.66% in 2020 30.06% in 2021
Logz.io
3.5% in 2020 5.99% in 2021 Datadog
12.18% in 2020 22.98% in 2021
Nagios
A majority of ops teams (roughly 34%) use the AWS cloud, while 24% use 12.57% in 2020 12.38% in 2021
Microsoft Azure, and 23% use Google Cloud Platform. Just around 13% said
None
their organization either doesn’t use a public cloud or they don’t know which 16.21% in 2020 12.17% in 2021
public cloud is in use. Azure saw the biggest change from 2020: Last year not Solarwinds
quite 18% provisioned the Microsoft cloud service. 5.4% in 2020 8.97% in 2021
Sensu
3.44% in 2020 8.76% in 2021
Other
14.54% in 2020 4.67% in 2021
 23

Working with development Looking to the future

Over 55% of ops teams told us their software development lifecycle was either The impact of a global pandemic and the challenges of remote work were
completely or mostly automated. Just over 27% said it was partially automated clearly felt in every DevOps role, and ops is no exception. Last year, like their
and 11% said it was just beginning. Almost 6% said there was no automation at counterparts in dev, security, and test, a slim majority of ops pros (31%) said
all. In 2020, just 8% of teams claimed full automation – nearly 19% said it this they believed soft skills like communication and collaboration would be most
year. important to their future careers. This year, 25% said programming would be
most important, followed by soft skills (24%), subject matter expertise (21%),
Some things remain steady in the ops/dev relationship, including the fact that and AI/ML (20%).
nearly 71% of ops pros said it was very or extremely important to them to have
visibility into development, while 79% agreed at some level they get enough They had other thoughts about the future, including:
notice to support dev projects. But in a continuing sign of shifting roles, nearly
77% of ops pros said their devs are able to provision testing environments,
which is an 8% increase from last year. And there is more actual DevSecOps “System administration skills are still a big plus”
happening: Just over 76% of ops teams agree at some level that devs are able
to receive and address security issues during the development process (that’s a
10% jump from last year).

Ops pros feel increasingly responsible for security in their organizations: 28% “Decentralization, ethical architecture and security”
said they were solely responsible (up from 21% last year) but the majority (34%)
believe they are responsible but as part of a bigger team.

“Can't say the world is really changing, all this cloud stuff is ok,
“Integration of diverse platforms or tools” but if you know the basics about computer networking, OSes
- welcome on board! Hard skills are essential of course, but
communication is a real problem sometimes.”

“Scripting, basic hardware and networking, and cloud basics”

 24

Keeping the DevOps momentum

The “gaining momentum” state of DevOps came through loud and clear
from survey respondents when asked about their investment plans for 2021.
Last year, DevOps teams were going to focus on the basics: automation, CI/
CD and DevOps, and apparently it worked, because this year’s priorities
show striking differences.

In 2021 the majority of survey takers will focus their investments on the cloud followed by AI. To put that
in comparison, cloud was the 4th place pick last year and AI was in a distant 8th place. Automation and
DevOps were the third and fourth place picks this year. Anecdotally, however, survey takers also expressed
a lot of interest in machine learning and data science.

It’s clear DevOps teams are doing the real (and hard) work required to move forward. And to end on a good
note, it seems the majority are ready for what’s ahead. Fully 48% of survey takers said they feel “somewhat
prepared” for the future, while 27% said they were “well prepared.” Just 6% said they feel overwhelmed, a
relatively small number considering the events of the last year.

So go forth and share these observations and results with your team, and see how your efforts compare.
DevOps is very much a journey and not a destination; the trick is to keep the momentum going.