Jimmy Xu, Director of DevSecOps & Cloud Security at Trace3, recently presented “Mastering DevSecOps.” Jimmy’s presentation generated lots of great questions, which he graciously answered below. If you missed his presentation or would like to watch it again, you can view it below.
With small Security and Dev teams, how do you manage segregation of duties?
This is a great question, and a common one. If done correctly, this is where the value of DevOps comes in. Consider these two options:
1. Leverage existing source code management platforms such as GitHub to enforce manual code review and approval before allowing new code to be merged into the main branch. You could enforce this as a mandatory gate as part of the code merge Pull Request, along with the requirement that code must pass all the build checks. I’ve seen the use of 1 to 2 mandatory approvers (another peer or manager on the dev team) but this depends upon the size of your dev team. If you leverage Infrastructure as Code (IaC), you could do the same when a developer makes any changes to infrastructure configuration.
2. Adopt a GitOps model for pull-based deployments as it eliminates any need for human action. It also eliminates the need for any developers or ops folks to access or make any changes in production as all changes are done through the pipeline. One person checks the change, separate approvers review it, then a machine deploys the change in production. The only caveat to this approach is that your organization must have sufficient DevOps maturity and embrace IaC, containers, and Kubernetes. We can help you get there.
What are your recommendations or experiences with static code analysis (own code) and automated software composition analysis (3rd party scans) as part of the deployment pipeline?
This is a big question and may be better answered offline. In general, this space has exploded in the past few years. There are plenty of open source tools for SAST, SCA, DAST, etc. and some Dev & cloud platform vendors offer native capabilities as well. There are also commercial vendors. Generally speaking, these tools are unique and should be used in your pipeline. Which ones to use, where to use, and how to use them depend on your pipeline setup, types of apps you have, the programming languages they must support, etc. It’s imperative however that you select the right tool and use it correctly.
Two common issues I see:
1. False positives: Open source tools, and SAST tools in particular, contain false positives that must be accounted for. Some SAST tools could be tuned, but that’s a lot of work. You could use them early to identify bugs, but be prepared to triage the findings and eliminate false positives. I don’t recommend you use SAST tools as the source of truth and fail the build unless you have successfully worked out the issues and have the tools tuned to an acceptable state. If possible, I recommend using IAST for a more accurate result.
2. False negatives: Open source tools and newly created tools by Dev and cloud platforms have issues with false negatives. Developers tend to use these tools and think they’re covered, but it creates a false sense of security. While I do not recommend restricting what tools developers should use, I do recommend good commercial tools for security reasons.
SCA tools are much easier to deal with. You should use them, use them early, and use them throughout the deployment process. Some tools now have browser plugins that could identify and flag vulnerable code libraries when developers are researching open source components. Most SCA tools also integrate with IDE for early identification, as well as code repo integration, way before the build phase. Take advantage of that. Commercial SCA vendors are also developing lots of new, cool features.
Feel free to follow up with me for specific recommendations as I know the vendor landscape very well and can answer specific questions.
If your organization is still immature at DevOps, would you recommend initiating DevSecOps or wait until we reach some level of maturity?
You should initiate DevSecOps concurrently, even if DevOps is immature, to ensure that security is aligned with your DevOps efforts from the outset. If your teams are tightly aligned, you could leverage each other’s strengths and resources to become strong force multipliers and help each other scale. You may in fact realize that security and DevOps share similar goals. We are currently working on a DevSecOps transformation project where both Security & DevOps leadership are in their early phase of building out a program. We have helped them develop this program from the ground up but a big part of that success is having both DevOps and Security leadership on the steering committee and closely aligned.
Which should I prioritize: Dev or Ops?
This really depends on the organization and the organization’s culture. We often see DevOps initiatives come from Ops. Nevertheless, if you tightly align security with Ops early, it could be a great force multiplier. As an example, the things Ops cares about, such as observability and consistency, greatly benefit security as well. If you can achieve some security observability by partnering with Ops (like seeing real world attacks against the app, or identifying gaps in cloud environments as a result of dev team misconfigurations), it may be easier to use real data to have conversations with Dev.
There are alternatives. We’ve worked with app teams who are open to learning and improving security (assuming there are no historical frictions and the conversations are conducted correctly) and are actively looking for recommendations on how to do it better. When this occurs, it could be an indication that things are moving in the right direction and that developers do care about code quality – and that includes security.