Software [In]security: What Works in Software Security
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Fifteen Common Activities from BSIMM2
The BSIMM (pronounced "bee simm") project was released one year ago as an in-depth, observation-based study of nine firms' software security initiatives. Part of what makes BSIMM interesting is its basis in actual data from real software security initiatives. Since its release, the study has more than tripled in size and now includes data from 30 firms. For more about the BSIMM, see:
- The Building Security In Maturity Model (BSIMM) (March 16, 2009)
- Cargo Cult Computer Security (January 28, 2010)
Now that BSIMM includes thirty firms, we are undertaking statistical analysis of the data (which we intend to publish shortly). In the meantime, the data show that of the 110 software security activities identified and described in the BSIMM, there are 15 that are very commonly observed. These common activities are worth careful consideration.
One Hundred and Ten Activities
One exceptionally easy way of analyzing the BSIMM data is to consider the 110 activities at once, keeping a count of the number of firms where a given activity is observed. This indicates the popularity of an activity (if nothing else). The chart below can be used as the basis for a BSIMM score card (which compares observations drawn from a target firm against these data):
)
In the score card above, we indicate those activities that are observed in 20 or more firms (of 30) by highlighting the activity. Decoding the BSIMM tags in the chart above requires a copy of the BSIMM which you can obtain for free on the BSIMM website.
Please note that just because an activity is common, it is not necessarily something you should adopt in your organization. The BSIMM simply describes what various leading firms are doing for software security, not about what you should do. That is, the BSIMM is a descriptive model, not a prescriptive model. Then again, the data should be very useful in devising your own strategic approach to software security.
Fifteen Common Activities
Without further ado, here are the fifteen most common BSIMM activities in order of how commonly they are observed.
[SE 1.2] Ensure host and network security basics are in place. The organization provides a solid foundation for software by ensuring that host and network security basics are in place. It is common for operations security teams to be responsible for duties such as patching operating systems and maintaining firewalls. (100%: 30 of 30 firms)
Take home lesson here...the firms we have observed did not start with software security until they had their network security ducks in a row.
[SFD 1.1] Build/publish security features (authentication, role management, key management, audit/log, crypto, protocols). Some problems are best solved only once. Rather than have each project team implement all of their own security features, the SSG provides proactive guidance by building and publishing security features for other groups to use. Project teams benefit from implementations that come pre-approved by the SSG, and the SSG benefits by not having to repeatedly track down the kinds of subtle errors that creep into features such as authentication, role management, audit/logging, key management, and cryptography. (97%: 29 of 30 firms)
Even though security is not thing, but rather a property, security features appear to be a natural way for developers to start.
[PT 1.1] Use external penetration testers to find problems. Many organizations are not willing to address software security until there is unmistakable evidence that the organization is not somehow magically immune to the problem. If security has not been a priority, external penetration testers demonstrate that the organization's code needs help. Penetration testers could be brought in to break a high-profile application in order to make the point. (93%: 28 of 30 firms)
Breaking things works, and it has sex appeal. But watch out for making enemies. Nobody likes it when you call their baby ugly.
[CP 1.3] Create policy. The SSG guides the rest of the organization by creating or contributing to policy that satisfies regulatory requirements and customer-driven security requirements. The policy provides a unified approach for satisfying the (potentially lengthy) list of external security drivers. As a result, project teams can avoid learning the details involved in complying with all applicable regulations. Likewise, project teams don't need to re-learn customer security requirements on their own. The SSG policy documents are sometimes focused around major compliance topics such as the handling of personally identifiable information or the use of cryptography. (87%: 26 of 30 firms)
For whatever reason, security people are enamored with policy.
[SM 1.4] Identify gate locations, gather necessary artifacts. The software security process will eventually involve release gates at one or more points in the software development lifecycle (SDLC) or SDLCs. The first two steps toward establishing these release gates is to identify gate locations that are compatible with existing development practices and to begin gathering the input necessary for making a go/no go decision. Importantly at this stage, the gates are not enforced. For example, the SSG can collect security testing results for each project prior to release, but stop short of passing judgment on what constitutes sufficient testing or acceptable test results. (80%: 24 of 30 firms)
Don't interpose on a process until you understand what will happen when you do so. Turning on a gate too early can bring production to a halt and cause a huge backlog/bottleneck.
[CP 1.1] Know all regulatory pressures and unify approach. If the business is subject to regulatory or compliance drivers such as FFIEC, GLBA, OCC, PCI DSS, SOX, SAS 70, HIPAA or others, the SSG acts as a focal point for understanding the constraints such drivers impose on software. The SSG creates a unified approach that removes redundancy from overlapping compliance requirements. A formal approach will map applicable portions of regulations to control statements explaining how the organization will comply. (80%: 24 of 30 firms)
Killing many birds with one stone is important since security is already seen as a tax on the organization.
[CP 1.2] Identify PII obligations. The way software handles personally identifiable information (PII) could well be explicitly regulated, but even if it is not, privacy is a hot topic. The SSG takes a lead role in identifying PII obligations stemming from regulation, customer demand, and consumer expectations. It uses this information to promote best practices related to privacy. For example, if the organization processes credit card transactions, the SSG will identify the constraints that PCI DSS places on the handling of cardholder data. (80%: 24 of 30 firms)
Privacy is important. People who build systems do well to consider privacy while they are dreaming up and implementing systems.
[T 1.1] Provide awareness training. The SSG provides awareness training in order to promote a culture of security throughout the organization. Training might be delivered by members of the SSG, by an outside firm, by the internal training organization, or through a computer-based training system. Course content is not necessarily tailored for a specific audience. For example, all programmers, quality assurance engineers, and project managers could attend the same Introduction to Software Security course. (80%: 24 of 30 firms)
People are an essential part of software security.
[SR 1.1] Create security standards. Software security requires much more than security features, but security features are part of the job as well. The SSG meets the organization's demand for security features by creating standards that explain the accepted way to adhere to policy and carry out specific security-centric operations. A standard might describe how to perform authentication using J2EE or how to determine the authenticity of a software update. (See [SFD1.1] Build and publish security features for one case where the SSG provides a reference implementation of a security standard.) (73%: 22 of 30 firms)
The notion of acting as an advocate instead of an adversary is common among software security initiatives. Creating standards and helping dev groups get their work done (properly) engenders good will. This can counter the "we just break things" problem.
[AA 1.1] Perform security feature review. To get started with architecture analysis, center the analysis process on a review of security features. Reviewers first identify the security features in an application (authentication, access control, use of cryptography, etc.) then study the design looking for problems that would cause these features to fail at their purpose or otherwise prove insufficient. At higher levels of maturity this activity is eclipsed by a more thorough approach to architecture analysis not centered on features. (73%: 22 of 30 firms)
Architectural analysis (sometimes called threat modelling) is hard. Getting started by over-focusing on security features is common.
[CMVM 1.2] Identify software defects found in operations monitoring and feed them back to development. Defects identified through operations monitoring are fed back to development and used to change developer behavior. The contents of production logs can be revealing (or can reveal the need for improved logging). (73%: 22 of 30 firms)
Simply pointing out problems does not miraculously solve them. Too many security people are satisfied with breaking things. A constructive approach that gets around to fixing things is better.
[ST 1.1] Ensure QA supports edge/boundary value condition testing. The QA team goes beyond functional testing to perform basic adversarial tests. They probe simple edge cases and boundary conditions. No attacker skills required. (70%: 21 of 30 firms)
Thinking like a bad guy is an uncommon skill (and may well be impossible to teach). Start simple.
[CMVM 1.1] Create or interface with incident response. The SSG is prepared to respond to an incident. The group either creates its own incident response capability or interfaces with the organization's existing incident response team. A regular meeting between the SSG and the incident response team can keep information flowing in both directions. (70%: 21 of 30 firms)
Response is an important part of security. Systems will break. Be prepared to handle that. Firms that close the defect loop and get back to root causes practice better software security.
[AM 1.2] Create data classification scheme and inventory. The organization agrees upon a data classification scheme and uses the scheme to inventory its software according to the kinds of data the software handles. This allows applications to be prioritized by their data classification. Many classification schemes are possible — one approach is to focus on PII. Depending upon the scheme and the software involved, it could be easiest to first classify data repositories, then derive classifications for applications according to the repositories they use. (67%: 20 of 30 firms)
Information assets must be identified before they can be properly protected.
[CR 2.1] Use automated tools along with manual review. Incorporate static analysis into the code review process in order to make code review more efficient and more consistent. The automation does not replace human judgment, but it does bring definition to the review process and security expertise to reviewers who are not security experts. (67%: 20 of 30 firms)
BSIMM activities come in three levels. This is the only level two activity on the list of the fifteen most common activities. This may indicate that marketing and sales of static analysis tools is incredibly effective, or it may indicate that many companies believe that they can solve the software security problem by throwing a tool at it. Or maybe code review is just a good idea.
The Rest of the Story
We've only scratched the surface of the BSIMM by covering 15 of 110 activities. Our analysis continues, so stay tuned for much more.