Preface to Improving Software Development Productivity: Effective Leadership and Quantitative Methods in Software Management
From the book
From the book
From the book
Some books are to be tasted,
others to be swallowed,
and some few to be chewed
and digested
Francis Bacon
Of Studies
Productivity is a measure relating a quantity or quality of output to the inputs required to produce it. Productivity is, in one sense, a measure of efficiency and/or quality. Fortunately, software development productivity has been consistently measured as the number of delivered source code statements per staff month over the last 50 years. The statements are the delivered statements, not the statements written during the development. Source code statements are specified in the programming language used by the software developer. Many statements are written and thrown away during the development process. The delivered statements may include multiple languages such as UML and C++. The statements are counted in the language written even though a UML statement may contain the power of 40 C++ statements. The input effort is the work required to produce the individual statement.
My interest in productivity improvement started as early as 1955 while I was an electrical engineering student at Utah State University. I was struggling with a full time course load in a 4-year curriculum, a part time job, an interest in campus politics and living in a fraternity house. Time management, by itself, wasn’t adequate to keep up the pace, so I had to somehow reduce the impact of homework and lab reports (increase productivity). I obviously found a simple and logical way to improve my efficiency. Years later I applied the same approach while simultaneously pursuing a PhD in electrical engineering, teaching full time, running a consulting business, and writing two books. I am in no way an outstanding intellect or student. I was guided by the concept that two heads are much more efficient than one when solving problems. This idea led me into a lifelong interest in organization management.
I began experimenting seriously with management technology and its impact on the software development cost and schedule during the mid-1970s. Early studies led me to believe the people aspects of software development had significant impacts on both software productivity and quality. My first serious experiment with software development teams began in 1975 that yielded a 175% increase in productivity and a nearly 3 order of magnitude decrease in errors. I have been told by several colleagues that the 1975 experiment was the first documented use of pair programming.
Chuck Tonies and I wrote in our 1979 Prentice-Hall text Software Engineering that the software engineer’s value V to an organization is dependent on three attributes of the Effectiveness Formula: communication skills C, management concept awareness M and technical ability T, or
V = C[M(T)]
The Effectiveness Formula is implicitly fundamental to the Agile development approaches that have become popular today. The formula is also fundamental to the quality and productivity improvements in the traditional development approaches as well. People, motivation, and communication are key attributes of all successful projects.
The primary purpose of this text is to explore the concepts that are the primary factors that drive a productive environment, to provide the means of evaluating the effectiveness of an organization’s development environment, and to project the productivity impacts of decisions made by managers during the inception and execution of their software development projects.
The productivity impacts go far beyond those associated with technology changes which have been the norm over the past several decades, and include the more dramatic impacts that are part of the management and communications attributes of the Effectiveness Formula.
We will explore the makeup of the Effectiveness Value ratings and the means of improving these ratings in an organization while simultaneously improving the organization’s productivity, effectiveness and estimating ability, and equally as important, give you a better understanding of the development process and the interaction between management style and the environment, and significantly improve development productivity. Instead of using the environment parameters to calculate development cost and schedule, the metrics can also be used to gauge the impact of any management decision that affects the environment.
The text can help answer questions about the project environment (Dilbert is real.) such as “What does the use of cubicles cost the project?”, or “What is the cost of changing the programming language of choice from Ada to C++?”, “How much will productivity improve by increasing my Capability Maturity Rating (CMMI) from level 3 to level 5?”
A second principal objective of this text is to establish an estimating methodology that can, in the hands of a trained analyst, produce realistic estimates of schedule and resources required for software development under a broad variety of project and environmental conditions.
The obvious, most likely audience for the text are software developers. Software developers include all managers and professionals interested in productivity improvement. The concepts described here apply equally to both agile and traditional software developments. The use of communications, teamwork and the environment are implicit in pair programming as an example of agile development.
Software development is the ideal vehicle to use for explaining the productivity concepts because of the wealth of documentation and historical data spanning over 50 years.
While I was trying to compile a list of the audience who would benefit from the material presented here, I thought back to the first beneficiary, a struggling student trying to complete his college career in electrical engineering possible with the parallel need to work part time. All of the concepts about communications, teamwork, and environment discussed in this text were part of making that career possible. With that in mind I expanded the list of beneficiaries of the ideas in this material to include not just software managers, engineers and programmers, but include those in any discipline (information technology, manufacturing, communication, education, etc.) that involve people and the effective, efficient use of those people. The famous Hawthorne Experiment in the early 20th century applied these concepts to equipment manufacturing with great success. The ideas are truly universal.
Software development is a people-centered process as described by DeMarco and Lister in their book “Peopleware”, not the traditional technology-centered process that has been popular since the 1950s. Although the traditional development process is still the major process used in large organizations today, this work provides a roadmap that can quantitatively support the transition from traditional to modern development styles and environments. Concepts of total quality management (TQM) including the use of software development teams and Theory Y management become obvious management technologies to be considered if more substantial gains in productivity and quality are to be realized. The transitions are not all milk and honey. There are short-term penalties to be paid for any process change, whether it is the latest state-of-the-art CASE tool, a new programming language, CMMI, or a development team approach.
My work in the field of software cost and schedule estimation began in 1978 while supporting a proposal effort for a large space-based software system. I was tasked by the Hughes Aircraft Co. Space & Communications Group (my employer) to develop a simulation model that would be the basis of their software development estimates. I derived a mathematic model that produced realistic estimates through the use of environment parameters including both developer capability and project-imposed constraints. The completed software development project data accumulated from multiple sources that date from as far back as 19601 until today helped to create the mathematic model, and for the organization Capability Calculator used in this text. It is amazing, maybe even frightening, that the model formulated in 1980 still produces realistic effort and schedule estimates independent of the development approaches today.
Much of this text explores this simple concept and its importance to software development productivity. Simply stated, the productivity of a development environment and the resulting software development cost and schedule are driven by the product of three important attributes: communication, management, and technology. Although this text focuses on software, the principles can also be applied to other fields without modification.
Improving Software Development Productivity: Effective Leadership and Quantitative Methods in Software Management is comprised of two major parts: Effective Leadership and Quantitative Methods.
Effective Leadership
The first seven chapters focus on effective leadership and measurement of organization capability.
• Chapter 1, Software Development Issues. This chapter discusses the “Software Crisis” and the technology approaches that have been used for the 40+ years since the 1960s to resolve the crisis issues. Productivity has improved very slowly from 1970 to the present in spite of large improvements in tools, languages and development approaches.
• Chapter 2, The Effectiveness Formula. This chapter discusses the importance of the three attributes of the Effectiveness Formula: communications, management, and technology in productivity improvement. The mechanics of effective communications, and some important culture issues that impede effective development improvement are also discussed.
• Chapter 3, Importance of Software Management. This chapter discusses two people management principles in effective development: the Hawthorne Effect, and Theory X/Theory Y management principles. These concepts are the basis of modern people management. The relationship between Agile software development and these principles are discussed as an example.
• Chapter 4, What We Learn From History. This chapter describes what we have learned from history about software development productivity, technology productivity contributions, the productivity impact of CMMI, and the result of optimistic pricing and scheduling.
• Chapter 5, Software Development Teams. Software development teams are explored this chapter. The good, the bad, and the ugly and their impacts on development productivity are discussed.
• Chapter 6, Measuring Organization Capability. A process for measuring software development productivity is presented that can be applied to your development organization to reveal its inherent capability, basic technology constant, and the relative standing of your organization with respect or industry. A tool to support the evaluation is contained in the chapter.
• Chapter 7, Faultless Software Corporation Case Study. This chapter presents a case study in capability assessment that summarizes the material in the first section of the text.
Quantitative Software Development Management
The remainder of the text discusses the application of quantitative management in the delivery of software products within cost and schedule constraints.
• Chapter 8, Product Complexity. This chapter discusses the impact of software system complexity on software development cost and schedule constraints.
• Chapter 9, Staffing Profiles. This chapter presents an evaluation of optimum development staffing profiles for software development. Optimum staffing is a function of product complexity and independent of development approach.
• Chapter 10, Seer Software Model Introduction. This chapter presents an introduction to the Jensen software model used for quantitative development management (the basis for the SEER and Sage estimating tools).
• Chapter 11, Development Environment. This chapter discusses the impact of the development environment on product cost and schedule. The environment evaluation includes considerations related to experience, volatility and other management constraints.
• Chapter 12, Product Characteristics. This chapter discusses the impact of constraints imposed by the product characteristics and requirements on productivity and the software development environment.
• Chapter 13, Development Schedule and Cost Constraints. This chapter uses the Faultless Software Corporation cast study to describe the estimation process on the software development cost and schedule. The estimates are limited by the constraints imposed by the organization capability and environment constraints.
• Chapter 14, Effective Size Estimation. Effective size is not simply an estimate of the number of new and modified software lines of code. This chapter explores the effective size needed to predict the estimate development cost and schedule.
• Chapter 15, Function Point Sizing. This chapter contains an introduction to function point sizing of traditional software products. The use of object points is also discussed as an alternate sizing method for object-based development.
• Chapter 16, Maintenance Estimating. Maintenance estimating is more than simple software enhancements estimates. The impact of knowledge retention on software maintenance is discussed as a major addition to the for software product support.
• Chapter 17, Summary. This chapter reviews the information presented in the first 16 chapters and applies the important concepts to non-software development environments.
• Appendix A, Software Estimating Models. This appendix discusses the evolution of quantitative software estimating models and the capabilities of each of the major estimating approaches.
• Appendix B, Additional Reading. This appendix contains a broad list of additional reading that covers the effective leadership topics (communication, people management styles and issues, and technology) as well as quantitative management and estimating topics.
• Appendix C, Terminology.
• Capability Calculator Access. The Capability Calculator spreadsheet used throughout the text to support the organization capability calculations and software effort and schedule estimates can be freely downloaded from the Prentice-Hall website for the book, www.informit.com/9780133562675, just click on the “Downloads” tab.
Acknowledgements
There are several individuals that I wish to acknowledge in the development of this material. First, and foremost is the contributions of Chuck Tonies and Ken Hubbard of Hughes Aircraft’s Space and Communication Group in supporting me in the development of the Jensen Model and encouraging me to pursue the experiments that became the foundation of the management concepts discussed here. I cannot forget the support of Frank Wolfe, Project Manager for his active support in the estimating activity and Don Forster, SCG Group Executive for granting me ownership of the estimating technology and encouraging me to commercialize the Jensen model and launching an estimating consulting career. Another key person who worked with me though the years in pursuing the Jensen model and many estimating ventures, Suzanne Lucas, a long term friend and colleague.
There are three engineers from the USAF Software Technology Support Center at Hill AFB who were part of the software estimating team and contributors and valued collaborators for all the estimating, training and technology extensions during my time at STSC: Les Dupaix, Mark Woolsey, Thom Rogers and Brent Baxter.
Finally I will not forget my wife of many years and many textbooks, Marge, for all her support in this endeavor. This one only took about 25 years to finish.
—Randall W. Jensen
1. Wolverton, R. W., “The Cost of Developing Large-Scale Software,” IEEE Transactions on Computers, June 1974