Register your product to gain access to bonus material or receive a coupon.
Software Build Systems: Principles and Experience
- By Peter Smith
- Published Mar 11, 2011 by Addison-Wesley Professional.
Book
- Sorry, this book is no longer in print.
Description
- Copyright 2011
- Dimensions: 7" x 9-1/8"
- Edition: 1st
- Book
- ISBN-10: 0-321-71728-7
- ISBN-13: 978-0-321-71728-3
“This book represents a thorough and extensive treatment of the software build process including the choices, benefits, and challenges of a well designed build process. I recommend it not only to all software build engineers but to all software developers since a well designed build process is key to an effective software development process.”
—Kevin Bodie, Director Software Development, Pitney Bowes Inc.
“An excellent and detailed explanation of build systems, an important but often overlooked part of software development projects. The discussion of productivity as related to build systems is, alone, well worth the time spent reading this book.”
—John M. Pantone, Objectech Corporation, VP, IT Educator and Course Developer
“Peter Smith provides an interesting and accessible look into the world of software build systems, distilling years of experience and covering virtually every type of tool in the build engineer’s toolbox. Well organized, well written, and very thorough; I would recommend this book to anyone with a build system under their responsibility.”
—Jeff Overbey, Project Co-Lead, Photran
“Software Build Systems teaches how to think about building software. It surveys the tools and techniques for building software products and the ways things go wrong. This book will appeal to those new to build systems as well as experienced build system engineers.”
—Monte Davidoff, Software Development Consultant, Alluvial Software, Inc.
Inadequate build systems can dramatically impact developer productivity. Bad dependencies, false compile errors, failed software images, slow compilation, and time-wasting manual processes are just some of the byproducts of a subpar build system. In Software Build Systems, software productivity expert Peter Smith shows you how to implement build systems that overcome all these problems, so you can deliver reliable software more rapidly, at lower cost.
Smith explains the core principles underlying highly efficient build systems, surveying both system features and usage scenarios. Next, he encapsulates years of experience in creating and maintaining diverse build systems–helping you make well-informed choices about tools and practices, and avoid common traps and pitfalls. Throughout, he shares a wide range of practical examples and lessons from multiple environments, including Java, C++, C, and C#. Coverage includes
• Mastering build system concepts, including source trees, build tools, and compilation tools
• Comparing five leading build tools: GNU Make, Ant, SCons, CMake, and the Eclipse IDE’s integrated build features
• Ensuring accurate dependency checking and efficient incremental compilation
• Using metadata to assist debugging, profiling, and source code documentation
• Packaging software for installation on your target machine
• Best practices for managing complex version-control systems, build machines, and compilation tools
If you’re a developer, this book will illuminate the issues involved in building and maintaining the build system that’s best for your team. If you’re a manager, you’ll discover how to evaluate your team’s build system and improve its effectiveness. And if you’re a build “guru,” you’ll learn how to optimize the performance and scalability of your build system, no matter how demanding your requirements are.
Sample Content
Online Sample Chapter
Table of Contents
PREFACE xxi
Why Do Build Systems Become Complex? xxiii
The True Cost of a Build System xxiv
The Focus of This Book xxvii
Who Should Read This Book? xxviii
How This Book Is Organized xxix
Summary xxxi
ACKNOWLEDGMENTS xxxiii
ABOUT THE AUTHOR xxxv
PART I THE BASICS 1
Chapter 1 BUILD SYSTEM OVERVIEW 3
What Is a Build System? 3
Compiled Languages 4
Interpreted Languages 6
Web-Based Applications 6
Unit Testing 7
Static Analysis 8
Documentation Generation 9
Components of a Build System 10
Version-Control Tools 10
Source and Object Trees 11
Compilation Tools and Build Tools 13
Build Machines 14
Release Packaging and Target Machines 15
The Build Process and Build Description 16
How a Build System Is Used 18
Build-Management Tools 19
Build System Quality 21
Summary 21
Chapter 2 A MAKE-BASED BUILD SYSTEM 23
Calculator Example 23
Creating a Simple Makefile 26
Simplifying the Makefile 28
Additional Build Targets 29
Using a Framework 31
Summary 33
Chapter 3 THE RUNTIME VIEW OF A PROGRAM 35
Executable Programs 36
Native Machine Code 36
Monolithic System Images 37
Full Program Interpretation 38
Interpreted Byte Codes 39
Libraries 40
Static Linking 41
Dynamic Linking 42
Configuration and Data Files 43
Distributed Programs 44
Summary 46
Chapter 4 FILE TYPES AND COMPILATION TOOLS 47
C/C++ 48
Compilation Tools 49
Source Files 50
Assembly Language Files 52
Object Files 53
Executable Programs 56
Static Libraries 57
Dynamic Libraries 58
C++ Compilation 59
Java 60
Compilation Tools 61
Source Files 62
Object Files 63
Executable Programs 65
Libraries 67
C# 68
Compilation Tools 68
Source Files 69
Executable Programs 71
Libraries 74
Other File Types 76
UML-Based Code Generation 77
Graphic Images 79
XML Configuration Files 81
Internationalization and Resource Bundles 81
Summary 82
Chapter 5 SUBTARGETS AND BUILD VARIANTS 83
Building Subtargets 84
Building Different Editions of the Software 86
Specifying the Build Variant 87
Varying the Code 90
Building Different Target Architectures 94
Multiple Compilers 94
Platform-Specific Files/Functions 95
Multiple Object Trees 96
Summary 98
PART II THE BUILD TOOLS 99
Chapter 6 MAKE 107
The GNU Make Programming Language 108
Makefile Rules to Construct the Dependency Graph 109
Makefile Rule Types 110
Makefile Variables 112
Built-In Variables and Rules 114
Data Structures and Functions 116
Understanding Program Flow 119
Further Reading 122
Real-World Build System Scenarios 123
Scenario 1: Source Code in a Single Directory 123
Scenario 2(a): Source Code in Multiple Directories 125
Scenario 2(b): Recursive Make over Multiple Directories 126
Scenario 2(c): Inclusive Make over Multiple Directories 130
Scenario 3: Defining New Compilation Tools 137
Scenario 4: Building with Multiple Variants 138
Scenario 5: Cleaning a Build Tree 140
Scenario 6: Debugging Incorrect Builds 142
Praise and Criticism 144
Praise 144
Criticism 146
Evaluation 148
Similar Tools 149
Berkeley Make 149
NMake 150
ElectricAccelerator and SparkBuild 151
Summary 153
Chapter 7 ANT 155
The Ant Programming Language 156
A Little More Than “Hello World” 157
Defining and Using Targets 158
Ant’s Flow of Control 161
Defining Properties 162
Built-In and Optional Tasks 164
Selecting Multiple Files and Directories 168
Conditions 170
Extending the Ant Language 172
Further Reading 173
Real-World Build System Scenarios 174
Scenario 1: Source Code in a Single Directory 174
Scenario 2(a): Source Code in Multiple Directories 175
Scenario 2(b): Many Directories, with Multiple build.xml Files 175
Scenario 3: Defining New Compilation Tools 179
Scenario 4: Building with Multiple Variants 183
Scenario 5: Cleaning a Build Tree 188
Scenario 6: Debugging Incorrect Builds 188
Praise and Criticism 191
Praise 191
Criticism 191
Evaluation 193
Similar Tools 193
NAnt 194
MSBuild 194
Summary 196
Chapter 8 SCons 197
The SCons Programming Language 198
The Python Programming Language 199
Simple Compiling 202
Managing Build Environments 206
Program Flow and Dependency Analysis 210
Deciding When to Rebuild 212
Extending the Language 214
Other Interesting Features 218
Further Reading 219
Real-World Build System Scenarios 219
Scenario 1: Source Code in a Single Directory 219
Scenario 2(a): Source Code in Multiple Directories 219
Scenario 2(b): Multiple SConstruct Files 220
Scenario 3: Defining New Compilation Tools 222
Scenario 4: Building with Multiple Variants 224
Scenario 5: Cleaning a Build Tree 226
Scenario 6: Debugging Incorrect Builds 226
Praise and Criticism 229
Praise 230
Criticism 231
Evaluation 231
Similar Tools 232
Cons 232
Rake 233
Summary 235
Chapter 9 CMAKE 237
The CMake Programming Language 238
CMake Language Basics 239
Building Executable Programs and Libraries 240
Control Flow 243
Cross-Platform Support 246
Generating a Native Build System 248
Other Interesting Features and Further Reading 254
Real-World Build System Scenarios 255
Scenario 1: Source Code in a Single Directory 255
Scenario 2: Source Code in Multiple Directories 256
Scenario 3: Defining New Compilation Tools 257
Scenario 4: Building with Multiple Variants 259
Scenario 5: Cleaning a Build Tree 260
Scenario 6: Debugging Incorrect Builds 260
Praise and Criticism 261
Praise 261
Criticism 262
Evaluation 262
Similar Build Tools 263
Automake 263
Qmake 264
Summary 264
Chapter 10 ECLIPSE 267
The Eclipse Concepts and GUI 268
Creating Projects 269
Building a Project 276
Running a Project 282
Using the Internal Project Model 285
Other Build Features 286
Further Reading 288
Real-World Build System Scenarios 288
Scenario 1: Source Code in a Single Directory 288
Scenario 2: Source Code in Multiple Directories 290
Scenario 3: Defining New Compilation Tools 291
Scenario 4: Building with Multiple Variants 292
Scenario 5: Cleaning a Build Tree 295
Scenario 6: Debugging Incorrect Builds 296
Praise and Criticism 296
Praise 297
Criticism 297
Evaluation 298
Similar Build Tools 299
CDT for Eclipse, C/C++ Development Tooling 299
Summary 301
PART III ADVANCED TOPICS 303
Chapter 11 DEPENDENCIES 305
The Dependency Graph 307
Incremental Compilation 307
Full, Incremental, and Subtarget Builds 308
The Problem with Bad Dependencies 310
Problem: Missing Dependencies Causing a Runtime Error 310
Problem: Missing Dependencies Causing a Compile Error 311
Problem: Unwanted Dependencies Causing Excess Rebuilding 312
Problem: Unwanted Dependencies Causing Failed Dependency Analysis 312
Problem: Circular Dependencies 313
Problem: Implicit Sequencing As a Substitute for Dependencies 314
Problem: The Clean Target Doesn’t Clean Everything 315
Step 1: Computing the Dependency Graph 315
Gathering Exact Dependencies 316
Caching the Dependency Graph 319
Updating the Cached Dependency Graph 320
Step 2: Determining Which Files Are Out-of-Date 324
Time Stamp-Based Methods 324
Checksum-Based Methods 326
Flag Comparison 328
Advanced Methods 329
Step 3: Sequencing the Compilation Steps 330
Summary 333
Chapter 12 BUILDING WITH METADATA 335
Debugging Support 336
Profiling Support 338
Coverage Support 340
Source Code Documentation 341
Unit Testing 344
Static Analysis 348
Adding Metadata to a Build System 349
Summary 350
Chapter 13 SOFTWARE PACKAGING AND INSTALLATION 351
Archive Files 352
Packaging Scripts 353
Other Archive Formats 356
Improvements 356
Package-Management Tools 359
The RPM Package Manager Format 360
The rpmbuild Process 361
An Example RPM Spec File 363
Creating the RPM File from the Spec File 369
Installing the RPM Example 371
Custom-Built GUI Installation Tools 373
The Nullsoft Scriptable Install System (NSIS) 374
The Installer Script 376
Defining the Pages 379
The License Page 380
Directory Selection 381
The Main Component 381
The Optional Components 383
Defining a Custom Page 385
The Installation Page and the Uninstaller 387
Summary 388
Chapter 14 VERSION MANAGEMENT 391
What Should Be Version-Controlled 392
Build Description Files 393
References to Tools 395
Large Binary Files 400
Source Tree Configurations 401
What Should Not Be in the Source Tree 402
Generated Files in the Source Tree 402
Generated Files Under Version Control 404
Build-Management Scripts 405
Version Numbering 406
Version-Numbering Systems 406
Coordinating and Updating the Version Number 407
Storing and Retrieving the Version Number 410
Summary 411
Chapter 15 BUILD MACHINES 413
Native and Cross-Compilation 414
Native Compilation 414
Cross-Compilation 415
Hybrid Environments 416
Centralized Development Environments 416
Why Build Machines Differ 418
Managing Multiple Build Machines 421
Open-Source Development Environments 424
GNU Autoconf 428
The High-Level Workflow 428
An Autoconf Example 430
Running autoheader and autoconf 434
Running the configure Script on the Build Machine 435
Using the Configuration Information 437
Summary 438
Chapter 16 TOOL MANAGEMENT 441
Rules for Managing Tools 442
Tool Rule #1: Take Notes 442
Tool Rule #2: Use Version Control for the Source Code 443
Tool Rule #3: Periodically Upgrade Tools 444
Tool Rule #4: Use Version Control for the Tool Binaries 445
Breaking the Rules 448
Writing Your Own Compilation Tools 449
Custom-Written Tools with Lex and Yacc 450
Summary 453
PART IV SCALING UP 455
Chapter 17 REDUCING COMPLEXITY FOR END USERS 457
Build Frameworks 458
Developer-Facing Portion of the Build Description 459
Framework Portion of the Build Description 460
Convention over Configuration 461
Maven: An Example Build Tool 462
Reasons to Avoid Supporting Multiple Variants 463
You’ll Have More Variants to Test 463
Source Code Becomes Messy 465
Build Times Can Increase 465
Higher Disk Space Requirements 466
Various Ways to Reduce Complexity 466
Use a Modern Build Tool 466
Automatically Detect Dependencies 467
Keep Generated Files out of the Source Tree 467
Ensure That Cleaning a Build Tree Works Correctly 468
Abort the Build After the First Error 468
Provide Meaningful Error Messages 470
Validate Input Parameters 470
Don’t Overengineer Build Scripts 471
Avoid Using Cryptic Language Features 471
Don’t Use Environment Variables to Control the Build Process 472
Ensure That Release and Debug Builds Are Similar 473
Display the Exact Command Being Executed 474
Version-Control References to Tools 475
Version-Control the Build Instructions 475
Automatically Detect Changes in Compilation Flags 475
Don’t Invoke the Version-Control Tool from the Build System 476
Use Continuous Integration as Often as Possible 476
Standardize on a Single Type of Build Machine 477
Standardize on a Single Compiler 477
Avoid Littering Code with #ifdefs 477
Use Meaningful Symbol Names 478
Remove Stale Code 478
Don’t Duplicate Source Files 479
Use a Consistent Build System 480
Scheduling and Staffing Build System Changes 480
Summary 482
Chapter 18 MANAGING BUILD SIZE 485
The Problem with Monolithic Builds 486
Component-Based Software 488
Advantages of Using Components 491
What Exactly Is a Component? 493
Integrating Components into a Single Product 498
People and Process Management 502
Development Team Structure 503
Component Line-Up Management 505
Managing the Component Cache 507
Coordinating New Software Features 509
Apache Ivy 512
Chapter 19 FASTER BUILDS 515
Measuring Build System Performance 516
Measuring Performance in the Start-Up Phase 516
Measuring Performance in the Compilation Phase 526
Performance-Measurement Tools 531
Fixing the Problem: Improving Performance 534
Build Avoidance: Eliminating Unnecessary Rebuilds 535
Object File Caching 536
Smart Dependencies 539
Other Build-Avoidance Techniques 544
Parallelism 545
Build Clusters/Clouds 546
Parallel Build Tools 546
Limitations of Scalability 547
Reducing Disk Usage 548
Summary 551
REFERENCES 553
INDEX 559