Practical Guide to Continuous Delivery, A

Description

• Copyright 2017
• Dimensions: 7" x 9-1/8"
• Pages: 288
• Edition: 1st

• eBook
• ISBN-10: 0-13-469155-5
• ISBN-13: 978-0-13-469155-8

Using Continuous Delivery, you can bring software into production more rapidly, with greater reliability. A Practical Guide to Continuous Delivery is a 100% practical guide to building Continuous Delivery pipelines that automate rollouts, improve reproducibility, and dramatically reduce risk.

Eberhard Wolff introduces a proven Continuous Delivery technology stack, including Docker, Chef, Vagrant, Jenkins, Graphite, the ELK stack, JBehave, and Gatling. He guides you through applying these technologies throughout build, continuous integration, load testing, acceptance testing, and monitoring. Wolff’s start-to-finish example projects offer the basis for your own experimentation, pilot programs, and full-fledged deployments.

A Practical Guide to Continuous Delivery is for everyone who wants to introduce Continuous Delivery, with or without DevOps. For managers, it introduces core processes, requirements, benefits, and technical consequences. Developers, administrators, and architects will gain essential skills for implementing and managing pipelines, and for integrating Continuous Delivery smoothly into software architectures and IT organizations.

• Understand the problems that Continuous Delivery solves, and how it solves them
• Establish an infrastructure for maximum software automation
• Leverage virtualization and Platform as a Service (PAAS) cloud solutions
• Implement build automation and continuous integration with Gradle, Maven, and Jenkins
• Perform static code reviews with SonarQube and repositories to store build artifacts
• Establish automated GUI and textual acceptance testing with behavior-driven design
• Ensure appropriate performance via capacity testing
• Check new features and problems with exploratory testing
• Minimize risk throughout automated production software rollouts
• Gather and analyze metrics and logs with Elasticsearch, Logstash, Kibana (ELK), and Graphite
• Manage the introduction of Continuous Delivery into your enterprise
• Architect software to facilitate Continuous Delivery of new capabilities



Sample Content

Table of Contents

Preface     xvii
Part I:  Foundations     1
Chapter 1:  Continuous Delivery: What and How?     3
1.1 Introduction: What Is Continuous Delivery?     3
1.2 Why Software Releases are So Complicated     3
    1.2.1 Continuous Integration Creates Hope     4
    1.2.2 Slow and Risky Processes     4
    1.2.3 It’s Possible to be Fast     4
1.3 Values of Continuous Delivery     4
    1.3.1 Regularity     5
    1.3.2 Traceability/Confirmability     6
    1.3.3 Regression     6
1.4 Benefits of Continuous Delivery     6
    1.4.1 Continuous Delivery for Time to Market     7
    1.4.2 One Example     7
    1.4.3 Implementing a Feature and Bringing It into Production     7
    1.4.4 On to the Next Feature     7
    1.4.5 Continuous Delivery Generates Competitive Advantages     8
    1.4.6 Without Continuous Delivery     8
    1.4.7 Continuous Delivery and Lean Startup     9
    1.4.8 Effects on the Development Process     9
    1.4.9 Continuous Delivery to Minimize Risk     10
    1.4.10 Faster Feedback and Lean     13
1.5 Generations and Structure of a Continuous Delivery Pipeline     14
    1.5.1 The Example     16
1.6 Conclusion     17
Endnotes     17
Chapter 2:  Providing Infrastructure     19
2.1 Introduction     19
    2.1.1 Infrastructure Automation: An Example     20
2.2 Installation Scripts     21
    2.2.1 Problems of Classical Installation Scripts     21
2.3 Chef     24
    2.3.1 Chef versus Puppet     25
    2.3.2 Other Alternatives     26
    2.3.3 Technical Foundations     26
    2.3.4 Chef Solo     33
    2.3.5 Chef Solo: Conclusion     35
    2.3.6 Knife and Chef Server     35
    2.3.7 Chef Server: Conclusion     39
2.4 Vagrant     40
    2.4.1 An Example with Chef and Vagrant     41
    2.4.2 Vagrant: Conclusion     43
2.5 Docker     43
    2.5.1 Docker’s Solution     44
    2.5.2 Creating Docker Containers     46
    2.5.3 Running the Example Application with Docker     49
    2.5.4 Docker and Vagrant     51
    2.5.5 Docker Machine     53
    2.5.6 Complex Configurations with Docker     55
    2.5.7 Docker Compose     57
2.6 Immutable Server     60
    2.6.1 Disadvantages of Idempotency     60
    2.6.2 Immutable Server and Docker     61
2.7 Infrastructure as Code     61
    2.7.1 Testing Infrastructure as Code     63
2.8 Platform as a Service (PaaS)     63
2.9 Handling Data and Databases     65
    2.9.1 Handling Schemas     66
    2.9.2 Test and Master Data     67
2.10 Conclusion     68
Endnotes     69

Part II:  The Continuous Delivery Pipeline     71
Chapter 3:  Build Automation and Continuous Integration     73
3.1 Introduction     73
    3.1.1 Build Automation: An Example     74
3.2 Build Automation and Build Tools     74
    3.2.1 Build Tools in the Java World     75
    3.2.2 Ant     76
    3.2.3 Maven     76
    3.2.4 Gradle     81
    3.2.5 Additional Build Tools     84
    3.2.6 Choosing the Right Tool     84
3.3 Unit Tests     86
    3.3.1 Writing Good Unit Tests     87
    3.3.2 TDD—Test-Driven Development     89
    3.3.3 Clean Code and Software Craftsmanship     90
3.4 Continuous Integration     91
    3.4.1 Jenkins     92
    3.4.2 Continuous Integration Infrastructure     97
    3.4.3 Conclusion     99
3.5 Measuring Code Quality     101
    3.5.1 SonarQube     102
3.6 Managing Artifacts     105
    3.6.1 Integration into the Build     108
    3.6.2 Advanced Features of Repositories     109
3.7 Conclusion     110
Endnotes     111
Chapter 4:  Acceptance Tests     113
4.1 Introduction     113
    4.1.1 Acceptance Tests: An Example     113
4.2 The Test Pyramid     114
4.3 What Are Acceptance Tests?     117
    4.3.1 Automated Acceptance Tests     117
    4.3.2 More Than Just an Increase in Efficiency     117
    4.3.3 Manual Tests     119
    4.3.4 What about the Customer?     119
    4.3.5 Acceptance versus Unit Tests     119
    4.3.6 Test Environments     120
4.4 GUI-Based Acceptance Tests     121
    4.4.1 Problems of GUI Tests     121
    4.4.2 Abstractions against Fragile GUI Tests     122
    4.4.3 Automation with Selenium     122
    4.4.4 Web Driver API     122
    4.4.5 Tests without Web Browser: HtmlUnit     123
    4.4.6 Selenium Web Driver API     123
    4.4.7 Selenium IDE     123
    4.4.8 Problems with Automated GUI Tests     124
    4.4.9 Executing GUI Tests     125
    4.4.10 Exporting the Tests as Code     125
    4.4.11 Manual Modifications of the Test Cases     125
    4.4.12 Test Data     126
    4.4.13 Page Object     126
4.5 Alternative Tools for GUI Tests     127
    4.5.1 PhantomJS     127
    4.5.2 Windmill     127
4.6 Textual Acceptance Tests     129
    4.6.1 Behavior-Driven Development     129
    4.6.2 Different Adaptors     131
4.7 Alternative Frameworks     133
4.8 Strategies for Acceptance Tests     134
    4.8.1 The Right Tool     134
    4.8.2 Rapid Feedback     135
    4.8.3 Test Coverage     135
4.9 Conclusion     136
Endnotes     137
Chapter 5:  Capacity Tests     139
5.1 Introduction     139
    5.1.1 Capacity Tests: An Example     139
5.2 Capacity Tests—How?     140
    5.2.1 Objectives of Capacity Tests     140
    5.2.2 Data Volumes and Environments     140
    5.2.3 Performance Tests Only at the End of the Implementation?     141
    5.2.4 Capacity Tests = Risk Management     141
    5.2.5 Simulating Users     141
    5.2.6 Documenting Performance Requirements     142
    5.2.7 Hardware for Capacity Tests     142
    5.2.8 Cloud and Virtualization     143
    5.2.9 Minimizing Risk by Continuous Testing     143
    5.2.10 Capacity Tests—Sensible or Not?     144
5.3 Implementing Capacity Tests     145
5.4 Capacity Tests with Gatling     146
    5.4.1 Demo versus Real Life     150
5.5 Alternatives to Gatling     151
    5.5.1 Grinder     152
    5.5.2 Apache JMeter     152
    5.5.3 Tsung     152
    5.5.4 Commercial Solutions     152
5.6 Conclusion     153
Endnotes     153
Chapter 6:  Exploratory Testing     155
6.1 Introduction     155
    6.1.1 Exploratory Tests: An Example     155
6.2 Why Exploratory Tests?     156
    6.2.1 Sometimes Manual Testing Is Still Better     156
    6.2.2 Test by the Customers     156
    6.2.3 Manual Tests for Non-Functional Requirements     157
6.3 How to Go About It?     157
    6.3.1 Missions Guide the Tests     157
    6.3.2 Automated Environment     158
    6.3.3 Showcases as a Basis     158
    6.3.4 Example: An E-Commerce Application     158
    6.3.5 Beta Tests     159
    6.3.6 Session-Based Tests     159
6.4 Conclusion     162
Endnotes     162
Chapter 7:  Deploy—The Rollout in Production     163
7.1 Introduction     163
    7.1.1 Deployment: An Example     164
7.2 Rollout and Rollback     164
    7.2.1 Benefits     164
    7.2.2 Disadvantages     164
7.3 Roll Forward     165
    7.3.1 Benefits     165
    7.3.2 Disadvantages     166
7.4 Blue/Green Deployment     166
    7.4.1 Benefits     166
    7.4.2 Disadvantages     167
7.5 Canary Releasing     168
    7.5.1 Benefits     168
    7.5.2 Disadvantages     169
7.6 Continuous Deployment     169
    7.6.1 Benefits     170
    7.6.2 Disadvantages     171
7.7 Virtualization     171
    7.7.1 Physical Hosts     172
7.8 Beyond Web Applications     172
7.9 Conclusion     174
Endnotes     174
Chapter  8:  Operations     175
8.1 Introduction     175
    8.1.1 Operate—An Example     176
8.2 Challenges in Operations     176
8.3 Log Files     177
    8.3.1 What Should Be Logged?     178
    8.3.2 Tools for Processing Log Files     180
    8.3.3 Logging in the Example Application     181
8.4 Analyzing Logs of the Example Application     182
    8.4.1 Analyses with Kibana     184
    8.4.2 ELK—Scalability     185
8.5 Other Technologies for Logs     189
8.6 Advanced Log Techniques     190
    8.6.1 Anonymization     190
    8.6.2 Performance     191
    8.6.3 Time     191
    8.6.4 Ops Database     191
8.7 Monitoring     191
8.8 Metrics with Graphite     192
8.9 Metrics in the Example Application     194
    8.9.1 Structure of the Example     194
8.10 Other Monitoring Solutions     197
8.11 Additional Challenges When Operating an Application     198
    8.11.1 Scripts     198
    8.11.2 Applications in a Client’s Data Center     198
8.12 Conclusion     199
Endnotes     199

Part III:  Management, Organization, and Architecture for Continuous Delivery     201
Chapter  9 Introducing Continuous Delivery into Your Enterprise     203
9.1 Introduction     203
9.2 Continuous Delivery Right from the Start     203
9.3 Value Stream Mapping     204
    9.3.1 Value Stream Mapping Describes the Sequence of Events     205
    9.3.2 Optimizations     205
9.4 Additional Measures for Optimization     206
    9.4.1 Quality Investments     207
    9.4.2 Costs     207
    9.4.3 Benefits     207
    9.4.4 Do not Check in on a Red Build!     208
    9.4.5 Stop the Line     208
    9.4.6     5 Whys     209
    9.4.7 DevOps     209
9.5 Conclusion     210
Endnotes     211
Chapter  10: Continuous Delivery and DevOps     213
10.1 Introduction     213
10.2 What Is DevOps?     213
    10.2.1 Problems     214
    10.2.2 The Client Perspective     214
    10.2.3 Pioneer: Amazon     215
    10.2.4 DevOps     215
10.3 Continuous Delivery and DevOps     217
    10.3.1 DevOps: More Than Continuous Delivery     217
    10.3.2 Individual Responsibility and Self-Organization     218
    10.3.3 Technology Decisions     218
    10.3.4 Less Central Control     219
    10.3.5 Technology Pluralism     219
    10.3.6 Exchange Between Teams     220
    10.3.7 Architecture     220
10.4 Continuous Delivery without DevOps?     221
    10.4.1 Terminating the Continuous Delivery Pipeline     222
10.5 Conclusion     223
Endnotes     224
Chapter  11: Continuous Delivery, DevOps, and Software Architecture     225
11.1 Introduction     225
11.2 Software Architecture     225
    11.2.1 Why Software Architecture?     226
11.3 Optimizing Architecture for Continuous Delivery     228
    11.3.1 Smaller Deployment Units     229
11.4 Interfaces     230
    11.4.1 Postel’s Law or the Robustness Principle     231
    11.4.2 Design for Failure     231
    11.4.3 State     232
11.5 Databases     233
    11.5.1 Keeping Databases Stable     233
    11.5.2 Database = Component     234
    11.5.3 Views and Stored Procedures     234
    11.5.4 A Database per Component     235
    11.5.5 NoSQL Databases     235
11.6 Microservices     235
    11.6.1 Microservices and Continuous Delivery     236
    11.6.2 Introducing Continuous Delivery with Microservices     237
    11.6.3 Microservices Entail Continuous Delivery     237
    11.6.4 Organization     237
11.7 Handling New Features     238
    11.7.1 Feature Branches     238
    11.7.2 Feature Toggles     238
    11.7.3 Benefits     239
    11.7.4 Use Cases for Feature Toggles     240
    11.7.5 Disadvantages     240
11.8 Conclusion     241
Endnotes     242
Chapter  12: Conclusion: What Are the Benefits?     243
Endnotes     244
Index     245



Updates

Submit Errata

