Grokking Continuous Delivery

Inhaltsverzeichnis des Buches

• contents
• front matter
• forewords
• preface
• acknowledgments
• about this book
• Who should read this book
• How this book is organized: A road map
• liveBook discussion forum
• About the author
• Part 1. Introducing continuous delivery
• 1 Welcome to Grokking Continuous Delivery
• Do you need continuous delivery?
• Why continuous delivery?
• Continuous delivery
• Integration
• Continuous integration
• What do we deliver?
• Delivery
• Continuous delivery/deployment
• Elements of continuous delivery
• Conclusion
• Summary
• Up next . . .
• 2 A basic pipeline
• Cat Picture Website
• Cat Picture Website source code
• Cat Picture Website pipelines
• What’s a pipeline? What’s a task?
• The basic tasks in a CD pipeline
• Gates and transformations
• CD: Gates and transformations
• Cat Picture Website service pipeline
• Running the pipeline
• Running once a day
• Trying continuous integration
• Using notifications
• Scaling manual effort
• Automation with webhooks
• Scaling with webhooks
• Don’t push changes when broken
• Cat Picture Website CD
• What’s in a name?
• Conclusion
• Summary
• Up next . . .
• Part 2. Keeping software in a deliverable state at all times
• 3 Version control is the only way to roll
• Sasha and Sarah’s start-up
• All kinds of data
• Source and software
• Repositories and versions
• Continuous delivery and version control
• Git and GitHub
• An initial commit—with a bug!
• Breaking main
• Pushing and pulling
• Are we doing continuous delivery?
• Keep version control releasable
• Trigger on changes to version control
• Triggering the User service pipeline
• Building the User service
• The User service in the cloud
• Connecting to the RandomCloud database
• Managing the User service
• The User service outage
• Outsmarted by automation
• What’s the source of truth?
• User service config as code
• Configuring Deployaker
• Config as code
• Rolling out software and config changes
• Conclusion
• Summary
• Up next . . .
• 4 Use linting effectively
• Becky and Super Game Console
• Linting to the rescue!
• The lowdown on linting
• The tale of Pylint and many, many issues
• Legacy code: Using a systematic approach
• Step 1: Configure against coding standards
• Step 2: Establish a baseline
• Step 3: Enforce at submission time
• Adding enforcement to the pipeline
• Step 4: Divide and conquer
• Isolation: Not everything should be fixed
• Enforcing isolation
• Not all problems are created equal
• Types of linting issues
• Bugs first, style later
• Jumping through the hoops
• Conclusion
• Summary
• Up next . . .
• 5 Dealing with noisy tests
• Continuous delivery and tests
• Ice Cream for All outage
• Signal vs. noise
• Noisy successes
• How failures become noise
• Going from noise to signal
• Getting to green
• Another outage!
• Passing tests can be noisy
• Fixing test failures
• Ways of failing: Flakes
• Reacting to failures
• Fixing the test: Change the code or the test?
• The dangers of retries
• Retrying revisited
• Why do we retry?
• Get to green and stay green
• Conclusion
• Summary
• Up next . . .
• 6 Speeding up slow test suites
• Dog Picture Website
• When simple is too simple
• New engineer tries to submit code
• Tests and continuous delivery
• Diagnosis: Too slow
• The test pyramid
• Fast tests first
• Two pipelines
• Getting the right balance
• Changing the pyramid
• Safely adjusting tests
• Test Coverage
• Enforcing test coverage
• Test coverage in the pipeline
• Moving tests in the pyramid with coverage
• What to move down the pyramid?
• Legacy tests and FUD
• Running tests in parallel
• When can tests run in parallel?
• Updating the pipelines
• Still too slow!
• Test sharding, aka parallel++
• How to shard
• More complex sharding
• Sharded pipeline
• Sharding the browser tests
• Sharding in the pipeline
• Dog Picture Website’s pipelines
• Conclusion
• Summary
• Up next . . .
• 7 Give the right signals at the right times
• CoinExCompare
• Life cycle of a change
• CI only before merge
• Timeline of a change’s bugs
• CI only before merging misses bugs
• A tale of two graphs: Default to seven days
• A tale of two graphs: Default to 30 days
• Conflicts aren’t always caught
• What about the unit tests?
• PR triggering still lets bugs sneak in
• CI before AND after merge
• Option 1: Run CI periodically
• Option 1: Setting up periodic CI
• Option 2: Require branches to be up-to-date
• Option 2: At what cost?
• Option 3: Automated merge CI
• Option 3: Running CI with the latest main
• Option 3: Merge events
• Option 3: Merge queues
• Option 3: Merge queue for CoinExCompare
• Where can bugs still happen?
• Flakes and PR-triggered CI
• Catching flakes with periodic tests
• Bugs and building
• CI vs. build and deploy
• Build and deploy with the same logic
• Improved CI pipeline with building
• Timeline of a change revisited
• Conclusion
• Summary
• Up next . . .
• Part 3. Making delivery easy
• 8 Easy delivery starts with version control
• Meanwhile at Watch Me Watch
• The DORA metrics
• Velocity at Watch Me Watch
• Lead time for changes
• Watch Me Watch and elite performers
• Increasing velocity at Watch Me Watch
• Integrating with AllCatsAllTheTime
• Incremental feature delivery
• Committing skipped tests
• Code review and “incomplete” code
• Keeping up the momentum
• Committing work-in-progress code
• Reviewing work-in-progress code
• Meanwhile, back at the end-to-end tests
• Seeing the benefits
• Decreasing lead time for changes
• Continuing AllCatsAllTheTime
• Deployment windows and code freezes
• Increased velocity
• Conclusion
• Summary
• Up next . . .
• 9 Building securely and reliably
• Top Dog Maps
• When the build process is a doc
• Attributes of secure and reliable builds
• Always releasable
• Automated builds
• Build as code
• Use a CD service
• Ephemeral build environments
• Miguel’s plan
• From a doc to a script in version control
• Automated containerized builds
• Secure and reliable build process
• Interface changes and bugs
• When builds cause bugs
• Builds and communication
• Semantic versioning
• The importance of being versioned
• Another outage!
• Build-time dependency bugs
• Pinning dependencies
• Version pinning alone isn’t a guarantee
• Pinning to hashes
• Conclusion
• Summary
• Up next . . .
• 10 Deploying confidently
• Plenty of deployment woes
• DORA metrics for stability
• DORA metrics at Plenty of Woofs
• Deploying less frequently?
• Deploying more frequently?
• Daily deployments vs. outages
• Steps toward increasing frequency
• Fixing problems with the process
• Rolling updates
• Fixing a bug with a rolling update
• Rollbacks
• Rollback strategy = immediate improvement
• Rollback policy in action
• Blue-green deployments
• Faster time to restore with blue-green
• Faster and more stable with canaries
• Requirements for canary deployments
• Canary deployments with baselines
• Time to restore with canary deployments
• Increasing deployment frequency
• DORA metrics with daily canary deployments
• Continuous deployment
• When to use continuous deployment
• Mandatory QA phases
• QA and continuous deployment
• Elite DORA performance
• Conclusion
• Summary
• Up next . . .
• Part 4. CD design
• 11 Starter packs: From zero to CD
• Starter packs: Overview
• Recap: Universal CD pipeline tasks
• Prototypical release pipeline
• Prototypical CI pipeline
• Both pipelines with triggering
• Greenfield: Getting to CD
• Gulpy
• Greenfield: Zero to CD
• First step: Does it build?
• Picking a CD system
• Setting up the initial automation
• State of the code: Linting
• State of the code: Unit tests
• State of the code: Coverage
• Moving past CI: Publishing
• Deployment
• Expanding testing
• Tasks for integration and end-to-end tests
• Completing the CI pipeline
• Gulpy’s complete pipelines
• Legacy: Getting to CD
• Rebellious Hamster
• First step: Prioritize incremental goals
• Focusing on the pain first
• Pain at Rebellious Hamster
• Know when things are broken
• Isolate and add tests
• Legacy pipeline with more tests
• Make deployments more automated
• Creating a release pipeline
• Rebellious Hamster’s release pipeline
• Rebellious Hamster’s complete pipelines
• Conclusion
• Summary
• Up next . . .
• 12 Scripts are code, too
• Purrfect Bank
• CD problems
• Purrfect Bank CD overview
• Payment Org’s bash libraries
• Transaction service pipeline
• Evolving from one big script
• Principles of well-designed tasks
• Breaking up the giant task
• Updated Transaction service pipeline
• Debugging bash libraries
• Investigating the bash library bug
• Why was the bug introduced?
• What is bash for?
• When is bash less good?
• Shell scripting vs. general-purpose languages
• Shell script to general-purpose language
• Migration plan
• From bash library to task with bash
• Reusable bash inside a task
• From bash to Python
• Tasks as code
• CD scripts are code, too
• Conclusion
• Summary
• Up next . . .
• 13 Pipeline design
• PetMatch
• Matchmaking CD pipelines
• CD pipeline problems
• End-to-end test pipeline
• End-to-end test pipeline and errors
• Finally behavior
• Finally as a graph
• Finally in the matchmaking pipeline
• End-to-end test pipeline and speed
• Parallel execution of tasks
• End-to-end test pipeline and test speed
• Parallel execution and test sharding
• End-to-end test pipeline with sharding
• End-to-end test pipeline and signal
• One CI pipeline
• Release pipeline and signal
• Differences in CI
• Combining pipelines
• Release pipeline
• Hardcoding in the release pipeline
• Pipeline reuse with parameterization
• Using reusable pipelines
• Updated pipelines
• Solving PetMatch’s CD problems
• CD features to look for
• Conclusion
• Summary
• Up next . . .
• Appendices
• Appendix A. CD systems
• Appendix B. Version control systems