Effective Software Testing: A developer's guide

Inhaltsverzeichnis des Buches

• front matter
• forewords
• preface
• acknowledgments
• about this book
• Who should read this book
• How this book is organized: A roadmap
• What this book does not cover
• About the code
• liveBook discussion forum
• about the author
• about the cover illustration
• 1 Effective and systematic software testing
• 1.1 Developers who test vs. developers who do not
• 1.2 Effective software testing for developers
• 1.2.1 Effective testing in the development process
• 1.2.2 Effective testing as an iterative process
• 1.2.3 Focusing on development and then on testing
• 1.2.4 The myth of “correctness by design”
• 1.2.5 The cost of testing
• 1.2.6 The meaning of effective and systematic
• 1.2.7 The role of test automation
• 1.3 Principles of software testing (or, why testing is so difficult)
• 1.3.1 Exhaustive testing is impossible
• 1.3.2 Knowing when to stop testing
• 1.3.3 Variability is important (the pesticide paradox)
• 1.3.4 Bugs happen in some places more than others
• 1.3.5 No matter what testing you do, it will never be perfect or enough
• 1.3.6 Context is king
• 1.3.7 Verification is not validation
• 1.4 The testing pyramid, and where we should focus
• 1.4.1 Unit testing
• 1.4.2 Integration testing
• 1.4.3 System testing
• 1.4.4 When to use each test level
• 1.4.5 Why do I favor unit tests?
• 1.4.6 What do I test at the different levels?
• 1.4.7 What if you disagree with the testing pyramid?
• 1.4.8 Will this book help you find all the bugs?
• Exercises
• Summary
• 2 Specification-based testing
• 2.1 The requirements say it all
• 2.1.1 Step 1: Understanding the requirements, inputs, and outputs
• 2.1.2 Step 2: Explore what the program does for various inputs
• 2.1.3 Step 3: Explore possible inputs and outputs, and identify partitions
• 2.1.4 Step 4: Analyze the boundaries
• 2.1.5 Step 5: Devise test cases
• 2.1.6 Step 6: Automate the test cases
• 2.1.7 Step 7: Augment the test suite with creativity and experience
• 2.2 Specification-based testing in a nutshell
• 2.3 Finding bugs with specification testing
• 2.4 Specification-based testing in the real world
• 2.4.1 The process should be iterative, not sequential
• 2.4.2 How far should specification testing go?
• 2.4.3 Partition or boundary? It does not matter!
• 2.4.4 On and off points are enough, but feel free to add in and out points
• 2.4.5 Use variations of the same input to facilitate understanding
• 2.4.6 When the number of combinations explodes, be pragmatic
• 2.4.7 When in doubt, go for the simplest input
• 2.4.8 Pick reasonable values for inputs you do not care about
• 2.4.9 Test for nulls and exceptional cases, but only when it makes sense
• 2.4.10 Go for parameterized tests when tests have the same skeleton
• 2.4.11 Requirements can be of any granularity
• 2.4.12 How does this work with classes and state?
• 2.4.13 The role of experience and creativity
• Exercises
• Summary
• 3 Structural testing and code coverage
• 3.1 Code coverage, the right way
• 3.2 Structural testing in a nutshell
• 3.3 Code coverage criteria
• 3.3.1 Line coverage
• 3.3.2 Branch coverage
• 3.3.3 Condition + branch coverage
• 3.3.4 Path coverage
• 3.4 Complex conditions and the MC/DC coverage criterion
• 3.4.1 An abstract example
• 3.4.2 Creating a test suite that achieves MC/DC
• 3.5 Handling loops and similar constructs
• 3.6 Criteria subsumption, and choosing a criterion
• 3.7 Specification-based and structural testing: A running example
• 3.8 Boundary testing and structural testing
• 3.9 Structural testing alone often is not enough
• 3.10 Structural testing in the real world
• 3.10.1 Why do some people hate code coverage?
• 3.10.2 What does it mean to achieve 100% coverage?
• 3.10.3 What coverage criterion to use
• 3.10.4 MC/DC when expressions are too complex and cannot be simplified
• 3.10.5 Other coverage criteria
• 3.10.6 What should not be covered?
• 3.11 Mutation testing
• Exercises
• Summary
• 4 Designing contracts
• 4.1 Pre-conditions and post-conditions
• 4.1.1 The assert keyword
• 4.1.2 Strong and weak pre- and post-conditions
• 4.2 Invariants
• 4.3 Changing contracts, and the Liskov substitution principle
• 4.3.1 Inheritance and contracts
• 4.4 How is design-by-contract related to testing?
• 4.5 Design-by-contract in the real world
• 4.5.1 Weak or strong pre-conditions?
• 4.5.2 Input validation, contracts, or both?
• 4.5.3 Asserts and exceptions: When to use one or the other
• 4.5.4 Exception or soft return values?
• 4.5.5 When not to use design-by-contract
• 4.5.6 Should we write tests for pre-conditions, post-conditions, and invariants?
• 4.5.7 Tooling support
• Exercises
• Summary
• 5 Property-based testing
• 5.1 Example 1: The passing grade program
• 5.2 Example 2: Testing the unique method
• 5.3 Example 3: Testing the indexOf method
• 5.4 Example 4: Testing the Basket class
• 5.5 Example 5: Creating complex domain objects
• 5.6 Property-based testing in the real world
• 5.6.1 Example-based testing vs. property-based testing
• 5.6.2 Common issues in property-based tests
• 5.6.3 Creativity is key
• Exercises
• Summary
• 6 Test doubles and mocks
• 6.1 Dummies, fakes, stubs, spies, and mocks
• 6.1.1 Dummy objects
• 6.1.2 Fake objects
• 6.1.3 Stubs
• 6.1.4 Mocks
• 6.1.5 Spies
• 6.2 An introduction to mocking frameworks
• 6.2.1 Stubbing dependencies
• 6.2.2 Mocks and expectations
• 6.2.3 Capturing arguments
• 6.2.4 Simulating exceptions
• 6.3 Mocks in the real world
• 6.3.1 The disadvantages of mocking
• 6.3.2 What to mock and what not to mock
• 6.3.3 Date and time wrappers
• 6.3.4 Mocking types you do not own
• 6.3.5 What do others say about mocking?
• Exercises
• Summary
• 7 Designing for testability
• 7.1 Separating infrastructure code from domain code
• 7.2 Dependency injection and controllability
• 7.3 Making your classes and methods observable
• 7.3.1 Example 1: Introducing methods to facilitate assertions
• 7.3.2 Example 2: Observing the behavior of void methods
• 7.4 Dependency via class constructor or value via method parameter?
• 7.5 Designing for testability in the real world
• 7.5.1 The cohesion of the class under test
• 7.5.2 The coupling of the class under test
• 7.5.3 Complex conditions and testability
• 7.5.4 Private methods and testability
• 7.5.5 Static methods, singletons, and testability
• 7.5.6 The Hexagonal Architecture and mocks as a design technique
• 7.5.7 Further reading about designing for testability
• Exercises
• Summary
• 8 Test-driven development
• 8.1 Our first TDD session
• 8.2 Reflecting on our first TDD experience
• 8.3 TDD in the real world
• 8.3.1 To TDD or not to TDD?
• 8.3.2 TDD 100% of the time?
• 8.3.3 Does TDD work for all types of applications and domains?
• 8.3.4 What does the research say about TDD?
• 8.3.5 Other schools of TDD
• 8.3.6 TDD and proper testing
• Exercises
• Summary
• 9 Writing larger tests
• 9.1 When to use larger tests
• 9.1.1 Testing larger components
• 9.1.2 Testing larger components that go beyond our code base
• 9.2 Database and SQL testing
• 9.2.1 What to test in a SQL query
• 9.2.2 Writing automated tests for SQL queries
• 9.2.3 Setting up infrastructure for SQL tests
• 9.2.4 Best practices
• 9.3 System tests
• 9.3.1 An introduction to Selenium
• 9.3.2 Designing page objects
• 9.3.3 Patterns and best practices
• 9.4 Final notes on larger tests
• 9.4.1 How do all the testing techniques fit?
• 9.4.2 Perform cost/benefit analysis
• 9.4.3 Be careful with methods that are covered but not tested
• 9.4.4 Proper code infrastructure is key
• 9.4.5 DSLs and tools for stakeholders to write tests
• 9.4.6 Testing other types of web systems
• Exercises
• Summary
• 10 Test code quality
• 10.1 Principles of maintainable test code
• 10.1.1 Tests should be fast
• 10.1.2 Tests should be cohesive, independent, and isolated
• 10.1.3 Tests should have a reason to exist
• 10.1.4 Tests should be repeatable and not flaky
• 10.1.5 Tests should have strong assertions
• 10.1.6 Tests should break if the behavior changes
• 10.1.7 Tests should have a single and clear reason to fail
• 10.1.8 Tests should be easy to write
• 10.1.9 Tests should be easy to read
• 10.1.10 Tests should be easy to change and evolve
• 10.2 Test smells
• 10.2.1 Excessive duplication
• 10.2.2 Unclear assertions
• 10.2.3 Bad handling of complex or external resources
• 10.2.4 Fixtures that are too general
• 10.2.5 Sensitive assertions
• Exercises
• Summary
• 11 Wrapping up the book
• 11.1 Although the model looks linear, iterations are fundamental
• 11.2 Bug-free software development: Reality or myth?
• 11.3 Involve your final user
• 11.4 Unit testing is hard in practice
• 11.5 Invest in monitoring
• 11.6 What’s next?
• Appendix. Answers to exercises
• References
• index