Code Contracts and Pex presentation give to Toronto Visual Basic User Group

1. Putting a Hit on Bugs
with Code Contracts

2. Software needs Reliability
• Two Components
 Correctness
• Does what it’s supposed to do and only that
 Robustness
• Acts appropriately in cases where it cannot do what it is
supposed to do

3. But What’s it Supposed to Do?

4. Describing Software
• How do developers express what software
is supposed to do?
A. Write it in English, allowing your users/clients to
approve it beforehand?
B. Write it in the comments?
C. Describe it in a format system based on discrete
mathematics
D. Poorly?
E. All of the above?

5. Correctness of a Routine
• State the conditions that must be true
before the routine can work correctly
 Pre-conditions
• State the conditions that will be true after
execution, if the routine has worked
correctly
 Post-conditions

6. Let’s Consider an Example
• Create a class that implements a time of day
 Exposes hour, minute, second properties
 Implementation could be as three separate integers or as
the number of seconds since midnight
• We’re going to look only at the process of
assigning the hour

7. Contracts
• Document assumptions
 Preconditions, postconditions, invariants
• Are executable
 Can perform checks at run-time
• Help with static verification
 Assist with early error detection
 Can be used by tools to generate test cases
• Different than assertions
 Assertions are not viewed as a contract, they are
a suggestion
 Difficult to use with test case generation tools

8. What Contracts Can I Write?
• Requires
 What must be true at method entry
• Ensures
 What must be true at method exit
 Includes exits on exceptions
• Invariants
 What must be true at all method exits
• Assertions
 What must be true at a particular point
• Assumptions
 What should be true at a particular point

9. What Can I Put In A Contract?
• Any boolean expression
 In your favorite programming language!
 Including method calls (but must
be marked Pure)
• Contract.Result
 refer to the return value of the method
• Contract.OldValue
 refer to values at method entry
• Quantifiers
 Contract.ForAll(0,A.Length, Function(i) A(i) > 0);
 Contract.Exists(0,A.Length, Function(i) A(i) > 0);

10. Public Overridable Function Add(value As Object) As Integer{
Contract.Requires( value IsNot Nothing )
Contract.Ensures( Count = Contract.OldValue(Of Integer)(Count) + 1 )
Contract.Ensures( Contract.Result(Of Integer)() = _
Contract.OldValue(Of Integer)(Count) )
Executable Runtime
Contract Checking
if (_size == _items.Length) EnsureCapacity(_size+1);
_items[_size] = value; .method public hidebysig newslot virtual instance int32 Add(object 'value') cil managed
{
.locals init (int32 'Contract.Old(Count)',
return _size++; int32 'Contract.Result<int>()')
ldarg.0
call instance int32 TabDemo.BaseList::get_Count()
} stloc.3
ldarg.1
ldnull
csc/vbc/… ceq
ldc.i4.0
.method public hidebysig newslot virtual instance int32 Add(object 'value') cil managed
ceq
{
ldstr quot;value != nullquot;
ldarg.1
call void __RewriterMethods::RewriterRequires$PST06000009(bool, string)
ldnull
/d:CONTRACTS_FULL
Release ldarg.0
ceq
ldfld int32 TabDemo.BaseList::count
ldc.i4.0
ldarg.0
ceq
ldfld object[] TabDemo.BaseList::items
call void [Microsoft.Contracts]Microsoft.Contracts.Contract::Requires(bool)
Compile ldlen
ldarg.0
conv.i4
call instance int32 TabDemo.BaseList::get_Count()
csc/vbc/…
ceq
ldarg.0
ldc.i4.0
call instance int32 TabDemo.BaseList::get_Count()
ceq
call !!0 [Microsoft.Contracts]Microsoft.Contracts.Contract::Old<int32>(!!0)
stloc.1
ldc.i4.1
ldloc.1
add
brtrue IL_004d
ceq
nop
call void [Microsoft.Contracts]Microsoft.Contracts.Contract::Ensures(bool)
ldarg.0
call !!0 [Microsoft.Contracts]Microsoft.Contracts.Contract::Result<int32>()
ccrewrite ldarg.0
ldarg.0
ldfld int32 TabDemo.BaseList::count
call instance int32 TabDemo.BaseList::get_Count()
ldc.i4.1
.method public hidebysig newslot virtual instance int32 Add(object 'value') cil managed call !!0 [Microsoft.Contracts]Microsoft.Contracts.Contract::Old<int32>(!!0)
add
{ ceq
call instance void TabDemo.BaseList::EnsureCapacity(int32)
ldarg.0 call void [Microsoft.Contracts]Microsoft.Contracts.Contract::Ensures(bool)
nop
ldfld int32 TabDemo.BaseList::count ldarg.0
nop
ldarg.0 ldfld int32 TabDemo.BaseList::count
ldarg.0
ldfld object[] TabDemo.BaseList::items ldarg.0
ldfld object[] TabDemo.BaseList::items
ldlen ldfld object[] TabDemo.BaseList::items
ldarg.0
conv.i4 ldlen
ldfld int32 TabDemo.BaseList::count
ceq conv.i4
ldarg.1
ldc.i4.0 ceq
stelem.ref
ceq ldc.i4.0
ldarg.0
stloc.1 ceq
dup
ldloc.1 stloc.1
ldfld int32 TabDemo.BaseList::count
brtrue.s IL_0029 ldloc.1
dup
ldarg.0 brtrue.s IL_0069
stloc.2
ldarg.0 ldarg.0
ldc.i4.1
ldfld int32 TabDemo.BaseList::count ldarg.0
add
ldc.i4.1 ldfld int32 TabDemo.BaseList::count
stfld int32 TabDemo.BaseList::count
add ldc.i4.1
ldloc.2
call instance void TabDemo.BaseList::EnsureCapacity(int32) add
stloc.0
ldarg.0 call instance void TabDemo.BaseList::EnsureCapacity(int32)
br IL_0072
ldfld object[] TabDemo.BaseList::items ldarg.0
ldloc.0
ldarg.0 ldfld object[] TabDemo.BaseList::items
stloc.s 'Contract.Result<int>()'
ldfld int32 TabDemo.BaseList::count ldarg.0
br IL_007a
ldarg.1 ldfld int32 TabDemo.BaseList::count
ldarg.0
stelem.ref ldarg.1
call instance int32 TabDemo.BaseList::get_Count()
ldarg.0 stelem.ref
ldloc.3
dup ldarg.0
ldc.i4.1
ldfld int32 TabDemo.BaseList::count dup
add
dup ldfld int32 TabDemo.BaseList::count
ceq
stloc.2 dup
ldstr quot;Count == Contract.Old(Count) + 1quot;
ldc.i4.1 stloc.2
call void __RewriterMethods::RewriterEnsures$PST0600000B(bool, string)
add ldc.i4.1
ldloc.s 'Contract.Result<int>()'
stfld int32 TabDemo.BaseList::count add
ldloc.s V_4
ldloc.2 stfld int32 TabDemo.BaseList::count
ceq
stloc.0 ldloc.2
ldstr quot;Contract.Result<int>() == Contract.Old(Count)quot;
br.s IL_004b stloc.0
call void __RewriterMethods::RewriterEnsures$PST0600000B(bool, string)
ldloc.0 br.s IL_008b
ldloc.s 'Contract.Result<int>()'
ret ldloc.0
ret
} ret
}
} // end of method BaseList::Add

11. Static Contract Checking
• No silver bullet
 But helps catch errors earliest
 Best used in a focused manner
• Guides development
 Discovers implicit assumptions
 Propagates assumptions
• Not only explicit contracts
 Dereferencing null
 Indexing arrays
 Arithmetic exceptions

12. What Do You Ship?
src
src
src
src
Release Contract Reference
Assemblies Assemblies
+ PowerLib.Contracts.d
PowerLib.dll
ll
(minimal runtime checks) All contracts, no code

13. Interface Contracts
<ContractClass(GetType(CloneableContract))> _
Public Interface ICloneable
Function Clone() As Object
End Interface
ContractClassFor(GetType(ICloneable))> _
Public Class CloneableContract
Implements ICloneable
Public FunctionClone() As Object Implements Icloneable.Clone
Contract.Ensures( Contract.Result(Of Object>() IsNot Nothing)
…
End Function
All classes implementing
End Class
the interface inherit the
contract

14. Code Contracts Summary
• Contract library class enables contracts
in all .NET languages
 No restrictions on what can be expressed
• Contracts are being used in the BCL
today
 Contract library is a core component of .NET 4.0
• Same contracts used for
 Runtime checking
 Static checking
 Documentation generation

15. Why People don’t Write Tests
• Testing is tedious
• Too easy to miss cases
• Old tests get stale
• Too much legacy code

16. What The Demo Showed
• Pex can be used to generate
comprehensive test suite with high
code coverage
• Pex finds contract violations and
potential error situations
• The generated test suite integrates
automatically with
Visual Studio Team Test

17. Pex Understands The Code
• Pex does not generate random inputs,
 enumerate all possible values, or
 make you write test input generators
• Instead, Pex analyzes your .NET code.
 Test inputs computed by Z3,
 Precise inter-procedural, path-sensitive analysis
• As a result, you get
a small test suite with high code coverate
coverage

18. Pex Summary
• Pex generates small test suites with
high code coverage and bug reports for
free
• Reduce test maintenance costs
by parameterized unit testing
• Pex has been used in Microsoft
to test core .NET components
 Almost always finds new bug pathways

19. Summary
• Code Contracts for .NET:
http://research.microsoft.com/Contracts/
• Pex: test generation for .NET
http://research.microsoft.com/Pex/

20. Questions?
• My contact information
 EMail: bjohnson@objectsharp.com
 Twitter: LACanuck
 Blog: http://www.objectsharp.com/blogs/bruce
 MSN: lacanadians@hotmail.com