1. May 4, 2011
Using Formal Tools to Improve the
Productivity of Verification at
STMicroelectronics
James Pascoe
1000 Aztec West, Almondsbury, Bristol, UK
ChipEx 2013 – Tel Aviv, Israel
May 1, 2013

2. May 1, 2013
Overview
• Who are we:
– The ‘CPU’ part of ‘CPU/GPU’ in TR&D (ST Bristol)
– We develop ARM based CPU sub-systems for a range of SoCs
• Organisation:
– System-level functional verification (Noida)
– Block-level activities (Bristol)
– Low-power and DFT verification (Grenoble)
• Formal evaluation:
– Sensor Control Block (SCB) – block-level verification
– Clock and Reset Manager (CRM) – block-level verification
– Documentation driven point-to-point connectivity checking

3. May 1, 2013
Subsystem

4. May 1, 2013
Verification Strategy
• Unit testing:
– Performed by Designers
– Is this block ready to enter verification?
– Designers typically implement HDL based test-benches
• Block-level:
– Performed by Verification engineers
– Does this block conform to its specification in isolation?
– Typically verified using a constrained random approach
• System-level testing:
– Point-to-point checks
– Functional testing
– Performance verification

5. May 1, 2013
Evaluating Formal
• We evaluated a formal tool (Jasper) to determine its potential to
enhance our productivity. Our study had three aims:
1. To close verification projects with appreciably less effort than
constrained random;
2. To promote greater use of assertions by encouraging designers
to develop formal properties for their blocks;
3. To augment or replace legacy in-house flows with mature
industry tools that reduce maintenance and other overheads.

6. May 1, 2013
Approach
• Projects increase in complexity:
– Sensor Control Block
• Digital memory mapped sensor block
• Developed in Bristol
• Constrained Random test-bench developed in parallel
– Clock and Reset Manager
• Provides clock and reset sequencing in subsystem (complex)
• Developed in Grenoble
• Very good micro-architectural documentation but no functional specification
– Point-to-point connectivity checking
• Flow developed to extract assertions from project specifications
• Use Jasper to verify connectivity at the subsystem level
• Also very useful in the context of low-power (e.g. UPF aware proofs)

7. May 1, 2013
Overview
Sensor
Control Block
Clock and
Reset
Manager
Point-to-point
connectivity
checking
Conclusions
Sensor Control Block

8. May 1, 2013
Sensor Control Block
• Provides a digital front-end to thermal sensors:
– Monitors chip temperature
– Selected for its simplicity:
• Connects to thermal sensors
• Samples sensors periodically
• Generates warning interrupts
– Well specified and understood
• Used Jasper to check:
– Registers
– APB interface and protocol
– Temperature functionality
– Sampling periods
– Interrupt properties

9. May 1, 2013
Results
• Found 3 bugs:
– Found subtle problem with PREADY signal on APB interface
• Not detected by Specman
– 1 RTL problem:
• Inverted ‘or’ concatenation for sensor overflow / underflow bits
– 1 specification problem:
• Registers listed as ‘Write Clear 1’ can be cleared with other values
• Validity:
– Didn’t expect to find much wrong 
• Designer had performed extensive unit testing prior to verification
• Some previous Specman verification performed
• Tried and tested components used in the development

10. May 1, 2013
Project Management
• We need to answer the following questions:
– How complete is the verification?
• Q: are there enough assertions to verify all features?
• A: measure ‘out-of-coi’ coverage to find coverage holes
– Is the environment over constrained?
• Q: are we masking bugs by constraining the environment too much?
• A: measure stimuli coverage and check that all legal scenarios are covered
– How good are the bounded proofs for my block?
• Q: do we require more depth in the search space?
• A: use bounded coverage analysis
– How does the combination of formal and dynamic cover the design?
• Q: how do I interpret Jasper coverage with constrained random coverage?
• A: Jasper is working on merging results into other UCDB files

11. May 1, 2013
Coverage
• Dead code analysis:
– Dead code due to RTL – 64 cover points
• Branches that will not be taken during reset
• 1 feature that was later deprecated (but masked with a tie-off in the RTL)
• Register implementation uses generic code to minimise flop generation at synthesis
• Justified exceptions with designer review
• Out of Cone of Influence:
– 219 of 1000 coverage points were uncovered
– > 150 uncovered points were due to the register model
– Remainder due to P1500 signals:
• Feature added when Jasper work was close to sign-off
• Later covered in Specman TB
– Exceptions justified by designer
• Bounded proof coverage – 100% covered

12. May 1, 2013
Overview
Sensor
Control Block
Clock and
Reset
Manager
Point-to-point
connectivity
checking
Conclusions
Clock and Reset Manager

13. May 1, 2013
Clock and Reset Manager
• Generates clocks and resets to blocks within subsystem:
– Sequences actions to control transitions between operating points
– Enables subsystem to be fully asynchronous
– Provides an abstract interface to the SoC
– Good micro-architectural documentation but difficult to verify against
– Critical block and complex
• Verification approach:
– Use Jasper to perform feature extraction
• Not possible with constrained random test-bench
– Prove critical properties using Jasper
• Event sequences, specific timings etc.
– Develop Specman test bench in parallel
• Compare approaches

14. May 1, 2013
Feature Extraction
• Key problem: CRM functionality not well specified:
– Once ‘cmd_in’ rises ‘cmd_ack’ should follow after ‘N’ cycles
– Once ‘cmd_in’ falls, ‘cmd_ack’ will follow after ‘M’ cycles
– where ‘N’ and ‘M’ are not known …
• Solution
– Use Visualise to evaluate the range of cycles the ‘ack’ follows
– Write number of properties with different values for ‘N’
– Find the lowest ‘N’ for which the property proves
– Keep the assertion of the lowest proven ‘N’
assert: $rose(cmd_in) |-> ##[1:80] cmd_ack
– Define cover item for the Min ‘N-1’ for regression
cover: $rose(cmd_in) ##1 !cmd_ack [*80-1]

15. May 1, 2013
Visualize

16. May 1, 2013
Results
• No RTL bugs were found. But:
– CRM was verified previously. Jasper was used to boost confidence 
– Specification is now well understood – solved a big problem
– Used to bound overlap conditions on functional stimuli
• Exhaustive proofs on all functional features
– 73 assertions – all fully proven
– 88 covers – all covered
– Not including test-mode
• Used coverage for:
– Checking for over-constraints – none found
– Ensuring that formal touched all functionality
– More details about coverage to follow

17. May 1, 2013
Coverage
• Dead code analysis:
– Dead code due to RTL – 4 cover points
• Branches that will not be taken during reset or test mode
• Due to synchronisers that have reset signals tied low
• Designer could have replaced them with synchronisers without reset
– Dead code due to no test mode constraint: 54 cover points (58-4)
• Test code that is not active in functional mode – designer reviewed and approved
• Wanted to make sure that no functional branches were made dead by mistake
– Dead code due to reset: 63 cover points (121-58)
• These are the (!resetn) branches
• Reviewed and designer approved
– Dead code due to functional assumptions: 4 cases
• Do we constrain away any valid branches?
• 4 cases were discovered and approved as part of test mode

18. May 1, 2013
Coverage
• Bounded Proof Coverage:
– Initially, not all of the assertions converged
• Out of 55 assertions, 28 did not converge
• Lowest bound for non-converging assertions is: 774 cycles
– Used bounded proof coverage
• Measured coverage within the bounds reached by non-converging properties
– Bounded proof coverage tells us that bound is acceptable because all cover
items are covered in less than 268 cycles
– Subsequently, more advanced engines were used and all properties converged
• Out of Cone of Influence:
– 46 branches were detected as out of cone of influence
– All branches relate to DFT

19. May 1, 2013
Overview
Sensor
Control Block
Clock and
Reset
Manager
Point-to-point
connectivity
checking
Conclusions
Point-to-Point Connectivity Checking

20. May 1, 2013
Point-to-Point Connectivity
• Point-to-point connectivity checking:
– Once everything is verified at the unit and block-level …
– Point-to-point connectivity checking provides a first check that blocks
have been assembled into the subsystem correctly
– Eliminates wiring errors - useful before functional system testing
• Developed a documentation driven point-to-point flow:
– 2564 reference connections generated from key project document
– Point-to-point checking flow setup in 1 day (with help of Jasper) 
– All 2564 properties have been proven
• Useful for low-power:
– Can check connectivity rules for changing power states
– Rule validity can be tied to power states

21. May 1, 2013
Overview
Sensor
Control Block
Clock and
Reset
Manager
Point-to-point
connectivity
checking
Conclusions
Conclusions

22. May 1, 2013
Conclusions
• Formal delivers!
• Found bugs quickly:
– SCB required 6 m/w of effort to build a Constrained Random test-bench
– The formal approach found almost all the bugs within the first week
• Potential for designer involvement is high:
– Designers found Jasper easier to learn than other formal tools
– Jasper was used to develop assertions and to perform unit-testing
• Unit-testing was then reused in the block-level verification
• Enabled us to verify blocks with incomplete specifications:
– Used formal to test implicit assumptions on the CRM
– Provided results that could be quickly verified by designers

23. May 1, 2013
Quality Improvements
• Insight can be captured as properties when:
– Interpreting specifications
– Making assumptions
• Formal provides a good way of stimulating early designs:
– No need for HDL test benches that are discarded once verification starts
– Formal is a great way of performing unit-testing
– Assertions are reused throughout the life-cycle of the IP
• Certain aspects can only be verified formally:
– Absence of deadlock
– Liveness properties
• Properties automatically validate system level behaviour:
– Detects system-level inconsistencies in specifications and assumptions

24. May 1, 2013
The Value of Formal
• Unit / block-level:
– Significant time savings
– No need to develop complicated Constrained Random test-benches
– Potential for property reuse is high
– Provides feedback early in the design process
– Allows designers to stimulate designs without having to write HDL TBs
– Properties add value to subsequent design phases
– Feature extraction can be performed
• System-level:
– Point-to-point checking is easy to setup and provides good insights
– Low level properties assist in verifying system-level assumptions
– Absence of dead-lock / liveness

25. May 1, 2013
Overview
Sensor
Control Block
Clock and
Reset
Manager
Point-to-point
connectivity
checking
Conclusions
Questions