Testing with VS2010 - A Bugs Life

A bug’s life Testing with VS 2010

Testing the world with Visual Studio 2010 How often do you get a bug report that you can’t reproduce? How often do you struggle to find the source of the bug in your code? How can you streamline the testing process and make sure you don’t repeat the same bugs? With Visual Studio 2010 & Team Foundation Server (TFS) 2010 there are a lot of new features for testers and developers that will solve these problems. In this session you will learn how to: automate more, reproduce bugs easier, maintain your tests and configuration and discover problems sooner. Abstract

In every day software development, we often encounter bugs that are difficult to reproduce and even more difficult to find. This problem is accentuated when it’s a reoccurring bug in the system. Visual Studio 2010 & Team Foundation Server (TFS) 2010 introduces heaps of new features that will help testers and developers in squashing these bugs once and for all. In this session you will learn how to: Reproduce bugs Automate your tests to find reoccurring bugs Maintain your tests and configuration Discover problems sooner Abstract

SSA @ SSW Loves C# and .NET (Java not anymore) Specializes in Windows Forms ASP.NET TFS testing Automated tests Peter Gfader

How to report bugs What to test to find bugs? Tests Unit tests Integration test Testing, reporting bugs and fixing bugs Test & Lab Manager (MTLM) VS2010 Missing in VS2010 Agenda

“Pass by my office then I show you”

Email Web application TFS web access How do you report bugs?

And we love emailsTeamCompanion glues those worlds together http://www.ssw.com.au/ssw/standards/rules/rulestobetterprojectmanagementwithtfs.aspx#TeamCompanionWorkItem How we report bugs

Instructions to tester Reproduce steps Screen captures Stacktrace (Windows Forms, WPF, ASP.NET, Silverlight) Win7 Recorder? How do you report bugs (Instructions)?

Reproduce steps Screen captures Stacktrace (Windows Forms, WPF, ASP.NET, Silverlight) Win7 Recorder ??? URL (Page) Version number Browser (Firefox or IE) http://www.ssw.com.au/ssw/standards/Support/BugReportOrEnhancement.aspx Instructions to tester

** JavaScriptCapturing unreported bugs #2

LadyLoghttp://www.ssw.com.au/ssw/Standards/Rules/RulesToBetterErrorHandling.aspx We suggest to use

** Triaging rule Runtime error  this release Suggestions / feature  next release Little UI ones  this release ** RULE Do you have rules on triaging?

** add pix Ryan His favourite quote “Test failed – 5 critical bugs found” Do you have a dedicated tester?

Code UI DB Deployment Reports Performance Security Whole system User requirements fulfilled What do we want to test?

What is the problem here? 1. What are we testing? [TestMethod] publicvoidTestCalculation() { varorderManager = newOrderManager(); using (var db = newNorthwindDataContext()) { var order = db.Orders.FirstOrDefault(); var actual = orderManager.GetTotalIncGST(order); var expected = order.OrderItems.Sum(oi => oi.Qty * price) * 1.1; Assert.AreEqual(actual, expected); } } 2.

“A unit test is a fast, in-memory, consistent, automated and repeatable test of a functional unit-of-work in the system.” “A unit of work is any functional scenario in the system that contains logic. It can be as short as a function, or it can span multiple classes and functions, and it provides internal or business value to the system under test.” “Art of Unit Testing” Roy Osherove What is a Unit test?

It talks to the database It communicates across the network It touches the file system It can't run at the same time as any of your other unit tests You have to do special things to your environment Configuration files, registry, ... Michael Feathers What is NOT a Unit test?

Integrates different pieces UI Business Layer Database Not in memory Touches File system Registry Database Other shared resources Integration test

Run much slower not run in-memory Less consistent Dependent from external resources Has side effects Harder to prepare E.g. Database setup Integration test

Unit tests – ALWAYS Integration tests – “allow” to be red Why is this important?

Readable Understand! What? Scenario? Expectation? Maintainable We change our application Trustworthy All  We are bug free Attributes of a good Unit test

[TestMethod] publicvoidActualEndTime_PlusOvertimeHardcoded_Valid() { // Arrange EmployeeShiftnewShift=newEmployeeShift(); newShift.EndTime=newDateTime(2000, 01, 01, 8, 0, 0); newShift.OvertimeAfter= 1.5; // Act DateTime actual = newShift.ActualEndTime; // Assert Assert.IsTrue(actual == newDateTime(2000, 01, 01, 9, 30, 0)); }

[TestMethod] publicvoidGetDirtyObjects_Insert2Shifts_DirtyObjectsFound() { using (var db = newNorthwindDataContext()) { // Arrange db.CreateNewEmployeeShift( 1 // EmployeeId "01/01/2000 02:00", "01/01/2000 10:00“); db.CreateNewEmployeeShift( 2 // EmployeeId "01/01/2000 15:00", "01/01/2000 19:00”); // Act vardirtyObjects = db.GetDirtyObjects(empShiftIdToVerify); // Assert Assert.IsTrue(dirtyObjects.Count() > 0); } }

[TestMethod] publicvoidAlwaysTrueRuleTest() { //Arrange var rules = newList<IRule<Shift>> { newWorkflowRule<Shift>() { Name = "Rule 1", Description = "This rule will always pass", Condition = "true", ErrorMessage = "This will never be shown" } }; var shift = newShift() { ShiftID = Guid.NewGuid(), StartTime = DateTime.Now, EndTime = DateTime.Now }; //Act var results = RuleManager.Execute<Shift>(shift, rules); //Assert Assert.IsTrue(results.Count == 0); Assert.IsTrue(results.IsValid); }

[TestMethod] publicvoidTestCalculation() { varnorthWindDataContextFake = MockRepository.GenerateStub<INorthwindDataContext>(); varorderitemsFake = Helper.CreateListOfOrders(); varorderFake = Helper.CreateNewOrder(); northWindDataContextFake.Stub( d => d.Orders.FirstOrDefault()).Return(orderFake); northWindDataContextFake.Stub( d => d.OrderItems).Return(orderitemsFake); varorderManager = newOrderManager(); using (var db = northWindDataContextFake) { var order = db.Orders.FirstOrDefault(); var actual = orderManager.GetTotalIncGST(order); var expected = order.OrderItems.Sum( oi => oi.Qty * oi.Price) * 1.1; Assert.AreEqual(actual, expected); } }

[TestMethod] publicvoidPresenter_Request_ReturnsRequestFromHttpContext() { // Arrange var view = MockRepository.GenerateStub<IView>(); varhttpContext = MockRepository.GenerateStub<HttpContextBase>(); var request = MockRepository.GenerateStub<HttpRequestBase>(); httpContext.Stub(h => h.Request).Return(request); // Act var presenter = newTestPresenter(view); presenter.HttpContext = httpContext; // Assert Assert.AreSame(request, presenter.Request); } http://webformsmvp.codeplex.com/

[TestMethod] publicvoidRedirectPresenterRedirectsOnActionAccepted() { // Arrange var view = MockRepository.GenerateMock<IRedirectView>(); varhttpContext = MockRepository.GenerateMock<HttpContextBase>(); varhttpResponse = MockRepository.GenerateMock<HttpResponseBase>(); httpContext.Expect(h => h.Response).Return(httpResponse); httpResponse.Expect(r => r.Redirect("~/RedirectTo.aspx")); var presenter = newRedirectPresenter(view) { HttpContext = httpContext }; // Act view.Raise(v => v.Load += null, view, newEventArgs()); view.Raise(v => v.ActionAccepted += null, view, newEventArgs()); presenter.ReleaseView(); // Assert httpContext.VerifyAllExpectations(); httpResponse.VerifyAllExpectations(); } http://webformsmvp.codeplex.com/

[TestMethod] publicvoid DataContextSubmitChangesValidateDirtyObjects_RuleResultSavedMeasureTime_RunProcessorAfterInsert5Shifts() { for (inti = 0; i < 5; i++) { //dummy entries DbHelper.CreateNewEmployeeShift( DbHelper.EmployeeId1SouthSuperIntendentEast, "01/01/2000 15:00“, "01/01/2000 16:00“, true); } Stopwatch stopwatch = new Stopwatch(); stopwatch.Start(); RuleProcessor.Default.Start(); System.Threading.Thread.Sleep(2000); RuleProcessor.Default.Stop(); stopwatch.Stop(); // no dirty object left vardirtyObjects = DbHelper.GetDirtyObjects(); Assert.IsTrue(dirtyObjects.Count() == 0,"Dirty objects found"); }

Hard to see Naming convention Unit test Test single small “unit of work” Integration test Test integration, functionality Unit test vs. Integration test

Test & Lab Manager (MTLM) VS2010 TestingReporting bugsFixing bugs

VS2008Scenario #1 Coding done Internal testing done Deploy to staging “Test please” to Tester…

No plan (no user stories) Lucky to find bugs Waste a lot of time on irrelevant things This is called “Exploratory testing” Tester tests

Subject: Bug found!! “I cant quite remember what I did but I got this after a while”

Hi Ryan The developer tests and replies...

Dev: We have 2 additionally reported bugs PM: No time to fix bugs... We must SHIP!  We call this the “Vista method” Finally at the end of Test phase

2 weeks development  BANG Tester has no plan – Exploratory testing Tester tests everything “Unable to reproduce” = “Works on my machine” Bug never gets fixed Ship a buggy product Problems

VS2010#1 Tester uses Test Lab Manager

#3 Dev sees bug Start VS2010 Open “My work items”

Bug in TFS Video.wmv Detailed steps IntelliTrace (drill through to Visual Studio) Automated recording Lots of details (you don’t need if you get intelliTrace) Stack trace System info What’s good?

#4 Dev rewinds debugging session Dev spots bugs

From recorded test we create an automated test Don’t fix bug yet#5 Create automated test

Changes to the test //Arrange ApplicationUnderTestappUT = ApplicationUnderTest.Launch(@”bin86ebugvalonCalculator2.exe"); //Act // SNIP SNIPSNIP //Assert Assert.IsTrue(appUT.Process.Responding, "App not responding");

VS2008: Developer tests manually his bug fix VS2010: Replay coded UI test #7 Dev verifies bug fix

Easy to reproduce the bug Create functional test (coded UI test) Automate the test from recording Good for future changes (Regression testing) What’s good with coded UI tests

Suddenly wake up because of a surprise!IntelliTrace

Suddenly wake up because of a surprise!

Rewind back to see what happened...IntelliTrace

Was called “Historical Debugging” Easy to spot the bug Record method calls with parameters Advanced call stack Get intellitrace logs from your testers Record / Trace testers execution Use same source code to debug as the tester had to test Get intellitrace logs from your build server What’s good in IntelliTrace

Hi Ryan Done – Fixed and test added PS: you don’t need to test this again PPS: testers are becoming less useful with 2010 Peter #8 Email to tester

Testing with VS2010 - A Bugs Life

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (13)

Similar to Testing with VS2010 - A Bugs Life

Similar to Testing with VS2010 - A Bugs Life (20)

More from Peter Gfader

More from Peter Gfader (20)

Recently uploaded

Recently uploaded (20)

Testing with VS2010 - A Bugs Life

Editor's Notes