Keynote: Testing the Xbox: Lessons for All

•

0 likes•599 views

Testing a game console isn’t all fun and games. However, with more than 50 million Xbox 360 consoles sold, and the amazing success of the Kinect sensor, it’s certainly a hotbed of excitement for software developers and testers alike. Veteran tester Alan Page is having a blast on the Xbox console team and shares an insider’s view of what it’s like to test one of the most popular entertainment systems ever created. Learn the details of testing the Xbox from the guts of the operating system to the latest applications—and everything in between. Discover how the Xbox team creatively balances test automation with hands-on testing, how they’ve turned traditional measures of code coverage and test pass rates on edge, and many other ways the Xbox team builds, tests, and delivers software that both end-users and game developers love. Join Alan to discover new ideas, see exciting demonstrations, and take away practical ideas that any team can use.

Technology Business

K2
Keynote
10/2/2013 10:00:00 AM

"Testing the Xbox: Lessons for
All"
Presented by:
Alan Page
Microsoft Corporation

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Alan Page
Microsoft
Alan Page is a principal SDET—a fancy name for tester—on the Xbox console team at
Microsoft. Edging up on twenty years in software testing, Alan has previously worked on a
variety of Microsoft products including Windows, Windows CE, Internet Explorer, and Office
Lync. He spent some time as Microsoft’s director of test excellence where he developed and ran
technical training programs for testers throughout the company.

9/19/2013

Testing the Xbox
ALAN PAGE
MICROSOFT

Lessons Learned
from Xbox One

1

9/19/2013

Lessons Learned
from Xbox One

My Story

2

9/19/2013

Build a Great
Team
LESSON 1

Three Rules:
- Build a Team, Not a product
- Don’t Ship your Org Chart
- Leverage your Strengths

4

9/19/2013

Have Super
Powers
LESSON 2

5

9/19/2013

You Don’t Have to
Manage to Lead
LESSON 3

6

9/19/2013

Communicate
Intent and
Priorities

Have a plan for
Debt
LESSON 5

8

9/19/2013

“The Tools, Systems, and
Culture we’re creating now…
…will be the Tools, Systems and
Culture we use for the lifetime
of the product.”

The Crowd is
Powerful
LESSON 6

9

9/19/2013

The Crowd is
Powerful
…and the User is
King
LESSON 6 (REVISED)

Evaluation Scale
5

Love It! – You love it so much you’d shout it out at the top of your
lungs through a bullhorn from every rooftop you encounter

4

Like It – You like it a lot and might mention it during a lull in
conversation at a dinner party

3

Meh – You can live with it and neither like it or hate it. It’s nothing
special. There are some improvements that can be made

2

Don’t Like It – The experience leaves a bad taste in your mouth.
You’d use it if you really had to otherwise, you’d stay away from it.

1

Hate it! – You hate it so much that you would only use it if you
were under a Hogwarts compulsion spell

0

Not Implemented

10

9/19/2013

Scenario

Love It Like It

Meh Don't Like Hate

Do this

43

10

1

1

1

Do that

3

2

9

14

20

Do the other thing

40

14

5

0

1

Do something else

20

3

5

7

15

11

9/19/2013

http://angryweasel.com/blog
http://twitter.com/alanpage

14

9/19/2013

Ease of Use
• Am I able to complete the scenario?
• Is it complicated? Do I need to perform
extraordinary steps to get what I need done?
• Are there glitches in the system that make it
difficult?
• Was it hard to find how to execute this
scenario? Are the features hidden?
• Is the experience consistent?

15

9/19/2013

Responsiveness
• Did I feel like the scenario was fast and
fluid?
• At any point did I feel like I had to wait
a frustrating amount of time?
• Was I effectively distracted while
waiting for an action to take place? (ex.
Movie or animation while I wait)

Usefulness

• Would you use this scenario yourself
to accomplish this specific outcome?
• Does this scenario meet a need for our
consumer?
• Does this let me do something that I
want to do?

16

9/19/2013

Visual Appeal

• Is the experience exciting to see
and hear?
• Is the UI polished?
• Does the UX make the experience
enjoyable?

17

Viewers also liked

One of the most cherished concepts of the Agile Manifesto is valuing individuals and interactions over processes and tools. Within this idea is the implicit assumption that individuals innately know how to interact. Dramatic lessons from aviation suggest otherwise. During the mid-1960s the frequent crashes of perfectly good aircraft alarmed the world’s airlines. Investigators discovered nothing lacking in the pilot’s “stick and rudder” skills; these accidents were the result of the flight crew’s inability to work as a team. Steve Adolph shares four leadership roles necessary for well managed communications in software development—Sheltering to create quiet, focused time needed to get the job done; Supporting to cover the backs of others; Boundary Spanning to connect the silos of communications; and Drum Beating to prevent communications from grinding to a halt. Some individuals are blessed with “natural leadership” talents, but, no worries, these skills can be learned. Join Steve to discover how.

I Thought YOU Were Flying the Plane: Preventing Projects from Falling Out of ...

TechWell

Although usability and user experience may seem synonymous, they are separate and much different concepts. While usability is well defined in standards, UX has no agreed upon definition because it relates to a more nebulous attribute-user satisfaction. Both are, however, key ingredients for successful system deployment. Because they don’t know how to measure and evaluate UX, many teams ignore this important attribute until the end of development. Philip Lew discusses how to model both usability and UX by breaking each attribute down into measurable characteristics-learnability, user effectiveness, user efficiency, content quality, user errors, and more. Phil shows you how to derive measurements and metrics that your development and team can employ to benchmark, analyze, and improve both usability and UX. Beyond the measurements, Phil discusses case studies in which measurements have driven significant usability and user experience improvements.

The Why and How of Usability and User Experience (UX) Testing

TechWell

Your shrinking project deadlines are increasing the need for automated tests—but, simultaneously, reducing the time available for writing them. The system requirements are continually changing. The implementation is changing. You spend more and more time maintaining old tests, leaving less time to write new ones. The tests take longer and longer to run. And when they fail, the problem is as likely to be in the tests as in the system. What’s a developer to do? Dale Emery and George Dinwiddie share hard-won lessons learned from their decades of software development and test automation. Discover the factors that make automated tests maintainable, expressive, informative, fast, reliable, and repeatable. Practice achieving these qualities in hands-on exercises. Apply new techniques and your existing software development expertise in new ways. Take home powerful principles and practices to meet the unique challenges of test automation and to help your project deliver sooner with greater confidence.

The Developer’s Guide to Test Automation

TechWell

Having difficulties getting your organization to recognize the value of QA? Is your “salmon team” losing to currents that impede continuous improvement and strategic planning? Colleen Kirtland and Harish Krishnankutty share their two-year uphill struggle to elevate QA to the position of trusted business partner. Move QA upstream before testing begins by aligning requirements to a business capability model (BCM). Translate the BCM model into key implementation assets with story maps. Before delivering test execution, swim like salmon to frame testing services by connecting day-to-day operational metrics to higher level business value metrics. Partner with your product and/or development teams to inject measurable quality gates upstream in the delivery lifecycle. Learn about the evolution of merging service level management (e.g., ITIL processes) with upstream QA, test, and solution delivery. Create a fun, vibrant team culture of uphill swimmers who advocate formal quality standards. Fight to fund and sustain a multi-year quality strategy while still meeting customer demands.

Swimming with the Salmon: Lessons in Moving Quality Upstream

TechWell

We have opportunities to coach people all the time. Much of what we see as coaching is actually undercover training. Real coaching is richer—offering support while explaining options. In this interactive session, Johanna Rothman invites you to explore how to coach, regardless of your position in the organization. Teaching is just one option for coaching. You have many other options, depending on your coaching stance. You may select a counselor’s stance if you are managing up or a partner’s stance if you are a peer. You might even select a reflective observer’s stance or a technical advisor’s stance, depending on the situation. We will explore what to do when you see opportunities for coaching but you haven’t been asked to coach. Bring your coaching concerns, whether you are coaching onsite, or coaching at a distance, coaching one-on-one, or coaching teams. Let’s learn and build our coaching skills together.

Congruent Coaching: An Exploration

TechWell

In response to increasing market demand for well-performing applications, many organizations implement performance testing programs, often at great expense. Sadly, these solutions alone are often insufficient to keep pace with emerging expectations and competitive pressures. Scott Barber shares the fundamentals of implementing T4APM™ including specific examples from recent client implementations. T4APM™ is a simple and universal approach that is valuable independently or as an extension of existing performance testing programs. The approach hinges on applying a simple and unobtrusive "Target, Test, Trend, Tune” cycle to tasks in your application lifecycle—from a single unit test through entire system production monitoring. Leveraging T4APM™ on a particular task may require knowledge specific to the task, but learning how to leverage the approach does not. Scott provides everything you need to become the T4APM™ coach and champion, and to help your team keep up with increasing demand for better performance, regardless of your current title or role.

Managing Application Performance: A Simplified Universal Approach

TechWell

No matter how thorough the test team or how expansive the test lab, Chris Munroe knows that defects still abound in mobile apps after launch. With more “non-software” companies launching mobile apps every day, testers have increased pressure to ensure apps are secure and function as intended. In retail and media especially, audiences are incredibly diverse and expect apps to work every time, everywhere, and on every device. These expectations make it imperative for companies to take every possible step to make their mobile apps defect free. This is increasingly difficult to do when all your testing occurs within the confines of the lab—and your users live in the wild. Using real-world examples from USA Today, Chris identifies why you need to test your mobile apps both inside and outside the lab—and do so in a way that is secure, effective, and timely.

Mobile App Testing: Moving Outside the Lab

TechWell

Are you running automated tests during development yet not providing automated feedback to the project stakeholders? Vikas Bhupalam approached this problem by leveraging and integrating monitoring, logging, and defect tracking systems to provide automatic feedback to stakeholders. Tests are executed using a Java-based framework, and the results are sent to a monitoring tool that shows up as traffic lights on a dashboard. The dashboard links to logs on the server that provide insights into failing tests and root causes of problems. Alerts can be triggered for specific conditions. Change requests are then automatically filed in the defect tracking system with the appropriate severity and priority set. The QA sign off in all environments is provided to DevOps and all other stakeholders in this automated process. Learn about the framework and the integration involved in bringing all these pieces together.

End-to-End Automation: Providing Stakeholders Feedback on Quality

TechWell

Transforming software development across geographic locations in a large company is difficult. But Nationwide Insurance found a way to successfully implement more than forty agile teams in multiple locations around the United States using their internal Application Development Center. Sara McClintock explains what worked for Nationwide: using Scrum practices, including creating standard work, and lean techniques, using tools such as A3s. Sara provides insight into some of the challenges Nationwide faced and shares some things they should have avoided. With the continued success that Nationwide has experienced in the Application Development Center in the past three years, the company has decided to continue with an Enterprise Transformation that will move their IT organization into full agile software development as they plan, build, and run projects across the company. The goal is to increase productivity, quality, and speed to market using Scrum, XP, lean, and CMMI practices and techniques.

Transforming the Large Organization

TechWell

Ken Pugh takes you beyond thinking of design patterns as “solutions to a problem in a context.” Patterns are really about handling variations in your problem domain while keeping code from becoming complex and difficult to maintain as the system evolves. Ken begins by describing the classic use of patterns. He shows how design patterns implement good coding practices and then explains key design patterns including Strategy, Bridge, Adapter, Façade, and Abstract Factory. In small group exercises, learn how to use patterns to create robust architectures that can readily adapt as new requirements arise. Lessons from these patterns are used to illustrate how to do domain analysis based on abstracting out commonalities in a problem domain and identifying particular variations that must be implemented. Leave with a working understanding of what design patterns are and a better way to build models of your application domains.

Design Patterns Explained: From Analysis through Implementation

TechWell

Viewers also liked (10)

I Thought YOU Were Flying the Plane: Preventing Projects from Falling Out of ...

The Why and How of Usability and User Experience (UX) Testing

The Developer’s Guide to Test Automation

Swimming with the Salmon: Lessons in Moving Quality Upstream

Congruent Coaching: An Exploration

Managing Application Performance: A Simplified Universal Approach

Mobile App Testing: Moving Outside the Lab

End-to-End Automation: Providing Stakeholders Feedback on Quality

Transforming the Large Organization

Design Patterns Explained: From Analysis through Implementation

Similar to Keynote: Testing the Xbox: Lessons for All

You name the testing topic, and Alan Page has an opinion on it, hands-on practical experience with it—or both. Spend the morning with Alan as he discusses a variety of topics, trends, and tales of software engineering and software testing. In an interactive format loosely based on discovering new testing ideas—and bringing new life to some of the old ideas—Alan shares experiences and stories from his twenty year career as a software tester. Topics may include philosophical rants about code coverage and test pass rates; thoughts on the developer/tester relationship and quality ownership; and insights on test leadership and the real future of test. Join Alan for a unique opportunity to participate in intriguing discussions about testing that will expand your testing knowledge, give you the insight you need to grow your own career, and help your organization succeed.

Alan Page: On Testing

TechWell

Testers frequently say that they are the voice of the customer or the customer advocate for their organization’s products. In some situations this can be a helpful mindset, but no matter how hard he tries, a software tester is not the customer. In fact, there is no one better suited to evaluate customer experience than the actual customer of your software. However, getting actionable feedback from customers can be time-consuming, difficult, and often too late to have any meaningful impact on the product. Alan Page shares his thoughts and a number of examples of how to get customer feedback quickly, how to make that feedback actionable, and how to use customer data to drive better software development and testing on any team—and for any product. In this fast-paced session of information and fun, Alan discusses product instrumentation, analysis techniques, reporting, A/B testing, and many other facets of customer feedback.

Inside the Mind of the 21st Century Customer

TechWell

You name the testing topic, and Alan Page has an opinion on it, hands-on practical experience with it—or both. Spend the afternoon with Alan as he discusses a variety of topics, trends, and tales of software engineering and software testing. In an interactive format loosely based on discovering new testing ideas—and bringing new life to some of the old ideas—Alan shares experiences and stories from his twenty year career as a software tester. Topics may include philosophical rants about code coverage and test pass rates; thoughts on the developer/tester relationship and quality ownership; and insights on test leadership and the real future of test. Join Alan for a unique opportunity to participate in intriguing discussions about testing that will expand your testing knowledge, give you the insight you need to grow your own career, and help your organization succeed.

Alan Page: On Testing

TechWell

Sedgwick e0498336-d0105-31189a-assessment 04

Colleen Sedgwick

You already know the value of usability testing. But how do you convince everyone else? This mini-workshop will explain what has worked for facilitators Julie Grundy, Information Architect and UX Designer, Duke University, and Melissa Eggleston, Consultant. We will help attendees gain confidence in their ability to bring usability testing into their organization—despite little resources and time. Our goal is to arm each attendee with a practical guerrilla testing approach and a feeling that they can beat down the bureaucracies of higher ed. Participants will learn how to sell and conduct a usability test with minimal resources. They will also create a script in the workshop, customized to their institution.

EdUI 2016: How to Implement Low-Tech, High-Impact Usability Testing

Melissa Eggleston

Testers are taught they are responsible for all testing. Some even say “It’s not tested until I run the product myself.” Eric Jacobson believes this old school way of thinking can hurt a tester’s reputation and—even worse—may threaten the team’s success. Learning to recognize opportunities where you may not have to test can eliminate bottlenecks and make you everyone’s favorite tester. Eric shares eight patterns from his personal experiences where not testing was the best approach. Examples include patches for critical production problems that can’t get worse, features that are too technical for the tester, cosmetic bug fixes with substantial test setup, and more. Challenge your natural testing assumptions. Become more comfortable with approaches that don’t require testing. Eliminate waste in your testing process by asking, “Does this need to be tested? By me?” Take back ideas to manage not testing including using lightweight documentation for justification. You may find that not testing may actually be a means to better testing.

Maybe We Don’t Have to Test It

TechWell

At Integration Monday, we have had feedback from the audience that people are struggling with understanding how to do automated testing with Logic Apps. Back in the day Mike Stephenson wrote a lot of guidance about automated testing & unit testing for BizTalk. So he took up the challenge of trying to help out on this one. In this session, we will discus some of the challenges around testing Logic Apps and then we will work through some examples of how testing can be performed and finally we will look at an approach which should put us in a solid place to be able to test Logic Apps both as an individual developer and via an automated build.

Automated Testing with Logic Apps and Specflow

BizTalk360

From Gatekeeper to Partner by Kelsey Shannahan

QA or the Highway

Share your #lovefordev

Olga Sergeeva

Invisible Tech: Go APE in the Classroom

International School of Curitiba

Usability Testing: Targeting Mobile Devices

Wayne Pau

Top 5 Apps - Why Commit?

James Abela

Claudia(madobe) Design Pattern.

Naoto MATSUMOTO

Graceful Failure with Selenium and Continuous Integration

Chris B. France

If you are a startup and want to launch a product or a service, you can not effort to let your IT focus on something else than product development. The last couple of years I unfortunately experienced that IT in startups tend to shift the focus over time from product development to internal operations support. Not because the product is ready and nothing to improve anymore, rather because the need of other departments for technical support and help on their essential tasks and processes grown. And the "IT guys" need to help out there to get it running. In this presentation I suggest to outsource every process functionality which is not affecting the core product of a startup to an SaaS service provider out there. Use an expert service provider for sending mail campaigns. Use an drag and drop landingpage creation tool including A/B testing ability for your landingpages. Use a dashboard service for reports and live stats. Use a customer care ticketing and instant faq tool for taking care of customer questions. My main point is: Don't do it internally and don't do it yourself. Pay overall 200 bucks a month and nobody gets hurt. So you can focus on your mission and not on stupid support tasks.

3 Things Rambo can teach you about IT in Startups - Startup Camp Berlin 2013

Martin Müller

Tech 20 Session 10

Greg Dhuyvetter

Aðalsteinn "Alli" Óttarsson", Technical Producer CCP. Synopsis: Massively multiplayer online game developer CCP has been pursuing a multi-product development effort and has teams scattered across the globe. In 2008 CCP decided transition the production of its flagship product EVE Online to agile development and at the same time release their most ambitious expansion to the online universe. In order to achieve this CCP shifted a large portion of their globally distributed resources from working on separate local projects to one unified development effort using Scrum. While getting an insight into the fascinating field of game development attendees will learn about the structure of the project, its roll-out and how the company as a whole transitioned to Agile throughout the release and how the entire development arm of the company has now unified around the framework. Alli covers the biggest hurdles and impediments the company was faced with and how they were solved as well as how the development teams and management embraced agility and the cultural change.

Bridging the Distance through Agile Game Development

Finnur Magnusson

W.O.W-Wonderful, Outstanding, Websites to engage, empower, and inspire

afeldmann

W.O.W. - Wonderful, Outstanding Websites to Engage, Empower, and Inspire

afeldmann

Oracle Solaris 11 Education

Pavel Anni

Similar to Keynote: Testing the Xbox: Lessons for All (20)

Alan Page: On Testing

Inside the Mind of the 21st Century Customer

Alan Page: On Testing

Sedgwick e0498336-d0105-31189a-assessment 04

EdUI 2016: How to Implement Low-Tech, High-Impact Usability Testing

Maybe We Don’t Have to Test It

Automated Testing with Logic Apps and Specflow

From Gatekeeper to Partner by Kelsey Shannahan

Share your #lovefordev

Invisible Tech: Go APE in the Classroom

Usability Testing: Targeting Mobile Devices

Top 5 Apps - Why Commit?

Claudia(madobe) Design Pattern.

Graceful Failure with Selenium and Continuous Integration

3 Things Rambo can teach you about IT in Startups - Startup Camp Berlin 2013

Tech 20 Session 10

Bridging the Distance through Agile Game Development

W.O.W-Wonderful, Outstanding, Websites to engage, empower, and inspire

W.O.W. - Wonderful, Outstanding Websites to Engage, Empower, and Inspire

Oracle Solaris 11 Education

More from TechWell

Do you ever feel you have lost confidence in your own abilities? Why does this happen? Isabel Evans spends a lot of time painting. Someone once commented, “Why are you doing this, when you are not very good at it?” And gradually she stopped drawing and painting, after being intimidated by a conventional vision of what good art should look like. At the same time, she experienced a parallel loss of confidence in her professional abilities. Attempting creative pursuits like drawing and painting is essential to cognitive, emotional, creative abilities and she began to understand the correlation between her creative activities and her confidence. Making errors, being wrong, failing – that is a generous gift we receive when we practice outside our skill level. By staying in a comfort zone and repeating successes, we stagnate. As Isabel started to create again she thought “I don’t feel good at it, I do feel good doing it” The difference was that she was learning, having ideas and the act of re-engaging with failure, together with the comradeship of friends and colleagues, including at Women Who Test, Isabel has regained her confidence in her professional abilities, and been able to reboot her career and joy. Join Isabel to share a journey from self-perceived failure, to recovery and renewed learning.

Failing and Recovering

TechWell

The DevOps movement is here. Companies across many industries are breaking down siloed IT departments and federating them into product development teams. Testing and its practices are at the heart of these changes. Traditionally, IT organizations have been staffed with mostly manual testers and a limited number of automation and performance engineers. To keep pace with development in the new “you build it, you own it” environment, testing teams and individuals must develop new technical skills and even embrace coding to stay relevant and add greater value to the business. DevOps really starts with testing. Join Adam Auerbach as he explains what DevOps is and how it relates to testing. He describes how testing must change from top to bottom and how to access your own environment to identify improvement opportunities. Adam dives into practices like service virtualization, test data management, and continuous testing so you can understand where you are now and identify steps needed to instill a DevOps testing culture in your team and organization.

Instill a DevOps Testing Culture in Your Team and Organization

TechWell

Imagine this … As soon as any developed functionality is submitted into the code repository, it is automatically subjected to the appropriate battery of tests and then released straight into production. Setting up the pipeline capable of doing just that is becoming more and more common and something you need to know about. But most organizations hit the same stumbling block—just what IS the appropriate battery of tests? Automated build architectures don't always lend themselves well to the traditional stages of testing. In this hands-on tutorial, Melissa Benua introduces you to key test design principles—applicable to organizations both large and small—that allow you to take full advantage of the pipeline's capabilities without introducing unnecessary bottlenecks. Learn how to make highly reliable tests that run fast and preserve just enough information to let testers and developers determine exactly what went wrong and how to reproduce the error locally. Explore ways to reduce overlap while still maintaining adequate test coverage. Take back ideas about which test areas could benefit from being combined into a single suite and which areas could benefit most from being broken out altogether.

Test Design for Fully Automated Build Architecture

TechWell

Many organizations invest a lot of effort in test automation at the system level but then have serious problems later on. As a leader, how can you ensure that your new automation efforts will get off to a good start? What can you do to ensure that your automation work provides continuing value? This tutorial covers both “theory” and “practice”. Dot Graham explains the critical issues for getting a good start, and Chris Loder describes his experiences in getting good automation started at a number of companies. The tutorial covers the most important management issues you must address for test automation success, particularly when you are new to automation, and how to choose the best approaches for your organization—no matter which automation tools you use. Focusing on system level testing, Dot and Chris explain how automation affects staffing, who should be responsible for which automation tasks, how managers can best support automation efforts to promote success, what you can realistically expect in benefits and how to report them. They explain—for non-techies—the key technical issues that can make or break your automation effort. Come away with your own clarified automation objectives, and a draft test automation strategy to use to plan your own system-level test automation.

System-Level Test Automation: Ensuring a Good Start

TechWell

Let’s build a mobile app quality and testing strategy together. Whether you have a web, hybrid, or native app, building a quality and testing strategy means (1) knowing what data and tools you have available to make agile decisions, (2) understanding your customers and your competitors, and (3) testing your app under real-world conditions. Jason Arbon guides you through the latest techniques, data, and tools to ensure the awesomeness of your mobile app quality and testing strategy. Leave this interactive session with a strategy for your very own app—or one you pretend to own. The information Jason shares is based on data from Appdiff’s next-gen mobile app testing platform, lessons from Applause/uTest’s crowd, text mining hundreds of millions of app store reviews, and in-depth discussions with top mobile app development teams.

Build Your Mobile App Quality and Test Strategy

TechWell

Technologies, testing processes, and the role of the tester have evolved significantly in the past few years with the advent of agile, DevOps, and other new technologies. It is critical that we testing professionals evaluate ourselves and continue to add tangible value to our organizations. In your work, are you focused on the trivial or on real game changers? Jennifer Bonine describes critical elements that help you artfully blend people, process, and technology to create a synergistic relationship that adds value. Jennifer shares ideas on mastering politics, maneuvering core vs. context, and innovating your technology strategies and processes. She explores how new processes can be introduced in an organization, what the role of organizational culture is in determining the success of a project, and how you can know what tools will add value vs. simply adding overhead and complexity. Jennifer reviews critically needed tester skills and discusses a continual learning model to evolve your skills and stay relevant. This discussion can lead you to technologies, processes, and skills you can stake your career on.

Testing Transformation: The Art and Science for Success

TechWell

We’ve all been there. We work incredibly hard to develop a feature and design tests based on written requirements. We build a detailed test plan that aligns the tests with the software and the documented business needs. And when we put the tests to the software, it all falls apart because the requirements were changed without informing everyone. Mary Thorn says help is at hand. Enter behavior-driven development (BDD), and Cucumber and SpecFlow, tools for running automated acceptance tests and facilitating BDD. Mary explores the nuances of Cucumber and SpecFlow, and shows you how to implement BDD and agile acceptance testing. By fostering collaboration for implementing active requirements via a common language and format, Cucumber and SpecFlow bridge the communication gap between business stakeholders and implementation teams. In this workshop, practice writing feature files with the best practices Mary has discovered over numerous implementations. If you experience developers not coding to requirements, testers not getting requirements updates, or customers who feel out of the loop and don’t get what they ask for, Mary has answers for you.

Implement BDD with Cucumber and SpecFlow

TechWell

Many teams go crazy because of brittle, high-maintenance automated test suites. Jim Holmes helps you understand how to create a flexible, maintainable, high-value suite of functional tests using Selenium WebDriver. Learn the basics of what to test, what not to test, and how to avoid overlapping with other types of testing. Jim includes both philosophical concepts and hands-on coding. Testers who haven't written code should not be intimidated! We'll pair you up to make sure you're successful. Learn to create practical tests dealing with advanced situations such as input validation, AJAX delays, and working with file downloads. Additionally, discover when you need to work together with developers to create a system that's more easily testable. This tutorial focuses primarily on automating web tests, but many of the same concepts can be applied to other UI environments. Demos and labs will be in C# and Java using WebDriver. Leave this tutorial having learned how to write high-value WebDriver tests—and stay sane while doing so.

Develop WebDriver Automated Tests—and Keep Your Sanity

TechWell

DevOps is a cultural shift aimed at streamlining intergroup communication and improving operational efficiency for development and operations groups. Over time, inclusion of other IT groups under the DevOps umbrella has become the norm for many organizations. But even broadening the boundaries of DevOps, the conversation has been largely devoid of the business units’ place at the table. A common mistake organizations make while going through the DevOps transformation is drawing a line at the IT boundary. If that occurs, a larger, more inclusive silo within the organization is created, operating in an informational vacuum and causing operational inefficiency and goal misalignment. Sharing his experiences working on both sides of the fence, Leon Fayer describes the importance of including business units in order to align technology decisions with business goals. Leon discusses inclusion of business units in existing agile processes, benefits of cross-departmental monitoring, and a business-first approach to technology decisions.

Ma 15

TechWell

Chris Parlette maintains that renting infrastructure on demand is the most disruptive trend in IT in decades. In 2016, enterprises spent $23B on public cloud IaaS services. By 2020, that figure is expected to reach $65B. The public cloud is now used like a utility, and like any utility, there is waste. Who's responsible for optimizing the infrastructure and reducing wasted expenses? It’s DevOps. The excess expense, known as cloud waste, comprises several interrelated problems: services running when they don't need to be, improperly sized infrastructure, orphaned resources, and shadow IT. There are a few core tenets of DevOps—holistic thinking, no silos, rapid useful feedback, and automation—that can be applied to reducing your cloud waste. Join Chris to learn why you should include continuous cost optimization in your DevOps processes. Automate cost control, reduce your cloud expenses, and make your life easier.

Eliminate Cloud Waste with a Holistic DevOps Strategy

TechWell

With the recent emergence of DevOps across the industry, testing organizations are being challenged to transform themselves significantly within a short period of time to stay meaningful within their organizations. It’s not easy to plan and approach these changes considering the way testing organizations have remained structured for ages. These challenges start from foundational organizational structures and can cut across leadership influence, competencies, tools strategy, infrastructure, and other dimensions. Sumit Kumar shares his experience assisting various organizations to overcome these challenges using an organized DevOps enablement framework. The framework includes radical restructuring, turning the tools strategy upside down, a multidimensional workforce enablement supported by infrastructure changes, redeveloped collaborations models, and more. From his real world experiences Sumit shares tips for approaching this journey and explains the roadmap for testing organizations to transform themselves to lead the quality in DevOps.

Transform Test Organizations for the New World of DevOps

TechWell

All too often, the triple constraints—time, cost, and quality—are bandied about as if they are the be-all, end-all. While they are important, leadership—the fourth and larger underpinning constraint—influences the first three. Statistics on project success and failure abound, and these measurements are usually taken against the triple constraints. According to the Project Management Institute, only 53 percent of projects are completed within budget, and only 49 percent are completed on time. If so many projects overrun budget and are late, we can’t really say, “Good, fast, or cheap—pick two.” Rob Burkett talks about leadership at every level of a team. He shares his insights and stories gleaned from his years of IT and project management experience. Rob speaks to some of the glaring difficulties in the workplace in general and some specifically related to IT delivery and project management. Leave with a clearer understanding of how to communicate with teams and team members, and gain a better understanding of how you can be a leader—up and down your organization.

The Fourth Constraint in Project Delivery—Leadership

TechWell

As teams grow, organizations often draw a distinction between feature teams, which deliver the visible business value to the user, and component teams, which manage shared work. Steve Berczuk says that this distinction can help organizations be more productive and scale effectively, but he recognizes that not all shared work fits into this model. Some work is best handled by “specialists,” that is people with unique skills. Although teams composed entirely of T-shaped people is ideal, certain skills are hard to come by and are used irregularly across an organization. Since these specialists often need to work closely with teams, rather than working from their own backlog, they don’t fit into the component team model. The use of shared resources presents challenges to the agile planning model. Steve Berczuk shares how teams such as those providing infrastructure services and specialists can fit into a feature+component team model, and how variations such as embedding specialists in a scrum team can both present process challenges and add significant value to both the team and the larger organization.

Resolve the Contradiction of Specialists within Agile Teams

TechWell

Metrics don’t have to be a necessary evil. If done right, metrics can help guide us to make better forward-looking decisions, rather than being used for simply managing or monitoring. They can help us identify trade-offs between options for what to do next versus punitive or worse, purely managerial measures. Steve Martin won’t be giving the Top Ten List of field-tested metrics you should use. Instead, in this interactive mini-workshop, he leads you through the critical thinking necessary for you to determine what is right for you to measure. First, Steve explores why you want to measure something—whether it’s for a team, a portfolio, or even an agile transformation. Next, he provides multiple real-life metrics examples to help drive home concepts behind characteristics of good and bad metrics. Finally, Steve shows how to run his field-tested agile game—Pin the Tail on the Metric. Take back this activity to help you guide metrics conversations at your organization.

Pin the Tail on the Metric: A Field-Tested Agile Game

TechWell

A hierarchy is an organizational network that has a top and a bottom, and where position is determined by rank, importance, and value. A holarchy is a network that has no top or bottom and where each person’s value derives from his ability, rather than position. As more companies seek the benefits of agile, leaders need to build and sustain delivery capability while scaling agile without introducing unnecessary process and overhead. The Agile Performance Holarchy (APH) is an empirical model for scaling and sustaining agility while continuing to deliver great products. Jeff Dalton designed the APH by drawing from lessons learned observing and assessing hundreds of agile companies and teams. The APH helps implement a holarchy—a system composed of interacting organizational units called holons—centered on a series of performance circles that embody the behaviors of high performing agile organizations. Jeff describes how APH provides guidelines in the areas of leadership, values, teaming, visioning, governing, building, supporting, and engaging within an all-agile organization. Join Jeff to see what the APH is all about and how you can use it in your team and organization.

Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams

TechWell

A Business-First Approach to DevOps Implementation

TechWell

DevOps is transforming software development with many organizations adopting lean development practices, implementing continuous integration (CI), and performing regular continuous deployment (CD) to their production environments. However, the database is largely ignored and often seen as a bottleneck in the DevOps process. Steve Jones discusses the challenges of database development and why many developers find the database to be an impediment to the CD process. Steve shares the techniques you can use to fit a database into the DevOps process. Learn how to store database code in a version control system, and the differences between that and application code. Steve demonstrates a CI process with SQL code and uses automated testing frameworks to check the code. Steve then shows how automated releases with manual gates can reduce the stress and risk of database deployments while ensuring consistent, reliable, repeatable releases to QA, UAT, and production.

Databases in a Continuous Integration/Delivery Process

TechWell

Organizations are moving rapidly into mobile technology, which has significantly increased the demand for testing of mobile applications. David Dangs says testers naturally are turning to automation to help ease the workload, increase potential test coverage, and improve testing efficiency. But should you try to automate all things mobile? Unfortunately, the answer is not always clear. Mobile has its own set of complications, compounded by a wide variety of devices and OS platforms. Join David to learn what mobile testing activities are ripe for automation—and those items best left to manual efforts. He describes the various considerations for automating each type of mobile application: mobile web, native app, and hybrid applications. David also covers device-level testing, types of testing, available automation tools, and recommendations for automation effectiveness. Finally, based on his years of mobile testing experience, David provides some tips and tricks to approach mobile automation. Leave with a clear plan for automating your mobile applications.

Mobile Testing: What—and What Not—to Automate

TechWell

Diversity is becoming the norm in everyday life. However, introducing global delivery models without a proper understanding of intercultural differences can lead to difficulty, frustration, and reduced productivity. Priyanka Sharma and Thena Barry say that in our diverse world, we need teams with people who can cross these boundaries, communicate effectively, and build the diverse networks necessary to avoid problems. We need to learn about cultural intelligence (CI) and cultural quotient (CQ). CI is the ability to relate and work effectively across cultures. CQ is the cognitive, motivational, and behavioral capacity to understand and respond to beliefs, values, attitudes, and behaviors of individuals and groups. Together, CI and CQ can help us build behavioral capacities that aid motivation, behavior, and productivity in teams as well as individuals. Priyanka and Thena show how to build a more culturally intelligent place with tools and techniques from Leading with Cultural Intelligence, as well as content from the Hofstede cultural model. In addition, they illustrate the model with real-life experiences and demonstrate how they adapted in similar circumstances.

Cultural Intelligence: A Key Skill for Success

TechWell

Why would a century-old utility with no direct competitors take on the challenge of transforming its entire IT application organization to an agile methodology? In an increasingly interconnected world, the expectations of customers continue to evolve. From smart meters to smart phones, IoT is creating a crisis point for industries not accustomed to rapid change. Glen Morris explains that pizzas can be tracked by the minute and packages at every stop, and customers now expect this same customer service model should exist for all industries—including power. Glen examines how to create momentum and transform non-IT-focused industries to an agile model. If you are struggling with gaining traction in your pursuit of agile within your business, Glen gives you concrete, practical experiences to leverage in your pursuit. Finally, he communicates how to gain buy-in from business partners who have no idea or concern about agile or its methodologies. If your business partners look at you with amusement when you mention the need for a dedicated Product Owner, join Glen as he walks you through the approaches to overcoming agile skepticism.

Turn the Lights On: A Power Utility Company's Agile Transformation

TechWell

More from TechWell (20)

Failing and Recovering

Instill a DevOps Testing Culture in Your Team and Organization

Test Design for Fully Automated Build Architecture

System-Level Test Automation: Ensuring a Good Start

Build Your Mobile App Quality and Test Strategy

Testing Transformation: The Art and Science for Success

Implement BDD with Cucumber and SpecFlow

Develop WebDriver Automated Tests—and Keep Your Sanity

Ma 15

Eliminate Cloud Waste with a Holistic DevOps Strategy

Transform Test Organizations for the New World of DevOps

The Fourth Constraint in Project Delivery—Leadership

Resolve the Contradiction of Specialists within Agile Teams

Pin the Tail on the Metric: A Field-Tested Agile Game

Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams

A Business-First Approach to DevOps Implementation

Databases in a Continuous Integration/Delivery Process

Mobile Testing: What—and What Not—to Automate

Cultural Intelligence: A Key Skill for Success

Turn the Lights On: A Power Utility Company's Agile Transformation

Recently uploaded

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Imagine a world where information flows as swiftly as thought itself, making decision-making as fluid as the data driving it. Every moment is critical, and the right tools can significantly boost your organization’s performance. The power of real-time data automation through FME can turn this vision into reality. Aimed at professionals eager to leverage real-time data for enhanced decision-making and efficiency, this webinar will cover the essentials of real-time data and its significance. We’ll explore: FME’s role in real-time event processing, from data intake and analysis to transformation and reporting An overview of leveraging streams vs. automations FME’s impact across various industries highlighted by real-life case studies Live demonstrations on setting up FME workflows for real-time data Practical advice on getting started, best practices, and tips for effective implementation Join us to enhance your skills in real-time data automation with FME, and take your operational capabilities to the next level.

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

Safe Software

[2024]Digital Global Overview Report 2024 Meltwater.pdf

hans926745

The presentation explores the development and application of artificial intelligence (AI) from its inception to its current status in the modern world. The term "artificial intelligence" was first coined by John McCarthy in 1956 to describe efforts to develop computer programs capable of performing tasks that typically require human intelligence. This concept was first introduced at a conference held at Dartmouth College, where programs demonstrated capabilities such as playing chess, proving theorems, and interpreting texts. In the early stages, Alan Turing contributed to the field by defining intelligence as the ability of a being to respond to certain questions intelligently, proposing what is now known as the Turing Test to evaluate the presence of intelligent behavior in machines. As the decades progressed, AI evolved significantly. The 1980s focused on machine learning, teaching computers to learn from data, leading to the development of models that could improve their performance based on their experiences. The 1990s and 2000s saw further advances in algorithms and computational power, which allowed for more sophisticated data analysis techniques, including data mining. By the 2010s, the proliferation of big data and the refinement of deep learning techniques enabled AI to become mainstream. Notable milestones included the success of Google's AlphaGo and advancements in autonomous vehicles by companies like Tesla and Waymo. A major theme of the presentation is the application of generative AI, which has been used for tasks such as natural language text generation, translation, and question answering. Generative AI uses large datasets to train models that can then produce new, coherent pieces of text or other media. The presentation also discusses the ethical implications and the need for regulation in AI, highlighting issues such as privacy, bias, and the potential for misuse. These concerns have prompted calls for comprehensive regulations to ensure the safe and equitable use of AI technologies. Artificial intelligence has also played a significant role in healthcare, particularly highlighted during the COVID-19 pandemic, where it was used in drug discovery, vaccine development, and analyzing the spread of the virus. The capabilities of AI in healthcare are vast, ranging from medical diagnostics to personalized medicine, demonstrating the technology's potential to revolutionize fields beyond just technical or consumer applications. In conclusion, AI continues to be a rapidly evolving field with significant implications for various aspects of society. The development from theoretical concepts to real-world applications illustrates both the potential benefits and the challenges that come with integrating advanced technologies into everyday life. The ongoing discussion about AI ethics and regulation underscores the importance of managing these technologies responsibly to maximize their their benefits while minimizing potential harms.

Artificial Intelligence: Facts and Myths

Joaquim Jorge

Histor y of HAM Radio presentation slide

vu2urc

With more memory available, system performance of three Dell devices increased, which can translate to a better user experience Conclusion When your system has plenty of RAM to meet your needs, you can efficiently access the applications and data you need to finish projects and to-do lists without sacrificing time and focus. Our test results show that with more memory available, three Dell PCs delivered better performance and took less time to complete the Procyon Office Productivity benchmark. These advantages translate to users being able to complete workflows more quickly and multitask more easily. Whether you need the mobility of the Latitude 5440, the creative capabilities of the Precision 3470, or the high performance of the OptiPlex Tower Plus 7010, configuring your system with more RAM can help keep processes running smoothly, enabling you to do more without compromising performance.

Boost PC performance: How more available memory can improve productivity

Principled Technologies

A Domino Admins Adventures (Engage 2024)

Gabriella Davis

The 7 Things I Know About Cyber Security After 25 Years | April 2024

Rafal Los

This presentation explores the impact of HTML injection attacks on web applications, detailing how attackers exploit vulnerabilities to inject malicious code into web pages. Learn about the potential consequences of such attacks and discover effective mitigation strategies to protect your web applications from HTML injection vulnerabilities. for more information visit https://bostoninstituteofanalytics.org/category/cyber-security-ethical-hacking/

HTML Injection Attacks: Impact and Mitigation Strategies

Boston Institute of Analytics

Join our latest Connector Corner webinar to discover how UiPath Integration Service revolutionizes API-centric automation in a 'Quote to Cash' process—and how that automation empowers businesses to accelerate revenue generation. A comprehensive demo will explore connecting systems, GenAI, and people, through powerful pre-built connectors designed to speed process cycle times. Speakers: James Dickson, Senior Software Engineer Charlie Greenberg, Host, Product Marketing Manager

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

DianaGray10

Scaling API-first – The story of a global engineering organization

Radu Cotescu

Partners Life - Insurer Innovation Award 2024

The Digital Insurer

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

In this session, we will delve into strategic approaches for optimizing knowledge management within Microsoft 365, amidst the evolving landscape of Copilot. From leveraging automatic metadata classification and permission governance with SharePoint Premium, to unlocking Viva Engage for the cultivation of knowledge and communities, you will gain actionable insights to bolster your organization's knowledge-sharing initiatives. In this session, we will also explore how to facilitate solutions to enable your employees to find answers and expertise within Microsoft 365. You will leave equipped with practical techniques and a deeper understanding of how there is more to effective knowledge management than just enabling Copilot, but building actual solutions to prepare the knowledge that Copilot and your employees can use.

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Drew Madelung

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Neo4j

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

GenCyber Cyber Security Day Presentation

Michael W. Hawkins

As privacy and data protection regulations evolve rapidly, organizations operating in multiple jurisdictions face mounting challenges to ensure compliance and safeguard customer data. With state-specific privacy laws coming up in multiple states this year, it is essential to understand what their unique data protection regulations will require clearly. How will data privacy evolve in the US in 2024? How to stay compliant? Our panellists will guide you through the intricacies of these states' specific data privacy laws, clarifying complex legal frameworks and compliance requirements. This webinar will review: - The essential aspects of each state's privacy landscape and the latest updates - Common compliance challenges faced by organizations operating in multiple states and best practices to achieve regulatory adherence - Valuable insights into potential changes to existing regulations and prepare your organization for the evolving landscape

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

TrustArc

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

apidays

Recently uploaded (20)

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

[2024]Digital Global Overview Report 2024 Meltwater.pdf

Artificial Intelligence: Facts and Myths

Histor y of HAM Radio presentation slide

Boost PC performance: How more available memory can improve productivity

A Domino Admins Adventures (Engage 2024)

The 7 Things I Know About Cyber Security After 25 Years | April 2024

HTML Injection Attacks: Impact and Mitigation Strategies

Connector Corner: Accelerate revenue generation using UiPath API-centric busi...

Scaling API-first – The story of a global engineering organization

Partners Life - Insurer Innovation Award 2024

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Automating Google Workspace (GWS) & more with Apps Script

How to Troubleshoot Apps for the Modern Connected Worker

GenCyber Cyber Security Day Presentation

TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments

Apidays New York 2024 - The value of a flexible API Management solution for O...

Keynote: Testing the Xbox: Lessons for All

1. K2 Keynote 10/2/2013 10:00:00 AM "Testing the Xbox: Lessons for All" Presented by: Alan Page Microsoft Corporation Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Alan Page Microsoft Alan Page is a principal SDET—a fancy name for tester—on the Xbox console team at Microsoft. Edging up on twenty years in software testing, Alan has previously worked on a variety of Microsoft products including Windows, Windows CE, Internet Explorer, and Office Lync. He spent some time as Microsoft’s director of test excellence where he developed and ran technical training programs for testers throughout the company.

3. 9/19/2013 Testing the Xbox ALAN PAGE MICROSOFT Lessons Learned from Xbox One 1

4. 9/19/2013 Lessons Learned from Xbox One My Story 2

5. 9/19/2013 October 31, 2011 3

6. 9/19/2013 Build a Great Team LESSON 1 Three Rules: - Build a Team, Not a product - Don’t Ship your Org Chart - Leverage your Strengths 4

7. 9/19/2013 Have Super Powers LESSON 2 5

8. 9/19/2013 You Don’t Have to Manage to Lead LESSON 3 6

9. 9/19/2013 Have a Strategy LESSON 4 7

10. 9/19/2013 Communicate Intent and Priorities Have a plan for Debt LESSON 5 8

11. 9/19/2013 “The Tools, Systems, and Culture we’re creating now… …will be the Tools, Systems and Culture we use for the lifetime of the product.” The Crowd is Powerful LESSON 6 9

12. 9/19/2013 The Crowd is Powerful …and the User is King LESSON 6 (REVISED) Evaluation Scale 5 Love It! – You love it so much you’d shout it out at the top of your lungs through a bullhorn from every rooftop you encounter 4 Like It – You like it a lot and might mention it during a lull in conversation at a dinner party 3 Meh – You can live with it and neither like it or hate it. It’s nothing special. There are some improvements that can be made 2 Don’t Like It – The experience leaves a bad taste in your mouth. You’d use it if you really had to otherwise, you’d stay away from it. 1 Hate it! – You hate it so much that you would only use it if you were under a Hogwarts compulsion spell 0 Not Implemented 10

13. 9/19/2013 Scenario Love It Like It Meh Don't Like Hate Do this 43 10 1 1 1 Do that 3 2 9 14 20 Do the other thing 40 14 5 0 1 Do something else 20 3 5 7 15 11

14. 9/19/2013 OverCommunicate LESSON 7 12

15. 9/19/2013 Have Fun LESSON 8 13

16. 9/19/2013 http://angryweasel.com/blog http://twitter.com/alanpage 14

17. 9/19/2013 Ease of Use • Am I able to complete the scenario? • Is it complicated? Do I need to perform extraordinary steps to get what I need done? • Are there glitches in the system that make it difficult? • Was it hard to find how to execute this scenario? Are the features hidden? • Is the experience consistent? 15

18. 9/19/2013 Responsiveness • Did I feel like the scenario was fast and fluid? • At any point did I feel like I had to wait a frustrating amount of time? • Was I effectively distracted while waiting for an action to take place? (ex. Movie or animation while I wait) Usefulness • Would you use this scenario yourself to accomplish this specific outcome? • Does this scenario meet a need for our consumer? • Does this let me do something that I want to do? 16

19. 9/19/2013 Visual Appeal • Is the experience exciting to see and hear? • Is the UI polished? • Does the UX make the experience enjoyable? 17

Keynote: Testing the Xbox: Lessons for All

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (10)

Similar to Keynote: Testing the Xbox: Lessons for All

Similar to Keynote: Testing the Xbox: Lessons for All (20)

More from TechWell

More from TechWell (20)

Recently uploaded

Recently uploaded (20)

Keynote: Testing the Xbox: Lessons for All