During this webinar, Anand Bagmar demonstrates how AI tools such as ChatGPT can be applied to various stages of the software development life cycle (SDLC) using an eCommerce application case study. Find the on-demand recording and more info at https://applitools.info/b59 Key takeaways: • Learn how to use ChatGPT to add AI power to your testing and test automation • Understand the limitations of the technology and where human expertise is crucial • Gain insight into different AI-based tools • Adopt AI-based tools to stay relevant and optimize work for developers and testers * ChatGPT and OpenAI belong to OpenAI, L.L.C.

1. @BagmarAnand
Unlocking the Power
of
ChatGPT and AI in Testing
A Real-World Look
Anand Bagmar
Software Quality Evangelist

2. ABOUT ME
BagmarAnand
linkedin.com/in/anandbagmar/

3. Hype in AI

4. Gartner’s Hype Cycle
for AI, 2021 report
https://www.gartner.com/en/articles/the-4-trends-that-prevail-
on-the-gartner-hype-cycle-for-ai-2021

5. https://www.gartner.com/en/articles/the-4-trends-that-prevail-on-the-gartner-hype-cycle-for-ai-2021

6. AGENDA
• What is ChatGPT?
• Using ChatGPT in Testing
• Using ChatGPT in Programming
• Limitations of ChatGPT
• Other AI Tools you can use
• Pitfalls of AI
• Summary
• Q&A

7. @BagmarAnand
ChatGPT
https://openai.com/blog/chatgpt/

8. @BagmarAnand
ChatGPT – in its own words

9. @BagmarAnand
ChatGPT is evolving!

10. @BagmarAnand
GPT-3
https://en.wikipedia.org/wiki/GPT-3

11. @BagmarAnand
ChatGPT – Interesting Stats
https://www.demandsage.com/chatgpt-statistics/
• ChatGPT was launched on 30th November 2022.
• The new and improved embedding model of ChatGPT was launched on
15th December 2022.
• ChatGPT has crossed one million users within a week of its launch.
• ChatGPT can only fetch data prior to the year 2021 as its training
stopped in the year 2021.
• Microsoft Azure supports OpenAI and provides them with the
computational power required for running ChatGPT.

12. ChatGPT in Testing
Example: Testing an eCommerce application – Amazon (USA)

13. @BagmarAnand
Testing Approach
• Getting started
• Framework
• Design patterns
• Code optimization
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

14. @BagmarAnand
Testing - Strategy
• Getting started
• Framework
• Design patterns
• Code optimization
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

15. @BagmarAnand
Testing - Strategy
Question asked:
Create a test strategy for
testing and automating
Amazon USA

16. @BagmarAnand
Testing - Strategy
Question asked:
Create a test strategy for
testing and automating
Amazon USA

17. @BagmarAnand
Testing - Strategy
Question asked:
Create a test strategy for
testing and automating
Amazon USA This is not useful.
Seems like a template

18. @BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case

19. @BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case

20. @BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case

21. @BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case

22. @BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case This is promising.
A great start!

23. @BagmarAnand
Testing – Strategy with Risk Mitigation
Next question:
What risks and their
mitigation strategies
should be included in
the above strategy?

24. @BagmarAnand
Testing – Strategy with Risk Mitigation
Next question:
What risks and their
mitigation strategies
should be included in
the above strategy?

25. @BagmarAnand
Testing – Strategy with Risk Mitigation
Next question:
What risks and their
mitigation strategies
should be included in
the above strategy?

26. @BagmarAnand
Testing – Test Design
• Getting started
• The 1st Test
• Design
patterns
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

27. @BagmarAnand
Test Design – Test Scenarios
Next question:
What are the most
critical test scenarios I
should test for this
website?

28. @BagmarAnand
Test Design – Test Scenarios
Next question:
What are the most
critical test scenarios I
should test for this
website?

29. @BagmarAnand
Test Design – Test Scenarios
Next question:
What are the most
critical test scenarios I
should test for this
website?
Great set of regression
scenarios!

30. @BagmarAnand
Test Design – Test Cases
Next question:
List 10 test cases for
"Product Search and
Filtering" scenario in
order of priority

31. @BagmarAnand
Test Design – Test Cases
Next question:
List 10 test cases for
"Product Search and
Filtering" scenario in
order of priority

32. @BagmarAnand
Test Design – Test Cases
Next question:
List 10 test cases for
"Product Search and
Filtering" scenario in
order of priority
Isn’t this amazing?

33. @BagmarAnand
Test Design – Test Data
Next question:
What test data do I
need to be available
for performing the
above test cases?

34. @BagmarAnand
Test Design – Test Data
Next question:
What test data do I
need to be available
for performing the
above test cases?

35. @BagmarAnand
Test Design – Test Data
Next question:
What test data do I
need to be available
for performing the
above test cases?
Well – I was hoping for more
specific, but it’s a start!

36. @BagmarAnand
Testing - Automation
• Getting started
• The 1st
Test
• Design patterns
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

37. @BagmarAnand
Automation – Strategy
Next question:
What is a good way to
automate the above
test cases?

38. @BagmarAnand
Automation – Strategy
Next question:
What is a good way to
automate the above
test cases?

39. @BagmarAnand
Automation – Getting Started
Next question:
I don't know Selenium.
How should I learn?

40. @BagmarAnand
Automation – Getting Started
Next question:
I don't know Selenium.
How should I learn?

41. @BagmarAnand
Automation – Getting Started
Next question:
I don't know Selenium.
How should I learn? This answer took some time
TestAutomationUniversity
(TAU) was missed!

42. @BagmarAnand
Automation – The 1st Test
Next question:
Implement the 1st test
case using Selenium
Java with Junit

43. @BagmarAnand
Automation – The 1st Test
Next question:
Implement the 1st test
case using Selenium
Java with Junit

44. @BagmarAnand
Automation – The 1st Test
Next question:
Implement the 1st test
case using Selenium
Java with Junit

45. @BagmarAnand
Automation – The 1st Test
Next question:
Implement the 1st test
case using Selenium
Java with Junit

46. @BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

47. @BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

48. @BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

49. @BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

50. @BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

51. @BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

52. @BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

53. @BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

54. @BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

55. @BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

56. @BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

57. @BagmarAnand
Testing – Test Execution
• Getting started
• The 1st Test
• Design
patterns
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

58. @BagmarAnand
Test Execution – Reports
Next question:
How to generate
reports and metrics for
automated tests?

59. @BagmarAnand
Test Execution – Reports
Next question:
How to generate
reports and metrics for
automated tests?

60. @BagmarAnand
Test Execution – Reports
Next question:
How to generate
reports and metrics for
automated tests?

61. @BagmarAnand
Test Execution – Reports
Next question:
How to generate
reports and metrics for
automated tests?
This is not what I was looking for!

62. @BagmarAnand
Test Execution – Reports
Follow-up question:
Show an example to
generate JunitReports
for the automated
tests

63. @BagmarAnand
Test Execution – Reports
Follow-up question:
Show an example to
generate JunitReports
for the automated
tests

64. @BagmarAnand
Test Execution – Reports
Follow-up question:
Show an example to
generate JunitReports
for the automated
tests

65. ChatGPT in Programming

66. @BagmarAnand
ChatGPT in Programming
Writing
code
Refactoring
Debugging
...

67. @BagmarAnand
Writing code
Writing
code
Refactoring
Debugging
...

68. @BagmarAnand
Writing code
Question asked:
Implement code for count-
and-say
https://leetcode.com/problems/count-and-say/

69. @BagmarAnand
Writing code
Question asked:
Implement code for count-
and-say
https://leetcode.com/problems/count-and-say/

70. @BagmarAnand
Writing code
Question asked:
Implement code for count-
and-say
https://leetcode.com/problems/count-and-say/

71. @BagmarAnand
Writing code
Question asked:
Implement code for count-
and-say
https://leetcode.com/problems/count-and-say/

72. @BagmarAnand
Writing code
Follow-up asked:
Give above solution in Java

73. @BagmarAnand
Writing code
Follow-up asked:
Give above solution in Java

74. @BagmarAnand
Writing code
Follow-up asked:
Give above solution in Java
This actually works J

75. @BagmarAnand
Writing code
Follow-up asked:
Use proper variable names
for above code

76. @BagmarAnand
Writing code
Follow-up asked:
Use proper variable names
for above code

77. @BagmarAnand
Writing code
Follow-up asked:
Use proper variable names
in the loops

78. @BagmarAnand
Writing code
Follow-up asked:
Use proper variable names
in the loops

79. @BagmarAnand
Refactoring
Writing code Refactoring
Debugging
...

80. @BagmarAnand
Refactor code
Question asked:
Given a link to code in
Github, refactor it

81. @BagmarAnand
Refactor code
Question asked:
Given a link to code in
Github, refactor it

82. @BagmarAnand
Refactor code
Question asked:
Given a link to code in
Github, refactor it
I have not verified the
validity of this refactoring

83. @BagmarAnand
Question asked:
Given a snippet of
code, refactor it
Refactor code

84. @BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it

85. @BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it

86. @BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it

87. @BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it
Large output pauses abruptly
”continue” proceeds

88. @BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it

89. @BagmarAnand
Complex code
Question asked:
Given a complex code,
asked ChatGPT to
refactor the code, and
then reduce the
complexity

90. @BagmarAnand
Complex code
Question asked:
Given a complex code,
asked ChatGPT to
refactor the code, and
then reduce the
complexity

91. @BagmarAnand
Complex code
Question asked:
Given a complex code,
asked ChatGPT to
refactor the code, and
then reduce the
complexity

92. @BagmarAnand
Complex code
Follow-up question:
Reduce complexity of
above code

93. @BagmarAnand
Complex code
Follow-up question:
Reduce complexity of
above code

94. @BagmarAnand
Debugging
Writing
code
Refactoring
Debugging
...

95. @BagmarAnand
Debugging
Question asked:
Given a buggy code, fix it

96. @BagmarAnand
Debugging
Question asked:
Given a buggy code, fix it

97. @BagmarAnand
Debugging
Question asked:
Given a buggy code, fix it

98. @BagmarAnand
Debugging
Question asked:
Given a buggy code, fix it
Not only fixed the code, ChatGPT
also explained the error

99. Limitations of ChatGPT

100. @BagmarAnand
Limitations of ChatGPT
https://openai.com/blog/chatgpt/

101. @BagmarAnand
Limitations based on my observations
• ChatGPT does not know your context – use this as a starting point
• Will not work on internal sites
• Data privacy could become a huge issue
• Will not work for new features that are being developed

102. AI tools impacting SDLC

103. @BagmarAnand
Github Copilot
• Github Copilot is not new
• It is super helpful to write clean code, and rapidly!
• It is helpful to implement product code, as well as automation code
• Refer to these links for more information about it:
• https://applitools.com/blog/using-github-copilot-to-automate-tests/
• https://applitools.com/event/make-testing-easy-github-copilot/
• https://github.com/features/copilot

104. @BagmarAnand
Applitools Visual AI
Applitools Ultrafast Cloud
applitools.info/b8h

105. @BagmarAnand
Visual AI simplifies functional testing

106. @BagmarAnand
Visual AI simplifies functional testing

107. @BagmarAnand
Visual AI handles dynamic content

108. @BagmarAnand
Visual AI handles dynamic content

109. @BagmarAnand
Visual AI handles shifting content

110. @BagmarAnand
Visual AI handles shifting content

111. @BagmarAnand
Visual AI handles shifting content

112. @BagmarAnand
Visual AI handles shifting content

113. @BagmarAnand
Visual AI accelerates
cross browser/device testing
Capture the entire UI with
a single snapshot
DOM & CSS sent to
Applitools
Ultrafast Test Cloud
Applitools Eyes
Visual AI analysis
Ultrafast Grid
renders DOM
snapshots in parallel
Login to Eyes to
view test results
.
Firewall
18.2x faster than traditional cloud execution grids.
Applitools
Ultrafast Test Cloud

114. @BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

115. @BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

116. @BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

117. @BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

118. @BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

119. @BagmarAnand
Automation – With Applitools Visual AI
While this is not the
complete test, it
shows the correct
setup of Applitools
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

120. @BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

121. @BagmarAnand
Advantages of AI in Testing
• Reduce mundane tasks – let the machine handle it!
• Reduce the error prone nature of manual testing
• Easier Scaling
• Learn new tools & technologies
• Make work more interesting

122. Potential Pitfalls of AI

123. @BagmarAnand
Potential pitfalls of AI
• Thinking AI will solve all problems!
• Results in superficial knowledge
• Data Privacy
https://twitter.com/ProtonPrivacy/status/1619007351750627342

124. Current Sentiment

125. @BagmarAnand
https://nypost.com/2023/01/25/chat-gpt-could-make-these-jobs-obsolete/
https://www.semafor.com/article/01/27/2023/openai-has-hired-an-army-of-contractors-to-make-basic-coding-obsolete
Buzz in social media

126. @BagmarAnand
Are you excited?
https://storage.needpix.com/rsynced_images/excited-2681489_1280.jpg
https://www.lavorareascuola.it/wp-content/uploads/2020/05/thinking-2681494_960_720.jpg
Current Sentiments
Or
Are you worried?

127. Summary
• ChatGPT and many other tools in the AI-space are solving very
interesting challenges
• Understand how to use them in your context
• Ask the right question, to get a good answer!
• Tools have limitations – choose them wisely
• Human mind & experience is “still” the essential ingredient in
SDLC

128. Some tips for you
Use this opportunity to:
• Upskill yourself in new tools and technologies
• Evolve your thought process
• Be creative and use new information to come with
unique solutions
• Be innovative and learn how to use tools and
technologies for different purposes

129. https://twitter.com/paulg/status/1618676092163997696?t=J4XUNYJ8ynidWgrUW02Ing&s=03

130. And a New Challenge
Have you thought of how will you test these new
AI tools and technologies?

131. HI + AI = SI
Human Intelligence + Artificial Intelligence = Super Intelligence

132. QUESTIONS?
BagmarAnand
linkedin.com/in/anandbagmar/

133. QUESTIONS?
Talk with a Testing Specialist
applitools.info/lbd

134. TRY APPLITOOLS
Try the only truly AI-driven
Visual Testing Platform with
a free-forever account
applitools.info/xm3

135. @BagmarAnand
Additional Learning
applitools.info/cdf

136. @BagmarAnand
Coming in March
applitools.info/7mp
TAU Conference
March 8-9
Let the Engineers Speak
March 16

137. @BagmarAnand
• https://applitools.com/blog/ai-the-magical-helping-hand-in-testing/
• https://applitools.com
• https://openai.com/blog/chatgpt/
• https://github.com/features/copilot
Resources

138. Thank You!
Join us in the lounge for additional networking