1. @BagmarAnand
Unlocking the Power
of
ChatGPT and AI in Testing
A Real-World Look
Anand Bagmar
Software Quality Evangelist

2. ABOUT ME
BagmarAnand
linkedin.com/in/anandbagmar/

3. Hype in AI

4. Gartner’s Hype Cycle
for AI, 2021 report
https://www.gartner.com/en/articles/the-4-trends-that-prevail-
on-the-gartner-hype-cycle-for-ai-2021

5. https://www.gartner.com/en/articles/the-4-trends-that-prevail-on-the-gartner-hype-cycle-for-ai-2021

6. AGENDA
• What is ChatGPT?
• Using ChatGPT in Testing
• Using ChatGPT in Programming
• Limitations of ChatGPT
• Other AI Tools you can use
• Pitfalls of AI
• Summary
• Q&A

7. @BagmarAnand
ChatGPT
https://openai.com/blog/chatgpt/

8. @BagmarAnand
ChatGPT – in its own words

9. @BagmarAnand
ChatGPT is evolving!

10. @BagmarAnand
GPT-3
https://en.wikipedia.org/wiki/GPT-3

11. @BagmarAnand
ChatGPT – Interesting Stats
https://www.demandsage.com/chatgpt-statistics/
• ChatGPT was launched on 30th November 2022.
• The new and improved embedding model of ChatGPT was launched on
15th December 2022.
• ChatGPT has crossed one million users within a week of its launch.
• ChatGPT can only fetch data prior to the year 2021 as its training
stopped in the year 2021.
• Microsoft Azure supports OpenAI and provides them with the
computational power required for running ChatGPT.

12. ChatGPT in Testing
Example: Testing an eCommerce application – Amazon (USA)

13. @BagmarAnand
Testing Approach
• Getting started
• Framework
• Design patterns
• Code optimization
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

14. @BagmarAnand
Testing - Strategy
• Getting started
• Framework
• Design patterns
• Code optimization
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

15. @BagmarAnand
Testing - Strategy
Question asked:
Create a test strategy for
testing and automating
Amazon USA

16. @BagmarAnand
Testing - Strategy
Question asked:
Create a test strategy for
testing and automating
Amazon USA

17. @BagmarAnand
Testing - Strategy
Question asked:
Create a test strategy for
testing and automating
Amazon USA This is not useful.
Seems like a template

18. @BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case

19. @BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case

20. @BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case

21. @BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case

22. @BagmarAnand
Testing - Strategy
Follow-up question:
Give a concrete
strategy for the above
use case This is promising.
A great start!

23. @BagmarAnand
Testing – Strategy with Risk Mitigation
Next question:
What risks and their
mitigation strategies
should be included in
the above strategy?

24. @BagmarAnand
Testing – Strategy with Risk Mitigation
Next question:
What risks and their
mitigation strategies
should be included in
the above strategy?

25. @BagmarAnand
Testing – Strategy with Risk Mitigation
Next question:
What risks and their
mitigation strategies
should be included in
the above strategy?

26. @BagmarAnand
Testing – Test Design
• Getting started
• The 1st Test
• Design
patterns
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

27. @BagmarAnand
Test Design – Test Scenarios
Next question:
What are the most
critical test scenarios I
should test for this
website?

28. @BagmarAnand
Test Design – Test Scenarios
Next question:
What are the most
critical test scenarios I
should test for this
website?

29. @BagmarAnand
Test Design – Test Scenarios
Next question:
What are the most
critical test scenarios I
should test for this
website?
Great set of regression
scenarios!

30. @BagmarAnand
Test Design – Test Cases
Next question:
List 10 test cases for
"Product Search and
Filtering" scenario in
order of priority

31. @BagmarAnand
Test Design – Test Cases
Next question:
List 10 test cases for
"Product Search and
Filtering" scenario in
order of priority

32. @BagmarAnand
Test Design – Test Cases
Next question:
List 10 test cases for
"Product Search and
Filtering" scenario in
order of priority
Isn’t this amazing?

33. @BagmarAnand
Test Design – Test Data
Next question:
What test data do I
need to be available
for performing the
above test cases?

34. @BagmarAnand
Test Design – Test Data
Next question:
What test data do I
need to be available
for performing the
above test cases?

35. @BagmarAnand
Test Design – Test Data
Next question:
What test data do I
need to be available
for performing the
above test cases?
Well – I was hoping for more
specific, but it’s a start!

36. @BagmarAnand
Testing - Automation
• Getting started
• The 1st
Test
• Design patterns
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

37. @BagmarAnand
Automation – Strategy
Next question:
What is a good way to
automate the above
test cases?

38. @BagmarAnand
Automation – Strategy
Next question:
What is a good way to
automate the above
test cases?

39. @BagmarAnand
Automation – Getting Started
Next question:
I don't know Selenium.
How should I learn?

40. @BagmarAnand
Automation – Getting Started
Next question:
I don't know Selenium.
How should I learn?

41. @BagmarAnand
Automation – Getting Started
Next question:
I don't know Selenium.
How should I learn? This answer took some time
TestAutomationUniversity
(TAU) was missed!

42. @BagmarAnand
Automation – The 1st Test
Next question:
Implement the 1st test
case using Selenium
Java with Junit

43. @BagmarAnand
Automation – The 1st Test
Next question:
Implement the 1st test
case using Selenium
Java with Junit

44. @BagmarAnand
Automation – The 1st Test
Next question:
Implement the 1st test
case using Selenium
Java with Junit

45. @BagmarAnand
Automation – The 1st Test
Next question:
Implement the 1st test
case using Selenium
Java with Junit

46. @BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

47. @BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

48. @BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

49. @BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

50. @BagmarAnand
Automation – With WebDriverManager
Next question:
Use
WebDriverManager in
the above example

51. @BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

52. @BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

53. @BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

54. @BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

55. @BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

56. @BagmarAnand
Automation – Design Patterns
Next question:
Implement the above
test using Page-
Object-Model design
pattern

57. @BagmarAnand
Testing – Test Execution
• Getting started
• The 1st Test
• Design
patterns
• Reporting
• Metrics
• Test scenarios
• Test cases
• Test data
• Risk
• Coverage
• Metrics
Strategy Test Design
Automation
Test
execution

58. @BagmarAnand
Test Execution – Reports
Next question:
How to generate
reports and metrics for
automated tests?

59. @BagmarAnand
Test Execution – Reports
Next question:
How to generate
reports and metrics for
automated tests?

60. @BagmarAnand
Test Execution – Reports
Next question:
How to generate
reports and metrics for
automated tests?

61. @BagmarAnand
Test Execution – Reports
Next question:
How to generate
reports and metrics for
automated tests?
This is not what I was looking for!

62. @BagmarAnand
Test Execution – Reports
Follow-up question:
Show an example to
generate JunitReports
for the automated
tests

63. @BagmarAnand
Test Execution – Reports
Follow-up question:
Show an example to
generate JunitReports
for the automated
tests

64. @BagmarAnand
Test Execution – Reports
Follow-up question:
Show an example to
generate JunitReports
for the automated
tests

65. ChatGPT in Programming

66. @BagmarAnand
ChatGPT in Programming
Writing
code
Refactoring
Debugging
...

67. @BagmarAnand
Writing code
Writing
code
Refactoring
Debugging
...

68. @BagmarAnand
Writing code
Question asked:
Implement code for count-
and-say
https://leetcode.com/problems/count-and-say/

69. @BagmarAnand
Writing code
Question asked:
Implement code for count-
and-say
https://leetcode.com/problems/count-and-say/

70. @BagmarAnand
Writing code
Question asked:
Implement code for count-
and-say
https://leetcode.com/problems/count-and-say/

71. @BagmarAnand
Writing code
Question asked:
Implement code for count-
and-say
https://leetcode.com/problems/count-and-say/

72. @BagmarAnand
Writing code
Follow-up asked:
Give above solution in Java

73. @BagmarAnand
Writing code
Follow-up asked:
Give above solution in Java

74. @BagmarAnand
Writing code
Follow-up asked:
Give above solution in Java
This actually works J

75. @BagmarAnand
Writing code
Follow-up asked:
Use proper variable names
for above code

76. @BagmarAnand
Writing code
Follow-up asked:
Use proper variable names
for above code

77. @BagmarAnand
Writing code
Follow-up asked:
Use proper variable names
in the loops

78. @BagmarAnand
Writing code
Follow-up asked:
Use proper variable names
in the loops

79. @BagmarAnand
Refactoring
Writing code Refactoring
Debugging
...

80. @BagmarAnand
Refactor code
Question asked:
Given a link to code in
Github, refactor it

81. @BagmarAnand
Refactor code
Question asked:
Given a link to code in
Github, refactor it

82. @BagmarAnand
Refactor code
Question asked:
Given a link to code in
Github, refactor it
I have not verified the
validity of this refactoring

83. @BagmarAnand
Question asked:
Given a snippet of
code, refactor it
Refactor code

84. @BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it

85. @BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it

86. @BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it

87. @BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it
Large output pauses abruptly
”continue” proceeds

88. @BagmarAnand
Refactor code
Question asked:
Given a snippet of
code, refactor it

89. @BagmarAnand
Complex code
Question asked:
Given a complex code,
asked ChatGPT to
refactor the code, and
then reduce the
complexity

90. @BagmarAnand
Complex code
Question asked:
Given a complex code,
asked ChatGPT to
refactor the code, and
then reduce the
complexity

91. @BagmarAnand
Complex code
Question asked:
Given a complex code,
asked ChatGPT to
refactor the code, and
then reduce the
complexity

92. @BagmarAnand
Complex code
Follow-up question:
Reduce complexity of
above code

93. @BagmarAnand
Complex code
Follow-up question:
Reduce complexity of
above code

94. @BagmarAnand
Debugging
Writing
code
Refactoring
Debugging
...

95. @BagmarAnand
Debugging
Question asked:
Given a buggy code, fix it

96. @BagmarAnand
Debugging
Question asked:
Given a buggy code, fix it

97. @BagmarAnand
Debugging
Question asked:
Given a buggy code, fix it

98. @BagmarAnand
Debugging
Question asked:
Given a buggy code, fix it
Not only fixed the code, ChatGPT
also explained the error

99. Limitations of ChatGPT

100. @BagmarAnand
Limitations of ChatGPT
https://openai.com/blog/chatgpt/

101. @BagmarAnand
Limitations based on my observations
• ChatGPT does not know your context – use this as a starting point
• Will not work on internal sites
• Data privacy could become a huge issue
• Will not work for new features that are being developed

102. AI tools impacting SDLC

103. @BagmarAnand
Github Copilot
• Github Copilot is not new
• It is super helpful to write clean code, and rapidly!
• It is helpful to implement product code, as well as automation code
• Refer to these links for more information about it:
• https://applitools.com/blog/using-github-copilot-to-automate-tests/
• https://applitools.com/event/make-testing-easy-github-copilot/
• https://github.com/features/copilot

104. @BagmarAnand
Applitools Visual AI
Applitools Ultrafast Cloud
applitools.info/b8h

105. @BagmarAnand
Visual AI simplifies functional testing

106. @BagmarAnand
Visual AI simplifies functional testing

107. @BagmarAnand
Visual AI handles dynamic content

108. @BagmarAnand
Visual AI handles dynamic content

109. @BagmarAnand
Visual AI handles shifting content

110. @BagmarAnand
Visual AI handles shifting content

111. @BagmarAnand
Visual AI handles shifting content

112. @BagmarAnand
Visual AI handles shifting content

113. @BagmarAnand
Visual AI accelerates
cross browser/device testing
Capture the entire UI with
a single snapshot
DOM & CSS sent to
Applitools
Ultrafast Test Cloud
Applitools Eyes
Visual AI analysis
Ultrafast Grid
renders DOM
snapshots in parallel
Login to Eyes to
view test results
.
Firewall
18.2x faster than traditional cloud execution grids.
Applitools
Ultrafast Test Cloud

114. @BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

115. @BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

116. @BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

117. @BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

118. @BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

119. @BagmarAnand
Automation – With Applitools Visual AI
While this is not the
complete test, it
shows the correct
setup of Applitools
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

120. @BagmarAnand
Automation – With Applitools Visual AI
Next question:
Implement the 1st test
case "Verify that the
search results match the
keywords entered by the
user" using Selenium Java
with WebDriverManager,
Junit and Applitools
Ultrafast Grid

121. @BagmarAnand
Advantages of AI in Testing
• Reduce mundane tasks – let the machine handle it!
• Reduce the error prone nature of manual testing
• Easier Scaling
• Learn new tools & technologies
• Make work more interesting

122. Potential Pitfalls of AI

123. @BagmarAnand
Potential pitfalls of AI
• Thinking AI will solve all problems!
• Results in superficial knowledge
• Data Privacy
https://twitter.com/ProtonPrivacy/status/1619007351750627342

124. Current Sentiment

125. @BagmarAnand
https://nypost.com/2023/01/25/chat-gpt-could-make-these-jobs-obsolete/
https://www.semafor.com/article/01/27/2023/openai-has-hired-an-army-of-contractors-to-make-basic-coding-obsolete
Buzz in social media

126. @BagmarAnand
Are you excited?
https://storage.needpix.com/rsynced_images/excited-2681489_1280.jpg
https://www.lavorareascuola.it/wp-content/uploads/2020/05/thinking-2681494_960_720.jpg
Current Sentiments
Or
Are you worried?

127.