https://event.shoeisha.jp/devsumi/20140213/session/368/

1. 13-­‐A-­‐6 #devsumiA
Mobageを支える
テストエンジニアリング
2014/02/13
!
Masaki Nakagawa
Manager, SWET Gr.
Platform Unit System Dept.
Mobage Business Sector Japan Region
DeNA Co., Ltd.
1
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

2. About Me
• Masaki Nakagawa
• @ikasam_a
• github.com/masaki
• CPAN Author
• Testing Casual
(metacpan.org/author/MASAKI)
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

3. http://gihyo.jp/magazine/wdpress/archive/2013/vol75
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

4. http://gihyo.jp/magazine/wdpress/archive/2013/vol77
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

5. http://atnd.org/events/40914
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

6. Mobage統合事業本部
Japanリージョン事業本部
プラットフォーム本部
システム部
SWETグループ
6
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

7. 今日話すこと
•
Mobage Open Platform について
•
SWETグループを作った背景
•
•
SWET って何？
SWETグループって何やってるの？
7
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

8. 今日話さないこと
•
Mobage Open Platform の詳細
•
特定言語に特化した話
•
テストコードの詳細
8
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

9. Mobage Open Platform について
9
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

10. Mobage Open Platform とは？
•
http://developer.dena.jp/mbga/
•
Mobageでゲームを公開できる仕組み
•
色々なデバイスに対応
•
Feature Phone
•
Smartphone
•
PC
10
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

11. Mobage Open Platform の歴史
•
2009/08 Open Platform Project スタート
•
2010/01 JP Platform (Feature Phone)
•
2010/10 JP Platform (Y!Mobage / PC)
•
2011/02 JP Platform (Smartphone Web)
•
2011/06 JP Platform (Smartphone App)
•
2012/12 JP Platform (ShellApp)
•
2013/05 JP Platform (mixi Mobage / Smartphone)
11
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

12. Mobage Open Platform の歴史
•
2011/06 WW Platform (Smartphone App)
•
2011/09 CN Platform (Smartphone App)
•
2012/02 KR Platform (Smartphone App)
•
2012/11 KR Platform (ShellApp)
•
2012/11 TC Platform (Smartphone App)
•
2013/01 CN Platform (ShellApp)
•
2013/02 KR Platform (Smartphone Web)
12
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

13. Works
• (2011/10) Joined DeNA Co.,Ltd.
• (2012/01) Platform QA Team
• (2013/08) SWET Group
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

14. SWETグループを作った背景
14
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

15. Works
• (2011/10) Joined DeNA Co.,Ltd.
• (2012/01) Platform QA Team
• (2013/08) SWET Group
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

16. Mobage Open Platform の歴史(2)
•
2011/06 WW Platform (Smartphone App)
•
2011/09 CN Platform (Smartphone App)
•
2012/02 KR Platform (Smartphone App)
•
2012/11 KR Platform (ShellApp)
•
2012/11 TC Platform (Smartphone App)
•
2013/01 CN Platform (ShellApp)
•
2013/02 KR Platform (Smartphone Web)
16
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

17. プラットフォームのグローバル展開
17
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

18. 立ち上げの背景
•
プラットフォームのグローバル展開
•
大規模システムの拡張とリファクタリング
•
デリバリーのスピードを落とさない
•
検証属人性の解消
18
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

19. 立ち上げの方針
•
End-to-End テストを確立する
•
テストを徹底的に自動化する
•
テストしやすい環境を提供する
19
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

20. ∼テストしやすい環境∼
•
単体テストのREDが消えない問題
•
リリース頻度・速度・影響範囲のバランス
•
テスト時間のコスト問題
•
CIの必要性
20
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

21. チーム立ち上げ
•
当初はQAチームとして出発
•
•
現在はSWETグループ（正式に組織化）
•
•
2012/01 より 3人でスタート
2014/02 現在 16人
なぜ独立したチームにしたか？
•
横串チームによる「戦略的横展開」を狙う
21
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

22. SWET って何？
22
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

23. Works
• (2011/10) Joined DeNA Co.,Ltd.
• (2012/01) Platform QA Team
• (2013/08) SWET Group
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

24. http://en.wikipedia.org/wiki/SET
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

25. a
Quality Assurance
job title
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

26. Names
• SET
• SDET
• SWET
(Software Engineer in Test)
by @Google
(Software Design Engineer in Test)
by @Microsoft
by @DeNA
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

27. Definition
•
Google Testing Blog
•
How Google Tests Software
• http://googletesting.blogspot.com
• http://googletesting.blogspot.jp/2011/01/how-google-testssoftware.html
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

28. http://googletesting.blogspot.jp/2011/02/how-googletests-software-part-two.html
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

29. the SET
• a developer role focused on testability
• review designs
• refactor code to testable
• write testing frameworks
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

30. http://googletesting.blogspot.jp/2011/02/how-googletests-software-part-two.html
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

31. the TE
• a role that puts testing first
• write automation scripts
• interpret test results
• drive test execution
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

32. http://googletesting.blogspot.jp/2011/02/how-googletests-software-part-two.html
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

33. Focus
•
SETs primary focus is on the developer
• Individual feature quality
Enabling developers to easily test
•
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

34. Developer
Productivity
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

35. our SWET = SET + TE
•
•
Developer Productivity
Quality Assurance
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

36. SWET Group
Mission Statement
•
• Improve the quality and
Keep the quality of Platform
productivity of Platform
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

37. As an Engineer
• can be developed test target
NOT a tester but a test engineer
•
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

38. SWETグループって何やってるの？
38
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

39. Our Works
•
Client Side Testing
•
…
•
Server Side Testing
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

40. http://perl-users.jp/articles/advent-calendar/2011/test/3
!
Perl Advent Calendar Japan 2011
Test Track - day 3
40
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

41. Classification of Testing
Test
Levels
Who
Which
How
What
Test
Design
Techniques
Test
Types
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

42. Who
Developer or User (Customer)
• Developer Testing
Acceptance Testing
•
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

43. Which
One or More
• Unit Testing
Integration Testing
•
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

44. How
Techniques for writing tests
•
White Box Testing
•
Gray Box Testing
•
Black Box Testing
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

45. What
the Purpose of testing
•
Non-Functional Testing
•
Functional Testing
• Performance / Stress / Usability / ...
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

46. Open Platform Architecture
Mobage
Mobile
App
SDK
Game
Server
Mobage
API
RESTful API
Mobile
Browser
API
mbga.jp
API
ProxyServer
Databases / Middlewares
Mobage Developers
PC
Browser
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

47. Targets
1. Mobage RESTful API (Web API)
2. Mobage Developers (Web Application)
3. mbga.jp ProxyServer (Mobile Web)
4. Mobage SDKs (Client SDK)
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

48. A) WebAPI
Test Client
API Server
module
API Server
API Server
DB
module
module
API Server
API Server
Cache
48
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

49. Conditions
• HTTP Request to API Server
• JSON Response from API Server
• no UI
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

50. Gray Box Fixture
Test Client
API Server
module
API Server
API Server
DB
module
module
API Server
API Server
Cache
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

51. Gray Box Fixture
• DB / Cache Manipulation
• insert test-case specific data
• delete cache for continuous tests
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

52. Specific Targeting
Test Client
API Server
API Server
API Server
module
API Server
API Server
DB
module
module
API Server
API Server
Cache
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

53. Specific Targeting
• programmable local DNS (MyDNS)
• emulates “inet_aton”
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

54. Domain Specific Client
• Mobage::API::Client (Perl)
• mobage-api-client (Ruby)
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

55. Message Dumping
• keep previous request in response
• interactive debugging
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

56. B) WebApp / Mobile Web
Test Browser
Web Application
M
API Server
API Server
DB
V
C
API Server
API Server
Cache
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

57. Conditions
• HTTP Request to Web App
• HTML Response from Web App
• with JavaScript
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

58. Basic Strategy
• Selenium WebDriver
• Ruby, RSpec, Capybara
• Headless Browser
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

59. Agent Switching
Browser
Emulator
Web Application
M
API Server
API Server
DB
V
C
API Server
API Server
Cache
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

60. Agent Switching
• Browser can’t touch HTTP Header
• Emulator can’t parse JavaScript
• Save/Restore pseudo agent state
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

61. Domain Specific Client
• called “mobage-browser”
• We don’t use “Page Object Pattern”
• “Domain Agent Pattern”
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

62. Mobile Emulation
• FireMobileSimulator
• Mobile WebDrivers (DEPRECATED)
• AndroidDriver, IPhoneDriver
• with Real Devices
• Selendroid, ios-driver, Appium
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

63. C) Client SDKs
Test Scripts
Test
Application
Mobage
SDK
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

64. Conditions
• create test apps using specific SDK
• use SDK as a developer
• run test scripts to automate test apps
• check SDK-API behaviors
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

65. UI Automation
• Multiple Devices Support
• Android, iOS
• Test Cases Consistency
• Calabash
• Appium
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

66. http://gihyo.jp/magazine/wdpress/archive/2013/vol77
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

67. Ex) Service Integration
• “Notification from Games” feature
• Game can broadcast notification
to users who is playing
• User can receive notification
at Mobage Chat Web
• User can toggle settings
to receive notification or not at mbga.jp
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

68. Scenario
Feature: Notification from Games
Scenario: User can receive notification
Given user login to mbga.jp
When user visit to settings page
And user enable to “notification” setting
And game broadcast notification using API
And user visit to Mobage Chat page
Then user can show notification message
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

69. Scenario
Feature: Notification from Games
Scenario: User can ignore notification
Given user login to mbga.jp
When user visit to settings page
And user disable to “notification” setting
And game broadcast notification using API
And user visit to Mobage Chat page
Then user can not show notification message
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

70. Multiple Sessions
• Game
• using “mobage-api-client”
• User
• using “mobage-browser”
• 3 Domain Specific Client
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

71. 今日話したこと
•
Mobage Open Platform について
•
SWETグループを作った背景
•
•
SWET って何？
SWETグループって何やってるの？
71
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.

72. ご清聴ありがとうございました
72
Copyright (C) 2013 DeNA Co.,Ltd. All Rights Reserved.