Resource: LCE13 Name: Introduction to QA Services and Case Study Date: 11-07-2013 Speaker: Milosz Wasilewski Video: https://www.youtube.com/watch?v=x1z0d82UZmA

1. Linaro QA Services
LCE13 - Dublin, July 2013

2. QA Services
Tasks (what do we do?):
• Manual testing
• Dashboard monitoring
• Reporting
• Porting tests to LAVA
Things that we (usually) don't do:
• creating new test cases
• fixing bugs in the software

3. Currently Active Tasks
• linux-linaro Android validation
• linux-linaro ubuntu validation
• Linaro OpenEmbedded validation (ARMv8 Foundation Model)
• Upstream "linux-next" branch monitoring & validation
• Upstream "linux-arm-soc-for-next" branch monitoring & validation
• big.LITTLE
– Functional and performance Testing & Reporting
– Test suite maintenance
• Test suites porting to LAVA

4. • Weekly Test
– Manually Test:
– Linux Linaro Android: TI Panda 4430 & 4460, Samsung Arndale,
Samsung Galaxy Nexus, ARM Versatile Express A9 & TC2
– Linux Linaro Ubuntu: TI Panda 4430 & 4460, Samsung Arndale, ARM
Versatile Express A9
– Linaro OpenEmbedded: LAMP & Minimal
– Consolidated results sent to broad audience
• Monthly Release Test
– Run all test manually as same as weekly test
– Sign-off the release
– Results released to broad audience
Manual Testing

5. • On-demand Test:
– Agreed with engineering teams
– If a new target comes in, it may become a member of regular test
target. We also accept one shot test if it's necessary
Manual Testing

6. • Linaro Android Test Case Automation
– "Command Line" test cases are expected to be automated
– UI related test cases (Browser, Email, YouTube, etc.) are hard to
automate and probably not worth to do
• Linux Linaro ubuntu Test Case Automation
– We're currently using headless (no UI) Ubuntu images
– "Command Line" test cases are expected to be automated
– Some test cases depend on hardware actions, probably LMP can help
here.
Test Case Automation

7. • Virtualization
• piglit (most of the job done by Tom Gall)
Test Case porting to LAVA

8. • Agreed on LCA-13 (http://cards.linaro.org/browse/CARD-521)
• CI loop builds for the following devices
– Samsung Origen
– STE Snowball
– ARM Versatile Express A9
– TI Panda Board
– HighBank
• Boot test in LAVA for above boards
– Failure analysis and report:
– Identify the root cause as possible as we can
– Send Email to related mailing lists
• Current responsibilities:
– "linux-next": Naresh Kamboju <naresh.kamboju@linaro.org>
– "linux-arm-soc-for-next": Botao Sun <botao.sun@linaro.org>
linux-next & linux-arm-soc-for-next

9. • Requesting new services
• Initial analysis (requirements, scope of work)
• Defining acceptance criteria and test plan
• Artifacts delivery
• Test case execution
• Reporting (bugs, results...)
New projects