Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Automated system testing for a learning management system

1.313 visualizaciones

Publicado el

Präsentation Masterarbeit, TU Graz, November 2019

Publicado en: Educación
  • Inicia sesión para ver los comentarios

  • Sé el primero en recomendar esto

Automated system testing for a learning management system

  1. 1. WISSEN TECHNIK LEIDENSCHAFT Automated system testing for a learning management system 31.10.2019 Lukas Krisper
  2. 2. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 2
  3. 3. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 3
  4. 4. A shift towards Agile Software Development • Over the last two decades … • Emerge of agile methodologies in software development • Acceleration of software delivery and feedback cycles • Automation of manual processes • Quality Assurance? • Only 20% of IT organisations have automated more than half of their test cases (State of Testing Report, 2018) Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 4 Motivation
  5. 5. New Releases in Summer 2019 • TeachCenter 3.0 • Learning Management System (LMS) • Migration of whole LMS to new Moodle Version (3.5) • New GUI for all users • Set of Unit-Tests existed • Sparely documented manual tests Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 5 Motivation
  6. 6. Research Questions • (RQ1) How can the existing core functionality of a large software system be assured automatically in new versions of the system? • (RQ2) Which test cases and test suites need to be designed to test a learning management system effectively? • (RQ3) Which test data is needed to test a learning management system and how can the data be provided to automated test cases? Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 6 Motivation
  7. 7. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 7
  8. 8. Focus of Automation Activities • Software Quality Characteristic*: Functional Suitability • Testing Level: System Testing • Testing Type: Regression Tests Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 8 Test Automation *According to ISO-25010
  9. 9. Software Quality Characteristic: Functional Suitability Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 9 Test Automation • Does the software product do what it is supposed to do? • Functional testing as basis of all testing activities • A software product that does not provide functional suitability is rather worthless • E.g. calculator that calculates very fast and has a nice UI, but the calculations are wrong 2 + 2 = 5
  10. 10. Testing Level: System Testing Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 10 Test Automation • Software product is considered in its entirety • Test environment has close resemblance to production environment • Tests conducted from user’s perspective • Typically using the systems GUI for interaction • Automated system tests as second line of defense • If tests fail at this level → malfunctioning system + wrong or missing test at lower testing level
  11. 11. Testing Type: Regression Tests Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 11 Test Automation • Is already existing functionality affected by the changes applied to a software? • The first type of tests to be considered when automating tests • Quickly run a set of tests after applying changes: • Extensions to a feature • Bug fixes • Refactorings • … • Especially useful when changes happen late in the project and time for manual testing is short
  12. 12. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 12
  13. 13. What Test Cases should be automated? • Important consideration! • Interview with first level support of TeachCenter 2.0 • Research in literature on usages of LMS • Results: • Knowledge on most important use cases for students and teachers E.g. reading the course’s contents, communicating with other participants, submitting material to the course, … • > 30 test cases combined to different test suites Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 13 Test Cases for a LMS
  14. 14. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 14
  15. 15. Real Data vs Synthetic Data • Real Data • Transfer data from production environment to test environment • Data has to be edited (according to laws on data protection/privacy*) • Important: Keep consistency in data set! • Synthetic Data • Newly created data for the purpose of testing • Always GDPR compliant Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 15 Test Data for a LMS * e.g. General Data Protection Regulation (GDPR)
  16. 16. Data for a LMS • A LMS typically contains data on • Courses (course descriptions, learning material, …) • Data on people involved in the courses (names, email-addresses, …) • Data from interaction of people with the courses (submissions, …) • TeachCenter 3.0 is tested with both synthetic and real data • Data used for automated tests is labeled as such Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 16 Test Data for a LMS
  17. 17. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 17
  18. 18. Overview on Infrastructure Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 18 Implementation and Infrastructure
  19. 19. Test Suite 1 Test Case 1 Test Case n Test Case 2 … Test Suite 1 Test Case 1 Test Case n Test Case 2 … Overview on Implementation Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 19 Implementation and Infrastructure Test Suite 1 Test Case 1 Test Case n Test Case 2 … Test Data Manager Test Driver Utility e.g. initialise headless Chrome browser with cleared cache e.g. get PDF files for submissions, references to test data stored in JSON files, … Page Object 1Page Object 1Page Object 1 TestNG Page Object 1Page Object 1 Helper Classes e.g. taking screenshots on errors See following slides
  20. 20. PageObject Design Pattern Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 20 Implementation and Infrastructure • Components of a website are modelled into reusable objects → “Page Objects” • Page Objects offer functionality to interact with website (click links, enter input fields, …) • Tests use Page Objects to interact with website (perform action → assert result of action)
  21. 21. Identification of Objects in UI Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 21 Implementation and Infrastructure • Tests should be executable regardless … • screen resolution • window sizes • language settings • Identification of objects using stable features (IDs or XPath expressions) DOM Element <button type= "button" id= "btn1" data-role= "next "> Next </button > ID btn1 XPath expression //button[@data-role=’next’]
  22. 22. Agenda • Motivation • Test Automation • Test Cases for an LMS • Test Data for an LMS • Implementation and Infrastructure • Conclusion Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 22
  23. 23. (RQ1) How can the existing core functionality of a large software system be assured automatically in new versions of the system? • Regression Tests for „Functional Suitabilty“ • System Testing Level • Large software system = many integrated components that interact with each other → “high” testing level to ensure stability and detect side effects • Continuous Integration Environment (Automation Server) Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 23 Conclusion
  24. 24. (RQ2) Which test cases and test suites need to be designed to test a learning management system effectively? • Identification of use cases • Interview with 1st level support + literature research • Prioritisation of use cases • Implementation of test cases based on use case • Execution of suites of test cases according to prioritisation of use cases • Multiple times a day ~ once a week Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 24 Conclusion
  25. 25. (RQ3) Which test data is needed to test a learning management system and how can the data be provided to automated test cases? • Synthetic and real test data • 1 configured course with various activities and resources • 1 student‘s account (enrolled to course) • 1 teacher‘s account (enrolled to course) • 1 PDF file • Test data is persisted on test system (references in JSON-files) … • Or provided to the test cases on demand • Separation of test data and automated test cases • No hardcoded test data Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 25 Conclusion
  26. 26. Thank you for your attention! Lukas Krisper, 31.10.2019 Automated system testing for a learning management system 26 Conclusion

×