With software testing, it’s a good idea to keep the end user in mind, but with open source software this is absolutely necessary. Open source projects largely rely on community engagements and involvement. Whether that be asking questions, reporting issues, or submitting pull requests.
Testing an open source project must account for working with the community in these ways. There must be processes in place to support community members in their efforts to contribute and to leverage the feedback they provide in order to impact the product’s quality positively.
In this SauceCon 2018 session, Meaghan will share strategies to maintain high quality in open source projects. She will draw on her experience testing open source projects at GitHub and discuss the ways collaborating with the community has allowed her to test more creatively while keeping the users in mind.
This session is about Maintaining quality in open source.
This is a personal journey and what I’ve learned from testing on an open source project both at GitHub and externally.
Throughout this session, I’ll share what have been some effective ways to build in high quality on open source projects.
My hope is that some of these lessons might resonate with you whether or not you work on an open source project.
Note: You’ve seen great automation talks today, but this will not be one of them. This is a process heavy talk
First, I’ll briefly tell you a little more about me. I’ve been at GH for the past 7 months. We are hiring!
I’ve been in quality for the past 6 years now.
Over the past few years, I have begun to work on open source projects and have become sort of an open source aficionado.
I'm excited to share some processes that have helped me in working with open source software.
At GitHub, I work under client apps which is for applications that GitHub builds to support developers in various ways.
It includes the products:
Electron - a powerful framework to build cross platform desktop apps
Atom - an awesome text editor. Recently released teletype for atom which allows for real-time collaboration and is great for pairing, especially across teams that are distributed
Desktop- a native desktop app to use GitHub. Works on both mac and windows
Editor Tools - which focuses on the GitHub plugin for IDE’s. The GitHub extension for Visual Studio is the most popular.
Throughout this session, I will feature some examples from these projects that highlight things we do to ensure quality.
When I first started testing open source software, I noticed there was something very different about open source projects compared to non-open source projects.
There is something that sets them apart. Something that forces me think about the software from a different perspective.
So what is it that distinguishes open source projects?
Community makes all the difference.
Working on open source, you don’t just have to consider your teammates in your own software delivery team
You have to take into account a whole community of contributors and maintainers.
From a quality engineer’s perspective,from my perspective it means a bigger team- there is more unexpected activity
there are sometimes more pull requests than you expect. Definitely more issues and questions about the software.
More to take into consideration
So community involvement makes open source unique.
The community affects these projects in so many ways and really pushes them forward.
Let’s talk for a few minutes about why involvement matters.
Contributions are really key in a few different aspects.
Engagement is good. Usually the more the better as contributors can have tremendous benefits to a project. It’s also good because it means people care about your product.
Having more eyes on the application is always a plus. With non open-sourced applications, of course there are users- but you might not interact with them. When projects are open-sourced, users find it easy to pop on over to the GitHub repository and open up an issue. The users want to be heard, they want you to know all the bugs they found, and they want them to be fixed.
In addition to bugs, users can also describe pain points in their workflows- which is a plus, and can often suggest enhancements and new features as well. This can even influence the product team about what to work on next.
In addition to all the points before this, community involvement should be awesome. It’s really cool and rewarding to have a project that people want to contribute to.
I’ve also been telling a story so far from my perspective of open-source projects that have a team of full time maintainers. I have to take a moment to stop and acknowledge all of the projects which are maintained solely by volunteers.
There are tons of projects out there where people put their blood, sweat, and tears into projects so that they can be used and enjoyed by people all across the world.
At the end of the day, contributions by contributors and maintainers make all the difference and can make or break a project
So involvement is good, but we want community members to be able to easily understand how to contribute, what the guidelines are, and make sure they feel supported to help out.
it’s important to build a community of contributors who are not just eager about the software, but can also help contribute in a positive way and promote quality in the project.
The goal is to have good steps in place to tell the community best practices for working together
Have a friendly readme. This is the Atom README that shows details about the project, resources for websites to visit for more information and support.
There is a link to follow them on Twitter
A section describing a code of conduct
Then there’s steps for installing Atom and getting set up to use it
This is an example of a great README that’s full of juicy information
Whereas READMEs help people use the project, contributing docs help people contribute to the project. It explains what types of contributions are needed and how the process works.
The contributing doc should include a code of conduct that describes expectations for participating in the project.
It sets ground rules for participants’ behavior and helps to facilitate a friendly, welcoming environment.
Most importantly, contributing docs include what types of contributions are needed and how to submit pull requests, bug reports, feature and enhancement requests.
Makes it easy
Makes it so that people follow the same steps to do stuff. More uniformity, less surprises. That’s always a motto I like to live by- so boring, I know. But extremely helpful
In addition to having those scripts locally it’s also great to use those scripts to run in CI and then set it up so that it is integrated with pull requests
Tests should build with every new commit
It should be required that tests pass before code can be merged to master
Templates are helpful to have in order to ask directed questions so that there is enough information in the description for anyone reading it.
An issue template typically has sections for a description, steps to reproduce, expected vs actual behavior
Similarly pull request templates are also really great at asking for all of the information you expect to see in the description.
Templates can be really helpful to directly ask all of the details you want to get from users although unfortunately people are people and break the rules sometimes and don’t fill out all the fields that you think are helpful
Open-sourced projects all have a community profile.
It can be interesting to check out how a project meets community standards and how welcoming they are
As a potential contributor, use the community profile checklist to see if a project meets the recommended community standards and decide if you'd like to contribute.
Next, I want to talk about the different types of contributions that the community makes and how to build quality processes around these areas.
There are 3 main avenues that the community can get involved in a project
People are going to email in support questions. It’s important to:
Make anything that needs to be made available, made available
Pre-canned responses for standardization
SLA for response time in order to
Give people information they need upfront in terms of how to contribute to a project. Make it accessible
Have good processes in place to support contributors and follow those processes in order to maintain/ensure quality
Be responsive. Be helpful. It’s good to build a strong community of people who enjoy working on and using a product so help keep it that way