Some considerations for the software development process. If you were present when I gave this talk at php|tek, please consider leaving me some feedback: http://joind.in/189

1. Getting IT Done
Considerations for Development Process
Wez Furlong

2. Why you might consider listening to me
• Over a decade of commercial development experience
• Desktop/Windows applications
• Web/PHP applications
• Systems/Server software
• Consulting and Product Development
• One-man projects and team efforts
• OpenSource (PHP: Streams and PDO, umem, gimli, freedce, + more)

3. Agenda
Note that I don’t hold all the answers by any means
Process for the sake of process is non-productive. Take what you need from this
session, and no more than that.
• Two broad types of development and how their focus differ
• Day-to-day pre-requisites for owning your development process
• Longer term considerations

4. Consult-ware: Bespoke
• You’re building software for one particular customer
• Often working in the form of a Project (has a defined end-goal)
• Not uncommon to be working as outsourced development resources for your
customer
• Customer satisfaction is key to retaining their business
• Customer needs drive development focus
How can I satisfy my customer while avoiding having them dictate everything I do?

6. Product Development
• You own the product
• You have more than one customer to satisfy
• Often Project based development, with maintenance considerations
• Release deadlines often linked to other efforts within and outside your organization;
Marketing, Sales, Trade Shows and Customer needs. Missing deadlines is not
good.
• Typically build a software package that customers will install on their systems
• Diversity in customer systems must not be underestimated
How can I meet deadlines, ensure quality and not die trying?

8. Owning your Development Process
It’s important to be in control of your development process.
Lack of control leads to:
• Un-focused, off-target, use of time
• Stress
• Low morale
• Decreased productivity
• Disaster

10. Pre-Requisites: Revision Control
How do you know whether you’ve got the latest version of the code? How do you know
if work has been done for a particular feature or bug? Do you know why that
particular line of code exists? Is it safe to remove it?
• You need one master copy of the source code
• You need to be able to review the changes made in the past, and why they were
made
• Make sure that your developers log useful information in their commit messages
• Where possible, commit messages should refer to a development issue (bug report
or task number) for additional background
• It should be part of your culture to keep the code in working order. In particular, the
code should be buildable and functional at the close of business each day.
• You should have a backup procedure for your revision control repository

11. Pre-Requisites: Issue Tracker
Do you know what you’re supposed to be doing right now? Do your developers? How
sure are you that it’s the right thing? Do you know if that bug has been fixed? Are
you waiting for something before you can make progress on that issue?
• You need a master list of what’s been done and what’s yet to be done
• You need to keep it organized and up-to-date for it to be useful
• It should allow you to prioritize, categorize and assign items (bug/task, when is it
needed, who is working on it?)
• You need to ensure that your developers know how to interrogate it to find out what
they should be working on
• Ideally, it should be convenient to cross-reference changes in your Revision Control
system to your Issue Tracking system and vice-versa
• You should have a backup procedure for your Issue Tracker

12. Pre-Requisites: Specs
Depending on the scale of your development organization, Specifications can range
from something relatively informal through to a hefty tome with excruciating detail.
Smaller shops will often skimp on up-front spec building so that they can engage
development sooner. This super-agile approach is fine so long as the customer
requirements and your solution for them is well understood, this is often so for
Cookie-cutter development.
Anything larger needs something with more detail. My guidelines for what to capture:
• Know why you’re building something (the use-case)
• Know how you’re thinking of implementing something
• Call out particular nuances (eg: use standard hashing, but pad with spaces)
Ideally, you will have enough detail that any one of your developers could pick up
implementation of the feature.

13. Pre-Requisites: Specs (cont.)
There needs to be a master copy of the Specs that your organization can use as a
reference point.
Changes to the specs should be avoided once development has started, and well
communicated to your organization if they are unavoidable. This goes both ways;
developers may need to implement something differently from the plan, and this
must be communicated up the chain to make sure it is acceptable.
The specs should be broken down into tasks and logged in your Issue Tracker. It should
be convenient for your developers to locate the pertinent section(s) of the specs
from the task; best to refer by section name rather than number in case the specs
change after the task is created.
Remember: goal is not process for the sake of process. When recording specs, keep in
mind who you’re writing them for. You need enough detail to make sure that your
team is able to implement them, and no more than that.

14. Pre-Requisites: Build and Deployment
Writing software is just one aspect to development. Does the code written by developer
A run on your workstation? Did it stop working because it required a new library to
be installed or upgraded? Is there some trick to getting things packaged up? Are
there configuration adjustments to be made on the customer’s machine to make it
work? Can you make an emergency code push at 2am with confidence?
• Make sure that your build process is scripted (ideally with a single script that does
everything necessary for the build)
• Make sure that all pre-requisites are recorded and taken care of via your build
• Make sure that your deployment/installation process is scripted
• Consider using system level packages (eg: rpm) to manage dependencies and pre/
post installation steps
• Ensure that your developers only use the above to maintain their development
environment.

15. Pre-Requisites: Testing
How confident are you that a code change didn’t break something? Are you sure that it
works in all web browsers? Are you sure that it works for a fresh install of the
software? How about a pre-existing install?
• You need a test plan, and you need to have people work through it. (my developers
physically check boxes off a grid on a whiteboard as platforms pass tests)
• The test plan should be derived from the Specs
• There needs to be a master copy of the test plan
• You should have unit tests, and these should be able to be run via a single script
• Unit tests should pass 100% all the time to be useful. If it is “ok” for a test case to
fail, it should be reflected in the test harness or recorded centrally.
• Consider using Virtual Machines for deployment testing

16. What does that buy me?

17. What does that buy me?
Confidence
No need to worry about losing code, missing things out, breaking the build or
breaking the install.
More time to think about important things
Now that I’m not distracted by issues cased by the lack of the above, I can finally
think about the implementation of this complex component.
Improved ability to forecast
With the build/test/deploy portions scripted, and with centralized task management,
the overhead of development becomes smaller and more predictable, which makes
it easier to figure out how long things will take.

18. Longer Term
With the daily process under control, you are free to think beyond the issues of the day.
I suggest turning your attention to the deadline and on breaking down development to
get to that point.
It is common to break projects into a set of milestones on the path to the deadline.
Depending on your organization, you may need to deploy code to customer(s) before the
deadline is reached.
There are a number of software development approaches you can use to do this, but I’m
going to talk about just one approach that has worked out well for me.

19. Scrum: an agile process

20. Scrum
Scrum is based on the concept of a Sprint, a focused effort of fixed duration towards
fixed goals
The project members form a Scrum Team
A Product Owner compiles all the changes planned for the product and prioritizes. This
list is called the Product Backlog
Before each Sprint, the the highest priority items are taken from the Product Backlog
and form the Sprint Backlog
During discussions with the Product Owner and Scrum Team, the Sprint Backlog is
refined into a detailed set of tasks
The Scrum Team is self-organizing and the members have joint responsibility for the
results

21. Scrum (cont.)
The Scrum Master coaches the team, removes impediments, works to ensure that the
team has the best possible chances for realizing the goals fixed for the Sprint
Each Sprint enhances the product value and adds improvements that can be delivered
to the customer(s)
Scrum helps raise team awareness of development, reducing the tendency to get
blinkered into your own set of tasks
More deliberate teamwork helps improve morale and makes it easier to progress difficult
tasks

23. Scrum Roles

24. The Scrum Team
Performs the actual work of problem solvers and designers.
Decide how the work is arranged and how assignments are distributed.
Everyone should be able to swap tasks with another member, if they wish.
Shouldn’t be required to become multi-discipline gurus, but that may be a side effect of
the process.

25. The Product Owner
Represents the voice of the customer
Ensures that the Team works on the right things from a business perspective
Administers the Product Backlog

26. The Scrum Master
Combination of coach, fixer and gate-keeper
Meets with the team every day in brief meetings, called Daily Scrums
Works to keep the team happy, focused and progressing

27. Implementation

28. The Pre-Requisites
Subversion – version control
Trac – issue tracking
Trac – Specs stored in wiki
Build and Deployment – invested effort into a “roll” script that builds everything as both
rpms for Linux systems and Sys V packages on Solaris
Testing – home-grown continuous integration builds out and runs unit tests across a
dozen different architectures and OS combinations on each subversion commit
Testing – we follow a manual test-plan as part of our software roll week and check off
platforms as they pass muster
VMs – each developer has a big workstation that can support multiple virtual machines
for development and testing purposes – one vm for dev, one for testing.
We have some custom Trac plugins to track time and plot the burndown chart (see later)

29. Implementing Scrum: Pre-Sprint
Before each Sprint, the team meets to review the Sprint Backlog and flesh out design
and time estimates
We use the estimates to guide what lands in the Sprint Backlog – it makes no sense to
schedule more work than you have man-hours available
Once the Sprint begins, the Sprint Backlog is frozen – no new items can be added
A Release Date is chosen – that cannot be altered
We use a Trac milestone to record the Sprint Backlog items and the release date

30. Daily Scrum
Every day, the team meets for a brief meeting, with the purpose of eliminating speed
impediments for the group
What have you done since the last meeting?
What will you do between now and the next meeting?
Is there anything preventing you from doing what you’ve planned?
These meetings should be kept brief (10-15 minutes) to avoid bogging things down.
If you have more to discuss, discuss it after the scrum with just those people that are
needed.

31. Monitoring Progress: Burndown Chart

32. So what about bugs?
Two classes of bugs:
• Bugs that you find during development
• Bugs that your customers find in production
General Rule: It’s better to find and fix bugs in development, otherwise you’ll end up
fixing bugs at 2am – a risky proposition

33. Planning for bugs
Bugs are the only exception to the rule about not modifying the Sprint Backlog
This can mess with your development schedule
Depending on the maturity of your code and process, and complexity of your code, you
should allocate a portion of development time out of each Sprint purely for fixing
bugs.
As your team becomes more experienced with the process, quality will increase and the
amount of bugs should decrease.
You may find that you need to reserve as much as 50% of your time for unexpected
issues when you start Scrum
This means that you should plan your product deliverables based on what is possible to
ship with only 50% of your developers
If you exhaust the Sprint Backlog, work with the Product Owner to pull in more items,
and decrease your bug budget for the next Sprint

34. Finishing the Sprint
A Sprint finishes on the Release Date
If the Sprint Backlog has not been completely burned down, the remaining items are
pushed into the next Sprint
The Release Date is never altered
Packages are ready ship to the customer by the end of the Release Date

35. Feedback Loop
No process is worth implementing if there’s no way to feed back and adjust to fit your
team.
It’s strongly recommended that you hold a review at the end of each Sprint and allow
each member of the team to comment on how things went so that you can
collectively adjust your plan for the next Sprint
You need to gather metrics so that you can measure how things change from Sprint to
Sprint:
• Burndown velocity
• Time spent on planned vs. unplanned issues (bugs)
• Problems encountered during the sprint
An important part of Scrum is to look at things from a team perspective, and your metrics
should reflect team rather than individual performance.

36. Time Out
We take a week out between Sprints to work on items that are not directly related to the
Product Backlog
This gives people a chance to take a breather and work on those non-essential items,
like code cleanup, refactoring, as well as allowing some time for innovation
We hold our Sprint Review in this time, and also use it to plan out the next Sprint

37. Getting IT Done?
Scrum helps you get IT done, and the more you do it, the more it helps.
Key benefits for us:
• The fixed/known cycle duration helps your team get into good development habits
• The repeatable nature of the process becomes second nature, takes less time and
reduces mistakes
• Customers are pleased because you never miss a deadline
• The short Sprint duration means that missed features are never too far away, which
is also great news for customers
• Burndown chart provides feedback for planning the next Sprint and helps self-
correct for bad time estimates
• Happy customers, no missed deadlines, and reasonable workload = Fun Work