The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
Extending Agile to Suite Big Projects
1. Extending Agile to Suite Big Projects 1
Extending Agile to Suite Big Projects
Muhammad Amin Bandeali, MSE Student California State University, Fullerton
Amin.Bandeali@csu.fullerton.edu
Abstract – It has been said that Agile methods only work for small, collocated, self-directed
teams that include on-site customers and trusted relationship among team members. But what if
the team is distributed around the world or the developers lack self-directed team skills? This
article presents a case for using key Agile practices in a broader range of projects, which
includes large physically distributed projects keeping in mind safety, reliability and criticality.
I.Introduction
“It [is] predicted that a decade would not see any programming
technique that would by itself bring an order-of-magnitude
improvement in software productivity.”
- (Brooks, 1995, p.vii)
Brooks’ bold and provocative “No Silver Bullet” statement was made in 1986. Even after two
decades his prediction stays and is widely accepted as the truth. It is also believed that the next
decade wouldn’t bring any change and his statement will stand the time. Agile methodologies
were being developed around the era this statement was made, perhaps even earlier. The ‘Agile’
name came about when the Agile Manifesto was signed in February 2001. It was signed after
highly regarded individuals from the software development industry came together to form what
is now termed ‘the Agile Movement.’ (Highsmith, 2002, p.xvii). These proponents of the Agile
methodologies all believed that the traditional methodologies of heavy documentation and
rigorous processes are in many cases no longer appropriate. Today’s unpredictable environment
requires rapidly changing software which can adapt and evolute with the ever emerging
requirements.
Many organizations are looking seriously at Agile methods to see if there are benefits to be
gained across a broader range of projects. The fundamental problem that these organizations
have with adopting Agile methodology is the myth that Agile cannot be used with large scale
projects, but as we would see later in this article that with some extensions to the well-known
Agile methodology, it can be applied to large scale projects. These organizations currently use
the more traditional “heavy-weight” type methodologies sometimes known as non-Agile
methodologies.
II.Non-Agile Methodologies
The serial, or waterfall, development lifecycle are very widely used throughout the industry and
have been under attack for years. Donald Reinertsen said, “The vast majority of academic
literature on product development is skewed towards the sequential approach. This is in striking
contrast to the enormous shift that has taken place in managerial practice towards a more
concurrent approach.”
Serial development cycle’s supporters believe that time spent early on in software production can
lead to greater economy later on in the software lifecycle. A bug found in the early stages of the
2. Extending Agile to Suite Big Projects 2
production lifecycle – specification or design is cheaper to fix in terms of money, effort and time
than later on in the process. McConnell estimates that quot;a requirements defect that is left
undetected until construction or maintenance will cost 50 to 200 times as much to fix as it would
have cost to fix at requirements time.quot; However, the defense industry abandoned the waterfall
methodology for iterative methods after experiencing high failure rates.
Critics of Agile methods are from the above community and they believe that Agile is an ad-hoc,
undisciplined, and technically mediocre style of development. The reality is that Agile methods
have less-elaborate process structures compared to the serial or iterative methodologies, but they
tend to “follow” the ones they have; unlike projects that thrive on process, procedure and
documentation, but have high failure rates – since those processes, procedures, forms and
documents are systematically ignored.
III.Iterative Methodology
If you want to innovate, you have to iterate! That is what the proponents of Iterative
Methodologies believe in. Their main argument is that Estimates of budgets, schedules, etc.
become more realistic as work progresses, because important issues are discovered earlier. It is
important because iteration helps in coping with the changes that software development
generally entails and software engineers can get their hands dirty by start working on a project
earlier. Agile methodology is also built on iterative concepts as discussed in the next section.
IV.Agile Methodology
What does a business executive want from a product development and project management
process – reliability, predictability and dependability. So that good business decisions could be
taken earlier than later. That is what Agile methodology is set to achieve. Agile development is
reshaping the way coders and entrepreneurs create software products based on Web-based
services. Agile teams work very efficiently. They create small chunks of functional code,
sometimes in as little as a week and once a component is finished, additional features are added,
with the process repeating indefinitely. Agile employs different strategies to achieve its goal.
Few of those well known methods are Scrum, XP and Lean. Scrum focuses on one month
sprints. XP advises shorter iterations (2 week, typical). Lean focuses on a single piece (one
feature, in the case of design projects), delivered in the shortest possible time.
This methodology has built a reputation for enabling managers to deliver products on time and
under budget, which helps explain why it has become a methodology of choice at companies like
Google and Lockheed Martin. This mindset has started as a revolt against over-exercised,
Dilbert-style software development projects. CNN has compiled a list of the 50 people, products,
trends, and ideas that are transforming the world of business. Agile Software Development ranks
#18.
V.How to apply Agile on large and distributed projects
Software Life cycles are full of trade-offs. Software is complex as people are complex and the
only thing that is certain in projects is change. This lethal combination of unpredictability is
more often helped by Agile principles. But there are implications. In exchange to working and
3. Extending Agile to Suite Big Projects 3
functional products, you do get less predictability. Some people believe that Agile development
is right for every project, while others argue that Agile is a hoax. But the reality is somewhere in
between. Most approaches could be applied to any project but the best way to methodology is the
way you know. But the key question is – how do you know if out-of-the-box Agile suits a
project?
DSDM (Dynamic Systems Development Method) is one of the initial development
methodologies based on the Agile philosophies. The DSDM methodology includes a project
suitability filter. This filter is a simple questionnaire that tries to highlight the key characteristics
of a project that is well suited to DSDM, or more generally to an Agile approach. Here are some
of the questions that matter most:
• Does the sponsor/senior management understand and accept the iterative philosophy?
• Can the organization accommodate the frequent delivery of increments?
• Will it be possible for the developers to have access to users (user representatives)
throughout the project?
• Will the development team remain the same throughout the project?
• Will the project use technologies suitable for prototyping?
• Will the development be computationally non-complex?
• Can the requirements be prioritized? Will users be able to define requirements
interactively?
“Yes” to the above questions means that the out-of-the-box Agile approach would work great for
the project. However a few of “No”s do not eliminate the Agile approach entirely. Through a
small amount of risk mitigation, and extensions, the Agile approach could be used.
VI.Agile on large and distributed projects
If you attempt Agile methods out of the box on large and distributed projects, you are likely to
miss the key benefits of Agile. This is because these methods require extensions to work on more
complex projects. Using a combination of the following extensions, Agile architecture could be
used with large scale, complex projects:
A.Agile Architecture
Agile architecture involves first setting up an Agile architecture team. The goal of the Agile
architecture is to address the complete project scope and provide a minimum set of architecture
compliance rules. First a small strategically selected team rapidly develops a high-level Agile
architecture and documents the results. The resultant product of the first level is a thin
architecture document that includes a simple, high-level diagram showing the major system
components. After this practice, the Agile architecture team focuses on the high-risk areas for
each iteration. When the architecture grows over time as the solutions accumulate, these
documents are updated in a timely manner. This is crucial to Agile architecture scalability.
B.Super Leads
A super lead oversees between three and five Agile teams working on a large scale project;
however he/she is not the Agile team lead. Super leads could also attend daily stand-up meetings
4. Extending Agile to Suite Big Projects 4
but if they do, they do not speak. The super lead's role is to mentor the less experienced Agile
team leads with filling the communication gap among them. Super leads also review the team
plans and provide feedback.
A primary benefit for having an on-site customer is to answer questions quickly so the Agile
team does not waste valuable time. A super lead can function as a customer proxy, especially on
large and distributed projects. In scenarios where the super lead does not know the answer they
might know who to call.
C.Light Weight Documentation
Since large and distributed teams cannot meet every day, therefore, having light weight
documentation can help in many regards. Training the team to be able to document efficiently
and lightly the processes that might help their other peers to successfully maintain velocity could
increase productivity and count towards successful project delivery.
VII.Advantages of adopting Agile on large and distributed projects
1) Edge: Since Agile development is iterative, the most important features with the most
priorities are delivered early. This could cause the revenue to start pouring in way before
anticipation. Completing a large project using traditional methodologies might take very long to
complete. Research also depicts that 80% of all market leaders in any industry had their flagship
product first in the market. Agile development is set out to do just that – capture the market
before any of your competitors.
2) Quality: With integrated testing, product owners can make adjustment to the system without
compromising quality. Agile would help improve quality in large projects by having a constant
eye on the most important quality attributes after each release, instead of the one and only final
release. With users’ active involvement throughout the product’s development, the visibility
increases for key stake holders, which can ensure expectations are met and expectations are
adequately managed and ultimately result in higher customer satisfaction.
3) Risk: Through incremental releases, product owners and team can easily identify any potential
risks early and mitigate them to ensure that any necessary decisions could be taken at the first
possible opportunity.
4) Cost: The above approach of fixed timescales and evolving requirements enables a fixed
budget. The scope of the product and its features are variable, rather than the cost.
5) Product: The ability to embrace change makes sure that the right product is delivered versus a
successful project to only find that the product is not what was expected, needed or hoped for.
Agility is all for the right product especially if it is being developed by geographically diverse
teams.
VIII.Conclusion
The factors that are involved in software development are far too extensive and unpredictable for
just one methodology to be the answer in today’s chaotic environment. Employing any of the
existing or emerging Agile methods does not exclude the use of another. Nevertheless, Agile is
about breaking through to the grassroots level, making real status visible and acted upon sooner,
which ultimately provides greater value to the customer. It is also becoming increasingly
conspicuous that a large development project can make use of rudiments of more than one
5. Extending Agile to Suite Big Projects 5
methodology. However, the focus is clearly shifting, from processes to practice, from heavy
documentation to iterative releases of working software, from price fixing and contract
negotiation to collaboration with the customer and from following a plan to adapting quickly to
change. And that is the set of underlying philosophies that form the Agile Manifesto. I would
like to wrap up this paper with the best of Agile Manifesto (Highsmith, 2002, p.xvii) itself:
• Individuals and Interactions over Processes and Tools
• Working Software over Comprehensive Documentation
• Customer Collaboration over Contract Negotiation
• Responding to Change over Following a Plan
References
[1] Highsmith, J. 2002, Agile software development ecosystems, Addison-Wesley, Boston.
[2] McMahon, Paul. E. “Extending Agile Methods: A Distributed Project and Organizational
Improvement Perspective.” May 2005.
[3] Schwaber, Ken. Agile Project Management with Scrum. Microsoft Press, 2004.
[4] Lynos, Kevan. “The Agile Approach”.
[5] Business 2.0 Staff. “The 50 Who Matter Now” CNN.
(http://money.cnn.com/galleries/2007/biz2/0706/gallery.50whomatter.biz2/33.html)
[6) Waters, Kelly. “Is Agile development right for your project”.
[7] Agile Alliance. (http://www.agilealliance.com/)
[8) Ambler, Scott “Why Agile Software Development Techniques Work: Improved Feedback”.
(http://www.ambysoft.com/essays/whyAgileWorksFeedback.html)
[9] WikiPedia. “Agile Software Development”.
[10] Fowler, Martin. “Is Design Dead?” Appeared in Extreme Programming Explained, G. Succi
and M. Marchesi, ed., Addison-Wesley, Boston. 2001.
[11) Elssamadisy, Amr. “Agile Team Size” InfoQ. 2007.
(http://www.infoq.com/news/2007/07/agile_team_size)
[12] Cohn, Mike. “Agile Estimating and Planning”. 2005.
[13] Brooks, Frederick. “The Mythical Man-Month: Essays on Software Engineering”. 1995.