How to Troubleshoot Apps for the Modern Connected Worker
Crocker.lori
1. The Art of Contract Technical
Monitoring
PM Challenge 2011
Managing Contracts Track
February 10, 2011
Lori Crocker, NASA JSC/EVA Office
2. Contract Technical Monitoring is an Art?
If you subscribe to the notion that anything that is not a science is
an art… then technical monitoring is definitely an art.
Leadership
Diplomacy
Persuasion
Teamwork
Communication
Empathy
2
3. How Do We Learn Contract Technical Monitoring?
The FAR and NASA FAR supplement addresses the role of the
COTR, and high level duties of the COTR, but not HOW to do
contract technical monitoring
That leaves HOW to do it up to the discretion and experience of the
individual
They don’t teach it in Engineering school…
We do get COTR training, which scares you to death, followed
by…
Watching others do it (best case)
Trial and error (typical case)
Engineers are not always great at things that aren’t so black and
white
3
4. And You Can Tell Me How to Do This Because…?
“Lori, what makes you particularly qualified to tell a new contract
technical monitor how to perform your duties?”
Technical monitor experience on 3 different contracts
Experience as both a contractor and a federal employee
But mostly, because I have done it the WRONG way (and paid the
price for that misstep), as well as the RIGHT way (and enjoyed the
success of that method )
…and I am willing to tell you about it!
NOT
4
6. Step 1: Develop Trust
Trust requires a relationship
Show by your actions that you have the best interest of the WORK
at heart
Show by your actions that you do not feel your job is to skewer
your contractor every chance you get
It is OK to be tough, as long as you are also FAIR
Trust requires that you know how the contractor does their work
You have little to offer until you have a thorough knowledge of the
contract SOW and how the contractor meets that SOW, so DO
YOUR HOMEWORK!
Trust is a two-way street; once you have established the previous
two requirements, you must INSIST that the contractor’s
management does not “filter” the message you receive from them
Clearly communicate this requirement with all contractor
management, especially if you can tell this is not their SOP!
Make sure you get a commitment from contractor management that
they intend to meet your requirement! 6
7. A Word of Warning…
You must be in a position to KNOW if information is being
filtered from you… so get out there and see the work!
7
8. Step 2: Don’t Just Find Problems, Help Fix Them
As you learn and monitor this technical work, guess what? You
will see problems and issues.
That doesn’t mean you need to take some kind of sick joy in finding
issues.
That doesn’t mean this contractor can’t do anything right.
If you act like this, it will become a self-fulfilling prophecy
Your job is not just to find problems – your job is to help fix
problems by contributing to solutions.
It is best to guide the contractor to a good solution, rather than
dictating it.
Contractor personnel know their processes and procedures best
Explain at a high level what you are after, and ask the contractor to tell
you the best way to achieve what you are after
Always be the dumbest person in the room – ask questions until there
are no questions left to ask
Contract monitoring is no place for someone uncomfortable with not
knowing more than everyone else
Ask how you can help, and follow through on what you are told. 8
9. Step 3: Communicate, Communicate, Communicate!
You have to be able to tell your contractor what they are doing
wrong, however…
How you say it does matter! (i.e. don’t be a jerk!)
“The Jerk: You have great skin. Are you a model?
Lady: No. I'm a Cosmetologist.
The Jerk: A Cosmetologist? Really? Wow. Must be tough to handle the
weightlessness.” from The Jerk (1979)
Your motivation for saying it matters!
To help performance improve = good
To find something wrong because you can = not good
If you don’t have the right motivation for telling your contractor what
they are doing wrong…
“Be a Technical Monitor that gets the reasons, not the excuses.”
Don’t forget to tell your contractor what they are doing right!
9
10. A Word About Award Fee
Contractors are in business to make a profit
Understand that award fee impacts the contractor’s profit and
bottom line
If Award Fee becomes adversarial, the results can be disastrous
for NASA.
If you attack your contractor on Award fee, or they live in fear that
you are going to, they may go into protect mode
Instead of finding out where problems are, your contractor may try
to filter those problems from you
Does that mean you shouldn’t point out weaknesses for Award
Fee or give a contractor a less than perfect score?
Of course not!
So what does it mean?...
10
11. More About Award Fee
Don’t make Award Fee into an adversarial process
Utilize Steps 1-3 year-round, including Award Fee time
Award Fee write-ups should never, never, ever be the first time
a contractor has heard from you about an issue
Be fair; remember that being tough is ok as long as you are also
fair
No contractor will like getting dinged on Award Fee, but if it is
fair, they will accept it as part of doing business with the
government
Realize that if our contractors fail, NASA fails
Make sure you communicate to our contractors how much you
understand that if they fail, we fail
Again, make Award Fee results about fixing the problems, not
about finding them
11
13. My NASA Monitor Doesn’t Know All This…
What should I do???
Anonymously send them the link to this pitch on the PM
Challenge website?
13
14. My NASA Monitor Doesn’t Know All This…
Try doing your part on Steps 1-3
Do your part to build a relationship – invite them over to see the
work you do and witness first hand that you know what you are
doing
Don’t filter information, even if it gets used against you
Do speak up if it does get used against you in an unfair manner,
and let your technical monitor know what they did is very non-
motivational
Do tell your TM if you think they are being unfair and question their
motives for being that way
Ask for their high level desires for problem areas
Tell them your potential solutions for these problem areas
Encourage their questions and ask for their help
Give reasons, not excuses
Ask for your TM’s feedback and take it without becoming defensive
14
15. My NASA Monitor Doesn’t Know All This…
Other avenues?
If all else fails, do speak up to your contracts
personnel so they can discuss it with the NASA CO
and COTR
15
16. Let’s Talk Some Examples
DISCLAIMER: NASA PROCUREMENT HAS NOT APPROVED MY
ANSWERS TO ANY OF THESE EXAMPLES. THESE ARE NOT
INTENDED TO BE HARD AND FAST RULES, BUT TO PROVIDE
YOU WITH SOME REAL-LIFE SITUATIONS AND SOME THINGS I
HAVE TRIED THAT EITHER DIDN’T WORK, OR DID WORK, TO
HELP IMPROVE PERFORMANCE ISSUES AND MAKE WORKING
TOGETHER AS PARTNERS MORE SUCCESSFUL.
16
17. Let’s Talk Some Examples
Contractor A provides hardware for NASA’s new game-changing,
long-duration space travel primate testing. NOTE: NO HARM
COMES TO THE PRIMATES DURING THIS TESTING. Several tests
have been cancelled due to hardware anomalies on Contractor A’s
accountable hardware. You are the NASA Technical Monitor for the
work Contractor A performs. What do you do about these issues?
A. Award Fee write-ups are due soon. Write a weakness on Contractor A
for these issues in hopes that you will get their management’s attention
to resolve the problems.
B. Work with Contractor A to determine the causes of the issues. Help
them resolve the issues. Since you are able to help them resolve the
issues, you really don’t need to mention it in Award Fee or impact their
score. Why tick them off?
C. Work with Contractor A to determine the causes of the issues. Help
them resolve the issues. Keep the lines of communication open and
make sure you are working as a team. Include the issues that were
performance weaknesses in the Award Fee write-up. Also mention
positive work and strengths that you have seen from Contractor A during
the issue resolution process in the Award Fee write-up.
17
18. Let’s Talk Some Examples
You are a Technical Monitor for Contractor B, who is designing the
EVA Nose-Scratcher. In the time you have been the Technical
Monitor for this contract, Contractor B has been late on several major
project milestones. Each time you ask for information from
Contractor B about why they missed a milestone, you get what feels
like a sales pitch about how great they are and why even though they
missed the milestone, they really didn’t miss the milestone. What
should you do?
A. Wait until Award Fee, then cut them off at the knees. This is ridiculous!
B. Don’t do anything. These jokers obviously have pictures of some high
ranking official, otherwise they would be out of business.
C. Build some two-way trust. Take their manager to lunch (YES, THIS IS
ALLOWED BY NASA ETHICS RULES!). Talk very frankly with this
manager about the impact of their missed milestones on the space
program. Tell the manager you are willing to work with them to fix the
delivery issues they are having, and that you are willing to do everything
in your power to make this process successful, but that you will no longer
tolerate the lack of information being provided to you.
18
19. Let’s Talk Some Examples
You are the manager of ACME Logistics Integration, Inc. You have a
contract with NASA to provide integration and logistics support for all
ping pong and foosball equipment that is manifested on flights to ISS.
You have developed a set of metrics as part of your contract that is
intended to give your NASA technical monitor a view of your work.
You receive your first award fee write up, and NASA has given you a
weakness for poor metrics, even though you submit these metrics
monthly and no one has ever asked you to make any changes. What
do you do?
A. You tell your technical monitor you are sorry, and will try to do better. The
customer is always right, aren’t they?
B. You call the technical monitor and give him a piece of your mind. Who
does he think he is?
C. You meet with your technical monitor, and partner a new set of metrics that
makes sense to both of you. You also ask him to provide more regular
feedback and not wait until award fee to mention a potential issue. He
agrees, and you all work together happily ever after.
19