Right of Boom
[After EOD]
Problem Sponsor: Joint Improvised Threat-Defeat
Agency (JIDA)
US Army & US Air Force Explosive O...
Who we talked to
Andreas
Pavlou
Nitish
Kulkarni Alex Zaheer Alex Richard
Gaurav
Sharma
Academic
Program MA IPS 2017
Physics 2015
ME 2016 CS...
The Emotional Journey
Don’t
understand
the problem:
Frustration
Conception of
problem
evolves:
Excitement
Key
organization...
DARPA PCAS
Team -
modifying
existing
systems
Joint Improvised Threat
Defeat Agency - work with
technicians who deal with
p...
Week 1: Launch Point
Hypothesis:
• Build a Live Tactical
Toolkit digital notebook
for differently located
coalition & US f...
Week 1: Live Tactical Threat Toolkit
Value Proposition Canvas for Host
Nation Explosive Ordnance Disposal
Products
& Servi...
Week 2: Needed to Get Out of the
Building
• Conducted 17 interviews
but none of them were
with end-users; failing to
adher...
Week 2/3: Our Initial LTTT MVP
Week 3: Anticipated Customer Workflow
Partner military unit
encounters IED
Link to EOD expert using
LTTT initiated: video
...
Week 3: LTTT Invalidated
“The last thing I’d want while defusing a bomb is a voice in my ear telling me
what to do.”
– EOD...
Products
& Services
-Higher-fidelity
post-incident
EOD analytics
and
reporting
-Higher-
accuracy device
classification
and...
Joint Improvised Threat
Defeat Agency
ARMY Research Lab
NAVEODTECHDIV
ARMY Asymmetric
Warfare Group
US Central Command
US ...
Week 4: What on Earth is actually
going on?
• We tried to work on a new product, but we couldn’t see
what was already ther...
Coalition EOD
Technicians
US POC
[JET]
[ONSITE]
FBI
[CONUS]
DOD EOD DATABASE
[CONUS]
JIDA DATABASE
[CONUS]
US AIR FORCE EO...
INTERNET
AFGHAN NETS
SIPRnet
NIPRnet
DOD
(DISA)
APAN
DOD
(NGA)
PiX
DOD (DISA)
APAN
AF RONNA
SharePoint 2010
BASEER
NIMS
CI...
Virtual Threat Toolkit:
Value Proposition Canvas for
JIDA J3 (JET Team)
Analysis of
organizational
strategies and
best pra...
Week 5: No Need for Another App
“Yeah, we don’t see anything really wrong with HYBRID -- it’s not at all a
bad application...
Joint Improvised Threat
Defeat Agency
ARMY Research Lab
NAVEODTECHDIV
ARMY Asymmetric Warfare
Group
US Central Command
US ...
Week 6: A New Approach
We discovered we needed a two-part
solution:
- Integrating existing technologies
- Helping our spon...
Customer Discovery, getting out
of the building, and getting
people in one room showed us
that under 15% of existing
syste...
INTERNET
AFGHAN NETS
SIPRnet
NIPRnet
DOD
(DISA)
APAN
DOD
(NGA)
PiX
DOD (DISA)
APAN
AF RONNA
SharePoint 2010
BASEER
NIMS
CI...
[UNCLASS]
JIDA DB
PIX/INDURE APP
HYBRID APP
INDURE
DB
[UNCLASS]
[FOUO]
[S/TS]
[UNCLASS]
HYBRID
Enhanced
deployment
via imp...
Products
& Services
Enhanced
development
of systems that
facilitate IED
reporting
- Development of products relevant
to mi...
Week 7 Onwards: Organizational
Learning
• We developed insights regarding the existing
solutions and how they connected (o...
Operational
Requirements
[MVP]
Technical
Specifications for
Solution
Organizational
analysis
enabling higher-
fidelity post-
incident EOD
analytics and
reporting
- -Development process that “...
Joint Improvised Threat
Defeat Agency
Specifically:
J2 Intelligence
J3 Operations
J8 Acquisitions
DIUx
Booz Allen Hamilton...
Final Hypotheses for JIDA
Hypothesis: JIDA requires greater inter-
divisional communication (between J
branches) to facili...
1. Tech integration
2. Deployment process
revamp
3. Smarter development
● Refining technical
recommendations
(combining Hy...
Week 9: Strategy of On-going Consultation
Cost Flow Diagram (T&M)
Back-End Android
Developer: SQL: (2x)
$58/hour
Project M...
IRL 1
IRL 4
IRL 3
IRL 2
IRL 7
IRL 6
IRL 5
IRL 8
IRL 9
First pass on MMC w/Problem Sponsor
Complete ecosystem analysis peta...
Thank you
● JIDA team
● H4D TA Team
● EOD Officers
● AWG Personnel
● SF FBI Bomb Squad
● Palo Alto Police Department
● US ...
Right of Boom Lessons Learned H4D Stanford 2016
Próxima SlideShare
Cargando en…5
×

Right of Boom Lessons Learned H4D Stanford 2016

42.645 visualizaciones

Publicado el

agile, bmnt, business model, corporate innovation, customer development, diux, dod, h4d, hacking for defense, lean, lean launchpad, lean startup, nsa, stanford, steve blank

Publicado en: Educación
  • Sé el primero en comentar

  • Sé el primero en recomendar esto

Right of Boom Lessons Learned H4D Stanford 2016

  1. Right of Boom [After EOD] Problem Sponsor: Joint Improvised Threat-Defeat Agency (JIDA) US Army & US Air Force Explosive Ordnance Disposal US Central Command DIUx Booz Allen Hamilton We seek to advance JIDA’s mission of countering improvised threats by developing systems, deployment strategies, and incentives for allied foreign militaries with the goal of improved intelligence fidelity. We seek to create a live tactical threat toolkit to enable real-time advice from US Explosive Ordnance Officers for the defeat of improvised explosive devices by partner and coalition forces. 124 Total Interviews
  2. Who we talked to
  3. Andreas Pavlou Nitish Kulkarni Alex Zaheer Alex Richard Gaurav Sharma Academic Program MA IPS 2017 Physics 2015 ME 2016 CS 2019 CS 2016 GSB MSx Subject Matter Expert? No (Software /Project Management) No (Sensor Fusion, Systems Integration) Yes (Software Engineering) Yes (Software Engineering) Yes (Military Information Logistics) Role Jack of All Trades Hardware Engineering SWE SWE PM Team
  4. The Emotional Journey Don’t understand the problem: Frustration Conception of problem evolves: Excitement Key organizational insights: Hope Recommendations and future plans: Satisfaction Are we relevant? Despair
  5. DARPA PCAS Team - modifying existing systems Joint Improvised Threat Defeat Agency - work with technicians who deal with problem day-in day-out General Atomics, Raytheon - contractors already working in this space may be able to provide key insight ARMY Research Lab NAVEODTECHDIV ARMY Asymmetric Warfare Group Usability- ensuring system is applicable to every forward deployment and tactical situation Bridging Gaps – enabling real time dialogue between US and host nation EOD Personnel-centered - creating a system designed around those who need it most Integrated - creating a tool that is an aid, not a hindrance, to time-critical operations Engineers - contractor/DARPA employees may provide technical insight Technicians - connections to those who need this product most Embedded operations need support from those who need product Translation experts - those who do job already Iterate - Multiple simulated field test scenarios with technicians to fit product to need Incorporation - product has many possible futures - can potentially be incorporated into other DoD product needs Primary - JIDA, EOD specialists in recurring forward deployment operations across systems Secondary - Military remote technical communications wings. Software can be co-opted for many different communications missions Fixed: Variable: Software development optimization IT Management/maintenance Deployment training Mission Achievement Criteria Enable clear, safe, common-sense communication of time-sensitive high- stress technical information Real-time IED Defeat Assistance for Coalition and Remote Forces: Week 1 Mission Model Canvas
  6. Week 1: Launch Point Hypothesis: • Build a Live Tactical Toolkit digital notebook for differently located coalition & US forces to address diffusing remote IED (Improvised Explosive Device) Experiments: • Interviewed 14 stakeholders including end users, planners, solution providers, and procurement teams • Ascertained organization structure and its relation to JIDA
  7. Week 1: Live Tactical Threat Toolkit Value Proposition Canvas for Host Nation Explosive Ordnance Disposal Products & Services Live Tactical Threat Toolkit linking US and Host Nation EOD forces - Timely and refined intelligence on IEDs and the tactics of insurgents using them - Real-time advice from US forces on how to defeat IEDs Customer Jobs Assess and defeat IEDs Lack of information on IEDs utilized by insurgents and their tactics Host Nation Explosive Ordnance Disposal Teams Gains Pains Gain Creators Pain Relievers -Less people dying -Improved counter-IED capability - Reduction of communication barrier with base-deployed US forces - Less likely to encounter IEDs without being forewarned
  8. Week 2: Needed to Get Out of the Building • Conducted 17 interviews but none of them were with end-users; failing to adhere to methodology •Frustration as we struggled to find EOD officers •Excitement as we finally managed to meet EOD units
  9. Week 2/3: Our Initial LTTT MVP
  10. Week 3: Anticipated Customer Workflow Partner military unit encounters IED Link to EOD expert using LTTT initiated: video feed/or minimal workflow depending on connectivity IED data logged Report sent to JIDA IED diagnosis and EOD process leveraging US expertise Resolution JIDA analyzes data and distributes intelligence as incentive for use
  11. Week 3: LTTT Invalidated “The last thing I’d want while defusing a bomb is a voice in my ear telling me what to do.” – EOD Staff Sergeant discussing reluctance to receive advice through LTTT “That’s what robots are for. We can’t do that with human lives. I can’t imagine me feeling comfortable from 50 kilometers telling someone to cut some wire. That’s scary proposition.” – Deployed EOD 1st Sergeant discussing discomfort with advisory role of LTTT We received a new problem statement from our sponsor that stated the real problem was ensuring sufficient intelligence collection for post-blast analytics.
  12. Products & Services -Higher-fidelity post-incident EOD analytics and reporting -Higher- accuracy device classification and intelligence - Gain and document intelligence from host- nation security forces as they encounter IEDs -Alignment of incentives between foreign EODs and JIDA such that blast data is reported -Bringing coalition forces into intelligence umbrella Customer Jobs -Analyze IED patterns and provide actionable intel -Lack of information on evolution of IED’s utilized by insurgents and their tactics -Reduced ability to provide actionable intel due to lack of raw data Gains Pains Gain Creators Pain Relievers -Accurate reporting data on IED usage -Development of predictive models for insurgent use of IEDs - -Reduction of communication barrier between US analysts and host nation security forces in the field - -Transition from material incentives to informational and culturally oriented incentives, allowing for more sustainable cooperation Week 3: Value Proposition Canvas for JIDA J2 (Intelligence)
  13. Joint Improvised Threat Defeat Agency ARMY Research Lab NAVEODTECHDIV ARMY Asymmetric Warfare Group US Central Command US Army EOD US Air Force EOD FBI Bomb Squad – SF Alaska Airlines/American Airlines -Post-Incident Reporting -Development of Reporting App Bridge - bridging the gap between forward deployed technicians and US remote technicians Incentivization – gamification to increase likelihood of use by partner militaries Reporting – a richer data set for JIDA’s mission Engineers - contractor/DARPA employees may provide technical insight EOD Technicians – iterative design process needs more user input JIDA SETAs Host Nation Forces Better intel JIDA More raw data Iterate - Multiple simulated field test scenarios with technicians to fit product to need Incorporation - product has many possible futures - can potentially be incorporated into other DoD product needs Primary - JIDA - Host Nation EOD - People in IED-impacted zones Secondary - Military remote technical communications. - Software can be co- opted for many different communications missions Transportation Industry – remote medical care Fixed: Software development IT Management/maintenance Deployment training Mission Achievement Criteria Enable clear, safe, common-sense communication of time-sensitive high-stress technical information and post-incident reporting and analysis dissemination Week 3: Accurate high-volume post- incident IED reporting Variable: Maintenance Equipment retrofit Equipment refurb
  14. Week 4: What on Earth is actually going on? • We tried to work on a new product, but we couldn’t see what was already there, not even an image. • It was almost impossible to get the sponsor to connect us with users of existing application • Frustration as we couldn’t understand why the current applications weren’t working and didn’t know what was broken
  15. Coalition EOD Technicians US POC [JET] [ONSITE] FBI [CONUS] DOD EOD DATABASE [CONUS] JIDA DATABASE [CONUS] US AIR FORCE EOD [CONUS] US ARMY EOD [CONUS] US NAVY EOD [CONUS] [UNCLASS] [UNCLASS] [UNCLASS] [FOUO [S/TS] Information flow, in an ideal world (still valid) [UNCLASS] [FOUO] [S/TS] [UNCLASS FOUO S/TS] [UNCLASS FOUO S/TS]
  16. INTERNET AFGHAN NETS SIPRnet NIPRnet DOD (DISA) APAN DOD (NGA) PiX DOD (DISA) APAN AF RONNA SharePoint 2010 BASEER NIMS CIDNE RED DOT DAIIS PORTALS HERMES Open Source Program CDC Dissemination Program Dept. of Army Intelligence Information System Smartphone Web Browser PiX Android/iOS Apps INDURE Android View/Report App INDURE Android View/Report App JIDA Hybrid Reporting App SOCCENT ATAK Tactical Ops App PiX SMS PiX WIKI Mediated WIKI INDURE API IRAQ/Other Nets Collaboration Portal Zimbra/Sharepoint 2007 FOUO/SBU INDURE Reporting/DB GPS Mapping
  17. Virtual Threat Toolkit: Value Proposition Canvas for JIDA J3 (JET Team) Analysis of organizational strategies and best practices to work with host nation forces - Increased ability to incorporate HYBRID app and into host nation EOD forces for higher fidelity intelligence Customer Jobs Enhance host nation EOD technicians Inadequate host nation use of HYBRID application Joint Expeditionary Team Gains Pains Gain Creators Pain Relievers Better situational awareness & IED reporting in conflict zones - -Less likely to encounter IEDs without being forewarned - -Creating better working relationships between US and host nation forces Products & Services
  18. Week 5: No Need for Another App “Yeah, we don’t see anything really wrong with HYBRID -- it’s not at all a bad application.” - JIDA JET Team “Can’t hand out Samsung tablets like candy.” - JIDA JET Team At this point, we were questioning our relevance to our sponsor.
  19. Joint Improvised Threat Defeat Agency ARMY Research Lab NAVEODTECHDIV ARMY Asymmetric Warfare Group US Central Command US Army EOD US Air Force EOD FBI Bomb Squad – SF Defense Threat Reduction Agency Palo Alto Police Department Post-Incident Reporting - Ensuring high volumes of detailed IED reporting data - Host nation forces continue to develop scalable intel reporting Bridging Gap - between IED intel collection and analysis Alignment of Incentives and Improved Reporting Information sharing and redundancy elimination EOD Technicians – iterative design process needs more user input JIDA JET- Insights regarding deployment of HYBRID Host Nation Forces - NEED: intel, interest - HOW: Incentivize - SABOTEURS: why help? JIDA - NEED: increased reporting - HOW: incentivizing coalition forces - SABOTEURS: Over-classification, inertia JIDA would benefit from changes to workflow and implementation strategies Primary - JIDA (Matt, Carrie, James) - EOD (Ike, Ethan, Christine, Whitner, especially J2 (Intelligence) and J5 (Strategic Plans and Policy) - Host Nation EOD (Sgt. Bashir) - Other coalition forces (Romania, Georgia, Bulgaria: no contacts) - Other entities such as FBI (Sean) - Palo Alto Police Department Fixed: Software redevelopment IT Management/maintenance Deployment retraining Mission Achievement Criteria Enable clear, common-sense communication of accurate actionable intel and post-incident reporting and analysis dissemination. Week 6: Accurate high-volume post-incident IED reporting Variable: Maintenance Equipment retrofit Equipment refurb [Full Names Redacted For Privacy]
  20. Week 6: A New Approach We discovered we needed a two-part solution: - Integrating existing technologies - Helping our sponsors modify their organization to better develop and deploy technologies
  21. Customer Discovery, getting out of the building, and getting people in one room showed us that under 15% of existing systems were relevant.
  22. INTERNET AFGHAN NETS SIPRnet NIPRnet DOD (DISA) APAN DOD (NGA) PiX DOD (DISA) APAN AF RONNA SharePoint 2010 BASEER NIMS CIDNE RED DOT DAIIS PORTALS HERMES Open Source Program CDC Dissemination Program Dept. of Army Intelligence Information System Smartphone Web Browser PiX Android/iOS Apps INDURE Android View/Report App INDURE Android View/Report App JIDA Hybrid Reporting App SOCCENT ATAK Tactical Ops App PiX SMS PiX WIKI Mediated WIKI INDURE API IRAQ/Other Nets Collaboration Portal Zimbra/Sharepoint 2007 FOUO/SBU INDURE Reporting/DB GPS Mapping Week 6: [RELEVANT] Customer Workflow
  23. [UNCLASS] JIDA DB PIX/INDURE APP HYBRID APP INDURE DB [UNCLASS] [FOUO] [S/TS] [UNCLASS] HYBRID Enhanced deployment via improved host nation partnerships Concerted campaign to promote importance of reporting to leadership
  24. Products & Services Enhanced development of systems that facilitate IED reporting - Development of products relevant to mission that actively catalyze increased intel gathering - Products that are easily pitched to Host Nation Customer Jobs Produce tech for intel reporting Lack of communication regarding pains between field and state based JIDA Gains Pains Gain Creators Pain Relievers Improved relations with Host Nation forces Useful products to give to Host Nation Development of system that not only overcomes in-theater technical challenges, but also helps incentivize reporting Week 7: Value Proposition Canvas for JIDA J6 (IT & Communications)
  25. Week 7 Onwards: Organizational Learning • We developed insights regarding the existing solutions and how they connected (or didn’t) • We diagrammed the ideal information workflow • We also discovered JIDA’s internal structure This enabled us to provide recommendations to improve existing solutions and their deployment
  26. Operational Requirements [MVP] Technical Specifications for Solution
  27. Organizational analysis enabling higher- fidelity post- incident EOD analytics and reporting - -Development process that “fails fast and fails often” and developers that are “proximate” to the problem, ideally developing with a team member with experience in theater - -Emphasis on needs-based development Customer Jobs Oversee improvement of Hybrid JIDA units operate in “silos” and lack adequate joint communication Gains Pains Gain Creators Pain Relievers Product that satisfies needs of various other JIDA parts, while also achieving mission Meetings involve representative from all parties related to problem -- dynamic engineering and rapid prototyping accounts for all parties’ input Virtual Threat Toolkit: Value Proposition Canvas for JIDA J8 Services
  28. Joint Improvised Threat Defeat Agency Specifically: J2 Intelligence J3 Operations J8 Acquisitions DIUx Booz Allen Hamilton US Central Command US Army EOD US Air Force EOD Defense Threat Reduction Agency ARMY Research Lab NAVEODTECHDIV FBI Bomb Squad – SF ARMY Asymmetric Warfare Group 1. Tech integration 2. Deployment process revamp 3. Smarter Development Bridge - bridging the gap between IED collection capabilities and analysis Make Using Hybrid more attractive High-speed intelligence allows for same-day incident analysis Improved Reporting – a richer data set for JIDA’s analysis Improved, targeted deployment strategy for HYBRID Existing technical solutions and technical staff (J6) within JIDA DIUx support in establishing forward- deployed team JIDA JET TEAM - Insights regarding deployment of HYBRID Host Nation Forces - NEED: intel, interest - HOW: Incentivize - SABOTEURS: why help? JIDA - NEED: increased reporting - HOW: incentivizing coalition forces - SABOTEURS: Bureaucratic Dissonance Team to address reporting problem in place -JIDA would benefit from an organizational process MVP outlining potential changes to workflow and implementation - Get all stakeholders in a room to ensure all parties are in sync Primary - JIDA(J2 (Intelligence)) and J8 (Acquisitions)) -IT & Communications JIDA (J6) - JET Team (J3) - Host Nation EOD Fixed: Overhead IT Management/maintenance Mission Achievement Criteria Enable clear, common-sense communication of accurate actionable intel and post-incident reporting and analysis dissemination. Accurate high-volume post-incident IED reporting: Week 8 Variable: $253,000 for 3 month consulting & development project
  29. Final Hypotheses for JIDA Hypothesis: JIDA requires greater inter- divisional communication (between J branches) to facilitate the development of effective technical solutions Hypothesis: Current technical solutions, if combined, will provide a solution of equal quality to standalone development in a shorter timeframe Hypothesis: Targeted deployment of HYBRID focusing on units with superior ethos of service in provinces with more IEDs and less militancy will gain more buy-in for HYBRID.
  30. 1. Tech integration 2. Deployment process revamp 3. Smarter development ● Refining technical recommendations (combining Hybrid & Indure, SMS capability) ● Ensuring features added to Hybrid are relevant and targeted ● Solidifying process solutions (best organizational structure practices focusing on information flow & incentivization) ● Further consultation with JIDA on bureaucratic obstacles and key stakeholders for implementation ● Gain JET team perspective for optimal deployment based on realities on the ground (given alternate perspective from JIDA ● Implement lean software development model that is agile + iterative Week 8: Critical Activities Validation from JIDA: •JIDA independently reached many of the same hypotheses via their work with BMNT • We gained significant credibility with JIDA by this point and it was satisfying
  31. Week 9: Strategy of On-going Consultation Cost Flow Diagram (T&M) Back-End Android Developer: SQL: (2x) $58/hour Project Manager: $53/hour Consultant (2x): $58/hour Front End Mobile Developer: Android: $58/hour UI/UX Developer: $58/hour Variable Costs (Labor) Overhead Cost: $1000/month Airfare of $1000/employee sent for deployment analysis in Iraq/Afghanistan Indirect Costs: Insurance, etc. $40/month/employee $/hour+13.5 % FICA + 9% profit margin USD 253,000 Quotable T&M Hourly Rate of $75 to include all expenses
  32. IRL 1 IRL 4 IRL 3 IRL 2 IRL 7 IRL 6 IRL 5 IRL 8 IRL 9 First pass on MMC w/Problem Sponsor Complete ecosystem analysis petal diagram Validate mission achievement (Right side of canvas) Problem validated through initial interviews Prototype low-fidelity Minimum Viable Product Value proposition/mission fit (Value Proposition Canvas) Validate resource strategy (Left side of canvas) Prototype high-fidelity Minimum Viable Product Establish mission achievement metrics that matter Post H4D Course Actions Discussing consultant project with JIDA and Booz Allen to optimize implementation of recommendations Catalyze a potential partnership between JIDA and DIUx in the works for a funded expeditionary team to analyze deployment of HYBRID Week 10: Final Assessment - IRL 6
  33. Thank you ● JIDA team ● H4D TA Team ● EOD Officers ● AWG Personnel ● SF FBI Bomb Squad ● Palo Alto Police Department ● US Army ● US Navy ● US Airforce ● Corporate partners ● Host nation forces

×