08448380779 Call Girls In Civil Lines Women Seeking Men
Whitepaper: Agricultural Systems + Data Outlook 2Q14
1. Whitepaper: Agricultural Systems + Data Outlook
The Data Guild, 20140220
How can data be leveraged to make food production and distribution systems more responsive,
resilient, and efficient?
An ecosystem of agricultural data has been quietly evolving, and is rapidly becoming a vital
component of global food security. The data rates and variety are vast: remote sensing via small
satellites, sensor networks in the fields, tractorsasdrones, and more. Many issues implied by
this category of data, however, are quite subtle and in some cases counterintuitive. Given that
this field is relatively new and not particularly organized yet, key learnings may be adapted from
other sectors where largescale data and analytics have already played a transformational role:
finance, intelligence, ecommerce, telecom, energy, etc.
We examine both key questions and the evolving vendor landscape for agricultural data in the
context of supply chain analysis, defining nomenclature for components of the ecosystem and
identifying key issues for consideration. Ultimately, this paper is at best an early draft for a much
longer and more comprehensive study: it provides a rubric for analyzing the complexities of
agricultural data, along with examples for the identified categories.
Impact
Farming represents the single largest employer globally, as the primary livelihood for 40% of the
world’s population. There are more than 500 million small farms worldwide , most of which are 1
family farms that rely on rainfed agriculture. The global domestic product for agriculture was 2
nearly $15 trillion in 2013 and the agricultural real estate in the U.S. alone is valued at over $2 3
trillion. The impact of these figures needs to be considered in the context of two factors:
resource consumption and production asymmetries.
In terms of resource consumption, recognize that 70% of the world’s freshwater resources goes
toward agriculture . This figure is estimated to reach 89% by 2050. Meanwhile, soils are being 4 5
1
“Small farms: Current Status and Key Trends”, Oksana Nagayets, Future of Small Farms (2005), p. 355
2
Agriculture, value added (% of GDP), The World Bank (2014)
3
National Agricultural Statistics Service, USDA (2014)
4
FAO Aquastat
5
UN Water Facts and Figures (2013)
Agricultural Data (Q2 2014) The Data Guild Page 1
6. ■ Titan Aerospace
○ lowaltitude: aerial imaging, aerostats, drone orchestration, etc.
■ TerrAvion
■ HoneyComb
■ PrecisionHawk
■ Raven Aerostar
■ Skycatch
● tractor telemetry
○ John Deere FarmSight, Apex
○ AgLeader SMS
○ Trimble FarmWorks
● farm robotics
○ Blue River
● sensor networks
○ localized weather
■ WeatherHawk
■ Ambient Weather
○ water usage
■ Hortau
■ PowWow Energy
■ Agronode
○ nutrient testing
■ Solum / Monsanto
○ pest management
■ Semios
■ Dolphin Engineering
● direct data entry
○ much of the most relevant data is entered manually by farmers
● inferential sensors
○ as in soft sensors14
● import from other sources (government, semipublic agencies, etc.)
○ weather
14
“Design of inferential sensors in the process industry: A review of Bayesian methods”, Shima
Khatibisepehr, Biao Huang, Swanand Khare, Journal of Process Control (Nov 2013)
Agricultural Data (Q2 2014) The Data Guild Page 6
7. ○ local rainfall
○ soil distribution
○ pest/disease spread
○ water allocations
○ snowpack variance/evaporation water cycle
○ soil compaction
○ hazards: pipelines, cables, underground irrigation
A number of issues beleaguer this data collection stage, including:
● Poor communications infrastructure in rural areas
○ lack of adequate cell coverage in rural areas (depends on the region)
○ satellite upload temporarily blocked by cloud cover and other weather events
○ coops among neighboring properties share towers, where overlap is possible
● Serious privacy concerns
○ see below in “Drivers: Privacy and Security Issues”
● Data quality
○ lack of calibration, high variance on devices (need for maintenance, etc.)
○ additional factors that explain variance in yield map results15
● Data Silos: Vendors must surface metadata to help overcome problems of data silos on
farms. Here, standards could play a key role in spurring innovation broadly, and creating
new synergies between players in the market. Specifically, support for popular geospatial
formats, data import/export, and effective licensing that does not impede data
aggregation downstream are all key needs.
The field of sensor design in general is undergoing a rapid evolution. For example, selfpowered
sensors from Piezonix can function continuously by scavenging energy. Arduino and other
hardware platforms have opened up new capabilities for rapid prototyping and small
formfactors, even into the hands of hobbyists – which is a particular boost for entrepreneurs.
Meanwhile, National Instruments has a large market share for production of sensors, and much
of its market among design engineers is outside of the U.S.
Mobile, lowaltitude data collection methods such as drones and aerostats may help augment
the remote sensing from higher altitudes – in other words, fill in gaps on demand, provide high
resolution baseline measures, etc. These could help augment communications where cell
coverage is sparse. Then again, use of such equipment may create negative reactions.
Increasingly, consumergrade mobile devices provide substantial platforms for the data
collection required in agriculture. Examples include Project Tango from Google, used for high
resolution 3D mapping – or for that matter, the widespread use of smartphones and tablets by
15
Yield Monitors and Maps: Making Decisions, Larry Lotz, Ohio State (1997)
Agricultural Data (Q2 2014) The Data Guild Page 7
10. Note that infrastructure businesses tend to bundle data enrichment and analytics services in
addition to a core function of data transport and storage. So far, vendors tend to differentiate with
particular valueadded specialties:
● timeseries analysis and geospatial analysis
● metadata alignment / schema / lineage for a wide variety of data sets
● blending farm data with other external data (Open Data from gov sources)
● support for curation, addressing data quality issues introduced during collection
● allowing customers to create data products for resell
● managing interfaces (aka “app stores”) for thirdparty data products
● integrating mobile devices with service fleets
As data products continue to leverage machine learning, other important issues for elastic
infrastructure include:
● contingencies to upload data at scale via alternative channels
● data preparation at scale: imputed missing values, feature engineering, etc.
● ultimately, provide for queries, approximated metrics, etc., to feed analytics
● compression technologies
● data processing and computation at the edge (as noted above)
Backlash based on privacy concerns from farmers could ostensibly change infrastructure
strategies significantly. Also, privacy laws in different regions (e.g., EU) will have impact on data
policies. Both factors indicate eventual regulations in this stage of data infrastructure. The
traditional IT vendors have addressed these kinds of issues before many times; however,
startups may encounter difficult challenges advocating at that level of policy and government.
Focusing on the core problems of elastic infrastructure, most of the vendors do not pay enough
attention to the needs of data preparation (curation, cleaning, metadata alignment) prior to
serving data to the analytics downstream. Experience from other domains (e.g., adtech, social
networks) shows that the bulk of the work performed in data infrastructure at scale is in 17
cleanup prior to analytics use cases. Marinexplore is an exception in this case, providing
metadata alignment across a wide variety of data sources.
Another issue concerns data workflows on a farm. Generally there are teams of people, working
concurrently at different locations. There are important requirements for data to be updated
across the team in realtime. Given the scale of the data, this will require effective use of tiered
architectures, balancing what data preparation gets handled in the cloud versus on a mobile
device. AmigoCloud is an exception in this case, providing realtime updates among the mobile
devices used by a team on a farm.
17
Data Jujitsu: The art of turning data into product, DJ Patil, O’Reilly Radar (2012)
Agricultural Data (Q2 2014) The Data Guild Page 10
14. aggressively extractive . Cloudbased services outside of a distressed region could help 19
disintermediate entire layers of corruption.
Farm Operations
At the fourth stage, which we label as farm operations, there are a variety of different functions
to manage, including:
● seed catalog selection
○ DuPont Pioneer
○ Yield Pop
● activity calendar
● weather – both short and long term predictions
● asset inventory
○ Trecker
● yield maps
○ Farmers Edge
○ Solapa 4
○ VitalFields
● livestock management
○ FarmerOn
● commodity price monitoring
○ FarmLogs
○ Agronometrics
○ HarvestMap
● accounting workflow
○ Granular
● contracts, deliveries
● harvest storage
An essential point of precision agriculture is that by combining analytics based on a variety of
data collected from sensors (including satellites, drones, etc.) along with field topography, farm
operation history, etc., the variability of crops at specific locations can be leveraged to improve
overall yield: modify the seed density, modify the inputs, etc. For example, consider the
statement from the acquisition of Climate Corporation by Monsanto:20
19
Many intermediaries buying harvests are effectively “loan sharks” who charge 50% interest rates, etc.
20
“Monsanto to Acquire The Climate Corporation, Combination to Provide Farmers with Broad Suite of
Tools Offering Greater OnFarm Insights”, Business Wire (20131002)
Agricultural Data (Q2 2014) The Data Guild Page 14
17. The sixth stage, which we label as market aggregation, concerns the kind of agriculture data
that most people are already familiar with:
● global market analytics
○ Mercaris
○ GroVentures (Africa focus)
● commodities trading
○ AgFlow
● market intelligence
○ Cleantech Group
○ Food Tank
○ Praescient Analytics
○ Palantir
○ Stratfor
● public policy
○ USDA
○ GODAN
○ FAO AQUASTAT
Traditionally this stage of agricultural data has been focused either on shortterm opportunities
(commodities trading) or very highlevel concerns from qualitative perspectives (policy making , 24
global food security, natural resource management).
Opportunities abound for leveraging feedback loops in the data, algorithmic modeling, aggregate
data services driving hyperlocal (perblock) recommender systems, etc. This is especially the
case as sensor networks become more pervasive and as remote sensing services continue to
provide better, higherresolution data. A key point is to focus the data services so that markets
steer away from shortterm extractive practices (hedge funds) and toward opportunities to apply
data to make food production and distribution systems more responsive, resilient, and efficient.
General Insights
An earlier question asked, who are the stakeholders in this system? We find a number of actors
who represent stakeholders in the flow of agricultural data, each of whom represent diverse,
sometimes conflicting, interests in the larger value chain:
● farmers
● corporate farms
● coops
● public/private partnerships, e.g., water districts
24
Of course, the impact of policy changes should be modeled and considered prior to implementation.
Agricultural Data (Q2 2014) The Data Guild Page 17
18. ● technology vendors
● shippers/storage
● wholesalers/distributors
● food processors
● end uses: groceries, restaurants, etc.
● public policy makers: USDA, CAP, etc.
● financial markets/traders
Which of these stakeholders require more transparency into the data flows? For example, do the
end use cases such as restaurants require traceability at the level of individual palettes, all the
way through food processors, shippers, etc., back to the origin at a farm? Does that need for
traceability conflict with legitimate concerns about data privacy, or could it open the door for data
security concerns and other abuses? In any case, we can use these identified stakeholders to
analyze the different issues identified for agricultural data.
Overall, the point of data flowing across these several stages is to generate actionable insights,
at very large scale, and in many cases with relatively low latency. That is a tall order, and voices
within agriculture lament the volume/velocity/variety of the data, and the “needle in a haystack”
effect of attempting to draw actionable insights from mountains of raw data.
Even so, it’s important to keep in mind that other verticals – e.g., finance and telecom – have
achieved this already for their own specific needs. Agriculture is known for relatively
conservative practices, with perhaps a 10year cycle for adopting new technologies. To change
that aspect in any way, one must understand the root causes: among which uncertainty and
enormous risks dominate whole markets, local communities, and families. Farmers earn 40
paychecks in a lifetime, and there is little margin for error.
Even so, as the following drivers indicate, there are good reasons to accelerate key areas of
technology adoption. Some of the more conservative bias against new technologies may need to
be adjusted due to other looming priorities.
Driver: Drought Outlook
Circa 2014, the predominant issue being discussed in California (and hence, proximate to many
of the technology vendors) is drought. Variance in snowpack levels causes serious shortfalls in
water resource allocations via aqueducts – with obvious impact on farm operations now in crisis.
In addition, variance in the timing of the water cycle stress natural resources and infrastructure
throughout these connections, from snowpack to farm or food processor usage: reservoirs, river
ways, aquifers, levees, seawater incursion, etc. Attempts to control nature usually fail sooner or
Agricultural Data (Q2 2014) The Data Guild Page 18
23. community discussions.
Moreover, OADA has substantial support from Monsanto, which has stated its aims to 30
integrate and assure data privacy:
The data created by a farmer, or generated from equipment the farmer owns or
leases, is owned by that farmer and should be easily managed.
Other interesting efforts toward these ends include:
● Standardized Precision Ag Data Exchange (SPADE) Project
● Spatial data infrastructures for precision farming data standards and system design
criteria, Martin Weis (2007)
Compelling solutions for farm operations will focus on interoperability, welldefined interfaces,
and the ability to accommodate “plugins” from alternative analytics sources. Similar effects
were observed among early Internet vendors as Web 2.0 practices drove adoption of open
standards. In particular, it will be important for vendors to:
● surface their products’ metadata to help avoid potential data silos
● allow for data import/export between vendors, while propagating schema and lineage
● support popular geospatial formats, datetime formats, etc.
● use effective data licensing that does not impede data aggregation downstream
Even so, the capital structure of corporate farms may conflict with adoption of more advanced
analytics and interoperability. If so, that could open opportunities for competition from smaller
farms and new kinds of dataintensive coops.
One of the open standards that is becoming quite important for agricultural data is RFID, in terms
of traceability of farm products, accountability of processing and procurement, etc. From a public
policy perspective, this also provides the capability to reverseengineer the procurement chain 31
in case of illness or contamination.
Arguably, there is an open standard used by analytics vendors in general, called PMML, which
could readily be used in agriculture. It provides for model portability, guards against vendor
lockin, allows analytics to scale (e.g., on cloudbased infrastructure) independent of where the
models are trained, etc. Vendors providing analytics products and services would need to agree
on PMML for model import/export. That is likely to occur over time anyway as more traditional IT
vendors move into this space.
30
“Guiding Principles on Data and Privacy”, David Friedberg, The Climate Corporation
31
“RFID's Role in Food Safety”, Mark Roberti, RFID Journal (20130729)
Agricultural Data (Q2 2014) The Data Guild Page 23
24. Driver: Funding Analysis
As recently as Q1 2012, the outlook for clean tech investments had been receding . Many of 32
the Silicon Valley venture capital firms backed away from agriculture. One notable exception is
Khosla Ventures, which has been consistently engaged in this area. They funded two recent
acquisitions by Monsanto Growth Ventures: Climate Corp and Solum.
Other predominant sources of capital investment include:
● family office investments
● strategic funds: Monsanto, Dow, BASF, etc.
● investment bankers
● challenge funds/incubators: StartUp Chile, Africa Enterprise Challenge Fund
● crowdfunding: Kiva, AgFunder, Angel List
One geopolitical aspect becomes apparent in an analysis of the agricultural data vendors: a
large cohort of Ag data startups are based in Southern Hemisphere, and most of these have 33 34
been involved with Startup Chile. So there is a competitive tension emerging between Monsanto
Growth Ventures and other strategic funds in the Northern Hemisphere (mostly Silicon Valley
since 2013) and incubators in the Southern Hemisphere.
This regional economic tension will likely be shadowed by public policy. For example, Mexico
recently ruled against allowing use of Monsanto GMO products . This tension echoes among 35
the influential buyers, e.g., Whole Foods has announced its intent to require GMO labeling by 36
2018. Mexico plays a unique role in the borderlands: it is within the Northern Hemisphere and
obviously sells much of its output to the United States, and yet politically and culturally it finds
resonance with other Latin American countries in the Southern Hemisphere.
This begs two questions. On the one hand, will other Silicon Valley venture capital firms rush
back into clean tech investments following the two recent (circa 2014Q2) successes of
Khosla/Monsanto? On the other hand, will national governments tilt the geopolitical playing field
by subsidizing incubators following the success of StartUp Chile?
32
“The state of cleantech venture capital: what lies ahead”, Matthew Nordan, GigaOm (20130327)
33
“Avance Proporción de Países Seleccionados Top 5”, slide 9, StartUp Chile (2014)
34
AngelList “Agriculture Startups” (2014)
35
“Mexico Judge Bans Monsanto’s GMO Corn”, Devon Pena, Environmental and Food Justice (20131011)
36
“Our Commitment to GMO Labeling”, Whole Foods Market
Agricultural Data (Q2 2014) The Data Guild Page 24
25. Conclusions
The following trends are in progress for each of the six stages:
Stage 1: Data Collection
● the needs of this stage are complex, but the vendor landscape is becoming crowded
○ implies much consolidation among startup vendors
○ new entrants face headwinds in the face of market fatigue
● remote sensing products tend to augment sensor networks
○ implies largescale use cases for data fusion, i.e., cloudbased apps
○ startups that focus too much on one data source are probably doomed
● farm robotics and tractor instrumentation (mobile) will augment static sensors
● absent key learnings from other verticals, startups tend to repeat critical mistakes
○ creating data silos
○ desire to "own" the data and the tech stack
○ lack of promoting standards for interoperability
● data quality, communication, and privacy issues beleaguer vendors
○ implies that regulatory policy will emerge, enterprise incumbents may dominate
○ as public policy fails to respond, private solutions emerge
● computation and decisions/alerts are pushed to the edge of new sensor networks
○ sensors become extended computational resources that can take action
○ compression techniques, coupled with computational resources in lowpower
packages create new opportunities to pervasive sensors nets that rely less on
alwayson network connectivity
● farm operations use cases will drive toward more realtime processing
○ implies pushing computation out to the edge, as a truism throughout the larger
Internet of Things space
○ other verticals (finance, telecom, search) confronted this need already
○ in terms of open source strategy, look to Twitter for leadership
Stage 2: Elastic Infrastructure
● traditional IT infrastructure vendors will move into the space, edging out the startups:
○ economies of scale for networking, storage, cloud services, etc.
○ incumbents can navigate regulatory policy more effectively
○ their business tendency to move up the stack for lucrative verticals
● startups attempting a “seed to sale” strategy are mostly doomed
● successful startups will differentiate by focusing on specialized use of infrastructure:
Agricultural Data (Q2 2014) The Data Guild Page 25
26. ○ emphasize features that address ongoing painpoints, such as data preparation at
scale prior to analytics, e.g., data federation, cleanup, curation, metadata
alignment, etc.
○ edge their way into the subsequent stage of analytics by specialized use of
elastic infrastructure: timeseries, geospatial, imaging, etc.
○ position themselves for acquisition by IT infrastructure incumbents
● Demand for better communications infrastructure grows
○ New opportunities for both established vendors and new entrants to fill gaps,
especially through strategic partnerships
○ Progressive communities establish publicprivate partnerships that include tax
incentives, financing, and other components that make buildout more
costeffective
● North/South Hemisphere tension emerges
○ IT incumbents from the industrialized North displace business models for startups
predominantly based in the developing South
○ long product cycles in the North may benefit the relatively nimble startups in the
South, if local politics do not interfere
○ this does not imply a clear “winner” between the two
Stage 3: Analytics
● analytics products are not an end in themselves; they feed metrics into farm operations
○ misplaced emphasis at this stage poses additional risks for siloed strategies
○ analytics offerings that are tightly coupled to feedback loops with users in specific
workflows will edge out static dashboards
○ machine learning becomes a key factor use cases where local optimization and
customization provides measurable benefit
● platforms leverage the coming generation of lowpower, highcomputation sensors
networks
○ present new opportunities for efficient, highlytargeted analytics that rely less on
constant connection to the cloud
● "seed to sale" strategies drive startups to bundle infrastructure services with data
collection analytics
○ implies significant duplication of resources and extra costs to farmers
○ duplication costs drive acquisitions and mergers, plus “seed to sale” aversion
○ on the other hand, bundled services of multiple vendors (through strategic
partnerships) could succeed if the value of such a bundle is obvious
● large analytics vendors may avoid infrastructure plays
○ left to IT incumbents who typically pursue up the stack in lucrative verticals
● calibration is a major issue in practice, huge downsides for analytics innovation
○ requires large capital investments, aggressive partnering, etc.
Agricultural Data (Q2 2014) The Data Guild Page 26
27. ○ will drive acquisitions nearterm by large analytics vendors
○ may drive "crowdsource" calibration services longterm
● as regulatory policy emerges, predictive analytics come under pressure
○ implies tradeoffs in favor of accountability at the cost of accuracy
○ corporate farms may be too conservative to navigate those issues
○ potential opportunities for new kinds of dataintensive coops
● without attention to interoperability and standards, analytics products become too static
and limit adoption
○ crucial needs being missed: feature engineering, model portability, tournaments
Stage 4: Farm Operations
● One Size Fits All, related to "seed to sale", represents an antipattern for startup viability
○ the strategy tends to collapse after early adopter phase wanes
● farmers demand immediate access to data via mobile devices
○ natural response to grappling with technology learning curves
○ accentuates needs for cloudbased infrastructure and realtime processing
● business needs for recommendation services at this stage
○ drives need for feedback loops and data products from aggregate stages
○ will tend to track a similar market evolution in ecommerce
● technology giants focused on yield optimization
○ ROI is a better metric for most farmers’ success (outside of the U.S.); however,
that is even more difficult to model
○ yield increases have come at the expense of disproportionately larger increases
for inputs
○ focus on the precision estimates of aggregate yield presumably serves the
interests of financial traders more so than farmers overall
● data at this stage presents a more lucrative target for bad actors
○ this heightens concerns about privacy/security
Stage 5: Distribution
● traceability is a driver at this stage
○ implies new kinds of business opportunities
○ surfaces new issues for privacy/security and accountability
● processing consumes more water+energy resources than farms do
○ must ensure accountability for consumed resources
○ accentuates needs for monitoring, data collection, elastic infrastructure, analytics,
etc., in parallel to farm sensors
○ potentially a big sustainability win and a big economic opportunity as the costs of
water+energy increase
● what are the water+energy implications of a harvest after it leaves the farm?
Agricultural Data (Q2 2014) The Data Guild Page 27
28. ○ consumers will demand transparency
○ implies opportunities for feedback loops and data products
Stage 6: Market Aggregation
● traditional focus at this stage have been ineffective
○ shortterm opportunities (commodities trading)
○ very highlevel qualitative concerns (policy)
● major opportunities for leveraging feedback loops in the data
○ algorithmic modeling, aggregate data services, etc.
● refocus data services to steer market opportunities
○ steer away from shortterm extractive practices (hedge funds)
○ apply data to make food production responsive, resilient, efficient
Outlook: Forced Asymmetries, Tail Wagging the Dog
Increasing variance in snowpack levels and rising rates for anthropomorphic evaporation in
wealthy regions (e.g., California) will stress local infrastructure which is already at crisis levels
(e.g., aqueducts and transportation). This will force even greater asymmetries in production as
well as in technology innovation – in terms of relatively wealthy versus poor regions, where the
latter increasingly gain the upper hand for technology and expertise. Of course, niches will
persist, such as local organic farms near metro areas in the U.S. Even so, some of the large
stakeholders have vested interests in undermining even these: technology giants (for political
momentum, homogenizing toward their agenda) and financial traders (surfacing risk through
exposed data, extracting capital).
In the wealthy regions: available water resources will be redirected to capitalintensive,
highmargin crops such as orchards, vineyards, and premium livestock to preserve capital.
Meanwhile, the productivity and longterm commercial viability of these properties is decreasing.
Some crops will push north as grow zones change. Other crops will be pushed toward imports
(e.g., Southern Hemisphere) or incentivize urban agriculture at scale. Extensive monocultures
(e.g., grain) become increasingly subject to systemic risks on several fronts.
As risks increase for capitalintensive crops in wealthy regions, this segment of farmers will
become more averse to technologies that open the door to potential privacy and security
breaches. They have far too much to lose, while multiple bad actors have too much to gain. In
particular, financial interests could engage in aggressively extractive practices at a scale that
would make the 2008 credit default swap crisis look small by comparison.
An essential tension is that technology giants will insist on owning the data collection, analytics,
Agricultural Data (Q2 2014) The Data Guild Page 28