Building a New “Big Data” Set

We recently completed gathering a new (and very large) data set that will be beneficial for Agricultural marketers.  The data already gathered includes:

Soil type (and soil composition)

Slope

Erosion

Precise location

Watershed boundaries

Land usage

The data covers the entire US and all US territories.  Over 3,300 counties, with 10’s of thousands of observations defined by type, shape, and location per county.  It will become tremendously beneficial in improving targeting, cross-selling, and new-product growth in agriculture.

What separates this data set from previous tools, other than sheer size and completeness, is that it is BOTH Spatial AND Tabular. 

This means not only can you look at it on a map, we can analyze it with statistical tools.  That solves a major issue of complexity, since you can only see two or three dimensions on a map, which becomes the analytical limit for spatial data. But with tabular data and statistical software, we can utilize many variables and increase analytical power by hundreds or thousands of times.

New Data Means New Techniques

Data of this scope demands Community Analysis.  Community Analysis is a set of techniques designed to understand interactions, causations, and correlations in complex systems.  Through the process of Community Analysis segments are created that are homogeneous, substantial, actionable, differentiable, and accessible.  Different approaches have been developed in public health, genetic and molecular biology and database marketing that is being now be applied to agricultural markets. 

Represent the Environment

Community Analysis combined with “Big Data” allows a fuller representation of the environment.  Rather than comparing pre-selected similar fields, we can incorporate harvest monitor data compare similar sections of similar fields, areas of different types seeing similar conditions, and areas of the same type seeing different conditions.  Allowing each environmental variable to be available individually allows specific comparisons around environmental factors that could otherwise be ignored in field-to-field testing.

Find and Manage Cohorts

Cohorts are similar pieces of land or similar operations that share a common need or have a common benefit potential.  Cohorts can be defined by their environmental variables, operational characteristics, local agricultural practices, and many other factors.  Using “Big Data” and Community Analysis techniques similarities and difference can be defined that fit specific needs, goals, risks, or characteristics.

Refining Cohorts

Useful cohorts are segments that benefit from a specific message or action.  As a result, cohorts can be created on several levels.  Overall cohorts that segment entire farm operations or areas, product-driven cohorts that are prospect or re-sell targets for a specific product or service, and message-driven cohorts that are likely to respond to a specific type of appeal to suit their psychographics.  The process of refining cohorts occurs continuously as situations, offers, and response mechanisms change.

What This Means Operationally

Macro-level organizations create and target communications, offers, and support to constituents in a closed-loop fashion where the response (or lack or response) dictates the on-going cohort relationship.  Segmented communication is designed to fit individual needs, albeit the individuals are in specific groups, and those groups are large enough in terms of numbers and dollars to merit specific macro-level attention.  This delivers a one-to-one relationship feel while also allowing a segmented approach to communication that can be tracked back to ROI.

More to Come!

Now that gathering soils data is complete, we are moving on to gathering more environment-related data including variables such as stream flow and hourly weather.  We will also incorporate data from the three major censuses (US, AG, and Economic) and multiple meta-data points.  It is a big data set, and a big job, but it is a major step in progress to now have all soils!

For questions about use in US Agribusiness, contact Alan Weber - Alan@D2SG.net.  For questions about environmental or public policy use, contact Doug Ballou - doug@blue-window.org  If you are in Asia, contact HawZan Chong, HawZan@D2SG.net