A Cyber-Physical Analyzer System for Precision Agriculture

Over the past decades, the need for more precise and sustainable farming operations derives from the rapidly growing population across the world and the demand for organic food across agriculture sectors [1,2]. Understanding soil and crop performance can strengthen farming productivity and resiliency. Currently, crop analysis is slow as the primary means of assessment is based on the manual data collection in field-based studies under various environmental conditions and soil quality [3]. Emerging technologies can be utilized for addressing existing limitations in farming practices that are not being met; for instance, remote sensors for time-series data collection, drones and satellites with high-quality cameras for spatial image collection and smart phone apps for real-time control and recommendations. Such data and extracted knowledge can transform current soil-crop analysis to the next-generation smart farming. 

The motivation behind this study lies in addressing the inherent limits of the existing approaches for data collection and filter out redundant data across farming, with particular interests in organic farming, using fertilizers from livestock waste. Up to this time, most of the efforts have been built upon the physical components and there is a growing need for cyber-based computational elements to satisfy a market-responsive organic crop supply. Internet of Things (IoT) devices can: (a) Automate the data collection from soil/crop analysis, (b) Correlate such data from different farming conditions and production goals and (c) Forecast soil performance for any particular farm. Advanced Cyber-Physical Systems (CPS) can complement human judgment by implementing real-time data analytics, proactive analysis and predictive models to detect and solve problems before they occur (e.g. process failures) [4,5]. 

Advanced farming plays a key role in food security, for example, using remote devices (e.g. drones and satellites) for big data analytics allows for better forecasting of crop yield estimation under a wide variety of conditions. Remote sensing and monitoring of terrestrial vegetation provide useful insights in agriculture infrastructures for determining sustainable, optimal strategies to meet national priorities [6] (e.g. feed the growing population, expected to reach nine billion by 2050 [1,7]). 

Prior research focused primarily on crop variety identification under real-world conditions (e.g. environment, irrigation and fertilization) and resources (e.g. water, energy and labor), as well as phenotyping studies for measuring pH soil level and soil nutrient depletion rate [8-10]. Later studies focused on productivity and sustainability challenges, e.g. crop analysis and optimization [11,12]. Crop analysis includes monitoring the plant phenotypes to identify crop variety [13]. Recent studies established foundational concepts in promoting soil performance and organic food commercialization through IoT technologies and improving efficiency and productivity by advancing integrated technologies and assessments [14-17]. 

During the past three decades, engineers have developed tools and methods for controlling the physical processes and simultaneously computer scientists have developed practical methods for cyber systems [18]. However, there is a huge gap between the cyber and physical worlds, where information and original materials are transformed and exchanged. CPS, as the fourth major industrial revolution after the first three revolutions (i.e. mechanization, electrification and automation), embraces physical components and advanced cyber-based technologies (e.g. high computation power, remote operations and communication protocols) to locally control systems and analytics [19] and optimize operation performance metrics (e.g. accuracy and reliability) [20]. For instance, advanced CPS utilizes low-cost, small size transistors that offer more flexibility of remote/in-line sensors to improve performance measurement in the smart manufacturing industry [21-23]. 

The integrated multi-scale monitoring of soil nutrients and crop yields study for data collection employs remote sensing, using a Moderate-Resolution Imaging Spectro Radiometer (MODIS) [24,25] and land remote sensing satellite (Landsat) [26] and a drone-carried multispectral camera. Collected data are Spectral Vegetation Indices (SVIs), e.g. indices of Wetness, Greenness and Brightness (WGB). Leveraging the Google Earth Engine public data archive, all historical and near-real-time Landsat (30 m) and MODIS (250 m) Normalized Difference Vegetation Index (NDVI) and Normalized Difference Water Index (NDWI) products, with surface reflectance data will be utilized to evaluate changes in crop moisture stress and productivity with crop phenology at our field site and the surrounding region [27]. 

Unmanned Aerial Vehicle (UAV) along with the related control and sensor technologies has developed at a fast pace in the last decade [28]. As a result, a large number of new UAV (also known as drone) applications were introduced across various domains. One of the most popular uses of drones has been in agriculture for monitoring crop health; these tasks are often associated with the concept of precision agriculture [29,30], which refers to a site-specific field management approach based on observation of the crop variability in the field. Accordingly, the provision of aerial image data in combination with AI algorithms has been widely employed in identifying weeds, diseases, pests and other crop stresses [31]. 

Furthermore, these soil analyzers can be georeferenced (associating a map or image with spatial locations), employing Global Positioning System (GPS) technology. Thus, through a repeated stratified sampling program farmers can: (1) Characterize the spatiotemporal variability in soil fertility and moisture throughout the dormant and growing season [32] and (2) Use model projections of future nutrient and moisture availability to inform near-/long-term soil amendments, using Geographic Information System (GIS) and computer-statistical techniques [33-35]. 

In general, collected data from airborne multi-spectral and hyper-spectral cameras in agriculture is used for calculating SVIs [36-38]. A number of SVIs have been proposed in the literature, e.g. NDVI, Transformed Chlorophyll Absorption in Reflectance Index (TCARI) and Optimized Soil-Adjusted Vegetation Index (OSAVI) [39,40]. Prior research reported of high correlation between SVIs-derived from UAV-collected spectral data and crop health and related traits, e.g. Leaf Area Index (LAI) [39,41,42].

Despite the confirmed economic and environmental benefits of the use of drones in precision farming, professional reports and scientific surveys indicate low rates of adoption of the technology [43]. Among the many barriers towards increased adoption, the need for new tools for agricultural data analytics that are ‘useful’ and ‘easy-to-use’ has been acknowledged as an essential factor [44]. This study aims to integrate novel tools and technologies that will reduce the efforts of deploying drones in agriculture and promote advanced technological solutions for environmentally and economically sustainable farming systems. 

Growers typically assess the field and crop nutrient status through chemical soil testing prior to planting, plant chemical tissue testing during the growing season and more recently through sensor-based measurements, e.g. NDVI [45]. However, relatively lower usage of these sensors is currently in place. 

Strategies developed in prior work have not been integrated to date to transform farming resources into sustainable, organic products [46]. Integrating sustainability assessment with CPS for real-time control facilitates complex decision making for socio-environmental responsibility with techno-economic aspects, which will be implemented in this study to address global concerns [47-49]. 

Ultimately, the convergence and miniaturization of analytical techniques and technologies are making the practice of mechanized, precision agriculture closer to widespread adoption [50]; however, developing a framework capable of describing current, while also projecting future soil/crop nutrient conditions is needed.

This study combines existing farming approaches with advanced cyber-enabled tools to construct an adaptive SeDS analyzer system and support farming. This study includes both qualitative and quantitative methods to address the lack of required data along with associated deficiencies across cyber-physical farming systems. The integrated method in this study investigates several parameters (e.g. soil physical, chemical and biological properties) and variables (e.g. crop yield and quality) under real-world conditions to extend dimensions of CPS and data-driven decision-making, as described in this section. 

The “bottom-up” approach proposed herein to characterizing and projecting local soil and environmental condition represents a potential paradigm shift from current strategies that look to solely optimize yield by relying on generic, empirical relationships between fertilizer rate and crop yield. 

Analytical and observation techniques for soil and crop analysis 

A standardized, multi-scale methodology is developed to quantitatively describe the spatiotemporal variability of within-field soil moisture, temperature, macro-nutrients, and crop phenology and crop yields, using both remote sensing (e.g. satellite and drones) and field monitoring techniques. The data will be used to train dynamic models. Overall, this subsection puts field-scale observations into a broader historical and regional context for assessing soil health and crop productivity.

Both NDVI and NDWI can be used to generate regional, crop-specific empirical models that measure and project crop yields based on sensing throughout the growing season. Likewise, surface reflectance data can be used to calculate indices of WGB. These indices will be used to develop date-of-year phenological LOESS models to predict a fields’ WGB for any day of the year. This baseline phenology model will be useful because it will reveal the impact of any treatment effect on crop yield and development through a residual analysis [51,52]; however, satellite access data operations (e.g. Landsat 8) do not provide real-time data access and the period for a full Earth coverage is typically in the range of two weeks. Thus, it takes time for acquired images to become available to end-users for processing. 

The spatial resolution of satellite-gathered images is in the range of 5-75 feet per pixel (a field-of-view of 10-50 km) and also in cloudy weather conditions there is missing imaging data for the land areas under cloud coverage [53,54]. Considering the listed drawbacks, drones represent an attractive alternative for sensory perception of earth phenomena and agricultural fields that are affordable and available for on-demand immediate surveying and processing of collected images. The spatial resolution of drone-acquired imagery ranges from 0.25 to 4 inches per pixel (a field-of-view of 10-500 m). Consequently, the fine spatiotemporal resolution provided by drone images enables more frequent soil health and crop performance assessment in smaller field areas. This capacity is highly valuable for farmers because it alleviates the assessment of the impact of different fertilizers based on various combinations and ratios. 

This study integrates hyper-spectral cameras carried by drones. These cameras have been increasingly used in agricultural applications [55]. Conversely to the conventional vision cameras that provide three image channels associated with the Red, Green and Blue (RGB) light wavelengths of the visible portion of the electromagnetic spectrum, hyperspectral cameras provide additional channels of light wavelengths in the near-infrared portion of the electromagnetic spectrum. Thus, these cameras capture crop information that is not visible to farmers or obtained by conventional RGB cameras. Such information significantly contributes to the crop performance assessment, as it is related to the pigmentation (natural coloring of plant tissue). 

In particular, vegetation with reduced levels of chlorophyll (as the primary photosynthetic pigment in plants) reflects less energy in the near-infrared region than healthy plants [56]. Similarly, reduced leaf moisture shifts the reflected spectral content, which is used for identifying water deficiency in crops [57]. Furthermore, the high-resolution drone imagery can be used for extracting additional crop features, which are not captured in low-resolution satellite images, e.g. leaf shape and color patterns [54]. More specifically, certain phenotypes (e.g. biochemical and physiological properties) are characterized by changes in the shape of the leaves, without affecting the spectral content of the crop. Also, analysis of the patterns of leaf color change can be utilized to differentiate diseased plants from nutrient deficient plants. This study explores the combination of field maps for several commonly used SVIs and leaf shape-color information in the design of data analytics algorithms for assessment of the impact of fertilizers on organic crop production. 

In this study, the bi-weekly estimate of standing pools of soil macro-nutrients (e.g. N, P, K, Ca, S and Mg) is taken at permanent plots through the Lachat Colorimetric analyses and ion chromatography once we have the extractions. Additionally, bi-weekly measurements of soil pH, moisture, temperature, NO₃ and LAI, as well as soil N₂O, NO, CH₄ and CO₂ emissions are carried out, employing various tools (e.g. GC, IRGA and GC-MS). The collected data provides an estimate of the nutrients available to plants in soil solution and exchange sites. Monthly estimates of net nutrient mineralization with the buried bag technique are also performed, which provides a net estimate of the mass of nutrients internally generated from the soil substrate after accounting for gaseous loss. 

Coinciding with the traditional soil and plant monitoring efforts described above, we also carried out parallel measurements, employing iMETOS MobiLab analyzers (MobiLab) and Horiba LAQUAtwin pocket meters. The instruments and consumables are low-cost and generally have respectable accuracy and precision. The Mobilab can carry out laboratory precision analysis for NO₃-, K+ and PO4-, whereas the pocket meters are capable of measuring soil pH, moisture, temperature, conductivity, NA+, K+, NO₃- and Ca₂+. The combination of these measurements provides the essential suite of information of site fertility and physical characteristic for farmers to integrate into precision farming systems. These instruments have also been identified as candidates to be further developed, so they can be deployed continuously in the field once the sensor analyzer prototypes are constructed, described in subtask 3.1. 

Data-driven decision making via real-time sensing, modeling and optimization 

This phase aims to provide effective and deployable decision support services by constructing data-driven models for identifying soil health problems and monitoring dynamic changes. Accurate Soil-Crop (SC) health monitoring is vitally important in improving SC sustainability to support farming operations [1]. Soil health diagnosis models focus on representing complicated interaction among soil functions by taking advantage of deep data-driven models learned from the collected data [58]. SC health monitoring provides nearly real-time data (time delay mainly introduced by network transmission) that helps farmers to make effective decisions. 

Soil health is an integrative property dealing with soil chemical, biological and physical properties and represents the soil fertility to support agricultural production and ecosystem services sustainably. It depends on the maintenance of four fundamental functions: Carbon transformations, nutrient cycles, soil structure and the regulation of pests and diseases. However, the highly sophisticated interactions of the functions make it insufficient to diagnose soil health just by considering each function separately. 

The formulated data-driven models in this study assess soil health and crop yields, using multi-source collected data (described in Phase I) and leveraging the strong hierarchical data representation ability of Deep Neural Networks (DNNs). The collected data and observation are classified into different categories according to the corresponding soil-crop function. DNNs provide an automated representation-learning scheme for raw data and enable the models to adapt to the complex relationship. The developed DNN models learn the relationship between the soil/crop observations and health/performance indicators. 

A stochastic gradient descent approach associated with the back propagation is applied for training the predictive-dynamic models to learn the interactions between soil characteristics. Given the initial soil quality and real-time environmental data (e.g. weather and irrigation), the dynamic change of soil quality is predicted by the formulated models herein. 

The encoder-decoder architecture of deep Recurrent Neural Networks (RNNs) is utilized to produce a variable-length output sequence [59]. The encoder and decoder are trained jointly to maximize the average log-likelihood (Equation 1): 

Where M is the number of all pairs of X and Y and P denotes the conditional distribution and the conditional independence assumption holds. The encoder processes an input sequence of soil quality data X=(x_1,x_2,...x_m), e.g. monthly data and emits a summary representation C; the decoder generates a sequence Y=(y_1,y_2,...y_m) to predict next week’s soil quality. 

We will implement the gated recurrent units as the hidden activation function due to its useful properties of avoiding the gradient vanishing problem and keeping long-term information [59]. The back propagation through time with the Adam algorithm through time will be applied to train the proposed deep RNN [60]. 

Crop productivity and soil nutrient analysis 

In concert with Phase 1 and 2, this phase aims to construct a cyber-physical farming platform for real-time monitoring and control of various soil-crop parameters (e.g. NPK values) and identifying the sustainable configurations of crop production to reveal overall crop performance via techno-economic and socio-environmental studies better. A self-contained soil monitoring data acquisition system is proposed for field deployment to obtain real-time and daily measurements of various soil quality characteristics of interest. The sensor analyzer system consists of several fundamental components, i.e. multiple soil quality sensors (e.g. pH and NPK), a sensor aggregation and control board, data logger and a data-driven decision process block as a communication system. 

Figure 1 shows the proposed cyber-physical platform for the communication protocol and the interaction between the components of the system being monitored. The sensor package shares the data with the onboard microcontroller and communicates via the transceiver, using 3G/4G protocols. To improve the sensor channel range, a repeater station could be introduced in the field under observation, which would collect the data from multiple sensor platforms and communicate it to a field station for synchronization. This data is stored in the cloud to facilitate remote access and sharing with stakeholders for making real-time and daily decisions to improve precision agriculture. 

 Figure 1: Communication schematic for data dissemination

Using existing IoT devices requires extensive efforts to correlate data from various IoT technologies and extract knowledge from the obtained data. Evaluation of crop varieties is based on the field-scale trials where specific crops are grown and manually harvested. Nutrient recommendations are based largely on the field-scale correlation and calibration studies of soil tests developed to match the soil conditions of the region (e.g. Bray phosphorus for low pH soils). These studies rely on the chosen soil test values being correlated with the crop response and the calibration data being sufficiently accurate to predict the change in yield that will occur from added fertilizers. 

While these assumptions often hold in practice, issues have also been noted in terms of the ability of current alerts to maintain micronutrient levels (e.g. Nitrogen (N), Phosphorus (P) and Potassium (K) or NPK values) [61,62]. Additionally, determination of N mineralization and availability have proven to be the most difficult tasks facing crop producers and despite the large quantity of recent research, a widely applicable soil test to predict N mineralization has not been established [63-65]. In-season plant tissue testing is also common for specific crops (e.g. potatoes and wheat) where a specific plant part is sampled at a defined growth stage and chemically analyzed for nutrient concentration. These values are then compared to data derived from field-scale studies where threshold values for supplemental fertilizer application were developed to elicit a yield or quality response. 

Advances in sensor technology have resulted in the development of technologies (e.g. NDVI) that have been shown to be well correlated to crop response, particularly in cereals, to applied N [66-68]. However, the usage of these sensors relies on the establishment of in-field reference strips to establish the response to applied N and lower fertilizer doses. 

The proposed cyber-physical platform integrates various technology (e.g. sensors, cameras, drones, satellite and weather stations), soil analysis (e.g. Lachat Colorimetric and ion chromatography), crop productivity analysis (e.g. forecasting and recommendation), IoT devices (e.g. wireless networks, network-connected cloud) and organic and synthetic fertilizer assessments (Figure 2). Additionally, this platform is able to maximize the efficiency and profitability of individual entities in the crop life cycle to maintain competitiveness in the market-place. The SeDS system includes web-/mobile-based apps that share real-time updates of soil-nutrient concentrations, moisture and crop accumulation to farmers, who can then decide to supply fertilizers with the demand to enhance crop performance while improving soil health. For instance, farm irrigation - if they receive a critical low moisture alert from the mobile app and no rain is in the forecast. Thus farmers will take proactive measures before soil moisture may reach the wilting point. Additionally, farmers will receive seasonal reports of nutrient mass balances from their fields to visualize the input, storage and loss of nitrogen and phosphorus along with other relevant nutrients, which will ultimately help them to achieve the 4Rs of nutrient stewardship (i.e. right source, right rate, right time and right place). Ultimately, the form and frequency of these reports are determined iteratively through farmer and researcher interaction, but eventually the responsibility of the annual and bi-weekly actions are tasked to the farmer equipped with nutrient and moisture sensors.
 

Figure 2: Simulated study in Aberdeen, Idaho, USA. 

Validation of theory soil quality prediction associated with SeDS data and observations can provide accurate estimations of soil health and crop yields for both local and global regions. SeDS system explores the correlations between the crop attributes acquired by remote sensing platforms and the crop attributes obtained by field measurements and investigates the relation of the aerial and ground measurements with the type of fertilizers and their impact on the farming productivity. The availability of ground measurements is employed for validation and comparison of the used SVIs from aerial imagery and the implemented data analytics methods for automated image interpretation. Since the discussed DNN models in Phase 2 are supervised; cross-validation is applied to evaluate the performance of the models, including: (1) Divide each dataset into training set, validation set and test set, (2) Train and tune parameters, using the training and validation sets and (3) Evaluate the performance of the models, using unseen data in the test set. The discussed onsite-field empirical plan is adopted for validation of the closed-loop real-time sensing, modeling, control and data-influenced decision making.

Smart farming challenges are associated with data processing from various IoT devices with different interoperability standards and communication platforms for sharing information and interpretation among decision makers. Currently, point solutions for soil/crop analysis exist for capturing one or a few specific factors, but there is no support for robust data analytics, sharing captured knowledge and forecasting the variables (soil/crop quality and yield). The proposed SeDS system can capture key factors (e.g. pH and micronutrients) and share time-series data among growers to assess farm productivity, using the common, available tools. 

The study facilitates exploration of tradeoffs by applying weights optimization within a multi-objective function and running various scenarios to incorporate the uncertainty sources. Several state government agencies and industry partners can use the outcomes of this study and support future ongoing research. The key research outcomes will not only monitor and control the crop performance, but will also improve the yield and quality by (a) Observing the in-depth views of each operation, (b) Recommending optimal parameters, (c) Explaining causes and deviations, (d) Providing real-time alerts of production losses and (e) Permitting decision makers to optimize farm productivity based on often-conflicting sustainability objectives. 

The existing intricacies of agro-ecological systems that play a critical role in feeding the world are not well-understood, thus a fundamental understanding is critical to maximizing effective management of pervasive data and extracted information. The fundamental innovation of this study lies in modernizing the soil/crop analysis through CPS and IoT devices, as well as crossing the boundaries among science and engineering disciplines. 

Moving beyond current techniques to monitor soil and plant health and further development of telecommunication technologies offer the opportunity to advance today’s framing infrastructure through cyber-physical real-time control and optimization platforms capable of (a) Streamlining the information transfer to farmers, (b) Minimizing the cost of monitoring and (c) Supporting precision farming systems. The emergence of innovative miniaturized soil macro-nutrient, pH, temperature and conductivity analyzers and meters allow farmers to carry out laboratory precision measurements on soils in the field, using available tools, e.g. soil sensing pocket meters. 

UAVs equipped with imaging sensors are well-suited to augment the agricultural decision support systems and to meet the objectives of precision agriculture to reduce the use of herbicides, pesticides, fertilizers and other chemical substances and to increase crop yield. Due to the low-cost and on-demand flight capabilities of UAVs and the ability to supply high-/multi-resolution aerial imagery at a multi-spatial scale, the technology offers numerous advantages in comparison to satellite and manned aircraft-carried imaging systems and has subsequently become the primary choice for remote sensing and field monitoring in modern farm management. 

Real-time data and information on the soil/crop health enable farmers to take early corrective actions and provide opportunities to enhance productivity and reduce potential losses. Identifying crop stress in early stages also reduces the use of resources, e.g. it eliminates the need for massive widespread applications of chemicals to treat pests and diseases, which, consequently, reduces environmental impacts.

Advanced sensing and comprehensive assessments can provide accurate soil/crop quality evaluation; however, current approaches are inconvenient and high-price and cannot provide real-time information. The potential paths for future research are as follows:

This study integrates mechanical inventions with cyber-enabled tools to better understand soil fertility and capture interactions between variables at multi-spatiotemporal levels. The proposed integrated platform allows farmers to participate and develop a data-driven decision making process in which the farmer can interact and make placed-based decisions throughout the growing season. Understanding the ramifications of agro-ecological systems, complex compounds, mechanisms, multifunctional performance and commercial viability, as well as elucidation of the effects of various parameters (e.g. micronutrients, data processing and communication) is the potential impacts of this study; which will provide a base of knowledge to promote farming and enhance sustainability benefits. Further, drones (as a non-invasive sensing technology) reduce ground traffic on the field, help in minimizing soil compacting and erosion during crop scouting and eliminate the need for ground vehicles (e.g. agricultural robotic platforms) for crop management by early monitoring.

Special thanks should be given to Dr’s. Aleksander Vakanski, Min Xian and Anish Sebastian for their valuable inputs.