Edited by: Monique Petitdidier, Institut Pierre-Simon Laplace, France
Reviewed by: Karine Zeitouni, University of Versailles Saint-Quentin, France; Télesphore Yao Brou, University of La Réunion, France; Fabrizio Niro, European Space Research Institute, Italy
*Correspondence: Mykola Lavreniuk
This article was submitted to Environmental Informatics, a section of the journal Frontiers in Earth Science
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Many applied problems arising in agricultural monitoring and food security require reliable crop maps at national or global scale. Large scale crop mapping requires processing and management of large amount of heterogeneous satellite imagery acquired by various sensors that consequently leads to a “Big Data” problem. The main objective of this study is to explore efficiency of using the Google Earth Engine (GEE) platform when classifying multi-temporal satellite imagery with potential to apply the platform for a larger scale (e.g., country level) and multiple sensors (e.g., Landsat-8 and Sentinel-2). In particular, multiple state-of-the-art classifiers available in the GEE platform are compared to produce a high resolution (30 m) crop classification map for a large territory (~28,100 km2 and 1.0 M ha of cropland). Though this study does not involve large volumes of data, it does address efficiency of the GEE platform to effectively execute complex workflows of satellite data processing required with large scale applications such as crop mapping. The study discusses strengths and weaknesses of classifiers, assesses accuracies that can be achieved with different classifiers for the Ukrainian landscape, and compares them to the benchmark classifier using a neural network approach that was developed in our previous studies. The study is carried out for the Joint Experiment of Crop Assessment and Monitoring (JECAM) test site in Ukraine covering the Kyiv region (North of Ukraine) in 2013. We found that GEE provides very good performance in terms of enabling access to the remote sensing products through the cloud platform and providing pre-processing; however, in terms of classification accuracy, the neural network based approach outperformed support vector machine (SVM), decision tree and random forest classifiers available in GEE.
Information on land cover/land use (LCLU) geographical distribution over large areas is extremely important for many environmental and monitoring tasks, including climate change, ecosystem dynamics analysis, food security, and others. Reliable crop maps can be used for more accurate agriculture statistics estimation (Gallego et al.,
The Google Earth Engine (GEE) provides a cloud platform to access and seamlessly process large amount of freely available satellite imagery, including those acquired by the Landsat-8 remote sensing satellite. The GEE also provides a set of the state-of-the-art classifiers for pixel-based classification that can be used for crop mapping. Though these methods are well-documented in the literature, there were no previous studies to compare all these methods for classification of multi-temporal satellite imagery for large scale crop mapping. Since the GEE platform does not include any neural network based models, we add a neural network based classifier (Skakun et al.,
The proposed study is carried out for the Joint Experiment for Crop Assessment and Monitoring (JECAM) test site in Ukraine. Agriculture is a major part of Ukrainian's economy accounting for 12% of the Ukrainian Gross Domestic Product (GDP). Globally, Ukraine was the largest sunflower producer (11.6 MT) and exporter, and the ninth largest wheat producer (22.2 MT) in the world in 2013, according to the U.S. Department of Agriculture (USDA) Foreign Agricultural Service (FAS) statistics.
The JECAM test site in Ukraine was established in 2011 as part of the collaborative activities within the GEOGLAM Community of Practice. The site covers the administrative region of Kyiv region with the geographic area of 28,100 km2 and almost 1.0 M ha of cropland (Figure
1 | Winter wheat | End of September | Mid of July |
2 | Winter rapeseed | End of August | Mid of July |
3 | Maize | End of April | Mid of August—begin of September |
4 | Soybean | End of April—Begin of May | Mid of September—begin of October |
5 | Spring crops | Begin of April | Mid of July—August |
6 | Sugar beet | End of April—Mid of May | End of September—Mid of November |
7 | Sunflower | End of April—Begin of May | Begin of September |
Ground surveys to collect data on crop types and other land cover classes were conducted in 2013 in Kyiv region (Figure
1 | Axx | Artificial | 6 | 1.6 | 23.0 | 0.1 |
2 | B11 | Winter wheat | 51 | 13.2 | 3960.8 | 17.4 |
3 | B32 | Winter rapeseed | 12 | 3.1 | 937.3 | 4.1 |
4 | B12, B14 | Spring crops | 9 | 2.3 | 455.9 | 2.0 |
5 | B16 | Maize | 87 | 22.5 | 7253.3 | 31.9 |
6 | B22 | Sugar beet | 8 | 2.1 | 632.5 | 2.8 |
7 | B31 | Sunflower | 30 | 7.8 | 2549.0 | 11.2 |
8 | B33 | Soybeans | 60 | 15.5 | 3252.3 | 14.3 |
9 | B19, B39, B40 | Other cereals | 32 | 8.3 | 1364.0 | 6.0 |
10 | C10, B60 | Forest | 17 | 4.4 | 1014.3 | 4.5 |
11 | E01, E02 | Grassland | 48 | 12.4 | 747.5 | 3.3 |
12 | F00 | Bare land | 10 | 2.6 | 67.2 | 0.3 |
13 | G01, G02 | Water | 16 | 4.1 | 448.3 | 2.0 |
Total | 386 | 100 | 22705.3 | 100 |
Remote sensing images acquired by the Operational Land Imager (OLI) sensor aboard Landsat-8 satellite were used for crop mapping over the study region. Landsat-8/OLI acquires images in eight spectral bands (bands 1–7, 9) at 30 m spatial resolution and in panchromatic band 8 at 15 m resolution (Roy et al.,
April 16 | 181/24 | 4.77 | June 19 | 181/24 | 62.58 |
181/25 | 0.60 | 181/25 | 33.37 | ||
181/26 | 0.02 | 181/26 | 26.43 | ||
May 02 | 181/24 | 0.01 | July 05 | 181/24 | 35.30 |
181/25 | 0.77 | 181/25 | 21.06 | ||
181/26 | 4.38 | 181/26 | 15.69 | ||
May 18 | 181/24 | 9.06 | August 06 | 181/24 | 24.35 |
181/25 | 14.93 | 181/25 | 12.69 | ||
181/26 | 14.32 | 181/26 | 40.86 |
The main issues that need to be addressed while dealing with satellite imagery for large areas (such as Kyiv region) are a non-regular coverage over the region and the presence of missing data due to the clouds and shadows. Therefore, before feeding satellite data to the classifiers, a pre-processing step should be performed to fill in missing values due to clouds and shadows. At present, there is no a standard approach for dealing with these issues. Compositing is a very popular approach, however missing values can still happen in composite products (Yan and Roy,
Different composites derived from Landsat-8 imagery and available in GEE were analyzed in the study. Landsat 8 8-Day Top-of-atmosphere (TOA) Reflectance Composites were used from GEE (Figure
The following pre-processing steps were applied for all Landsat-8 images:
Conversion of digital numbers (DNs) values to the TOA reflectance values using conversion coefficients in the metadata file (Roy et al.,
To decrease an impact of atmosphere to the image quality conversion from the TOA reflectance to the surface reflectance (SR) has been done using the Simplified Model for Atmospheric Correction (SMAC; Rahman and Dedieu,
Detection of clouds and shadows were done using Fmask algorithm (Zhu and Woodcock,
Landsat-8 satellite images were reconstructed from missing pixel values (clouds and shadows) using self-organizing Kohonen maps (SOMs; Skakun and Basarab,
These pre-processing steps were performed outside GEE platform. After these products were generated they were uploaded in the GEE cloud platform for the further classification using classification algorithms available in GEE.
In the study, classification of multi-temporal satellite imagery was performed at per-pixel basis. Multiple classification techniques were evaluated in the study. At first, all classification algorithms available in GEE were analyzed and used for classifying multi-temporal 8-days Landsat-8 TOA composites from GEE. Then, the best classification algorithms in terms of overall classification accuracy were compared with the neural network classifier that used multi-temporal SR values generated outside GEE. The presented approaches were compared in terms of classification accuracy at pixel level. GEE offers several classification algorithms among which are decision trees, random forests, support vector machine (SVM), and Naïve Bayes classifier.
SVM became popular in a recent decade for solving problems in classification, regression, and novelty detection. An important property of SVMs is that the determination of the model parameters corresponds to a convex optimization problem, and so any local solution is also a global optimum (Bishop,
A decision tree (DT) classifier is built from a set of training data using the concept of information entropy. At each node of the tree, one attribute of the data that most effectively splits its set of samples into subsets enriched in one class or the other is selected. Its criterion is the normalized information gain that results from choosing an attribute for splitting the data. The attribute with the highest normalized information gain is chosen to make the decision. The algorithm then recurs on the smaller sublists. One disadvantage of the DT classifier is the considerable sensitivity to the training dataset, so that a small change to the training data can result in a very different set of subsets (Bishop,
Since the major problem of the DT classifier is overfitting, RF overcomes it by constructing an ensemble of DTs (Breiman,
Other classifiers included in the GEE platform that are less popular in the remote sensing community:
Multinomial logistic regression is generalization of linear regression using the softmax transformation function and the main task is to minimize an error function by taking the negative logarithm of the likelihood, which means cross-entropy. The main difference from other models and algorithms is the outcome score that could be considered as a probability value (Bishop,
This approach applied implementation of the linear perceptron to multiclass problems. The main idea of the perceptron is that the summing node of the neural model computes a linear combination of the inputs applied to its synapses, as well as incorporates an externally applied bias. The resulting sum, that is, the induced local field, is applied to a hard limiter. Accordingly, the neuron produces an output equal to 1 if the hard limiter input is positive, and -1 if it is negative (Haykin,
Bayes classifier is a simple probabilistic approach which is based on the Bayes theorem and assumption of independence between input features. Within the learning procedure, it minimizes the average risk of classification error. Main advantage of this classifier is that it requires a small number of training data to compute the decision surface. At the same time, its derivation is contingent on the assumption that the underlying distributions be Gaussian, which may limit its area of application (Haykin,
It is the specialized version of the SVM which represents a histogram intersection kernel SVMs (IKSVMs). The runtime complexity of the classifier is logarithmic in the number of support vectors as opposed to linear for the standard approach. It allows to put IKSVM classifiers in the same order of computational cost for evaluation as linear SVMs (Maji et al.,
The algorithm can be expressed as a linear-threshold algorithm that is similar to MultiClassPerceptron. However, while the perceptron uses an additive weight-update scheme, the Winnow classifier uses a multiplicative scheme. A primary advantage of this algorithm is that the number of mistakes that it makes is relatively little affected by the presence of large numbers of irrelevant attributes in the examples. The number of mistakes grows only logarithmically with the number of irrelevant attributes. Classifier is computationally efficient (both in time and space; Littlestone,
A variant of SVM with simple and effective sub-GrAdient SOlver algorithm for approximately minimizing the objective function that has a fast rate of convergence results. At each iteration, a single training example is chosen at random and used to estimate a sub-gradient of the objective, and a step with pre-determined step-size is taken in the opposite direction. Solution is found in probability solely due to the randomization steps employed by the algorithm and not due to the data set. Therefore, the runtime does not depend on the number of training examples and thus Pegasos is especially suited for large datasets (Shalev-Shwartz et al.,
It should be noted that neural network (NN) classifiers are not available in GEE. Our proposed neural network approach based on committee of NNs, in particular Multi-Layer Perceptron (MLPs), is utilized to improve performance of individual classifiers. The MLP classifier has a hyperbolic tangent activation function for neurons in the hidden layer and logistic activation function in the output layer. Within training cross-entropy (CE) error function is minimized (Bishop,
A committee of MLPs was used to increase performance of individual classifiers. The committee was formed using MLPs with different parameters trained on the same training data. Outputs from different MLPs were integrated using the technique of average committee. Under this technique the average class probability over classifiers is calculated, and the class with the highest average posterior probability for the given input sample is selected.
Ensemble based neural network model for crop classification was recently validated in the JECAM experiment within study areas in five different countries and agriculture conditions, and provided the best result compare to SVM, maximum likelihood, decision tree and logit regression (Waldner et al.,
Therefore, this approach is used as a benchmark for assessing classification techniques available in GEE.
The first set of experiments was carried out to select the best input (TOA 8-day composites or restored values) and evaluating different classifiers available in GEE. Table
CART | 75 |
GMO Max Entropy | 72 |
Random Forest | 68 |
MultiClassPerceptron | 60 |
IKPamir | 57 |
Winnow | 49 |
FastNaiveBayes | 32 |
Pegasos | – |
VotingSvm | – |
MarginSvm | – |
One of the best GEE classifiers (CART and RF) on atmospherically corrected Landsat-8 imagery were compared to the committee of NN that was implemented outside GEE, in the Matlab environment using a Netlab toolbox (
1 | Artificial | 87.1 | 33.2 | 80.8 | 54.9 | 64.8 | 94.9 |
2 | Winter wheat | 87.8 | 91.3 | 82.7 | 90.0 | 91.3 | 95.0 |
3 | Winter rapeseed | 88.1 | 92.8 | 67.2 | 86.5 | 97.7 | 92.9 |
4 | Spring crops | 10.4 | 6.5 | 13.2 | 14.2 | 39.2 | 45.2 |
5 | Maize | 75.3 | 92.7 | 73.3 | 83.9 | 85.9 | 89.7 |
6 | Sugar beet | 56.1 | 48.4 | 44.8 | 27.3 | 91.1 | 95.3 |
7 | Sunflower | 75.0 | 75.9 | 67.0 | 61.7 | 87.5 | 83.9 |
8 | Soybeans | 74.0 | 50.8 | 56.8 | 51.3 | 77.0 | 68.0 |
9 | Other cereals | 74.1 | 53.6 | 64.8 | 34.9 | 82.4 | 69.5 |
10 | Forest | 93.8 | 89.8 | 89.4 | 94.8 | 82.7 | 97.3 |
11 | Grassland | 60.1 | 77.2 | 52.7 | 61.6 | 72.8 | 93.1 |
12 | Bare land | 90.1 | 83.9 | 84.6 | 80.2 | 98.5 | 85.2 |
13 | Water | 98.5 | 99.6 | 99.3 | 99.8 | 98.1 | 99.8 |
The GEE platform offers powerful capabilities in handling large volumes of remote sensing imagery that can be used, for example, for classification purposes such as crop mapping for large territories. In order to deal with irregular observation and missing values due to clouds and shadows, a compositing approach was applied. Several compositing products were available. Also, using the rich JavaScript and/or Python APIs, it is possible to filling missing values using MODIS images using, for example, approaches developed by Roy et al. (
The GEE platforms provides a set of classification algorithms. The best results in the GEE were obtained for the DT-based classifiers, namely CART and RF. The best accuracy achieved on atmospherically corrected and restored Landsat-8 images was 76.9 and 69.9% for CART and RF, respectively. The classifier GMO Max Entropy shows better result than RF but there is no Python implementation (only Javascript). It is known that CART and RF tend to overfit and require fine-tuning. These accuracies were significantly lower than for the ensemble of MLPs that obtained 84.7% overall accuracy (Figure
Accuracy of 85% is usually considered as target accuracy for agriculture applications (McNairn et al.,
The crops that did not pass the 85% threshold for PA and UA:
The activities within this paper were targeted on the comparison of pixel-based approaches to crop mapping in Ukraine, and exploring efficiency of the GEE cloud platform for large scale crop mapping with target to apply the platform for large areas involving multiple sensors and large volumes of data (at the order terabytes). Crop classification was performed by using multi-temporal Landsat-8 imagery over the JECAM tests site in Ukraine. Several inputs (products and composites) were evaluated by means of different classifiers available at GEE and our own approach. The classifiers included state-of-the-art techniques such as SVM, decision tree, random forest, and neural networks. Unlike other applications such as general land cover mapping or forest mapping, crop discrimination requires acquisition of multi-temporal profiles of crop growth dynamics. Therefore, images at multiple dates (or composites at several time intervals) need to be acquired or generated to discriminate particular crops. At the same, it causes the problem of missing data due to clouds and shadows when dealing with large territory, for example Kyiv oblast in the study. Therefore, it is very difficult to find an optimal solution in terms of temporal resolution and cloud-free composites, and missing data still occur. One way to handle this problem was to restore the missing values using self-organizing Kohonen maps. Another approach would be to use ancillary data such as MODIS to predict and fill in missing values in the Landsat time-series. But these are not trivial procedures, are not available at hand in GEE and require a lot of efforts from the end-user to be implemented. Therefore, 8-day TOA reflectance composites were used in GEE.
Better performance in terms of overall accuracy was shown for atmospherically corrected and restored Landsat images over the 8-day TOA reflectance composites from GEE. One of the explanations would that quality of restored data was better than TOA composites from GEE.
As to classification algorithms, ensemble of neural networks outperformed SVM, decision tree, and random forest classifiers. At some extent, it contradicts with recent studies where SMV and RF show better performance than neural networks. In our opinion, these are due: (i) in many cases, out-of-date neural networks techniques are used and not full potential of neural networks is explored in the remote sensing community; (ii) for SVM, DT and RF built in GEE parameters were used.
For the following agriculture classes an 85% threshold of producer's and user's accuracies was achieved: winter wheat, winter rapeseed, maize, and sugar beet. Such crops as sunflower, soybeans, and spring crops showed worse performance (below 85%).
In general, GEE provided very good performance in enabling access to remote sensing products through the cloud platform, and allowing seamless execution of complex workflow for satellite data processing. The user at once gets access to many satellite scenes or composites that are ready for processing. Although our particular study did not directly deal with large volumes of data, we still showed effectiveness of the GEE cloud platform to build large scale applications when accessing and processing satellite data with no reference to the volume. Indeed, the approaches addressed in the study can be applied at country level thanks to the capabilities offered in GEE. Another point addressed in the study was validity—in other words, what kind of accuracy can be expected for crop mapping with GEE when utilizing existing products and processing chains.
However, in our opinion, several improvements should be made to enable large scale crop mapping through GEE, especially within operational context:
Provide at hand tools to deal with missing data due to clouds and shadows.
To add neural networks classifiers (e.g., Tensorflow deep learning library) and allow optimization of existing classifiers especially SVM.
Future works should include:
Large scale crop mapping (potentially at national scale) using optical and SAR imagery taking into account the coverage of Landsat-8, Proba-V, Sentinel-1, and Sentinel-2 imagery for the whole country (such as Ukraine).
In the future we will implement parcel-based classification approach using GEE connected pixel method and some others, to improve a pixel-based crop classification map.
AS was the project manager. ML provided crop classification maps using ensemble of neural networks and conducted experiments in Google Earth Engine. NK and AN were scientific coordinators of the research and provided results interpretation. SS has prepared training and test sets and preprocessed Landsat-8 data for crop mapping.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
This research was conducted in the framework of the “Large scale crop mapping in Ukraine using SAR and optical data fusion” Google Earth Engine Research Award funded by the Google Inc.