Main authors:	Pim van den Berg, Cleveland Rex Steward, Marti Vidal Morant, Emily Ongus, Thijs van der Zaan
iSQAPERiS editor:	Jane Brandt
Source document:	van den Berg, P., Steward, S. R., Vidal Morant, M., Ongus, E., van der Zaan, T. 2018. Evaluation of the performance of the Soil Quality App (SQAPP) for the greater Albaida region. Report submitted to Wageningen University and Research as part of the authors' MSc programme. 107 pp

 

In 2018, as part of their MSc programme, students of Wageningen University and Research undertook a field evaluation of the performance of SQAPP (beta version) in terms of its accuracy, relevance and functionality. Their feedback was used in the continued development of SQAPP (»Developing SQAPP).

---

Contents table

1. Introduction

2. Core concepts and approach

3. Materials and methods

4. Results and data analysis

5. Discussion

6. Recommendations for improvement

7. Conclusions and outlook

8. References

1. Introduction

1.1 Background

Unsustainable use of agricultural areas causes soil degradation (Oldeman et al., 1991). Degraded soils are more prone to soil threats which can result in a vicious circle of soil degradation. For the European continent there are several soil threats identified; the main ones include soil organic matter decline, soil compaction, soil erosion, soil salinization, pollution, sealing and landslides (Van Beek et al., 2010). These threats have been an increasing concern in the European Union (EU) (Glæsner et al., 2014; Montanarella, 2007) with several soil protection strategies as a response, e.g. EU Common Agricultural Policy (Bowyer & Keenleyside, 2017). However, the impact of such strategies remains debatable (Glæsner et al., 2014). Soil degradation together with climate change (Olesen & Bindi, 2002), a growing demand for food due to a growing population (Godfray et al., 2010), and an increase in meat consumption due to a changing diet (Popp et al., 2010) puts heightened pressure on agricultural productivity. This pressure on agricultural land and the ineffectiveness of current European policies calls for new or revised strategies related to land management to reach a more sustainable use of soils.

To reach a more sustainable use of agricultural soils in Europe, the European Union, the Chinese government and the Swiss government, funded the iSQAPER project as part of the European Union Horizon 2020 programme for research and innovation. iSQAPER has been initiated by several European and Chinese organisations. The project is being coordinated by the Soil Physics and Land Management (SLM) Group of Wageningen University. The project’s core mission is the protection and promotion of soil quality to ensure sustainable agricultural productivity. In support of this ideology, the main objective of iSQAPER is to provide an “interactive soil quality assessment in Europe and China for agricultural productivity and environmental resilience providing decision makers with science-based, easy to apply and cost-effective tools to manage soil quality and function” (iSQAPER information system, n.d.).

ISQAPER wants to achieve this objective by integrating existing soil quality data into an open access information system in the form of a mobile app, the Soil Quality Assessment Application (SQAPP). Currently SQAPP is still in its beta. SQAPP provides users with soil quality information anywhere in the world using data aggregated from several global datasets, primarily from ISRIC’s Soilgrids (ISRIC, n.d.). The provided data consists of a range of physical and chemical soil quality indicators and associated soil threats. Additionally, the app provides management advice based on the aforementioned data and field characteristics. The expectation is that farmers can act on this knowledge to manage their soils more sustainably. Furthermore, it could inform policy makers about regionally specific soil threats and protection measures and therefore aid in the formation of soil protection legislation. This will lead to an overall higher quality of soils that are less susceptible to soil threats and ensure longevity of the agricultural systems that depend on them.  

1.2 Study objectives

This study was commissioned by Coen Ritsema, chair of the SLM department and head of the Wageningen team responsible for SQAPP development and project management of the overall iSQAPER project. The commissioner stressed the need for a multi-actor approach in the development of SQAPP whereby farmers, scientists, practitioners, agricultural service providers and policy makers play a role in testing, evaluating, and improving the app. It was made clear that SQAPP has not been thoroughly tested in its current level of development, especially by the primary end-users envisioned: farmers themselves.

In meetings with the commissioner, a need for further verification of the global data used by the app was expressed, as was the necessity for direct farmer feedback on the usefulness and impact of the app and the information it contains. The opportunity was thus presented to explore what other information may be incorporated to better match the needs and interests of such end-users. Interest was also shown into seeing how different farmers from varying backgrounds and sectors of agriculture interpret the potential of such tool.

Based on the above points of discussion, the main issue that was identified was the lack of information on the performance and impressions of the app. The objective of our study was thus identified: to gain information on the accuracy of soil quality data, relevance of the information provided to the farmers, and overall practical functionality of the app.

1.3 Study area

The study area is located in the region around Albaida, consisting of the South-West of the Valencia province and the Alicante province, as shown in Figure 1. This region lies in the Mediterranean climate zone. The climate zone combined with the soil types forms pedoclimatic zones (iSQAPER information system, n.d. & Toth et al., 2017). Most of the times during the summer, between June and September, there is a dry period that usually lasts 3 – 5 months. Contrary to the summer, the winter is characterized by intense rainfall events and contributes to a yearly precipitation of 300 – 500 mm (García-Orenes et al., 2009).

Figure 1

The most common crops in the study area are citrus plantations, rain fed olive, vineyards, fruit and cereals (ISQAPER, n.d., a). The traditional agricultural practices entail intensive tilling and the use of inorganic fertilizers. Because of water scarcity, wastewater and salty water is often used for irrigation, locally leading to the loss of soil structure and aquifer contamination (Pedro-Monzonís et al., 2015). Recently, however, organic farming systems have been introduced, which use leguminous species that provide soil cover and animal manure to improve the quality of the soil. These soils are mainly developed on calcareous materials and on quaternary sediments (ISQAPER, n.d., a). Unsuitable land management practices and the highly irregular and occasional intensive rainfall can lead to increased rates of soil erosion and land degradation processes, leading to reduced soil fertility. Furthermore, soils in the project area generally have poor soil structure and low organic content (ISQAPER, n.d., a).

1.4 Soil Quality APP (SQAPP)

There is a wealth of global soil information available and even a number of apps aimed at making this data more accessible to landowners and the general public. One of the most comprehensive is the “SoilInfo” app by ISRIC which makes use of many of the same databases to provide soil quality indicators and inform users of the properties of their soil (Hengl & Mendes de Jesus, 2015). SQAPP aims to expand upon the offerings of such tools by better tailoring information to specific decision-makers. It seeks to accomplish this by benchmarking soil quality in relation to farming systems and pedo-climatic conditions, linking soil quality to soil threats, and providing management advice on how to improve soil quality (iSQAPER information system, n.d.).

SQAPP pulls data from global datasets, with a resolution of 250m, based on the input location of the user. This data consists of information on the chemical, physical and biological soil properties. The soil threats are defined through existing soil threat datasets that have been altered based on the local conditions, e.g. slope, rainfall and land use. The app then provides recommendations, or agricultural management practices (AMPs) to improve the properties and threats that score the worst, while taking the local characteristics into account.

1.5 Research questions

In response to the aforementioned issues, the authors address the following research question and sub-questions:

How can the Soil Quality App (SQAPP) be improved based on its performance in the Albaida region, SE Spain?

Sub-questions:

• How accurate is the data provided by the app compared to data acquired through field measurement and lab analysis, for the greater Albaida region?
• How relevant is the information provided by the app to end-users in the region?
• How functional is the app as a tool for conveying information to local farmers?  

2. Core concepts and approach

2.1 Soil quality

Soil quality has been a debated concept since its first utilisation in 1971 (Mausel, 1971). A recently published review (Bünemann et al., 2018) depicted the main arguments and struggles to consensuate a definition. A generally used definition of soil quality refers to it as “the capacity of a soil to function within ecosystem and land-use boundaries to sustain biological productivity, maintain environmental quality, and promote plant and animal health” (Doran and Parkin, 1994).

Soil quality is often assessed by the evaluation of a set of indicators. Since a single indicator cannot predict soil quality, a combination of those are usually selected aiming to describe different soil properties. Generally, the inclusion of a minimum set of indicators referring to soil physical, chemical and biological properties is considered essential. However, one must take into account the existing trade-off between complexity and accuracy: including too many indicators can restrict the aggregation of an overall soil quality index (Bünemann et al., 2018). A solid body of indicators describing the three groups of soil properties should be the aim to describe soil quality.

The materialisation of the soil quality concept into indexed values is a complex subject. Here, the benchmarking of what is an adequate or acceptable score for an indicator or index becomes more subjective. Finding a reference soil with optimum indicator values is not always possible, since soil properties are determined by the soil type and local climatic conditions. In their review, Bünemann et al. (2018) state that very often native undisturbed soils, selected as reference, may not always score better for certain indicators on agricultural productivity or environmental performance. Additionally, what may seem as optimal according to expert opinion, may not be for farmers or other land users (Andrews et al. 2003). Experts, decision makers, and land users may not have the same requirements for soil quality indices, and divergent understandings of its meaning and relevance may further entangle the standardisation of a common assessment procedure.

The app summarizes output into biological, chemical and physical properties as indicators of soil quality. It then translates these information into soil parameters and threat levels needing attention and provides recommendations. This was highlighted in our analyses of relevance , this is further elaborated in our results and discussion.

2.2 Accuracy, relevance and functionality

Our analysis of app performance is focused on the accuracy and relevance of the information provided, as well as its functionality as a tool in the hands of end-users.

Accuracy Accuracy as a concept can be defined as the difference between the estimated value and the true value. In order to determine accuracy, quantitative measurements of data are necessary. This is expressed by knowing or estimating the actual (true) value , (Walther and Moore, 2005). As such, the accuracy in this case can refer to the ability of the app to report information to the best estimation of the true value. This was done by checking on the accuracy of the app output using our VSA.

Relevance The relevance of the information provided by the app can be analysed using the concept of actionable knowledge as stated by Cash et al. (2003): “Science and technology must play a role in sustainable development whilst effectively managing the boundaries between knowledge and action in ways that simultaneously enhance salience, credibility and legitimacy of the information produced”. Actionable Knowledge as a concept defines the boundaries of stakeholder participation in the decision making process and fostering solutions together (Geertsema et al., 2018). The concept of actionable knowledge will be channeled into our study’s investigation of relevance. Relevance, in the context of this study, is used to describe the meaningfulness of the information provided by the app and subsequent action through adjustments in management practices by the end-users. This was tested via communication of the recommendations to the farmers during farmer interviews. We also assessed the recommendations provided based on various levels of suitability to the specific farmers’ context. This is further highlighted in the results and discussion sections.

Functionality Functionality, in the present assessment, can be defined as the effectiveness of the app as a medium, the accessibility of the language used, and the usability of the tool. Within the concept of usability, Iwarsson and Ståhl (2003) suggest four components that need to be satisfied: a personal component related to human functioning, an environmental component related to barriers within the environment that may inhibit action, an activity component related to the activities that need to be performed, and, finally, an analysis of the three aforementioned parts ensuring individual and group preferences are met within the targeted environment. That means, that the functionality of the app is not only limited by the design of the interface, but it also encompasses the socio-environmental context in which the user is placed, as well as the individual attitude towards the technology.

2.3 Approach

In seeking to understand the accuracy and relevance of the app, our study assesses how soil properties, threats, and suitable management practices are uniquely interpreted and reported by the application, by field measurements and observations, and by farmers and landowners themselves. The diagram below illustrates the categorical information sought from these three distinct sources, forming the core of our study:

Figure 2

The differences in how this information is interpreted at each level (SQAPP, scientific analysis, and farmer perception) is central to understanding the accuracy and relevance of the app - concepts which are defined above. As the app was designed to close the gap between scientific data and end users, the practical functionality of the app as a tool toward conveying this information and achieving this goal will also be assessed.

3. Materials and Methods

To assess the accuracy, relevance and functionality of SQAPP local information had to be compared to the app’s predictions. Since the accessibility of existing field data is limited, field visits were conducted in the Albaida region. During the field visits the app was run for that location, soil measurements were taken, and the landowners were interviewed.

3.1 Accuracy of reported soil data and threats

The first objective of the study was to assess the accuracy of soil quality parameter and soil threat information provided by the app. The information targeted by this portion of the study is illustrated in Figure 3 below:

Figure 3

Visual Soil Assessment (VSA) Since we did not have a laboratory at our disposal we performed in-field soil measurements using a Visual Soil Assessment tool (VSA) as described by ISQAPER (Alaoui & Schwilch, 2016) to assess soil quality. The app provides a sizeable list of soil properties and threats. With our VSA we were able to assess a handful of these through direct measurements and some indirectly via empirical relationships. The properties and threats we were able to assess in this fashion were bulk density, texture, percentage coarse fragments, pH, electrical conductivity, water erosion and wind erosion. All the properties and threats assessed with the VSA are listed below. . Critically, we first validated our VSA by comparing it to lab data provided by Universidad Miguel Hernández or obtained from farmer cooperatives. This VSA validation is not a result of our study, but is a necessary step to legitimise using our field assessment to judge the app’s accuracy.

• List of indicators assessed in VSA: Bulk density - Silt content - Sand content Clay content - Course fragments - pH Electrical conductivity - Compaction - Water erosion Wind erosion - Acidification - Salinization

Lab data Due to limitations in scope and extent of our VSA data, we sought additional soil data in the form of laboratory soil analyses. Data for 6 farm plots was acquired from Universidad Miguel Hernández (UMH). Their dataset included information on nitrogen, phosphorus, organic matter, pH and electrical conductivity.

Laboratory soil data was also obtained by contacting individual farmers and farmer cooperatives. A consultant in Villanueva de Castellón, for example, was willing to share lab data for 2 plots. The scopes of these lab studies varied and only partially aligned with parameters highlighted in the app. As with our VSA data, our assessment of accuracy was therefore limited to those parameters shared by both the lab reports and the app. The locations of the acquired lab data and VSA measurements are mapped in Figure 4.

Figure 4

• List of VSA properties that we were able to validate using lab data: Silt content - Sand content - Clay content Electrical conductivity - pH

App runs: properties & threats The above collected data were compared to the soil properties and threats provided by the app to determine the accuracy of the app data. Lab data were prioritised over VSA data in this process when multiple values were available.

To assess the accuracy of the app data, we consider the average difference between the app’s prediction and measured data or lab data. The standard deviation of differences between the app’s prediction and measured data has additionally been determined. For most indicators, we simply calculate the absolute difference between lab/VSA and the app’s predictions. However, since the absolute differences in nutrient availability of the soil are generally very small, percent (relative) difference gives a better impression of the magnitude.

Average Difference = (L-A)/L ×100%,

Where L is lab/vsa data and A represents the app’s predicted value.

Soil threats, on the other hand, are classified based on the level of the threat, which, for most threats, relates to certain soil quality indicators. The app has a classification system to determine whether a threat is low, moderate or high. To assess the accuracy of this classification, we subject our VSA/lab data to the app’s classification system, and check whether the threat level is the same as predicted by the app. The overall score of all samples show us how accurate the app predicts soil threats. To identify the threat level of erosion by wind and water, we use alternative methods, proposed by Stocking (2013) and University of Hertforshire (2011) respectively. Due to time and material limitations, we were unable to assess app performance when predicting organic matter decline, nutrient depletion, contamination, and biodiversity decline.

3.2 Relevance of app content

Relevance of app content to farmers in the region was explored using two distinct strategies: First, per app run, an assessment of the recommended agricultural management practices (AMPs), were reviewed for suitability and feasibility using an elimination method. The second means of assessing relevance was carried out by conducting semi-structured interviews and characterizing and comparing farmer interpretation of soil quality, soil threats, and management as well as seeking direct feedback on app output for their land.

Contextual assessment of summary and management advice Management recommendations provided by the app were critically assessed for all sites visited by the project team. We assessed the different recommendations using an elimination method into 3 different levels to find the most relevant AMPs for our study area. First, the recommended practices were assessed based solely on field characteristics without regard to farm-specific context. In doing so, the applicability matrix used by the app in assigning recommendations is tested. Second, farm characteristics were taken into account and physical suitability of the recommendations are assessed based on specific farm context. For example, the current irrigation and conservation practices are taken into account. Last,the recommended practices were assessed on their ability to improve the soil properties and remediate the soil threats given in the summary of the app, this was done alongside the cropping system of the specific sites. This assessment was based on our expert view and literature. The knowledge targeted via this methodology is represented by the following portions of our diagrammatic framework (Figure 5)

Figure 5
Figure 6

Semi-structured interviews The second part of the assessment of the relevance of app content was carried out by holding 12 semi-structured interviews. The purpose of the interview is to gather information on the opinion of the farmers on soil properties and threats of their field. Furthermore, information about the management practices is gathered. This information was first recorded and then the results of the app were discussed. The opinion of the farmers on the given values for the soil properties and soil threats were recorded. After that the app’s proposed recommendations and the general idea of a soil quality app were discussed. This information is represented in Figure 6.

A list of the soil quality indicators used by farmers used to describe their soil quality were produced. The indicators were proposed by the farmer without guided questions. Some of these indicators may be in line with the ones provided by the app, while others may not. Similarly, soil threats perceived by the farmer were outlined. In order to obtain the information on soil threats, guided questions on the threats considered by the application were formulated. All responses were recorded even if the definition of soil threats by farmers did not match our own. The recorded farming practices were used to provide indispensable contextual background and enabled comparison with app recommendations and insights into adoption dynamics.

The outputs of the app were presented to the farmer, and the consensus between the farmer opinion on soil properties, threats and recommendations were analysed. First, the level of agreement on each category of soil properties (physical, chemical and biological) for all the farmers were combined. Second, we presented an analysis of the mismatch between what the app perceives as a soil threat and the farmer perception. Any difference between farmer perceived threats and the app were qualified as mismatch if, either the farmer perceives it but not the app (and vice versa), or if the app does not provide data for a perceived threat by the farmer. It has not been considered a mismatch when the app perceives a threat but the farmer is not able to agree or disagree on that output. Very often, the app gives a value for a given threats, such as erosion in tons per hectare. However, farmers are unable to produce values to compare with. In that case, the app qualitative information on the threat (low, medium or high risk, for instance) were used to compare. Finally, each of the recommendations given by the app were qualified on the likelihood to be considered as a future AMP by the farmer or not. Additionally, we recorded if that particular recommendation is already in practice or not.

3.3 App functionality

Expert opinion Throughout the study the research team has interfaced with SQAPP extensively. This provided ample opportunity to reflect on the app based on our own direct experiences. These experiences, both positive and negative, were distilled into recommendations for improvements.

Questionnaire The iSQAPER project targets farmers as key end-users of the app. It is important that the app is a functional and understandable tool for farmers. To assess the functionality of the concepts used in the app, farmer questionnaires were held through sending out questionnaires by mail and by distributing hard copies through farmer cooperatives. Unfortunately the app was only available in English at the time the questionnaires were held and the respondents did not get a chance to test the app themselves. The questionnaires consisted of 4 components: The first component was about the general background of the farmer. This data was meant to be used to identify possible trends in the rest of the data, however sampling size was too small to identify any trends. The second component was about the practicality of a mobile app. This component had questions about the possession of phones, use of phones, and reception. The third component was used to assess the terminology used in different parts of the app. This component consisted of a list of terms used in the app and the farmer could score his familiarity with these terms. Finally, there was a component with some open ended questions about points of improvements for clarity and completeness of the terminology. It was considered to add a section about the graphs and layout used in the app, however the questionnaire was already quite long and we figured that this is not the most important part to be assessed. A total of 19 questionnaires were returned. The information in the returned questionnaires were summarised and analysed to identify trends and potential points for improvement.

4. Results and data analysis

4.1 Accuracy

VSA Validation Validation for our VSA methods was carried out by comparing our measurements with lab data at 6 sites. This is a rough validation, due to limitations in quantity and scope of comparable data sets. The result is a general understanding of the dependability of several indicators from our VSA. Not all parameters measured in our tests overlapped with available lab data and thus several cannot be validated using this method. The conclusions on validity of VSA measurements are displayed in the table below. Because validation of the VSA is not the main focus of this report, only the level of validity will be shown here.

Table 1: Validity of the parameters of the VSA measurements

Parameter	Validity
Clay Content	Low
Silt Content	High *
Sand Content	High *
Coarse Fragments	Inconclusive
pH	High
Electric Conductivity	High
Bulk Density & Compaction	Inconclusive

      * Validation showed high validity of VSA data, but method should be questioned since clay content validity is low

As can be concluded from the table above (Table 1), some soil properties could be measured with reasonable accuracy (high) and others were consistently mismeasured (low), while we lacked lab data to validate our method for other properties (inconclusive). This needs to be taken into account while drawing conclusions from this data.

App Accuracy: soil properties and field characteristics 30 app runs were conducted in order to assess the accuracy of soil parameter and threat reporting by SQAPP. Field measurements were conducted for 19 sites via visual soil assessments and laboratory data was assembled for 11 sites. Both types of data were available 4 sites, allowing a wider range of parameters to be assessed. A map of these locations is available in Figure 4. This section will present the accuracy of most indicators provided by the app, together with identified trends. Discussion of these results and how deviations may be linked to land management will follow in section 5. For each VSA we perform or lab data set we analyse, we calculate the difference between indicator values and those predicted by the app. The accuracy of the prediction will be assessed by calculating the average and standard deviation of these differences. A summary for each of the indicators and relevant field characteristics will follow.

Figure 7
Figure 8
Figure 9

• Slope: The app showed its ability to predict slope angles in practice. Our analysis shows an average deviation of 1.41% compared with our field measurements, with a standard deviation of 1.4%. The app’s predicted slope deviated between 0% and 4.15% from the measured slope. The difference between the measured value and the app’s prediction usually ranged between two percent, except for three test sites.
• Rainfall Data: Rainfall data was compared with data from another source (climate-data.org, 30 years of rainfall data) for 11 points all over Spain. Our analysis shows an average difference between the sources of 111 mm/year for all sites, with an average a deviation of 17%. Subjecting these data to the rainfall classification system of the app shows that the sources result in a different classification in 4 of the cases. Neither consistent underprediction nor overprediction was observed for this indicator.
• Soil Particle Distribution: The app predicts a percentage of silt-, clay- and sand particles in the soil. Our VSA provided a % range for each type of particle. We checked whether the app’s predicted value matched this range. For silt particles, this was the case for 100% of the samples. The app often overestimated sand content and only predicted it well in 56% of the cases, while clay content was often underestimated and matched the VSA result in only 33% of the samples.
• Bulk Density: The average difference between predicted and measured values was 0.08 ton/m³, while the standard deviation of differences was 0.06 ton/m³. Although this doesn’t seem to be high, considering the fact that bulk density values usually range from 1.0 to 1.5 ton/m³ in this pedoclimatic zone, the difference is quite substantial. Neither consistent underprediction nor overprediction was observed for this indicator.
• Coarse Fragments: Compared to our VSA data, the differences with app results on coarse fragments (%) are substantial. The average difference was 10.25% with a standard deviation of 9.52%. These values are fairly high, questioning the accuracy of either the app or our VSA method. The scatter plot (Figure 7) shows there is no pattern in deviation between app and VSA: neither consistent underprediction nor overprediction was observed for this indicator. Since lab data was not available for this indicator, we can’t draw a conclusion on the app’s performance to predict coarse fragment content.
• Soil Organic Matter The app appeared to be moderately capable of predicting soil organic matter content in our case study. The average deviation was 0.68% with a standard deviation of 0.13%. No evident over or under prediction was noted. However, the number of samples in this study was limited (n=8), which may affect the representativeness of the data.
• Soil pH Soil pH was predicted quite accurately, with an average difference of 0.51 and a standard deviation of 0.17. The sample size for this indicator was decent (n=25). However, as Figure 8 shows, the app consistently underestimates the value for soil pH.
• Electrical Conductivity Compared to our measurements for soil EC, the differences with the app were 0.2 dS/m on average, with a standard deviation of 0.12 dS/m. Figure 9 shows that the app consistently overestimates the electrical conductivity of the soil, compared to our measurements. While we measure rather consistent values for EC, the app predicts values in a much wider range.
• Soil Nutrients Although the absolute differences for soil nutrients (P, N, K) between app predictions and lab data seemed low, the percentage difference is extremely high, as displayed in the table below (Table 2).

Table 2: Summary of the percentage and absolute differences for soil nutrients (P, K & N)

Nutrient	n	Average Difference	SD Differences	Average |Difference|	Over or under estimation
Available P	10	112%	59.18%	51.96 mg/kg	Systematic Under
Exchangeable K	4	91%	142.73%	0.16 meq/100g	n.a.
Total N	10	24%	364.72%	0.822 g/kg	n.a.

App Accuracy: Soil Threats To assess the app’s capability of predicting soil threat levels, we used its own classification system. For several locations, the app was not able to provide information for some threats, so the number of samples differ per threat. Figure 10 depicts how accurately the app predicts soil threat class, and whether threats are under- or over-predicted compared to our measurements and observations. These results are limited due to reliance on the app’s classification scheme.

Figure 10

4.2 Relevance

Contextual assessment of summary and management advice This was done using 3 different levels of elimination. A total of 18 app runs for different sites were assessed, these included different plots within the same farmer’s field: 4 citrus, 4 vineyards, 1 Olive, 2 Kaki, 1 potato, 2 pomegranate,1 apricot,1 grassland and 2 peach plots. For each app run, 10 AMPs were recommended based on the apps matrix and input data. We, however, noted that 23 AMPs were consistently repeated, see Figure 11.

Figure 11

11 out of the 23 AMPs were found to be relevant for the region. The 11 recommendations included eliminations of AMPs that were not relevant to the threats and parameters identified, the second elimination was based on physical suitability and the third elimination was based on the suitability of threats and parameters identified and the cropping systems. However, there are certain recommendations that were suitable but not priority, those were also eliminated at level 3 see figure 11. These recommendations were eliminated because of their redundancy. There were repetitions of AMPs within the same major AMP class. Notably within the nutrient major class composting, animal manures and slurry would constantly be recommended see figure 11.To highlight this, for example, in instances where the app recognized susceptibility to compaction as a threat with bulk densities greater than 1.5t/m³ , no till as an AMP was still suggested, even at sites where the soils were predominantly clay. Another example, was the suggestion of planting pits on already existing permanent orchards. Farmers 1 (CA001), 4 (AL 001-1), and 18 (MO002) received recommendations such as rotational grazing or area enclosure which were not relevant to the current management practices i.e. citrus, peach and vineyard respectively.

Semi-structured Interviews Twelve interviews were performed, of which 10 interviewees were male and 2 were female. Five subjects are full time farmers, one is an agricultural engineer that provides advice to farmers, two are part-time farmers, three are land owners involved in their farm management, and one is a full time farm manager.

Different soil indicators were used to describe soil quality by the farmers. Six of those indicators are explicitly mentioned in the app, while six others were not (Figure 12). The most used indicator was soil texture, followed by organic matter content and calcareous material content. Although some indicators used require analysis to determine its value, farmers often rely on the general knowledge about the area. For instance, a farmer stated the soil had a high pH, without having the soil ever tested. Some farmers did test the soil, and could be more technical on their description. Most of the farmers used between two to four indicators to depict the soil, with texture being always present.

Figure 12
Figure 13

Many of the soil threats described by the farmers were in the app’s considered list of soil threats. Only nutrient immobilization (due to high amount of calcium carbonate), soil-borne pests, deterioration of soil structure, and decrease in soil water availability are not considered by the app. Interestingly, some farmers did not identify any soil threat occurring in their fields (Figure 13).

Most farmers agree on the predictions of soil physical and chemical properties. Some of those farmers, however, mainly accept the values as plausible. None of the farmers agreed on predictions for biological properties of their soil, due to a lack of knowledge on biological soil quality indicators (Table 3). Farmers tended to accept the values given for parameters that they were less familiar with, as long as the parameter depicted the qualitative category in which their classified their soil. For instances, specific values for EC couldn’t be agreed, but if the values indicated a salinity level in which they could reflect on their soils, then the parameter was accepted. Some parameters were more often sources of disagreement than others. For instance, organic matter was very often stated lower than the predicted by the app, while pH was almost always matching farmers perception.

Table 3: Percentage of farmers that agree with different soil properties proposed by the app

Heading	Agrees or accepts the values	Partially agrees	Disagrees	Unable to evaluate
Physical properties	64%	27%	9%
Chemical properties	64%	36%
Biological properties				100%

When presented with the soil threats predicted by the app, many farmers did not recognise water erosion happening in their fields. Additionally, soil compaction, and pH, presented the highest degree of mismatch (Table 4). Certain threats predicted by the app, for instance pH extremes, matched in value with the farmer expectations, however, farmers did not perceive it as a threat to soil quality or to crop productivity. None of the farmers could judge whether soil biodiversity loss was an occurring phenomenon in their fields, and none of the app runs produced data to detect it as a threat.

Table 4: Percentage mismatch between the app’s predicted soil threats and the farmers’ perception
(the methodology followed to obtain the following values is explained in Section 3.2)

Soil threat	% mismatch
Soil erosion by water	45.5%
Soil erosion by wind	0%
Soil compaction	63.6%
Soil salinisation	18.2%
Soil OM decline	0%
Nutrients	23.3%
Extremes of pH	54.5%
Contamination	0%
Biodiversity loss	0%

The recommendations were presented and discussed with the farmers. From that discussion, we could state that 51% of the recommendations given were proposing AMPs already in practice (Table 5). On average only 4% of the recommendations were considered as plausible future AMPs by the farmers. There were no obvious trends of individual farmers being exclusively negative on the practices proposed, or farmers predisposed to adopt more recommendations. Most of the farmers categorised the recommended practices following the values expressed on Table 5, describing the majority of the practices as already in use, or not to be considered, and in a much lesser extent to be considered.

Table 5: Percentage of recommendations that were already in practice or were/were not considered by the farmers

General qualification of recommendations	% of recommendations
To be considered	4%
Already in practice	51%
Not to be considered	45%

The underlying reasons explaining why certain practices were not considered are exposed in Figure 14. It can be seen that 47% of the not considered recommendations were discarded due to the practice of an alternative measure. For instance, farmers applying manure did not consider application of compost when recommended to do so. Although one may not consider them as equivalent practices, farmers felt that they covered the same functions, and were inclined to the practice that required less financial input or that was more easily achievable (due to the high availability to manure, for instance).

Figure 14

4.3 Functionality

The questionnaires that were conducted consisted of questions related to the possession and use of mobile phones, a 1-5 scoring system to indicate their familiarity with terms used in the app and open-ended questions with suggestions for improvements. A total of 19 responses were received.

The questionnaire showed that every respondent has a phone and has cellular reception at their field. However, 18 out of 19 respondents had connection to the internet on their field and only 11 out of 19 respondents uses apps often. The results of the suitability of the app as a medium are summarised in the table below (Table 6).

Table 6: Results survey analysis on suitability of app as medium

Uses a phone	100%
Uses apps	58%
Has network connecting at field	100%
Has internet connection at field	94%

The table below (Table 7) shows the familiarity of the respondents to the terminology used in different parts of the app. The table shows the averaged scoring of different types of terminology. Terms related to soil properties score lowest on average while the respondents were pretty familiar to the soil threats and AMPs discussed in the questionnaire. The results of the terminology are summarised in the table below (Table 7).

Table 7: Familiarity of farmers with different types of terminology on a score from 1 - 5

Topic	Averaged score
Familiarity with terminology soil properties	3.7
Familiarity with terminology soil threats	4.5
Familiarity with terminology AMPs	4.5

5. Discussion

5.1 Accuracy

In order to assess the app’s capabilities to predict soil parameters and soil threats, we performed an accuracy analysis. We compared data acquired from our visual soil assessment (VSA) with app data provided for the same location. If lab data was available, we also compared these to the data provided by the app. Besides the quantitative analysis on soil property accuracy, we performed a more qualitative analysis on the classification of soil threats by the app.

This way, the VSA played an important role in our research. Although our field measurement were done in a systematic way, the VSA put some limitations on the study. Firstly, the method was validated using lab data that was acquired from farmers, cooperatives, and UMH. Unfortunately, these sets were quite limited, preventing us from confirming the validity of our method for some indicators. On top of that, comparing VSA results with lab data showed that our method was unsuitable to measure certain indicators. Although the methods are straight-forward, drawing conclusions from data that is not validated is not scientifically justified. In the end, this reduced the number of app soil properties that could be considered in our accuracy analysis.

Data on field slopes are generally interpolated and processed from altitude maps. As mentioned before, maps are processed to a resolution of 250m. As terrace widths are often smaller than 250 meters, the process can neglect this highly local topography and predict a larger slope than there is in reality.

While working with the app we noticed that the location that the app automatically provides does not always match reality. It sometimes uses a previous location or a complete random location. Since this results in outputs unrelated to the end-user’s field, the issue should be fixed. Also, we suggest to make an extra function in the location tab that allows you to choose a location or navigate the map by entering coordinates or an address.

While running the app, some of the soil properties and soil threats never had any data, for example the amount of wind erosion per year. Also, some data layers seem to be inconsistent. For example, the app regularly provided no (rainfall data) or highly inaccurate data (altitude).

For some soil parameters, we were unable to assess the app’s accuracy, since we lacked lab data to validate our VSA. This goes for bulk density, coarse fragments and soil texture. For other parameters, the app performed well (soil organic carbon) or reasonable (pH, EC).

SQAPP uses the Global Soil Dataset for Earth System Modelling for the prediction of nutrient levels (Exchangeable potassium, Olsen phosphorus and Total nitrogen). There big differences between lab data and app data for nutrient. WIth most of the nutrients missing the threat level by two classes in the classification system used by SQAPP (Fleskens et al., 2017). Also electric conductivity (EC) data was retrieved from this source. These values were generally slightly overestimated. Soil pH data was retrieved from Soilgrids - from our analysis, these values were consistently underestimated.

The app uses rainfall data to select suitable AMPs. From our comparison with a third party source, we conclude that there is a substantial deviation between these sources, often of a magnitude that would result in different classification. Although we can’t assess the accuracy of this source, we want to stress the importance of reliable rainfall data for the app to be able to estimate soil erosion by water and provide suitable AMPs.

With only three classes (low, moderate, high), the precision of threat results is not too rigorously tested - there could be considerable variation of results even if they fall within the same threat class (EC is not particularly accurate, but results always fell within the “low” threat class for salinization.

The app classifies soil parameters to identify soil threats using global threshold values (Figure 1). However, this classification system is not crop specific, since crop type is not incorporated in the app, which limits the value of its judgement. For example, the app uses the electrical conductivity of the soil to classify salinization threat. Since crops type is not specified, the low risk class ranges from 0 to 2 dS/m, while almonds, not an unfamiliar crop in this area, begin suffering from salinization at EC values higher than 1.13.

Compaction, water erosion and acidification were fairly accurately predicted by the app, compared to our measurements. The pH, however, was consistently underpredicted by the app. The fact that for most samples the pH ranged around the class threshold of 8.0 led to a relatively high number of misclassifications. Our VSA appeared to be unsuitable for assessing the accuracy of water erosion vulnerability. This soil threat will be left out of the rest of the discussion. For wind erosion risk, however, the app predicted a high risk, or provided no data at all. Our conclusion often didn’t match with the app’s results, which might have been due to difficulty in determining wind erosion with our VSA.

The biggest limitation to our study was the amount of resources available. Both time and lab facilities were lacking. This limited the amount of parameters we could analyse, the amount of measurements we could take and the methods we could use. If more resources were available a more thorough study could have been conducted and more app parameters could have been analysed. For future research we would recommend facilitating a laboratory so that the missing parameters can be analysed.  

5.2 Relevance

Contextual assessment of summary and management advice We tried to judge the app run as if we were the farmers. As such, we did not change any of the parameters after performance of the VSA to assess whether there would be a difference in the recommendations provided. We recognize that if we had input soil data that farmers had available or data from our VSA, we would have yielded more relevant information.

However, out of 23 identified recommendations from the 18 app runs, only 11 AMPs were found to be most relevant to the study area. These sites may be farmer’s fields or plots within farmer’s fields depending on the size of the field. Site 4 (AL 001-1) & 5 (AL 001-2) are plots within the same farmers’ field. Whilst the app identified the same threats and parameters needing attention. The app provided similar recommendations but in different order of priority. This was similar for farmer 8 (CA 003-1) & 9 (CA 003-2).For sites 17 (MO001) and 18 (MO002), the app identified the exact same threats and parameters needing attention but provided different recommendations. The only difference between site 17 & 18 was soil cover.The app recommended sprinkler and flood irrigation for sites 8 & 9 when they both had drip irrigation already in place. For site 17 the app suggested claying soils when the threat that needed attention was susceptibility to compaction. Soils with finer particles like soils with higher clay content are more susceptible to compaction (Batey, 2009).

The app supplies 10 AMPs irrespective of the site. For example, there may only be 5 AMPs that are suitable enough to combat a certain soil threat. The app will still fill the recommendations list to 10 AMPs, thus including 5 less suitable AMPs. End-users may not recognise the overexploitation of recommended AMPs in the case where the soil quality is “good” or threats are low.

To highlight how characterisation of the different AMPs into their various classes, may be problematic and distinctively off. Our observations, for example, showed that in farmers 1(CA001), 4 (AL 001-1), and 18 (MO002), received recommendations such as rotational grazing or area enclosure which were not relevant to the current management practices i.e. citrus, peach and vineyard respectively. If the farmer was in any way practicing silvo-pastoralism this would be sensible. However, the farmer does not and unfortunately has no way of inputting this information on the app. Within the water management major AMP class for example, ridge and furrow and soil bunds were recommended irrespective of the input being permanent cropland and in this studies context case fruit tree orchards.

The app does not have provision for the end-user to input their current management practise or current land use outside the boundaries of the app i.e. permanent crop, arable etc. This results in recommendations that may not necessarily be applicable for example, recommendations of straw mulch when chipped branches would be a more suitable recommendation due to availability of material.

Farmers should also not be recommended to convert their land to forest or otherwise abandon their livelihood. Whilst this makes sense for the app’s logic especially when the soil threats identified are in relation to soil structure and nutrient class, these sorts of sweeping conversions (and other similar recommendations) may be useful within a research or policy context but as soon as such a recommendation is presented to a farmer it may suggest that the app is not designed with their best interest in mind.

Semi-structured interviews From the interviews it was evident that the farming community in the area is not a homogeneous group, not in opinions, understanding of soil quality, farming practices, nor uses of technology. This complicates, and perhaps precludes the formulation of a unified or reduced ‘farmer opinion’ on the app. Even farmers with similar management practices and crop types expressed themselves differently when faced with the management advice of the app. Nevertheless, some of the issues and themes encountered occurred frequently enough to warrant discussion.

Farmers describe soil quality using few indicators and are often more reliant on the evaluation of the crop productivity to depict their soils. This may indicate that the dominant concept of soil quality is what some describe as soil productivity (Bünemann et al., 2018). Although the app reports a wide range of indicators to describe soil properties, some farmers mentioned a few indicators not considered by the app. An interesting and recurring one is the presence of calcium carbonate, which plays a big role in nutrient availability for the plants. Some of the indicators, though, were not fully understood yet by farmers. Biological soil quality indicators, for example, are still largely unknown. The discussion is, then, does an indicator become irrelevant due to that lack of understanding? One approach to deal with this discussion is to show such indicators only for expert or advanced users. However, another approach may consider that although the indicator is not fully understood, it still provides a benchmark of certain aspect of soil quality that the farmer can use to reflect, monitor and compare. Hiding the least known indicators may not be necessarily better for the end user, however, it must be noted that more extensive and detailed explanatory information on such indicators will be needed. By linking these indicators to threats and management advice, the app can add meaning and relevance to an indicator that wasn’t previously identified as a priority.

Most of the interviewed farmers did not analyse the soils, which can be interpreted in two ways. On one hand, it is an optimal gap where the app could provide the lacking information, but on the other hand, it may also indicate that farmers are not very dependable on soil information to select their management practices. Further research in this aspect is needed, since it is crucial to understand whether farmers are willing to engage with the information that the app provides.

Farmers’ perceptions of soil threats are particularly diverse. Some farmers may not identify any threats while others in the same pedoclimatic zone and cropping system do. Contrasting definitions of soil threat between the app and the farmers were also noted. For instance, high values of pH were often referred as a threat by the app, while many farmers did not feel it as an issue of concern to their soils. That could have implications for the adoption of the recommendations based on that threat. Providing information on the mechanisms by which these threats can impact the soil and crop productivity could bring these positions closer to one another. Interestingly, water erosion is often underestimated by farmers, which tend to disagree with the outputs predicted by the app. The disagreement is an interesting observation, since it reflects how the information is received by the farmer, and how it is processed. When faced with contradicting information to their perceptions, farmers may either examine their assumptions or discredit the information given. The drivers that may incline towards one or the other option largely depend on how fundamental their knowledge is, or how trustworthy the input from the new source is.

The recommendations provided by the app proved to show a bit of redundancy with the practices already in place. That is a factor which may dramatically decrease the relevance of such recommendations. The fact that only 4% of the recommendations are to be considered as plausible AMPs casts doubts on the efficiency of the application as a transformative tool towards the adoption of sustainable land management practices. In that sense, being able to refine the recommendations by allowing the input of practices already in place may help to reduce such overlapping.

Some of the recommendations were considered a bit general, and not local and context specific enough, in that sense, the app could be more adaptive to context-sensitive interpretations of quality by allowing enterprising end-users to specify additional field characteristics such as crop type and current AMPs.

5.3 Functionality

Based on our own experiences with using the app and the questionnaires filled in by the farmers, there are some points for improvements that need to be discussed. Even though the amount of useful respondents to the questionnaire was limited, some useful information emerged from the questionnaire.

Using the app for the first time proved to be very difficult. There is only a very brief tutorial available that can only be watched once. After this the user is supposed to find out by him or herself what kind of information can or needs to be filled in and what kind of information is provided by the app. Especially, the cumulative probability density functions of the soil properties are difficult to understand without any guidance.

Some of the field characteristics that are allowed to be changed by the user are difficult, if not impossible, to know. Especially the landscape position is almost impossible to be determined in the field, e.g. the difference between flat plains and smooth plains is very small.

Based on the questionnaires we can state that the terminology in the soil threats and recommendations sections are clear. However, many of the soil properties terms remain unclear to the user. Especially, cation exchange capacity, course fragments volume and phosphorus using the Olsen method were not well understood. In the open-ended questions farmers did mention phosphorus as an important nutrient, however the method used to derive phosphorus was not familiar to them. The questionnaire also showed that the medium type used does have some practical implications for the respondents as about half of them does not use apps that often. However, we have to note that the sampling size was very small and might not be representable for the entire Albaida region. Furthermore, 7 questionnaires were returned with a score of 5 for the familiarity of all the terms. We doubt that this is reality and the presented scores might be overestimated due to this.

The app allows the user to save and store a location with its relating input and output information. This is a useful feature, however when opening this location and changing the input coordinates the field characteristics do not automatically update. This results in a mismatch between actual field characteristics and field characteristics of a different location.

In determining the apps potential end-users, it is worth debating on the static nature of the app. How often would a potential end-user have to use the app? For example, apps that provide information on market prices, climate information, are dynamic. Whilst on the other hand, after using SQAPP for a single site, it may take time before the app is used again. As such it might make more sense to an extension officer or an advisory service provider or a farmers’ cooperative as middle men within the value chain. This is because these agents are mobile and as such need insight on the various farmers they work with.  

6. Recommendations for improvement

Our study culminates in the following 11 core recommendations for improvement of the iSQAPER Soil Quality App (Table 8)

Table 8: Core recommendations for the improvement of SQAPP: the soil quality app

Area	Recommendation no.	Details
Accuracy	A1	Insert the the option to let users search and specify location using direct address and coordinate entry
	A2	Reconsider source and evaluate accuracy of datasets for: ‘Soil pH’ Nutrient Availability: ‘Exchangeable Potassium’ ‘Phosphorus using the Olsen method’ ‘Total Nitrogen’ ‘Electrical Conductivity’ ‘Wind erosion vulnerability (classified)’
	A3	Work to fill gaps in datasets for: 'Rainfall data’ ‘Altitude’ ‘Soil Wind Erosion in Agricultural Soil’ ‘Wind erosion vulnerability (classified)
Relevance	R1	Allow for entry of optional field characteristics including crop type and AMPs
	R2	Allow user to specify ‘user type’ during profile creation (Farmer vs Researcher) and curate recommendations accordingly
	R3	App should not give 10 AMPs arbitrarily, instead listing all that exceed a score threshold
	R4	Require that ‘land cover’ be manually entered rather than auto filling ‘arable’ or ‘other’
Functionality	F1	Include a detailed/guided tutorial that can be reviewed by the user at any time
	F2	Soil properties terminology has to be clarified and links to more information could be provided
	F3	‘Landscape position’ should be auto filled (locked to latitude and longitude) and not allowed to be changed manually
	F4	Re-specifying coordinates for a saved location should automatically update field characteristics to match (altitude, precipitation, landscape position, and slope)

7. Conclusions and outlook

In our study we sought understanding on how SQAPP performed on three levels: accuracy, relevance, and functionality. We were encouraged by our results and the response of farmers in the region that this can be an impactful tool in moving towards sustainable soil use and continued productivity in the region.

The app aims to provide a “holistic assessment of soil quality” and inform users about agricultural management practices available to them to improve their soil. It was designed with the intention of guiding users to not only access this data but to understand where they stand by providing comparative contextual information and how they can overcome threats and limitations to the quality of their soil.

Farmers already access agricultural information by a variety of means and this should be taken into consideration in the design and roll-out of the app. The app is seemingly designed to rest in the palms of farmers themselves but it is crucial to understand that the product will exist in a complex web of stakeholders and interests. Farmer cooperatives, agricultural advisory services/extension services, and consultants could potentially serve as end-users. To reiterate this, the language and terminology used by the app greatly influences the potential end-user and as such a review on the language is worth considering. This was well expressed by farmer interview 3. The potential for more varied end-users could be useful for investigation in future studies.

Although this study is limited to the greater Albaida region, several conclusions can be drawn for applicability in other regions of the world. For example, from the farmer interviews it is distinct that while the app’s main objective may be in the holistic assessment of soil quality, farmers were interested in recommendations that related soil quality in terms of soil nutrition and how this related to overall productivity. Interpretations of the SQAPP to local languages may greatly improve the user-friendliness of the app. A review of the AMPs to the specific local context as we observed would greatly increase the validity and relevance of the app.

In terms of accuracy, we can conclude that SQAPP has some trouble accurately predicting soil parameters. Some parameters were fairly accurately predicted (like SOC, slope, rainfall, soil erosion by water), while for others, estimations were highly inaccurate or no data was provided at all (wind erosion, nutrient availability). For the prediction of some other parameters, the app showed potential, but is not accurate yet (pH, EC). Since erroneously predicted data may lead to a wrong classification of soil properties or threats and thus to unsuitable management advice; if possible higher accuracy should be pursued.

Following the philosophy of the greater iSQAPER project, our study embraced a multi-actor approach whereby stakeholder feedback was central to assessing indicator performance and management recommendations reported in the app. It is clear that to change the way people relate to their soil and promote sustainable productivity, data must not simply be made available, but conveyed in such a way that it is convincing and actionable. This highlights a need for the scientific community to step aside from traditional, technocentric research in order to better grasp the environment in which soil information is diffused and adopted. Only then can knowledge transfer be expected to also bring about transformation. The Soil Quality App, with its obvious potential, should aim for no less.

8. References

• Alam, M., Islam, M., Salahin, N. and Hasanuzzaman, M., 2014. Effect of tillage practices on soil properties and crop productivity in wheat-mungbean-rice cropping system under subtropical climatic conditions. The Scientific World Journal, 2014.
• Andrews, S.S., Flora, C.B., Mitchell, J.P., Karlen, D.L., (2003). Growers’ perceptions and acceptance of soil quality indices. Geoderma 114, 187–213.
• Batey, T. (2009). Soil compaction and soil management – a review. Soil Use and Management, 25(4), pp.335-345.
• Bouma, J., & Droogers, P. (2007). Translating soil science into environmental policy: A case study on implementing the EU soil protection strategy in The Netherlands. environmental science & policy, 10(5), 454-463.
• Bowyer, C., & Keenleyside, C. (2017). Joining the dots - Soil health, Agriculture and Climate
• Bünemann, E. K., Bongiorno, G., Bai, Z., Creamer, R. E., De Deyn, G., de Goede, R., ... & Pulleman, M. (2018). Soil quality–A critical review. Soil Biology and Biochemistry, 120, 105-125.
• Cash, D.W., Clark, W.C., Alcock, F., Dickson, N.M., Eckley, N., Guston, D.H., Jäger, J. and Mitchell, R.B., (2003). Knowledge systems for sustainable development. Proceedings of the national academy of sciences, 100(14), pp.8086-8091.
• Cresswell, H.P. and Hamilton, G.J., 2002. Bulk density and pore space relations.
• Doran, J.W., Coleman, D.C., Bezdicek, D.F., Stewart, B.A., Doran, J.W., Parkin, T.B., (1994). Defining and Assessing Soil Quality, in: Defining Soil Quality for a Sustainable Environment. Soil Science Society of America and American Society of Agronomy, pp. 1–21.
• Doran, J.W., Zeiss, M.R., (2000). Soil health and sustainability: managing the biotic component of soil quality. Appl. Soil Ecol. 15, 3–11.
• Fleskens, L., Ritsema, C., Bai. Z., Geissen, V., Yang, X., Mendes de Jesus, J. (2017) Pilot soil quality assessment tool
• García -Orenes, F., Cerdà, A., Mataix-Solera, J., Guerrero, C., Bodí, M. B., Arcenegui, V., ... & Sempere, J. G. (2009). Effects of agricultural management on surface soil properties and soil–water losses in eastern Spain. Soil and Tillage Research, 106(1), 117-123.
• Geertsema, W., Rossing, W.A., Landis, D.A., Bianchi, F.J., Rijn, P.C., Schaminée, J.H., Tscharntke, T. and Werf, W., (2016). Actionable knowledge for ecological intensification of agriculture. Frontiers in Ecology and the Environment, 14(4), pp.209-216.
• Glæsner, N., Helming, K., de Vries, W., (2014). Do Current European Policies Prevent Soil Threats and Support Soil Functions? Sustainability 6, 9538–9563. doi:10.3390/su6129538
• Godfray, H. C. J., Beddington, J. R., Crute, I. R., Haddad, L., Lawrence, D., Muir, J. F., & Toulmin, C. (2010). Food security: the challenge of feeding 9 billion people. science, 327(5967), 812-818.
• Hengl, T., de Jesus, J. M., Heuvelink, G. B., Gonzalez, M. R., Kilibarda, M., Blagotić, A., ... & Guevara, M. A. (2017). SoilGrids250m: Global gridded soil information based on machine learning. PLoS one, 12(2), e0169748.
• Hengl, T., & Mendes de Jesus, J. (2015). SoilInfo App: global soil information on your palm. In EGU General Assembly Conference Abstracts (Vol. 17).
• ISQAPER. (n.d., a). Case Study Site 4: Costera, Valencia, Spain. Retrieved from: http://isqaper-project.eu/project-information-gb/copyright-and-disclaimer-gb/26-english-category/project-info/project-case-studieshttp://isqaper-project.eu/project-information-gb/copyright-and-disclaimer-gb/26-english-category/project-info/project-case-studiesGarcía
• ISQAPER. (n.d., b). Pedoclimatic zones of Europe. Retrieved May 31, 2018, from http://isqaper-is.eu/indicators/pedoclimatic-zones-of-europe/104-overview-spatial-assessment-of-pedoclimatic-zones
• iSQAPER information system. (n.d.). Retrieved from: http://www.isqaper-is.eu/
• ISRIC (n.d.). SoilGrids - global gridded soil information Retrieved from: https://www.isric.online/index.php/explore/soilgrids
• IWARSSON, S. and STÅHL, A. (2003). Accessibility, usability and universal design—positioning and definition of concepts describing person-environment relationships. Disability and Rehabilitation, 25(2), pp.57-66.
• Karlen, D. L., Andrews, S. S., & Doran, J. W. (2001). Soil quality: current concepts and applications.
• Montanarella, L. (2007). Trends in land degradation in Europe. In Climate and land degradation (pp. 83-104). Springer, Berlin, Heidelberg.
• Oldeman, L. R., Hakkeling, R. U., & Sombroek, W. G. (1991). World map of the status of human-induced soil degradation: an explanatory note. Global Assessment of Soil Degradation (GLASOD).
• Olesen, J. E., & Bindi, M. (2002). Consequences of climate change for European agricultural productivity, land use and policy. European journal of agronomy, 16(4), 239-262.
• Peel, M. C., Finlayson, B. L., & McMahon, T. A. (2007). Updated world map of the Köppen-Geiger climate classification. Hydrology and earth system sciences discussions, 4(2), 439-473.
• Pedro-Monzonís, M., Ferrer, J., Solera, A., Estrela, T., & Paredes-Arquiola, J. (2015). Key issues for determining the exploitable water resources in a Mediterranean river basin. Science of the Total Environment, 503, 319-328.
• Popp, A., Lotze-Campen, H., & Bodirsky, B. (2010). Food consumption, diet shifts and associated non-CO2 greenhouse gases from agricultural production. Global environmental change, 20(3), 451-462.
• Tóth, G., Song, X., Tóth, B., Kismányoky T., Fernández-Ugalde, O. (2017) Spatial analysis of crop systems in relation to pedoclimatic conditions in Europe and China. iSQAPER Project Deliverable 2.3 34 pp [Unpublished document]
• Van Beek, C.L., Tóth, T., Hagyó, A., Tóth, G., Recatalá Boix, L., Añó Vidal, C., Malet, J.P., Maquaire, O., Van den Akker, J.J.H., Van der Zee, S.E.A.T.M. and Verzandvoort, S., (2010). The need for harmonizing methodologies for assessing soil threats in Europe. Soil use and management, 26(3), pp.299-309.
• Walther, B. and Moore, J. (2005). The concepts of bias, precision and accuracy, and their use in testing the performance of species richness estimators, with a literature review of estimator performance. Ecography, 28(6), pp.815-829.

---

Note: Download the full report