| Main authors: | Luuk Fleskens, Coen Ritsema, Zhanguo Bai, Violette Geissen, Jorge Mendes de Jesus, Vera da Silva, Aleid Teeuwen, Xiaomei Yang |
| iSQAPERiS editor: | Jane Brandt |
| Source document: | Fleskens, L et al. (2020) Tested and validated final version of SQAPP. iSQAPER Project Deliverable 4.2, 143 pp |
| Contents table |
| 1. Internal review session at plenary iSQAPER meeting Ljubljana, June 2019 |
| 2. External review session at Soil Horizons workshop at the Wageningen Soil Conference, August 2019 |
1. Internal review session at plenary iSQAPER meeting Ljubljana, June 2019
A review session was organized at the plenary iSQAPER project meeting in Ljubljana with project participants in order to review the SQAPP and inform any final adjustments to be made. The session was born out of a felt need to conduct a peer-review of SQAAP within and outside iSQAPER, to discuss issues concerning the validation of SQAPP against measurements, data origin, data coverage, reliability and accuracy of the data, use of pedo-transfer functions if possible, and the possibility for users to include data and feedback (so that users will not just discard the app, but will make it better and interactive). Four parallel sessions were organized, and each participant could rotate to 3 groups to provide input on specific themes.
The themes distinguished were:
- soil quality;
- soil threats;
- AMP recommendations;
- user experience.
The tables below indicate the points raised and how they were addressed.
1.1 Soil quality/soil properties
| Issue raised | How addressed in SQAPP development |
| Where do the data come from? Important to add in the app information about database, number of samples used for make the scoring curve, resolution (area used for the scoring curve). This information should be added in a button called: more information in the app. | The data source and reliability of spatial data is mentioned in info buttons. These data are interpolated across varying scales with variable numbers of samples depending on the original studies. Pedoclimatic zones used are not easily mappable (too many zones, too dispersed zones). A future option is to include statistics on its total area and spatial distribution. |
| Some of the results that we get from the app are really wrong (completely out of reality: nutrients and OM for example) - for this we need to be more detailed? but how to be more detailed (difficult because of temporal and spatial variation, especially for biology)? | This aspect can be solved by better datasets. But these were not (timely) available at global/pan-European/China scale. For now a reliability estimate is provided in info buttons. The challenge for dynamic indicators is even larger. Standardized measurement protocols for sampling on which to base or verify spatial data is one aspect, allowing the user to input data on dynamic dependent variables might be another, but as SQAPP is intended for monitoring long-term changes rather then short-term variations this was not considered a priority. |
| Maybe use pedotransfer functions or feedback from the users, we need more details! | Linked to the above point, some experimentation was done with pedotransfer functions in WP6. This could be a promising avenue to replace poor indicator maps. |
| Maybe gather more data from other European projects! | Data layers were sourced from other projects. Few projects have produced large scale datasets. |
| Important to include a message which says that the results are only an approximation. | This is made clear in the disclaimer of SQAPP for the app as a whole, and in info sheets for specific types of users. |
| The feedback should be added in some way (but maybe problems with data: fines given to people or also we should find a way to control the quality of the data)= this is also the point of the interactive part of the app | This point addresses the interactive reporting of soil data and how to overcome issues with using user contributed data. We will make a start with assessing user-contributed data that are less likely to be inaccurate, e.g. evaluations of AMPs. Using data to improve datasets is a long-term goal towards SQAPP can evolve, potentially catalyzed by social functionality or use of the app for mandatory reporting requirements. |
| But how to include new indicators? | New indicators require an app update, can be data-driven or functionality driven (e.g. the inclusion of a pesticide module relying on user data on pesticide use has already been pursued). |
| Indicators: Active lime content (Spain very calcareous area); Drainage class | New datasets can be included in a future release. |
1.2 Soil threats
| Issue raised | How addressed in SQAPP development |
| Reliability of the data = state clearly; sometime very bad results (for example peatlands= they found less OM) | The data source and reliability of spatial data is mentioned in info buttons. |
| Conversion factors | Common conversion factors are provided in info buttons. |
| Easiness and effectiveness of the indicators? More for extension, not for farmers= training | The classification system of soil threats is simple with three colour codes. The exact determination of the soil threats may be complicated; farmers should focus on those relevant in the local area, supported by extension. |
| Farmers not sure how to assess the severity of threats | The tool can function for awareness creation about soil threats. Providing own data may indeed be complex for farmers. Advisors and researchers could take the lead here. |
| Add nuances and not total values | The colour coding scheme of threat levels gives an indicative idea about the severity of soil threats. For the functioning of SQAPP these are more important than the actual absolute values. |
| Feedback= we need to make sure that they are used | It will be interesting to monitor the feedback for soil properties (for which farmers and land users often have data) vs. those of soil threats. For use see also the same point in the Soil quality/Soil properties table. |
1.3 AMP recommendations
| Issue raised | How addressed in SQAPP development |
| Provide more details in the AMP descriptions or give a link e.g. to WOCAT database | The number of examples of AMPs have been greatly increased (n=391). Within the app links are not desired as it would take users out of the SQAPP. However, links are provided in iSQAPERIs where the AMP database is provided. |
| Consider optimal effects of AMP combination on soil properties – referring to the cluster analysis (D6.1) | Combinations of AMPs would be complex as the number of options multiplies exponentially. Defining combined AMPs is a solution that is implemented that facilitates this |
| Provide as much as tangible advice on an AMP e.g., how many tons/ha/year of organic matter/manure/compost are needed to improve approximately how many percentage of soil organic matter (SOM) content. | This is a step too far for now. Effects of better management are assumed but not quantified. |
1.4 User experience
| Issue raised | How addressed in SQAPP development |
| Consistency in capitalization of letters in SQAPP | Resolved |
| When changing data got stuck | Bugs have been resolved. App users can report bugs they encounter for resolution |
| AMPs: what is the relation between 18 WOCAT / 72 SQAPP | A linkage table has been provided on iSQAPERIS |
| Geographically confine examples or inspire users with global examples? | This discussion point was in feedback from end users implemented as broad global examples being communicated without specifying the country it comes from. The iSQAPERIS website has the links for interested app users to explore further |
| Threat-level not country-specific | Indeed, general thresholds for soil threats were considered; these may deviate from country-specific norms. |
| Download for Chinese users | Android version can be installed using .apk file instead of Google Play |
| Missing data and need to code something | Where there is missing data, a user may indeed have no clue about what value to enter. Consultation with farm advisors may be a strategy here. |
| User guide/Youtube video | Instruction video has been prepared |
| Are you curious about your soil intro screen: repeat at every start-up | If the user logs out or did not log into the app in the first place this is the case |
| Explain why SQAPP is different from other soil apps | Fact sheets, instruction video and background information explain what SQAPP can do |
| Make a flyer to attract people to download the app | Materials have been produced and are used to launch SQAPP on World Soil Day 2020 |
|
Add info button for (characteristics – specify land cover) Add conversion units in info button soil properties and soil threats Add technical details about soil parameters such as bulk density and coarse fragments in info button Add axis titles and an explanation for cumulative density functions, perhaps add a screen between values and cumulative density functions to explain the curve Change “Provide feedback” to “Improve your data” |
These practical suggestions have been implemented in info buttons and instruction materials |
| At the end of the SQAPP screens, include questions on barriers to implementation | The current set-up of the evaluation screen requires the user to evaluate AMPs as either already implemented, inappropriate, potentially interesting and definitely interesting – the latter two are taken forward for ranking as exploratory options. It is assumed that the user is made aware about these options but not necessarily already has a clear idea about them and possible barriers to implementation. This aspect could be explored in the future. |
2. External review session at Soil Horizons workshop at the Wageningen Soil Conference August 2019
The iSQAPER and LANDMARK projects joined together to hold a side event on soil quality and soil functions at the Wageningen Soil Conference in August 2019. The workshop, attended by ca. 100 participants, included a demonstration of SQAPP and the collection of feedback from workshop participants. Figure 11 gives a first impression of the perception of SQAPP. It is viewed as simple, quick, easy and global.
Figure 11
In focus group demonstrations and discussions of SQAPP a number of issues were raised by the participants. These have been listed in Table 11 together with a comment on how these were or could be addressed in (future) SQAPP development.
| Issue raised | How addressed in SQAPP development |
| Recommendations are only accessible if you are registered / logged in. This doesn't have to be necessary, right? | Recommendations are given based on field characteristics for which user input is required. This requires to save user input for a location first, which is why registering is required to view recommendations. Registering is a quick process, as only an email and password are required and no further personal data are solicited. |
| Is it possible to display the uncertainties in the data in the app? Via a pop-up with a disclaimer, for example | Yes this has been implemented |
| Can a farmer indicate which objectives he / she considers important? For example, a farmer who has an above-average interest in climate, if he can indicate this with a kind of slider at the beginning, and then also receive other recommendations, more focused on climate than on production, for example | The implicit objective in SQAPP is that the ensemble of poorly scoring relative soil quality parameters and high risk soil threats are addressed. Soil quality is hypothesized to support multiple ecosystem services which could be considered objectives. But the SQAPP cannot indicate if and how much individual ecosystem services are improved. What the farmer can do is specify the domain of management recommendations (s)he is interested in to get those recommendations highlighted. Alternatively, farmers can be pointed to the iSQAPERIS website where the full overview of AMPs and examples can be consulted per domain. |
| Can we register whether the recommendations are actually being followed up on? By means of an extra button "I am going to implement this". It is interesting to see how much effect the app actually has, and encourages action | In the final screen, the app user is asked to answer the question “Do you think you will implement one of the recommendations?” While less definite action-oriented, this formulation is truer to what can be measured, namely the intention of the user, rather than actual implementation. |
| Can you see where in the world how many people use the app? | Yes, there are two ways to do this. First, data on saved locations can be analysed to this effect. However, anyone can save a location anywhere in the world, so this information may not tell much about the geographical spread of users. The second source of data are the statistics from the app stores on app downloads in different countries. This statistic gives good data on the spatial distribution of app downloads, but not on its actual use. |
| Idea for a follow-up project: link SQAPP and Soil Navigator | SQAPP offers the potential to link to other apps. Within the current project this was not an objective, but further development of functionality is on the radar of the development theme. |
| As the privacy conditions are now, the data entered by the user himself may not be used to improve the databases. If we ultimately want that, then the conditions must be changed, and the users until now be asked if they are still okay with their data being used | The privacy conditions state that data may be used: - To improve SQAPP and the information it provides to you - For scientific research purposes Individual (point) data will not be disclosed for privacy reasons. In the future, one may conceive that user-contributed point data could be explicitly intended for building up trusted, open access sample data, but this is currently not the case. In this case, users would be asked to explicitly state their point-data can be made public so that it does not affect data already contributed |
| For regions where more specific data is available (eg NL) also use this | As the relative soil quality is determined based on data for pedo-climatic zones which often occur in multiple regions/countries, ideally the same data source is used. Juxtaposing more specific data for defined areas would still mean that all probability density curves for affected pedo-climatic zones has to be recalculated. These factors make that adding local datasets is not as easy as it seems. An alternative is to calculate the pedo-climatic zones with the more specific dataset only. This would then give higher quality results in the area considered, but the data to construct the probability density curves would also be available for the region only. |
| Add a notification (pop-up) to the soil properties: “If you have better data yourself, you can enter it here”. Now you really have to look for this possibility. | The button text “Provide feedback” has been renamed to “Enter own data” to clarify this issue. |
| It would be useful for policy makers if SQAPP could be used on a regional scale, instead of point / plot | As indicated in »SQAPP spatial modelling, a regional scale implementation has been set up. As SQAPP relies partially on user input, there are some scenario options (land cover and crop data) that need to be defined for a ‘real’ regional application. A regional application was not envisaged as part of the project concept, but has emerged as a potential powerful future development. |
| Add euros | Data on costs of AMPs and examples of AMPs is scarce and tend to vary significantly. Some require a piece of equipment (used on an unknown number of hectares), others may require costs for investment and maintenance over several years. Another aspect could be availability of subsidies for implementation, which tend to change over time and are region-specific. That is why we have for now chosen a simple ‘complexity’ indicator giving a qualitative scoring of cost, labour and know-how needed. |
| Provide more information about how the graphs are constructed. For example: what about if all the soils in your pedoclimatic zone are super bad, except yours? Then this one will come out very well, while it might not be that good at all. And is it possible to show the graphs only the data of your pedoclimatic zone from your country? "Farmers don't find it interesting to compare themselves with people from other countries," it was noted. | The procedure to construct probability density functions is explained in FAQs and instruction video. Indeed, theoretically all soils in a pedo-climatic zones could be bad/degraded. As management information is not (yet) taken into account in the construction of maps, this may particularly apply to the ranges in natural areas being reduced and not represented as land cover is a co-variate in most soil data mapping procedures. Still, the relative score does indicate this would then be the best farmer in the pedo-climatic zone. Regionalizing of the pedoclimatic zones to countries would introduce other biases and limit the comparability in smaller countries while still not offering a more regional perspective in very large ones. A country-specific approach to pedo-climatic zones would be more useful if based on a dedicated higher accuracy soil data system for that country. |
| The practical details of the recommendations are also important, if a farmer really wants to apply them. Can we possibly here. refer (link) to the info in WOCAT? | Links within the app would be confusing as it requires the user to navigate away from the app. The iSQAPERIS section »Agricultural management practices recommended by SQAPP is referred to as a first-stop portal for further advice and links. |
| Idea: Integrate feature that brings together users with the same threats / recommendations in the same region. So that a kind of study clubs can arise around a specific theme | Innovations for social collaboration are indeed an important future development potential for SQAPP. |
| Expectations management: clearly indicate that this tool is a support, and not the hard truth. | Under the About section we have added the following text: “ Scale is an important consideration when using this app. The analysis provided is based on global and regional data and should not be interpreted as exact data for a specific point on the map, such as your garden or farm field. Within the app, you may upload local data to refine the soil quality analysis and resulting recommendations. European and Chinese geography contains the richest data layers within SQAPP.” |
| Idea: add intro text (eg after choosing the location, before viewing soil properties). Indicate here what the app does and does not do. Similar to previous comments on disclaimer / expectations management | Support text is provided in information buttons and the more information section in SQAPP in order not to break the functional flow of the app. |
| Introduce SQAPP in study clubs (such as Kringloopwijzer in K&K) to make it land with farmers | Information sheets on the use of SQAPP by various audiences have been produced. Following the launch event we will monitor and target groups |
| Do not throw the app over the fence when the project is over, keep maintaining! And implement it in education; learned young is done old. | Fully agreed! SQAPP has already been integrated in the student education of participating organisations. |