Introduction 

Forest fires are a complex phenomenon driven by different biological, physical, and human factors, at least in southern Europe, experiencing around 85% of the annual burned area on the continent ([6]). In 2022, the burned area increased to 837.000 ha, with countries like Romania, France, and Portugal joining Spain and Italy as the biggest contributors to the burned areas within the EU27 ([17]). Interactions between extreme climate conditions, vegetation, and human behaviour are the most critical and important factors driving forest fires ([5], [4]) not only in southern Europe, but recently in northern and western Europe. Fire activity is also influenced by other variables, such as topography, stand and forest fuel characteristics, size and compartmentalization of forests, land ownership fragmentation, landscape vegetation, management practices, land use changes, rural exodus, and urban expansion ([21], [3]). These variables are mingled in various combinations and influence wildfire risk. Humans through their activities and behaviour affect mostly when and where fires are ignited and how they eventually propagate. In Europe, human activities accounted for more than 95% of ignitions ([10]). Therefore, if new fire prevention-oriented tools could be created and effectively used, they could potentially contribute to the reduction of people caused ignitions and relieve some of the burden that fire management agencies must carry each fire season (less ignitions, less fatigue, better and timely first response).

Several such initiatives have been developed by the forest services, civil protection agencies, academia, and industry, focusing on the importance of effectively mapping fuel types across extended landscapes. The lack of updated and accurate fuel maps for almost all European countries (with the exception of some southern countries) has serious implications for how fire risk is estimated, as it makes it impossible to conduct fire behaviour simulations or run daily fire hazard predictions using similar methods as those applied in the latest version of NFDRS or Wildfire Analyst which is currently implemented in numerous fire agencies.

A concerted project named LANDFIRE, aimed to determine fuel types at the country scale was started many years ago by the Forest Service of the United States using the vegetation fuel groups developed by Anderson ([1]) and Scott & Burgan ([18]); recently, an adaptation to the European landscapes was carried out by Aragoneses et al. ([2]). However, mapping fuel types at larger scales inherently leads to uncertainty in the results, mostly due to the lack of ground data to train the models. The production of a reliable and fine-scale pan-European fuel map requires a rigorous process for validating spatial outputs produced with remote sensing, geoprocessing, Machine Learning (ML), and Deep Learning (DL) techniques with ground-obtained data (i.e., location, photographs, and fuel characterization). Keane & Dickinson ([9]) and Tinkham et al. ([20]) suggested a photoload sampling technique to support similar surface fuel mapping.

Mapping vegetation characteristics, such as fuel types, canopy base height, or biomass, is a daunting task due to the environmental complexity and the features of interest to be identified and distinguished in each landscape. Vegetation can show very different vertical and horizontal spatial characteristics. Grasslands, shrublands, and forests with short to very tall trees can have a vertical structure with high spatial complexity. The latest advancements in remote sensing, from satellites to drones and portable sensors, have remarkable improved our capability to map vegetation parameters; vegetation height and biomass can be estimated by combining remote sensing products with predicting models that use different categories of sophisticated algorithms based on ML and DL. Nevertheless, it is well known that such predictions may suffer from potential errors which can be quite significant, especially when the training process is fed with inaccurate data, the number of training samples is limited, or the attribute to be predicted is not representative of the real-world variability. A solution to the abovementioned issue is to conduct field inventories using mobile-based surveying approaches to support data collection that are necessary for all three steps of AI classification, i.e., training, testing, and validation.

So far, traditional remote sensing products used for fuel mapping in forested lands have been constrained by sensor inability to penetrate the canopy and identify understory features, thus limiting their capacity to capture fuel loads and the distribution of surface fuels ([8]). This limitation is often solved by generating a set of rules considering the type and state of tree vegetation and/or on-site environmental characteristics ([16]). Regardless of the methodology applied, surface fuel classification is based on general rules or variables obtained through statistical models that, while based on empirical data, tend to aggregate predictions around means, missing the opportunity to account for the variability existing in the real world and requiring expert-based adjustments ([12]). This calls for a specific and broad-scale inventory of the main fuel types, especially forest fuels.

Mobile applications for forestry use ([7], [13], [19]) and for wildfire purposes already exist. A concrete example in wildfire purposes is the WFA Pocket ([14]), which implements the classical fire spread models and has been used by thousands of users worldwide. However, to the best of our knowledge, there are not many mobile applications widely used to collect field data and characterize fuel types. The FIRE-RES project aims to integrate fire management approaches through a user-friendly and intituitive mobile application which can be used by non-experts to collect fuel data. Therefore, the main objective of this work is to develop a mobile application to collect oriented and geolocated images in a quick, simple, and reliable manner, thereby collecting user-acquired ground truth data with consistency across large areas and capturing the full range of European fuel types and landscapes. The collected images will then be interpreted by experts using an online viewer and standard guides to translate the images into fuel types. To make the process more rigorous, each image will be interpreted by more than one expert. This redundancy of interpretations will allow for checks on consensus among experts. If an image has different interpretations, a consensus is reached through further interaction among experts. This process will provide a robust way to validate the pan-European fuel map we are planning to generate during our follow-up research.

  Materials and methods 

Overview: the FIRE-RES Geo-Catch App and viewer

The FIRE-RES Geo-Catch framework consists of an App for mobile phones (FIRE-RES Geo-Catch App, hereafter “App”) and a web-GIS portal (FIRE-RES Geo-Catch Viewer, hereafter “web-GIS” or “viewer”), both available online via common browsers. The software was developed as a lightweight and easy-to-use application for a smartphone. By design, it relies on well-supported application programming interfaces (APIs) to collect and share low-memory-footprint online data. The following sections describe the components of the whole framework, including the App, viewer, and storage repositories.

App user interface

The first icon row, respectively from left to right, displays (i) a background light/dark toggle button, (ii) a link button to a manual page that provides detailed information about the App, and (iii) a log button. The latter opens a log-space with a detailed list of messages that provides useful feedback when the App is tested internally and is improved only among FIRE-RES partners.

The second row has a single large button (with a camera icon) that opens the video-stream connecting to the device camera. The last button provides a list of available projects to be chosen by the user, to which their field collected data will be assigned. This option is mandatory, as each captured image must be connected to a project, thus giving greater credibility and reachability to specific data stored on the server; this is useful for meta-analysis, as system administrators can choose at will from which project they can extract the uploaded information. Another mandatory step that must be taken upon the first usage of the App is user validation, described in the next section.

The interface (Fig. 1) keeps a dark background for battery efficiency. Users can change this option by clicking the first icon on the top left of the interface. The importance of this function is not for aesthetic-related reasons but because the App will be used in the field and the light background is a better choice for some outdoor conditions (e.g., strong sunlight).

User validation

To avoid abuse of the App, a white-list approach is used. Photos are loaded and stored on the server only if the user is validated through a verified email address by simply clicking the provided link. To ensure privacy, the email is not stored anywhere but is only used to validate a user’s unique ID (UID), a 10-character alphanumeric string that is persistently assigned to the user’s phone and browser. The same phone will get different UIDs if accessing the App from different browsers on the same device (e.g., Chrome™, Mozilla Firefox, etc.). The users will also get a secret token that they can use to delete their images stored on the server. This validation approach keeps full privacy for the user while preventing anonymous, unwanted, or unauthorized use of the App, e.g., uploading random images or trying to use the uploading capabilities for intrusion in the server.

Collecting geotagged and oriented images

When the user taps the photo icon, the interface for capturing images is activated (Fig. 2). The interface consists of two square areas over a red background (Fig. 2 - left, area A and area B). One square has the video stream from the camera (Fig. 2 - left, area A), overlaid at its top part with coordinates, accuracy, and horizontal and vertical angles in degrees. The accuracy is directly obtained from the APIs, with a 95% confidence level. When the users are satisfied with the image and have checked the geolocation accuracy, they can tap on the video stream square, triggering a snapshot of the video frame and showing it as an image in the bottom square (Fig. 2 - left, area B), along with its coordinates, accuracy, and orientation values (Fig. 2 - central panel). After the user has taken the photos, the Geo-Catch App will double-check that they are accurately geotagged and will inform the user if the camera is not enabled to read and save the location in images. Up to this point, the image is not yet stored, but this step is necessary for double-checking and evaluating the captured image (photo quality and accuracy). Upon approval, the user can tap the captured image (Fig. 2 - left, area B) to finalize it and finally save on the smartphone.

When the device is online, both photo and geolocation information are uploaded to the Interdepartmental Research Center of Geomatics (CIRGEO) server, which belongs to the University of Padua, Italy. If an internet connection is not available, the data package (image with accompanying information) will be cached and synchronized the next time that the FIRE-RES Geo-Catch App detects an internet connection. Users will not lose cached images if the App is shut down, since images captured while offline are stored until synchronized with the server. The local stored copy will be deleted only when the server has successfully received the uploaded image.

Technical details on the workflow are shown in Fig. 3and described below, with user actions in italics:

1. “the user in the field opens the App” from the device browser or directly from the installed App icon, and this opens the video stream interface (Fig. 3A).
2. “The user tap the video stream box” (Fig. 2 - left, square A). This action triggers the following: (2.1) the image is temporarily saved in the device memory as a JPEG compressed format with 90% compression and shown in the bottom square (Fig. 2 - middle, square B); (2.2) the following additional information is stored as an ASCII string in the EXIF tag “ImageDescription” directly in the JPEG image memory blob: (a) project name; (b) geolocation: longitude and latitude; (c) orientation: horizontal and vertical Euler angles; (d) timestamp in POSIX time: milliseconds from 1° January 1970; (e) horizontal spatial accuracy: in meters; (f) user ID. An example of the information ASCII string below: FIRELinks|9.2174032|44.3598822|358|86|1679734943295|3.2|q7p1w0a1nk.
3. “The user tap the captured image” (Fig. 2 - middle, square B). The tagged image is stored internally on the smartphone using the IndexedDB JavaScript API. The unique key used in the database is the timestamp (refer to point 2.2d above), the value is the image blob, and the information is the ASCII string presented above.
4. One of two possible automatic actions follows (Fig. 3B). (a) The device is online, which triggers the uploading of the image to the server. If the upload is successful, then the dataset is deleted from the IndexedDB of the device. (b) The device is offline: nothing happens; the image is kept in the internal device database via the IndexedDB JavaScript API.

If event (4b) is true, when the device becomes online later, (4a) is activated. This action is referred to as a “lazy” strategy of synchronizing data. In other words, the action is triggered without the user having to wait or check for the proper upload. If no upload occurs due to the smartphone being offline, the data is safely stored on the user’s device.

Framework

The proposed App can also work completely offline, thanks to its Progressive Web App (PWA) framework. The PWA framework enables web-Apps to work as smartphone Apps without the requirement for the standard Android or iOS frameworks to be installed (i.e., Google Play™, Apple Store™). This has the advantage of working as a simple web-App, thus accessible via a web browser, or as a standalone application.

The App’s activities and interface (Fig. 3B) are built on HTML-JavaScript as the client-side environment and PHP as the server-side environment. The latter collects the raw image data as a byte-array blob and decodes it for storing the image in a file-based structure that is stored on the server. A specific web-GIS portal, FIRE-RES Geo-Catch Viewer, processes all information stored on the server (Fig. 4), thus decoding the EXIF from all images, tagging users to each image, and allowing users to view each photo with its associated data. A potential advantage of this approach is that every image keeps all information, making it completely transferable to other software that might want to process the image using information in the EXIF tag “ImageDescription”.

From photos to fuel types

Our purpose and follow-up plan include the assignment on the acquired photos by experts of a fuel model type that will subsequently be used to train, test, and validate an AI algorithm. This pattern will be provided to a wide range of forestry and wildfire research end-user in the scientific community, enabling them to predict the fuel model from the collected images automatically. Labels will be given by experts who can interpret and assign the correct fuel model type. In order to achieve an objective evaluation, each image will be assessed by two or more different experts, thus cross-validating each interpretation. If all independent experts assign the same fuel model type, then the agreed type will be selected directly; otherwise, a consensus will be reached through interaction among experts. These actions can be quickly deployed via the FIRE-RES Geo-Catch Viewer, which provides the necessary tools for experts to view and label the images directly online and then interact to provide consensus when necessary. Labelled images can be used for training and validation of an AI model that will extract vegetation features from the scene’s background and further classify them into fuel model types. Fig. 5shows the first application of an automatic segmentation process using the method proposed by Kirillov et al. ([11]) in different main fuel types.

Location and orientation accuracy

Recommended image accuracy in terms of geo-position and orientation is related to the targeted scale on several projects. In the FIRE-RES project, we target to validate a map where the cell spatial resolution is 100 × 100 m and the horizontal geolocation accuracy is 10 m (90^th percentile Circular Error - CE90), i.e., a minimum of 90 percent of horizontal errors fall within the stated CE90 value. Smartphone’s geolocation accuracy, even under forest canopies, is far below this tolerance threshold, and therefore correction of gross errors is required. It must be noted that this implies filtering out gross errors (outliers). Testing and validation of smartphone expected accuracies have already been conducted in previous research in an urban forest condition ([15]), finding CE95 < 5 m.

Regarding orientation accuracy, rapid tests have been done by orienting the smartphone perpendicular to regular building facades and comparing the direction of horizontal angles using existing cartography. Results showed that the readings fell within ± 20°, but for some of them, an offset of 180° was noticed. This is probably due to the smartphone’s internal interpretation of its orientation regarding the axis of the internal compass. However, additional studies are planned to further investigate this matter.

  Conclusions 

A reliable fuel map is a useful dataset to support the assessment of fire effects and mitigate the negative consequences of wildfires through early warning systems, fire risk estimation, and monitoring in real time the state of forests susceptibility to wildfires. Fuel map classifications are well structured hierarchically and well tested for fire simulations. The FIRE-RES Geo Catch mobile application can contribute to provide more robust and accurate fuel maps at a pan-European scale, improving the accuracy of fuel type classification by using obtained field inventory data. People involved in acquiring geolocated imagery to support fuel map through ground control points will also contribute during validation phases by providing specialized knowledge. The geolocated images need to be well distributed across all over Europe, and our target is to obtain them in a collaborative manner involving forest services, civil protection agencies, research centres, academia, and private companies. Involving the public in general could also be an option, thus taking advantage of more regular data acquisitions. In addition, the databank that will be created through this in-situ crowdsourcing of images can be used for other purposes. Applicability will be further increased if the images are taken recurrently for the same areas during different time periods. Finally, this tool can be used to assess forest status by identifying main natural and anthropogenic alterations, capturing changes not only in fuel types but also in the state and condition of a forest, whether phenological or due to natural or human impacts, identifying the main drivers of forest change, that are not necessarily always driven by fire events.

  Acknowledgements 

This mobile application was funded by the European Union’s Horizon 2020 Research and Innovation Programme, by the project entitled “Innovative technologies & socio-ecological-economic solutions for fire resilient territories in Europe - FIRE-RES”, grant agreement no. 101037419. We would like to thank each of our colleagues who anonymously participated during the testing phase of the App, which has been very helpful in improving and making the App stable and operational.

  References