Chapter 1 Overview and motivating data
1.1 Aim and scope
We wrote this book with two aims in mind. First, provide an introduction to programming for those who work with forestry and ecology data. Second, guide and illustrate implementation of fundamental forestry data analysis techniques using a contemporary and powerful programming language.
We recognize that understanding and responding to current environmental and management challenges requires strong quantitative and analytical skills. Increasingly, forestry and ecology professionals assume the role of data analyst due to unprecedented environmental data availability and the questions these data are used to answer. Following a definition given by Bravo et al. (2016), a data analyst has the ability to make appropriate calculations, convert data to graphical representations, interpret the information presented in graphical or mathematical forms, and make judgments or draw conclusions based on quantitative data analysis. In addition to these skills, and what turns out to dominate much of an analyst’s time, is basic data wrangling, which is the task of transforming data from one format into another to make it more appropriate or valuable for subsequent purposes such as analysis. Some estimate that upwards of 50% to 80% of an analyst’s time is spent data wrangling (S. Lohr 2014).
The analyst’s job is best tackled using a flexible and easily understood programming language with high-level functionality, meaning lots of ready-made functions to accomplish routine data wrangling, analysis, visualization, and reporting tasks. While there are several high-level language contenders meeting these general criteria, currently R (R Core Team 2024) and Python (Python Core Team 2015) dominate the arena. Both are open-source, meaning the code is freely available and can be freely redistributed and modified. One language isn’t better than the other—choice between the two depends on your use case and the questions you’re trying to answer. While Python is often touted as a general-purpose language with an easy-to-understand syntax1, the same could be said for R. Relative to Python, R’s origin and evolution favored interactive data analysis, thereby giving it an edge for the type of work pursued in this book. The focus here is to provide an introduction to R, which has become the more prevalent of these two languages in the environmental sciences.
There are of course numerous excellent books and online resources for learning R, several of which we recommend in subsequent chapters to supplement and extend the topics covered. In fact, many of these books would serve well as your introduction to R. Our personal experience, however, is that it’s easier and more enjoyable to learn challenging and abstract topics when their relevance is tied to your field of interest using real-world examples. To this end, we motivate programming topics using datasets collected to answer specific forestry and environmental science questions.
After working through the fundamentals of programming using R in the first half of this book, our attention turns more fully to the analysis of data collected for forest inventory and monitoring efforts. Following a definition given by Kershaw et al. (2016), forest inventory is the procedure for obtaining current information on the quantity, quality, and condition of the forest resource, associated vegetation and composition, and many of the characteristics of the land area on which the forest is located. Forest monitoring quantifies change in the composition, structure, size, and health of the forest over an extended time period.
Forest mensuration describes the tools and procedures for measuring forest characteristics as well as the statistical methodologies for generalizing these measurements. There are several contemporary forest mensuration books that offer a comprehensive treatment of forest measurement techniques and associated statistical methods for estimating a wide range of forest characteristics. Several books in particular, Laar and Akça (2007), Kershaw et al. (2016), and Burkhart, Avery, and Bullock (2018) provide an excellent foundation for applied forest mensuration with a focus squarely on providing quantitative information to support silviculture and management decisions. We mention these books because they provide an introductory-level background and motivation for the practical programming and applied data analysis covered in this book. Because our focus is on implementing quantitative methods, this book also pairs nicely with material of a more statistical nature, for example, forest sampling (Gregoire and Valentine 2007; Mandallaz 2007), growth and yield modeling (Weiskittel et al. 2011), and advanced environmental data analysis topics using R (Robinson and Hamann 2011; Mehtätalo and Lappi 2020; Green, Finley, and Strawderman 2020).
1.2 Motivating forestry datasets
We are in a data-rich era that provides extraordinary opportunities to understand complex ecological processes at local to global scales. Proliferation of environmental data is a result of investments to collect information for regulatory, monitoring, and resource management objectives, and technological advances in spatially-enabled networks along with geospatial information storage, analysis, and distribution systems. The sources generating this unprecedented volume of data are increasingly diverse and specialized, e.g., monitoring station instruments, remotely located sensors, and georeferenced field measurements.
By automating time-consuming, tedious, and repetitive data wrangling tasks, computing environments, such as R, allow the analyst to focus on answering the questions for which the data were collected. In forestry, data are collected to answer a wide variety of questions. For example, field crews collect forest inventory data to answer questions about the amount and location of timber or non-timber resources. Monitoring data are collected to understand change in forest characteristics. Highly detailed individual tree data are collected to understand allometry, which is the growth and size relationship between different parts of an organism. Experimental manipulations of trees, stands, or forests are used to better understand how environmental change and disturbance events impact individual growth rates and trends in population demographics.
The next several sections give a high-level tour of forestry datasets collected to answer a range of questions central to traditional forestry operations, silviculture, and management, as well as broader research questions related to climate and environmental change. When an analyst receives a dataset, the first task is to understand the data format and to identify features for consideration in the subsequent wrangling and analysis.
We first consider datasets with fairly simple structure, a single flat file, meaning the data are organized in rows and columns typically with rows corresponding to observational units (things we measure, e.g., trees) and columns corresponding to the various measurements and characteristics recorded for each unit. For the most part, in these examples we have left the datasets in their original form and have done minimal cleaning and wrangling. For example, you’ll notice the datasets use different measurement systems and approaches for organizing information. We explore these datasets, and others with more complex structure, in more detail throughout the remainder of the book to motivate and illustrate programming tools for cleaning, wrangling, summary, and analysis.
The HTML and printed versions of this book offer slightly different figures to introduce the datasets. When maps of the datasets are presented, the HTML version allows for more interactive data exploration, similar to a geographic information system (GIS) viewer where you can pan, zoom, turn data layers on and off, and click on data features to reveal more information. The printed version includes only static maps and hence is slightly less exciting. The figures (and this entire book for that matter) were created using R and, after working through the subsequent chapters and exercises, you’ll have the skills to recreate these dynamic and informative data products. We provide the following datasets, as well as others used in this book, as supplementary material.
1.2.1 Fernow Experimental Forest biomass dataset
Initially, we might have in mind a modest-sized and uncomplicated dataset that serves a fairly specific purpose. For example, in forestry it’s convenient to have a mathematical formula that relates easy to measure tree or stand characteristics to more difficult to measure characteristics. These mathematical formulas, referred to as allometric equations, are a common feature in forestry tasks. For the practicing forester, these formulas are often encountered when computing stem volume or biomass using diameter at breast height (DBH; 4.5 ft or 1.37 m from the ground) and perhaps species—in this case DBH is the easily obtained measurement and volume or biomass is the more difficult to measure characteristic.
| Species | DBH (in) | All woody dry (kg) | Stem dry (kg) |
|---|---|---|---|
| Liriodendron tulipifera | 7.8 | 129.5 | 80.5 |
| Prunus serotina | 2.4 | 6.3 | 4.9 |
| Acer rubrum | 6.0 | 105.1 | 54.7 |
| Prunus serotina | 6.7 | 118.3 | 72.4 |
| Prunus serotina | 3.4 | 20.1 | 14.0 |
Creating an allometric equation typically requires fitting a regression model that relates the difficult to measure characteristic to one or more easily measured characteristics. For example, one would likely need a dataset that contains measurements of both DBH and total biomass on a collection of trees to create a general allometric equation for this relationship. Clearly such a dataset would be quite expensive and time-consuming to collect, because it would entail harvesting trees to directly measure their biomass. Many such valuable datasets exist in the forestry literature, see, e.g., Jenkins et al. (2002) and Radtke et al. (2015).
Table 1.1 contains data for five of 88 trees felled and measured as part of a United States Forest Service (USFS) study on the Fernow Experimental Forest (FEF), located in the central Appalachians of West Virginia (Wood, Kochenderfer, and Adams 2016). Data include DBH, tree height, as well as green and dry weight of tree stem, top, small branches, large branches, and leaves. Table 1.1 provides a subset of these data, where each row holds the measurements for one tree. Oddly these data include a mix of English system, i.e., DBH in inches, and International System of Units commonly known as the metric system, i.e., weights in kilograms. These data, like many Forest Service datasets collected on experimental forests, are well documented and include detailed metadata.2 Metadata are data that provide information about other data, i.e., data about data, and typically describe data collection protocol, measurement units, and file organization.
FIGURE 1.1: Relationship between tree DBH and component weight by species using the Fernow Experimental Forest biomass dataset. Points represent data observed on the 88 trees and blue lines are the best fit lines through the observations.
Figure 1.1 shows the relationship between DBH3 and dry weight of the total tree weight (All woody), stem, all large branches, and all small branches, separated by tree species. The points in this figure are the 88 tree measurements and blue best fit lines are from an allometric equation that characterizes the relationship between DBH and component weight. Given DBH and species of a new tree, its biomass components are estimated using the mathematical formula represented by the blue lines. We’ll encounter these data again in Chapter 9 when we learn how to make informative graphics for effectively communicating data and analysis results.
1.2.2 Elk County timber cruise dataset
A timber cruise is a forest inventory to locate and estimate the quantity of timber in a given area according to species, size, quality, potential products, and other characteristics. We’ll see later that many decisions go into designing an inventory. These decisions include the number and placement of measurement points, which trees to measure around a given point, and what type of measurements to take on the selected trees.
FIGURE 1.2: Location of timber cruise points on a 271 acre forested property in Elk County, Pennsylvania. Locations colored by acceptable growing stock basal area.
Figure 1.2 shows the resulting dataset from a timber cruise of a 271 acre forested property in Elk County, Pennsylvania. The cruise data comprise 54 points where a forester measured tree characteristics such as species, DBH, height, and stock quality potential. Points were located on a 7-by-7 chain (1 chain equals 66 ft) grid resulting in about 1 point per 5 acres. We colored point locations based on the basal area (ft\(^2\)/acre) for trees recorded as acceptable growing stock (AGS). The dotted orange line shows the transect, or course, the forester walked between points. If you’re reading the HTML version of this book, mousing-over gives the point number and a single click gives a list of AGS trees used to calculate the basal area color reflected in the figure legend.
| Point ID | Trees | Species | DBH (in) | Height (logs) | Product |
|---|---|---|---|---|---|
| 1 | 1 | Black Cherry | 12 | 1 | AGS Sawtimber |
| 1 | 1 | Black Cherry | 10 | 1.5 | AGS Pulpwood |
| 54 | 1 | Beech | 30 | 1 | Snag |
| 54 | 4 | Beech | NA | NA | Regen |
Table 1.2 shows a subset of trees measured on the first and last point, where each row corresponds to a set of characteristics shared by one or more trees. Information in the first row of this table tells us inventory point 1 has one black cherry with a DBH of 12 in and one 16 foot log that is designated as AGS sawtimber. Obviously one would need to consult the cruise metadata to figure out the DBH and height units as well as the stem qualities needed for an AGS designation. The last row in Table 1.2 records four beech stems at point 54 that were considered regeneration. In such cruise data, regeneration typically refers to stems that are quite small with little or no DBH; hence, the NA values for DBH and height are expected. As we’ll see later, NA values can indicate data missing for other reasons.
We’ll revisit these data when we look at efficient ways to generate property-level tables and figures of key forest characteristics summarized by species and product type (Chapter 8). For consulting foresters, who collect and analyze these data routinely, canned programs that automate wrangling, summarizing, and reporting tasks are particularly valuable.
1.2.3 Free-Air Carbon Dioxide Enrichment experiment dataset
We often encounter data from highly structured and complex experiments. Such data typically present challenges in organization/storage, exploratory data analysis (EDA), statistical analysis, and interpretation of analysis results. An example dataset comes from the Aspen Free-Air Carbon Dioxide Enrichment (FACE) experiment conducted from 1997-2009 on the Rhinelander Experimental Forest West Unit. The Aspen FACE Experiment was a multidisciplinary study to assess the effects of increasing tropospheric ozone (O3) and carbon dioxide (CO2) concentrations on the structure and functioning of northern forest ecosystems. The design allowed researchers to test the effects of these gasses alone (and in combination) on many ecosystem attributes, including growth, leaf development, root characteristics, and soil carbon. The dataset considered here comprises tree diameter measurements at 10 cm above the ground from 1997 to 2008 for Populus tremuloides grown inside twelve 30 m diameter treatment arrays in which the concentrations of tropospheric O3 and CO2 were controlled (Mark E. Kubiske 2013). Because there was no confinement, there was no significant change in the natural, ambient environment other than elevating these trace gas concentrations (Hendrey et al. 1999). Although individual tree measurements are similar to those in the FEF dataset in Section 1.2.1, (i.e., height and diameter), the study design specifies various tree genetics, varying gas treatments, and treatment replicates. These data are an example of longitudinal or time series data, meaning researchers repeatedly measured individuals (trees in this case) over time. Table 1.3 provides a subset of measurements for the first five trees as well as five trees selected at random in the dataset. Here, a row identifies each tree’s experimental treatment, genetic description (Clone), and diameter growth over time.
| Treat | Clone | Diam. 1997 | Diam. 1998 | Diam. 1999 | |
|---|---|---|---|---|---|
| 1 | Control | 8L | 0.4 | 1.34 | 2.52 |
| 2 | Control | 216 | 0.9 | 3.11 | 5.19 |
| 3 | Control | 8L | 0.25 | 0.96 | 1.71 |
| 4 | Control | 216 | 0.78 | 3.03 | 5.03 |
| 5 | Control | 216 | 0.75 | 1.92 | 3.47 |
| 1017 | Elevated O3 | 8L | 0.6 | 2.01 | 3.12 |
| 1860 | Elevated CO2+O3 | 259 | 0.68 | 1.22 | 1.54 |
| 679 | Control | 8L | NA | 0.42 | 0.43 |
| 129 | Control | 42E | 0.26 | 0.54 | 1.25 |
| 930 | Elevated CO2 | 271 | 0.75 | 2.41 | 4.21 |
Notice the 1997 diameter measurements in Table 1.3 contain an NA, which indicates missing data—you’d see a lot more missing data if all the data were displayed. These missing observations resulted from tree mortality or data recording error. When analyzing these data to identify treatment effects, it’s essential to make a decision on how to handle such missing data.
FIGURE 1.3: Free-Air Carbon Dioxide Enrichment experiment aspen diameter growth by treatment. Bars on each point indicate 95% confidence intervals.
Figure 1.3 shows an EDA plot of the treatment effects compared with the control (i.e., no gasses added). This figure suggests a possible positive growth response to elevated CO2 and negative growth response to elevated O3, compared with the control. This is consistent with other studies on atmospheric CO2 fertilization, as well as studies on the damaging effect of O3 on leaf tissue. The figure also shows experimental units that receive both elevated CO2 and O3 do not differ substantially from the control, which could suggest that any CO2 fertilization is offset by O3 damage.
Subsequent chapters focus on creating these and other EDA plots and summaries (Chapters 7 and 9), approaches to communicating uncertainty in analysis results (Chapter 11), and documenting data processing and analysis steps to promote open science, reproducibility, and transparency (Chapters 3 and 14; Powers and Hampton (2019)).
1.2.4 Penobscot Experimental Forest inventory and LiDAR dataset
Coupling forest inventory with remotely sensed datasets using regression models offers an attractive approach to mapping forest characteristics at stand, regional, continental, and global scales. Light Detection and Ranging (LiDAR) data have shown great potential for use in estimating spatially explicit forest variables over a range of geographic scales (Asner et al. 2009; Næsset 2011; Babcock et al. 2013; Neigh et al. 2013; Finley et al. 2017). Encouraging results from these and many other studies have spurred massive investment in LiDAR sensors, sensor platforms, as well as extensive campaigns to collect field-based calibration data.
Much of the interest in LiDAR-based forest mapping is to support carbon monitoring, reporting, and verification (MRV) initiatives, such as those defined by the United Nations Programme on Reducing Emissions from Deforestation and Forest Degradation (UN-REDD (2008)) and Paris Agreement (UNFCCC (2015)). In these, and similar initiatives, aboveground biomass (AGB) is a primary forest characteristic of interest because it provides a nearly direct measure of forest carbon, i.e., carbon comprises ~50% of wood biomass (West 2004). Most efforts to quantify and/or manage forest ecosystem services (e.g., carbon, biodiversity, water) seek high spatial resolution wall-to-wall data products such as gridded maps with associated measures of uncertainty. In fact the international initiatives noted above include language concerning the level of spatially explicit acceptable error in total forest carbon estimates.
A long-term monitoring effort on the Penobscot Experimental Forest (PEF) in Bradley and Eddington, Maine, provides valuable data to assess methods to quantify forest biomass change over time. The dataset comprises LiDAR-based canopy height estimates from the Goddard’s LiDAR, Hyperspectral and Thermal Imager (G-LiHT) collected in Summer 2017 and AGB measurements on a set of georeferenced forest inventory plots. The G-LiHT LiDAR data were used to generate a canopy height surface (meters above the ground) at a fine resolution over the PEF. We obtained the forest inventory data associated with each plot from the PEF’s database of several on-going, long-term silvicultural experiments described by Kenefic et al. (2015). These silvicultural experiments are conducted in a series of management units (MUs) that partition the PEF.
| MU | Plot | Inventory | Month | Year | Species code | DBH (in) | AGB (lbs) |
|---|---|---|---|---|---|---|---|
| 22 | 41 | 7 | 6 | 1988 | 1 | 0.7 | 0.7 |
| 12 | 33 | 12 | 10 | 1994 | 1 | 2.4 | 15.5 |
| 28 | 42 | 10 | 7 | 1996 | 6 | 12.8 | 983.8 |
| 16 | 53 | 19 | 7 | 2002 | 6 | 6.4 | 176.2 |
| 27 | 63 | 10 | 8 | 1996 | 1 | 1.1 | 2.2 |
| 20 | 44 | 7 | 6 | 1977 | 15 | 0.9 | 2.3 |
| 90 | 12 | 22 | 6 | 2005 | 1 | 1.6 | 5.6 |
| 24 | 41 | 10 | 7 | 1995 | 6 | 1.4 | 4.1 |
| 29B | 12 | 11 | 8 | 1997 | 4 | 0.7 | 1.1 |
| 20 | 22 | 9 | 6 | 1986 | 4 | 13.2 | 997.4 |
Like the FACE data described in Section 1.2.3, the PEF data are longitudinal data that comprise repeated measurements on trees within permanent sample plots (PSPs). These periodic remeasurements are part of several PEF experiments designed to improve understanding of forest response to silvicultural treatments. Within a given MU the plot remeasurements are indexed by consecutive inventory numbers. These inventory numbers are used instead of dates because sometimes multiple inventories occur within the same year and month. Table 1.4 shows 10 trees selected at random from the 316,837 tree measurements in the PEF dataset. Following the column labels, the first row describes a 0.7 in DBH tree that weighs 0.7 lbs measured on plot 41 in management unit 22 in inventory 7 conducted in 6/1988.
FIGURE 1.4: Change in plot-level aboveground biomass (AGB) within a subset of the PEF management units. Each line traces a permanent sample plot’s AGB over time.
Figure 1.4 illustrates how plot-level AGB changed over time within a subset of MUs. As part of the experimental design, some MUs were not harvested (MU 10 and 32B), others were prescribed periodic light harvests (MU 12 and 20),and some received single heavy harvests (MU 15, 22, and 8). From an analyst’s perspective, this figure is helpful for revealing that some plots were not measured for all inventories within a MU, e.g., it appears that plots were added in the mid-1990s to MU 32A and 32B.
The HTML version of Figure 1.5 shows the PEF LiDAR canopy height surface, forest inventory plot locations, and MU boundaries. Clicking on a plot shows the MU in which the plot resides, its identification number, and current basal area (ft\(^2\)/ac). Clicking in a MU polygon (i.e., between plots) brings up a figure of the MU’s basal area changed over time. The printed version Figure 1.5 shows the PEF MU boundaries and plot locations colored by most current basal area (ft\(^2\)/ac).
FIGURE 1.5: Penobscot Experimental Forest management unit boundaries and forest inventory plot locations colored by basal area.
We’ll revisit these data when practicing data aggregation approaches, e.g., how to move from tree-level measurements to plot-, MU-, and ultimately forest-level summaries. We’ll also exploit the longitudinal nature of these data (i.e., repeated measures over time) to explore monitoring methods designed to quantify change in forest characteristics. Finally, these data are particularly valuable for learning how remotely sensed data, such as LiDAR, can help inform forest inventory efforts and produce high-resolution spatially-explicit dynamic maps.
1.2.5 Harvard Forest tree census
In 2014, the Harvard Forest, located in Petersham, MA, in cooperation with the Smithsonian/CTFS Forest Global Earth Observatory (ForestGEO) completed an initial mapping of a 35 ha (85 ac) forest dynamics monitoring plot. Every tree over 1 cm DBH in the plot was geolocated, tagged, and measured—83,801 trees in total. The Harvard Forest dataset is a census, or a complete enumeration of all units (e.g., trees) in a population. Such datasets are often referred to as stem maps because each tree is mapped with spatial coordinates.
Toward the end of 2020, field crews completed a second census of the Harvard Forest Global Earth Observatory, with 6,992 new woody stems mapped, tagged, and measured. Many of the longest-running research experiments on the Harvard Forest are in the ForestGEO plot footprint (Orwig, Foster, and Ellison 2015). These closely monitored trees, along with numerous sensors the Harvard Forest has installed to track forest canopy gas exchange, provide invaluable information about tree- to forest-level physiology.
A census of trees within a large area is a particularly rare and valuable resource for learning about and testing different forest mensuration methods. We often want to use measurements from a small number of trees to learn about forest-wide characteristics. A census allows us to test such methods by comparing their generalizations—based on a subset of the census data—to the true values of all trees in the census. Hence, we’ll use the Harvard Forest tree census to explore advantages and disadvantages of different sampling and statistical methods for forest inventory and monitoring (Chapter 13).
Figure 1.6 shows the location of all trees greater than 12.7 cm (5 in) measured in the first ForestGEO plot census with locations colored by tree family. The HTML map version allows you to click on a tree’s location to see its unique identification number, species, genus, family, and DBH.
FIGURE 1.6: Stem map of all trees larger than 12.7 cm (5 in) from the first Harvard Forest tree census.
1.3 How to learn: The most important section in this book!
There are several ways to engage with the content in this book. One way is not to engage at all. Leave the book closed on your shelf and do something else with your time. That might or might not be a good strategy, depending on what else you do with your time, but you won’t learn much from this book!
Another way is to read the book “passively” in a comfortable chair away from your computer. With this strategy you’ll probably learn more than if you leave the book closed.
A third option is to read the book while you’re at your computer, enter the R commands as you read about them, and work through the chapter exercises. You’ll likely learn more this way.
A fourth strategy is even better. In addition to reading, entering the commands, and working through the practice exercises, you think about what you’re doing, and ask yourself questions (which you then go on to answer). For example, after working through some R code computing the logarithm of positive numbers you might ask yourself, “What would R do if I asked it to calculate the logarithm of a negative number? What would R do if I asked it to calculate the logarithm of a really large number such as one trillion?” You can explore these questions easily by just trying things out in the R console. As you read, you’ll notice we occasionally use R functions without sufficient or any explanation. In such cases, an “active” learning strategy is to use the help resources we suggest, read the function’s manual page, then test its behavior with your own example data.
If your goal is to maximize the time you have to binge-watch a Netflix series, the first strategy might be optimal. But if your goal is to learn a lot about computing and analysis that will form a sound foundation upon which to build a new skill set or even career, the fourth strategy is probably going to be best.
References
Syntax is the set of rules that defines combinations of symbols to structure statements or expressions in a computer language.↩︎
The FEF metadata are available at https://www.fs.usda.gov/rds/archive/Catalog/RDS-2016-0016.↩︎
To be consistent with tree weight measurements, we changed the DBH measurement units from English to metric system.↩︎