R Views
https://rviews.rstudio.com/
Recent content on R ViewsHugo -- gohugo.ioen-usThu, 14 Oct 2021 00:00:00 +0000FDA and the Dynamics of Curves
https://rviews.rstudio.com/2021/10/14/fda-and-the-dynamics-of-curves/
Thu, 14 Oct 2021 00:00:00 +0000https://rviews.rstudio.com/2021/10/14/fda-and-the-dynamics-of-curves/
<script src="/2021/10/14/fda-and-the-dynamics-of-curves/index_files/header-attrs/header-attrs.js"></script>
<p>An elegant application of Functional Data Analysis is to model longitudinal data as a curve and then study the curve’s dynamics. For example, in pharmacokinetics and other medical studies analyzing multiple measurements of drug or protein concentrations in blood samples, it may be interest to determine if the concentrations in subjects undergoing one type of treatment rise quicker than those undergoing an alternative treatment. In this post, I will generate some plausible fake data for measurements taken over time for two groups of subjects, use the techniques of Functional Data Analysis to represent these data as a continuous curve for each subject, and look at some of the dynamic properties of the curves, in particular their velocities and accelerations.</p>
<div id="synthetic-data" class="section level3">
<h3>Synthetic Data</h3>
<p>The <a href="https://cran.r-project.org/package=fdaoutlier">fdaoutlier</a> package contains functions to generate a number of stochastic models with a mechanism to generate reasonable outliers for each type of model. THe curves produced by model 4 look like they can serve plausible synthetic concentration curves. Instead of thinking of normal curves and outliers, I imagine the two related sets of curves to be the results of two different treatments influencing some measured concentration curve. In the example below, the curves associated with <strong>treatment 2</strong> are of the form: <span class="math display">\[X_i(t) = \mu t(1 - t)^m + e_i(t),\]</span>
while those associated with <strong>treatment 1</strong> are of the form:
<span class="math display">\[X_i(t) = \mu(1 - t)t^m + e_i(t)\]</span> where:</p>
<ul>
<li><span class="math inline">\(t\in [0,1]\)</span>,</li>
<li><span class="math inline">\(e_i(t)\)</span> is a Gaussian process with zero mean and covariance function of the form: <span class="math display">\[\gamma(s,t) = \alpha\exp\{-\beta|t-s|^\nu\},\]</span></li>
<li><span class="math inline">\(m\)</span> is a constant</li>
<li><span class="math inline">\(\mu\)</span> is the mean value</li>
<li><span class="math inline">\(\alpha\)</span>, <span class="math inline">\(\beta\)</span>, and <span class="math inline">\(\nu\)</span> are coefficients in the covariance function</li>
</ul>
<pre class="r"><code>library(fdaoutlier)
library(tidyverse)
library(fda)
library(gganimate)
library(gridExtra)</code></pre>
<p>The following code generates two different sets of longitudinal data from Model 4 which is described in the <a href="https://cran.r-project.org/web/packages/fdaoutlier/vignettes/simulation_models.html">vignette</a> to the <a href="https://cran.r-project.org/package=fdaoutlier">fdaoutlier</a> package.</p>
<pre class="r"><code>set.seed(95139)
n_curves <- 100
n_obs <- 50
mod4 <- simulation_model4(n = n_curves, p = n_obs, outlier_rate = .5, seed = 50, plot = FALSE)
index <- 1:n_curves
index1 <- mod4$true_outliers
# curves_mat is an n_curves x n_obs matrix
curves_mat <- mod4$data
treat <- rep(2,n_obs)
curves <- data.frame(index, treat, curves_mat)
curves <- curves %>% mutate(treat = if_else((index %in% index1),1,2))</code></pre>
<p>There are 50 curves for each treatment and 50 points for each curve. This is probably the simplest case possible, and it is good enough to show how to explore the dynamics of curves. However, if you work with real longitudinal data you know that things are rarely this simple. But please be assured that FDA can deal with considerably more complexity, including a variable number of measurements for each subject, different measurement times for each subject, and situations where you have far fewer than 50 points for each subject. I have explored some of these situations in previous posts. For example, look <a href="https://rviews.rstudio.com/2021/05/14/basic-fda-descriptive-statistics-with-r/">here</a> to see how to work with different time points, and <a href="https://rviews.rstudio.com/2021/07/08/exploratory-fda-with-sparse-data/">here</a> for some ideas for working with sparse data. The first link above also points to basic references that should help you to get started with your data.</p>
<p>Now, I reformat the data into a long form data frame and plot both sets of curves.</p>
<pre class="r"><code>time <- 1:n_obs
curves_l <- pivot_longer(curves, cols = !c("index", "treat"), names_to = "Xval") %>%
mutate(time = rep(time,100), .before = "treat", treat = as.factor(treat)) %>%
dplyr::select(-Xval)
p <- curves_l %>% ggplot(aes(time,value, color = treat)) +
geom_line(aes(group = index)) +
scale_color_manual(values=c("navy blue", "dark grey")) +
ggtitle("Model 4 Curves")
p</code></pre>
<p><img src="/2021/10/14/fda-and-the-dynamics-of-curves/index_files/figure-html/unnamed-chunk-3-1.png" width="672" /></p>
<p>So far we just have plots based on a relatively small number of data points. Nevertheless, our eyes extrapolate, and we imagine seeing two sets of continuous curves with one set clearly rising to its maximum value faster than the other set. But we really don’t have continuous curves yet, we just have points.</p>
<p>The next step is to invoke the mathematics of FDA to embed the points in an infinite dimensional vector space where the data for each subject is modeled by a continuous function (curve). Exactly how this happens is a bit involved, but the general idea is that we create a basis and then use the data and some techniques from the linear algebra of Hilbert spaces to estimate the time dependent coefficients that enable modeling each subject’s data points as a linear combination of the basis functions. So, from here on out we are not going to be working with the raw data anymore. We will be working with vector space models of the data. The upside is that we can now now have real curves (continuous function), and can calculate the values of the functions and other properties such as the first and second derivatives at any time point. If you are interested in the math, please have a look at the references listed at the bottom of my post on <a href="https://rviews.rstudio.com/2021/06/10/functional-pca-with-r/">Functional PCA</a>.</p>
</div>
<div id="modeling-the-data-as-functions-with-a-b-spline-basis" class="section level3">
<h3>Modeling the data as functions with a B-Spline Basis</h3>
<p>Here we use the function <code>fda::create.bspline.basis()</code> to create a B-Spline basis covering the interval of our observations. The plot of the curves above indicates that specifying a knot at every multiple of 5 ought to be adequate for representing our data. Note that using cubic splines <code>n_order</code> = 4 ensures that the splines will have continuous first and second derivatives at the knots.</p>
<pre class="r"><code>knots = c(seq(0,n_obs,5)) #Location of knots
n_knots = length(knots) #Number of knots
n_order = 4 # order of basis functions: for cubic b-splines: order = 3 + 1
n_basis = length(knots) + n_order - 2;
spline_basis = create.bspline.basis(rangeval = c(0,n_obs), nbasis = n_basis, norder = n_order)
#plot(spline_basis)</code></pre>
<p>The following code uses the function <code>fda::Data2fda()</code> and the <code>spline-basis</code> to convert the Model 4 data points into functions and produce one fda object for each of the two groups of treatments.</p>
<pre class="r"><code># df1 is an (n_curves/2) x (n_obs) matrix
df1 <- curves_mat[index1,] # data for treatment 1
index2 <- index[!(index %in% index1)]
df2 <- curves_mat[index2,] # data for treatment 2
# Use the b-spline basis to create represent the curves as vectors in the function space
df1_obj <- Data2fd(argvals = 1:n_obs, y = t(df1), basisobj = spline_basis, lambda = 0.5)
df2_obj <- Data2fd(argvals = 1:n_obs, y = t(df2), basisobj = spline_basis, lambda = 0.5)</code></pre>
</div>
<div id="computing-derivatives" class="section level3">
<h3>Computing Derivatives</h3>
<p>Here we evaluate each function along with its first and second derivatives at a finer time scale than we used to originally display our data.</p>
<pre class="r"><code>tfine <- seq(0,50,by=.5)
# Each matrix is 101 x 50 rows are different times, columns are curves
pos1 <- as.vector(eval.fd(tfine, df1_obj)); pos2 <- as.vector(eval.fd(tfine, df2_obj))
vel1 <- as.vector(eval.fd(tfine, df1_obj,1)); vel2 <- as.vector(eval.fd(tfine, df2_obj,1))
acc1 <- as.vector(eval.fd(tfine, df1_obj,2)); acc2 <- as.vector(eval.fd(tfine, df2_obj,2)) </code></pre>
<p>Note that the velocities and accelerations computed above were returned as matrices. We convert them to vectors and put them into a data frame for plotting.</p>
<pre class="r"><code>time <- rep(tfine,50)
id1 <- rep(1:50,each=101)
id2 <- rep(51:100,each=101)
derv1 <- data.frame(time, id1, pos1, vel1, acc1)
derv2 <- data.frame(time, id2, pos2, vel2, acc2)
pv1 <- derv1 %>% ggplot(aes(time,vel1,col=id1)) + geom_line(aes(group = id1)) + ggtitle("Velocity Treatment 1")
pv2 <- derv2 %>% ggplot(aes(time,vel2,col=id2)) + geom_line(aes(group = id2)) + ggtitle("Velocity Treatment 2")
pa1 <- derv1 %>% ggplot(aes(time,acc1,col=id1)) + geom_line(aes(group = id1)) +ggtitle("Acceleration Treatment 1")
pa2 <- derv2 %>% ggplot(aes(time,acc2,col=id2)) + geom_line(aes(group = id2)) + ggtitle("Acceleration Treatment 2")
grid.arrange(pv1, pa1, pv2, pa2, nrow = 2,ncol = 2, padding = unit(1, "line"))</code></pre>
<p><img src="/2021/10/14/fda-and-the-dynamics-of-curves/index_files/figure-html/unnamed-chunk-7-1.png" width="672" /></p>
<p>The velocities for treatment 1 head upwards before they decrease while the velocities for treatment 2 go straight down, and the curve up again. The accelerations for treatment 1 slope downward slightly while those of treatment two slope upward. So the plots indicate that the two sets of curves look like they behave differently.</p>
</div>
<div id="testing-for-differences" class="section level3">
<h3>Testing for Differences</h3>
<p>We can test for differences using the function <code>fda::tperm.fd()</code> which implements a resampling method to do pointwise t-tests. The functional data representing the curves for the two samples are combined in a single array and the labels for the curves are randomly shuffled. Recalculating the maximum value of the t-statistic for each point enables computing a null distribution. Then, at each time point, the observed data are compared with the 1 - <span class="math inline">\(\alpha\)</span> quantile of the null distribution.</p>
<p>The plot below shows the result of performing the t-test to compare the first derivatives of the two treatments. I use the function <code>fda.deriv.fd()</code> to calculate the first derivative for each treatment on the fly, just to show another way of doing things in the <code>fda</code> package. You can easily modify the code to compare accelerations.</p>
<pre class="r"><code>dfdt1 <- deriv.fd(df1_obj,1)
dfdt2 <- deriv.fd(df2_obj,1)
tres <- tperm.fd(dfdt1,dfdt2,plotres=FALSE)
max_q <- tres$qval
tres_dat <-tibble(time = tres$argvals, t_values = tres$Tvals,
q_vals = tres$qvals.pts)
p <- tres_dat %>% ggplot(aes(time,t_values, colour = "t_value")) + geom_line() +
geom_line(aes(time, q_vals, colour = "q_vals")) +
geom_line(aes(time, max_q,colour = "max_q"), linetype= "dashed") +
labs(x = "time", y = "") +
ggtitle("Statistics for Pointwise t-test")
p</code></pre>
<p><img src="/2021/10/14/fda-and-the-dynamics-of-curves/index_files/figure-html/unnamed-chunk-8-1.png" width="672" /></p>
<p>The blue curve shows the t-statistic for the observed values. The green curve represents the 95% quantiles, and the dashed red line is the 95% quantile of the maximum of null distribution t-statistics. The t-test confirms that the derivatives are indeed different except in the regions of overlap around time = 10 and time = 40.</p>
</div>
<div id="examine-phase-plots" class="section level3">
<h3>Examine Phase Plots</h3>
<p>Phase plots are useful for evaluating how the curves develop in the abstract <em>phase space</em> created by looking at position versus velocity, or velocity versus acceleration. Here we pick three curves from each treatment and plot velocity versus acceleration.</p>
<pre class="r"><code>phase1 <- derv1 %>% filter(id1 %in% 15:17)
phase2 <- derv2 %>% filter(id2 %in% 51:53)
pph1 <- phase1 %>% ggplot(aes(vel1,acc1,col=id1)) + geom_point() + ggtitle("Treat 1 v vs. acc")
pph2 <- phase2 %>% ggplot(aes(vel2,acc2,col=id2)) + geom_point() + ggtitle("Treat 2 v vs. acc")
grid.arrange(pph1, pph2, ncol = 2, padding = unit(1, "line"))</code></pre>
<p><img src="/2021/10/14/fda-and-the-dynamics-of-curves/index_files/figure-html/unnamed-chunk-9-1.png" width="672" />
Based on this small sample, it certainly appears that the curves associated with the two different treatments inhabit different regions of phase space.</p>
<p>Finally, it is easy and helpful to animate a phase diagram to show how a function develops in phase space over time. The animation below shows the first curve for treatment 1.</p>
<pre class="r"><code>pos <- eval.fd(tfine, df2_obj[1])
vel <- eval.fd(tfine, df2_obj[1],1)
acc <- eval.fd(tfine, df2_obj[1],2)
phase_dat <- tibble(tfine, vel, acc)
p <- ggplot(phase_dat, aes(vel,acc)) + geom_point() + ggtitle("Trajectory in Phase Space")
anim <- p + transition_time(tfine) + shadow_mark()
anim_save("anim.gif", anim)
#anim</code></pre>
<p><img src="anim.gif" /></p>
</div>
<script>window.location.href='https://rviews.rstudio.com/2021/10/14/fda-and-the-dynamics-of-curves/';</script>
August 2021: "Top 40" New CRAN Packages
https://rviews.rstudio.com/2021/09/27/august-2021-top-40-new-cran-packages/
Mon, 27 Sep 2021 00:00:00 +0000https://rviews.rstudio.com/2021/09/27/august-2021-top-40-new-cran-packages/
<p>One hundred sixty new packages covering a wide array of topics made it to CRAN in August. I thought I would emphasize the breadth of topics by expanding the number of categories organizing my “Top 40” selections beyond core categories that appear month after month. Here are my picks in fourteen categories: Archaeology, Computational Methods, Data, Education, Finance, Forestry, Genomics, Machine Learning, Medicine, Science, Statistics, Time Series, Utilities, and Visualization. Based on informal impressions formed over the last several months, I believe a new category combining applications in forestry, animal populations, climate change could become a regular core category.</p>
<h3 id="archaeology">Archaeology</h3>
<p><a href="https://cran.r-project.org/package=DIGSS">DIGSS</a> v1.0.2: Provides a simulation tool to estimate the rate of success that surveys including user-specific characteristics have in identifying archaeological sites given specific parameters of survey area, survey methods, and site properties. See <a href="https://www.cambridge.org/core/journals/american-antiquity/article/abs/effectiveness-of-subsurface-testing-a-simulation-approach/B667DE186230F25072CA7B2F002783A7">Kintigh (1988)</a> for background and the <a href="https://cran.r-project.org/web/packages/DIGSS/vignettes/DIGSS.html">vignette</a> for examples.</p>
<p><img src="DIGSS.png" height = "400" width="400" alt="Example of a field map with artifacts plotted"></p>
<h3 id="computational-methods">Computational Methods</h3>
<p><a href="https://cran.r-project.org/package=simlandr">simlandr</a> v0.1.1: Provides a set of tools for constructing potential landscapes for dynamical systems using Monte-Carlo simulation which is especially suitable for formal psychological models. There are vignettes on <a href="https://cran.r-project.org/web/packages/simlandr/vignettes/simulation.html">Dynamic Models and Simulations</a>, <a href="https://cran.r-project.org/web/packages/simlandr/vignettes/landscape.html">Constructing Potential Landscapes</a>, and <a href="https://cran.r-project.org/web/packages/simlandr/vignettes/barrier.html">Calculating the Lowest Elivation Path</a>.</p>
<p><img src="simlandr.png" height = "400" width="400" alt="Barrier Simulation Plot"></p>
<h3 id="data">Data</h3>
<p><a href="https://cran.r-project.org/package=metaboData">metaboData</a> v0.6.2: Provides access to remotely stored <a href="https://github.com/aberHRML/metaboData/releases">data sets</a> from a variety of biological sample matrices analyzed using mass spectrometry metabolomic analytical techniques. See the <a href="https://cran.r-project.org/web/packages/metaboData/vignettes/metaboData.html">vignette</a>.</p>
<p><a href="https://cran.r-project.org/package=metadat">metadat</a> v1.0-0: Contains a collection of data sets useful for teaching meta analysis. See <a href="https://cran.r-project.org/web/packages/metadat/readme/README.html">README</a> for more information.</p>
<p><a href="https://cran.r-project.org/package=nflreadr">nflreadr</a> v1.1.0: Provides functions for downloading data from the GitHub repository for the <a href="https://github.com/nflverse">nflverse project</a>. There is a brief <a href="https://cran.r-project.org/web/packages/nflreadr/vignettes/exporting_nflreadr.html">Introduction</a> and several short vignettes that serve as the data dictionary for the various files <a href="https://cran.r-project.org/web/packages/nflreadr/vignettes/dictionary_draft_picks.html">Draft Picks</a>, <a href="https://cran.r-project.org/web/packages/nflreadr/vignettes/dictionary_ff_rankings.html">Rankings</a>, etc.</p>
<p><a href="https://cran.r-project.org/package=OCSdata">OCSdata</a> v1.0.2: Provides functions to access and download data from the <a href="https://www.opencasestudies.org/">Open Case Studies</a> repositories on <a href="https://github.com/opencasestudies">GitHub</a>. See the <a href="https://cran.r-project.org/web/packages/OCSdata/vignettes/instructions.html">vignette</a> to get started.</p>
<p><a href="https://cran.r-project.org/package=rATTAINS">rATTAINS</a> v0.1.2: Implements an interface to United States Environmental Protection Agency (EPA) <a href="https://www.epa.gov/waterdata/attains">ATTAINS</a> database used to track information provided by states about water quality assessments conducted under federal Clean Water Act requirements. There is a <a href="https://cran.r-project.org/web/packages/rATTAINS/vignettes/Introduction.html">vignette</a>.</p>
<p><a href="https://cran.r-project.org/package=taylor">taylor</a> v0.2.1: Provides access to a curated data set of Taylor Swift songs, including lyrics and audio characteristics. Data comes <a href="https://genius.com/artists/Taylor-swift">Genius</a> and the <a href="https://open.spotify.com/artist/06HL4z0CvFAxyc27GXpf02">Spotify</a> API. See <a href="https://cran.r-project.org/web/packages/taylor/readme/README.html">README</a> for examples,</p>
<p><img src="taylor.gif" height = "300" width="300" alt="Apple Music gif of Taylor Swify"></p>
<h3 id="education">Education</h3>
<p><a href="https://CRAN.R-project.org/package=karel">karel</a> v0.1.0: Provides an R implementation of Karel the robot, a programming language for teaching introductory concepts about general programming in an interactive and fun way, by writing programs to make Karel achieve tasks in the world she lives in. There are several vignettes including one on <a href="https://cran.r-project.org/web/packages/karel/vignettes/control_es_4.html">Control Structures</a> and another on <a href="https://cran.r-project.org/web/packages/karel/vignettes/descomp_es_3.html">Algorithmic Decomposition</a>.</p>
<p><img src="karel.gif" height = "300" width="500" alt="Gif of karel the robot moving along"></p>
<p><a href="https://cran.r-project.org/package=roger">roger</a> v0.99-0: Implements tools for grading the coding style and documentation of R scripts. This is the R component of <a href="https://gitlab.com/roger-project">Roger the Omni Grader</a>, an automated grading system for computer programming projects based on Unix shell scripts. Look <a href="https://roger-project.gitlab.io/">here</a> for more information.</p>
<h3 id="finance">Finance</h3>
<p><a href="https://cran.r-project.org/package=dispositionEffect">dispositionEffect</a> v1.0.0: Implements four different methodologies to evaluate the presence of the <a href="https://en.wikipedia.org/wiki/Disposition_effect">disposition effect</a> and other irrational investor behaviors based on investor transactions and financial market data. There is a <a href="https://cran.r-project.org/web/packages/dispositionEffect/vignettes/getting-started.html">Getting Started Guide</a>, and vignettes on <a href="https://cran.r-project.org/web/packages/dispositionEffect/vignettes/de-analysis.html">Analysis</a>, <a href="https://cran.r-project.org/web/packages/dispositionEffect/vignettes/de-parallel.html">Disposition Effects in Parallel</a>, and <a href="https://cran.r-project.org/web/packages/dispositionEffect/vignettes/de-timeseries.html">Time Series Disposition Effects</a>.</p>
<p><img src="dispositionEffect.png" height = "300" width="500" alt="Plot showing volatility and Disposition Effect"></p>
<p><a href="https://cran.r-project.org/package=HDShOP">HDShOP</a> v0.1.1: Provides functions to construct shrinkage estimators of high-dimensional mean-variance portfolios and performs high-dimensional tests on optimality of a given portfolio. See <a href="https://www.sciencedirect.com/science/article/abs/pii/S0377221717308494?via%3Dihub">Bodnar et al. (2018)</a>, <a href="https://ieeexplore.ieee.org/document/8767989">Bodnar et al. (2019)</a>, and <a href="https://ieeexplore.ieee.org/document/9258421">Bodnar et al. (2020)</a> for background.</p>
<p><a href="https://cran.r-project.org/package=tcsinvest">tcsinvest</a> v0.1.1: Implements an interface to the <a href="https://tinkoffcreditsystems.github.io/invest-openapi/">Tinkoff Investments API</a> which enables analysts and traders can interact with account and market data from within R. Clients for both REST and Streaming protocols have been implemented. There is a <a href="https://cran.r-project.org/web/packages/tcsinvest/vignettes/base.html">vignette</a>.</p>
<h3 id="forestry">Forestry</h3>
<p><a href="https://cran.r-project.org/package=APAtree">APAtree</a> v1.0.1: Provides functions to map the area potentially available (APA) using the approach from <a href="https://academic.oup.com/forestry/article/85/5/567/650068">Gspaltl et al. (2012)</a> and also aggregation functions to calculate stand characteristics based on APA-maps and the neighborhood diversity index as described in <a href="https://www.sciencedirect.com/science/article/pii/S1470160X2100738X?via%3Dihub">Glatthorn (2021)</a>. See the <a href="https://cran.r-project.org/web/packages/APAtree/vignettes/APAtree-vignette.html">vignette</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=efdm">efdm</a> v0.1.0: Implements the European Forestry Dynamics Model (<a href="https://ec.europa.eu/jrc/en/european-forestry-dynamics-model">EFDM</a>), a large-scale forest model that simulates the development of a forest and estimates volume of wood harvested for any given forested area. See <a href="https://op.europa.eu/en/publication-detail/-/publication/4715d130-0803-4e99-abed-915fec152c7b/language-en">Packalen et al. (2015)</a> for background and the <a href="https://cran.r-project.org/web/packages/efdm/vignettes/example.html">vignette</a> for examples.</p>
<h3 id="genomics">Genomics</h3>
<p><a href="https://cran.r-project.org/package=molnet">molnet</a> v0.1.0: Implements a network analysis pipeline that enables integrative analysis of multi-omics data including metabolomics. It allows for comparative conclusions between two different conditions, such as tumor subgroups, healthy vs. disease, or generally control vs. perturbed. The case study presented in the <a href="https://cran.r-project.org/web/packages/molnet/vignettes/Molnet_Vignette.html">vignette</a> uses data published by <a href="https://www.cell.com/cell/fulltext/S0092-8674(20)31400-8?_returnURL=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0092867420314008%3Fshowall%3Dtrue">Krug (2020)</a>.</p>
<p><img src="molnet.png" height = "300" width="500" alt="Illustration of network analysis pipeline"></p>
<p><a href="https://cran.r-project.org/package=simtrait">simtrait</a> v1.0.21P Provides functions to simulate complex traits given a SNP genotype matrix and model parameters with an emphasis on avoiding common biases due to the use of estimated allele frequencies. Traits can follow three models: random coefficients, fixed effect sizes, and multivariate normal. GWAS method benchmarking functions as described in <a href="https://www.biorxiv.org/content/10.1101/858399v1">Yao and Ochoa (2019)</a> are also provided. See the <a href="https://cran.r-project.org/web/packages/simtrait/vignettes/simtrait.html">vignette</a>.</p>
<p><img src="simtrait.png" height = "300" width="500" alt="Plot showing agreement of theoretical and RC kinship covariance matrices"></p>
<p><a href="https://cran.r-project.org/package=statgenIBD">statgenIBD</a> v1.0.1: Provides functions to calculate biparental, three and four-way crosses Identity by Descent (<a href="https://en.wikipedia.org/wiki/Identity_by_descent">IBD</a>) probabilities using Hidden Markov Models and inheritance vectors following <a href="https://www.jstor.org/stable/29713">Lander & Green (1987)</a> and <a href="https://www.pnas.org/content/108/11/4488">Huang (2011)</a>. See the <a href="https://cran.r-project.org/web/packages/statgenIBD/vignettes/IBDCalculations.html">vignette</a> for examples.</p>
<p><img src="statgenIBD.png" height = "300" width="500" alt="Plot of IBD probabilities"></p>
<h3 id="machine-learning">Machine Learning</h3>
<p><a href="https://cran.r-project.org/package=text2map">text2map</a> v0.1.0: Provides functions for computational text analysis for the social sciences including functions for working with word embeddings, text networks, and document-term matrices. For background on the methods used see <a href="https://link.springer.com/article/10.1007%2Fs42001-019-00048-6">Stoltz and Taylor (2019)</a>, <a href="https://link.springer.com/article/10.1007%2Fs42001-020-00075-8">Taylor and Stoltz (2020)</a>, <a href="https://sociologicalscience.com/articles-v7-23-544/">Taylor and Stoltz (2020)</a>, and <a href="https://www.sciencedirect.com/science/article/abs/pii/S0304422X21000504?via%3Dihub">Stoltz and Taylor (2021)</a>. There is a <a href="https://cran.r-project.org/web/packages/text2map/vignettes/CMDist-concept-movers-distance.html">Quick Start Guide</a> and a vignette on <a href="https://cran.r-project.org/web/packages/text2map/vignettes/concept-class-analysis.html">Concept Class Analysis</a>.</p>
<p><img src="text2map.png" height = "300" width="500" alt="Plot illustrating closeness of concepts"></p>
<p><a href="https://cran.r-project.org/package=NPRED">NPRED</a> v1.0.5: Uses partial informational correlation (PIC) to identify the meaningful predictors from a large set of potential predictors. Details can be found in <a href="https://agupubs.onlinelibrary.wiley.com/doi/full/10.1002/2013WR013845">Sharma & Mehrotra, (2014)</a>, <a href="https://www.sciencedirect.com/science/article/abs/pii/S1364815216301578?via%3Dihub">Sharma et al.(2016)</a>, and <a href="https://www.sciencedirect.com/science/article/abs/pii/S0309170805002137?via%3Dihub">Mehrotra & Sharma (2006)</a>. See the <a href="https://cran.r-project.org/web/packages/NPRED/vignettes/NPRED.html">vignette</a> for examples.</p>
<p><img src="NPRED.png" height = "300" width="500" alt="Illustration of using partial weights"></p>
<p><a href="https://cran.r-project.org/package=stabiliser">stabiliser</a> v0.1.0: Implements an approach to variable selection through stability selection and the use of an objective threshold based on permuted data. See <a href="https://www.nature.com/articles/s41598-020-79317-8">Lima et al (2021)</a> and <a href="https://rss.onlinelibrary.wiley.com/doi/10.1111/j.1467-9868.2010.00740.x">Meinshausen & Buhlmann (2010)</a> for details and the <a href="https://cran.r-project.org/web/packages/stabiliser/vignettes/stabiliser.html">vignette</a> for an example.</p>
<p><img src="stabiliser.png" height = "500" width="300" alt="Plot measuring stability of variables"></p>
<h3 id="medicine">Medicine</h3>
<p><a href="https://cran.r-project.org/package=dreamer">dreamer</a> v3.0.0: Fits longitudinal dose-response models utilizing a Bayesian model averaging approach as outlined in <a href="https://onlinelibrary.wiley.com/doi/10.1002/bimj">Gould (2019)</a> for both continuous and binary responses. See the <a href="https://cran.r-project.org/web/packages/dreamer/vignettes/dreamer.html">vignette</a>.</p>
<p><img src="dreamer.png" height = "300" width="500" alt="Plot from dreamer package"></p>
<p><a href="https://cran.r-project.org/package=smartDesign">smartDesign</a> v0.72: Implements the SMART trial design, as described by <a href="https://www.tandfonline.com/doi/abs/10.1080/19466315.2021.1883472?journalCode=usbr20">He et al. (2021)</a> which includes multiple stages of randomization where participants are randomized to an initial treatment in the first stage and then subsequently re-randomized between treatments in the following stage. There is a <a href="https://cran.r-project.org/web/packages/smartDesign/vignettes/DTR.html">Dynamic Treatment Tutorial</a> and a <a href="https://cran.r-project.org/web/packages/smartDesign/vignettes/SST.html">Sequential Design Tutorial</a>.</p>
<h3 id="science">Science</h3>
<p><a href="https://cran.r-project.org/package=bootf2">bootf2</a> v0.4.1: Provides functions to compare dissolution profiles with confidence intervals of the <a href="https://cran.r-project.org/web/packages/bootf2/vignettes/bootf2.html">similarity factor f2</a> and also functions to simulate dissolution profiles. There are multiple vignettes including and <a href="https://cran.r-project.org/web/packages/bootf2/vignettes/introduction.html">Introduction</a> a <a href="https://cran.r-project.org/web/packages/bootf2/vignettes/sim.dp.html">Simulation Example</a>.</p>
<p><img src="bootf2.png" height = "300" width="500" alt="Plot of dissolution profiles."></p>
<p><a href="https://cran.r-project.org/package=track2KBA">track2KBA</a> v1.0.1: Provides functions to prepare and analyze animal tracking data in order to identify areas of potential interest for population level conservation. See <a href="https://onlinelibrary.wiley.com/doi/10.1111/ddi.12411">Lascelles et al. (2016)</a> for background on the methodology employed and the <a href="https://cran.r-project.org/web/packages/track2KBA/vignettes/track2kba_workflow.html">vignette</a> for examples and workflow.</p>
<p><img src="track2KBA.png" height = "400" width="300" alt="Plot shows estimated minimum number of birds in space around breeding island."></p>
<h3 id="statistics">Statistics</h3>
<p><a href="https://cran.r-project.org/package=chyper">chyper</a> v0.3.1: Provides functions to work with the conditional hypergeometric distribution. See the <a href="https://cran.r-project.org/web/packages/chyper/vignettes/Guide.html">vignette</a>.</p>
<p><a href="https://cran.r-project.org/package=sprtt">sprtt</a> v0.1.0: Provides functions to perform sequential t-tests including those of <a href="https://academic.oup.com/sf/article-abstract/27/2/170/1991955">Wald (1947)</a>, <a href="https://www.jstor.org/stable/2332385?origin=crossref">Rushton (1950)</a>, <a href="https://www.jstor.org/stable/2334026?origin=crossref">Rushton (1952)</a>, and <a href="https://www.jstor.org/stable/2333131?origin=crossref">Hajnal (1961)</a>. There is an <a href="https://cran.r-project.org/web/packages/sprtt/vignettes/usage-sprtt.html">Introduction</a> to the package, a <a href="https://cran.r-project.org/web/packages/sprtt/vignettes/use-case.html">Use Case</a>, and a vignette on the <a href="https://cran.r-project.org/web/packages/sprtt/vignettes/sequential_testing.html">Sequential t-test</a>.</p>
<p><a href="https://cran.r-project.org/package=SurvMetrics">SurvMetrics</a> v0.3.5: Implements popular evaluation metrics commonly used in survival prediction including Concordance Index, Brier Score, Integrated Brier Score, Integrated Square Error, Integrated Absolute Error and Mean Absolute Error. For detailed information, see <a href="https://projecteuclid.org/journals/annals-of-applied-statistics/volume-2/issue-3/Random-survival-forests/10.1214/08-AOAS169.full">Ishwaran et al. (2008)</a> and <a href="https://link.springer.com/article/10.1007%2Fs10985-016-9372-1">Moradian et al. (2017)</a>. The <a href="https://cran.r-project.org/web/packages/SurvMetrics/vignettes/SurvMetrics-vignette.html">vignette</a> offers examples.</p>
<p><img src="SurvMetrics.png" height = "400" width="300" alt="Boxplot comparing models"></p>
<h3 id="time-series">Time Series</h3>
<p><a href="https://cran.r-project.org/package=DCSmooth">DCSmooth</a> v1.0.2: Implements nonparametric smoothing techniques for data on a lattice or functional time series which allow for modeling a dependency structure of the error terms of the nonparametric regression model. See <a href="https://www.tandfonline.com/doi/abs/10.1198/106186002420">Beran & Feng (2002)</a>, <a href="https://www.jstor.org/stable/2533197?origin=crossref">Mueller & Wang (1994)</a>, <a href="https://ideas.repec.org/p/pdn/ciepap/144.html">Feng & Schaefer (2021)</a>, and <a href="https://ideas.repec.org/p/pdn/ciepap/143.html">Schaefer & Feng (2021)</a> for the background and the <a href="https://cran.r-project.org/web/packages/DCSmooth/vignettes/DCSmooth.html">vignette</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=STFTS">STFTS</a> v0.1.0: Implements statistical hypothesis tests of functional time series including a functional stationarity test, a functional trend stationarity test and a functional unit root test.</p>
<p><a href="https://cran.r-project.org/package=WASP">WASP</a> v1.4.1: Implements wavelet-based variance transformation methods for system modeling and prediction. For details see <a href="https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2019WR026962">Jiang et al. (2020)</a>, <a href="https://www.sciencedirect.com/science/article/abs/pii/S1364815220309646?via%3Dihub">Jiang et al. (2020)</a>, and <a href="https://www.sciencedirect.com/science/article/abs/pii/S0022169421008660?via%3Dihub">Jiag et al. (2021)</a> There is a <a href="https://cran.r-project.org/web/packages/WASP/vignettes/WASP.html">vignette</a> with examples.</p>
<p><img src="WASP.png" height = "300" width="500" alt="Plot showing Daubechies wavelets"></p>
<h3 id="utilities">Utilities</h3>
<p><a href="https://cran.r-project.org/package=ExpImage">ExpImage</a> v0.2.0: Provides an image editing tool for researchers which includes functions for segmentation and for obtaining biometric measurements. There are several vignettes including: <a href="https://cran.r-project.org/web/packages/ExpImage/vignettes/Contagem_de_bovinos.html">Contagem de bovinos</a>, <a href="https://cran.r-project.org/web/packages/ExpImage/vignettes/Contagem_de_objetos.html">Contagem de objetos</a>, and <a href="https://cran.r-project.org/web/packages/ExpImage/vignettes/Edicao_de_imagens.html">Como editar imagens</a>.</p>
<p><img src="ExpImage.png" height = "300" width="300" alt="Image of leaf with seeds to be counted"></p>
<p><a href="https://cran.r-project.org/package=meltr">meltr</a> v1.0.0: Provides functions to read non-rectangular data, such as ragged forms of csv (comma-separated values), tsv (tab-separated values), and fwf (fixed-width format) files. See <a href="https://cran.r-project.org/web/packages/meltr/readme/README.html">README</a> to get started.</p>
<p><a href="https://cran.r-project.org/package=plumbertableau">plumbertableau</a> v0.1.0: Implements tools for building <code>plumber</code> APIs that can be used in <a href="https://www.tableau.com/">Tableau</a> workbooks. There is a package <a href="https://cran.r-project.org/web/packages/plumbertableau/vignettes/introduction.html">Introduction</a> and vignettes on <a href="https://cran.r-project.org/web/packages/plumbertableau/vignettes/r-developer-guide.html">Writing Extensions</a>, <a href="https://cran.r-project.org/web/packages/plumbertableau/vignettes/tableau-developer-guide.html">Using Extensions in Tableau</a>, and <a href="https://cran.r-project.org/web/packages/plumbertableau/vignettes/publishing-extensions.html">Publishing Extensions to RStudio Connect</a>.</p>
<p><a href="https://cran.r-project.org/package=string2path">string2path</a> v0.0.2: Provides functions to extract glyph information from a font file, translate the outline curves to flattened paths or tessellated polygons, and return the results as a <code>data.frame</code>. See <a href="https://cran.r-project.org/web/packages/string2path/readme/README.html">README</a> for an example.</p>
<p><img src="string2path.png" height = "300" width="500" alt="Japanese kana and kanji as glyphs on an x-y grid"></p>
<p><a href="https://cran.r-project.org/package=trackdown">trackdown</a> v1.0.0: Uses <a href="https://www.google.com/drive/">Googel Drive</a> to implement tools for collaborative writing and editing of R Markdown and Sweave documents. There are some <a href="https://cran.r-project.org/web/packages/trackdown/vignettes/trackdown-tech-notes.html">Tech Notes</a> and vignettes on <a href="https://cran.r-project.org/web/packages/trackdown/vignettes/trackdown-features.html">Features</a> and <a href="https://cran.r-project.org/web/packages/trackdown/vignettes/trackdown-workflow.html">Workflow</a>.</p>
<h3 id="visualization">Visualization</h3>
<p><a href="https://cran.r-project.org/package=aRtsy">aRtsy</a> v0.1.1: Provides algorithms for creating artwork in the <code>ggplot2</code> language that incorporate some form of randomness. See <a href="https://cran.r-project.org/web/packages/aRtsy/readme/README.html">README</a> for examples and package use.</p>
<p><img src="aRtsy.png" height = "200" width="400" alt="aRtsy generated abstract art"></p>
<p><a href="https://cran.r-project.org/package=ggcleveland">ggcleveland</a> v0.1.0: Provides functions to produce <code>ggplot2</code> versions of the visualization tools described in William Cleveland’s book <a href="https://www.amazon.com/Visualizing-Data-William-S-Cleveland/dp/0963488406/ref=sr_1_3?dchild=1&keywords=Visualizing+Data+cleveland&qid=1632504146&sr=8-3"><em>Visualizing Data</em></a>. The <a href="https://cran.r-project.org/web/packages/ggcleveland/vignettes/ggplot-cleveland.html">vignette</a> contains several examples.</p>
<p><img src="ggcleveland.png" height = "200" width="400" alt="William Cleveland inspired qqplots"></p>
<p><a href="https://cran.r-project.org/package=ggtikz">ggtikz</a> v0.0.1: Provides tools to annotate <code>ggplot2</code> plots with <a href="https://www.overleaf.com/learn/latex/TikZ_package">TikZ</a> code using absolute data or relative coordinates. See the <a href="https://cran.r-project.org/web/packages/ggtikz/vignettes/examples.pdf">vignette</a>.</p>
<p><img src="ggtikz.png" height = "300" width="300" alt="Scatter plot annotated with text and lines"></p>
<p><a href="https://cran.r-project.org/package=tidycharts">tidycharts</a> v0.1.2: Provides functions to generate charts compliant with the International Business Communication Standards (<a href="https://www.ibcs.com/">IBCS</a>) including unified bar widths, colors, chart sizes, etc. There is a <a href="https://cran.r-project.org/web/packages/tidycharts/vignettes/Getting_Started.html">Getting Started</a> guide and vignettes on <a href="https://cran.r-project.org/web/packages/tidycharts/vignettes/EDA-for-palmer-penguins-data-set.html">EDA</a>, <a href="https://cran.r-project.org/web/packages/tidycharts/vignettes/customize-package.html">Customization</a>, and <a href="https://cran.r-project.org/web/packages/tidycharts/vignettes/join_charts.html">Joining Charts</a>.</p>
<p><img src="tidycharts.png" height = "300" width="500" alt="tidycharts IBCS compliant histogram"></p>
<script>window.location.href='https://rviews.rstudio.com/2021/09/27/august-2021-top-40-new-cran-packages/';</script>
A Guide to Binge Watching R / Medicine 2021
https://rviews.rstudio.com/2021/09/09/a-guide-to-binge-watching-r-medicine/
Thu, 09 Sep 2021 00:00:00 +0000https://rviews.rstudio.com/2021/09/09/a-guide-to-binge-watching-r-medicine/
<p><a href="https://r-medicine.org/">R / Medicine</a> is a big deal. This year, the conference grew by 13% with 665 people from over 60 countries signing up for the virtual event which was held last month. 34% percent of the registrants were from outside of the United States and 17% identified as physicians.</p>
<p><img src="rmed.png" height = "300" width="500" alt="Global map with locations of R Medicine registrants indicated"></p>
<p>The conference is now an established international event where experts report on the advanced use of the R language, Machine Learning, and statistical analysis, and discuss the successes and challenges associated with bringing these technologies to day-to-day medical practice.</p>
<p>Almost all of the talks, including keynotes, regular talks, lightning talks, pre-conference workshops and poster sessions are available online. <a href="https://r-medicine.org/schedule/">Find the links</a> on the R / Medicine site or look through the <a href="https://www.youtube.com/playlist?list=PL4IzsxWztPdmHxCpS_c2l_jbMfrywWciZ">playlist </a> on the <a href="https://www.r-consortium.org/">R Consortium Youtube</a> Channel. Note that the posters can be viewed by going to the <a href="https://spatial.chat/s/R-Medicine2021?room=231308">conference spatial.chat site</a>. (If you and a friend visit at the same time you should be able to “walk around” the posters and chat about what you see.)</p>
<p>To kick off an evening of binge watching the conference I would begin with the keynotes.</p>
<h3 id="the-keynotes">The Keynotes</h3>
<p><a href="https://medicine.umich.edu/dept/lhs/karandeep-singh-md-mmsc">Dr. Karandeep Singh</a> sets the hook for his talk, <a href="https://www.youtube.com/watch?v=l71wLKUr26E&list=PL4IzsxWztPdmHxCpS_c2l_jbMfrywWciZ&index=7">Bringing Machine Learning Models to the Bedside at Scale</a>, two minutes into the video when he asks:</p>
<blockquote>
<p>Who are the twenty sickest patients in the hospital right now who are not in the ICU?</p>
</blockquote>
<p>This straightforward question immediately gets to the promise and the problems of introducing large scale machine learning algorithms into the hospital, and indicates how medical practice interacts with big money questions about allocating resources. Both physicians and administrators would like to identify high risk patients and treat them proactively while being able to confidently spend less on unnecessary test for low risk patients. About (5:10) into the talk, Karandeep begins discussing the challenges associated with introducing machine learning models.</p>
<p><img src="chal.png" height = "300" width="500" alt="Slide with list of challenges discussed. Is there infrastructure to support models? Should we implement a model? Once implemented, how do we measure model performance? Is a model “good enough” to use? Do users agree on how to use the model? Is the model effective when used? What does governance look like for machine learning models?"></p>
<p>In the remainder of the talk he describes the technical infrastructure and then the governance or “social infrastructure” needed for success.</p>
<p>If you enjoy a good detective story, and take pride in your ability to interpret a well-done statistical plot you are certainly going to want to watch <a href="http://ziadobermeyer.com/">Ziad Obermeyer’s</a> keynote <a href="https://www.youtube.com/watch?v=JfKYO1W4uuA&list=PL4IzsxWztPdmHxCpS_c2l_jbMfrywWciZ&index=27">Dissecting Algorithmic Bias</a>. About two minutes into the video Professor Obermeyer sets the stage with the warning:</p>
<blockquote>
<p>The single greatest threat to all of the gains that we can make in using algorithms in medicine is letting them go wrong in increasingly well known ways.</p>
</blockquote>
<p>and the observation that due to the focus of the US health care management on “high risk care management” an estimated 150 to 200 million Americans are sorted by algorithms every year. He goes on to work through a case study that illustrates how an algorithm built with good intentions had the effect of scaling up racial bias.</p>
<p><img src="bias.png" height = "300" width="600" alt="Dot plot with regression line of algorithm risk score versus realized cost to show the racial bias in high risk care management"></p>
<p>A second case study features an algorithm that “fights against” racial bias. Along the way, Ziad weaves two common themes into his presentation:</p>
<ol>
<li>So many of the ways that algorithms can go wrong come from training algorithms with the wrong target variables, often “convenient and tempting proxies”.</li>
<li>The necessity of follow-up work to fix underlying problems.</li>
</ol>
<p>In the remainder of this post, I have organized the talks into six categories that you may find helpful for setting your viewing program: Clinical Practice, Clinical Trials, Medical Data, R in Production, R Tools, and Short Courses. The majority of the talks have a machine learning angle. There is quite a bit of Shiny and several R packages, not all of them on CRAN, are featured. I have provided links when I could find them. I don’t want to spoil anyone’s fun in searching through the videos for “Easter Eggs”, but the <em>Reproducible Research with R</em> short course contains the first preview on the <a href="https://quarto.org/">Quarto</a> Publishing system in a talk from anyone at RStudio. (Note that the video needs some editing. Start watching at 9 minutes.)</p>
<h3 id="clinical-practice">Clinical Practice</h3>
<ul>
<li>Building an Interpretable ML Model API for Interpretation of CNVs in Patients with Rare Diseases - Francisco Requena</li>
<li>Subgroup Identification and Precision Medicine with the personalized R Package - Jared Huling</li>
<li>R and Shiny Dashboards to Facilitate Quality Improvement in Anesthesiology and Periopeartive Care - Robert Lobato</li>
<li><code>tidytof</code>: Predicting Patient Outcomes from Single-cell Data using Tidy Data Principles - Timothy Keyes</li>
<li>Assessing ML Model Performance in DIverse Populations and Across Time - Victor Castro, Roy Perlis</li>
</ul>
<h3 id="clinical-trials">Clinical Trials</h3>
<ul>
<li>Designing Early Phase Clinical Trials with <a href="https://github.com/zabore/ppseq"><code>ppseq</code></a> - Emily Zabor</li>
<li>Collaborative, Reproducible Exploration of Clinical Trial Data - Michael Kane</li>
<li>Graphical Displays in R for Clinical Trials - Steven Schwager</li>
<li><a href="https://www.gitmemory.com/presagia-analytics/ctrialsgov"><code>ctrialsgov</code></a>: Access, Visualization, and Discovery of the ClinicalTrials.gov Database - Taylor Arnold</li>
</ul>
<h3 id="medical-data">Medical Data</h3>
<ul>
<li>Scaling Up and Deploying Shiny and Text Mining for National Health Decisions - Andreas Soteriade, Chris Beeley</li>
<li>Mapping African Health Data with <a href="https://afrimapr.github.io/afrimapr.website/"><code>afrimapr</code></a> Package, Training & Community - Andy South</li>
<li>You R What You Measure: Digital Biomarkers for Insights in Personalized Health - Irene van den Broek</li>
<li>Shiny and REDCap for a Global Research Consortium - Judith Lewis, Stephany Duda</li>
<li>Diving into Registry Data: Using R for Large Norwegian Health Registries - Julia Romanowska</li>
<li><a href="https://cran.r-project.org/web/packages/ReviewR/index.html"><code>ReviewR</code></a>: A Shiny App for Reviewing Clinical Records - Laura Wiley, David Mayer</li>
<li><a href="https://cran.r-project.org/package=DOPE"><code>DOPE</code></a>: An R package for Processing and Classifying Drug Names - Layla Bouzoubaa</li>
<li><a href="https://cran.r-project.org/package=medicaldata"><code>medicaldata</code></a> for Teaching #Rstats - Peter Higgins</li>
<li>Stem Cell Transplant Outcomes Reporting using R/Shiny - Richard Hanna, Stephan Kadauke</li>
</ul>
<h3 id="r-in-production">R in Production</h3>
<ul>
<li>Second Server to the Right and Straight On ‘til Production: Deploying a GxP Shiny Application - Marcus Adams</li>
<li>Target Markdown and <a href="https://docs.ropensci.org/stantargets/"><code>stantargets</code></a> for Bayesian model validation pipelines - Will Landau</li>
<li><a href="https://www.themillerlab.io/publication/genetex/"><code>GENETEX</code></a>: A Genomics Report Text Mining R Package to Capture Real-world Clinico-genomic Data - David Miller, Sophia Shalhout</li>
</ul>
<h3 id="r-tools">R Tools</h3>
<ul>
<li>Generalized Additive Models for Longitudinal Biomedical Data - Ariel Mundo<br /></li>
<li>Multistate Data Using the <a href="https://cran.r-project.org/package=survival"><code>survival</code></a> Package - Beth Atkinson<br /></li>
<li>Bayesian Random-Effects Meta-analysis using <a href="https://cran.r-project.org/package=survival"><code>bayesmeta</code></a> - Christian Rover<br /></li>
<li>An <a href="https://cran.r-project.org/package=arsenal"><code>arsenal</code></a> of R Functions for Statistical Summaries - Ethan Heinzen, Beth Atkinson, Jason Sinnwell</li>
<li>R Markdown and <a href="https://cran.r-project.org/package=officedown"><code>officedown</code></a> to Automate Clinical Trial Reporting - Damian Rodziewicz</li>
<li>Creating and Styling PPTX Slides with <a href="https://cran.r-project.org/package=rmarkdown"><code>rmarkdown</code></a> - Emil Hvitfeldt</li>
<li><a href="https://github.com/ML4LHS/runway"><code>runway</code></a>: an R Package to Visualize Prediction Model Performance - Jie Cao, Karandeep Singh</li>
<li><a href="https://CRAN.R-project.org/package=clinspacy"><code>clinspacy</code></a>: An R package for Clinical Natural Language Processing - Jie Cao, Karandeep Singh</li>
<li>Data Visualization for Machine Learning Practitioners - Julie Silge</li>
<li>Animated Data Visualizations with <a href="https://CRAN.R-project.org/package=gganimate"><code>gganimate</code></a> for Science Communication during the Pandemic - Kristen Panthagani</li>
<li>Incorporating Risk-of-Bias Assessments into Evidence Syntheses with <a href="https://cran.r-project.org/package=robvis"><code>robvis</code></a> - Luke McGuinness, Randall Boyes, Alex Fowler</li>
<li>‘gpmodels’: A Grammar of Prediction Models - Sean Meyer, Karandeep Singh</li>
<li>CONSORT Diagrams in R with <a href="https://github.com/tgerke/ggconsort"><code>ggconsort</code></a> - Travis Gerke<br /></li>
</ul>
<h3 id="short-courses">Short Courses</h3>
<ul>
<li>Secure Medical Data Collection: Best Practices with Excel, and Leveling Up to REDCap and <a href="https://github.com/kamclean/collaborator"><code>CollaboratoR</code></a> - Peter Higgins, Will Beasley, Kenneth MacLean, Amanda Miller</li>
<li>Introduction to R for Medical Data - Ted Laderas, Daniel Chen, Mara Alexeev</li>
<li>An Introductory R Guide for Targeted Maximum Likelihood Estimation in Medical Research - Ehsan Karim, Hanna Frank</li>
<li>Mapping Spatial Health Data - Marynia Kolak, Susan Paykin</li>
<li>From SAS to R - Joe Krsszun<br /></li>
<li>Reproducible Research with R - Alison Hill, Stephan Kaduke, Paul Villanueva</li>
</ul>
<script>window.location.href='https://rviews.rstudio.com/2021/09/09/a-guide-to-binge-watching-r-medicine/';</script>
July 2021: "Top 40" New CRAN Packages
https://rviews.rstudio.com/2021/08/26/july-2021-top-40-new-cran-packages/
Thu, 26 Aug 2021 00:00:00 +0000https://rviews.rstudio.com/2021/08/26/july-2021-top-40-new-cran-packages/
<p>One hundred eighty-three new packages stuck to CRAN in July. Here are my “Top 40” picks in eleven categories: Data, Ecology, Finance, Genomics, Machine Learning, Medicine, Science, Statistics, Time Series, Utilities, and Visualization. Although I don’t have any formal specification for these categories, I do my best to main my subjective sense of consistency from month to month. Nevertheless, watching the monthly ebb and flow of the number of packages that fit into the various categories is interesting. This month, developers seemed to be focused on utilities. I classified forty-five packages as utilities this month.</p>
<h3 id="data">Data</h3>
<p><a href="https://cran.r-project.org/package=beans">beans</a> v0.1.0: Contains data on 13,611 beans from <a href="https://www.sciencedirect.com/science/article/abs/pii/S0168169919311573?via%3Dihub">Koklu and Ozkan (2020)</a>. The beans have been quantified using 16 morphologic image features and labeled with one of 6 values.</p>
<p><a href="https://CRAN.R-project.org/package=geckor">geckor</a> v0.1.1: Provides functions to collect current and historical cryptocurrency market data using the public <a href="https://www.coingecko.com/en/api">CoinGecko API</a>. There is a brief <a href="https://cran.r-project.org/web/packages/geckor/vignettes/supported-currencies-and-exchanges.html">overview</a> and vignettes on <a href="https://cran.r-project.org/web/packages/geckor/vignettes/current-data.html">Current</a> and <a href="https://cran.r-project.org/web/packages/geckor/vignettes/historical-data.html">Historical</a> market data.</p>
<p><a href="https://cran.r-project.org/package=ISLR2">ISLR2</a> v1.0: Contains the data sets used in the book <a href="https://www.statlearning.com/"><em>An Introduction to Statistical Learning with Applications in R, Second Edition</em></a>.</p>
<p><a href="https://cran.r-project.org/package=sageR">sageR</a> v0.3.0: Provides the data sets used in the book <em>Statistiques pour l’économie et la gestion, Théorie et applications en entreprise</em>. Look <a href="https://fbertran.github.io/sageR/">here</a> for descriptions of the data with code and plots.</p>
<p><img src="sageR.png" height = "500" width="500"></p>
<h3 id="ecology">Ecology</h3>
<p><a href="https://cran.r-project.org/package=DA">DA</a> v1.2.0: Provides functions for Discriminant Analysis (DA) for evolutionary inference, especially population genetic structure and community structure inference as described in <a href="https://www.authorea.com/users/304610/articles/460985-da-ecological-and-evolutionary-inference-using-supervised-discriminant-analysis?commit=668bd97318a7646fd432e4824228415ed209b692">Qin et al. (2020)</a>. See the <a href="https://cran.r-project.org/web/packages/DA/vignettes/DA.html">Introduction</a>.</p>
<p><img src="DA.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=Ostats">Ostats</a> v0.1.0: Provides functions to calculate O-statistics, or overlap statistics, which measure the degree of community-level trait overlap. See <a href="https://onlinelibrary.wiley.com/doi/10.1111/ecog.03641">Read et al. (2018)</a> for background. There is an <a href="https://cran.r-project.org/web/packages/Ostats/vignettes/Ostats-introduction.html">Introduction</a> and a vignette on <a href="https://cran.r-project.org/web/packages/Ostats/vignettes/Ostats-multivariate.html">Multivariate Ostats</a>.</p>
<p><img src="Ostats.png" height = "400" width="400"></p>
<h3 id="finance">Finance</h3>
<p><a href="https://cran.r-project.org/package=ffp">ffp</a> v0.1.0: Implements numerical entropy-pooling for scenario analysis as described in <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1696802">Meucci (2010)</a> See the <a href="https://cran.r-project.org/web/packages/ffp/vignettes/Replicating-Meucci-Paper.html">vignette</a>.</p>
<p><img src="ffp.png" height = "300" width="500"></p>
<h3 id="genomics">Genomics</h3>
<p><a href="https://cran.r-project.org/package=utr.annotation">utr.annotation</a> v1.0.4: Implements a fast tool for annotating potential deleterious variants in the untranslated regions for both human and mouse species. See <a href="https://www.biorxiv.org/content/10.1101/2021.06.23.449510v2">Liu & Dougherty (2021)</a> for background and the <a href="https://cran.r-project.org/web/packages/utr.annotation/vignettes/introduction.pdf">vignette</a> for and introduction and examples.</p>
<p><a href="https://cran.r-project.org/package=OmicNavigator">OmicNavigator</a> v1.4.3: Provides a for interactive exploration of the results from ‘omics’ experiments to facilitate novel discoveries from high-throughput biology. There is a <a href="https://cran.r-project.org/web/packages/OmicNavigator/vignettes/OmicNavigatorUsersGuide.pdf">User’s Guide</a> and a vignette on the package’s <a href="https://cran.r-project.org/web/packages/OmicNavigator/vignettes/OmicNavigatorAPI.pdf">API</a>.</p>
<h3 id="machine-learning">Machine Learning</h3>
<p><a href="https://cran.r-project.org/package=BOSO">BOSO</a> v1.0.3: Implements the BiLevel Optimization Selector Operator feature selection algorithm for linear regression as described in <a href="https://www.biorxiv.org/content/10.1101/2020.11.18.388579v1">Valcarcel et al. (2020)</a>. See the <a href="https://cran.r-project.org/web/packages/BOSO/vignettes/BOSO.htm">vignette</a> for examples.</p>
<p><img src="Boso.png" height = "400" width="500"></p>
<p><a href="https://cran.r-project.org/package=FeatureTerminatoR">FeatureTerminatoR</a> v1.0.0: Implements a feature selection engine that removes features with minimal predictive power. See <a href="https://link.springer.com/article/10.1007%2Fs40595-018-0107-y">Boughaci (2018)</a> for background and the <a href="https://cran.r-project.org/web/packages/FeatureTerminatoR/vignettes/feature_terminatoR_howto.html">vignette</a> for and example.</p>
<p><a href="https://cran.r-project.org/package=hhcartr">hhcartr</a> v1.0.0: Implements the HHCART-G algorithm described in <a href="https://onlinelibrary.wiley.com/doi/10.1111/anzs.12275">Wickramarachchi et al. (2019)</a>. See the <a href="https://cran.r-project.org/web/packages/hhcartr/vignettes/hhcartr.pdf">vignette</a>.</p>
<p><img src="hhcartr.png" height = "500" width="400"></p>
<p><a href="https://cran.r-project.org/package=recometrics">recometrics</a> v0.1.3: Implements evaluation metrics for implicit-feedback recommender systems based on low-rank matrix factorization models, given the fitted model matrices and data. See the <a href="https://cran.r-project.org/web/packages/recometrics/vignettes/Evaluating_recommender_systems.html">vignette</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=tidylda">tidylda</a> v0.0.1: Implements an algorithm for Latent Dirichlet Allocation (LDA) as described in <a href="https://www.jmlr.org/papers/volume3/blei03a/blei03a.pdf">Blei et at. (2003)</a> using <a href="https://www.tidyverse.org/">tidyverse</a> principles. See <a href="https://cran.r-project.org/web/packages/tidylda/readme/README.html">README</a> for an overview.</p>
<h3 id="medicine">Medicine</h3>
<p><a href="https://cran.r-project.org/package=dcurves">dcurves</a> v0.2.0: Implements a decision curve analysis method for evaluating and comparing prediction models that incorporates clinical consequences and requires only the data set on which the models are tested. See <a href="https://journals.sagepub.com/doi/10.1177/0272989X06295361">Vickers (2006)</a>, <a href="https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/1472-6947-8-53">Vickers (2008)</a>, and <a href="https://onlinelibrary.wiley.com/doi/10.1002/bimj.201800240">Pfeiffer (2020)</a> for background, and the <a href="https://cran.r-project.org/web/packages/dcurves/vignettes/dca.html">vignette</a> for examples.</p>
<p><img src="dcurves.png" height = "200" width="300"></p>
<p><a href="https://cran.r-project.org/package=KHQ">KHQ</a> v0.2.0: Provides methods to calculate scores for each dimension of the The King’s Health Questionnaire <a href="https://obgyn.onlinelibrary.wiley.com/doi/10.1111/j.1471-0528.1997.tb11006.x">(KHQ)</a> ; converts KHQ item scores to <a href="https://journals.sagepub.com/doi/10.1177/0272989X07301820">KHQ5D</a> scores; and also calculates the utility index of the KHQ5D. See the <a href="https://cran.r-project.org/web/packages/KHQ/vignettes/KHQ.html">vignette</a>.</p>
<h3 id="science">Science</h3>
<p><a href="https://cran.r-project.org/package=AvInertia">AvInertia</a> v0.0.1: Provides functions to compute the center of gravity and moment of inertia tensor of any flying bird. See the <a href="https://cran.r-project.org/web/packages/AvInertia/vignettes/how-to-analyze-data.html">vignette</a> for some insight into bird design.</p>
<p><a href="https://cran.r-project.org/package=bayesnec">bayesnec</a> v1.0.1: Provides functions to fit dose concentration response curves to toxicity data, and derive No-Effect-Concentration (NEC), No-Significant-Effect-Concentration (NSEC), and Effect-Concentration (of specified percentage ‘x’, ECx) thresholds from non-linear models fitted using Bayesian MCMC fitting methods via <a href="https://cran.r-project.org/web/packages/brms/index.html">brms</a> and <a href="https://mc-stan.org/">Stan</a>. There are vignettes on <a href="https://cran.r-project.org/web/packages/bayesnec/vignettes/example1.html">Single model usage</a>, <a href="https://cran.r-project.org/web/packages/bayesnec/vignettes/example2.html">Multi model usage</a>, <a href="https://cran.r-project.org/web/packages/bayesnec/vignettes/example2b.html">Model details</a>, <a href="https://cran.r-project.org/web/packages/bayesnec/vignettes/example3.html">Priors</a>, and <a href="https://cran.r-project.org/web/packages/bayesnec/vignettes/example4.html">Comparing posterior predictions</a>.</p>
<h3 id="statistics">Statistics</h3>
<p><a href="https://cran.r-project.org/package=itdr">itdr</a> v1.0: Provides functions to estimate the sufficient dimension reduction subspaces, i.e., central mean subspace or central subspace in regression, using Fourier transformation proposed by <a href="https://www.tandfonline.com/doi/abs/10.1198/016214506000000140">Zhu & Zeng (2006)</a>, the convolution transformation proposed by <a href="https://www.sciencedirect.com/science/article/pii/S0047259X0900147X?via%3Dihub">Zeng & Zhu (2010)</a>, and an iterative Hessian transformation methods proposed by <a href="https://projecteuclid.org/journals/annals-of-statistics/volume-30/issue-2/Dimension-reduction-for-conditional-mean-in-regression/10.1214/aos/1021379861.full">Cook & Li (2002)</a>. See the <a href="https://cran.r-project.org/web/packages/itdr/vignettes/itdr-vignette.html">vignette</a> for an example.</p>
<p><a href="https://cran.r-project.org/package=mlfit">mlfit</a> v0.5.2: Extends the [Iterative Proportional Fitting] <a href="https://trid.trb.org/view/881554">(IPF)</a> algorithm which operates on count data to nested structures with constraints. See <a href="https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/39167/eth-3088-01.pdf?sequence=1&isAllowed=y">Müller & Axhausen (2011)</a> for background and look <a href="https://mlfit.github.io/mlfit/">here</a> for and example.</p>
<p><a href="https://cran.r-project.org/package=optimall">optimall</a> v0.1.0: Provides functions for the survey sampling design process with specific tools for multi-wave and multi-phase designs. Users can perform optimum allocation according to <a href="https://www.jstor.org/stable/2342192?origin=crossref">Neyman (1934)</a> or <a href="https://www.tandfonline.com/doi/abs/10.1080/00031305.2012.733679">Wright (2012)</a>. There is a <a href="https://cran.r-project.org/web/packages/optimall/vignettes/optimall-vignette.html">User’s Guide</a>, there are vignettes <a href="https://cran.r-project.org/web/packages/optimall/vignettes/Multiwave-Object.html">Multiwave Object</a> and <a href="https://cran.r-project.org/web/packages/optimall/vignettes/using-optimall_shiny.html">Splitting Strata with Optimall Shiny</a>.</p>
<p><a href="https://cran.r-project.org/package=posterior">posterior</a> v1.0.1: Provides tools for both users and package developers for fitting and working with Bayesian models; including tools to: efficiently convert between different formats of draws from distributions, provide consistent methods for common operations, provide summaries in convenient formats, and implement state of the art posterior inference diagnostics. See <a href="https://projecteuclid.org/journals/bayesian-analysis/volume-16/issue-2/Rank-Normalization-Folding-and-Localization--An-Improved-R%cb%86-for/10.1214/20-BA1221.full">Vehtari et al. (2021)</a> for background. There is an <a href="https://cran.r-project.org/web/packages/posterior/vignettes/posterior.html">Introduction</a> and a vignette on the <a href="https://cran.r-project.org/web/packages/posterior/vignettes/rvar.html">Random Variable Datatype</a>.</p>
<p><a href="https://cran.r-project.org/package=PropensitySub">PropensitySub</a> v0.2.0: Provides functions to estimate treatment effects in strata via inverse probability weighting or propensity score matching when subjects have missing strata labels. See the <a href="https://cran.r-project.org/web/packages/PropensitySub/vignettes/how_to.html">vignette</a> for examples.</p>
<p><img src="psub.png" height = "200" width="400"></p>
<p><a href="https://cran.r-project.org/package=ReplicationSuccess">ReplicationSuccess</a> v1.0.0: Provides utilities for the design and analysis of replication studies featuring both traditional methods based on statistical significance and more recent methods such as the skeptical p-value <a href="https://rss.onlinelibrary.wiley.com/doi/10.1111/rssa.12493">Held L. (2020)</a>, the harmonic mean chi-squared test <a href="https://rss.onlinelibrary.wiley.com/doi/10.1111/rssc.12410">Held, L. (2020)</a>, and intrinsic credibility <a href="https://royalsocietypublishing.org/doi/10.1098/rsos.181534">Held, L. (2019)</a>. See the <a href="https://cran.r-project.org/web/packages/ReplicationSuccess/vignettes/ReplicationSuccess.pdf">vignette</a> for examples.</p>
<p><img src="RS.png" height = "200" width="400"></p>
<h3 id="time-series">Time Series</h3>
<p><a href="https://cran.r-project.org/package=kinematics">kinematics</a> V1.0.0: Provides functions to analyze time series representing two-dimensional movements. It accepts a data frame with a time (t), horizontal (x) and vertical (y) coordinate as columns, and returns several dynamical properties such as speed, acceleration or curvature. See the <a href="https://cran.r-project.org/web/packages/kinematics/vignettes/example.html">vignette</a> for examples.</p>
<p><img src="kinematics.png" height = "400" width="400"></p>
<p><a href="https://cran.r-project.org/package=profoc">profoc</a> v0.8.3: Provides methods to combine probabilistic forecasts using CRPS learning algorithms proposed in <a href="https://arxiv.org/abs/2102.00968">Berrisch & Ziel (2021)</a> including multiple online learning algorithms such as Bernstein online aggregation as described in <a href="https://arxiv.org/abs/1404.1356">Wintenberger (2014)</a>.</p>
<p><a href="https://cran.r-project.org/package=tscopula">tscopula</a> v0.2.1: Provides functions for the analysis of time series using copula models. See <a href="https://www.mdpi.com/2227-9091/9/1/14"> McNeil (2021)</a> and <a href="https://arxiv.org/abs/2006.11088">Bladt & McNeil (2020)</a> for background. There are vignettes on <a href="https://cran.r-project.org/web/packages/tscopula/vignettes/bitcoin.html">Bitcoin Analysis</a>, <a href="https://cran.r-project.org/web/packages/tscopula/vignettes/tscm_models.html">Models with Margins</a>, <a href="https://cran.r-project.org/web/packages/tscopula/vignettes/tscopulas.html">Basic Time Series Copula Processes</a>, and <a href="https://cran.r-project.org/web/packages/tscopula/vignettes/vtscopulas.html">Copula Processes with V-Transforms</a>.</p>
<p><img src="tscopula.png" height = "300" width="500"></p>
<h3 id="utility">Utility</h3>
<p><a href="https://cran.r-project.org/package=codemeta">codemeta</a> v0.1.0: Provides core utilities to generate metadata with a minimum of dependencies as specified by the <a href="https://codemeta.github.io/">Codemata Project</a>. See <a href="https://cran.r-project.org/web/packages/codemeta/readme/README.html">README</a> for examples.</p>
<p><img src="codemeta.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=ctf">ctf</a> v0.1.0: Provides functions to read and write data in Column Text Format (CTF), a new tabular data format that is a simple column store. See the <a href="https://cran.r-project.org/web/packages/ctf/vignettes/overview.html">vignette</a>.</p>
<p><a href="https://cran.r-project.org/package=fedmatch">fedmatch</a> v2.0.2: Provides tools for matching two un-linked data sets using exact matches, fuzzy matches, or multi-variable matches. There is an <a href="https://cran.r-project.org/web/packages/fedmatch/vignettes/Intro-to-fedmatch.html">Introduction</a> and vignettes on <a href="https://cran.r-project.org/web/packages/fedmatch/vignettes/Fuzzy-matching.html">Fuzzy Matching</a>, <a href="https://cran.r-project.org/web/packages/fedmatch/vignettes/Multivar_matching.html">Multivar Matching</a>, <a href="https://cran.r-project.org/web/packages/fedmatch/vignettes/Using-tier-match.html">Tier Matching</a>, and <a href="https://cran.r-project.org/web/packages/fedmatch/vignettes/Using-clean-strings.html">Using clean_strings</a>.</p>
<p><a href="https://cran.r-project.org/package=fusen">fusen</a> v0.2.4: Implements a method to use R Markdown to build an R package. Users start by including documentation, functions, examples and tests in the same file. Then inflating the R Markdown template copies the relevant chunks and sections in the appropriate files required for package development. There is an <a href="https://cran.r-project.org/web/packages/fusen/vignettes/How-to-use-fusen.html">Introduction</a> and a <a href="https://cran.r-project.org/web/packages/fusen/vignettes/Maintain-packages-with-fusen.html">vignette</a> on maintaining packages.</p>
<p><a href="https://cran.r-project.org/package=katex">katex</a> v1.10: Provides functions to convert <a href="https://www.latex-project.org/">LaTeX</a> math expressions to HTML and <a href="https://www.w3.org/Math/">MATHML</a> for use in markdown documents or package manual pages in a way which eliminates the need for embedding the <a href="https://www.mathjax.org/">MathJax</a> library into your web pages. See the <a href="https://cran.r-project.org/web/packages/katex/vignettes/katex-tests.html">vignette</a>.</p>
<p><img src="katex.png" height = "200" width="300"></p>
<p><a href="https://cran.r-project.org/package=pacs">pacs</a> v0.3.3: Provides utilities for CRAN package maintainers and R packages developers, including tools for validating packages and exploring the complexity of a specific package. See <a href="https://cran.r-project.org/web/packages/pacs/readme/README.html">README</a> for a list of features and examples.</p>
<p><a href="https://cran.r-project.org/package=RcppFarmHash">RcppFarmHash</a> v0.0.2 Implements an interface to the The Google <a href="https://opensource.googleblog.com/2014/03/introducing-farmhash.html">FarmHash</a> family of hash functions is used by the Google <a href="https://cloud.google.com/bigquery">BigQuery</a> data warehouse. Look <a href="https://dirk.eddelbuettel.com/code/rcpp.farmhash.html">here</a> to get started.</p>
<p><a href="https://cran.r-project.org/package=shinyauthr">shinyauthr</a> v10.0: Provides in-app user authentication for Shiny applications, allowing developers to secure publicly hosted apps and build dynamic user interfaces from user information. See <a href="https://cran.r-project.org/web/packages/shinyauthr/readme/README.html">README</a> to get started.</p>
<p><a href="https://cran.r-project.org/package=tangram.pipe">tantram.pipe</a> v1.0.0: Allows users to build tables with customizable rows by specifying the type of data to use for each row, as well as how to handle missing data and the types of comparison tests to run on the table columns. See the <a href="https://cran.r-project.org/web/packages/tangram.pipe/vignettes/Customizable-Table-Building.html">vignette</a>.</p>
<p><img src="tangram.png" height = "300" width="600"></p>
<h3 id="visualization">Visualization</h3>
<p><a href="https://cran.r-project.org/package=chameleon">chameleon</a> v0.2-0: Provides functions to assign distinct colors to arbitrary multi-dimensional data, considering its structure. See the <a href="https://cran.r-project.org/web/packages/chameleon/vignettes/examples.html">vignette</a> for examples.</p>
<p><img src="chameleon.png" height = "300" width="300"></p>
<p><a href="https://cran.r-project.org/package=deeptime">deeptime</a> v0.1.0: Extends plotting packages such as <code>ggplot2</code> and <code>lattice</code> to facilitate the plotting of data over long time intervals, including, but not limited to, geological, evolutionary, and ecological data. Look <a href="https://github.com/willgearty/deeptime">here</a> for examples.</p>
<p><img src="deeptime.png" height = "300" width="600"></p>
<p><a href="https://cran.r-project.org/package=figpatch">figpatch</a> v0.1.0.1: Provides functions to arrange external figures with <code>patchwork</code> alongside <code>ggplot2</code> plots. See <a href="https://cran.r-project.org/web/packages/figpatch/readme/README.html">README</a> for examples.</p>
<p><img src="patchwork.png" height = "400" width="400"></p>
<p><a href="https://cran.r-project.org/package=spiralize">spiralize</a> v1.0.2: Provides functions to visualize data along an <a href="https://en.wikipedia.org/wiki/Archimedean_spiral">Archimedean spiral</a> which has the advantages of being able to visualize data with very a long axis with high resolution and reveal periodic patterns in time series. Look <a href="https://cran.r-project.org/web/packages/spiralize/vignettes/spiralize.html">here</a> for links to the five vignettes.</p>
<p><img src="spiralize.png" height = "500" width="500"></p>
<script>window.location.href='https://rviews.rstudio.com/2021/08/26/july-2021-top-40-new-cran-packages/';</script>
R / Medicine 2021
https://rviews.rstudio.com/2021/08/12/r-medicine-2021/
Thu, 12 Aug 2021 00:00:00 +0000https://rviews.rstudio.com/2021/08/12/r-medicine-2021/
<p><img src="rmed.png" height = "500" width="100%"></p>
<p><a href="https://r-medicine.com/">R/Medicine 2021</a>, the premier conference for the use of R in clinical applications is less than two weeks away! This conference reflects the increasing importance of data science, computational statistics and machine learning to clinical applications, and emphasizes the effectiveness of the R language as a vehicle for making data driven medicine accessible to clinicians with diverse backgrounds. The two conference keynote talks <em>Bringing Machine Learning Models to the Bedside</em> by <a href="https://medicine.umich.edu/dept/lhs/karandeep-singh-md-mmsc">Karandeep Singh</a> and <em>Dissecting Algorithmic Bias</em> by <a href="https://publichealth.berkeley.edu/people/ziad-obermeyer/">Ziad Obermeyer</a> directly address important technical and ethical issues confronting modern data driven medicine.</p>
<p>R/Medicine will offer six short courses spread out over the two days of August 24th and August 25th. These courses are <strong>included in the registration price</strong>. The Tuesday short courses will be:</p>
<ul>
<li><em>Secure Medical Data Collection: Best Practices with Excel, and Leveling Up to REDCap and collaboratoR</em> taught by Peter Higgins, Will Beasley, Kenneth McLean</li>
<li><em>Intro to R for Medical Data</em> taught by Ted Laderas, Daniel Chen, Mara Alexeev</li>
<li><em>An Introductory R Guide for Targeted Maximum Likelihood Estimation in Medical Research</em> taught by Ehsan Karim</li>
</ul>
<p>The Wednesday short courses will be:</p>
<ul>
<li><em>Mapping Spatial Health Data</em> taught by Marynia Kolak</li>
<li><em>Reproducible Research with R</em> taught by Alison Hill and Stephan Kadauke</li>
<li><em>From SAS to R</em> taught by Joe Korszun</li>
</ul>
<p>The conference will proper will run from 11:00 to 19:30 EDT on Thursday, August 26th, and from 10:50 to 18:00 EDT on Friday, August 27th. The <a href="https://r-medicine.com/schedule/">full schedule is here</a>. Priced at only $50 full fare, $25 for academics and $10 for students and trainees, this is an affordable, important conference that you will not want to miss.</p>
<h3 id="you-can-register-here-https-bit-ly-2vlqger">You can <a href="https://bit.ly/2VLqGer"><strong>register here</strong></a>.</h3>
<p>To get an idea of the international scope of the conference, and a feel for what the virtual conference experience might be like, have a look at the <em>R Journal</em> article written by the organizing team about last year’s conference: <a href="https://journal.r-project.org/archive/2021-1/rmed2020.pdf"><em>R Medicine 2020: The Power of Going
Virtual</em></a>.</p>
<p>And finally, to get an idea of the various ways that R is contributing to getting everyday work done in clinical practice: the following are the packages from the Medicine category that made it to my lists of “Top 40” new CRAN packages in posts over the past twelve months.</p>
<h3 id="top-40-picks-for-new-cran-packages-for-medicine">“Top 40” Picks for new CRAN Packages for Medicine</h3>
<p><a href="https://cran.r-project.org/package=afdx">afdx</a> v1.1.1: Provides functions to estimate diagnosis performance (Sensitivity, Specificity, Positive predictive value, Negative predicted value) of a diagnostic test when there is no golden standard by estimating the attributable fraction using either a <a href="https://cran.r-project.org/web/packages/afdx/vignettes/af_logit_exponential.html">logitexponential model</a> or a <a href="https://cran.r-project.org/web/packages/afdx/vignettes/latentclass.html">latent class model</a>.</p>
<p><a href="https://cran.r-project.org/package=aldvmm">aldvmm</a> v0.8.4: Fits health state utility adjusted limited dependent variable mixture models, i.e. finite mixtures of normal distributions with an accumulation of density mass at the limits, and a gap between 100% quality of life and the next smaller utility value. See <a href="https://journals.sagepub.com/doi/10.1177/1536867X1501500307">Alava and Wailoo (2015)</a> for background and the <a href="https://cran.r-project.org/web/packages/aldvmm/vignettes/aldvmm_vignette.pdf">vignette</a> for an example.</p>
<p><a href="https://cran.r-project.org/package=babsim.hospital">babsim.hospital</a> v11.5.14: Implements a discrete-event simulation model for a hospital resource planning. Motivated by the challenges faced by health care institutions in the current COVID-19 pandemic, it can be used by health departments to forecast demand for intensive care beds, ventilators, and staff resources. See <a href="https://www.jstatsoft.org/article/view/v090i02">Ucar, Smeets & Azcorra (2019)</a>, <a href="https://www.rcpjournals.org/content/futurehosp/6/1/17">Lawton & McCooe (2019)</a> and the <a href="https://www.th-koeln.de/informatik-und-ingenieurwissenschaften/babsimhospital_78996.php">website</a> for background, and the <a href="https://cran.r-project.org/web/packages/babsim.hospital/vignettes/babsim-vignette-introduction.pdf">vignette</a> to get started.</p>
<p><a href="https://cran.r-project.org/package=beats">beats</a> v0.1.1: Provides functions to import data from UFI devices and process electrocardiogram (ECG) data. It also includes a Shiny app for finding and exporting heart beats. See <a href="https://cran.r-project.org/web/packages/beats/readme/README.html">README</a> to get started.</p>
<p><a href="https://cran.r-project.org/package=bhmbasket">bhmbasket</a> v0.9.1: Provides functions to evaluate basket trial designs with binary endpoints using Bayesian hierarchical models and Bayesian decision rules. See <a href="https://journals.sagepub.com/doi/10.1177/1740774513497539">Berry et al. (2013)</a>, <a href="https://onlinelibrary.wiley.com/doi/abs/10.1002/pst.1730">Neuenschwander et al. (2016)</a> and <a href="https://link.springer.com/article/10.1177%2F2168479014533970">Fisch et al. (2015)</a> for background and the <a href="https://cran.r-project.org/web/packages/bhmbasket/vignettes/reproduceExNex.html">vignette</a> for an example.</p>
<p><a href="https://cran.r-project.org/package=bp">bp</a> v1.0.1: Provides functions to aid in the analysis of blood pressure data of all forms by providing both descriptive and visualization tools for researchers. There is a <a href="https://cran.r-project.org/web/packages/bp/vignettes/bp.html">vignette</a>.</p>
<p><a href="https://cran.r-project.org/package=card">card</a> v0.1.0: Provides tools to help assess the autonomic regulation of cardiovascular physiology with respect to electrocardiography, circadian rhythms, and the clinical risk of autonomic dysfunction on cardiovascular health through the perspective of epidemiology and causality. For background on the analysis of circadian rhythms through cosinor analysis see <a href="https://tbiomed.biomedcentral.com/articles/10.1186/1742-4682-11-16">Cornelissen (2014)</a> and <a href="https://www.tandfonline.com/doi/abs/10.1080/09291010600903692">Refinetti et al. (2014)</a>. There are two vignettes: <a href="https://cran.r-project.org/web/packages/card/vignettes/circadian.html">circadian</a> and <a href="https://cran.r-project.org/web/packages/card/vignettes/cosinor.html">cosinor</a>.</p>
<p><a href="https://cran.r-project.org/package=causalCmprsk">causalCmprisk</a> v1.0.0: Provides functions to estimate average treatment effects of two static treatment regimes on time-to-event outcomes with competing events. The method uses propensity scores weighting for emulation of baseline randomization. See the <a href="https://cran.r-project.org/web/packages/causalCmprsk/vignettes/cmp_rsk_RHC.html">vignette</a>.</p>
<p><a href="https://cran.r-project.org/package=ccoptimalmatch">ccoptimalmatch</a> v0.1.0: Uses sub-sampling to create pseudo-observations of controls to optimally match cases with controls. See <a href="https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-021-01256-3">Mamoiris (2021)</a> for the theory and the <a href="https://cran.r-project.org/web/packages/ccoptimalmatch/vignettes/ccoptimalmatching_vignette.html">vignette</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=CHOIRBM">CHOIRBM</a> v0.0.2: Provides functions for visualizing body map data collected with the Collaborative Health Outcomes Information Registry <a href="https://choir.stanford.edu/">CHOIR)</a>. See the <a href="https://cran.r-project.org/web/packages/CHOIRBM/vignettes/plot-one-patient.html">vignette</a>.</p>
<p><a href="https://cran.r-project.org/package=clinDataReview">clinDataReview</a> v1.1.0: Provides functions to create interactive tables, listings and figures and associated reports for exploratory analysis in a clinical trial setting. There are vignettes on <a href="https://cran.r-project.org/web/packages/clinDataReview/vignettes/clinDataReview-dataPreprocessing.html">Prerocessing</a>, <a href="https://cran.r-project.org/web/packages/clinDataReview/vignettes/clinDataReview-dataVisualization.html">Visualization</a>, and <a href="https://cran.r-project.org/web/packages/clinDataReview/vignettes/clinDataReview-reporting.html">Creating Reports</a>.</p>
<p><a href="https://cran.r-project.org/package=clinUtils">clinUtils</a> v0.0.4: Provides utility functions to facilitate importing, exploring, and reporting clinical data along with datasets in <a href="https://www.cdisc.org/standards/foundational/sdtm">SDTM</a> and <a href="https://www.cdisc.org/standards/foundational/adam">ADaM</a> format. See the <a href="https://cran.r-project.org/web/packages/clinUtils/vignettes/clinUtils-vignette.html">Introduction</a>.</p>
<p><a href="https://cran.r-project.org/package=cmprskcoxmsm">cmprskcoxmsm</a> v0.2.0: Provides functions to estimate treatment effect a under marginal structure model for the cause-specific hazard of competing risk events. Functions also estimate the risk of the potential outcomes, risk difference and risk ratio. See <a href="https://www.tandfonline.com/doi/abs/10.1198/016214501753168154">Hernan et al. (2001)</a> for the theory and the <a href="https://cran.r-project.org/web/packages/cmprskcoxmsm/vignettes/weight_cause_cox.pdf">vignette</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=coder">coder</a> v0.13.5: Provides functions to classify individuals or items based on external code data identified by regular expressions. A typical use case considers patients with medically coded data, such as codes from the International Classification of Diseases. There is an <a href="https://cran.r-project.org/web/packages/coder/vignettes/coder.html">overview</a> and vignettes on <a href="https://cran.r-project.org/web/packages/coder/vignettes/classcodes.html">class codes</a>, <a href="https://cran.r-project.org/web/packages/coder/vignettes/Interpret_regular_expressions.html">interpreting regular expressions</a>, and <a href="https://cran.r-project.org/web/packages/coder/vignettes/ex_data.html">example data</a>.</p>
<p><a href="https://cran.r-project.org/package=covid19us">covid19us</a> v0.1.2: Implements wrapper around the <a href="https://covidtracking.com/api/">COVID Tracking Project API</a> providing data on cases of COVID-19 in the US.</p>
<p><a href="https://cran.r-project.org/package=covidcast">covidcast</a> v0.4.2: Provides an interface to Delphi’s <a href="https://cmu-delphi.github.io/delphi-epidata/api/covidcast.html">COVIDcast Epidata</a> including tools for data access, maps and time series plotting, and basic signal processing, and a collection of numerous indicators relevant to the COVID-19 pandemic in the United States. There is a <a href="https://cran.r-project.org/web/packages/covidcast/vignettes/covidcast.html">Getting Started Guide</a>, and vignettes on <a href="https://cran.r-project.org/web/packages/covidcast/vignettes/correlation-utils.html">Computing Signal Correlations</a>, <a href="https://cran.r-project.org/web/packages/covidcast/vignettes/external-data.html">Combining Data Sources</a>, <a href="https://cran.r-project.org/web/packages/covidcast/vignettes/multi-signals.html">Manipulating Multiple Signals</a>, and <a href="https://cran.r-project.org/web/packages/covidcast/vignettes/plotting-signals.html">Plotting and Mapping Signals</a>.</p>
<p><a href="https://cran.r-project.org/package=dataquieR">dataQuieR</a> v1.0.4: Provides functions to assess data quality issues in studies. See the <a href="https://www.tmf-ev.de/EnglishSite/Home.aspx">TMF Guideline</a> and the <a href="https://dfg-qa.ship-med.uni-greifswald.de">DFG Project</a> for background, and the <a href="https://cran.r-project.org/web/packages/dataquieR/vignettes/DQ-report-example.html">vignette</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=epigraphdb">epigraphdb</a> v0.2.1: Provides access to the <a href="https://epigraphdb.org">EpiGraphDB</a> platform. There is an <a href="https://cran.r-project.org/web/packages/epigraphdb/vignettes/using-epigraphdb-r-package.html">overview</a>, vignettes on the <a href="https://cran.r-project.org/web/packages/epigraphdb/vignettes/using-epigraphdb-api.html">API</a>, <a href="https://cran.r-project.org/web/packages/epigraphdb/vignettes/meta-functionalities.html">Platform Functionality</a>, <a href="https://cran.r-project.org/web/packages/epigraphdb/vignettes/meta-functionalities.html">Meta Functions</a> and three case studies on <a href="https://cran.r-project.org/web/packages/epigraphdb/vignettes/case-1-pleiotropy.html">SNP protein associations</a>, <a href="https://cran.r-project.org/web/packages/epigraphdb/vignettes/case-2-alt-drug-target.html">Drug Targets</a> and <a href="https://cran.r-project.org/web/packages/epigraphdb/vignettes/case-3-literature-triangulation.html">Causal Evidence</a></p>
<p><a href="https://cran.r-project.org/package=EpiNow2">EpiNow2</a> v1.2.1: Provides functions to estimate the time-varying reproduction number, rate of spread, and doubling time using a range of open-source tools <a href="https://wellcomeopenresearch.org/articles/5-112/v1">Abbott et al. (2020)</a> for background, <a href="https://www.medrxiv.org/content/10.1101/2020.06.18.20134858v3">Gostic et al. (2020)</a> for current best practices, and <a href="https://cran.r-project.org/web/packages/EpiNow2/readme/README.htm">README</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=escalation">escalation</a> v0.1.2: Implements methods for working with dose-finding clinical trials and includes a common interface to various dose-finding methodologies such as the continual reassessment method (CRM) by <a href="https://www.jstor.org/stable/pdf/2531628.pdf?seq=1">O’Quigley et al. (1990)</a>, the Bayesian optimal interval design (BOIN) by <a href="https://mdanderson.elsevierpure.com/en/publications/bayesian-optimal-interval-designs-for-phase-i-clinical-trials">Liu & Yuan (2015)</a>, and the 3+3 described by <a href="https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.4780131802">Korn et al. (1994)</a>. There are vignettes on <a href="https://cran.r-project.org/web/packages/escalation/vignettes/DosePaths.html">Working with dose-paths</a>, <a href="https://cran.r-project.org/web/packages/escalation/vignettes/DoseSelectorInterface.html">Working with dose selectors</a>, and
<a href="https://cran.r-project.org/web/packages/escalation/vignettes/Simulation.html">Simulating dose-escalation trials</a>.</p>
<p><a href="https://cran.r-project.org/package=eventglm">eventglm</a> v1.0.2 Implements methods for doing event history regression for marginal estimands, including cumulative incidence the restricted mean survival, as described in the methodology reviewed in <a href="https://journals.sagepub.com/doi/10.1177/0962280209105020">Andersen & Perme (2010)</a>. See the <a href="https://cran.r-project.org/web/packages/eventglm/vignettes/example-analysis.html">vignette</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=eventTrack">eventTrack</a> v1.0.0: Implements the hybrid framework for event prediction in clinical trials as described in <a href="https://www.sciencedirect.com/science/article/pii/S155171441100139X?via%3Dihub">Fang & Zheng (2011)</a>. See the <a href="https://cran.r-project.org/web/packages/eventTrack/vignettes/eventTrack.html">vignette</a> for an example.</p>
<p><a href="https://cran.r-project.org/package=goldilocks">goldilocks</a> v0.3.0: Implements the Goldilocks adaptive trial design for a time to event outcome using a piecewise exponential model and conjugate Gamma prior distributions as described in <a href="https://www.tandfonline.com/doi/abs/10.1080/10543406.2014.888569?journalCode=lbps20">Broglio et al. (2014)</a>. See the <a href="https://cran.r-project.org/web/packages/goldilocks/vignettes/broglio.html">vignette</a> for an example.</p>
<p><a href="https://cran.r-project.org/package=healthyR">healthyR</a> v0.1.1: Implements hospital data analysis workflow tools including modeling tools, and tools to review common administrative hospital data such as average length of stay, readmission rates, average net pay amounts by service lines, and more. See the <a href="https://cran.r-project.org/web/packages/healthyR/vignettes/getting-started.html">vignette</a>.</p>
<p><a href="https://cran.r-project.org/package=hlaR">hlaR</a> v0.1.0: Implements a tool for the eplet analysis of donor and recipient HLA (human leukocyte antigen) mismatches. There are vignettes on <a href="https://cran.r-project.org/web/packages/hlaR/vignettes/allele-haplotype.html">Imputation</a> and <a href="https://cran.r-project.org/web/packages/hlaR/vignettes/eplet-mm.html">Eplet Mismatch</a> and a <a href="https://emory-larsenlab.shinyapps.io/hlar_shiny/">Shiny App</a> as well.</p>
<p><a href="https://cran.r-project.org/package=inTextSummaryTable">inTextSummaryTable</a> v3.0.1: Provides functions to create tables of summary statistics or counts for clinical data for <a href="https://www.pharmasug.org/proceedings/2015/CP/PharmaSUG-2015-CP04.pdf">TLFs</a>. These tables can be exported as in-text table for a Clinical Study Report in MS Word format or a presentation MS PowerPoint format, or as interactive table. There is an <a href="https://cran.r-project.org/web/packages/inTextSummaryTable/vignettes/inTextSummaryTable-introduction.html">Introduction</a> and six more vignettes including <a href="https://cran.r-project.org/web/packages/inTextSummaryTable/vignettes/inTextSummaryTable-aesthetics.html">Aesthetics</a> and <a href="https://cran.r-project.org/web/packages/inTextSummaryTable/vignettes/inTextSummaryTable-visualization.html">Visualization</a>.</p>
<p><a href="https://cran.r-project.org/package=IPDfromKM">IPDfromKM</a> v0.1.10: Implements a method to reconstruct individual patient data from Kaplan-Meier (KM) survival curves, visualize and assess the accuracy of the reconstruction, and perform secondary analysis on the reconstructed data. The package also implements iterative KM estimation algorithm proposed in <a href="https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-12-9">Guyot (2012)</a>.</p>
<p><a href="https://cran.r-project.org/package=metaSurvival">metaSurvival</a> v0.1.0: Provides a function to assess information from a summary survival curve and test the between-strata heterogeneity. See the <a href="https://github.com/shubhrampandey/metaSurvival">GitHub repo</a> for an example.</p>
<p><a href="https://cran.r-project.org/package=nCov2019">nCov2019</a> v0.4.4: Implements an interface to <a href="https://disease.sh/">disease.sh - Open Disease Data API</a> to access real time and historical data of COVID-19 cases, vaccine and therapeutics data. There is a <a href="https://cran.r-project.org/web/packages/nCov2019/vignettes/nCov2019.html">vignette</a>.</p>
<p><a href="https://cran.r-project.org/package=NHSDataDictionaRy">NHSDataDictionaRy</a> v1.2.1: Provides a common set of simplified web scraping tools for working with the <a href="https://datadictionary.nhs.uk/data_elements_overview.html">NHS Data Dictionary</a>.This package was commissioned by the <a href="https://nhsrcommunity.com/">NHS-R community</a> to provide this consistency of lookups. See the <a href="https://cran.r-project.org/web/packages/NHSDataDictionaRy/vignettes/introduction.html">vignette</a> to get started.</p>
<p><a href="https://cran.r-project.org/package=NMADiagT">NMADiagT</a> v0.1.2: Implements the hierarchical summary receiver operating characteristic model developed by <a href="doi:10.1093/biostatistics/kxx025">Ma et al. (2018)</a> and the hierarchical model developed by <a href="doi:10.1080/01621459.2018.1476239">Lian et al. (2019)</a> for performing meta-analysis. It is able to simultaneously compare one to five diagnostic tests within a missing data framework.</p>
<p><a href="https://cran.r-project.org/package=packDAMipd">packDAMipd</a> v0.1.2: Provides functions to construct both time-homogenous and time-dependent Markov models for cost-effectiveness analyses, perform decision analyses, and conduct deterministic and probabilistic sensitivity analyses. There are vignettes on <a href="https://cran.r-project.org/web/packages/packDAMipd/vignettes/Deterministic_sensitivity_analysis.html">deterministic</a> and <a href="https://cran.r-project.org/web/packages/packDAMipd/vignettes/Probabilstic_sensitivity_analysis.html">probabilistic</a> sensitivity analyses, <a href="https://cran.r-project.org/web/packages/packDAMipd/vignettes/Simple_sick_sicker.html">simple</a> “sick-sicker” models, <a href="https://cran.r-project.org/web/packages/packDAMipd/vignettes/Sick_sicker_age_dependent.html">age-dependent</a> “sick-sicker” models, and <a href="https://cran.r-project.org/web/packages/packDAMipd/vignettes/cycle_dependent.html">cycle dependent</a> models.</p>
<p><a href="https://cran.r-project.org/package=patientProfilesVis">patientProfilesVis</a> v2.0.1: Provides functions to create patient specific profile visualizations for exploration, diagnostic or monitoring purposes during a clinical trial which display the evolution of parameters such as laboratory measurements, ECG data, vital signs, adverse events and more. There is <a href="https://cran.r-project.org/web/packages/patientProfilesVis/vignettes/patientProfiles-template-SDTM.html">template</a> for creating patient profiles from CDISC SDTM datasets, and an <a href="https://cran.r-project.org/web/packages/patientProfilesVis/vignettes/patientProfilesVis-introduction.html">Introduction</a> to the package.</p>
<p><a href="https://cran.r-project.org/package=psrwe">psrwe</a> v1.2: Provides tools to incorporate real-world evidence (RWE) into regulatory and health care decision making and includes functions which implement the PS-integrated RWE analysis methods proposed in <a href="https://www.tandfonline.com/doi/abs/10.1080/10543406.2019.1657133?journalCode=lbps20">Wang et al. (2019)</a>, <a href="https://www.tandfonline.com/doi/abs/10.1080/10543406.2019.1684309?journalCode=lbps20">Wang et al. (2020)</a>, and <a href="https://www.tandfonline.com/doi/abs/10.1080/10543406.2020.1730877?journalCode=lbps20">Chen et al. (2020)</a>. There is a <a href="https://cran.r-project.org/web/packages/psrwe/vignettes/vignette.html">vignette</a> on propensity score integration.</p>
<p><a href="https://cran.r-project.org/package=QDiabetes">QDiabetes</a> v1.0-2: Calculates the risk of developing type 2 diabetes using risk prediction algorithms derived by <a href="https://clinrisk.co.uk/ClinRisk/Welcome.html">ClinRisk</a>. Look <a href="https://github.com/Feakster/qdiabetes">here</a> for information and examples.</p>
<p><a href="https://cran.r-project.org/package=raveio">raveio</a> v0.0.3: implements an interface to the <a href="https://openwetware.org/wiki/RAVE">RAVE</a> (R analysis and visualization of human intracranial electroencephalography data) project which aims at analyzing brain recordings from patients with electrodes placed on the cortical surface or inserted into the brain. See <a href="https://www.biorxiv.org/content/10.1101/2020.06.02.129676v1">Mafnotti et al. (2020)</a> for background.</p>
<p><a href="https://cran.r-project.org/package=reconstructKM">reconstructKM</a> v0.3.0: Provides functions for reconstructing individual-level data (time, status, arm) from Kaplan-MEIER curves published in academic journals. See <a href="https://www.nejm.org/doi/10.1056/NEJMc1808567">Sun et al. (2018)</a> for background and the <a href="https://cran.r-project.org/web/packages/reconstructKM/vignettes/introduction.html">vignette</a> for the reconstruction procedure.</p>
<p><a href="https://cran.r-project.org/package=ReviewR">RevieweR</a> v2.3.6: Implements a portable <code>Shiny</code> tool to explore patient-level electronic health record data and perform chart review in a single integrated framework. This tool supports the <a href="https://www.ohdsi.org/data-standardization/the-common-data-model/">OMOP</a> common data model as well as the <a href="https://mimic.physionet.org/">MIMIC-III</a> data model, and chart review through a <a href="https://www.project-redcap.org/">REDCap</a> API. See the <a href="https://reviewr.thewileylab.org/">RevieweR Website</a> for more information. There are several vignettes including <a href="https://cran.r-project.org/web/packages/ReviewR/vignettes/deploy_local.html">Local</a>, <a href="https://cran.r-project.org/web/packages/ReviewR/vignettes/deploy_docker.html">Docker</a>, <a href="https://cran.r-project.org/web/packages/ReviewR/vignettes/deploy_bigquery.html">BigQuery</a> and <a href="https://cran.r-project.org/web/packages/ReviewR/vignettes/deploy_server.html">Shiny Server</a> deployment and performing a <a href="https://cran.r-project.org/web/packages/ReviewR/vignettes/usage_perform_chart_review.html">Chart Review</a>.</p>
<p><a href="https://cran.r-project.org/package=RHRT">RHRT</a> v1.0.1: Provides methods to scan RR interval data for Premature Ventricular Complexes and parameterise and plot the resulting Heart Rate Turbulence. See <a href="https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(98)08428-1/fulltext">Schmidt et al. (1999)</a> and <a href="https://iopscience.iop.org/article/10.1088/1361-6579/ab98b3">Blesius et al. (2020)</a> and the <a href="https://cran.r-project.org/web/packages/RHRT/vignettes/rhrt-vignette.html">vignette</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=SAMBA">SAMBA</a> v0.9.0: Implements several methods, as proposed in <a href="doi:10.1101/2019.12.26.19015859">Beesley & Mukherjee (2020)</a> for obtaining bias-corrected point estimates along with valid standard errors using electronic health records data with misclassifird EHR-derived disease status. See the <a href="https://cran.r-project.org/web/packages/SAMBA/vignettes/UsingSAMBA.html">vignette</a> for details.</p>
<p><a href="https://cran.r-project.org/package=SteppedPower">SteppedPower</a> v0.1.0: Provides tools for power and sample size calculations and design diagnostics for longitudinal mixed models with a focus on stepped wedge designs using methods introduced in <a href="https://www.sciencedirect.com/science/article/pii/S1551714406000632?via%3Dihub">Hussey and Hughes (2007)</a> and extensions discussed in <a href="https://journals.sagepub.com/doi/10.1177/0962280220932962">Li et al. (2020)</a>. See the <a href="https://cran.r-project.org/web/packages/SteppedPower/vignettes/Getting_Started.html">vignette</a> to get started.</p>
<p><a href="https://cran.r-project.org/package=tboot">tboot</a> v0.2.0: Provides functions to simulate clinical trial data with realistic correlation structures and assumed efficacy levels by using a tilted bootstrap resampling approach. There is a tutorial on <a href="https://cran.r-project.org/web/packages/tboot/vignettes/tboot.html">The Tilted Bootstrap</a> and another on <a href="https://cran.r-project.org/web/packages/tboot/vignettes/tboot_bmr.html">Bayesian Marginal Reconstruction</a>.</p>
<p><a href="https://cran.r-project.org/package=Tplyr">Tplyr</a> v0.1.3: Implement a tool to simplify table creation and the data manipulation necessary to create clinical reports. There is a <a href="https://cran.r-project.org/package=Tplyr">Getting Started Guide</a>, and vignettes on <a href="https://cran.r-project.org/web/packages/Tplyr/vignettes/desc.html">Layers</a>, <a href="https://cran.r-project.org/web/packages/Tplyr/vignettes/options.html">Options</a>, and <a href="https://cran.r-project.org/web/packages/Tplyr/vignettes/table.html">Tables</a>.</p>
<p><a href="https://cran.r-project.org/package=visR">visR</a> v0.2.0: Provides functions to generate clinical graphs and tables with sensible defaults based on graphical principles as described in: <a href="https://onlinelibrary.wiley.com/doi/10.1002/pst.1912">Vandemeulebroecke et al. (2018)</a>, <a href="https://ascpt.onlinelibrary.wiley.com/doi/10.1002/psp4.12455">Vandemeulebroecke et al. (2019)</a>, and <a href="https://bmjopen.bmj.com/content/9/9/e030215">Morris et al. (2019)</a>. Vignettes include <a href="https://cran.r-project.org/web/packages/visR/vignettes/CDISC_ADaM.html">Survival Analysis using CDISC ADaM standard</a>, <a href="https://cran.r-project.org/web/packages/visR/vignettes/Consort_flow_diagram.html">Creating Consort Flow Diagram</a>, <a href="https://cran.r-project.org/web/packages/visR/vignettes/Styling_KM_plots.html">Styling Survival Plots</a> and <a href="https://cran.r-project.org/web/packages/visR/index.html">Survival Analysis</a>.</p>
<script>window.location.href='https://rviews.rstudio.com/2021/08/12/r-medicine-2021/';</script>
R Views Call for Documentation
https://rviews.rstudio.com/2021/08/04/r-views-blog-contest/
Wed, 04 Aug 2021 00:00:00 +0000https://rviews.rstudio.com/2021/08/04/r-views-blog-contest/
<p>We at R Views believe that the R ecosystem, centered around CRAN and Bioconductor, is the world’s largest open repository of statistical knowledge. R packages provide implementations and examples for a tremendous number of statistical methods, procedures, and algorithms. Yet, the virtual library of the R ecosystem is far from being complete. There is plenty of room for examples that explore some little travelled path of computational statistics or illuminate a familiar field with clarity. We invite you to contribute to expanding the knowledge available in the R ecosystem through blogging.</p>
<p>Documentation for R packages begins with the package pdf file which provides a detailed description for each individual function, indicates relationships among functions and often includes references to the statistical and scientific literature. Going down an organizational level you can view the source code for each function. Moving up a level, README files and package vignettes provide a coherent overview of the capabilities of the package as a whole and often include example and elaborate use cases.</p>
<p>While the burden to explain what R packages do falls mainly on package authors, all of us data scientists, statisticians, researchers, students and “ordinary” R users can add to the knowledge encoded in R by elaborating on the capabilities of R functions and packages, developing additional examples and contributing statistical analyses in areas where we may have developed some expertise.</p>
<p><img src="https://imgs.xkcd.com/comics/unpublished_discoveries.png" height = "300" width="500">
<sup><a href="https://www.explainxkcd.com/wiki/index.php/1805:_Unpublished_Discoveries">xkcd 1805 Unpublished Discoveries</a></sup></p>
<p>We are not talking about Nobel Laureate work here. But nearly everyone who regularly uses R has some gem stashed away somewhere. We are looking to mine those gems and organize them in a way that can be shared with the R Community.</p>
<p>We are asking you to please dig up your gem and organize it into a blog post. Tell us what your gem does, how it works, and why it is of some value. Some of the most popular R Views posts introduce technical topics by providing R based tutorials. Others show how to efficiently accomplish everyday tasks or present an elegant calculation or visualization. Think in terms of a well explained example, or if you are really ambitious, you could write that package vignette that you wish existed. Look below for some guidance on more elaborate ideas.</p>
<h2 id="call-for-posts">Call for Posts</h2>
<p>Please submit you posts using the following form: <a href="https://rstd.io/rviews-2021">rstd.io/rviews-2021</a>. All submissions will appear on the <a href="https://community.rstudio.com/c/60"><em>Call for Docs</em></a> page on the RStudio Community Site, and you will need an RStudio Community profile.</p>
<p>The deadline for submission is <strong>Friday, September 23, 2021</strong>. We planning awards of a sort to the best entries and would like to announce these by November 1, 2021.</p>
<h3 id="awards">Awards</h3>
<p>The award categories and the associated prizes are as follows:</p>
<ul>
<li><strong>Honorable Mentions</strong>:
<ul>
<li>One year free RStudio Cloud Premium</li>
<li>A bunch of hex stickers of RStudio packages</li>
</ul></li>
<li><strong>Runners Up</strong>:
<ul>
<li>All prizes listed above, plus</li>
<li>Some number of RStudio t-shirts, books, and mugs (worth up to $200)</li>
</ul></li>
<li><strong>Grand Prizes</strong>:
<ul>
<li>All prizes listed above, plus</li>
<li>Special & persistent recognition by RStudio in the form of a winners page, and a badge that will be publicly visible on your RStudio Community profile</li>
<li>A selected group of submissions will be invited to appear on R Views</li>
</ul></li>
</ul>
<p><sup>* Please note that we may not be able to send t-shirts, books, or other larger items to non-US addresses.</sup><br>
<sup>* Please note that all articles on R Views will go through an editorial review before being published there.</sup><br>
<sup>* With the author’s permission, some posts may appear on R Views before the awards announcements. </sup>.</p>
<h3 id="announcement">Announcement</h3>
<p>The names and work of all winners will be highlighted in an announcement on R Views, and we will announce them on RStudio’s social platforms, including RStudio Community (unless the winner prefers not to be mentioned).</p>
<h2 id="need-inspiration">Need inspiration?</h2>
<ul>
<li>Edgar Ruiz <a href="https://rviews.rstudio.com/2019/06/19/a-gentle-intro-to-tidymodels/">Gentle Introduction to tidymodels</a> is a good example of a homemade package vignette.</li>
<li>Jonathan Regenstein’s series on <a href="https://rviews.rstudio.com/categories/reproducible-finance-with-r/">Reproducible Finance with R</a> presents several masterful examples of Financial use cases.</li>
</ul>
<h2 id="guidance">Guidance</h2>
<ul>
<li>Posts should be well-written, contain enough code to support the main argument and have a few plots and/or tables.</li>
<li><strong>Novelty</strong> - We are looking for original content. That is, your article should not have appeared on blogging platforms, other than your personal website or blog.</li>
<li><strong>Contribution</strong> - Your article should provide novel insight and utility. For example, an exploration of a package or use case that is currently poorly documented will be better received than one which already has excellent documentation and vignettes either on CRAN or in the R community.</li>
<li><strong>Reproducible</strong> - Your submission will need to link to a repo which includes an .Rmd file that reproduces all code, images, and other output in the submitted article. If there is code that requires special resources, such as access to a private database, it is fine to simply refer to these in your repo.</li>
<li><strong>Please confine your posts to less than 1,000 words</strong>, excluding code and image captions. Your article should be self-contained, but may refer to additional content on your repo or personal website.</li>
</ul>
<h3 id="categories">Categories</h3>
<ul>
<li><strong>Homemade Package Vignette</strong> - Package authors put in a lot of effort to build their library and the documentation that supports it, but they can’t cover everything. They may not have developed a vignette yet, or written additional documentation explaining the set of functionality that is especially useful.
With this type of article, introduce readers to a package – or small group of packages – highlighting interesting features or extensions. It should not replicate any existing documentation, instead it should be distinct from – and complementary to – any existing package documentation.</li>
<li><strong>Entry Point into a Topic</strong> - It’s often daunting to get started as you’re exploring a new field or approaching a new problem or methodology. It’s incredibly pleasing when you find the resource that serves as your entry point into that new domain, demystifying the space, pointing out useful packages and examples, getting you started on your work. With this article type, explore resources centered around one problem, highlighting how R helps you approach it.</li>
</ul>
<script>window.location.href='https://rviews.rstudio.com/2021/08/04/r-views-blog-contest/';</script>
June 2021: "Top 40" New CRAN Packages
https://rviews.rstudio.com/2021/07/26/june-2021-top-40-new-cran-packages/
Mon, 26 Jul 2021 00:00:00 +0000https://rviews.rstudio.com/2021/07/26/june-2021-top-40-new-cran-packages/
<p>One hundred ninety-seven new packages made it to CRAN in June. Here are my selections for the “Top 40” in ten categories: Computational Methods, Data, Finance, Genomics, Machine Learning, Medicine, Statistics, Time Series, Utilities, and Visualization. The Medicine category includes multiple packages for medical reporting and table building. Note that eight new packages were removed from CRAN by the time I began my research for this post on July 16th, so they were not considered for the “Top 40”.</p>
<h3 id="computational-methods">Computational Methods</h3>
<p><a href="https://cran.r-project.org/package=disordR">disordR</a> v0.0-2: Provides tools for manipulating values of associative maps which are stored in arbitrary order. When associating keys with values one needs both parts to be in 1-1 correspondence. See the <a href="https://cran.r-project.org/web/packages/disordR/vignettes/disordR.html">vignette</a> for the theory and examples.</p>
<p><a href="https://cran.r-project.org/package=ICvectorfields">ICvectorfields</a> v0.0.2: Provides functions for converting time series of spatial abundance or density data in raster format to vector fields of population movement using the digital image correlation technique. See the <a href="https://cran.r-project.org/web/packages/ICvectorfields/vignettes/Using_ICvectorfields.html">vignette</a>.</p>
<p><img src="ICvectorfields.png" height = "400" width="400"></p>
<p><a href="https://cran.r-project.org/package=rim">rim</a> v0.4.1: Provides an interface to the computer algebra system <a href="https://maxima.sourceforge.io/">Maxima</a> which includes running Maxima commands from within R, generating output in LaTeX and <a href="https://www.w3.org/Math/">MathML</a> and R Markdown. Look <a href="https://rcst.github.io/rim/">here</a> for examples.</p>
<h3 id="data">Data</h3>
<p><a href="https://cran.r-project.org/package=PlanetNICFI">PlanetNICFI</a> v1.0.3: Provides functions to download and process <a href="https://www.planet.com/nicfi/">Planet NICFI</a> satellite imagery from Norway’s International Climate and Forest Initiative utilizing the <a href="https://developers.planet.com/docs/basemaps/reference/#tag/Basemaps-and-Mosaics">Planet Mosaics API</a>. See the <a href="https://cran.r-project.org/web/packages/PlanetNICFI/vignettes/planet_nicfi_functionality.html">vignette</a>.</p>
<p><img src="PlanetNICFI.png" height = "300" width="300"></p>
<p><a href="https://cran.r-project.org/package=rgovcan">rgovcan</a> v1.0.3: Provides access to data and other resources available through the <a href="https://open.canada.ca/en">Canadian Open Government portal</a>. See <a href="https://cran.r-project.org/web/packages/rgovcan/readme/README.html">README</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=wcde">wcde</a> v0.0.1: Implements an interface to the <a href="http://dataexplorer.wittgensteincentre.org/">Wittgenstein Human Capital Data Explorer</a>. See the <a href="https://cran.r-project.org/web/packages/wcde/vignettes/intro.html">vignette</a> for an overview.</p>
<p><img src="wcde.png" height = "500" width="300"></p>
<h3 id="finance">Finance</h3>
<p><a href="https://cran.r-project.org/package=etrm">etrm</a> v1.0.1: Provides functions to perform core tasks within Energy Trading and Risk Management including calculating maximum smoothness forward price curves for electricity and natural gas contracts with flow delivery, as presented in <a href="https://jod.pm-research.com/content/15/1/52">Benth et al. (2007)</a> and portfolio insurance trading strategies for price risk management in the forward market as described in <a href="https://www.sciencedirect.com/science/article/abs/pii/0304405X76900246?via%3Dihub">Black (1976)</a>. There are vignettes describing the <a href="https://cran.r-project.org/web/packages/etrm/vignettes/msfc_forward_curve.html">Maximum Smoothness Forward Curve</a> and <a href="https://cran.r-project.org/web/packages/etrm/vignettes/portfolio_insurance_strategies.html">Portfolio Insurance Trading Strategies</a>.</p>
<p><img src="etrm.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=mshap">mshap</a> v0.1.0: Provides functions to compute <em>mSHAP</em> values on two-part models as proposed by <a href="https://arxiv.org/abs/2106.08990">Matthews & Hartman (2021)</a> using the <em>TreeSHAP</em> algorithm described in <a href="https://www.nature.com/articles/s42256-019-0138-9">Lundberg et al. (2020)</a>. See the vignettes <a href="https://cran.r-project.org/web/packages/mshap/vignettes/mSHAP.html">mSHAP</a> and <a href="https://cran.r-project.org/web/packages/mshap/vignettes/mshap_plots.html">mshap plots</a>.</p>
<p><img src="mshap.png" height = "400" width="400"></p>
<h3 id="genomics">Genomics</h3>
<p><a href="https://cran.r-project.org/package=harmony">harmony</a> v0.1.0: Implements the harmony algorithm for single cell integration described in <a href="https://www.nature.com/articles/s41592-019-0619-0">Korsunsky et al. (2019)</a>. See the <a href="https://cran.r-project.org/web/packages/harmony/vignettes/quickstart.html">vignette</a>.</p>
<p><img src="harmony.jpeg" height = "300" width="500"></p>
<h3 id="machine-learning">Machine Learning</h3>
<p><a href="https://cran.r-project.org/package=AutoScore">AutoScore</a> v0.2.0: Implements an interpretable machine learning framework to automate the development of a clinical scoring model for predefined outcomes. See <a href="https://medinform.jmir.org/2020/10/e21798/">Xie et al. 2020</a> for the details, and the <a href="https://cran.r-project.org/web/packages/AutoScore/vignettes/Guide_book.html">Guide Book</a> to get started.</p>
<p><img src="AutoScore.png" height = "200" width="400"></p>
<p><a href="https://cran.r-project.org/package=daiR">daiR</a> v0.9.0: Implements an interface for the Google Cloud Services <a href="https://cloud.google.com/document-ai/">Document AI API</a> with additional tools for output file parsing and text reconstruction. See the <a href="https://dair.info/">package website</a> for more information and examples. There are six vignettes including a <a href="https://cran.r-project.org/web/packages/daiR/vignettes/using_document_ai.html">User Guide</a>, <a href="https://cran.r-project.org/web/packages/daiR/vignettes/basics.html">Basic processing</a>, and <a href="https://cran.r-project.org/web/packages/daiR/vignettes/extracting_tables.html">Extracting Tables</a>.</p>
<p><img src="daiR.png" height = "500" width="300"></p>
<p><a href="https://cran.r-project.org/package=luz">luz</a> v0.1.0: Implements a high level interface to <a href="https://en.wikipedia.org/wiki/Torch_(machine_learning)">torch</a> providing utilities to reduce the the amount of code needed for common tasks, abstract away torch details and make the same code work on both CPUs and GPUs. See <a href="https://arxiv.org/abs/2002.04688">Howard et al. (2020)</a> and <a href="https://zenodo.org/record/3828935#.YPitIxNKj0o">Falcon et al. (2019)</a> for background and the vignettes <a href="https://cran.r-project.org/web/packages/luz/vignettes/get-started.html">Get started with Luz</a>, <a href="https://cran.r-project.org/web/packages/luz/vignettes/custom-loop.html">Custom Loops</a> and <a href="https://cran.r-project.org/web/packages/luz/vignettes/accelerator.html">Accelerator API</a>.</p>
<p><a href="https://cran.r-project.org/package=mcboost">mcboost</a> v0.3.0: Implements <a href="https://proceedings.mlr.press/v80/hebert-johnson18a.html">Multi-Calibration Boosting</a> and <a href="https://arxiv.org/abs/1805.12317">Multi-Accuracy Boosting</a> to for the multi-calibrate the predictions of machine learning models. See the vignettes <a href="https://cran.r-project.org/web/packages/mcboost/vignettes/mcboost_basics_extensions.html">Basics and Extensions</a> and <a href="https://cran.r-project.org/web/packages/mcboost/vignettes/mcboost_example.html">Health Survey Example</a>.</p>
<p><a href="https://cran.r-project.org/package=RFpredInterval">RFpredInterval</a> v1.0.2: Implements various prediction interval methods with random forests and boosted forests. See <a href="https://arxiv.org/abs/2106.08217">Alakus et al. (2021)</a> and <a href="https://journals.sagepub.com/doi/10.1177/0962280219829885">Roy and Larocque (2020)</a> for the mathematical background.</p>
<h3 id="medicine">Medicine</h3>
<p><a href="https://cran.r-project.org/package=aldvmm">aldvmm</a> v0.8.4: Fits health state utility adjusted limited dependent variable mixture models, i.e. finite mixtures of normal distributions with an accumulation of density mass at the limits, and a gap between 100% quality of life and the next smaller utility value. See <a href="https://journals.sagepub.com/doi/10.1177/1536867X1501500307">Alava and Wailoo (2015)</a> for background and the <a href="https://cran.r-project.org/web/packages/aldvmm/vignettes/aldvmm_vignette.pdf">vignette</a> for an example.</p>
<p><img src="aldvmm.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=clinDataReview">clinDataReview</a> v1.1.0: Provides functions to create interactive tables, listings and figures and associated reports for exploratory analysis in a clinical trial setting. There are vignettes on <a href="https://cran.r-project.org/web/packages/clinDataReview/vignettes/clinDataReview-dataPreprocessing.html">Prerocessing</a>, <a href="https://cran.r-project.org/web/packages/clinDataReview/vignettes/clinDataReview-dataVisualization.html">Visualization</a>, and <a href="https://cran.r-project.org/web/packages/clinDataReview/vignettes/clinDataReview-reporting.html">Creating Reports</a>.</p>
<p><img src="clin.png" height = "300" width="300"></p>
<p><a href="https://cran.r-project.org/package=clinUtils">clinUtils</a> v0.0.4: Provides utility functions to facilitate importing, exploring, and reporting clinical data along with datasets in <a href="https://www.cdisc.org/standards/foundational/sdtm">SDTM</a> and <a href="https://www.cdisc.org/standards/foundational/adam">ADaM</a> format. See the <a href="https://cran.r-project.org/web/packages/clinUtils/vignettes/clinUtils-vignette.html">Introduction</a>.</p>
<p><a href="https://cran.r-project.org/package=inTextSummaryTable">inTextSummaryTable</a> v3.0.1: Provides functions to create tables of summary statistics or counts for clinical data for <a href="https://www.pharmasug.org/proceedings/2015/CP/PharmaSUG-2015-CP04.pdf">TLFs</a>. These tables can be exported as in-text table for a Clinical Study Report in MS Word format or a presentation MS PowerPoint format, or as interactive table. There is an <a href="https://cran.r-project.org/web/packages/inTextSummaryTable/vignettes/inTextSummaryTable-introduction.html">Introduction</a> and six more vignettes including <a href="https://cran.r-project.org/web/packages/inTextSummaryTable/vignettes/inTextSummaryTable-aesthetics.html">Aesthetics</a> and <a href="https://cran.r-project.org/web/packages/inTextSummaryTable/vignettes/inTextSummaryTable-visualization.html">Visualization</a>.</p>
<p><img src="inText.png" height = "300" width="300"></p>
<p><a href="https://cran.r-project.org/package=patientProfilesVis">patientProfilesVis</a> v2.0.1: Provides functions to create patient specific profile visualizations for exploration, diagnostic or monitoring purposes during a clinical trial which display the evolution of parameters such as laboratory measurements, ECG data, vital signs, adverse events and more. There is <a href="https://cran.r-project.org/web/packages/patientProfilesVis/vignettes/patientProfiles-template-SDTM.html">template</a> for creating patient profiles from CDISC SDTM datasets, and an <a href="https://cran.r-project.org/web/packages/patientProfilesVis/vignettes/patientProfilesVis-introduction.html">Introduction</a> to the package.</p>
<p><img src="patient.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=RHRT">RHRT</a> v1.0.1: Provides methods to scan RR interval data for Premature Ventricular Complexes and parameterise and plot the resulting Heart Rate Turbulence. See <a href="https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(98)08428-1/fulltext">Schmidt et al. (1999)</a> and <a href="https://iopscience.iop.org/article/10.1088/1361-6579/ab98b3">Blesius et al. (2020)</a> and the <a href="https://cran.r-project.org/web/packages/RHRT/vignettes/rhrt-vignette.html">vignette</a> for examples.</p>
<p><img src="RHRT.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=visR">visR</a> v0.2.0: Provides functions to generate clinical graphs and tables with sensible defaults based on graphical principles as described in: <a href="https://onlinelibrary.wiley.com/doi/10.1002/pst.1912">Vandemeulebroecke et al. (2018)</a>, <a href="https://ascpt.onlinelibrary.wiley.com/doi/10.1002/psp4.12455">Vandemeulebroecke et al. (2019)</a>, and <a href="https://bmjopen.bmj.com/content/9/9/e030215">Morris et al. (2019)</a>. Vignettes include <a href="https://cran.r-project.org/web/packages/visR/vignettes/CDISC_ADaM.html">Survival Analysis using CDISC ADaM standard</a>, <a href="https://cran.r-project.org/web/packages/visR/vignettes/Consort_flow_diagram.html">Creating Consort Flow Diagram</a>, <a href="https://cran.r-project.org/web/packages/visR/vignettes/Styling_KM_plots.html">Styling Survival Plots</a> and <a href="https://cran.r-project.org/web/packages/visR/index.html">Survival Analysis</a>.</p>
<p><img src="visR.png" height = "300" width="500"></p>
<h3 id="statistics">Statistics</h3>
<p><a href="https://cran.r-project.org/package=admix">admix</a> v0.3.2: Implements several methods to estimate the unknown quantities related to two-component admixture models, where the two components can belong to any distribution. See <a href="https://link.springer.com/article/10.3103%2FS1066530710010023">Bordes & Vandekerkhove (2010)</a>, <a href="https://rss.onlinelibrary.wiley.com/doi/10.1111/rssb.12148">Patra & Sen (2016)</a>, and <a href="https://hal.archives-ouvertes.fr/hal-03201760">Milhaud et al. (2021)</a> for background. There are vignettes on <a href="https://cran.r-project.org/web/packages/admix/vignettes/admixture-clustering.html">Clusterine</a>, <a href="https://cran.r-project.org/web/packages/admix/vignettes/admixture-weight-estimation.html">Estimation</a>, and <a href="https://cran.r-project.org/web/packages/admix/vignettes/test-hypothesis.html">Hypothesis Testing</a>.</p>
<p><a href="https://cran.r-project.org/package=ahMLE">ahMLE</a> v1.18: Implements methods for fitting additive hazards model which include the maximum likelihood method as well as Aalen’s method for estimating the additive hazards model. See <a href="https://arxiv.org/abs/2004.06156">Chengyuan Lu(2021)</a> for details and the <a href="https://cran.r-project.org/web/packages/ahMLE/vignettes/ahMLE_manual.html">vignette</a> for an example.</p>
<p><a href="https://cran.r-project.org/package=bayesrules">bayesrules</a> v0.0.1: Provides datasets and functions used for analysis and visualizations in the online <a href="https://www.bayesrulesbook.com">Bayes Rules</a> book. There is a vignette on <a href="https://cran.r-project.org/web/packages/bayesrules/vignettes/conjugate-families.html">Conjugate Families</a> and another on <a href="https://cran.r-project.org/web/packages/bayesrules/vignettes/model-evaluation.html">Model Evaluation</a>.</p>
<p><img src="bayesrules.png" height = "400" width="400"></p>
<p><a href="https://cran.r-project.org/package=dbglm">dbglm</a> v1.0.0: Provides a function to fit generalized linear models on moderately large datasets, by taking an initial sample, fitting in memory, then evaluating the score function for the full data in the database. See <a href="https://www.tandfonline.com/doi/abs/10.1080/10618600.2019.1610312?journalCode=ucgs20">Lumley (2019)</a> for the details.</p>
<p><a href="https://cran.r-project.org/package=fasano.franceschini.test">fasano.franceschini.test</a> v1.0.1: Implements the the 2-D Kolmogorov-Smirnov (KS) two-sample test as defined in <a href="https://academic.oup.com/mnras/article/225/1/155/1007281">Fasano & Franceschini (1987)</a>. See the <a href="https://cran.r-project.org/web/packages/fasano.franceschini.test/vignettes/fasano-franceschini-test.html">vignette</a> for examples.</p>
<p><img src="ff.png" height = "400" width="400"></p>
<p><a href="https://cran.r-project.org/package=flatness">flatness</a> v0.1.4: Provides S3 classes, plotting functions, indices and tests to analyze the flatness of histograms including functions for flatness tests introduced in <a href="https://journals.ametsoc.org/view/journals/mwre/136/6/2007mwr2219.1.xml">Jolliffe & Primo (2008)</a>, flatness indices described in <a href="https://journals.ametsoc.org/view/journals/mwre/147/2/mwr-d-18-0369.1.xml">Wilks (2019)</a>, and the procedure for multiple hypothesis described in <a href="https://rss.onlinelibrary.wiley.com/doi/10.1111/j.2517-6161.1995.tb02031.x">Benjamini & Hochberg (1995)</a>. See the <a href="https://cran.r-project.org/web/packages/flatness/vignettes/flatness.html">vignette</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=outlierensembles">outlierensembles</a> v0.1.0: Provides ensemble functions for detecting outliers and anomalies including a new method based on Item Response Theory described in <a href="https://arxiv.org/abs/2106.06243">Kandanaarachchi (2021)</a> and methods described in <a href="https://epubs.siam.org/doi/10.1137/1.9781611972825.90">Schubert et al. (2012)</a>, <a href="https://www.sciencedirect.com/science/article/pii/S1570868316301240?via%3Dihub">Chiang et al. (2017)</a>, and <a href="https://dl.acm.org/doi/10.1145/2830544.2830549">Aggarwal and Sathe (2015)</a>. See the <a href="https://cran.r-project.org/web/packages/outlierensembles/vignettes/outlierensembles.html">vignette</a> for examples.</p>
<p><img src="outlier.png" height = "300" width="250"></p>
<p><a href="https://cran.r-project.org/package=susieR">susieR</a> v0.11.42: Implements methods for variable selection in linear regression based on the <em>Sum of Single Effects</em> model, as described in <a href="https://www.biorxiv.org/content/10.1101/501114v4">Wang et al (2020)</a>. The <em>Iterative Bayesian Stepwise Selection</em> algorithm allows fitting models to large data sets with thousands of samples and hundreds of thousands of variables. There are ten short vignettes including <a href="https://cran.r-project.org/web/packages/susieR/vignettes/trendfiltering_derivations.pdf">Trend Fitting</a> and a <a href="https://cran.r-project.org/web/packages/susieR/vignettes/mwe.html">minimal example</a>.</p>
<h3 id="time-series">Time Series</h3>
<p><a href="https://cran.r-project.org/package=psdr">psdr</a> v1.0.1: Provides functions to generate and compare power spectral density plots given time series data and to compare the dominant frequencies of multiple groups of time series. Look <a href="https://www.mathworks.com/help/matlab/ref/fft.html">here</a> and <a href="https://www.mathworks.com/help/signal/ug/power-spectral-density-estimates-using-fft.html">here</a> for the mathematical background. For examples look <a href="https://yhhc2.github.io/psdr/articles/Introduction.html">here</a> or see this <a href="https://cran.r-project.org/web/packages/psdr/vignettes/Examples.html">vignette</a>. There is also an <a href="https://cran.r-project.org/web/packages/psdr/vignettes/Introduction.html">Introduction</a>.</p>
<p><img src="psdr.png" height = "400" width="400"></p>
<p><a href="https://cran.r-project.org/package=proteus">proteus</a> v1.0.0 Implements a <em>Sequence-to-Sequence Variational Model</em> for time-feature analysis based on a wide range of different distributions. Look <a href="https://rpubs.com/giancarlo_vercellino/proteus">here</a> for an overview with examples.</p>
<p><img src="proteus.png" height = "300" width="500"></p>
<h3 id="utilities">Utilities</h3>
<p><a href="https://cran.r-project.org/package=archive">archive</a> v1.0.2: Implements bindings to <a href="http://www.libarchive.org">libarchive</a>, the multi-format archive and compression library which offer connections and direct extraction for many archive formats including <code>tar</code>, <code>ZIP</code>, <code>7-zip</code>, <code>RAR</code>, <code>CAB</code> and compression formats including <code>gzip</code>, <code>bzip2</code>, <code>compress</code>, <code>lzma</code> and <code>xz</code>. See <a href="https://cran.r-project.org/web/packages/archive/readme/README.html">README</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=pasadr">pasadr</a> v1.0: Implements the anomaly detection method described in <a href="https://dl.acm.org/doi/10.1145/3243734.3243781">Aoudi et al. (2018)</a>.</p>
<p><a href="https://cran.r-project.org/package=recogito">recogito</a> v0.1.1: Provides <code>htmlwidgets</code> bindings to the <a href="https://github.com/recogito/recogito-js">recogito</a> and <a href="https://github.com/recogito/annotorious">annotorious</a> libraries to annotate text and areas of interest in images. See <a href="https://cran.r-project.org/web/packages/recogito/readme/README.html">README</a> to get started.</p>
<p><img src="recogito.gif" height = "500" width="600"></p>
<p><a href="https://cran.r-project.org/package=rextendr">rextendr</a> v0.2.0: Provides functions to compile and load <a href="https://www.rust-lang.org/">Rust</a> code from R along with helper functions to create R packages that use Rust code. There is a vignette on <a href="https://cran.r-project.org/web/packages/rextendr/vignettes/package.html">Using Rust code in R packages</a> and another on <a href="https://cran.r-project.org/web/packages/rextendr/vignettes/setting_up_rust.html">Setting up Rust</a>.</p>
<p><a href="https://cran.r-project.org/package=shinymeta">shinymeta</a> v0.2.0.1: Provides tools for capturing logic in a Shiny app and exposing it as code that can be run outside of Shiny (e.g., from an R console). It also provides tools for bundling both the code and results to the end user. See <a href="https://cran.r-project.org/web/packages/shinymeta/readme/README.html">README</a> for examples</p>
<p><img src="shinymeta.gif" height = "300" width="400"></p>
<h3 id="visualization">Visualization</h3>
<p><a href="https://cran.r-project.org/package=dynplot">dynplot</a> v1.1.1: Provides functions to visualize a single-cell trajectory as a graph or dendrogram as a dimensionality reduction or heatmap of the expression data, or a comparison between two trajectories as a pairwise scatterplot or dimensionality reduction projection. See <a href="https://www.nature.com/articles/s41587-019-0071-9">Saelens et al. (2019)</a> for background and the <a href="https://cran.r-project.org/web/packages/dynplot/vignettes/plotting-a-toy.html">vignette</a> for examples.</p>
<p><img src="dynplot.png" height = "400" width="400"></p>
<p><a href="https://cran.r-project.org/package=gridpattern">gridpattern</a> 0.2.1: Provides <em>grid grabs</em> to fill in a user-defined plot area with various patterns, including geometric and image-based patterns, and support for custom user-defined patterns. There is a <a href="https://cran.r-project.org/web/packages/gridpattern/vignettes/tiling.html">vignette</a>.</p>
<p><img src="gridpattern.png" height = "300" width="300"></p>
<p><a href="https://cran.r-project.org/package=netplot">netplot</a> v0.1-1: Implements a graph visualization engine that puts an emphasis on aesthetics while providing default parameters that yield <em>out-of-the-box-nice</em> visualizations. There is a vignette with <a href="https://cran.r-project.org/web/packages/netplot/vignettes/base-and-grid.html">base plot </a> examples, and another showing graph drawing with <a href="https://cran.r-project.org/web/packages/netplot/vignettes/examples.html">netplot</a>.</p>
<p><img src="netplot.png" height = "300" width="300"></p>
<p><a href="https://cran.r-project.org/package=NGLVieweR">NGLVieweR</a> v1.3.1: Implements an <a href="https://www.htmlwidgets.org/">htmlwidgets</a> interface to <a href="http://nglviewer.org/ngl/api/">NGL.js</a> enabling users to visualize and interact with protein databank <a href="https://www.rcsb.org/">PDB</a> and structural files in R and Shiny applications. See the <a href="https://cran.r-project.org/web/packages/NGLVieweR/vignettes/NGLVieweR.html">vignette</a> for examples.</p>
<p><img src="NGLVieweR.png" height = "300" width="300"></p>
<p><a href="https://cran.r-project.org/package=precisePlacement">precisePlacement</a> v0.1.0: Provides a selection of tools that make it easier to place elements onto a base R plot exactly where you want them. See the <a href="https://cran.r-project.org/web/packages/precisePlacement/vignettes/Overview.html">vignette</a> for an overview.</p>
<p><img src="precise.png" height = "300" width="300"></p>
<script>window.location.href='https://rviews.rstudio.com/2021/07/26/june-2021-top-40-new-cran-packages/';</script>
Exploratory Functional PCA with Sparse Data
https://rviews.rstudio.com/2021/07/08/exploratory-fda-with-sparse-data/
Thu, 08 Jul 2021 00:00:00 +0000https://rviews.rstudio.com/2021/07/08/exploratory-fda-with-sparse-data/
<script src="/2021/07/08/exploratory-fda-with-sparse-data/index_files/header-attrs/header-attrs.js"></script>
<script src="/2021/07/08/exploratory-fda-with-sparse-data/index_files/htmlwidgets/htmlwidgets.js"></script>
<script src="/2021/07/08/exploratory-fda-with-sparse-data/index_files/plotly-binding/plotly.js"></script>
<script src="/2021/07/08/exploratory-fda-with-sparse-data/index_files/typedarray/typedarray.min.js"></script>
<script src="/2021/07/08/exploratory-fda-with-sparse-data/index_files/jquery/jquery.min.js"></script>
<link href="/2021/07/08/exploratory-fda-with-sparse-data/index_files/crosstalk/css/crosstalk.css" rel="stylesheet" />
<script src="/2021/07/08/exploratory-fda-with-sparse-data/index_files/crosstalk/js/crosstalk.min.js"></script>
<link href="/2021/07/08/exploratory-fda-with-sparse-data/index_files/plotly-htmlwidgets-css/plotly-htmlwidgets.css" rel="stylesheet" />
<script src="/2021/07/08/exploratory-fda-with-sparse-data/index_files/plotly-main/plotly-latest.min.js"></script>
<p>I have written about the basics of Functional Data Analysis in three prior posts. In <a href="https://rviews.rstudio.com/2021/05/04/functional-data-analysis-in-r/">Post 1</a>, I used the <a href="https://cran.r-project.org/package=fda"><code>fda</code></a> package to introduce the fundamental concept of using basis vectors to represent longitudinal or time series data as a curve in an abstract vector space. In <a href="https://rviews.rstudio.com/2021/05/14/basic-fda-descriptive-statistics-with-r/">Post 2</a>, I continued to rely on the <code>fda</code> package to show basic FDA descriptive statistics. In <a href="https://rviews.rstudio.com/2021/06/10/functional-pca-with-r/">Post 3</a>, I introduced Functional PCA with the help of the <a href="https://cran.r-project.org/package=fdapace"><code>fdapace</code></a> package. In this post, I pick up where that last post left off and look at how one might explore a sparse, longitudinal data set with the FPCA tools provided in the <code>fdapace</code> package. I will begin by highlighting some of the really nice tools available in the <a href="https://cran.r-project.org/package=brolgar"><code>brolgar</code></a> package for doing exploratory longitudinal data analysis. While the first three posts made do with artificial data constructed with an algorithm for generating data from a Wiener Process, in this post I’ll use the <code>wages</code> data set available in <code>brolgar</code>.</p>
<p><code>brolgar</code> is a beautiful, tidyverse inspired package, based on the <a href="https://tsibble.tidyverts.org/"><code>tsibble</code></a> data structure that offers a number of super helpful functions for visualizing and manipulating longitudinal data. See the arXiv paper <a href="https://arxiv.org/abs/2012.01619">Tierney, Cook and Prvan (2021)</a> for an overview, and the seven package vignettes for examples. Collectively, these vignettes offer a pretty thorough exploration of the <code>wages</code> data set. Using <code>wages</code> for this post should provide a feeling for what additional insight FDA may have to offer.</p>
<div id="longitudinal-data" class="section level3">
<h3>Longitudinal Data</h3>
<p>Let’s look at the data. <code>wages</code> contains measurements on hourly wages associated with years of experience in the workforce along with several covariates for male high school dropouts who were between 14 and 17 years old when first measured. In what follows, I will use only<code>ln_wages</code>, the natural log of wages in 1990 dollars, and <code>xp</code> the number of years of work experience.</p>
<pre class="r"><code>library(brolgar)
library(fdapace)
library(tidyverse)
library(plotly)
dim(wages)</code></pre>
<pre><code>## [1] 6402 9</code></pre>
<pre class="r"><code>head(wages)</code></pre>
<pre><code>## # A tsibble: 6 x 9 [!]
## # Key: id [1]
## id ln_wages xp ged xp_since_ged black hispanic high_grade
## <int> <dbl> <dbl> <int> <dbl> <int> <int> <int>
## 1 31 1.49 0.015 1 0.015 0 1 8
## 2 31 1.43 0.715 1 0.715 0 1 8
## 3 31 1.47 1.73 1 1.73 0 1 8
## 4 31 1.75 2.77 1 2.77 0 1 8
## 5 31 1.93 3.93 1 3.93 0 1 8
## 6 31 1.71 4.95 1 4.95 0 1 8
## # … with 1 more variable: unemploy_rate <dbl></code></pre>
<p>A summary of the variables shows that wages vary from 2 dollars per hour to a high of about 73 dollars per hour. Time varies between 0 and almost 13 years.</p>
<pre class="r"><code>wages %>% select(ln_wages, xp) %>% summary()</code></pre>
<pre><code>## ln_wages xp id
## Min. :0.708 Min. : 0.001 Min. : 31
## 1st Qu.:1.591 1st Qu.: 1.609 1st Qu.: 3194
## Median :1.842 Median : 3.451 Median : 6582
## Mean :1.897 Mean : 3.957 Mean : 6301
## 3rd Qu.:2.140 3rd Qu.: 5.949 3rd Qu.: 9300
## Max. :4.304 Max. :12.700 Max. :12543</code></pre>
<p>Next, we construct the <code>tsibble</code> data structure to make use of some of the very convenient <code>brolgar</code> sampling functions, and count the number of observations for each subject.</p>
<pre class="r"><code>wages_t <- as_tsibble(wages,
key = id,
index = xp,
regular = FALSE)
num_obs <- wages_t %>% features(ln_wages,n_obs)
summary(num_obs$n_obs)</code></pre>
<pre><code>## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 1.00 5.00 8.00 7.21 9.00 13.00</code></pre>
<p>Just like the <code>brolgar</code> package vignettes, we filter out subjects with less than 3 observations. Then we use the <code>sample_n_keys()</code> function to generate a random sample of 10 wages versus year’s experience curves and plot them.</p>
<pre class="r"><code>df <- wages_t %>% add_n_obs() %>% filter(n_obs > 3)
set.seed(123)
df %>%
sample_n_keys(size = 10) %>%
ggplot(aes(x = xp, y = ln_wages, group = id, color = id)) +
geom_line()</code></pre>
<p><img src="/2021/07/08/exploratory-fda-with-sparse-data/index_files/figure-html/unnamed-chunk-4-1.png" width="672" /></p>
<p><code>brolgar</code> makes it easy to generates lots of panels with small numbers of curves in order to get a feel for the data.</p>
<pre class="r"><code>df %>% ggplot(aes(x = xp, y = ln_wages, group = id, colour = id)) +
geom_line() +
facet_sample(n_per_facet = 3, n_facets = 20)</code></pre>
<p><img src="/2021/07/08/exploratory-fda-with-sparse-data/index_files/figure-html/unnamed-chunk-5-1.png" width="672" /></p>
<p>Finally, for comparison with the plots produced by <code>fdapace</code> we plot the curves for the first two subjects in the <code>tsibble</code>.</p>
<pre class="r"><code>df %>% filter(id == 31 | id == 36) %>%
ggplot(aes(x = xp, y = ln_wages, group = id, color = id)) +
geom_line()</code></pre>
<p><img src="/2021/07/08/exploratory-fda-with-sparse-data/index_files/figure-html/unnamed-chunk-6-1.png" width="672" /></p>
<p>We finish our initial look at the data by noting that we are really dealing with sparse data here. Some curves have only 4 measurements, no curve has more than 13 measurements, and all subjects were measured at different times. This is a classic longitudinal data set.</p>
</div>
<div id="functional-pca" class="section level3">
<h3>Functional PCA</h3>
<p>As I mentioned in my previous post, Principal Components by Conditional Expectation (PACE), described in <a href="https://anson.ucdavis.edu/~mueller/jasa03-190final.pdf">Yao, Müller & Wang (2005)</a>, was designed for sparse data. The method works by pooling the data. Curves are not individually smoothed. Instead, estimates of the FPC scores are obtained from the entire ensemble of data. (See equation (5) in the reference above.)</p>
<p>The first step towards using FPCA functions in the <a href="https://cran.r-project.org/package=brolgar"><code>fdapace</code></a> package is to reshape the data so that the time and wages data for each subject are stored as lists in separate columns of a <code>tibble</code> where each row contains all of the data for a single id. (Standard <code>dplyr</code> operations might not work as expected on a <code>tsibble</code>.) The following code pulls just the required data into a <code>tibble</code> before the <code>dplyr</code> code in the somewhat untidy ‘for loop’ builds the data structure we will use for the analysis.</p>
<pre class="r"><code>df2 <- df %>% select(id, xp, ln_wages) %>% as_tibble()
uid <- unique(df2$id)
N <- length(uid)
Wages <- rep(0,N)
Exp <- rep(0,N)
for (k in 1:N){
Wages[k] <- df2 %>% filter(id == uid[k]) %>% select(ln_wages) %>% pull() %>% list()
Exp[k] <- df2 %>% filter(id == uid[k]) %>% select(xp) %>% pull() %>% list()
}
df3 <- tibble( uid, Wages, Exp )
glimpse(df3)</code></pre>
<pre><code>## Rows: 764
## Columns: 3
## $ uid <int> 31, 36, 53, 122, 134, 145, 155, 173, 207, 222, 223, 226, 234, 24…
## $ Wages <list> <1.491, 1.433, 1.469, 1.749, 1.931, 1.709, 2.086, 2.129>, <1.98…
## $ Exp <list> <0.015, 0.715, 1.734, 2.773, 3.927, 4.946, 5.965, 6.984>, <0.31…</code></pre>
<p>he <code>FPCA()</code> function computes the functional principal components. Note that in the function call the input data are the two columns of lists we created above. The <code>dataType</code> parameter specifies the data as being sparse. <code>error = FALSE</code> means we are using a simple model that does not account for unobserved error. <code>kernel =</code>epan` means that the we are using the <a href="https://www.gabormelli.com/RKB/Epanechnikov_Kernel">Epanechnikov</a> for smoothing the pooled data to compute the mean and covariance. (For this data set, Epanechnikov seems to yield better results than the default Gaussian kernel.)</p>
<pre class="r"><code>res_wages <- FPCA(df3$Wages,df3$Exp, list(dataType='Sparse', error=FALSE, kernel='epan', verbose=TRUE))</code></pre>
<p>The plot method for the resulting <code>FPCA</code> object provides a visual summary of the results. Going clockwise from the upper left, the Design Plot shows that collectively the data are fairly dense over the support except at the upper range near twelve years. The computed mean function for the data shows a little dip in wages near the beginning and then a clear upward trend until it abruptly drops towards the end. The first three eigenfunctions are plotted at the bottom right, and the scree plot at the bottom left shows that the first eigenfunction accounts for about 60% of the total variation and that it takes about eight eigenfunctions to account for 99% of the variance. Note that the default label for the <em>time</em> access in all of the <code>fdapace</code> plots is <em>s</em> for support.</p>
<pre class="r"><code>plot(res_wages)</code></pre>
<p><img src="/2021/07/08/exploratory-fda-with-sparse-data/index_files/figure-html/unnamed-chunk-9-1.png" width="672" />
We can obtain the exact estimates for the <em>cumulative Fraction of Variance Explained</em> by picking <code>cumFVE</code> out of the <code>FPCA</code> object.</p>
<pre class="r"><code>round(res_wages$cumFVE,3)</code></pre>
<pre><code>## [1] 0.591 0.739 0.806 0.860 0.908 0.936 0.957 0.975 0.983 0.989 0.993</code></pre>
<p>The following plot shows the smoothed curves estimated for the first two subjects. These are the same subject plotted in the third plot above. The circles indicate the input data. Everything looks pretty good here.</p>
<pre class="r"><code>CreatePathPlot(res_wages, subset=1:2, main = 'Estimated Paths for IDs 31 and 36'); grid()</code></pre>
<p><img src="/2021/07/08/exploratory-fda-with-sparse-data/index_files/figure-html/unnamed-chunk-11-1.png" width="672" />
But looking at just one more subject shows how quickly things can apparently go off the rails. The green curve for subject id 53 after 1.77 years is pure algorithmic imagination. Although there are several data points in the first two years, there is nothing thereafter.</p>
<pre class="r"><code>CreatePathPlot(res_wages, subset= 1:3, showMean=TRUE)</code></pre>
<p><img src="/2021/07/08/exploratory-fda-with-sparse-data/index_files/figure-html/unnamed-chunk-12-1.png" width="672" /></p>
<p>The value of the FPCA analysis lies in estimating the mean function and the covariance operator which are constructed from the pooled data, and not in predicting an individual paths.</p>
<p>The covariance surface is easily plotted with the help of the extractor function <code>GetCovSurface()</code> which fetches the time grid and associated covariance surface stored in the <code>FPCA</code> object. These are in the right format for a three dimensional, interactive <code>plotly</code> visualization. Rotating the plot and changing the viewing angle reveals quite a bit about the details of the estimated covariance surface.</p>
<pre class="r"><code>covS <- GetCovSurface(df3$Wages,df3$Exp)
x <- covS$workGrid
Surf <- covS$cov
fig <- plot_ly(x = x, y = x, z = ~Surf) %>%
add_surface(contours = list(
z = list(show=TRUE,usecolormap=TRUE,
highlightcolor="#ff0000", project=list(z=TRUE))))
fig <- fig %>%
layout(scene = list(camera=list(eye = list(x=1.87, y=0.88, z=-0.64))))
fig</code></pre>
<div id="htmlwidget-1" style="width:672px;height:480px;" class="plotly html-widget"></div>
<script type="application/json" data-for="htmlwidget-1">{"x":{"visdat":{"3e4258572446":["function () ","plotlyVisDat"]},"cur_data":"3e4258572446","attrs":{"3e4258572446":{"x":[0.001,0.25498,0.50896,0.76294,1.01692,1.2709,1.52488,1.77886,2.03284,2.28682,2.5408,2.79478,3.04876,3.30274,3.55672,3.8107,4.06468,4.31866,4.57264,4.82662,5.0806,5.33458,5.58856,5.84254,6.09652,6.3505,6.60448,6.85846,7.11244,7.36642,7.6204,7.87438,8.12836,8.38234,8.63632,8.8903,9.14428,9.39826,9.65224,9.90622,10.1602,10.41418,10.66816,10.92214,11.17612,11.4301,11.68408,11.93806,12.19204,12.44602,12.7],"y":[0.001,0.25498,0.50896,0.76294,1.01692,1.2709,1.52488,1.77886,2.03284,2.28682,2.5408,2.79478,3.04876,3.30274,3.55672,3.8107,4.06468,4.31866,4.57264,4.82662,5.0806,5.33458,5.58856,5.84254,6.09652,6.3505,6.60448,6.85846,7.11244,7.36642,7.6204,7.87438,8.12836,8.38234,8.63632,8.8903,9.14428,9.39826,9.65224,9.90622,10.1602,10.41418,10.66816,10.92214,11.17612,11.4301,11.68408,11.93806,12.19204,12.44602,12.7],"z":{},"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"surface","contours":{"z":{"show":true,"usecolormap":true,"highlightcolor":"#ff0000","project":{"z":true}}},"inherit":true}},"layout":{"margin":{"b":40,"l":60,"t":25,"r":10},"scene":{"camera":{"eye":{"x":1.87,"y":0.88,"z":-0.64}},"xaxis":{"title":[]},"yaxis":{"title":[]},"zaxis":{"title":"Surf"}},"hovermode":"closest","showlegend":false,"legend":{"yanchor":"top","y":0.5}},"source":"A","config":{"showSendToCloud":false},"data":[{"colorbar":{"title":"Surf","ticklen":2,"len":0.5,"lenmode":"fraction","y":1,"yanchor":"top"},"colorscale":[["0","rgba(68,1,84,1)"],["0.0416666666666667","rgba(70,19,97,1)"],["0.0833333333333334","rgba(72,32,111,1)"],["0.125","rgba(71,45,122,1)"],["0.166666666666667","rgba(68,58,128,1)"],["0.208333333333333","rgba(64,70,135,1)"],["0.25","rgba(60,82,138,1)"],["0.291666666666667","rgba(56,93,140,1)"],["0.333333333333333","rgba(49,104,142,1)"],["0.375","rgba(46,114,142,1)"],["0.416666666666667","rgba(42,123,142,1)"],["0.458333333333333","rgba(38,133,141,1)"],["0.5","rgba(37,144,140,1)"],["0.541666666666667","rgba(33,154,138,1)"],["0.583333333333333","rgba(39,164,133,1)"],["0.625","rgba(47,174,127,1)"],["0.666666666666667","rgba(53,183,121,1)"],["0.708333333333333","rgba(79,191,110,1)"],["0.75","rgba(98,199,98,1)"],["0.791666666666667","rgba(119,207,85,1)"],["0.833333333333333","rgba(147,214,70,1)"],["0.875","rgba(172,220,52,1)"],["0.916666666666667","rgba(199,225,42,1)"],["0.958333333333333","rgba(226,228,40,1)"],["1","rgba(253,231,37,1)"]],"showscale":true,"x":[0.001,0.25498,0.50896,0.76294,1.01692,1.2709,1.52488,1.77886,2.03284,2.28682,2.5408,2.79478,3.04876,3.30274,3.55672,3.8107,4.06468,4.31866,4.57264,4.82662,5.0806,5.33458,5.58856,5.84254,6.09652,6.3505,6.60448,6.85846,7.11244,7.36642,7.6204,7.87438,8.12836,8.38234,8.63632,8.8903,9.14428,9.39826,9.65224,9.90622,10.1602,10.41418,10.66816,10.92214,11.17612,11.4301,11.68408,11.93806,12.19204,12.44602,12.7],"y":[0.001,0.25498,0.50896,0.76294,1.01692,1.2709,1.52488,1.77886,2.03284,2.28682,2.5408,2.79478,3.04876,3.30274,3.55672,3.8107,4.06468,4.31866,4.57264,4.82662,5.0806,5.33458,5.58856,5.84254,6.09652,6.3505,6.60448,6.85846,7.11244,7.36642,7.6204,7.87438,8.12836,8.38234,8.63632,8.8903,9.14428,9.39826,9.65224,9.90622,10.1602,10.41418,10.66816,10.92214,11.17612,11.4301,11.68408,11.93806,12.19204,12.44602,12.7],"z":[[0.0331322070546811,0.0318695963697909,0.0306084275995855,0.0293443616254352,0.028072629395297,0.0267906480987916,0.0255006756921525,0.0242120268543572,0.0229422198561734,0.0217163929495041,0.0205645744597996,0.0195170186693932,0.0185987116402154,0.0178248600414893,0.0171991275937116,0.0167153708708982,0.0163621251457203,0.016127955920449,0.0160055878036825,0.0159933918024623,0.0160938921540552,0.0163099523455379,0.0166398999476985,0.0170729652007625,0.0175861660276314,0.0181434002723871,0.018697160461305,0.0191929524421927,0.0195760566546932,0.0197996745792023,0.0198329265466436,0.019666951979124,0.0193177557384191,0.0188253291304183,0.0182494927870589,0.0176634036179944,0.0171456377144425,0.0167714924650897,0.0166040332006384,0.0166856083812501,0.0170309009009039,0.0176227077845935,0.0184112786443383,0.0193172496887705,0.0202373545430986,0.0210516141248012,0.0216308185565193,0.0218436758463872,0.0215636271123677,0.0206756173712682,0.0190828850096431],[0.0318695963697909,0.031511328351703,0.0310844213553877,0.0305886167447588,0.0300221313024635,0.0293838087120821,0.0286753652366703,0.0279034036844076,0.0270807724091334,0.0262268026862442,0.0253660533977032,0.0245255219649181,0.0237308522495712,0.0230026783208871,0.0223544820291294,0.0217928829847496,0.0213202272841608,0.0209382638901337,0.0206512098636519,0.0204668224419947,0.0203949617226654,0.0204440719199803,0.0206166411527903,0.0209048771066281,0.0212876675483224,0.0217295972350372,0.0221825046990564,0.0225897571489521,0.0228929803263033,0.0230403620782594,0.0229950257762506,0.0227416870288337,0.0222901364657058,0.0216749595037037,0.0209518737204194,0.0201916473732674,0.0194725899040552,0.0188723116905523,0.0184592270070578,0.0182843430388284,0.0183741443774614,0.0187255391079382,0.0193036144352677,0.0200423581397086,0.0208478281555037,0.021602876289728,0.0221726533066038,0.0224106243518809,0.0221653460138802,0.0212884113617055,0.0196435736264059],[0.0306084275995855,0.0310844213553877,0.0314203443157761,0.0316211732112498,0.0316899997689659,0.0316296698778145,0.0314444829686355,0.031141735926594,0.0307328651729079,0.0302339329167516,0.0296652411030064,0.0290499875995216,0.0284121506968899,0.0277741613522838,0.0271552007357069,0.026570865378461,0.0260343645616434,0.0255586067552207,0.0251579795896333,0.0248486796578694,0.0246470601734909,0.024566269920893,0.0246120535026586,0.0247787839835087,0.0250466839220888,0.0253809574004011,0.0257333265645521,0.0260462056967514,0.0262593329966311,0.0263180703825426,0.0261819359281641,0.025831592457205,0.0252727742295653,0.0245364639953456,0.0236756287150415,0.0227594766668228,0.0218662830175306,0.0210755270415093,0.0204597776663962,0.0200767218002543,0.019961924623402,0.0201230862453322,0.0205364563355298,0.0211456469793341,0.0218625706246616,0.0225699549560327,0.0231250360536958,0.0233645023153083,0.023111210667939,0.0221832362790041,0.0204052535287975],[0.0293443616254352,0.0305886167447588,0.0316211732112498,0.0324527742841928,0.0330926122589903,0.033549427345952,0.0338325744107203,0.0339529229691742,0.0339234846317267,0.0337597059483445,0.0334794012503232,0.0331023272155123,0.0326494526229137,0.0321420923387922,0.0316012214453457,0.031047322714557,0.0305008998002475,0.02998334029036,0.0295174074411659,0.0291265943169415,0.0288329729453783,0.0286537846443632,0.0285975020695321,0.0286602597534576,0.0288234546599859,0.0290531349281225,0.0293016258764072,0.0295116403872773,0.0296227592326892,0.0295795907918275,0.0293402725455583,0.028883589112671,0.0282131669338688,0.0273579821762354,0.0263694175927364,0.0253158112109128,0.0242755760487848,0.0233296661615941,0.022553794222206,0.0220106712168221,0.0217426696601811,0.0217654972684213,0.0220634583583041,0.0225866023071851,0.0232496861286319,0.0239327005742675,0.0244828996383591,0.0247187370588077,0.0244365136706508,0.0234204805254652,0.021456414903528],[0.028072629395297,0.0300221313024635,0.0316899997689659,0.0330926122589903,0.0342457668280061,0.035165226524543,0.035867100741192,0.036367996968833,0.0366849499816742,0.0368352230856204,0.0368361319229153,0.036705028868662,0.0364595123559421,0.0361178439234699,0.0356995164882531,0.0352259038660699,0.0347208562860593,0.0342109531388896,0.0337249828906253,0.0332922721567286,0.0329398044529304,0.0326885091584365,0.0325494155058391,0.0325204245478211,0.0325843242153551,0.0327085147322396,0.0328467995228084,0.032943463587786,0.0329395682686644,0.0327808689153244,0.0324261328975807,0.0318542158157446,0.0310683701757549,0.0300969734622301,0.0289908442452867,0.0278180542076627,0.0266573229710324,0.0255907851168261,0.0246965027479584,0.0240408889623787,0.0236712848970544,0.023609124657479,0.0238441881948054,0.0243302842342403,0.0249824555037043,0.0256757120523867,0.0262455376714748,0.0264908882983998,0.0261807728004444,0.0250653643770753,0.0228917047983721],[0.0267906480987916,0.0293838087120821,0.0316296698778145,0.033549427345952,0.035165226524543,0.0365001589179228,0.0375779327997819,0.0384222209715625,0.0390557934481177,0.0394996564073907,0.0397724863651319,0.0398906214993328,0.039868742804271,0.0397211907255714,0.0394636825816669,0.0391150591393827,0.0386985955588714,0.0382423748739331,0.0377783067772102,0.0373396493141163,0.0369573124635487,0.0366556042743079,0.0364482293719683,0.0363352361293477,0.0363013778603103,0.0363161774963071,0.0363359156859208,0.0363077011780925,0.0361755685166264,0.0358880933379644,0.0354064210283272,0.03471116976732,0.0338067269710181,0.0327221039924163,0.0315084612945201,0.0302341663872877,0.0289784558548719,0.0278244848982743,0.0268521019279102,0.0261304237510619,0.0257103226997689,0.0256171332160939,0.0258440117422347,0.0263463244850819,0.0270372918356324,0.0277851146244285,0.0284120989309966,0.0286967926700521,0.028380506127086,0.0271793781075005,0.0248021151327367],[0.0255006756921525,0.0286753652366703,0.0314444829686355,0.0338325744107203,0.035867100741192,0.0375779327997819,0.0389963219307895,0.0401534282277709,0.0410786174610846,0.0417978607923005,0.0423326308575331,0.0426996401654115,0.0429116121443023,0.0429790556104094,0.0429127816213682,0.0427266795680036,0.0424400690487865,0.0420788384185495,0.0416747176013895,0.0412625019030786,0.0408757045452815,0.0405416265908022,0.0402769245879492,0.040084463293372,0.0399518369263898,0.0398516815407925,0.0397438379186396,0.039579424852938,0.0393067502032849,0.0388786081930561,0.0382599652836571,0.0374346055545078,0.0364093221534189,0.0352148242199604,0.0339034295561093,0.0325443539260933,0.0312176325491698,0.0300074319152829,0.0289950491331545,0.0282515992223047,0.0278303977209859,0.0277592460923794,0.0280330007461299,0.0286068255988149,0.0293904679760304,0.0302439668339692,0.0309755461573968,0.0313429736802349,0.0310600244238471,0.0298094247004417,0.0272624847552774],[0.0242120268543572,0.0279034036844076,0.031141735926594,0.0339529229691742,0.036367996968833,0.0384222209715625,0.0401534282277709,0.0415997646370282,0.0427971577911487,0.0437769617221728,0.0445642600859563,0.0451772221872844,0.0456277189660057,0.04592318372548,0.0460694929360384,0.0460744279739409,0.0459510134802218,0.0457197707341108,0.0454089123023939,0.045051980609218,0.0446833151434125,0.0443325374328273,0.0440194809255232,0.0437506202194773,0.0435174524559551,0.0432968591513079,0.0430533531143708,0.0427431422914126,0.0423198800384787,0.041741670224981,0.0409784095422716,0.0400181407658664,0.0388710841047306,0.0375705432865608,0.0361707306601078,0.034742269401235,0.0333663571800869,0.0321283085190226,0.0311107307504811,0.0303862751998386,0.0300098953209992,0.0300107465165768,0.0303840689693109,0.0310834720514554,0.0320140457697783,0.0330268552853245,0.0339157681373235,0.0344181265782517,0.0342211476124548,0.0329756318229061,0.0303172817711952],[0.0229422198561734,0.0270807724091334,0.0307328651729079,0.0339234846317267,0.0366849499816742,0.0390557934481177,0.0410786174610846,0.0427971577911487,0.0442529887701707,0.0454824432758555,0.0465143235143619,0.0473688329525233,0.0480579198058665,0.048586988191066,0.0489577712307832,0.0491720297546636,0.0492355165620847,0.0491612917511296,0.0489712182000815,0.0486947136249981,0.0483647338695585,0.0480120695132449,0.0476596314143326,0.047318167274733,0.0469841324033539,0.0466397997562264,0.0462554189796303,0.04579321924752,0.045213025409274,0.0444790265391244,0.0435668192610594,0.042469482036325,0.0412014299600058,0.0397992897434228,0.0383198276984895,0.0368356380231976,0.035429515100045,0.0341881768360146,0.0331955548265636,0.0325255539276368,0.0322341671793313,0.032351035192074,0.0328707640014087,0.0337444298930393,0.0348717546066247,0.0360946142218224,0.0371929840285189,0.0378850222502866,0.0378333851627186,0.0366595435632467,0.033966503058887],[0.0217163929495041,0.0262268026862442,0.0302339329167516,0.0337597059483445,0.0368352230856204,0.0394996564073907,0.0417978607923005,0.0437769617221728,0.0454824432758555,0.0469544296254086,0.0482248415510343,0.0493159068889525,0.0502401949657233,0.0510020717135082,0.051600337508263,0.0520317811264457,0.0522952875855835,0.0523958186937653,0.052347151470331,0.0521721525117859,0.0518999967310636,0.051560940778863,0.0511802713868282,0.0507732101226357,0.0503419353831115,0.0498750765127259,0.0493495297935976,0.0487342891617836,0.0479959315171738,0.0471052125440611,0.0460438847500474,0.0448105437488646,0.0434243246084322,0.0419257373646328,0.0403746734207767,0.0388462433236286,0.0374253059640954,0.0362003045305237,0.0352565956086472,0.0346691587521296,0.0344945578485685,0.0347622238454742,0.0354653549447156,0.0365518603713253,0.0379158575355582,0.0393904494187039,0.040742988478514,0.0416746727400368,0.0418267364762657,0.0407951715946082,0.0381544909245256],[0.0205645744597996,0.0253660533977032,0.0296652411030064,0.0334794012503232,0.0368361319229153,0.0397724863651319,0.0423326308575331,0.0445642600859563,0.0465143235143619,0.0482248415510343,0.0497295966756182,0.0510522508284168,0.0522060672925228,0.0531950813066915,0.0540164141691486,0.0546634615566294,0.0551297349076588,0.0554129627094953,0.055518616756928,0.0554616372084989,0.0552653191753962,0.0549572900391486,0.0545637429668059,0.0541037683390233,0.053585385320088,0.053004092967106,0.0523440333779606,0.051581464630932,0.0506900638309376,0.0496473902063883,0.0484415545052407,0.0470769076000458,0.0455776199433563,0.0439884858786634,0.0423729877906212,0.0408092384462426,0.0393846008910634,0.038189555773451,0.0373109890085399,0.0368247986457371,0.0367877087338195,0.0372283639033316,0.038137988574964,0.0394610180011997,0.0410862019494618,0.0428389243668108,0.0444759976719848,0.0456848692074931,0.0460896282272149,0.0452658894575249,0.0427651789303845],[0.0195170186693932,0.0245255219649181,0.0290499875995216,0.0331023272155123,0.036705028868662,0.0398906214993328,0.0426996401654115,0.0451772221872844,0.0473688329525233,0.0493159068889525,0.0510522508284168,0.0526018411772825,0.0539782445988262,0.0551855000657116,0.056220099760657,0.0570737433973107,0.0577366805901296,0.058201457900842,0.0584665898396193,0.0585392042675024,0.0584355150619547,0.0581784704229294,0.0577930229923928,0.0573005020972329,0.0567138801987863,0.0560352553140835,0.0552560839383807,0.054360049409957,0.053328049662346,0.0521445050236724,0.0508039225309999,0.0493164881911971,0.0477115747013433,0.0460385277666891,0.0443647670349988,0.04277178250366,0.0413497730546252,0.0401914639491365,0.0393852824208924,0.0390078286098411,0.0391155741302234,0.0397358852831468,0.0408576437369137,0.0424218328801814,0.0443125440979638,0.046349115163758,0.0482806604983484,0.0497849724381056,0.0504742642018614,0.0499099438517527,0.0476271646380002],[0.0185987116402154,0.0237308522495712,0.0284121506968899,0.0326494526229137,0.0364595123559421,0.039868742804271,0.0429116121443023,0.0456277189660057,0.0480579198058665,0.0502401949657233,0.0522060672925228,0.0539782445988262,0.0555697905486779,0.0569847184814562,0.0582196437939824,0.0592661127814069,0.0601133755308818,0.0607514927861801,0.0611745694532348,0.0613835618155174,0.061387750097617,0.0612040069372007,0.0608536489163157,0.0603576672737655,0.0597318805962031,0.058983602697914,0.0581108584551825,0.0571044154250173,0.0559522457302509,0.0546455605943084,0.0531852375642895,0.0515873300079298,0.0498865216933156,0.0481368933504082,0.0464100300839153,0.0447910174000128,0.043373031408999,0.0422510429655798,0.041514840834889,0.0412413717499742,0.0414863944493764,0.042275579018315,0.0435953084654497,0.0453834893143447,0.0475207426909342,0.0498226133977412,0.0520340121657551,0.0538278673371518,0.0548105000575356,0.0545360015284652,0.052530482460023],[0.0178248600414893,0.0230026783208871,0.0277741613522838,0.0321420923387922,0.0361178439234699,0.0397211907255714,0.0429790556104094,0.04592318372548,0.048586988191066,0.0510020717135082,0.0531950813066915,0.0551855000657116,0.0569847184814562,0.0585963735587847,0.0600176639883751,0.061241256945453,0.062257490072733,0.0630567248468819,0.0636317684888794,0.0639801498475518,0.0641057577250964,0.0640191546452811,0.0637360515952995,0.063274057972641,0.062648625762394,0.0618696058873551,0.0609397352232774,0.0598557726821016,0.0586122210548296,0.0572068762776654,0.0556469736621194,0.0539545341813076,0.0521697166573959,0.0503515180409112,0.0485758316043286,0.0469313817098358,0.0455142129926303,0.0444212530086321,0.0437431948848057,0.0435567709287507,0.0439164860155415,0.0448459744950298,0.0463292085754549,0.0483017799036967,0.0506425125125869,0.0531659366943929,0.0556167609033422,0.0576682820873136,0.0589272621375833,0.0589476194755151,0.0572539355838214],[0.0171991275937116,0.0223544820291294,0.0271552007357069,0.0316012214453457,0.0356995164882531,0.0394636825816669,0.0429127816213682,0.0460694929360384,0.0489577712307832,0.051600337508263,0.0540164141691486,0.056220099760657,0.0582196437939824,0.0600176639883751,0.0616121307218852,0.0629978135304456,0.0641678843030938,0.0651154760881114,0.0658351243262892,0.0663240697163271,0.0665833174235592,0.0666181657746142,0.0664378041517355,0.0660537395296178,0.0654772992279524,0.0647170520509045,0.0637773114496324,0.0626586901024346,0.0613610332114099,0.0598882547154044,0.0582539468743659,0.0564863485347136,0.0546314284187607,0.0527533869635769,0.050932558636991,0.0492612081923167,0.0478378833053571,0.0467608548245097,0.0461209398623877,0.0459938501190088,0.0464321946920124,0.0474573139732284,0.0490511191524726,0.0511480492320706,0.0536272691498109,0.0563055137478691,0.0589316217540477,0.0611846483990364,0.062678081017518,0.0629725651941309,0.0615982752030172],[0.0167153708708982,0.0217928829847496,0.026570865378461,0.031047322714557,0.0352259038660699,0.0391150591393827,0.0427266795680036,0.0460744279739409,0.0491720297546636,0.0520317811264457,0.0546634615566294,0.0570737433973107,0.0592661127814069,0.061241256945453,0.0629978135304456,0.0645333100972503,0.0658450782765156,0.0669309523068001,0.0677896675792781,0.068421012364542,0.0688258673468949,0.0690062249473209,0.0689651310582874,0.0687063639849809,0.0682337313746592,0.0675501867353708,0.0666573746184522,0.0655563892916451,0.0642502577019007,0.0627480047056168,0.0610694440316023,0.0592494193188867,0.0573402919545905,0.0554119769009098,0.0535494934426646,0.0518485076341788,0.0504095219009486,0.0493312595848625,0.0487035830371259,0.0486001403838893,0.0490708988766152,0.0501347147520288,0.051772022780174,0.0539176208888705,0.0564535281944212,0.0592021990009788,0.0619210517798117,0.0643001544683514,0.0659655850158709,0.0664909288536984,0.0654181840902741],[0.0163621251457203,0.0213202272841608,0.0260343645616434,0.0305008998002475,0.0347208562860593,0.0386985955588714,0.0424400690487865,0.0459510134802218,0.0492355165620847,0.0522952875855835,0.0551297349076588,0.0577366805901296,0.0601133755308818,0.062257490072733,0.0641678843030938,0.0658450782765156,0.0672913751291343,0.0685105823250321,0.0695073140434355,0.070285971913449,0.0708496397913598,0.0711992024309695,0.0713329478099026,0.0712467515840892,0.0709347669694883,0.070390491316937,0.069608214532828,0.0685850591094622,0.0673238537979696,0.0658367880766665,0.0641492850889093,0.0623031293376317,0.0603578749677877,0.0583899685967409,0.0564896130930861,0.0547558642479761,0.0532906154317949,0.0521920250661299,0.0515477453532323,0.0514281618677676,0.051879779055568,0.0529188175908524,0.0545249656843273,0.0566350945847835,0.0591367698553657,0.061861739126727,0.064580298867567,0.0669983577770905,0.068759722155404,0.0694561217415781,0.0686463861879824],[0.016127955920449,0.0209382638901337,0.0255586067552207,0.02998334029036,0.0342109531388896,0.0382423748739331,0.0420788384185495,0.0457197707341108,0.0491612917511296,0.0523958186937653,0.0554129627094953,0.058201457900842,0.0607514927861801,0.0630567248468819,0.0651154760881114,0.0669309523068001,0.0685105823250321,0.069864660771131,0.0710044800140192,0.0719401604443124,0.0726784629233871,0.0732209509670114,0.0735629059229249,0.0736933452404863,0.0735963371236817,0.0732535753663923,0.0726479601160189,0.0717678054186407,0.0706112552626868,0.0691904418657728,0.0675348095869117,0.065692947907517,0.0637323921756593,0.0617372112051954,0.0598036597815166,0.0580345032357235,0.0565326927574597,0.0553949301746461,0.0547054523864568,0.0545301977218941,0.0549113968697175,0.0558625014438278,0.057363196237199,0.0593541128195264,0.0617309347674406,0.0643380045051197,0.0669623185873697,0.0693297398868033,0.0711059789698952,0.0719049280351492,0.0713059001829667],[0.0160055878036825,0.0206512098636519,0.0251579795896333,0.0295174074411659,0.0337249828906253,0.0377783067772102,0.0416747176013895,0.0454089123023939,0.0489712182000815,0.052347151470331,0.055518616756928,0.0584665898396193,0.0611745694532348,0.0636317684888794,0.0658351243262892,0.0677896675792781,0.0695073140434355,0.0710044800140192,0.0722990125927399,0.0734068810914066,0.0743390128301715,0.0750986315464315,0.0756794756670089,0.076065315001788,0.0762311856672073,0.0761466265710056,0.075780857900844,0.0751093385037901,0.0741206534827394,0.0728224358154073,0.0712451420777093,0.0694429483429996,0.067491651440042,0.0654840517630202,0.0635236807961097,0.0617178308359329,0.060170681698369,0.0589770306205514,0.0582168546282454,0.0579507298666269,0.0582159700752847,0.0590231699298102,0.0603526522165691,0.0621502240248153,0.0643218025811392,0.0667269834018504,0.0691724614632012,0.0714071769938292,0.0731217847031828,0.0739551043288354,0.0735092539496809],[0.0159933918024623,0.0204668224419947,0.0248486796578694,0.0291265943169415,0.0332922721567286,0.0373396493141163,0.0412625019030786,0.045051980609218,0.0486947136249981,0.0521721525117859,0.0554616372084989,0.0585392042675024,0.0613835618155174,0.0639801498475518,0.0663240697163271,0.068421012364542,0.070285971913449,0.0719401604443124,0.0734068810914066,0.0747071348803353,0.0758555631684153,0.0768571240245939,0.0777047859678467,0.0783785506966801,0.0788462522396025,0.0790666738538563,0.0789953500438669,0.0785928135615148,0.0778340945249359,0.0767174138310883,0.0752698128492905,0.0735482048684632,0.0716357088710055,0.0696344276166984,0.0676564827666857,0.0658150176284293,0.0642163197260308,0.0629535863969765,0.0621023879018121,0.061717597256679,0.0618313585729791,0.0624514729788767,0.0635594121154091,0.0651071451363291,0.0670122289257222,0.0691512267404509,0.0713524169495847,0.0733897236751991,0.0749805227340939,0.0757900605957289,0.075444345101303],[0.0160938921540552,0.0203949617226654,0.0246470601734909,0.0288329729453783,0.0329398044529304,0.0369573124635487,0.0408757045452815,0.0446833151434125,0.0483647338695585,0.0518999967310636,0.0552653191753962,0.0584355150619547,0.061387750097617,0.0641057577250964,0.0665833174235592,0.0688258673468949,0.0708496397913598,0.0726784629233871,0.0743390128301715,0.0758555631684153,0.0772451452905528,0.0785136830202586,0.0796533474708173,0.0806412705413001,0.0814399083834682,0.0819996339966568,0.0822642710045249,0.0821798978394456,0.081706164761703,0.0808278827214659,0.0795636374776064,0.0779685686437122,0.0761303232849857,0.0741595405776082,0.0721777216884198,0.070305399846015,0.0686525563853742,0.0673120344566068,0.0663558236564226,0.0658336118704106,0.0657727555273533,0.0661786504479392,0.0670343867389329,0.0682986586983935,0.0699012934552287,0.0717364804592963,0.0736547254506154,0.0754555175425166,0.0768834080097065,0.0776303138573719,0.0773460695850045],[0.0163099523455379,0.0204440719199803,0.024566269920893,0.0286537846443632,0.0326885091584365,0.0366556042743079,0.0405416265908022,0.0443325374328273,0.0480120695132449,0.051560940778863,0.0549572900391486,0.0581784704229294,0.0612040069372007,0.0640191546452811,0.0666181657746142,0.0690062249473209,0.0711992024309695,0.0732209509670114,0.0750986315464315,0.0768571240245939,0.0785136830202586,0.0800736658544262,0.081527673681703,0.0828501372891118,0.0839994229280531,0.084919882111281,0.0855466525199673,0.085813999688535,0.0856671466414864,0.0850758390493886,0.0840460870116161,0.0826259658085248,0.080902954094887,0.0789934166472068,0.0770276203127833,0.0751345617927553,0.0734298575278577,0.072008113536416,0.0709396406065027,0.0702704963000985,0.0700244602327785,0.0702054224577551,0.0707986936857298,0.0717699947986146,0.0730614264944729,0.0745845358079433,0.0762115637406577,0.0777669010031268,0.079021472046981,0.0796929205297642,0.0794537853683119],[0.0166398999476985,0.0206166411527903,0.0246120535026586,0.0285975020695321,0.0325494155058391,0.0364482293719683,0.0402769245879492,0.0440194809255232,0.0476596314143326,0.0511802713868282,0.0545637429668059,0.0577930229923928,0.0608536489163157,0.0637360515952995,0.0664378041517355,0.0689651310582874,0.0713329478099026,0.0735629059229249,0.0756794756670089,0.0777047859678467,0.0796533474708173,0.081527673681703,0.0833153353332167,0.0849875216533874,0.0864990551751751,0.0877900769011238,0.0887900878799529,0.0894252995595545,0.0896298066760721,0.0893596351395245,0.0886065868477193,0.0874072786352219,0.0858433748977925,0.0840320495417437,0.0821095463954886,0.0802129743792947,0.0784651138009087,0.0769648698729929,0.0757836661338225,0.0749665006787499,0.0745356841725799,0.0744951355422329,0.0748332970184869,0.0755231887018097,0.0765188450798398,0.0777482904039718,0.0791041891881894,0.0804342146468295,0.0815338496952199,0.0821445328592555,0.0819594966089163],[0.0170729652007625,0.0209048771066281,0.0247787839835087,0.0286602597534576,0.0325204245478211,0.0363352361293477,0.040084463293372,0.0437506202194773,0.047318167274733,0.0507732101226357,0.0541037683390233,0.0573005020972329,0.0603576672737655,0.063274057972641,0.0660537395296178,0.0687063639849809,0.0712467515840892,0.0736933452404863,0.076065315001788,0.0783785506966801,0.0806412705413001,0.0828501372891118,0.0849875216533874,0.0870201393595641,0.0888990586348729,0.0905611958981878,0.0919328005582537,0.0929357782139176,0.0934975781889819,0.093564362644805,0.0931152728196424,0.0921736423997681,0.0908105091740811,0.08913783903358,0.0872928194407002,0.0854180751676755,0.0836436352676376,0.0820748008423383,0.0807872298262874,0.0798281487438137,0.0792212833346261,0.0789727505795509,0.0790754419614093,0.0795101149518328,0.0802423419962743,0.081215495753131,0.0823409428008623,0.0834874856536821,0.0844727420062443,0.0850593943383111,0.0849588020344645],[0.0175861660276314,0.0212876675483224,0.0250466839220888,0.0288234546599859,0.0325843242153551,0.0363013778603103,0.0399518369263898,0.0435174524559551,0.0469841324033539,0.0503419353831115,0.053585385320088,0.0567138801987863,0.0597318805962031,0.062648625762394,0.0654772992279524,0.0682337313746592,0.0709347669694883,0.0735963371236817,0.0762311856672073,0.0788462522396025,0.0814399083834682,0.0839994229280531,0.0864990551751751,0.0888990586348729,0.0911457744978129,0.0931730359723613,0.0949053043926594,0.096263180544068,0.0971719073718473,0.0975728659467336,0.097436692815716,0.0967748746785305,0.0956455762095412,0.0941502865248955,0.0924209006149407,0.0906006525186964,0.0888246222175502,0.0872050955894278,0.0858245059766227,0.0847356952841487,0.0839671130397174,0.0835297202332216,0.0834225616148196,0.0836348243437206,0.0841433537785074,0.0849057710511616,0.0858503692034513,0.0868648084228072,0.0877862633440928,0.0883959647032726,0.0884207553569314],[0.0181434002723871,0.0217295972350372,0.0253809574004011,0.0290531349281225,0.0327085147322396,0.0363161774963071,0.0398516815407925,0.0432968591513079,0.0466397997562264,0.0498750765127259,0.053004092967106,0.0560352553140835,0.058983602697914,0.0618696058873551,0.0647170520509045,0.0675501867353708,0.070390491316937,0.0732535753663923,0.0761466265710056,0.0790666738538563,0.0819996339966568,0.084919882111281,0.0877900769011238,0.0905611958981878,0.0931730359723613,0.0955556456009848,0.0976322345247785,0.0993240829971134,0.100557818214925,0.101275000146099,0.101443110451299,0.101065832101195,0.100189455009278,0.0989022326285972,0.097325211198836,0.0955960861991019,0.0938504714530358,0.0922059136606917,0.0907525481483566,0.0895514382434187,0.0886389000624342,0.0880335342703454,0.0877424884744388,0.0877643064447572,0.0880870380626791,0.0886816214752877,0.0894916606878204,0.0904215758088382,0.0913257358299202,0.0920015175738187,0.0921890192445248],[0.018697160461305,0.0221825046990564,0.0257333265645521,0.0293016258764072,0.0328467995228084,0.0363359156859208,0.0397438379186396,0.0430533531143708,0.0462554189796303,0.0493495297935976,0.0523440333779606,0.0552560839383807,0.0581108584551825,0.0609397352232774,0.0637773114496324,0.0666573746184522,0.069608214532828,0.0726479601160189,0.075780857900844,0.0789953500438669,0.0822642710045249,0.0855466525199673,0.0887900878799529,0.0919328005582537,0.0949053043926594,0.0976322345247785,0.10003516025407,0.102036986984579,0.103568146491787,0.104574317256765,0.105024910606696,0.104920958426604,0.104300434195248,0.103238788654741,0.101843106124674,0.10024003194464,0.098559962311594,0.0969216927634622,0.0954216709187161,0.0941300950979937,0.093093363656403,0.0923401727443969,0.091887716123396,0.0917449588334424,0.0919112734279505,0.0923701939418145,0.0930792672556463,0.0939578908043491,0.0948756865613284,0.0956443435828908,0.0960157406076288],[0.0191929524421927,0.0225897571489521,0.0260462056967514,0.0295116403872773,0.032943463587786,0.0363077011780925,0.039579424852938,0.0427431422914126,0.04579321924752,0.0487342891617836,0.051581464630932,0.054360049409957,0.0571044154250173,0.0598557726821016,0.0626586901024346,0.0655563892916451,0.0685850591094622,0.0717678054186407,0.0751093385037901,0.0785928135615148,0.0821798978394456,0.085813999688535,0.0894252995595545,0.0929357782139176,0.096263180544068,0.0993240829971134,0.102036986984579,0.104326290372479,0.106127404614697,0.10739266324478,0.108097273867193,0.108244393823937,0.10786831546042,0.107034659498267,0.105836563213491,0.104386463396965,0.102804375737443,0.101205133078558,0.0996879074202355,0.0983307132578995,0.0971905928021063,0.0963078613964445,0.0957113408970778,0.0954214646357943,0.0954491972577069,0.0957901778901475,0.0964148164781206,0.0972560718556443,0.0981973626594604,0.0990635044577494,0.0996175316874479],[0.0195760566546932,0.0228929803263033,0.0262593329966311,0.0296227592326892,0.0329395682686644,0.0361755685166264,0.0393067502032849,0.0423198800384787,0.045213025409274,0.0479959315171738,0.0506900638309376,0.053328049662346,0.0559522457302509,0.0586122210548296,0.0613610332114099,0.0642502577019007,0.0673238537979696,0.0706112552626868,0.0741206534827394,0.0778340945249359,0.081706164761703,0.0856671466414864,0.0896298066760721,0.0934975781889819,0.0971719073718473,0.100557818214925,0.103568146491787,0.106127404614697,0.108175829322502,0.109673444920823,0.110603525948291,0.110974840307782,0.110822298930238,0.110205812197299,0.109207117067596,0.107924291723597,0.106464007765531,0.104932444018865,0.103426784755773,0.102029527482395,0.100806902172034,0.0998109018765983,0.0990827757904326,0.0986552350119151,0.0985512045964603,0.0987782095222593,0.0993188002335625,0.10011850049598,0.101073565803754,0.102021357898257,0.102736190883306],[0.0197996745792023,0.0230403620782594,0.0263180703825426,0.0295795907918275,0.0327808689153244,0.0358880933379644,0.0388786081930561,0.041741670224981,0.0444790265391244,0.0471052125440611,0.0496473902063883,0.0521445050236724,0.0546455605943084,0.0572068762776654,0.0598882547154044,0.0627480047056168,0.0658367880766665,0.0691904418657728,0.0728224358154073,0.0767174138310883,0.0808278827214659,0.0850758390493886,0.0893596351395245,0.093564362644805,0.0975728659467336,0.101275000146099,0.104574317256765,0.10739266324478,0.109673444920823,0.111383857546067,0.112515877824602,0.113085776518446,0.11313222591449,0.112713393545381,0.111903411034972,0.110788305687717,0.109461229512625,0.108016993157218,0.106546529565121,0.105132493899998,0.103847111519071,0.102752419483958,0.101901732035687,0.101340326749818,0.101103449998262,0.101210604058146,0.101656224873726,0.102397931691148,0.103344400801675,0.104345513394756,0.105187564744777],[0.0198329265466436,0.0229950257762506,0.0261819359281641,0.0293402725455583,0.0324261328975807,0.0354064210283272,0.0382599652836571,0.0409784095422716,0.0435668192610594,0.0460438847500474,0.0484415545052407,0.0508039225309999,0.0531852375642895,0.0556469736621194,0.0582539468743659,0.0610694440316023,0.0641492850889093,0.0675348095869117,0.0712451420777093,0.0752698128492905,0.0795636374776064,0.0840460870116161,0.0886065868477193,0.0931152728196424,0.097436692815716,0.101443110451299,0.105024910606696,0.108097273867193,0.110603525948291,0.112515877824602,0.113834054290717,0.114582176500992,0.114804433460968,0.114560320775369,0.113920213195428,0.112961677971017,0.111766452052585,0.1104177482942,0.108997707407521,0.107585206582271,0.106254448586631,0.105074460818783,0.104108931896927,0.103415179632419,0.103040923420831,0.103018007293712,0.103353057139428,0.104015987465594,0.104928133739159,0.105952437353606,0.106888328103055],[0.019666951979124,0.0227416870288337,0.025831592457205,0.028883589112671,0.0318542158157446,0.03471116976732,0.0374346055545078,0.0400181407658664,0.042469482036325,0.0448105437488646,0.0470769076000458,0.0493164881911971,0.0515873300079298,0.0539545341813076,0.0564863485347136,0.0592494193188867,0.0623031293376317,0.065692947907517,0.0694429483429996,0.0735482048684632,0.0779685686437122,0.0826259658085248,0.0874072786352219,0.0921736423997681,0.0967748746785305,0.101065832101195,0.104920958426604,0.108244393823937,0.110974840307782,0.113085776518446,0.114582176500992,0.115494941123467,0.115874222037705,0.115782796235101,0.115290479775698,0.114470159706538,0.113395492262777,0.112139897763217,0.110776336371563,0.109377438541185,0.108015694584173,0.10676338545883,0.10569173692745,0.104868571561808,0.104353732351723,0.104191849914758,0.104402571445975,0.10496906278508,0.105826333685744,0.106851565637909,0.107858882603496],[0.0193177557384191,0.0222901364657058,0.0252727742295653,0.0282131669338688,0.0310683701757549,0.0338067269710181,0.0364093221534189,0.0388710841047306,0.0412014299600058,0.0434243246084322,0.0455776199433563,0.0477115747013433,0.0498865216933156,0.0521697166573959,0.0546314284187607,0.0573402919545905,0.0603578749677877,0.0637323921756593,0.067491651440042,0.0716357088710055,0.0761303232849857,0.080902954094887,0.0858433748977925,0.0908105091740811,0.0956455762095412,0.100189455009278,0.104300434195248,0.10786831546042,0.110822298930238,0.11313222591449,0.114804433460968,0.115874222037705,0.116396969505733,0.116439605607111,0.116073681660397,0.115370703138586,0.114399832477675,0.113227633932602,0.111919278851983,0.11054051606788,0.109159649703249,0.107848711631506,0.106683010165943,0.105738359857973,0.105085579561552,0.104782242435146,0.104862115769381,0.105323231845509,0.106116085261734,0.107133962584367,0.108207633789301],[0.0188253291304183,0.0216749595037037,0.0245364639953456,0.0273579821762354,0.0300969734622301,0.0327221039924163,0.0352148242199604,0.0375705432865608,0.0397992897434228,0.0419257373646328,0.0439884858786634,0.0460385277666891,0.0481368933504082,0.0503515180409112,0.0527533869635769,0.0554119769009098,0.0583899685967409,0.0617372112051954,0.0654840517630202,0.0696344276166984,0.0741595405776082,0.0789934166472068,0.0840320495417437,0.08913783903358,0.0941502865248955,0.0989022326285972,0.103238788654741,0.107034659498267,0.110205812197299,0.112713393545381,0.114560320775369,0.115782796235101,0.116439605607111,0.116601695766022,0.116343710815491,0.115738304031513,0.114853358010791,0.113751798687446,0.1124934301592,0.111138036961544,0.109748815904429,0.108395027857676,0.107152714978651,0.106102529495525,0.105324178609466,0.104887590576054,0.104841501813262,0.105200685521992,0.105933497377517,0.10695177152194,0.108105193171818],[0.0182494927870589,0.0209518737204194,0.0236756287150415,0.0263694175927364,0.0289908442452867,0.0315084612945201,0.0339034295561093,0.0361707306601078,0.0383198276984895,0.0403746734207767,0.0423729877906212,0.0443647670349988,0.0464100300839153,0.0485758316043286,0.050932558636991,0.0535494934426646,0.0564896130930861,0.0598036597815166,0.0635236807961097,0.0676564827666857,0.0721777216884198,0.0770276203127833,0.0821095463954886,0.0872928194407002,0.0924209006149407,0.097325211198836,0.101843106124674,0.105836563213491,0.109207117067596,0.111903411034972,0.113920213195428,0.115290479775698,0.116073681660397,0.116343710815491,0.11617874694784,0.115654261082425,0.114839384688616,0.113796324023148,0.112582235147372,0.111252805074603,0.109866583642804,0.108488883731186,0.107193929179962,0.106064024350804,0.105184925339355,0.104637267764746,0.10448470451514,0.104760160903368,0.105452201444788,0.106493830431803,0.107755996492083],[0.0176634036179944,0.0201916473732674,0.0227594766668228,0.0253158112109128,0.0278180542076627,0.0302341663872877,0.0325443539260933,0.034742269401235,0.0368356380231976,0.0388462433236286,0.0408092384462426,0.04277178250366,0.0447910174000128,0.0469313817098358,0.0492612081923167,0.0518485076341788,0.0547558642479761,0.0580345032357235,0.0617178308359329,0.0658150176284293,0.070305399846015,0.0751345617927553,0.0802129743792947,0.0854180751676755,0.0906006525186964,0.0955960861991019,0.10024003194464,0.104386463396965,0.107924291723597,0.110788305687717,0.112961677971017,0.114470159706538,0.115370703138586,0.115738304031513,0.115654261082425,0.115197657606072,0.114440579543786,0.113446797531765,0.112273291906639,0.110973839594665,0.109603724990492,0.108224442207695,0.106907088500316,0.105733113809483,0.104791298065786,0.10417035678817,0.103947420975635,0.104173656593322,0.104859223430166,0.105960334589026,0.107371136840097],[0.0171456377144425,0.0194725899040552,0.0218662830175306,0.0242755760487848,0.0266573229710324,0.0289784558548719,0.0312176325491698,0.0333663571800869,0.035429515100045,0.0374253059640954,0.0393846008910634,0.0413497730546252,0.043373031408999,0.0455142129926303,0.0478378833053571,0.0504095219009486,0.0532906154317949,0.0565326927574597,0.060170681698369,0.0642163197260308,0.0686525563853742,0.0734298575278577,0.0784651138009087,0.0836436352676376,0.0888246222175502,0.0938504714530358,0.098559962311594,0.102804375737443,0.106464007765531,0.109461229512625,0.111766452052585,0.113395492262777,0.114399832477675,0.114853358010791,0.114839384688616,0.114440579543786,0.113732820598261,0.112782915020332,0.111649540448044,0.110386561338576,0.109047759328712,0.107691903485977,0.106386972828554,0.105212260134586,0.104257101912314,0.10361523764427,0.103374465910063,0.103602363239967,0.104330125041191,0.105537609543634,0.107142928208123],[0.0167714924650897,0.0188723116905523,0.0210755270415093,0.0233296661615941,0.0255907851168261,0.0278244848982743,0.0300074319152829,0.0321283085190226,0.0341881768360146,0.0362003045305237,0.038189555773451,0.0401914639491365,0.0422510429655798,0.0444212530086321,0.0467608548245097,0.0493312595848625,0.0521920250661299,0.0553949301746461,0.0589770306205514,0.0629535863969765,0.0673120344566068,0.072008113536416,0.0769648698729929,0.0820748008423383,0.0872050955894278,0.0922059136606917,0.0969216927634622,0.101205133078558,0.104932444018865,0.108016993157218,0.1104177482942,0.112139897763217,0.113227633932602,0.113751798687446,0.113796324023148,0.113446797531765,0.112782915020332,0.111875121276362,0.110784869490238,0.109567564507343,0.108277140617219,0.106971198287026,0.10571561697311,0.104587524853554,0.103675434780255,0.103075372474765,0.10288218522253,0.103176151056989,0.104006449881658,0.10537453878966,0.107221272697958],[0.0166040332006384,0.0184592270070578,0.0204597776663962,0.022553794222206,0.0246965027479584,0.0268521019279102,0.0289950491331545,0.0311107307504811,0.0331955548265636,0.0352565956086472,0.0373109890085399,0.0393852824208924,0.041514840834889,0.0437431948848057,0.0461209398623877,0.0487035830371259,0.0515477453532323,0.0547054523864568,0.0582168546282454,0.0621023879018121,0.0663558236564226,0.0709396406065027,0.0757836661338225,0.0807872298262874,0.0858245059766227,0.0907525481483566,0.0954216709187161,0.0996879074202355,0.103426784755773,0.106546529565121,0.108997707407521,0.110776336371563,0.111919278851983,0.1124934301592,0.112582235147372,0.112273291906639,0.111649540448044,0.110784869490238,0.10974373936335,0.108583802309521,0.107360319768912,0.106131218338772,0.104961746361344,0.103927768433203,0.103116685438074,0.102624858350806,0.102550506302654,0.102981682926133,0.103980257251917,0.105564530119846,0.107694434575614],[0.0166856083812501,0.0182843430388284,0.0200767218002543,0.0220106712168221,0.0240408889623787,0.0261304237510619,0.0282515992223047,0.0303862751998386,0.0325255539276368,0.0346691587521296,0.0368247986457371,0.0390078286098411,0.0412413717499742,0.0435567709287507,0.0459938501190088,0.0486001403838893,0.0514281618677676,0.0545301977218941,0.0579507298666269,0.061717597256679,0.0658336118704106,0.0702704963000985,0.0749665006787499,0.0798281487438137,0.0847356952841487,0.0895514382434187,0.0941300950979937,0.0983307132578995,0.102029527482395,0.105132493899998,0.107585206582271,0.109377438541185,0.11054051606788,0.111138036961544,0.111252805074603,0.110973839594665,0.110386561338576,0.109567564507343,0.108583802309521,0.107495079258876,0.10635843647708,0.105233110398018,0.104185005564433,0.10328984449144,0.102634201910419,0.102313510365311,0.102426061550053,0.103062381590745,0.104290400275489,0.106138458121161,0.108579766828597],[0.0170309009009039,0.0183741443774614,0.019961924623402,0.0217426696601811,0.0236712848970544,0.0257103226997689,0.0278303977209859,0.0300098953209992,0.0322341671793313,0.0344945578485685,0.0367877087338195,0.0391155741302234,0.0414863944493764,0.0439164860155415,0.0464321946920124,0.0490708988766152,0.051879779055568,0.0549113968697175,0.0582159700752847,0.0618313585729791,0.0657727555273533,0.0700244602327785,0.0745356841725799,0.0792212833346261,0.0839671130397174,0.0886389000624342,0.093093363656403,0.0971905928021063,0.100806902172034,0.103847111519071,0.106254448586631,0.108015694584173,0.109159649703249,0.109748815904429,0.109866583642804,0.109603724990492,0.109047759328712,0.108277140617219,0.107360319768912,0.10635843647708,0.105329925935416,0.104335476670581,0.103442203722576,0.102726300320326,0.102273609833579,0.102177504351418,0.102533352112481,0.103429024147376,0.104931622509005,0.10707193345879,0.10982959053582],[0.0176227077845935,0.0187255391079382,0.0201230862453322,0.0217654972684213,0.023609124657479,0.0256171332160939,0.0277592460923794,0.0300107465165768,0.032351035192074,0.0347622238454742,0.0372283639033316,0.0397358852831468,0.042275579018315,0.0448459744950298,0.0474573139732284,0.0501347147520288,0.0529188175908524,0.0558625014438278,0.0590231699298102,0.0624514729788767,0.0661786504479392,0.0702054224577551,0.0744951355422329,0.0789727505795509,0.0835297202332216,0.0880335342703454,0.0923401727443969,0.0963078613964445,0.0998109018765983,0.102752419483958,0.105074460818783,0.10676338545883,0.107848711631506,0.108395027857676,0.108488883731186,0.108224442207695,0.107691903485977,0.106971198287026,0.106131218338772,0.105233110398018,0.104335476670581,0.103499551655562,0.102793080777742,0.102292254661182,0.102081401508485,0.102250190761531,0.102888025057679,0.104075363901974,0.10587217683983,0.108304635908844,0.111352283115469],[0.0184112786443383,0.0193036144352677,0.0205364563355298,0.0220634583583041,0.0238441881948054,0.0258440117422347,0.0280330007461299,0.0303840689693109,0.0328707640014087,0.0354653549447156,0.038137988574964,0.0408576437369137,0.0435953084654497,0.0463292085754549,0.0490511191524726,0.051772022780174,0.0545249656843273,0.057363196237199,0.0603526522165691,0.0635594121154091,0.0670343867389329,0.0707986936857298,0.0748332970184869,0.0790754419614093,0.0834225616148196,0.0877424884744388,0.091887716123396,0.0957113408970778,0.0990827757904326,0.101901732035687,0.104108931896927,0.10569173692745,0.106683010165943,0.107152714978651,0.107193929179962,0.106907088500316,0.106386972828554,0.10571561697311,0.104961746361344,0.104185005564433,0.103442203722576,0.102793080777742,0.102304063161057,0.102049436620413,0.102109946452767,0.10256902067649,0.103506783839688,0.104992022722983,0.107072464157825,0.109764198317764,0.113041726217119],[0.0193172496887705,0.0200423581397086,0.0211456469793341,0.0225866023071851,0.0243302842342403,0.0263463244850819,0.0286068255988149,0.0310834720514554,0.0337444298930393,0.0365518603713253,0.0394610180011997,0.0424218328801814,0.0453834893143447,0.0483017799036967,0.0511480492320706,0.0539176208888705,0.0566350945847835,0.0593541128195264,0.0621502240248153,0.0651071451363291,0.0682986586983935,0.0717699947986146,0.0755231887018097,0.0795101149518328,0.0836348243437206,0.0877643064447572,0.0917449588334424,0.0954214646357943,0.0986552350119151,0.101340326749818,0.103415179632419,0.104868571561808,0.105738359857973,0.106102529495525,0.106064024350804,0.105733113809483,0.105212260134586,0.104587524853554,0.103927768433203,0.10328984449144,0.102726300320326,0.102292254661182,0.102049436620413,0.102066807188804,0.102418123127492,0.10317716528555,0.104411357542059,0.106174391548663,0.10849840217679,0.11138628362159,0.114804908845702],[0.0202373545430986,0.0208478281555037,0.0218625706246616,0.0232496861286319,0.0249824555037043,0.0270372918356324,0.0293904679760304,0.0320140457697783,0.0348717546066247,0.0379158575355582,0.0410862019494618,0.0443125440979638,0.0475207426909342,0.0506425125125869,0.0536272691498109,0.0564535281944212,0.0591367698553657,0.0617309347674406,0.0643218025811392,0.0670122289257222,0.0699012934552287,0.0730614264944729,0.0765188450798398,0.0802423419962743,0.0841433537785074,0.0880870380626791,0.0919112734279505,0.0954491972577069,0.0985512045964603,0.101103449998262,0.103040923420831,0.104353732351723,0.105085579561552,0.105324178609466,0.105184925339355,0.104791298065786,0.104257101912314,0.103675434780255,0.103116685438074,0.102634201910419,0.102273609833579,0.102081401508485,0.102109946452767,0.102418123127492,0.103068252153533,0.104120637527244,0.105627032920497,0.107624065572147,0.110127276876865,0.113126101434321,0.116579914741623],[0.0210516141248012,0.021602876289728,0.0225699549560327,0.0239327005742675,0.0256757120523867,0.0277851146244285,0.0302439668339692,0.0330268552853245,0.0360946142218224,0.0393904494187039,0.0428389243668108,0.046349115163758,0.0498226133977412,0.0531659366943929,0.0563055137478691,0.0592021990009788,0.061861739126727,0.0643380045051197,0.0667269834018504,0.0691512267404509,0.0717364804592963,0.0745845358079433,0.0777482904039718,0.081215495753131,0.0849057710511616,0.0886816214752877,0.0923701939418145,0.0957901778901475,0.0987782095222593,0.101210604058146,0.103018007293712,0.104191849914758,0.104782242435146,0.104887590576054,0.104637267764746,0.10417035678817,0.10361523764427,0.103075372474765,0.102624858350806,0.102313510365311,0.102177504351418,0.102250190761531,0.10256902067649,0.10317716528555,0.104120637527244,0.105442771078341,0.107177942564458,0.109345907566447,0.111947428967978,0.114961243178256,0.118342017148305],[0.0216308185565193,0.0221726533066038,0.0231250360536958,0.0244828996383591,0.0262455376714748,0.0284120989309966,0.0309755461573968,0.0339157681373235,0.0371929840285189,0.040742988478514,0.0444759976719848,0.0482806604983484,0.0520340121657551,0.0556167609033422,0.0589316217540477,0.0619210517798117,0.064580298867567,0.0669623185873697,0.0691724614632012,0.0713524169495847,0.0736547254506154,0.0762115637406577,0.0791041891881894,0.0823409428008623,0.0858503692034513,0.0894916606878204,0.0930792672556463,0.0964148164781206,0.0993188002335625,0.101656224873726,0.103353057139428,0.104402571445975,0.104862115769381,0.104841501813262,0.10448470451514,0.103947420975635,0.103374465910063,0.10288218522253,0.102550506302654,0.102426061550053,0.102533352112481,0.102888025057679,0.103506783839688,0.104411357542059,0.105627032920497,0.107177942564458,0.109081468818869,0.111343399732056,0.113954504237764,0.116888383585565,0.120100020384222],[0.0218436758463872,0.0224106243518809,0.0233645023153083,0.0247187370588077,0.0264908882983998,0.0286967926700521,0.0313429736802349,0.0344181265782517,0.0378850222502866,0.0416746727400368,0.0456848692074931,0.0497849724381056,0.0538278673371518,0.0576682820873136,0.0611846483990364,0.0643001544683514,0.0669983577770905,0.0693297398868033,0.0714071769938292,0.0733897236751991,0.0754555175425166,0.0777669010031268,0.0804342146468295,0.0834874856536821,0.0868648084228072,0.0904215758088382,0.0939578908043491,0.0972560718556443,0.10011850049598,0.102397931691148,0.104015987465594,0.10496906278508,0.105323231845509,0.105200685521992,0.104760160903368,0.104173656593322,0.103602363239967,0.103176151056989,0.102981682926133,0.103062381590745,0.103429024147376,0.104075363901974,0.104992022722983,0.106174391548663,0.107624065572147,0.109345907566447,0.111343399732056,0.113614179481031,0.11614652614003,0.118916692761225,0.121886586438237],[0.0215636271123677,0.0221653460138802,0.023111210667939,0.0244365136706508,0.0261807728004444,0.028380506127086,0.0310600244238471,0.0342211476124548,0.0378333851627186,0.0418267364762657,0.0460896282272149,0.0504742642018614,0.0548105000575356,0.0589272621375833,0.062678081017518,0.0659655850158709,0.068759722155404,0.0711059789698952,0.0731217847031828,0.0749805227340939,0.0768834080097065,0.079021472046981,0.0815338496952199,0.0844727420062443,0.0877862633440928,0.0913257358299202,0.0948756865613284,0.0981973626594604,0.101073565803754,0.103344400801675,0.104928133739159,0.105826333685744,0.106116085261734,0.105933497377517,0.105452201444788,0.104859223430166,0.104330125041191,0.104006449881658,0.103980257251917,0.104290400275489,0.104931622509005,0.10587217683983,0.107072464157825,0.10849840217679,0.110127276876865,0.111947428967978,0.113954504237764,0.11614652614003,0.11851894361148,0.12105993339361,0.123745862555688],[0.0206756173712682,0.0212884113617055,0.0221832362790041,0.0234204805254652,0.0250653643770753,0.0271793781075005,0.0298094247004417,0.0329756318229061,0.0366595435632467,0.0407951715946082,0.0452658894575249,0.0499099438517527,0.0545360015284652,0.0589476194755151,0.0629725651941309,0.0664909288536984,0.0694561217415781,0.0719049280351492,0.0739551043288354,0.0757900605957289,0.0776303138573719,0.0796929205297642,0.0821445328592555,0.0850593943383111,0.0883959647032726,0.0920015175738187,0.0956443435828908,0.0990635044577494,0.102021357898257,0.104345513394756,0.105952437353606,0.106851565637909,0.107133962584367,0.10695177152194,0.106493830431803,0.105960334589026,0.105537609543634,0.10537453878966,0.105564530119846,0.106138458121161,0.10707193345879,0.108304635908844,0.109764198317764,0.11138628362159,0.113126101434321,0.114961243178256,0.116888383585565,0.118916692761225,0.12105993339361,0.123328316520917,0.125720682867589],[0.0190828850096431,0.0196435736264059,0.0204052535287975,0.021456414903528,0.0228917047983721,0.0248021151327367,0.0272624847552774,0.0303172817711952,0.033966503058887,0.0381544909245256,0.0427651789303845,0.0476271646380002,0.052530482460023,0.0572539355838214,0.0615982752030172,0.0654181840902741,0.0686463861879824,0.0713059001829667,0.0735092539496809,0.075444345101303,0.0773460695850045,0.0794537853683119,0.0819594966089163,0.0849588020344645,0.0884207553569314,0.0921890192445248,0.0960157406076288,0.0996175316874479,0.102736190883306,0.105187564744777,0.106888328103055,0.107858882603496,0.108207633789301,0.108105193171818,0.107755996492083,0.107371136840097,0.107142928208123,0.107221272697958,0.107694434575614,0.108579766828597,0.10982959053582,0.111352283115469,0.113041726217119,0.114804908845702,0.116579914741623,0.118342017148305,0.120100020384222,0.121886586438237,0.123745862555688,0.125720682867589,0.127840734674854]],"type":"surface","contours":{"z":{"show":true,"usecolormap":true,"highlightcolor":"#ff0000","project":{"z":true}}},"frame":null}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.2,"selected":{"opacity":1},"debounce":0},"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>
<p>The next plot, the <a href="https://en.wikipedia.org/wiki/Modes_of_variation">Modes of Variation Plot</a> shows the modes of variation about the mean for the first two eigenfunctions. The mean function is indicated by the red line. The dark gray shows the variation of the first eigenfunction, and the light gray shows the variation of the second. The plot indicates that all of the subjects begin their careers with similar entry level wages, start to diverge by their second year, reach peak variation around year eight and then begin to converge again by year eleven.</p>
<pre class="r"><code>CreateModeOfVarPlot(res_wages, main = "Modes of Variation of Eigenvectors")</code></pre>
<p><img src="/2021/07/08/exploratory-fda-with-sparse-data/index_files/figure-html/unnamed-chunk-14-1.png" width="672" />
### Some Conclusions</p>
<p>This short post neither presents a comprehensive view of functional principal components analysis, nor does it provide the last word on the <code>wages</code> data set. Nevertheless, juxtaposing the highly visual but traditional exploratory analysis conducted by the authors of the <code>brolgar</code> package with a basic FPCA look does provide some insight into the promise and pitfalls of using FPCA to explore sparse, longitudinal data sets.</p>
<p>On the promise side:</p>
<ol style="list-style-type: decimal">
<li><p>FDA offers a global perspective that facilitates thinking about an individual subject’s <em>time path</em> as a whole. For the <code>wages</code> data set, we see individual trajectories developing in a wages/time space that can be parsimoniously represented, analyzed and compared.</p></li>
<li><p>FPCA works with very sparse data, does not require the same number of observations for each subject, and does not demand that the observations be taken at the same time points.</p></li>
<li><p>There is really no concept of missing data per se, and no need for data imputation. The amount of information required to represent a subject can vary over a wide range.</p></li>
</ol>
<p>As to the pitfalls:</p>
<ol style="list-style-type: decimal">
<li><p>As with plain old multivariate PCA, eigenvectors may not have any obvious meaning, and trajectories in an abstract space may be difficult to interpret.</p></li>
<li><p>There is really no way to avoid the missing data <em>daemon</em>. The <code>wages</code> data set shows the sensitivity of FPCA trajectories to both the number and the locations of the observed data points. It is not possible to reconstruct individual trajectories for which there are too few observations.</p></li>
</ol>
<p>That’s all for now. Thank you for following along.</p>
</div>
<script>window.location.href='https://rviews.rstudio.com/2021/07/08/exploratory-fda-with-sparse-data/';</script>
May 2021: "Top 40" New CRAN Packages
https://rviews.rstudio.com/2021/06/24/may-2021-top-40-new-cran-packages/
Thu, 24 Jun 2021 00:00:00 +0000https://rviews.rstudio.com/2021/06/24/may-2021-top-40-new-cran-packages/
<p>Two hundred five packages made it to CRAN in May, but seven were removed before this post went to print. Here are my “Top40” picks in ten categories: Computational Methods, Data, Genomics, Machine Learning, Medicine, Science, Statistics, Time Series, Utilities, and Visualization.</p>
<h3 id="computational-methods">Computational Methods</h3>
<p><a href="https://cran.r-project.org/package=madgrad">madgrad</a> v0.1.0: Implements MADGRAD, a Momentumized, Adaptive Dual Averaged Gradient method for stochastic optimization. See <a href="https://arxiv.org/abs/2101.11075">Defazio & Jelassi (2021)</a> for details and <a href="https://cran.r-project.org/web/packages/madgrad/readme/README.html">README</a> to get started.</p>
<p><img src="madgrad.gif" height = "250" width="450"></p>
<p><a href="https://cran.r-project.org/package=TriDimRegression">TriDimRegression</a> v1.0.0.0: Provides functions to fit 2D and 3D transformations using <a href="https://mc-stan.org/">Stan</a> which return posterior distributed for fitted parameters. There are vignettes on <a href="https://cran.r-project.org/web/packages/TriDimRegression/vignettes/transformation_matrices.html">Transformation Matrices</a>, <a href="https://cran.r-project.org/web/packages/TriDimRegression/vignettes/calibration.html">Eye Gaze Mapping</a>, and <a href="https://cran.r-project.org/web/packages/TriDimRegression/vignettes/comparing_faces.html">Comparing Faces</a>. See <a href="https://cran.r-project.org/web/packages/TriDimRegression/readme/README.html">README</a> to get started.</p>
<p><img src="Tri.png" height = "300" width="500"></p>
<h3 id="data">Data</h3>
<p><a href="https://cran.r-project.org/package=AtmChile">AtmChile</a> v0.1.0: Provides access to air quality and meteorological information from the Chile’s National Air Quality System <a href="https://sinca.mma.gob.cl/">(S.I.N.C.A.)</a>. See <a href="https://cran.r-project.org/web/packages/AtmChile/readme/README.html">READMW</a> to get started.</p>
<p><a href="https://cran.r-project.org/package=basemaps">basemaps</a> v0.0.1: Provides a lightweight interface to access spatial basemaps from open sources such as <a href="https://www.openstreetmap.org/#map=5/38.007/-95.844">OpenStreetMap</a>, <a href="https://www.mapbox.com/">Mapbox</a> and others. See <a href="https://cran.r-project.org/web/packages/basemaps/readme/README.html">README</a> to get started.</p>
<p><img src="basemaps.png" height = "400" width="400"></p>
<p><a href="https://cran.r-project.org/package=causaldata">causaldata</a> v0.1.1: Contains the data sets to run the example problems in the online causal inference textbooks <a href="https://theeffectbook.net/"><em>The Effect</em></a> and <a href="https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/"><em>Causal Inference: What If</em></a> and more.</p>
<p><a href="https://cran.r-project.org/package=exoplanets">exoplanets</a> v0.2.1: Provides access to NASA’s <a href="https://exoplanetarchive.ipac.caltech.edu/index.html">Exoplanet Archive</a>. See the <a href="https://cran.r-project.org/web/packages/exoplanets/vignettes/exoplanets.html">vignette</a> to get started.</p>
<p><img src="exoplanets.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=frenchdata">frenchdata</a> v0.1.1: Provides access to Kenneth’s French <a href="http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html">finance data library</a>. See the <a href="https://cran.r-project.org/web/packages/frenchdata/vignettes/basic_usage.html">vignette</a> for basic usage.</p>
<p><img src="frenchdata.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=tradepolicy">tradepolicy</a> v0.5.0: Provides access to the data sets from <a href="https://zenodo.org/record/4277741#.YNIS_TZKj0o">Yotov et al. (2016)</a> along with an <a href="https://r.tiid.org/R_structural_gravity/">online book</a> containing commentary and the code to recreate the original analysis.</p>
<p><img src="trade.png" height = "300" width="500"></p>
<h3 id="genomics">Genomics</h3>
<p><a href="https://cran.r-project.org/package=artemis">artemis</a> v1.0.7: Provides a modeling framework for the design and analysis of experiments collecting environmental DNA. There is an <a href="https://cran.r-project.org/web/packages/artemis/vignettes/artemis-overview.html">Introduction</a> and also vignettes on <a href="https://cran.r-project.org/web/packages/artemis/vignettes/modeling.html">Modeling eDNA and qPCR Data</a> and <a href="https://cran.r-project.org/web/packages/artemis/vignettes/simulation.html">Simulating eDNA Data</a>.</p>
<p><img src="artemis.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=MAGEE">MAGEE</a> v1.0.0: Provides functions to perform variant set-based main effect tests, gene-environment interaction tests, and joint tests for association, as proposed in <a href="https://onlinelibrary.wiley.com/doi/10.1002/gepi.22351">Wang et al. (2020)</a>. See the <a href="https://cran.r-project.org/web/packages/MAGEE/vignettes/MAGEE.pdf">vignette</a> for details.</p>
<p><a href="https://cran.r-project.org/package=MultIS">MultIS</a> v0.5.1: Implements a bioinformatical approach to detect the multiple integration of viral vectors within the same clone. See the <a href="https://cran.r-project.org/web/packages/MultIS/vignettes/QuickStart.html">vignette</a> for how to use the package.</p>
<p><img src="MultIS.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=TopDom">TopDom</a> v0.10.0: Provides functions to identify topological domains in genomes from Hi-C sequence data as described in <a href="https://academic.oup.com/nar/article/44/7/e70/2467818">Shin et al. (2016)</a>. See <a href="https://cran.r-project.org/web/packages/TopDom/readme/README.html">README</a> to get started.</p>
<h3 id="machine-learning">Machine Learning</h3>
<p><a href="https://cran.r-project.org/package=cjbart">cjbart</a> v0.1.0: Implements a tool for analyzing conjoint experiments using Bayesian Additive Regression Trees (BART), a machine learning method developed by <a href="https://projecteuclid.org/journals/annals-of-applied-statistics/volume-4/issue-1/BART-Bayesian-additive-regression-trees/10.1214/09-AOAS285.full">Chipman & McCulloch (2010)</a>. See the <a href="https://cran.r-project.org/web/packages/cjbart/vignettes/cjbart-demo.html">vignette</a> for examples.</p>
<p><img src="cjbart.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=fastText">fastText</a> v1.0.1: Implements an interface to Facebook’s <a href="https://github.com/facebookresearch/fastText">fastText Library</a>. See <a href="https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00051/43387/Enriching-Word-Vectors-with-Subword-Information">Bojanowski et al. (2017)</a> for a description of the algorithm. There is a <a href="https://cran.r-project.org/web/packages/fastText/vignettes/language_identification.html">Benchmark</a> vignette and an <a href="https://cran.r-project.org/web/packages/fastText/vignettes/the_fastText_R_package.html">Introduction</a>.</p>
<p>.</p>
<h3 id="medicine">Medicine</h3>
<p><a href="https://cran.r-project.org/package=afdx">afdx</a> v1.1.1: Provides functions to estimate diagnosis performance (Sensitivity, Specificity, Positive predictive value, Negative predicted value) of a diagnostic test when there is no golden standard by estimating the attributable fraction using either a <a href="https://cran.r-project.org/web/packages/afdx/vignettes/af_logit_exponential.html">logitexponential model</a> or a <a href="https://cran.r-project.org/web/packages/afdx/vignettes/latentclass.html">latent class model</a>.</p>
<p><img src="afdx.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=covidcast">covidcast</a> v0.4.2: Provides an interface to Delphi’s <a href="https://cmu-delphi.github.io/delphi-epidata/api/covidcast.html">COVIDcast Epidata</a> including tools for data access, maps and time series plotting, and basic signal processing, and a collection of numerous indicators relevant to the COVID-19 pandemic in the United States. There is a <a href="https://cran.r-project.org/web/packages/covidcast/vignettes/covidcast.html">Getting Started Guide</a>, and vignettes on <a href="https://cran.r-project.org/web/packages/covidcast/vignettes/correlation-utils.html">Computing Signal Correlations</a>, <a href="https://cran.r-project.org/web/packages/covidcast/vignettes/external-data.html">Combining Data Sources</a>, <a href="https://cran.r-project.org/web/packages/covidcast/vignettes/multi-signals.html">Manipulating Multiple Signals</a>, and <a href="https://cran.r-project.org/web/packages/covidcast/vignettes/plotting-signals.html">Plotting and Mapping Signals</a>.</p>
<p><img src="covidcast.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=eventTrack">eventTrack</a> v1.0.0: Implements the hybrid framework for event prediction in clinical trials as described in <a href="https://www.sciencedirect.com/science/article/pii/S155171441100139X?via%3Dihub">Fang & Zheng (2011)</a>. See the <a href="https://cran.r-project.org/web/packages/eventTrack/vignettes/eventTrack.html">vignette</a> for an example.</p>
<p><img src="eventTrack.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=goldilocks">goldilocks</a> v0.3.0: Implements the Goldilocks adaptive trial design for a time to event outcome using a piecewise exponential model and conjugate Gamma prior distributions as described in <a href="https://www.tandfonline.com/doi/abs/10.1080/10543406.2014.888569?journalCode=lbps20">Broglio et al. (2014)</a>. See the <a href="https://cran.r-project.org/web/packages/goldilocks/vignettes/broglio.html">vignette</a> for an example.</p>
<h3 id="science">Science</h3>
<p><a href="https://cran.r-project.org/package=CopernicusDEM">CopernicusDEM</a> v1.0.1: Provides an interface to the <a href="https://spacedata.copernicus.eu/explore-more/news-archive/-/asset_publisher/Ye8egYeRPLEs/blog/id/434960">Copernicus DEM</a> Digital Elevation Model of the European Space Agency with 90 and 30 meters resolution using the <a href="https://aws.amazon.com/cli/">AWS CLI</a> command line tool. See the <a href="https://cran.r-project.org/web/packages/CopernicusDEM/vignettes/Copernicus_Digital_Elevation_Models.html">vignette</a> for an example.</p>
<p><img src="DEM.png" height = "200" width="400"></p>
<p><a href="https://cran.r-project.org/package=nimbleCarbon">nimbleCarbon</a> v0.1.2: Provides functions and a custom probability distribution for Bayesian analyses of radiocarbon dates within the <code>nimble</code> modeling framework, including a suite of functions for prior and posterior predictive checks for demographic inference as described in <a href="https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0251695">Crema & Shoda (2021)</a>. See the <a href="https://cran.r-project.org/web/packages/nimbleCarbon/vignettes/nimble_carbon_vignette.html">Introduction</a>.</p>
<p><img src="nimbleCarbon.png" height = "300" width="500"></p>
<h3 id="statistics">Statistics</h3>
<p><a href="https://cran.r-project.org/package=bayesmodels">bayesmodels</a> v0.1.0: Implements a framework to bring a number of Bayesian models into the <code>tidymodels</code> ecosystem. See the <a href="https://cran.r-project.org/web/packages/bayesmodels/vignettes/modeltime-integration.html">vignette</a> for an overview.</p>
<p><img src="bayesmodels.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=div">div</a> v0.3.1: Provides functions to facilitate the analysis of teams in a corporate setting, assess the diversity per grade and job, search for bias and also provides methods to simulate the effects of bias. See <a href="http://www.de-brouwer.com/assets/div/div-white-paper.pdf">De Brouwer (2021)</a> and <a href="https://onlinelibrary.wiley.com/doi/book/10.1002/9781119632757">De Brouwer (2020)</a> for background. Look <a href="http://www.de-brouwer.com/div/">here</a> to get started.</p>
<p><img src="div.png" height = "200" width="400"></p>
<p><a href="https://cran.r-project.org/package=HotellingEllipse">HotellingEllipse</a> v0.1.1: Provides functions to compute the semi-axes lengths and coordinate points of Hotelling ellipse. See <a href="https://pubs.rsc.org/en/content/articlelanding/2014/AY/C3AY41907J#!divAbstract">Bro & Smilde (2014)</a> and <a href="https://analyticalsciencejournals.onlinelibrary.wiley.com/doi/full/10.1002/cem.2763">Brenton (2016)</a> for background. Look <a href="https://github.com/ChristianGoueguel/HotellingEllipse">here</a> and at the <a href="https://cran.r-project.org/web/packages/HotellingEllipse/vignettes/HotellingEllipse.html">vignette</a> for examples.</p>
<p><img src="Hottelling.png" height = "300" width="500"></p>
<p><a href="https://CRAN.R-project.org/package=makemyprior">makemyprior</a> v1.0.0: Provides tools to construct and visualize joint priors for variance parameters. Vignettes provide examples for <a href="https://cran.r-project.org/web/packages/makemyprior/vignettes/latin_square.html">Latin Square</a>, <a href="https://cran.r-project.org/web/packages/makemyprior/vignettes/make_prior.html">i.i.d. models</a>, <a href="https://cran.r-project.org/web/packages/makemyprior/vignettes/neonatal_mortality.html">neonatal mortality</a>, and <a href="https://cran.r-project.org/web/packages/makemyprior/vignettes/wheat_breeding.html">wheat breeding</a>.</p>
<p><img src="makemyprior.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=Rage">Rage</a> v1.0.0: Provides functions for calculating life history metrics using matrix population models (MPMs) as described in <a href="https://www.biorxiv.org/content/10.1101/2021.04.26.441330v2">Jones et al. (2021)</a>. There is a <a href="https://cran.r-project.org/web/packages/Rage/vignettes/a01_GettingStarted.html">Getting Started Guide</a> and vignettes on <a href="https://cran.r-project.org/web/packages/Rage/vignettes/a02_VitalRates.html">Vital Rates</a>, <a href="https://cran.r-project.org/web/packages/Rage/vignettes/a03_LifeHistoryTraits.html">Life History Traits</a>, <a href="https://cran.r-project.org/web/packages/Rage/vignettes/a04_AgeFromStage.html">Deriving Age</a>, and <a href="https://cran.r-project.org/web/packages/Rage/vignettes/a05_TernaryPlots.html">Ternary Plots</a>.</p>
<p><img src="Rage.png" height = "300" width="300"></p>
<p><a href="https://cran.r-project.org/package=unusualprofile">unusualprofile</a> v0.1.0: Provides functions to calculate <a href="https://link.springer.com/article/10.1007%2Fs13171-019-00164-5">Mahalanobis distance</a> for every row of a set of outcome variables. There is an <a href="https://cran.r-project.org/web/packages/unusualprofile/vignettes/tutorial_unusualprofile.html">Introduction</a> and a vignette on the <a href="https://cran.r-project.org/web/packages/unusualprofile/vignettes/unusualprofile_calculations.html">calculations</a>.</p>
<p><img src="unusualprofile.png" height = "200" width="400"></p>
<h3 id="time-series">Time Series</h3>
<p><a href="https://cran.r-project.org/package=gsignal">gsignal</a> v0.3-2: Implements the <a href="https://octave.sourceforge.io/packages.php">Ovtave signal</a> package which provides a variety of signal processing tools, such as signal generation and measurement, correlation and convolution, filtering, filter design, filter analysis and conversion, power spectrum analysis, system identification, decimation and sample rate change, and windowing. See the <a href="https://cran.r-project.org/web/packages/gsignal/vignettes/gsignal.html">vignette</a> for an introduction.</p>
<p><img src="gsignal.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=legion">legion</a> v0.1.0 Provides functions for implementing multivariate state space models such as Vector Exponential Smoothing and Vector Error-Trend-Seasonal models, for time series analysis and forecasting as described in <a href="https://journals.sagepub.com/doi/10.1177/1471082X0901000401">de Silva et al. (2010)</a> There is a <a href="https://cran.r-project.org/web/packages/legion/vignettes/legion.html">Function Overview</a> and vignettes on <a href="https://cran.r-project.org/web/packages/legion/vignettes/ves.html">Vector ES</a> and <a href="https://cran.r-project.org/web/packages/legion/vignettes/vets.html">Vector ETS</a>.</p>
<h3 id="utilities">Utilities</h3>
<p><a href="https://cran.r-project.org/package=parsermd">parsermd</a> v0.1.2: Implements formal grammar and parser for R Markdown documents using the <a href="https://www.boost.org/doc/libs/1_76_0/libs/spirit/doc/x3/html/index.html">Boost Spirit X3</a> library. It also includes a collection of high level functions for working with the resulting abstract syntax tree. There is a <a href="https://cran.r-project.org/web/packages/parsermd/vignettes/parsermd.html">Getting Started Guide</a> and a vignette on <a href="https://cran.r-project.org/web/packages/parsermd/vignettes/templates.html">Rmd Templates</a>.</p>
<p><a href="https://cran.r-project.org/package=riskmetric">riskmetric</a> v0.1.0: Provides facilities for assessing R packages against a number of metrics to help quantify their robustness. Look <a href="https://pharmar.github.io/riskmetric/">here</a> for background on the package and <a href="https://www.pharmar.org/about/">here</a> for background on the R Consortium, R Validation Hub project. There is a <a href="https://cran.r-project.org/web/packages/riskmetric/riskmetric.pdf">Quick Start Guide</a> and a vignette on <a href="https://cran.r-project.org/web/packages/riskmetric/vignettes/extending-riskmetric.html">Extending riskmetric</a>.</p>
<p><a href="https://cran.r-project.org/package=shinyvalidate">shinyvalidate</a> v0.1.0: Provides functions to improve the user experience of Shiny apps by providing feedback when required inputs are missing, or input values are not valid. See <a href="https://cran.r-project.org/web/packages/shinyvalidate/readme/README.html">README</a> to get started.</p>
<p><a href="https://cran.r-project.org/package=ttt">ttt</a> v1.0: Provides tools to create structured, formatted HTML tables. See the <a href="https://cran.r-project.org/web/packages/ttt/vignettes/ttt-intro.html">vignette</a>.</p>
<h3 id="visualization">Visualization</h3>
<p><a href="https://cran.r-project.org/package=fitbitViz">fitbitViz</a> v1.0.1: Implements a connection to the <a href="https://dev.fitbit.com/build/reference/web-api/">Fitbit Web API</a> to provide <code>ggplot2</code>, <code>Leaflet</code> and <code>Rayshader</code> visualizations. See the <a href="https://cran.r-project.org/web/packages/fitbitViz/vignettes/fitbit_viz.html">vignette</a> for examples.</p>
<p><img src="fitbitViz.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=ggbreak">ggbreak</a> v0.0.3: Implements scale functions for setting axis breaks for <code>ggplot2</code>. See the <a href="https://cran.r-project.org/web/packages/ggbreak/vignettes/ggbreak.html">vignette</a>.</p>
<p><img src="ggbreak.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=ggpp">ggpp</a> v0.4.0: Provides extensions to <code>ggplot2</code> to add inserts to plots using both <em>native</em> and <a href="https://www.christophenicault.com/post/npc_ggplot2/"><em>npc</em></a> data coordinates. See the <a href="https://cran.r-project.org/web/packages/ggpp/vignettes/grammar-extensions.html">vignette</a> for examples.</p>
<p><img src="ggpp.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=ggseg">ggseg</a> v1.6.3: Implements a <code>ggplot</code> geom for plotting brain atlases using simple features. The largest component of the package is the data for two built-in atlases. See <a href="https://journals.sagepub.com/doi/10.1177/2515245920928009">Mowinckel & Vidal-Piñero (2020)</a> for background. There is an <a href="https://cran.r-project.org/web/packages/ggseg/vignettes/ggseg.html">Introduction</a> along with vignettes on <a href="https://cran.r-project.org/web/packages/ggseg/vignettes/externalData.html">external data</a>, <a href="https://cran.r-project.org/web/packages/ggseg/vignettes/freesurfer_files.html">Freesurfer files</a>, <a href="https://cran.r-project.org/web/packages/ggseg/vignettes/geom-sf.html">using atlases</a>.</p>
<p><img src="ggseg.png" height = "2500" width="550"></p>
<p><a href="https://cran.r-project.org/package=ichimoku">ichimoku</a> v0.2.0: Implements <a href="https://www.investopedia.com/terms/i/ichimokuchart.asp">Ichimoku Kinko Hyo</a>, also commonly known as <a href="https://www.amazon.com/Charts-Trading-Success-Ichimoku-Technique/dp/0956517102">cloud charts</a>, including static and interactive visualizations with tools for creating, backtesting and developing quantitative <em>ichimoku</em> strategies. There is a <a href="https://cran.r-project.org/web/packages/ichimoku/vignettes/reference.html">Reference</a> and a vignette on <a href="https://cran.r-project.org/web/packages/ichimoku/vignettes/strategies.html">Strategies</a>.</p>
<p><img src="ichimoku.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=liminal">liminal</a> v0.1.2: Provides functions for composing interactive visualizations and creating linked interactive graphics for exploratory high-dimensional data analysis. See <a href="https://arxiv.org/abs/2012.06077">Lee et al. (2020)</a> for background. There is a vignette on <a href="https://cran.r-project.org/web/packages/liminal/vignettes/liminal.html">Exploring Non-linear Embeddings</a> and another on the the geometry of <a href="https://cran.r-project.org/web/packages/liminal/vignettes/geometry_parameter_space.html">Parameter Space</a>.</p>
<p><img src="liminal.png" height = "200" width="300"></p>
<p><a href="https://cran.r-project.org/package=mipplot">mipplot</a> v0.3.1: Provides generic functions to produce area, bar, box, and line plots following Integrated Assessment Modeling Consortium <a href="https://www.iamconsortium.org/">(IAMC)</a> submission format in order to visualize climate migration scenarios. See the <a href="https://cran.r-project.org/web/packages/mipplot/vignettes/mipplot-first-steps.html">vignette</a> for first steps.</p>
<p><img src="mipplot.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=qqboxplot">qqboxplot</a> v0.1.0: Implements Q-Q boxplots as an extension to <code>ggplot2</code>. There is a vignette on <a href="https://cran.r-project.org/web/packages/qqboxplot/vignettes/qqboxplot-basic-usage.html">Basic Usage</a> and another that provides <a href="https://cran.r-project.org/web/packages/qqboxplot/vignettes/qqboxplot-paper-replication.html">Examples</a>.
<img src="qqboxplot.png" height = "300" width="500"></p>
<script>window.location.href='https://rviews.rstudio.com/2021/06/24/may-2021-top-40-new-cran-packages/';</script>
Summer Conferences!
https://rviews.rstudio.com/2021/06/17/summer-conferences/
Thu, 17 Jun 2021 00:00:00 +0000https://rviews.rstudio.com/2021/06/17/summer-conferences/
<p>Summer is here, but it is not too late sign up for some summer conferences. The following short list promises interesting speakers, a wide range of topics and plenty of R content.</p>
<p><img src="sc.png" height = "300" width="100%"></p>
<p>June (21 - 23) - The <a href="https://psiweb.org/conferences">PSI 2021</a> conference is online and the <a href="https://psiweb.org/conferences/conference-registration">Registration Portal</a> is still open. Keynote speakers <a href="https://www.ft.com/alan-smith">Alan Smith</a> and <a href="https://www.ft.com/ian-bott">Ian Bott</a> from the Financial Times, and <a href="https://www.statcollab.com/people/janet-wittes/">Janet Wittes</a>, President of the WCG Statistics Collaborative, head the program.</p>
<p>June (21 - 24) - The <a href="https://community.amstat.org/biop/events/ncb/index">Nonclinical Biostatistics Conference 2021</a> is virtual. <a href="https://en.wikipedia.org/wiki/Wendy_L._Martinez">Wendy Martinez</a> of the Bureau of Labor statistics will present <em>A Conversation About Data Ethics</em>, <a href="https://en.wikipedia.org/wiki/Nassim_Nicholas_Taleb">Nassim Taleb</a> of <em>Black Swan</em> fame will deliver the keynote on <em>Statistical Consequences of Fat Tails</em>, and <a href="http://dicook.org/">Di Cook</a> and RStudio’s Carson Sievert will both talk in the Statistical Computational & Visualization Session. <a href="https://community.amstat.org/biop/events/ncb/registration">Registration</a> is open.</p>
<p>July (1 to 2) - <a href="https://r-hta.org/events/workshop/2021/">R for HTA Annual Workshop</a> This online workshop from the Health Technology Assessment Consortium will be focused on R for trial and model-based cost-effectiveness analysis. <a href="https://onlinestore.ucl.ac.uk/conferences-and-events/faculty-of-mathematical-physical-sciences-c06/department-of-statistical-science-f61/f61-workshop-r-for-health-technology-assessment-2021">Registration</a> is open until June 30.</p>
<p>July (5 - 9) - <a href="https://user2021.r-project.org/">useR!2021</a> looks like it is going to be a blockbuster of a conference. The <a href="https://user2021.r-project.org/program/keynotes/">keynote talks</a> alone would be worth the price of admission. This exceptional lineup comprises a remarkably diverse, international group of long-time contributors, new faces, R developers, statisticians, journalists, and educators representing the global R community and speaking on a wide range of topics. I am very pleased to be presenting <em>A little bit about RStudio</em> on July 9 at UTC 9PM. <a href="https://user2021.r-project.org/participation/registration/">Registration</a> closes on June 25.</p>
<p>July (28 - 30) - <a href="https://juliacon.org/2021/">Juliacon 2021</a> will be online and everywhere and <strong>Free</strong>! Long time R contributor <a href="http://janvitek.org/">Jan Vitek</a>, Xiaoye Li of the Lawrence Livermore National Laboratory, and Soumith Chintala of Facebook AI Research will be the keynote speakers. It is free but you need to <a href="https://juliacon.org/2021/tickets/">Register</a>.</p>
<p>Aug (8 - 12) - The <a href="https://ww2.amstat.org/meetings/jsm/2021/">JSM</a> will be online. A keyword search using R and Shiny will turn up quite a few interesting talks. I am particularly looking forward to <a href="https://ww2.amstat.org/meetings/jsm/2021/onlineprogram/AbstractDetails.cfm?abstractid=318815">the talk</a> <em>Simulating Clinical Trials Data with Synthetic.Cdisc.Data and Respectables]</em> by Gabe Becker and Adrian Waddell and <a href="https://ww2.amstat.org/meetings/jsm/2021/onlineprogram/ActivityDetails.cfm?sessionid=220593">the session</a> on <em>Tools to Enable the Use of R by the Biopharmaceutical Industry in a Regulatory Setting</em> which contains five talks from members of the R Consortium’s <a href="https://www.pharmar.org/">R Validation Hub</a> working group.</p>
<p>August (24 - 27) - <a href="https://r-medicine.com/">R/Medicine 2021</a> is online and on track to repeat the last year’s international success. <a href="https://bit.ly/3zuZPTj">Registration</a> is open. The deadline for submitting <a href="https://r-medicine.com/abstract">Abstracts</a> is June 25. Workshops being planned include:</p>
<ul>
<li>R/Med 101: Intro to R for Clinicians and Healthcare Professionals</li>
<li>R Markdown for Reproducible Research (R<sup>3</sup>)</li>
<li>SAS 2 R: Getting off the Island!</li>
<li>From Excel to R+REDcap</li>
<li>Spatial Analysis of Healthcare Data</li>
</ul>
<p>Sept (6 to 9) - <a href="https://rss.org.uk/training-events/conference2021/">RSS 2021 International Conference</a> The Royal Statistical Society conference hopes to be in person in Manchester, UK. The <a href="https://rss.org.uk/training-events/conference2021/conference-programme/">keynote speakers</a> will be Tom Chivers and David Chivers,
Melinda Mills, Jonty Rougier, Eric Tchetgen Tchetgen,
Bin Yu, and <strong>Hadley Wickham</strong>. Submissions for poster presentations are currently open with a deadline of July 1. Registration is open with an early booking discount available until June 4.</p>
<script>window.location.href='https://rviews.rstudio.com/2021/06/17/summer-conferences/';</script>
Functional PCA with R
https://rviews.rstudio.com/2021/06/10/functional-pca-with-r/
Thu, 10 Jun 2021 00:00:00 +0000https://rviews.rstudio.com/2021/06/10/functional-pca-with-r/
<script src="/2021/06/10/functional-pca-with-r/index_files/header-attrs/header-attrs.js"></script>
<p>In two previous posts, <a href="https://rviews.rstudio.com/2021/05/04/functional-data-analysis-in-r/">Introduction to Functional Data Analysis with R</a> and <a href="https://rviews.rstudio.com/2021/05/14/basic-fda-descriptive-statistics-with-r/">Basic FDA Descriptive Statistics with R</a>, I began looking into FDA from a beginners perspective. In this post, I would like to continue where I left off and investigate Functional Principal Components Analysis (FPCA), the analog of ordinary Principal Components Analysis in multivariate statistics. I’ll begin with the math, and then show how to compute FPCs with R.</p>
<p>As I have discussed previously, although the theoretical foundations of FDA depend on some pretty advanced mathematics, it is not necessary to master this math to do basic analyses. The R functions in the various packages insulate the user from most of the underlying theory. Nevertheless, attaining a deep understanding of what the R functions are doing, or looking into any of the background references requires some level of comfort with the notation and fundamental mathematical ideas.</p>
<p>I will define some of the basic concepts and then provide a high level roadmap of the mathematical argument required to develop FPCA from first principals. It is my hope that if you are a total newcomer to Functional Data Analysis you will find this roadmap useful in apprehending the big picture. This synopsis closely follows the presentation by Kokoszka and Reimherr (Reference 1. below).</p>
<p>We are working in <span class="math inline">\(\mathscr{H}\)</span>, a separable <a href="https://en.wikipedia.org/wiki/Hilbert_space#:~:text=A%20Hilbert%20space%20is%20a,of%20calculus%20to%20be%20used.">Hilbert space</a> of square integrable random functions where each random function, <span class="math inline">\(X(\omega,t)\)</span>, where <span class="math inline">\(\omega \in \Omega\)</span> the underlying space of probabilistic outcomes, and <span class="math inline">\(t \in [0,1]\)</span>. (After the definitions below, I will suppress the independent variables and in most equations assume <span class="math inline">\(EX = 0\)</span>.)</p>
<div id="definitions" class="section level3">
<h3>Definitions</h3>
<ul>
<li>A Hilbert Space <span class="math inline">\(\mathscr{H}\)</span> is an infinite dimensional vector space with an inner product denoted by <span class="math inline">\(<.,.>\)</span>. In our case, the <em>vectors</em> are functions.</li>
<li><span class="math inline">\(\mathscr{H}\)</span> is separable if there exists an orthonormal basis. That is, there is an orthogonal collection of functions <span class="math inline">\((e_i)\)</span> in <span class="math inline">\(\mathscr{H}\)</span> such that <span class="math inline">\(<e_i,e_j>\; = 0\)</span> if <span class="math inline">\(i = j\)</span> and 0 otherwise, and every function in <span class="math inline">\(\mathscr{H}\)</span> can be represented as a linear combination of these functions.</li>
<li>The inner product of two functions <span class="math inline">\(X\)</span> and <span class="math inline">\(Y\)</span> in <span class="math inline">\(\mathscr{H}\)</span> is defined as <span class="math inline">\(<X,Y>\; = \int X(\omega,t) Y(\omega,t)dt\)</span>.</li>
<li>The norm of <span class="math inline">\(X\)</span> is defined in terms of the inner product: <span class="math inline">\(\parallel X(\omega) \parallel ^2\; = \int X(\omega, t)^2 dt < \infty\)</span>.</li>
<li><span class="math inline">\(X\)</span> is said to be square integrable if <span class="math inline">\(E\parallel X(\omega) \parallel ^2 < \infty\)</span>.</li>
<li>The <a href="https://math.stackexchange.com/questions/1687111/understanding-the-definition-of-the-covariance-operator">covariance operator</a> <span class="math inline">\(C(y): \mathscr{H} \Rightarrow \mathscr{H}\)</span> for any square integrable function <span class="math inline">\(X\)</span> is given by: <span class="math inline">\(C(y) = E[<X - EX,y>(X - EX)]\)</span></li>
</ul>
</div>
<div id="the-road-to-functional-principal-components" class="section level3">
<h3>The Road to Functional Principal Components</h3>
<p>As we have seen, the fundamental idea of Functional Data Analysis is to represent a function <span class="math inline">\(X\)</span> by a linear combination of basis elements. In the previous posts we showed how to accomplish this using a basis constructed from more or less arbitrarily selected B-spline vectors. But, is there a an empirical, some would say <em>natural</em> basis that can be estimated from the data? The answer is yes, and that is what FPCA is all about.</p>
<p>A good way to start is to look at the distance between a vector <span class="math inline">\(X\)</span> and its projection down into the space spanned by some finite, p-dimensional basis <span class="math inline">\((u_k)\)</span> which is expressed in the following equation,</p>
<p><span class="math inline">\(D = E\parallel X - \sum_{k=1}^{p}<X, u_k>u_k\parallel^2\)</span> <span class="math inline">\((*)\)</span></p>
<p>This expands out to:</p>
<p><span class="math inline">\(= E [< (X - \sum_{k=1}^{p}<X, u_k>u_k, X - \sum_{k=1}^{p}<X, u_k>u_k)>]\)</span></p>
<p>and with a little algebra this:</p>
<p><span class="math inline">\(= E\parallel X \parallel^2 - \sum_{k=1}^{p}E<X, u_k>^2\)</span></p>
<p>It should be clear that we would want to find a basis that makes <span class="math inline">\(D\)</span> as small as possible, and that minimizing <span class="math inline">\(D\)</span> is equivalent to maximizing the term to be subtracted in the line above.</p>
<p>A little algebra shows that, <span class="math inline">\(E<X, u_k>^2 \;=\; <C(u_k),u_k>\)</span> where <span class="math inline">\(C(u_k)\)</span> is the covariance operator defined above.</p>
<p>Now, we are almost at our destination. There is a theorem (e.g. Theorem 11.4.1 in reference 1.) that says for any fixed number of basis elements p, the distance D above is minimized if <span class="math inline">\(u_j = v_j\)</span> where the <span class="math inline">\(v_j\)</span> are the eigenfunctions of <span class="math inline">\(C(y)\)</span> with respect to the unit norm. From this it follows that <span class="math inline">\(E<X, v_j>^2 \;=\; <C(v_,),v_j>\; =\; <\lambda_j, v_j>\; =\; \lambda_j\)</span>.</p>
<p>Going back to equation <span class="math inline">\((*)\)</span>, we can expand <span class="math inline">\(X\)</span> in terms of the basis <span class="math inline">\((v_j)\)</span> so <span class="math inline">\(D = 0\)</span> and we have what is called the <a href="https://en.wikipedia.org/wiki/Karhunen%E2%80%93Lo%C3%A8ve_theorem">Karhunen–Loève</a> expansion: <span class="math inline">\(X = \mu + \sum_{j=1}^{\infty}\xi_jv_j\)</span></p>
<p>where:</p>
<ul>
<li><span class="math inline">\(\mu = EX\)</span><br />
</li>
<li>The deterministic basis functions <span class="math inline">\((v_j)\)</span> are called the <em>functional principal components</em></li>
<li>The <span class="math inline">\((v_j)\)</span> have unit norm and are unique up to their signs. (You can work with <span class="math inline">\(v_j\)</span> or <span class="math inline">\(-v_j\)</span>.)</li>
<li>The eigenvalues are such that: <span class="math inline">\(\lambda_1 > \lambda_2 > . . . \lambda_p\)</span></li>
<li>The random variables <span class="math inline">\(\xi_j =\; <X - \mu,v_j>\)</span> are called the <em>scores</em>.</li>
<li><span class="math inline">\(E\xi_j = 0\)</span>, <span class="math inline">\(E\xi_j^2 = \lambda_j\)</span> and <span class="math inline">\(E|\xi_i\xi_j| = 0,\; if\; i\;\neq\;j\)</span>.</li>
</ul>
<p>And finally, with one more line:<br />
<span class="math inline">\(\sum_{j=1}^{\infty}\lambda_j \: = \: \sum_{j=1}^{\infty}E[<X,v_j>^2] = E\sum_{j=1}^{\infty}<X,v_j>^2 \; = \; E\parallel X \parallel^2 \; < \infty\)</span></p>
<p>we arrive at our destination, the variance decomposition:
<span class="math inline">\(E\parallel X - \mu \parallel^2 \;= \;\sum_{j=1}^{\infty}\lambda_j\)</span></p>
</div>
<div id="lets-calculate" class="section level3">
<h3>Let’s Calculate</h3>
<p>Now that we have enough math to set the context, let’s calculate. We will use the same simulated Brownian motion data that we used in the previous posts, and also construct the same B-spline basis that we used before and save it in the fda object <code>W.obj</code>. I won’t repeat the code here.</p>
<p>The following plot shows <strong>120</strong> simulated curves, each having <strong>1000</strong> points scattered over the interval <strong>[0, 100]</strong>. Each curve has unique observation times over that interval.</p>
<p><img src="/2021/06/10/functional-pca-with-r/index_files/figure-html/unnamed-chunk-1-1.png" width="672" />
For first attempt at calculating functional principal components we’ll use the <code>pca.fd()</code> function from the <code>fda</code> package. So set up wise, we are picking up our calculations exactly where we left off in the previous post. As before, the basis representations of these curves are packed into the fda object <code>W.obj</code>. The function <code>pca.fd()</code> takes <code>W.obj</code> as input. It needs the non-orthogonal B-spline basis to seed its computations and the estimate the covariance matrix and the orthogonal eigenvector basis <span class="math inline">\(v_j\)</span>. The <code>nharm = 5</code> parameter requests computing 5 eigenvalues.</p>
<p>The method of calculation roughly follows the theory outlined above. It starts with a basis representation of the functions, computes the covariance matrix, and calculates the eigenfunctions.</p>
<pre class="r"><code>fun_pca <- pca.fd(W.obj, nharm = 5)</code></pre>
<p>The object produced by <code>pca.fd()</code> is fairly complicated. For example, the list <code>fun_pca$harmonics</code> does not contain the eignevectors themselves, but rather coefficients that enable the eigenvectors to be computed from the original basis. However, because there is a special plot method for <code>plot.pca.fd()</code> it is easy to plot the eigenvectors.</p>
<pre class="r"><code>plot(fun_pca$harmonics, lwd = 3)</code></pre>
<p><img src="/2021/06/10/functional-pca-with-r/index_files/figure-html/unnamed-chunk-3-1.png" width="672" /></p>
<pre><code>## [1] "done"</code></pre>
<p>It is also to obtain the eivenvalues <span class="math inline">\(\lambda_j\)</span>,</p>
<pre class="r"><code>fun_pca$values</code></pre>
<pre><code>## [1] 37.232207 3.724524 1.703604 0.763120 0.547976 0.389431 0.196101
## [8] 0.163289 0.144052 0.116587 0.089307 0.057999 0.054246 0.050683
## [15] 0.042738 0.035107 0.031283 0.024905 0.019079 0.016428 0.011657
## [22] 0.007392 0.002664</code></pre>
<p>and, the proportion of the variance explained by each eigenvalue.</p>
<pre class="r"><code>fun_pca$varprop</code></pre>
<pre><code>## [1] 0.81965 0.08199 0.03750 0.01680 0.01206</code></pre>
</div>
<div id="a-different-approach" class="section level3">
<h3>A Different Approach</h3>
<p>So far in this short series of FDA posts, I have been mostly using the <code>fda</code> package to calculate. In 2003 when it was released, it was ground breaking work. It is still the package that you are most likely to find when doing internet searches, and is the foundation for many subsequent R packages. However, as the <a href="https://cran.r-project.org/web/views/FunctionalData.html">CRAN Task View</a> on Functional Data Analysis indicates, new work in FDA has resulted in several new R packages. The more recent <a href="https://cran.r-project.org/package=fdapace"><code>fdapace</code></a> takes a different approach to calculating principal components. The package takes its name from the <strong>(PACE)</strong> Principal Components by Conditional Expectation algorithm described in the paper by Yao, Müller and Wang (Reference 4. below). The package <a href="https://cran.r-project.org/web/packages/fdapace/vignettes/fdapaceVig.html">vignette</a> is exemplary. It describes the methods of calculation, develops clear examples and provides a list of references to guide your reading about PACE and FDA in general.</p>
<p>A very notable feature of the PACE algorithm is that it is designed specifically to work with sparse data. The vignette describes the two different methods of calculation that package functions employ for sparse and non-sparse data. In this post, In this post we are not working with sparse data, but hope to do so in the future. See the vignette for examples of FPCA with sparse data.</p>
<pre class="r"><code>library(fdapace)</code></pre>
<p>The <code>fdapace</code> package requires data for the functions (curves) and associated times be organized in lists. We begin by using the <code>fdapace::CheckData()</code> function to check the data set up in the tibble <code>df</code>. (See previous post on descriptive statistics for the details on the data construction.)</p>
<pre class="r"><code>CheckData(df$Curve,df$Time)</code></pre>
<p>No error message is generated, so we move on th having the <code>FPCA()</code> function calculate the FPCA outputs including:</p>
<pre class="r"><code>W_fpca <- FPCA(df$Curve,df$Time)</code></pre>
<p>the eigenvalues:</p>
<pre class="r"><code>W_fpca$lambda</code></pre>
<pre><code>## [1] 37.5386 3.2098 1.0210 0.3533</code></pre>
<p>the cumulative percentage of variance explained by the eigenvalue</p>
<pre class="r"><code>W_fpca$cumFVE</code></pre>
<pre><code>## [1] 0.8875 0.9634 0.9875 0.9959</code></pre>
<p>and the scores:</p>
<pre class="r"><code>head(W_fpca$xiEst)</code></pre>
<pre><code>## [,1] [,2] [,3] [,4]
## [1,] 0.8187 1.1971 -1.77755 0.20087
## [2,] -8.7396 2.4165 -0.33768 -0.10565
## [3,] -2.7517 -0.4879 -0.06747 -0.13953
## [4,] 3.9218 0.9419 0.39098 -0.25449
## [5,] -1.4400 0.7691 2.11549 -0.59947
## [6,] 7.3952 0.5114 -2.16391 0.05199</code></pre>
<p>All of these are in fairly good agreement with what we computed above. I am, however, a little surprised by the discrepancy in the value of the second eigenvalue. The default plot method for <code>FPCA()</code> produces a plot indicating the density of the data, a plot of the mean of the functions reconstructed from the eigenfunction expansion, a scree plot of the eigenvalues and a plot of the first three eigenfunctions.</p>
<pre class="r"><code>plot(W_fpca)</code></pre>
<p><img src="/2021/06/10/functional-pca-with-r/index_files/figure-html/unnamed-chunk-12-1.png" width="672" /></p>
<p>Finally, it has probably already occurred to you that if you know the eigenvalues and scores, the Karhunen–Loève expansion can be used to simulate random functions. It can be shown that for the Wiener process:</p>
<p><span class="math inline">\(v_j(t) = \sqrt2 sin(( j - \frac{1}{2})\pi t)\)</span> and <span class="math inline">\(\lambda_j = \frac{1}{(j - \frac{1}{2})^2\pi^2}\)</span></p>
<p>This gives us:</p>
<p><span class="math inline">\(W(t) = \sum_{j=1}^{\infty} \frac {\sqrt2}{(j - \frac{1}{2})\pi)} N_j sin(( j - \frac{1}{2})\pi t\)</span></p>
<p>where the <span class="math inline">\(N_j \:are\; iid \;N(0,1)\)</span>.</p>
<p>The <code>fdapace</code> function <code>fdapace::Wiener()</code> uses this information to simulate an alternative, smoothed version of the Brownian motion, Wiener process.</p>
<pre class="r"><code>set.seed(123)
w <- Wiener(n = 1, pts = seq(0,1, length = 100))
t <- 1:100
df_w <- tibble(t, as.vector(w))
ggplot(df_w, aes(x = t, y = w)) + geom_line()</code></pre>
<p><img src="/2021/06/10/functional-pca-with-r/index_files/figure-html/unnamed-chunk-13-1.png" width="576" /></p>
<p>In future posts, I hope to continue exploring the <code>fdapace</code> package, including its ability to work with sparse data.</p>
</div>
<div id="references" class="section level3">
<h3>References</h3>
<p>I found the following references particularly helpful.</p>
<ol style="list-style-type: decimal">
<li>Kokoszka, P. and Reimherr, M. (2017). <a href="https://www.amazon.com/Introduction-Functional-Analysis-Chapman-Statistical-ebook/dp/B075Z9QCV9/ref=sr_1_1?dchild=1&keywords=Introduction+to+functional+data+analysis&qid=1623276309&sr=8-1"><em>Introduction to Functional Data Analysis</em></a>. CRC.</li>
<li>Hsing, T and Eubank, R. (2015). <a href="https://www.amazon.com/Theoretical-Foundations-Functional-Introduction-Probability/dp/0470016914/ref=sr_1_1?dchild=1&keywords=theoretical+foundations+of+functional+data+analysis&qid=1623276176&sr=8-1"><em>Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators</em></a> Wiley</li>
<li>Wang, J., Chiou, J. and Müller, H. (2015). <a href="https://arxiv.org/pdf/1507.05135.pdf"><em>Review of Functional Data Analysis</em></a></li>
<li>Yao, F., Müller, H, Wang, J. (2012). <a href="https://anson.ucdavis.edu/~mueller/jasa03-190final.pdf"><em>Functional Data Analysis for Sparse Longitudinal Data</em></a> JASA J100, I 470</li>
</ol>
</div>
<script>window.location.href='https://rviews.rstudio.com/2021/06/10/functional-pca-with-r/';</script>
R for Public Health
https://rviews.rstudio.com/2021/06/02/r-for-public-health/
Wed, 02 Jun 2021 00:00:00 +0000https://rviews.rstudio.com/2021/06/02/r-for-public-health/
<p>The COVID19 pandemic has raised the profile of public health workers at all levels from the nurses and doctors working on the front lines at our hospitals, to high level state and federal public health officials. I think its a good bet that eighteen months ago few of us had any clear idea about how the public health care system works, or thought much about the people charged with the awesome responsibility to keep us safe. We are all a little bit wiser now. It strikes me as obvious that we will have a continuing need to improve our public health systems and that this need will create opportunities for data scientists to make significant contributions, in research, logistics, data management, reporting, public communication and many more areas. The <a href="https://blog.rstudio.com/2021/05/18/managing-covid-vaccine-distribution-with-a-little-help-from-shiny/">recent post</a> by my colleague Jesse Mostipak describes how R and Shiny made a big difference in the nitty gritty work required to roll out vaccine distribution in West Virginia. This four minute video about how the West Virginia Army National Guard built a COVID vaccine inventory management system is inspiring.</p>
<div style="text-align:center">
<iframe width="400" height="250" src="https://www.youtube.com/embed/CYilc-rEgjg" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</div>
<p>The following are some R resources that you may find helpful if you are seeking to increase your R skills with a eye toward public health applications. Some may be useful to public health professionals seeking to learn R, others may be interesting to R users who want to investigate data science in a public health context.</p>
<h3 id="courses">Courses</h3>
<ul>
<li><p>The Johns Hopkins Graduate Institute of Epidemiology and Biostatistics has a slew of of <a href="https://www.jhsph.edu/departments/epidemiology/continuing-education/graduate-summer-institute-of-epidemiology-and-biostatistics/">summer courses</a> (many of them online), including <a href="https://www.jhsph.edu/departments/epidemiology/continuing-education/graduate-summer-institute-of-epidemiology-and-biostatistics/courses/introduction-to-r-for-public-health-researchers.html">Introduction to R For Public Health Researchers</a>.</p></li>
<li><p>If you are feeling optimistic about traveling and can make your to Estonia this summer you might consider <a href="http://bendixcarstensen.com/SPE/">Statistical Practice in Epidemiology using R</a>.</p></li>
<li><p>If you hurry, you may be able to get into the Coursera course offered by Imperial College London <a href="https://www.coursera.org/specializations/statistical-analysis-r-public-health">Statistical Analysis with R for Public Health Specialization</a>.</p></li>
<li><p>There are several <a href="https://www.classcentral.com/course/introduction-statistics-data-analysis-pu-13079">Coursera Courses</a> for R in a public health context.</p></li>
<li><p>Frank Harrell’s free online course <a href="https://hbiostat.org/bbr/">Biostatistics for Biomedical Research</a>, available on YouTube: <a href="https://www.youtube.com/channel/UC-o_ZZ0tuFUYn8e8rf-QURA">BBRcourse</a>, is an excellent introduction to the basic statistical concepts underlying all medical applications.</p></li>
</ul>
<h3 id="books">Books</h3>
<p>If you can learn on your own with the help of a good book, here are some ideas.</p>
<p><img src="Handbook.png" height = "300" width="500"></p>
<ul>
<li><p><a href="https://epirhandbook.com/contact-tracing-1.html">The Epidemiologist R Handbook</a>: R for applied epidemiology and public health is a delightful, brief, modern introduction to R that covers the basics of the <a href="https://www.tidyverse.org/">Tidyverse</a>, <a href="https://rmarkdown.rstudio.com/">R Markdown</a>, <a href="https://shiny.rstudio.com/">Shiny</a> and <a href="https://rdatatable.gitlab.io/data.table/">data.table</a>.</p></li>
<li><p><a href="https://web.stanford.edu/class/bios221/book/">Modern Statistics for Modern Biology</a>: This book is focused more on genomics than public health applications, but it is probably the best introductory statistical text available. The modern statistics part in the title means statistics as a data driven, computational based science. Every topic is illustrated with well-crafted R code and visualizations. The book is great read.</p></li>
<li><p><a href="https://bookdown.org/taragonmd/phds/">Population Health Data Science with R</a>: An introduction to R by authors who see population health as a systems framework for studying and improving the health of populations through collective action and learning.</p></li>
<li><p><a href="https://global.oup.com/academic/product/epidemiology-with-r-9780198841333?cc=us&lang=en&#">Epidemiology with R</a>: This is a new, reasonably priced book that covers the basics. It emphasizes reproducibility with R Markdown. Code examples are written in a base R style that matches the extensively used <a href="https://cran.r-project.org/package=Epi">Epi</a> package.</p></li>
</ul>
<h3 id="r-packages">R Packages</h3>
<p>If working through a book on you own seems too much of a stretch, then digest an R package or two. Here are a few examples of packages on public health themes with sufficient documentation to make interesting self-learning projects.</p>
<ul>
<li><p><a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5931789/">EpiModel</a>: An R Package for Mathematical Modeling of Infectious Disease over Networks</p></li>
<li><p><a href="https://cran.r-project.org/package=PHEindicatormethods">PHEindicators</a>: Common Public Health Statistics and their Confidence Intervals</p></li>
<li><p><a href="https://www.jstatsoft.org/article/view/v091i12">SimInf</a>: An R Package for Data-Driven Stochastic Disease Simulations</p></li>
<li><p><a href="https://cran.r-project.org/package=SPARSEMODr">SPARSEMODr</a>: Construct spatial, stochastic disease models that show how parameter values fluctuate in response to public health interventions</p></li>
</ul>
<h3 id="shiny">Shiny</h3>
<p>Finally, if you find yourself in a situation similar to the West Virginia Army National Guard team featured in the video, you may just want to teach yourself Shiny. The RStudio <a href="https://shiny.rstudio.com/tutorial/">Shiny Tutorial</a>, along with Hadley Wickham’s book <a href="https://mastering-shiny.org/">Mastering Shiny</a>, is a very good place to start. If you need more structure, you might check out the <a href="https://www.udemy.com/topic/shiny/?utm_source=adwords&utm_medium=udemyads&utm_campaign=DSA_Catchall_la.EN_cc.US&utm_content=deal4584&utm_term=_._ag_95911180068_._ad_436653296108_._kw__._de_c_._dm__._pl__._ti_dsa-437115340933_._li_9032191_._pd__._&matchtype=b&gclid=Cj0KCQjw2NyFBhDoARIsAMtHtZ7tXUSdrjLIvbkpb2BdzwBYYCelutNMt6RUZDCaI7lST-fAXjwwaeQaAumnEALw_wcB">Udemy Courses</a>, or work through the online workshops from <a href="https://library.capture.duke.edu/Panopto/Pages/Viewer.aspx?id=7a59e23a-1f7f-4bd7-8ebc-a943014170b4">Duke University</a> or the <a href="(https://uomresearchit.github.io/r-shiny-course/)">University of Manchester</a>. And by all means, immerse yourself in the examples, posts and podcasts of the <a href="https://shinydevseries.com/">Shiny Developer Series</a>.</p>
<script>window.location.href='https://rviews.rstudio.com/2021/06/02/r-for-public-health/';</script>
April 2021: "Top 40" New CRAN Packages
https://rviews.rstudio.com/2021/05/25/april-2021-top-40-new-cran-packages/
Tue, 25 May 2021 00:00:00 +0000https://rviews.rstudio.com/2021/05/25/april-2021-top-40-new-cran-packages/
<p>One hundred seventy-nine new packages made it to CRAN in April. Here are my “Top 40” picks in twelve categories: Computational Methods, Data, Genomics, Machine Learning, Mathematics, Medicine, Networks, Operations Research, Statistics, Time Series, Utilities, and Visualization.</p>
<h3 id="computational-methods">Computational Methods</h3>
<p><a href="https://cran.r-project.org/package=abess">abess</a> v0.1.0: Provides a toolkit for solving the best subset selection problem in linear regression, logistic regression, Poisson regression, Cox proportional hazard model, multiple-response Gaussian, and multinomial regression. It implements and generalizes algorithms described in <a href="https://www.pnas.org/content/117/52/33117">Zhu et al. (2020)</a> that exploit a novel sequencing-and-splicing technique to guarantee exact support recovery and globally optimal solution in polynomial times. There is an <a href="https://cran.r-project.org/web/packages/abess/vignettes/abess-guide.html">Introduction</a>.</p>
<p><a href="https://cran.r-project.org/package=eat">eat</a> v0.1.0: Provides functions to determine production frontiers and technical efficiency measures through non-parametric techniques based upon regression trees. See <a href="https://www.sciencedirect.com/science/article/abs/pii/S0957417420306072?via%3Dihub">Esteve et al. (2020)</a> for details. There is an <a href="https://cran.r-project.org/web/packages/eat/vignettes/EAT.html">Introduction</a>.</p>
<p><img src="eat.png" height = "300" width="500"></p>
<h3 id="data">Data</h3>
<p><a href="https://cran.r-project.org/package=childdevdata">childdevdata</a> v1.1.0: Bundles publicly available data sets with individual milestone data for children aged 0-5 years, with the aim of supporting the construction, evaluation, validation and interpretation of methodologies that aggregate milestone data into informative measures of child development. See <a href="https://cran.r-project.org/web/packages/childdevdata/readme/README.html">README</a>.</p>
<p><img src="child.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=datagovindia">datagovindia</a> v0.0.3: Allows users to search the <a href="https:data.gov.in/ogpl_apis">open data platform</a> of the government of India to communicate with the more than 80,000 available APIs. See the <a href="https://cran.r-project.org/web/packages/datagovindia/vignettes/datagovindia_vignette.html">vignette</a>.</p>
<p><a href="https://cran.r-project.org/package=lehdr">lehdr</a> v0.2.4: Provides functions to query the <a href="https://lehd.ces.census.gov/data/lodes/LODES7/">LODES FTP server</a> to obtain longitudinal <a href="https://lehd.ces.census.gov/">Employer-Household Dynamics</a> data and optionally aggregate Census block-level data. See the <a href="https://cran.r-project.org/web/packages/lehdr/vignettes/getting_started.html">vignette</a>.</p>
<p><a href="https://cran.r-project.org/package=rbioapi">rbioapi</a> v0.7.0: Provides a consistent R interface to the Biologic Web Services API and fully supports <a href="https://cran.r-project.org/web/packages/rbioapi/vignettes/rbioapi_mieaa.html">miEAA</a>, <a href="https://cran.r-project.org/web/packages/rbioapi/vignettes/rbioapi_panther.html">PANTHER</a>, <a href="https://cran.r-project.org/web/packages/rbioapi/vignettes/rbioapi_reactome.html">Reactome</a>, <a href="https://cran.r-project.org/web/packages/rbioapi/vignettes/rbioapi_string.html">String</a>, and <a href="https://cran.r-project.org/web/packages/rbioapi/vignettes/rbioapi_uniprot.html">UniProt</a>. See this <a href="https://cran.r-project.org/web/packages/rbioapi/vignettes/rbioapi.html">vignette</a> to get started.</p>
<p><a href="https://cran.r-project.org/package=tidywikidatar">tidywikidatar</a> v0.2.0: Provides functions to query <a href="https://wikidata.org/">Wilidata</a>, get tidy data frames in response, and cache data in a local <code>SQLite</code> database. See <a href="https://cran.r-project.org/web/packages/tidywikidatar/readme/README.html">README</a>.</p>
<h3 id="genomics">Genomics</h3>
<p><a href="https://cran.r-project.org/package=protti">protti</a> v0.1.1: Provides functions and workflows for proteomics quality control and data analysis of both limited proteolysis-coupled mass spectrometry and regular bottom-up proteomics experiments. See <a href="https://www.nature.com/articles/nbt.2999">Feng et. al. (2014)</a> for background. There are vignettes for various workflows: <a href="https://cran.r-project.org/web/packages/protti/vignettes/data_analysis_dose_response_workflow.html">Dose Response</a>, <a href="https://cran.r-project.org/web/packages/protti/vignettes/data_analysis_single_dose_treatment_workflow.html">Single Treatment Dose Response</a>, <a href="https://cran.r-project.org/web/packages/protti/vignettes/input_preparation_workflow.html">Input Preparation</a>, and <a href="https://cran.r-project.org/web/packages/protti/vignettes/quality_control_workflow.html">Quality Control</a>.</p>
<p><img src="protti.png" height = "400" width="400"></p>
<p><a href="https://cran.r-project.org/package=Rediscover">Rediscover</a> v0.1.0: Implements an optimized method for identifying mutually exclusive genomic events based on the Poisson-Binomial distribution that takes into account that some samples are more mutated than others. See <a href="https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1114-x">Canisius et al. (2016)</a>. The <a href="https://cran.r-project.org/web/packages/Rediscover/vignettes/Rediscover.html">vignette</a> provides an introduction.</p>
<p><img src="Rediscover.png" height = "300" width="500"></p>
<h3 id="machine-learning">Machine Learning</h3>
<p><a href="https://cran.r-project.org/package=geocmeans">geocmeans</a> v0.1.1: Provides functions to apply spatial fuzzy unsupervised classification, visualize and interpret results, as well as indices for estimating the spatial consistency and classification quality. See <a href="https://www.sciencedirect.com/science/article/abs/pii/S0031320306003451?via%3Dihub">Cai et al. (2007)</a>, <a href="https://www.sciencedirect.com/science/article/abs/pii/S1051200412002357?via%3Dihub">Zaho et al. (2013)</a>, and <a href="https://journals.openedition.org/cybergeo/36414">Gelb & Appaericio (2021)</a> for background. There is an <a href="https://cran.r-project.org/web/packages/geocmeans/vignettes/introduction.html">Introduction</a> and an additional <a href="https://cran.r-project.org/web/packages/geocmeans/vignettes/adjustinconsistency.html">vignette</a>.</p>
<p><img src="geocmeans.png" height = "400" width="400"></p>
<p><a href="https://cran.r-project.org/package=Rforestry">Rforestry</a> v0.9.0.4: Provides fast implementations of Honest Random Forests, Gradient Boosting, and Linear Random Forests, with an emphasis on inference and interpretability. See <a href="https://arxiv.org/abs/1906.06463">Kunzel et al. (2019)</a>. See <a href="https://cran.r-project.org/web/packages/Rforestry/readme/README.html">README</a> to get started.</p>
<h3 id="mathematics">Mathematics</h3>
<p><a href="https://cran.r-project.org/package=elasdics">elasdics</a> v0.1.2: Provides functions to align curves and to compute mean curves based on the elastic distance defined in the square-root-velocity framework. For information on the framework see <a href="https://link.springer.com/book/10.1007%2F978-1-4939-4020-2">Srivastava and Klassen (2016)</a>, For more theoretical details see <a href="https://arxiv.org/abs/2104.11039">Steyer et al. (2021)</a></p>
<p><a href="https://cran.r-project.org/package=jordan">jordan</a> v1.0-1: Provides functions to manipulate Jordan Algebras, commutative but non-associative algebraic structures that satisfy the Jordan Identify: (xy)x<sup>2</sup> = x(yx<sup>2</sup>). See <a href="http://math.nsc.ru/LBRT/a1/files/mccrimmon.pdf">McCrimmon (204)</a>.</p>
<h3 id="medicine">Medicine</h3>
<p><a href="https://cran.r-project.org/package=ccoptimalmatch">ccoptimalmatch</a> v0.1.0: Uses sub-sampling to create pseudo-observations of controls to optimally match cases with controls. See <a href="https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-021-01256-3">Mamoiris (2021)</a> for the theory and the <a href="https://cran.r-project.org/web/packages/ccoptimalmatch/vignettes/ccoptimalmatching_vignette.html">vignette</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=nCov2019">nCov2019</a> v0.4.4: Implements an interface to <a href="https://disease.sh/">disease.sh - Open Disease Data API</a> to access real time and historical data of COVID-19 cases, vaccine and therapeutics data. There is a <a href="https://cran.r-project.org/web/packages/nCov2019/vignettes/nCov2019.html">vignette</a>.</p>
<p><img src="nCov2019.png" height = "400" width="400"></p>
<p><a href="https://cran.r-project.org/package=hlaR">hlaR</a> v0.1.0: Implements a tool for the eplet analysis of donor and recipient HLA (human leukocyte antigen) mismatches. There are vignettes on <a href="https://cran.r-project.org/web/packages/hlaR/vignettes/allele-haplotype.html">Imputation</a> and <a href="https://cran.r-project.org/web/packages/hlaR/vignettes/eplet-mm.html">Eplet Mismatch</a> and a <a href="https://emory-larsenlab.shinyapps.io/hlar_shiny/">Shiny App</a> as well.</p>
<p><a href="https://cran.r-project.org/package=ReviewR">RevieweR</a> v2.3.6: Implements a portable <code>Shiny</code> tool to explore patient-level electronic health record data and perform chart review in a single integrated framework. This tool supports the <a href="https://www.ohdsi.org/data-standardization/the-common-data-model/">OMOP</a> common data model as well as the <a href="https://mimic.physionet.org/">MIMIC-III</a> data model, and chart review through a <a href="https://www.project-redcap.org/">REDCap</a> API. See the <a href="https://reviewr.thewileylab.org/">RevieweR Website</a> for more information. There are several vignettes including <a href="https://cran.r-project.org/web/packages/ReviewR/vignettes/deploy_local.html">Local</a>, <a href="https://cran.r-project.org/web/packages/ReviewR/vignettes/deploy_docker.html">Docker</a>, <a href="https://cran.r-project.org/web/packages/ReviewR/vignettes/deploy_bigquery.html">BigQuery</a> and <a href="https://cran.r-project.org/web/packages/ReviewR/vignettes/deploy_server.html">Shiny Server</a> deployment and performing a <a href="https://cran.r-project.org/web/packages/ReviewR/vignettes/usage_perform_chart_review.html">Chart Review</a>.</p>
<p><img src="RevieweR.png" height = "300" width="500"></p>
<h3 id="networks">Networks</h3>
<p><a href="https://CRAN.R-project.org/package=greed">greed</a> v0.5.1: Provides an ensemble of algorithms to enable clustering of networks and data matrices with different type of generative models. Model selection and clustering is performed in combination by optimizing the Integrated Classification Likelihood. The optimization is performed with a combination of greedy local search and a genetic algorithm. See <a href="https://arxiv.org/abs/2002.11577">Côme et al. (2021)</a> for background and the vignettes on <a href="https://cran.r-project.org/web/packages/greed/vignettes/GMM.html">Gaussian Mixture Models</a> and <a href="https://cran.r-project.org/web/packages/greed/vignettes/graph-clustering-with-sbm.html">Clustering</a>.</p>
<p><img src="greed.png" height = "400" width="400"></p>
<h3 id="operations-research">Operations Research</h3>
<p><a href="https://cran.r-project.org/package=critpath">critpath</a> v0.1.2: Provides functions to compute critical paths, schedules, <a href="https://www.investopedia.com/terms/p/pert-chart.asp">PERT</a> charts and <a href="https://en.wikipedia.org/wiki/Gantt_chart">Gantt</a> charts. There is a vignette on <a href="https://cran.r-project.org/web/packages/critpath/vignettes/CPMandPERT.html">CPM and PERT</a> and another on the <a href="https://cran.r-project.org/web/packages/critpath/vignettes/LESS.html">LESS Method</a>.</p>
<p><img src="critpath.png" height = "350" width="350"></p>
<p><a href="https://cran.r-project.org/package=himach">himach</a> v0.1.2: Provides functions to compute the best routes between airports for supersonic aircraft flying subsonic over land. There is an <a href="https://cran.r-project.org/web/packages/himach/vignettes/Supersonic_Routes.html">Introduction to Supersonic Routing</a> and a vignette on <a href="https://cran.r-project.org/web/packages/himach/vignettes/Supersonic_Routes_in_depth.html">Advanced Supersonic Routing</a>.</p>
<p><img src="himach.png" height = "350" width="350"></p>
<h3 id="statistics">Statistics</h3>
<p><a href="https://cran.r-project.org/package=convdistr">convdistr</a> v1.5.3: Provides functions to compute convolutions of probability distributions via a method that creates a new random number function for individual random samples from the random generator function of each distribution. There is an <a href="https://cran.r-project.org/web/packages/convdistr/vignettes/using-convdistr.html">Introduction</a> and a vignette on <a href="https://cran.r-project.org/web/packages/convdistr/vignettes/sample_size.html">Sample Size</a>.</p>
<p><img src="convdistr.png" height = "350" width="500"></p>
<p><a href="https://cran.r-project.org/package=gamlss.lasso">gamlss.lasso</a> v1.0-0: Provides an interface for extra high-dimensional smooth functions for Generalized Additive Models for Location Scale and Shape (GAMLSS) including lasso, ridge, elastic net and least angle regression. The <a href="https://www.gamlss.com/">gamlss website</a> provides considerable information.</p>
<p><img src="gamlss.jpeg" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=GGMnonreg">GGMnonreg</a> v1.0.0: Provides functions to estimate non-regularized Gaussian graphical models, Ising models, and mixed graphical models. See <a href="https://www.tandfonline.com/doi/abs/10.1080/00273171.2019.1575716?journalCode=hmbr20">Williams et al. (2019)</a>, <a href="https://bpspsychub.onlinelibrary.wiley.com/doi/abs/10.1111/bmsp.12173">Williams & Rast (2019)</a>, and <a href="https://psyarxiv.com/fb4sa/">Williams (2020)</a> for details. <a href="https://cran.r-project.org/web/packages/GGMnonreg/readme/README.html">README</a> contains examples.</p>
<p><img src="GGMnonreg.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=relevance">relevance</a> v1.1: Implements the concepts of relevance and significance measures introduced in <a href="https://stat.ethz.ch/~stahel/relevance/stahel-relevance2103.pdf">Stahel (2021)</a> to augment inference with p-values. See the <a href="https://cran.r-project.org/web/packages/relevance/vignettes/relevance-descr.pdf">vignette</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=sasfunclust">sasfunclust</a> v1.0.0: Implements the sparse and smooth functional clustering method described in <a href="https://arxiv.org/abs/2103.15224">Centofanti et al. (2021)</a> that aims to classify a sample of curves into homogeneous groups while jointly detecting the most informative portions of domain. See <a href="https://cran.r-project.org/web/packages/sasfunclust/readme/README.html">README</a> to get started.</p>
<p><img src="sasfunclust.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=survMS">survMS</a> v0.0.1: Provides functions to simulate data from the <a href="https://www.sciencedirect.com/science/article/pii/S0169716103230248">Accelerated Hazard</a>, <a href="https://en.wikipedia.org/wiki/Accelerated_failure_time_model">Accelerated Failure Time</a>, and <a href="https://socialsciences.mcmaster.ca/jfox/Books/Companion-2E/appendix/Appendix-Cox-Regression.pdf">Cox</a> survival models. See <a href="https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.2059">Bender et al. (2004)</a> for the methods used to implement the Cox model, and the <a href="https://cran.r-project.org/web/packages/survMS/vignettes/how-to-simulate-survival-models.html">vignette</a> and <a href="https://github.com/mathildesautreuil/survMS/">GitHub</a> for an introduction and examples.</p>
<p><img src="survMS.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=TestGardener">TestGardener</a> v0.1.4: Provides functions to develop, evaluate, and score multiple choice examinations, psychological scales, questionnaires, and similar types of data involving sequences of choices among one or more sets of answers. See <a href="https://www.mdpi.com/2624-8611/2/4/26">Ramsay et al. (2020)</a> and <a href="https://journals.sagepub.com/doi/10.3102/1076998619885636">Ramsay et al. (2019)</a> for the methodology and the vignettes <a href="https://cran.r-project.org/web/packages/TestGardener/vignettes/SDSAnalysis.html">Symptom Distress Analysis</a> and <a href="https://cran.r-project.org/web/packages/TestGardener/vignettes/SweSATQuantitativeAnalysis.html">SweSAT Quantitative Analysis</a>.</p>
<p><img src="TestG.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=wpa">wpa</a> v1.5.0: Provides opinionated functions to enable easier and faster analysis of Workplace Analytics data. See the <a href="https://cran.r-project.org/web/packages/wpa/vignettes/intro-to-wpa.html">vignette</a> for an introduction.</p>
<p><img src="wpa.png" height = "400" width="400"></p>
<h3 id="time-series">Time Series</h3>
<p><a href="https://cran.r-project.org/package=garchmodels">garchmodels</a> v0.1.1: Implements a framework for using <a href="https://www.investopedia.com/terms/g/garch.asp#:~:text=GARCH%20is%20a%20statistical%20modeling,an%20autoregressive%20moving%20average%20process.">GARCH</a> models with the <code>tidymodels</code> ecosystem. It includes both univariate and multivariate methods from the <code>rugarch</code> and <code>rmgarch</code> packages. There is a <a href="https://cran.r-project.org/web/packages/garchmodels/vignettes/getting-started.html">Getting Started Guide</a> and a <a href="https://cran.r-project.org/web/packages/garchmodels/vignettes/tuning_univariate_algorithms.html">vignette</a> on tuning univariate GARCH models.</p>
<p><img src="garchmodels.png" height = "600" width="400"></p>
<p><a href="https://cran.r-project.org/package=tensorTS">tensorTS</a> v0.1.1: Provides functions for estimating, simulating and predicting factor and autoregressive models for matrix and tensor valued time series. See <a href="https://arxiv.org/abs/1905.07530">Chen et al. (2020)</a>, <a href="https://www.sciencedirect.com/science/article/abs/pii/S0304407620302050?via%3Dihub">Chen et al. (2020)</a>, and <a href="https://arxiv.org/abs/2006.02611">Han et al. (2020)</a> for the math.</p>
<h3 id="utilities">Utilities</h3>
<p><a href="https://cran.r-project.org/package=diffmatchpatch">diffmatchpatch</a> v0.1.0: Implements a wrapper for Google’s <a href="https://github.com/google/diff-match-patch">diff-match-patch</a> library. It provides basic tools for computing diffs, finding fuzzy matches, and constructing / applying patches to strings. See <a href="https://github.com/google/diff-match-patch">README</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=erify">erify</a> v0.2.0: Provides several validator functions to check if arguments passed by users have valid types, lengths, etc., and if not, to generate informative and good-formatted error messages in a consistent style. See the <a href="https://cran.r-project.org/web/packages/erify/vignettes/erify.html">vignette</a> to get started.</p>
<p><a href="https://cran.r-project.org/package=juicr">juicr</a> v0.1: Provides a GUI interface for automating data extraction from multiple images containing scatter and bar plots, semi-automated tools to tinker with extraction attempts, and a fully-loaded point-and-click manual extractor with image zoom, calibrator, and classifier. See the <a href="https://cran.r-project.org/web/packages/juicr/vignettes/juicr_basic_vignette_v0.1.pdf">vignette</a> for examples, and the <a href="https://www.youtube.com/c/LajeunesseLab/">Youtube channel</a> for a course on meta analysis.</p>
<p><a href="https://CRAN.R-project.org/package=mailmerge">mailmerge</a> v 0.2.1: Allows users to mail merge using markdown documents and gmail, parse markdown documents as the body of email, use the <code>yaml</code> header to specify the subject line of the email, preview the email in the RStudio viewer pane, and send (draft) email using <code>gmailr</code>. See the <a href="https://cran.r-project.org/web/packages/mailmerge/vignettes/introduction.html">vignette</a> for examples.</p>
<p><a href="https://cran.r-project.org/package=m61r">m61r</a> v0.0.2: Provides <code>dplyr</code> and <code>tidyr</code> like data manipulation functions using only base R and no dependencies. See the <a href="https://cran.r-project.org/web/packages/m61r/vignettes/base_r.pdf">vignette</a> for examples.</p>
<h3 id="visualization">Visualization</h3>
<p><a href="https://cran.r-project.org/package=flametree">flametree</a> v0.1.2: Implements a generative art system for producing tree-like images using an L-system to create the structures. See <a href="https://cran.r-project.org/web/packages/flametree/readme/README.html">README</a> to get started.</p>
<p><img src="flametree.png" height = "500" width="500"></p>
<p><a href="https://cran.r-project.org/package=leafdown">leafdown</a> v1.0.0: Provides drill down functionality for <code>leaflet</code> choropleths in <code>shiny</code> apps. There is an <a href="https://cran.r-project.org/web/packages/leafdown/vignettes/Introduction.html">Introduction</a> and a <a href="https://cran.r-project.org/web/packages/leafdown/vignettes/Showcase_electionapp.html">Showcase</a> example.</p>
<p><img src="leafdown.png" height = "600" width="400"></p>
<p><a href="https://cran.r-project.org/package=mapping">mapping</a> v1.2: Provides coordinates, linking and mapping functions for mapping workflows of different geographical statistical units. Geographical coordinates automatically link with the input data to generate maps. See the <a href="https://cran.r-project.org/web/packages/mapping/vignettes/a-journey-into-mapping.html">vignette</a> to get started.</p>
<p><img src="mapping.png" height = "350" width="500"></p>
<p><a href="https://cran.r-project.org/package=materialmodifier">materialmodifier</a> v1.0.0: Provides functions to apply image processing effects to modify the perceived material properties such as gloss, smoothness, and blemishes. Look <a href="https://github.com/tsuda16k/materialmodifier">here</a> for documentation and practical tips of the package is available at</p>
<p><img src="material.png" height = "400" width="600"></p>
<p><a href="https://cran.r-project.org/package=svplots">svplots</a> v0.1.0: Implements two versions of sample variance plots illustrating the squared deviations from sample variance as described in <a href="https://www.tandfonline.com/doi/abs/10.1080/03610918.2020.1851716?journalCode=lssp20">Wijesuriya (2020)</a>. See the <a href="https://cran.r-project.org/web/packages/svplots/vignettes/Svplots_and_Testing_Hypothes.html">vignette</a>.</p>
<p><img src="svplots.png" height = "300" width="500"></p>
<p><a href="https://cran.r-project.org/package=vivid">vivid</a> v0.1.0: Provides a suite of plots for displaying variable importance and two-way variable interaction. Plots include partial dependence plots laid out in “pairs plot”” or <a href="https://www.zenplot.com/en?gclid=CjwKCAjwy42FBhB2EiwAJY0yQi2mfla-DMC2uuDglAGzh1mUx4sYyT5p8uEmfWgeMv5gKBh_5V2RDxoC6jEQAvD_BwE">zenplots</a> style. There is an <a href="https://cran.r-project.org/web/packages/vivid/vignettes/vivid.html">Introduction</a> and a <a href="https://cran.r-project.org/web/packages/vivid/vignettes/vividQStart.html">Quick Start Guide</a>.</p>
<p><img src="vivid.png" height = "500" width="300"></p>
<script>window.location.href='https://rviews.rstudio.com/2021/05/25/april-2021-top-40-new-cran-packages/';</script>
Basic FDA Descriptive Statistics with R
https://rviews.rstudio.com/2021/05/14/basic-fda-descriptive-statistics-with-r/
Fri, 14 May 2021 00:00:00 +0000https://rviews.rstudio.com/2021/05/14/basic-fda-descriptive-statistics-with-r/
<script src="/2021/05/14/basic-fda-descriptive-statistics-with-r/index_files/header-attrs/header-attrs.js"></script>
<script src="/2021/05/14/basic-fda-descriptive-statistics-with-r/index_files/htmlwidgets/htmlwidgets.js"></script>
<script src="/2021/05/14/basic-fda-descriptive-statistics-with-r/index_files/plotly-binding/plotly.js"></script>
<script src="/2021/05/14/basic-fda-descriptive-statistics-with-r/index_files/typedarray/typedarray.min.js"></script>
<script src="/2021/05/14/basic-fda-descriptive-statistics-with-r/index_files/jquery/jquery.min.js"></script>
<link href="/2021/05/14/basic-fda-descriptive-statistics-with-r/index_files/crosstalk/css/crosstalk.css" rel="stylesheet" />
<script src="/2021/05/14/basic-fda-descriptive-statistics-with-r/index_files/crosstalk/js/crosstalk.min.js"></script>
<link href="/2021/05/14/basic-fda-descriptive-statistics-with-r/index_files/plotly-htmlwidgets-css/plotly-htmlwidgets.css" rel="stylesheet" />
<script src="/2021/05/14/basic-fda-descriptive-statistics-with-r/index_files/plotly-main/plotly-latest.min.js"></script>
<p>In a previous post, I introduced the topic of Functional Data Analysis (FDA). In that post, I provided some background on Functional Analysis, the mathematical theory that makes FDA possible, identified FDA resources that might be of interest R users, and showed how to turn a series of data points into an FDA object. In this post, I will pick up where I left off and move on to doing some very basic FDA descriptive statistics.</p>
<p>Let’s continue with the same motivating example from last time. We will use synthetic data generated by a Brownian motion process and pretend that it is observed longitudinal data. However, before getting to the statistics, I would like to take a tiny, tidy diversion. The functions in <code>fda</code> and other fundamental FDA R packages require data structured in matrices. Consequently, the examples in the basic FDA reference works (listed below) construct matrices using code that seems to be convenient for the occasion. I think this makes adapting sample code to user data a little harder than it needs to be. There ought to be standard data structures for working with FDA data. I propose tibbles or data frames with function values packed into lists.</p>
<p>The following function generates <code>n_points</code> data points for each of <code>n_curve</code> Brownian motion curves that represent the longitudinal data collected from <code>n_curve</code> subjects.</p>
<pre class="r"><code>library(fda)
library(tidyverse)
library(plotly)
# Function to simulate data
fake_curves <- function(n_curves = 100, n_points = 80, max_time = 100){
ID <- 1:n_curves
x <- vector(mode = "list", length = n_curves)
t <- vector(mode = "list", length = n_curves)
for (i in 1:n_curves){
t[i] <- list(sort(runif(n_points,0,max_time)))
x[i] <- list(cumsum(rnorm(n_points)) / sqrt(n_points))
}
df <- tibble(ID,t,x)
names(df) <- c("ID", "Time", "Curve")
return(df)
}</code></pre>
<p>Notice that each curve is associated with a unique set of random time points that lie in the interval [0, max_time]. Not being restricted to situations where data from all subjects must be observed at the same times is a big deal. However, in practice you may encounter problems that will require curve alignment procedures. We will ignore this for now. Note that the variables <code>Time</code> and <code>Curve</code> contain lists of data points in each cell.</p>
<pre class="r"><code>set.seed(123)
n_curves <- 40
n_points <- 80
max_time <- 100
df <- fake_curves(n_curves = n_curves,n_points = n_points, max_time = max_time)
head(df)</code></pre>
<pre><code>## # A tibble: 6 x 3
## ID Time Curve
## <int> <list> <list>
## 1 1 <dbl [80]> <dbl [80]>
## 2 2 <dbl [80]> <dbl [80]>
## 3 3 <dbl [80]> <dbl [80]>
## 4 4 <dbl [80]> <dbl [80]>
## 5 5 <dbl [80]> <dbl [80]>
## 6 6 <dbl [80]> <dbl [80]></code></pre>
<p>Later on, this kind of structure will be convenient for data sets that contain both FDA curves and other scalar covariates. Note that if you are using the RStudio IDE running the function <code>View(df)</code> will show you an expanded view of the tibble under a tab labeled df that should look something like this:</p>
<p><img src="df.png" height = "400" width="600"></p>
<p>Next, we unpack the data into a long form tibble and plot.</p>
<pre class="r"><code>df_1 <- df %>% select(!c(ID,Curve)) %>% unnest_longer(Time)
df_2 <- df %>% select(!c(ID,Time)) %>% unnest_longer(Curve)
ID <- sort(rep(1:n_curves,n_points))
df_l <- cbind(ID,df_1,df_2)
p <- ggplot(df_l, aes(x = Time, y = Curve, group = ID, col = ID)) +
geom_line()
p</code></pre>
<p><img src="/2021/05/14/basic-fda-descriptive-statistics-with-r/index_files/figure-html/unnamed-chunk-3-1.png" width="672" />
Now that we have the data, remember that FDA treats each curve as a basic data element living in an infinite dimensional vector space. The vectors, <span class="math inline">\(X\)</span>, are random functions: <span class="math inline">\(X: \Omega \Rightarrow \mathscr{H}\)</span> where <span class="math inline">\(\Omega\)</span> is an underlying probability space and <span class="math inline">\(\mathscr{H}\)</span> is typically a complete Hilbert Space of square integrable functions. That is, for each <span class="math inline">\(\omega \in \Omega\)</span>, <span class="math inline">\(\parallel X(\omega) \parallel ^2 = \int X((\omega)(t))^2 dt < \infty\)</span>. In multivariate statistics we work with random variables that live in a Euclidean space, here we are dealing with random functions that live in a Hilbert space. In this context, square integrable means <span class="math inline">\(E \parallel X \parallel ^2 < \infty\)</span>. You will find lucid elaborations of all of this in the references below which I have reproduced below from the previous post.</p>
<p>The bridge from the theory to practice is the ability to represent the random functions as a linear combination of basis vectors. This was the topic of the previous post. Here is some compact code to construct the basis.</p>
<pre class="r"><code>knots = c(seq(0,max_time,5)) #Location of knots
n_knots = length(knots) #Number of knots
n_order = 4 # order of basis functions: for cubic b-splines: order = 3 + 1
n_basis = length(knots) + n_order - 2;
basis = create.bspline.basis(rangeval = c(0,max_time), n_basis)
plot(basis)</code></pre>
<p><img src="/2021/05/14/basic-fda-descriptive-statistics-with-r/index_files/figure-html/unnamed-chunk-4-1.png" width="672" />
This next bit of code formats the data in the long form tibble into the matrix input expected by the <code>fda</code> functions and creates an <code>fda</code> object that contains the coefficients and basis functions used to smooth data. Note the smoothing constant of lambda = .5.</p>
<pre class="r"><code>argvals <- matrix(df_l$Time, nrow = n_points, ncol = n_curves)
y_mat <- matrix(df_l$Curve, nrow = n_points, ncol = n_curves)
W.obj <- Data2fd(argvals = argvals, y = y_mat, basisobj = basis, lambda = 0.5)</code></pre>
<p>Next somewhat anticlimactically after all of the preparation and theory, we use the <code>fda</code> functions <code>mean.fd()</code> and <code>std.fd()</code> to calculate the pointwise mean and standard deviation from information contained in <code>fda</code> object. In order to use these objects to calculate the pointwise confidence interval for the mean it is necessary to construct a couple of new <code>fda</code> objects for the upper and lower curves. Then, we plot the smoothed curves for our data along with the pointwise mean and pointwise 95% confidence bands for the mean.</p>
<pre class="r"><code>W_mean <- mean.fd(W.obj)
W_sd <- std.fd(W.obj)
# Create objects for the standard upper and lower standard deviation
SE_u <- fd(basisobj = basis)
SE_l <- fd(basisobj = basis)
# Fill in the sd values
SE_u$coefs <- W_mean$coefs + 1.96 * W_sd$coefs/sqrt(n_curves)
SE_l$coefs <- W_mean$coefs - 1.96 * W_sd$coefs/sqrt(n_curves)
plot(W.obj, xlab="Time", ylab="", lty = 1)</code></pre>
<pre><code>## [1] "done"</code></pre>
<pre class="r"><code>title(main = "Smoothed Curves")
lines(SE_u, lwd = 3, lty = 3)
lines(SE_l, lwd = 3, lty = 3)
lines(W_mean, lwd = 3)</code></pre>
<p><img src="/2021/05/14/basic-fda-descriptive-statistics-with-r/index_files/figure-html/unnamed-chunk-6-1.png" width="672" />
Finally, we compute the covariance/correlation matrix for our sample of smoothed curves and use <a href="https://plotly.com/r/3d-surface-plots/"><code>plotly</code></a> to create an interactive plot of the three dimensional correlation surface along with a contour map.</p>
<pre class="r"><code>days <- seq(0,100, by=2)
cov_W <- var.fd(W.obj)
var_mat <- eval.bifd(days,days,cov_W)</code></pre>
<pre class="r"><code>fig <- plot_ly(x = days, y = days, z = ~var_mat) %>%
add_surface(contours = list(
z = list(show=TRUE,usecolormap=TRUE, highlightcolor="#ff0000", project=list(z=TRUE))))
fig <- fig %>%
layout(scene = list(camera=list(eye = list(x=1.87, y=0.88, z=-0.64))))
fig</code></pre>
<p><div id="htmlwidget-1" style="width:672px;height:480px;" class="plotly html-widget"></div>
<script type="application/json" data-for="htmlwidget-1">{"x":{"visdat":{"10fc4fe7d04f":["function () ","plotlyVisDat"]},"cur_data":"10fc4fe7d04f","attrs":{"10fc4fe7d04f":{"x":[0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78,80,82,84,86,88,90,92,94,96,98,100],"y":[0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78,80,82,84,86,88,90,92,94,96,98,100],"z":{},"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"type":"surface","contours":{"z":{"show":true,"usecolormap":true,"highlightcolor":"#ff0000","project":{"z":true}}},"inherit":true}},"layout":{"margin":{"b":40,"l":60,"t":25,"r":10},"scene":{"camera":{"eye":{"x":1.87,"y":0.88,"z":-0.64}},"xaxis":{"title":[]},"yaxis":{"title":[]},"zaxis":{"title":"var_mat"}},"hovermode":"closest","showlegend":false,"legend":{"yanchor":"top","y":0.5}},"source":"A","config":{"showSendToCloud":false},"data":[{"colorbar":{"title":"var_mat","ticklen":2,"len":0.5,"lenmode":"fraction","y":1,"yanchor":"top"},"colorscale":[["0","rgba(68,1,84,1)"],["0.0416666666666667","rgba(70,19,97,1)"],["0.0833333333333333","rgba(72,32,111,1)"],["0.125","rgba(71,45,122,1)"],["0.166666666666667","rgba(68,58,128,1)"],["0.208333333333333","rgba(64,70,135,1)"],["0.25","rgba(60,82,138,1)"],["0.291666666666667","rgba(56,93,140,1)"],["0.333333333333333","rgba(49,104,142,1)"],["0.375","rgba(46,114,142,1)"],["0.416666666666667","rgba(42,123,142,1)"],["0.458333333333333","rgba(38,133,141,1)"],["0.5","rgba(37,144,140,1)"],["0.541666666666667","rgba(33,154,138,1)"],["0.583333333333333","rgba(39,164,133,1)"],["0.625","rgba(47,174,127,1)"],["0.666666666666667","rgba(53,183,121,1)"],["0.708333333333333","rgba(79,191,110,1)"],["0.75","rgba(98,199,98,1)"],["0.791666666666667","rgba(119,207,85,1)"],["0.833333333333333","rgba(147,214,70,1)"],["0.875","rgba(172,220,52,1)"],["0.916666666666667","rgba(199,225,42,1)"],["0.958333333333333","rgba(226,228,40,1)"],["1","rgba(253,231,37,1)"]],"showscale":true,"x":[0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78,80,82,84,86,88,90,92,94,96,98,100],"y":[0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78,80,82,84,86,88,90,92,94,96,98,100],"z":[[0.0300141214765612,0.015686905073093,0.00564093884922246,0.00447256728960355,0.00995490109738018,0.0130378171941865,0.00725655484167828,-0.00232554798509603,-0.00801467720093724,-0.00790263578918598,-0.00586684380172273,-0.00503976012652639,-0.00456780472318001,-0.00268331461816891,0.000485556784993114,0.00291483970576381,0.00310419846129833,0.00197754095554643,0.00105330118737587,0.00158461271469904,0.00455930865447297,0.0102627510155511,0.0162165565814951,0.0193760713693298,0.0195639488111623,0.0194701497541815,0.0210337796708257,0.0226918603753865,0.0221081111780568,0.0192972748291135,0.0166251175189173,0.0157786309103451,0.015320834914587,0.013141952079488,0.00962070316816562,0.00762430715900963,0.0091589006740241,0.0123535086113619,0.0144475321088493,0.01451241822894,0.0134516599587157,0.012184664428522,0.0113758786592171,0.0115914994526642,0.0126039892034018,0.0133920758986437,0.0132433983079006,0.0128192790963617,0.0130612176454187,0.0134560954089633,0.0120361759133872],[0.015686905073093,0.0226633175061417,0.0244188932917484,0.0250220663890372,0.0258019153407324,0.025348163273158,0.0226239217260867,0.0185622655307192,0.0145795755299542,0.0118809819100223,0.011460364200486,0.0136427970615,0.0161148749202248,0.0160327747314867,0.0135243329261398,0.0116890454110662,0.0127990751660581,0.0153004431188909,0.0167891143432511,0.0173100860678814,0.0193573876765801,0.0245339491722834,0.0304523843988157,0.0338539590040724,0.0343832480444693,0.0345881359849426,0.0363282575780583,0.0382053253515058,0.0380840687391034,0.0356116588086777,0.0322177082620627,0.0291246356267592,0.0264160928835393,0.0239139417559029,0.0219667756937601,0.0214499198734321,0.0229121181584461,0.0255041836435166,0.0280467134870814,0.029803440233887,0.0304812318149881,0.0298735894418936,0.0280434804242242,0.0251388192376269,0.021711875112217,0.0187192720325778,0.0169049313606271,0.0160916414016255,0.0158522428499515,0.01530729642547,0.0131250828735328],[0.00564093884922246,0.0244188932917484,0.0404127823425623,0.0483733511242439,0.0480174477822059,0.0440280234846942,0.0403499317976119,0.0371119657865627,0.0335517590970443,0.0303526265888372,0.0296435643360046,0.0324526349389525,0.035152744283584,0.0331828317842279,0.0272677591173969,0.0234183102217875,0.026004021647384,0.0319101495611166,0.0363122500732657,0.0380973901937379,0.0398641478320654,0.0435528802177005,0.0478255817033191,0.0506029719293355,0.0516070386956147,0.0523610379614717,0.0538230640251236,0.0543773005126641,0.0518705141914604,0.0465888704428848,0.0412579332623144,0.0380833303825961,0.0367667012492371,0.0364074277070802,0.0366508994378863,0.0376885139603343,0.0396474612639257,0.0423031433367703,0.0453253497513456,0.0479775655285843,0.0491169711378746,0.0478109208618154,0.0440381258969864,0.0380544047895784,0.0312487428714465,0.0261432922601097,0.0245335471295971,0.0251112973955133,0.0258242178246622,0.025351634247369,0.0231045235574797],[0.00447256728960355,0.0250220663890372,0.0483733511242439,0.0658210619904763,0.0737960986504746,0.0738656199344664,0.0682277414378763,0.0607111433530042,0.0554453449883687,0.0541034188934748,0.055901990858814,0.059498497561025,0.0617408273533381,0.0592276422063684,0.0532303001501739,0.049692855274256,0.0528918336035986,0.0596210061121686,0.0649634904915878,0.0676832204828881,0.0702249458765112,0.0744621022049338,0.0793427260471337,0.0831343126365808,0.0852729502426609,0.0863633201706759,0.0866778286903351,0.0849565483575114,0.0796664880504596,0.0718051109167692,0.0649003343733642,0.0616710344942862,0.0611598429786833,0.0615355063179176,0.0620058163799042,0.0628176104091112,0.0643008086961966,0.0669369601728294,0.0711766273093092,0.0758858192786237,0.0783459919564488,0.0764513991188953,0.0708230606303023,0.0627955601389712,0.0544365433170178,0.0485467178603732,0.0470326094185766,0.0480965264079801,0.0490696898156232,0.0485475588451775,0.0463898309169458],[0.00995490109738018,0.0258019153407324,0.0480174477822059,0.0737960986504746,0.0971085914615697,0.10870177301888,0.102995904418255,0.0896655781362864,0.081799993583024,0.0833119041988271,0.0889376174543645,0.0937835994937921,0.0960328548021177,0.0947403483065957,0.0913260396462496,0.0895748831718718,0.0921893839315523,0.0971309463344363,0.10129314750544,0.104270865706126,0.108360280334708,0.115145590152833,0.122893279873863,0.129099970422491,0.132632271361622,0.133726780892375,0.132535733834645,0.128635652589831,0.121538220159228,0.112574159748106,0.104893234765713,0.1008406212425,0.0992267908756421,0.098035543560513,0.0965617776673574,0.0954114900412902,0.095367918350293,0.097695246427747,0.103646726979766,0.111458756334261,0.11635087444094,0.114697704490176,0.108018873309467,0.099107724487911,0.0904693162179292,0.084320421295265,0.0821118946249729,0.0822810561400769,0.0825592190496153,0.0817673680001241,0.0798161590756366],[0.0130378171941865,0.025348163273158,0.0440280234846942,0.0738656199344664,0.10870177301888,0.131429901425073,0.130338371899877,0.117199239378325,0.109151554151168,0.111744052608463,0.118935157239576,0.125205625814209,0.129141263988743,0.130406449035972,0.129868680057772,0.129598577985106,0.131177622726623,0.134021491522753,0.137061062815098,0.140530208014422,0.145965791500655,0.154228736045967,0.163249590518587,0.170292721388971,0.174149949926465,0.175140552199302,0.173523514301647,0.169051018118542,0.161419129688686,0.151947374787411,0.143578738926672,0.138481517872149,0.135442907580638,0.132469174994164,0.129046543615972,0.12614119351053,0.124904427310549,0.126970654171923,0.133934823992636,0.143706461230417,0.150509664902736,0.149900457099137,0.143403495752272,0.134024472210158,0.124513190276249,0.117363566209445,0.114287591739835,0.113914062663951,0.114141378734128,0.113797207304149,0.112638483329245],[0.00725655484167828,0.0226239217260867,0.0403499317976119,0.0682277414378763,0.102995904418255,0.130338371899877,0.139688182032744,0.137397260506744,0.133823360055384,0.134174652061097,0.138509726555246,0.146085330009431,0.153846414021606,0.158293537808086,0.158999056330915,0.158607120297858,0.159403311653335,0.16170471194128,0.165336738878575,0.170135419462684,0.175947389971655,0.182424713480816,0.188439315162361,0.192711088947525,0.194949003168609,0.195851100558978,0.195717042964027,0.19308095573795,0.186158898282678,0.176003247922498,0.166502699904055,0.160623618434228,0.15714977087869,0.153889591090921,0.150268773980867,0.147330275514936,0.146229194592453,0.148287939051957,0.154740975922443,0.163838107212786,0.170844469911743,0.171861305927171,0.166853349020735,0.1568316981457,0.144658773502766,0.135048316540061,0.131476061460251,0.132143744571539,0.133992284753065,0.135289287908722,0.135629046967152],[-0.00232554798509603,0.0185622655307192,0.0371119657865627,0.0607111433530042,0.0896655781362864,0.117199239378325,0.137397260506744,0.149021050937968,0.152158909046904,0.150528992318004,0.151479317345255,0.160019309553339,0.17117274984456,0.177717176035795,0.178183990982668,0.176858462579558,0.177467053922628,0.180500336734544,0.185609272525172,0.191740163307645,0.197134651598366,0.200408010855296,0.201792587780331,0.201934709781931,0.201704089367035,0.202193824141059,0.203687257405181,0.203189865661295,0.197077963680589,0.186104880523814,0.175400959541291,0.169000153079858,0.165789629595956,0.1634337203065,0.161065507288953,0.159286823481326,0.158833987946666,0.160725829315896,0.165959553302808,0.173410608511967,0.179832686438706,0.182195069327086,0.178698403652472,0.168035338100033,0.153134227850431,0.141159134579828,0.137488888247712,0.139624747338963,0.143184656221075,0.145766370540687,0.146947457223577],[-0.00801467720093724,0.0145795755299542,0.0335517590970443,0.0554453449883687,0.081799993583024,0.109151554151168,0.133823360055384,0.152158909046904,0.160759913976905,0.162051089760505,0.164280153376763,0.173284826371914,0.184250152549762,0.189965437082023,0.189370893621324,0.18755764430121,0.188800283197836,0.193037568943746,0.199106383544097,0.205431500374914,0.210025584183091,0.211435296519902,0.210414957220078,0.208268791568572,0.20625481507554,0.205584833476341,0.206553546822989,0.205795268916695,0.199235499638045,0.187544239635493,0.176135990325352,0.169354995508846,0.166439337958515,0.165382190838407,0.164904232232213,0.164453645143262,0.163936442039167,0.164963430649948,0.169454935750345,0.176647601384842,0.183094390867664,0.185548505510878,0.182030825701433,0.171074994035733,0.155719029247019,0.143505326205363,0.140040182937765,0.142402140335997,0.145642027044141,0.147207893536936,0.146945012119781],[-0.00790263578918598,0.0118809819100223,0.0303526265888372,0.0541034188934748,0.0833119041988271,0.111744052608463,0.134174652061097,0.150528992318004,0.162051089760505,0.170700287551338,0.179151255634655,0.188978321865683,0.197230041086307,0.200018353165535,0.197936248422864,0.196057767628283,0.198383944655527,0.203844472671486,0.210131452167584,0.215632561163899,0.219431055209156,0.220875817626191,0.220257272829271,0.218115273498233,0.215312661145223,0.21303526611469,0.211703065462037,0.20861645111564,0.200474760734835,0.188090698271463,0.176390333969868,0.169382638465077,0.166692815903987,0.166839231539204,0.168089613361411,0.168461052099373,0.166954905361007,0.166553187359872,0.171034412275775,0.179607818029667,0.186913366283649,0.188377265566261,0.183365368612479,0.172349353976725,0.158581939873738,0.148096884178574,0.14520462632401,0.146838672902345,0.148201202752072,0.147091220206974,0.143904555096121],[-0.00586684380172273,0.011460364200486,0.0296435643360046,0.055901990858814,0.0889376174543645,0.118935157239576,0.138509726555246,0.151479317345255,0.164280153376763,0.179151255634655,0.194134442321539,0.207015416909719,0.215285372260236,0.216547915360939,0.212685614130958,0.20985999742071,0.212748494985508,0.219347973081767,0.226063519070894,0.231102545847802,0.234474791840914,0.236211400126787,0.236115684455659,0.233972826276096,0.230460982724484,0.227151286625036,0.224790732613606,0.220674462452998,0.211423082374906,0.197991035415441,0.185666601415132,0.178728193736514,0.176661049081538,0.177732671916507,0.179763491086094,0.180126859813348,0.177508440228303,0.175920882370973,0.180448181283924,0.19018508658607,0.198237102472678,0.198945893456674,0.192639373221394,0.181216192008275,0.168281008165233,0.159144488146665,0.157417469158103,0.159614774347633,0.1606122604985,0.158391638832914,0.154040475012052],[-0.00503976012652639,0.0136427970615,0.0324526349389525,0.059498497561025,0.0937835994937921,0.125205625814209,0.146085330009431,0.160019309553339,0.173284826371914,0.188978321865683,0.207015416909719,0.226109065876101,0.240714743303262,0.244563014165053,0.239192032493299,0.233947541377799,0.236123607806666,0.243449752785962,0.251354795250806,0.257347545901448,0.261016807203268,0.2621094067394,0.260642534865696,0.256757706272572,0.251901309402189,0.248824606448451,0.248855362704169,0.247400421292272,0.238677363741663,0.223596057395594,0.20975865541166,0.203180237592353,0.202363714525955,0.20396716548771,0.205412522534698,0.204885570505838,0.201735186368487,0.199829771641696,0.203924162419897,0.213173923859123,0.221135350177003,0.222354612483207,0.216311175705738,0.203857529124282,0.189059282651913,0.179195166835087,0.179292120516932,0.184811112138843,0.189007860808978,0.189421680781059,0.187875481454371],[-0.00456780472318001,0.0161148749202248,0.035152744283584,0.0617408273533381,0.0960328548021177,0.129141263988743,0.153846414021606,0.171172749844561,0.184250152549762,0.197230041086307,0.215285372260236,0.240714743303262,0.264141654392061,0.273778057962538,0.269502116479285,0.262858202435585,0.263266344191042,0.269620287478599,0.278215275521839,0.28610711893243,0.29111219571012,0.291611942233209,0.288115755804973,0.281735341820846,0.275199716477354,0.272855206772115,0.276772608953709,0.279639324830909,0.272228571166917,0.255241826705762,0.239308832172306,0.232761966132105,0.23301750425598,0.234863600825815,0.23540920982273,0.234084084927079,0.231119034228133,0.229545466111812,0.23291543691702,0.240652454195423,0.248051476711448,0.25073177323223,0.246327861020472,0.233237750290338,0.215832110290119,0.204454269302239,0.206439741046498,0.216054057681938,0.224544341762145,0.228886688889647,0.231786167715914],[-0.00268331461816891,0.0160327747314867,0.0331828317842279,0.0592276422063684,0.0947403483065957,0.130406449035972,0.158293537808086,0.177717176035795,0.189965437082023,0.200018353165535,0.216547915360938,0.244563014165053,0.273778057962538,0.290572798312891,0.292573151941868,0.288651200741483,0.286737733825309,0.289042514381638,0.296222610813786,0.30611901054109,0.313756619998902,0.315175948630781,0.310969671127717,0.302955086674437,0.294940907638863,0.292727259572112,0.299164990782613,0.304961507880513,0.298318983747391,0.279644005272815,0.261547573354341,0.253979104530278,0.254119171117953,0.25607802348289,0.256769361369846,0.255910333902806,0.253830406873757,0.252820756332902,0.25554153363934,0.261859488272072,0.268847967829996,0.273294804653498,0.271331579071523,0.259269362595121,0.241320972989505,0.22960097427405,0.232798630393512,0.244527788254317,0.254931804696063,0.261016412991644,0.266649722847255],[0.000485556784993114,0.0135243329261398,0.0272677591173969,0.0532303001501739,0.0913260396462496,0.129868680057772,0.158999056330915,0.178183990982668,0.189370893621324,0.197936248422864,0.212685614130957,0.239192032493299,0.269502116479285,0.292573151941868,0.304084866530968,0.306439429693472,0.303057606677035,0.300265425619857,0.30480328575577,0.316236859897483,0.326957094436583,0.33064485798658,0.327388486556694,0.318988840468754,0.309791901709214,0.306688773929157,0.313271127157899,0.319490267082992,0.312455188069513,0.29233381903075,0.272353023428209,0.263086482662052,0.262224030835217,0.264331737959722,0.266211221915455,0.266899648450175,0.266091910995561,0.265725022785431,0.268205597825174,0.273848879004003,0.280878738094951,0.286898206164275,0.28739066816697,0.27767308912116,0.261422740636704,0.250677200915191,0.254094213698673,0.26535821852726,0.274730759985274,0.279843297835096,0.285697207017158],[0.00291483970576381,0.0116890454110662,0.0234183102217875,0.049692855274256,0.0895748831718718,0.129598577985106,0.158607120297858,0.176858462579558,0.18755764430121,0.196057767628283,0.20985999742071,0.233947541377799,0.262858202435585,0.288651200741483,0.306439429693472,0.314389455940099,0.312704592994684,0.309204068220854,0.313232618676246,0.325430199741382,0.337731985119672,0.343248592386973,0.341306289683202,0.332931048721084,0.322502032644519,0.317751596028588,0.322945736593217,0.327901860910561,0.319408078209404,0.29765805792069,0.276251029677522,0.266088035103276,0.26494596200812,0.26741130770582,0.270155472518723,0.271934759777765,0.272178336907421,0.27264423587875,0.275594391087112,0.281489521543979,0.288989130876933,0.296033547649337,0.297999655017049,0.289977868944833,0.275209671095333,0.265087608829077,0.267839041596672,0.277613000565924,0.285352127040801,0.289202483083125,0.294513551512573],[0.00310419846129833,0.0127990751660581,0.026004021647384,0.0528918336035986,0.0921893839315523,0.131177622726623,0.159403311653335,0.177467053922628,0.188800283197836,0.198383944655527,0.212748494985508,0.236123607806666,0.263266344191042,0.286737733825309,0.303057606677035,0.312704592994684,0.317105958549478,0.320795024198311,0.328855480815721,0.341169712939617,0.352418798771269,0.35791411030228,0.356392769700643,0.347634698529288,0.335706140920473,0.328959663575778,0.332178669150744,0.33512445671338,0.324411072215951,0.300642835576825,0.278414340680476,0.269212797906219,0.26966279248818,0.27282133348202,0.275151272423422,0.276521303328098,0.277252984769834,0.278887013123408,0.283196363791438,0.290286387461154,0.298594810104402,0.305911837133245,0.307727613410256,0.299305263018981,0.283911780284874,0.2728180297753,0.274256187773986,0.282911699996743,0.290363130038896,0.294624453490113,0.300145057934403],[0.00197754095554643,0.0153004431188909,0.0319101495611166,0.0596210061121686,0.0971309463344363,0.134021491522753,0.16170471194128,0.180500336734544,0.193037568943746,0.203844472671486,0.219347973081767,0.243449752785962,0.269620287478599,0.289042514381637,0.300265425619857,0.309204068220854,0.320795024198311,0.334779962483662,0.349582114419545,0.363171397029049,0.37306441301572,0.377071908776566,0.374260042755413,0.36420041359492,0.350871125241779,0.342656786946715,0.344349675078645,0.345606385730735,0.332936500117839,0.30749841883916,0.285099361878255,0.277894716559139,0.280884918805161,0.285014172433834,0.286574313382245,0.287204809707052,0.288632814369124,0.292006196968851,0.298266092384556,0.306929616209746,0.316089864753118,0.323293649031453,0.324150294421663,0.314137195281634,0.296837874960971,0.283941985800991,0.2841000136905,0.292396017144489,0.300764836732139,0.306716257668493,0.313335009814453],[0.00105330118737587,0.0167891143432511,0.0363122500732657,0.0649634904915878,0.10129314750544,0.137061062815098,0.165336738878575,0.185609272525172,0.199106383544097,0.210131452167584,0.226063519070894,0.251354795250806,0.278215275521839,0.296222610813786,0.30480328575577,0.313232618676246,0.328855480815721,0.349582114419545,0.370984151742953,0.389094311990697,0.400406401317044,0.402341684864012,0.395951074689911,0.383309110744288,0.369163304706409,0.360934139985263,0.362547165055812,0.36348332400828,0.350264985949383,0.324423770577444,0.302500548182388,0.297055887456202,0.3021188462484,0.307373832604049,0.309134496410136,0.310343729395571,0.313679062288349,0.319603670188119,0.327992114572648,0.337913388479374,0.347630916505408,0.354837546540918,0.35508391850748,0.3437550001667,0.324742995420216,0.310447344309692,0.3101171483734,0.318920635807337,0.328733235184467,0.336661005466127,0.345040636002025],[0.00158461271469904,0.0173100860678814,0.0380973901937379,0.0676832204828881,0.104270865706126,0.140530208014422,0.170135419462684,0.191740163307645,0.205431500374914,0.215632561163899,0.231102545847802,0.257347545901448,0.28610711893243,0.30611901054109,0.316236859897483,0.325430199741382,0.341169712939617,0.363171397029049,0.389094311990697,0.413877394668567,0.429739458769652,0.431049389052446,0.421249419374843,0.406064538201852,0.39155133264059,0.384097988440287,0.386666813598334,0.388458935775705,0.375894171990615,0.350548981412123,0.329156465360131,0.324417648570495,0.330269311677518,0.336262693460866,0.338975973627527,0.342014272811809,0.348387788586863,0.357504949598187,0.367888852076066,0.378410118034793,0.388286895272678,0.39594582808551,0.396587107492231,0.385004366348378,0.365192506093068,0.350347696748992,0.350321470200624,0.359986585156321,0.370722880057251,0.379700972961929,0.389882261546216],[0.00455930865447297,0.0193573876765801,0.0398641478320654,0.0702249458765112,0.108360280334708,0.145965791500655,0.175947389971655,0.197134651598366,0.210025584183091,0.219431055209156,0.234474791840914,0.261016807203268,0.29111219571012,0.313756619998902,0.326957094436583,0.337731985119672,0.352418798771269,0.37306441301572,0.400406401317044,0.429739458769652,0.450915402098339,0.456227225263344,0.448679218709221,0.43399327559022,0.418806196598807,0.410669689965659,0.413411559059499,0.415804873446741,0.40353871876728,0.37793526971434,0.355949790034467,0.350500579454025,0.355639283522414,0.361021505897904,0.363677148260339,0.368010410311134,0.377389876800323,0.389759186059976,0.401796610629334,0.412270451715893,0.422039039195408,0.430764527316019,0.432960884568757,0.422295336741163,0.402562095543856,0.387682358610529,0.387994106900862,0.39774123973533,0.407466892609289,0.415137602465859,0.426143307695915],[0.0102627510155511,0.0245339491722834,0.0435528802177005,0.0744621022049338,0.115145590152833,0.154228736045967,0.182424713480816,0.200408010855296,0.211435296519902,0.220875817626191,0.236211400126787,0.262109406739399,0.291611942233209,0.315175948630781,0.33064485798658,0.343248592386973,0.35791411030228,0.377071908776566,0.402341684864012,0.431049389052446,0.456227225263344,0.471713187952435,0.475315172245749,0.466116373399275,0.44970496080387,0.438174077985257,0.439084189309815,0.440733742656996,0.427329602525772,0.399728810463848,0.375438585067662,0.367819908526032,0.370753134432285,0.373607043309506,0.373997849489237,0.377789201111481,0.389187998900115,0.404339817600011,0.417561860121391,0.427471401556844,0.436985789181323,0.447303940848557,0.452003218086965,0.443239184389206,0.424289997295981,0.409556408396027,0.409624129776975,0.41788435133975,0.423866209365925,0.427140109465569,0.437317726577243],[0.0162165565814951,0.0304523843988157,0.0478255817033191,0.0793427260471338,0.122893279873863,0.163249590518587,0.188439315162361,0.201792587780331,0.210414957220078,0.220257272829271,0.236115684455659,0.260642534865696,0.288115755804973,0.310969671127718,0.327388486556694,0.341306289683202,0.356392769700643,0.374260042755413,0.395951074689911,0.421249419374843,0.448679218709221,0.475315172245749,0.492653238320889,0.491359229692647,0.475083372807843,0.460460307802116,0.458942148751755,0.458992746593247,0.444068798352341,0.414644974161663,0.388213917260719,0.378121765487047,0.378168987573464,0.377661321888484,0.374913305773777,0.377248275544171,0.389748533039457,0.406965494551783,0.421086008099859,0.430464443312404,0.439622691428143,0.450989028425408,0.457544656787058,0.450414823856973,0.432358481381503,0.417768285509471,0.417094601932727,0.42299538157567,0.424227190249541,0.422121121075806,0.430582794486151],[0.0193760713693298,0.0338539590040724,0.0506029719293355,0.0831343126365808,0.129099970422491,0.170292721388971,0.192711088947525,0.201934709781931,0.208268791568572,0.218115273498233,0.233972826276096,0.256757706272572,0.281735341820845,0.302955086674437,0.318988840468754,0.332931048721084,0.347634698529288,0.36420041359492,0.383309110744288,0.406064538201852,0.43399327559022,0.466116373399275,0.491359229692647,0.496771232159396,0.484303531324282,0.470809040547772,0.46829506710943,0.466794883037271,0.451513381679334,0.422953449440548,0.396915965782739,0.38544867037553,0.382926223188107,0.379686126428383,0.374831462072606,0.376230891865352,0.389241041266658,0.407645941943051,0.422630307690389,0.432136955591859,0.440866806017986,0.451495296269285,0.457420609055407,0.450202304850737,0.432455846616029,0.417852599798406,0.416199279096981,0.419921230569029,0.417696722356713,0.411964100245248,0.418921787662896],[0.0195639488111623,0.0343832480444693,0.0516070386956147,0.0852729502426609,0.132632271361622,0.174149949926465,0.194949003168609,0.201704089367035,0.20625481507554,0.215312661145223,0.230460982724484,0.251901309402189,0.275199716477354,0.294940907638863,0.309791901709214,0.322502032644519,0.335706140920473,0.350871125241779,0.369163304706409,0.39155133264059,0.418806196598807,0.44970496080387,0.475083372807843,0.484303531324282,0.4784961826378,0.470560720604065,0.469663561445729,0.467992489536655,0.454029492570107,0.428650367716082,0.40512472162131,0.393582438634855,0.389439068788739,0.384767773448362,0.379374196969909,0.38079646670035,0.394092004671593,0.413050632004732,0.428916568775937,0.439173916265262,0.447546656956644,0.456126506677243,0.459390917895988,0.450343846596568,0.431880843911928,0.416789056124265,0.414162394520702,0.416601552990779,0.413172633257409,0.406482117247161,0.41267686709026],[0.0194701497541815,0.0345881359849426,0.0523610379614717,0.0863633201706759,0.133726780892375,0.175140552199302,0.195851100558978,0.202193824141059,0.205584833476341,0.21303526611469,0.227151286625036,0.248824606448451,0.272855206772115,0.292727259572112,0.306688773929156,0.317751596028588,0.328959663575778,0.342656786946715,0.360934139985263,0.384097988440287,0.410669689965659,0.438174077985257,0.460460307802116,0.470809040547772,0.470560720604065,0.469115575603143,0.472779236162325,0.474077243404792,0.462592335719025,0.439458303186664,0.417359987582511,0.406044181113312,0.401496599158257,0.396577111193187,0.391377322675403,0.393220575043668,0.406984823385701,0.426508784170891,0.443127157996766,0.454057902463434,0.462402232173585,0.469814678020682,0.471139860930387,0.459943837840332,0.439542676278332,0.423002454362375,0.419638658976955,0.422068758056881,0.419303690994989,0.413417849679406,0.41954907849355],[0.0210337796708257,0.0363282575780583,0.0538230640251236,0.0866778286903351,0.132535733834645,0.173523514301647,0.195717042964027,0.203687257405181,0.206553546822989,0.211703065462037,0.224790732613606,0.248855362704169,0.276772608953709,0.299164990782613,0.313271127157899,0.322945736593217,0.332178669150744,0.344349675078645,0.362547165055812,0.386666813598334,0.413411559059499,0.439084189309815,0.458942148751755,0.46829506710943,0.469663561445729,0.472779236162324,0.48360904363182,0.491807243355981,0.483588029512162,0.459842077823593,0.436136545559384,0.424554952905488,0.420693840142059,0.4163763933165,0.411438785581337,0.413730177194226,0.428485347922813,0.449087992599913,0.466245834218447,0.4772629818291,0.486039930540316,0.494647711593813,0.49669230318913,0.484169386589381,0.460711868512451,0.44158988113099,0.437839440090394,0.441536229630547,0.440592365552886,0.436179795156215,0.442730297236699],[0.0226918603753865,0.0382053253515058,0.0543773005126641,0.0849565483575114,0.128635652589831,0.169051018118542,0.19308095573795,0.203189865661295,0.205795268916695,0.20861645111564,0.220674462452998,0.247400421292272,0.279639324830909,0.304961507880513,0.319490267082992,0.327901860910561,0.33512445671338,0.345606385730735,0.36348332400828,0.388458935775705,0.415804873446741,0.440733742656996,0.458992746593247,0.466794883037271,0.467992489536655,0.474077243404792,0.491807243355981,0.507677692879297,0.503896944557904,0.480587427684073,0.455785648259172,0.44368092602074,0.439958344049395,0.436039663502974,0.4316896020926,0.435015834082681,0.451340200927786,0.473390263877326,0.491077480554345,0.501960036355071,0.511242844448911,0.521749517586555,0.525448601144149,0.512201578850094,0.485824153938282,0.464086249148102,0.459953954192522,0.464751205895275,0.46499924937509,0.461199799648683,0.467835041630759],[0.0221081111780568,0.0380840687391034,0.0518705141914604,0.0796664880504596,0.121538220159228,0.161419129688686,0.186158898282678,0.197077963680589,0.199235499638045,0.200474760734835,0.211423082374906,0.238677363741663,0.272228571166917,0.298318983747391,0.312455188069513,0.319408078209404,0.324411072215951,0.332936500117839,0.350264985949383,0.375894171990615,0.40353871876728,0.427329602525772,0.444068798352341,0.451513381679334,0.454029492570107,0.462592335719025,0.483588029512162,0.503896944557905,0.506227456261832,0.48949010008663,0.46879757155209,0.456414732531753,0.450397343347703,0.445509862382192,0.442519137322143,0.448194405159151,0.466632497873868,0.490196733611043,0.508662304073023,0.5197451973432,0.529102197886016,0.539854500427431,0.543599932673212,0.529687626975937,0.502006082931013,0.478983167376673,0.474042569654484,0.478062683511896,0.476929702418906,0.471344663130109,0.476823445684704],[0.0192972748291135,0.0356116588086776,0.0465888704428848,0.0718051109167692,0.112574159748106,0.151947374787411,0.176003247922498,0.186104880523813,0.187544239635493,0.188090698271463,0.197991035415441,0.223596057395594,0.255241826705762,0.279644005272815,0.29233381903075,0.29765805792069,0.300642835576825,0.30749841883916,0.324423770577444,0.350548981412123,0.37793526971434,0.399728810463848,0.414644974161663,0.422953449440548,0.428650367716082,0.439458303186664,0.459842077823594,0.480587427684073,0.48949010008663,0.484230845366375,0.472375416874993,0.460698163707857,0.451088686975665,0.444296767638183,0.442943624095666,0.451521912188858,0.47214524953392,0.497085633467737,0.516481498387758,0.527988647690225,0.53678025377018,0.54578138088254,0.547617292200202,0.532915275809014,0.505573049014237,0.48275875834055,0.476829521694732,0.478590441778143,0.474134533299673,0.465100874572452,0.468674607513855],[0.0166251175189173,0.0322177082620627,0.0412579332623144,0.0649003343733642,0.104893234765713,0.143578738926672,0.166502699904055,0.175400959541291,0.176135990325352,0.176390333969868,0.185666601415132,0.20975865541166,0.239308832172306,0.261547573354341,0.272353023428209,0.276251029677522,0.278414340680476,0.285099361878255,0.302500548182388,0.329156465360131,0.355949790034467,0.375438585067662,0.388213917260719,0.396915965782739,0.40512472162131,0.417359987582511,0.436136545559384,0.455785648259172,0.46879757155209,0.472375416874993,0.468435111231379,0.459422170015031,0.449080840343203,0.441476994009423,0.440606162951622,0.450393539252134,0.472361361980421,0.498432291172835,0.518457243656538,0.529834621887771,0.53751031395185,0.54443319605555,0.544251490695122,0.528812802680059,0.502029288859946,0.479877658124455,0.473702912379025,0.474225042628651,0.467709999290762,0.45657461194633,0.459386589339875],[0.0157786309103451,0.0291246356267592,0.0380833303825961,0.0616710344942862,0.1008406212425,0.138481517872149,0.160623618434228,0.169000153079858,0.169354995508846,0.169382638465077,0.178728193736514,0.203180237592353,0.232761966132105,0.253979104530278,0.263086482662052,0.266088035103276,0.269212797906219,0.277894716559139,0.297055887456202,0.324417648570495,0.350500579454025,0.367819908526032,0.378121765487047,0.38544867037553,0.393582438634855,0.406044181113312,0.424554952905488,0.44368092602074,0.456414732531752,0.460698163707857,0.459422170015031,0.455349888117668,0.449872476135279,0.444155568067038,0.442066619777388,0.450174908996043,0.47205200752875,0.498808781421859,0.518817411678937,0.529109381436008,0.535375475961557,0.541203508738676,0.540263440520645,0.524279252988556,0.497245062359597,0.475425119387053,0.470333511356097,0.472349735238402,0.467288600496787,0.457500998193171,0.461873900988569],[0.015320834914587,0.0264160928835393,0.0367667012492371,0.0611598429786833,0.0992267908756421,0.135442907580638,0.15714977087869,0.165789629595956,0.166439337958515,0.166692815903987,0.176661049081538,0.202363714525955,0.23301750425598,0.254119171117953,0.262224030835217,0.26494596200812,0.26966279248818,0.280884918805161,0.3021188462484,0.330269311677518,0.355639283522414,0.370753134432285,0.378168987573464,0.382926223188107,0.389439068788739,0.401496599158257,0.420693840142059,0.439958344049395,0.450397343347703,0.451088686975665,0.449080840343203,0.449872476135279,0.451376901968089,0.44988736596414,0.448051597036438,0.454871804888492,0.475976224283362,0.502392063128317,0.521892469247847,0.531363112983211,0.536802187192439,0.542042016419592,0.540666665428192,0.524246217019632,0.496956662604507,0.475579902202605,0.472015999154859,0.476445338154734,0.474326860754641,0.467414468485694,0.473757022857713],[0.013141952079488,0.0239139417559029,0.0364074277070802,0.0615355063179176,0.098035543560513,0.132469174994164,0.153889591090921,0.1634337203065,0.165382190838407,0.166839231539204,0.177732671916507,0.20396716548771,0.234863600825815,0.25607802348289,0.264331737959722,0.26741130770582,0.27282133348202,0.285014172433834,0.307373832604049,0.336262693460866,0.361021505897904,0.373607043309506,0.377661321888484,0.379686126428384,0.384767773448362,0.396577111193187,0.4163763933165,0.436039663502974,0.445509862382192,0.444296767638183,0.441476994009423,0.444155568067038,0.44988736596414,0.454087083196024,0.457858625031747,0.467995106514124,0.488749400593422,0.513223984062976,0.532025928598614,0.54248276607584,0.548642488569836,0.552915213701676,0.549990785589102,0.533117094808669,0.506768673133801,0.486646693534791,0.483833995147439,0.488987622430972,0.488309143092758,0.482972478937609,0.489123905867787],[0.00962070316816562,0.0219667756937601,0.0366508994378863,0.0620058163799043,0.0965617776673574,0.129046543615972,0.150268773980867,0.161065507288953,0.164904232232213,0.168089613361412,0.179763491086095,0.205412522534698,0.23540920982273,0.256769361369846,0.266211221915455,0.270155472518723,0.275151272423422,0.286574313382245,0.309134496410136,0.338975973627527,0.363677148260339,0.373997849489237,0.374913305773777,0.374831462072606,0.379374196969909,0.391377322675403,0.411438785581337,0.4316896020926,0.442519137322143,0.442943624095666,0.440606162951622,0.442066619777388,0.448051597036438,0.457858625031747,0.471457839062233,0.489491979422758,0.511752752149262,0.53451075146009,0.55327142593128,0.566188267317039,0.574062810549737,0.576978154350557,0.57168312334756,0.554459777554331,0.530026666886978,0.511538831164126,0.508179224538887,0.51177524286709,0.510371975484521,0.50481592197545,0.508754992172626],[0.00762430715900963,0.0214499198734321,0.0376885139603343,0.0628176104091112,0.0954114900412902,0.12614119351053,0.147330275514936,0.159286823481326,0.164453645143262,0.168461052099373,0.180126859813348,0.204885570505838,0.234084084927079,0.255910333902806,0.266899648450175,0.271934759777765,0.276521303328098,0.287204809707052,0.310343729395571,0.342014272811809,0.368010410311134,0.377789201111481,0.377248275544171,0.376230891865352,0.38079646670035,0.393220575043668,0.413730177194226,0.435015834082681,0.448194405159151,0.451521912188858,0.450393539252133,0.450174908996043,0.454871804888492,0.467995106514124,0.489491979422758,0.515745875129392,0.543133321982381,0.568622932017401,0.58939272141059,0.604032520366637,0.612543973118782,0.614574451294114,0.608108703220587,0.591006647618568,0.567817206203912,0.549778303687962,0.54475151836718,0.545929738009707,0.54330405084826,0.537569126338575,0.540123215159413],[0.0091589006740241,0.0229121181584461,0.0396474612639257,0.0643008086961966,0.095367918350293,0.124904427310549,0.146229194592453,0.158833987946666,0.163936442039167,0.166954905361007,0.177508440228303,0.201735186368487,0.231119034228132,0.253830406873757,0.266091910995561,0.272178336907421,0.277252984769834,0.288632814369124,0.313679062288349,0.348387788586863,0.377389876800323,0.389187998900115,0.389748533039457,0.389241041266658,0.394092004671593,0.406984823385701,0.428485347922813,0.451340200927786,0.466632497873868,0.47214524953392,0.472361361980421,0.47205200752875,0.475976224283362,0.488749400593422,0.511752752149262,0.543133321982381,0.579891489816235,0.615003446952348,0.640693227552964,0.655024335003,0.661899741910047,0.66410144349512,0.658911965568111,0.642567491498452,0.618860277638933,0.599138653325696,0.5916170129157,0.590659911199867,0.587575328027475,0.58300659709459,0.586932405944075],[0.0123535086113619,0.0255041836435166,0.0423031433367703,0.0669369601728294,0.097695246427747,0.126970654171923,0.148287939051957,0.160725829315896,0.164963430649948,0.166553187359872,0.175920882370973,0.199829771641696,0.229545466111812,0.252820756332902,0.265725022785431,0.27264423587875,0.278887013123408,0.292006196968851,0.319603670188119,0.357504949598187,0.389759186059976,0.404339817600011,0.406965494551783,0.407645941943052,0.413050632004732,0.426508784170891,0.449087992599913,0.473390263877326,0.490196733611043,0.497085633467737,0.498432291172835,0.498808781421859,0.502392063128317,0.513223984062976,0.53451075146009,0.568622932017401,0.615003446952348,0.661529964350107,0.693685379505064,0.708373019034013,0.713916640876153,0.716660320288563,0.713043239830729,0.697407018116191,0.672802436012822,0.650989436642837,0.640775220956778,0.637641384304702,0.634093496532438,0.630810773091416,0.636646075038659],[0.0144475321088493,0.0280467134870814,0.0453253497513456,0.0711766273093092,0.103646726979766,0.133934823992636,0.154740975922443,0.165959553302808,0.169454935750345,0.171034412275775,0.180448181283924,0.203924162419897,0.23291543691702,0.25554153363934,0.268205597825174,0.275594391087112,0.283196363791438,0.298266092384556,0.327992114572648,0.367888852076066,0.401796610629334,0.417561860121391,0.421086008099859,0.422630307690389,0.428916568775937,0.443127157996766,0.466245834218447,0.491077480554345,0.508662304073023,0.516481498387758,0.518457243656538,0.518817411678937,0.521892469247847,0.532025928598614,0.55327142593128,0.58939272141059,0.640693227552964,0.693685379505064,0.731958672093633,0.75117309266341,0.759059121077132,0.761790173772067,0.757212197393526,0.741394265942365,0.717420333900737,0.695389236232093,0.68298265091228,0.676993648752599,0.671773520528878,0.668122107397169,0.673287800893748],[0.01451241822894,0.029803440233887,0.0479775655285843,0.0758858192786237,0.111458756334261,0.143706461230417,0.163838107212786,0.173410608511967,0.176647601384842,0.179607818029667,0.19018508658607,0.213173923859123,0.240652454195423,0.261859488272072,0.273848879004003,0.281489521543979,0.290286387461154,0.306929616209746,0.337913388479374,0.378410118034794,0.412270451715893,0.427471401556844,0.430464443312404,0.432136955591859,0.439173916265262,0.454057902463434,0.4772629818291,0.501960036355071,0.5197451973432,0.527988647690225,0.529834621887771,0.529109381436008,0.531363112983211,0.54248276607584,0.566188267317039,0.604032520366637,0.655024335003,0.708373019034013,0.751173092663411,0.778226097529673,0.792040596706031,0.795054719943551,0.788084512150767,0.771682049115599,0.749655104696312,0.72906715082152,0.715289796331835,0.7063609893558,0.698680729923973,0.693144760907573,0.695144568018487],[0.0134516599587157,0.0304812318149881,0.0491169711378746,0.0783459919564488,0.11635087444094,0.150509664902736,0.170844469911743,0.179832686438706,0.183094390867664,0.186913366283649,0.198237102472678,0.221135350177003,0.248051476711448,0.268847967829996,0.280878738094951,0.288989130876933,0.298594810104402,0.316089864753118,0.347630916505408,0.388286895272678,0.422039039195408,0.436985789181323,0.439622691428143,0.440866806017986,0.447546656956644,0.462402232173585,0.486039930540316,0.511242844448911,0.529102197886016,0.53678025377018,0.53751031395185,0.535375475961557,0.536802187192439,0.548642488569836,0.574062810549737,0.612543973118782,0.661899741910047,0.713916640876153,0.759059121077132,0.792040596706031,0.811823445088911,0.818042298701227,0.812281860505905,0.796706957442559,0.77561236261166,0.755422795274529,0.741041135892844,0.730912484310399,0.722030820366355,0.715146602345967,0.714766766980591],[0.012184664428522,0.0298735894418936,0.0478109208618154,0.0764513991188953,0.114697704490176,0.149900457099137,0.171861305927171,0.182195069327086,0.185548505510878,0.188377265566261,0.198945893456674,0.222354612483207,0.25073177323223,0.273294804653498,0.286898206164275,0.296033547649337,0.305911837133245,0.323293649031453,0.354837546540918,0.39594582808551,0.430764527316019,0.447303940848557,0.450989028425408,0.451495296269285,0.456126506677243,0.469814678020682,0.494647711593813,0.521749517586555,0.539854500427431,0.545781380882539,0.54443319605555,0.541203508738676,0.542042016419592,0.552915213701676,0.576978154350557,0.614574451294114,0.66410144349512,0.716660320288563,0.761790173772067,0.795054719943551,0.818042298701227,0.831782934255224,0.834025627384994,0.821986847722224,0.799500733226275,0.777019090184183,0.7626638177211,0.754086287475037,0.746539338907004,0.740315104458059,0.740745009549303],[0.0113758786592171,0.0280434804242242,0.0440381258969864,0.0708230606303023,0.108018873309467,0.143403495752272,0.166853349020735,0.178698403652472,0.182030825701433,0.183365368612479,0.192639373221394,0.216311175705738,0.246327861020472,0.271331579071523,0.28739066816697,0.297999655017049,0.307727613410256,0.324150294421663,0.35508391850748,0.396587107492231,0.432960884568757,0.452003218086965,0.457544656787058,0.457420609055407,0.459390917895988,0.471139860930387,0.49669230318913,0.525448601144149,0.543599932673212,0.547617292200202,0.544251490695122,0.540263440520645,0.540666665428192,0.549990785589103,0.57168312334756,0.608108703220587,0.65891196556811,0.713043239830729,0.757212197393526,0.788084512150767,0.812281860505905,0.834025627384994,0.846204553107233,0.839402248954447,0.816347064838136,0.79191208929948,0.777821061626795,0.771088195639605,0.765329870251256,0.760592032978596,0.763350199941972],[0.0115914994526642,0.0251388192376269,0.0380544047895784,0.0627955601389712,0.099107724487911,0.134024472210158,0.1568316981457,0.168035338100033,0.171074994035733,0.172349353976725,0.181216192008275,0.203857529124282,0.233237750290338,0.259269362595121,0.27767308912116,0.289977868944833,0.299305263018981,0.314137195281635,0.3437550001667,0.385004366348379,0.422295336741163,0.443239184389206,0.450414823856973,0.450202304850737,0.450343846596568,0.459943837840332,0.484169386589381,0.512201578850094,0.529687626975937,0.532915275809014,0.528812802680059,0.524279252988556,0.524246217019632,0.533117094808669,0.554459777554332,0.591006647618568,0.642567491498452,0.697407018116191,0.741394265942365,0.771682049115599,0.79670695744256,0.821986847722224,0.839402248954446,0.837968384106799,0.81921307224918,0.79717672855382,0.783171392528298,0.775419497351596,0.769108763063164,0.764893743882469,0.768895828209003],[0.0126039892034018,0.021711875112217,0.0312487428714465,0.0544365433170178,0.0904693162179292,0.124513190276249,0.144658773502766,0.153134227850431,0.155719029247019,0.158581939873738,0.168281008165233,0.189059282651913,0.215832110290119,0.241320972989505,0.261422740636704,0.275209671095333,0.283911780284874,0.296837874960971,0.324742995420216,0.365192506093068,0.402562095543856,0.424289997295981,0.432358481381503,0.432455846616029,0.431880843911928,0.439542676278332,0.460711868512451,0.485824153938282,0.502006082931013,0.505573049014237,0.502029288859946,0.497245062359597,0.496956662604507,0.5067686731338,0.530026666886978,0.567817206203912,0.618860277638933,0.672802436012822,0.717420333900737,0.749655104696312,0.77561236261166,0.799500733226275,0.816347064838136,0.81921307224918,0.808757619221069,0.793236718475193,0.77965794064671,0.768713844728515,0.759639720544742,0.754464983981018,0.758013176984464],[0.0133920758986437,0.0187192720325778,0.0261432922601097,0.0485467178603732,0.084320421295265,0.117363566209445,0.135048316540061,0.141159134579828,0.143505326205363,0.148096884178574,0.159144488146665,0.179195166835087,0.204454269302239,0.22960097427405,0.250677200915191,0.265087608829077,0.2728180297753,0.283941985800991,0.310447344309692,0.350347696748992,0.387682358610529,0.409556408396027,0.417768285509471,0.417852599798406,0.416789056124265,0.423002454362375,0.44158988113099,0.464086249148102,0.478983167376673,0.48275875834055,0.479877658124455,0.475425119387052,0.475579902202605,0.486646693534791,0.511538831164126,0.549778303687962,0.599138653325696,0.650989436642838,0.695389236232093,0.72906715082152,0.755422795274529,0.777019090184183,0.79191208929948,0.79717672855382,0.793236718475193,0.783864544186173,0.772508441505872,0.760737253457883,0.749743800435967,0.742878890922022,0.745651321486086],[0.0132433983079006,0.0169049313606271,0.0245335471295971,0.0470326094185766,0.0821118946249729,0.114287591739835,0.131476061460251,0.137488888247712,0.140040182937765,0.14520462632401,0.157417469158103,0.179292120516932,0.206439741046498,0.232798630393512,0.254094213698673,0.267839041596672,0.274256187773986,0.2841000136905,0.3101171483734,0.350321470200625,0.387994106900862,0.409624129776975,0.417094601932727,0.416199279096981,0.414162394520702,0.419638658976955,0.437839440090394,0.459953954192522,0.474042569654484,0.476829521694731,0.473702912379025,0.470333511356097,0.472015999154859,0.483833995147439,0.508179224538888,0.54475151836718,0.5916170129157,0.640775220956778,0.68298265091228,0.715289796331835,0.741041135892844,0.7626638177211,0.777821061626796,0.783171392528298,0.77965794064671,0.772508441505872,0.765873337377905,0.758848748779751,0.749517114130461,0.741708533689831,0.745000769558399],[0.0128192790963617,0.0160916414016255,0.0251112973955133,0.0480965264079801,0.0822810561400769,0.113914062663951,0.132143744571539,0.139624747338963,0.142402140335997,0.146838672902345,0.159614774347633,0.184811112138843,0.216054057681938,0.244527788254317,0.26535821852726,0.277613000565924,0.282911699996743,0.292396017144489,0.318920635807337,0.359986585156321,0.39774123973533,0.41788435133975,0.42299538157567,0.419921230569029,0.416601552990779,0.422068758056881,0.441536229630547,0.464751205895275,0.478062683511897,0.478590441778143,0.474225042628651,0.472349735238402,0.476445338154734,0.488987622430972,0.51177524286709,0.545929738009707,0.590659911199867,0.637641384304702,0.676993648752599,0.7063609893558,0.730912484310399,0.754086287475036,0.771088195639605,0.775419497351596,0.768713844728515,0.760737253457883,0.758848748779751,0.75936507004516,0.756232969455926,0.752382638643508,0.759727708669011],[0.0130612176454187,0.0158522428499515,0.0258242178246622,0.0490696898156232,0.0825592190496153,0.114141378734128,0.133992284753065,0.143184656221075,0.145642027044141,0.148201202752072,0.1606122604985,0.189007860808978,0.224544341762145,0.254931804696063,0.274730759985274,0.285352127040801,0.290363130038896,0.300764836732139,0.328733235184467,0.370722880057251,0.407466892609289,0.423866209365925,0.424227190249541,0.417696722356713,0.413172633257409,0.419303690994989,0.440592365552886,0.46499924937509,0.476929702418906,0.474134533299673,0.467709999290762,0.467288600496787,0.474326860754641,0.488309143092758,0.510371975484521,0.54330405084826,0.587575328027474,0.634093496532438,0.671773520528878,0.698680729923973,0.722030820366355,0.746539338907004,0.765329870251256,0.769108763063164,0.759639720544742,0.749743800435966,0.749517114130462,0.756232969455926,0.764217043125277,0.773730616715519,0.790970576667741],[0.0134560954089633,0.01530729642547,0.025351634247369,0.0485475588451775,0.0817673680001241,0.113797207304149,0.135289287908722,0.145766370540687,0.147207893536936,0.147091220206974,0.158391638832914,0.189421680781059,0.228886688889647,0.261016412991644,0.279843297835096,0.289202483083125,0.294624453490113,0.306716257668493,0.336661005466127,0.379700972961929,0.415137602465859,0.427140109465569,0.422121121075806,0.411964100245248,0.406482117247161,0.413417849679406,0.436179795156215,0.461199799648683,0.471344663130109,0.465100874572452,0.45657461194633,0.457500998193171,0.467414468485694,0.482972478937609,0.50481592197545,0.537569126338575,0.58300659709459,0.630810773091416,0.668122107397169,0.693144760907573,0.715146602345967,0.740315104458059,0.760592032978596,0.764893743882469,0.754464983981018,0.742878890922022,0.741708533689831,0.752382638643508,0.773730616715519,0.803127594121945,0.836494412362482],[0.0120361759133872,0.0131250828735328,0.0231045235574797,0.0463898309169458,0.0798161590756366,0.112638483329245,0.135629046967152,0.146947457223577,0.146945012119781,0.143904555096121,0.154040475012052,0.187875481454371,0.231786167715914,0.266649722847255,0.285697207017158,0.294513551512573,0.300145057934403,0.313335009814453,0.345040636002025,0.389882261546216,0.426143307695915,0.437317726577243,0.430582794486151,0.418921787662896,0.41267686709026,0.41954907849355,0.442730297236699,0.467835041630759,0.476823445684704,0.468674607513855,0.459386589339875,0.461873900988569,0.473757022857713,0.489123905867787,0.508754992172626,0.540123215159413,0.586932405944075,0.636646075038659,0.673287800893748,0.695144568018487,0.714766766980591,0.740745009549303,0.763350199941972,0.768895828209003,0.758013176984464,0.745651321486086,0.745000769558399,0.759727708669011,0.790970576667741,0.836494412362482,0.890690855519201]],"type":"surface","contours":{"z":{"show":true,"usecolormap":true,"highlightcolor":"#ff0000","project":{"z":true}}},"frame":null}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.2,"selected":{"opacity":1},"debounce":0},"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>
This covariance surface for the Brownian motion random walk we are using for data is interesting in its own right. There is a theoretical result that says that the estimate for the covariance function for this process is <span class="math inline">\(\hat{c}(t,s) = min(t,s)\)</span>. You can find the proof in the book by Kokoszka and Reimherr referenced below. Using the mouse to hover over some points in the graph makes this result seem plausible.</p>
<div id="references" class="section level4">
<h4>References</h4>
<div id="books" class="section level5">
<h5>Books</h5>
<ul>
<li>Kokoszka, P. and Reimherr, M. (2017). <em>Introduction to Functional Data Analysis</em>. CRC.</li>
<li>Ramsay, J.O. and Silverman, B.W. (2005). <em>Functional Data Analysis</em>. Springer.</li>
<li>Ramsay, J.0., Hooker, G. and Graves, S. (2009) <em>Functional Data Analysis with R and MATLAB</em> Springer.</li>
</ul>
</div>
<div id="online-resources" class="section level5">
<h5>Online Resources</h5>
<ul>
<li>Cao, J. (2019). <a href="https://www.youtube.com/watch?v=SUp_Nq8NwfE"><em>Functional Data Analysis Course</em></a></li>
<li>Staicu, A. and Park, Y. (2016) <a href="https://www4.stat.ncsu.edu/~staicu/FDAtutorial/"><em>Short Course on Applied Functional Data Analysis</em></a></li>
</ul>
</div>
<div id="recommended-papers" class="section level5">
<h5>Recommended Papers</h5>
<ul>
<li>Sørensen, H. Goldsmith, J. and Sangalli, L. (2013). <a href="https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.5989"><em>An introduction with medical applications fo functional data analysis</em></a> Wiley</li>
<li>Wang, J., Chiou, J. and Müller, H. (2015). <a href="https://arxiv.org/pdf/1507.05135.pdf"><em>Review of Functional Data Analysis</em></a></li>
<li>Yao, F., Müller, H, Wang, J. (2012). <a href="https://anson.ucdavis.edu/~mueller/jasa03-190final.pdf"><em>Functional Data Analysis for Sparse Longitudinal Data</em></a> JASA J100, I 470</li>
</ul>
</div>
</div>
<script>window.location.href='https://rviews.rstudio.com/2021/05/14/basic-fda-descriptive-statistics-with-r/';</script>
Introduction to Functional Data Analysis with R
https://rviews.rstudio.com/2021/05/04/functional-data-analysis-in-r/
Tue, 04 May 2021 00:00:00 +0000https://rviews.rstudio.com/2021/05/04/functional-data-analysis-in-r/
<script src="/2021/05/04/functional-data-analysis-in-r/index_files/header-attrs/header-attrs.js"></script>
<p>Suppose you have data that looks something like this.</p>
<p><img src="/2021/05/04/functional-data-analysis-in-r/index_files/figure-html/unnamed-chunk-1-1.png" width="672" />
This plot might depict 80 measurements for a participant in a clinical trial where each data point represents the change in the level of some protein level. Or it could represent any series of longitudinal data where the measurements are taken at irregular intervals. The curve looks like a time series with obvious correlations among the points, but there are not enough measurements to model the data with the usual time series methods. In a scenario like this, you might find <a href="https://en.wikipedia.org/wiki/Functional_data_analysis">Functional Data Analysis</a> (FDA) to be a viable alternative to the usual multi-level, mixed model approach.</p>
<p>This post is meant to be a “gentle” introduction to doing FDA with R for someone who is totally new to the subject. I’ll show some “first steps” code, but most of the post will be about providing background and motivation for looking into FDA. I will also point out some of the available resources that a newcommer to FDA should find helpful.</p>
<p>FDA is a branch of statistics that deals with data that can be conceptualized as a function of an underlying, continuous variable. The data in FDA are smooth curves (or surfaces) in time or space. To fix a mental model of this idea, first consider an ordinary time series. For example, you might think of the daily closing prices of your favorite stock. The data that make up a time series are the individual points which are considered to be random draws from an underlying stochastic process.</p>
<p>Now, go up a level of abstraction, and consider a space where the whole time series, or rather an imaginary continuous curve that runs through all of your data points is the basic item of analysis. In this conceptual model, the curve comprises an infinite number of points, not just the few you observed. Moreover, unlike in basic time series analysis, the observed points do not need to be equally spaced, and the various curves that make up your data set do not need to be sampled at the same time points.</p>
<p>Mathematically, the curves are modeled as functions that live in an infinite dimensional vector space, what the mathematicians call a <a href="https://iopscience.iop.org/article/10.1088/1742-6596/839/1/012002/pdf">Hilbert Space</a>. One way to think of this is that you are dealing with the ultimate large p small n problem. Each curve has infinitely many points, not just the 3 or 30 or 3,000 you happen to have.</p>
<p>The theory of Hilbert Spaces is part of the area of mathematical analysis called <a href="https://en.wikipedia.org/wiki/Functional_analysis">Functional Analysis</a>, a subject usually introduced as part of a second or third course in mathematical analysis, or perhaps in a course on <a href="https://quantum.phys.cmu.edu/QCQI/qitd114.pdf">Quantum Mechanics</a>.</p>
<p>It would be a heavy lift to expect someone new to Functional Data Analysis to start with the mathematics. Fortunately, this is not really necessary. The practical applications of FDA and the necessary supporting software have been sufficiently developed so that anyone familiar with the basics of ordinary vector spaces should have sufficient background to get started. The salient points to remember are:</p>
<ul>
<li>Hilbert space is an infinite dimensional linear vector space</li>
<li>The vectors in Hilbert space are functions</li>
<li>The inner product of two functions in the Hilbert space is defined as the integral of two functions, but it behaves very much like the familiar dot product.</li>
</ul>
<p>Moreover, for the last twenty years or so mathematical statisticians have been writing R packages to put FDA within the reach of anyone with motivation and minimal R skills. The CRAN Task View on <a href="https://cran.r-project.org/view=FunctionalData">Functional Data Analysis</a> categorizes and provides brief explanations for forty packages that collectively cover most of the established work on FDA. The following graph built with functions from the <a href="https://cran.r-project.org/package=cranly"><code>cranly</code></a> package shows part of the network for two core FDA packages.</p>
<p><img src="fda.png" height = "600" width="100%"></p>
<p>The <a href="https://cran.r-project.org/package=fda"><code>fda</code></a> package emphasized in the network plot above is the logical place for an R user to begin investigating FDA. With thirty-two reverse depends, thirty-eight reverse imports and thirteen reverse suggest, fda is at the root of Functional Data Analysis software for R. Moreover, in a very real sense, it is at the root of modern FDA itself. <code>fda</code> was written to explicate the theory developed in the 2005 book by Ramsay and Silverman<span class="math inline">\(^{1}\)</span>. Kokoszka and Reimnerr state that the first edition of this book published in 1997: “is largely credited with solidifying FDA as an official subbranch of statistics” (p xiv)<span class="math inline">\(^{2}\)</span>. The <a href="https://cran.r-project.org/package=refund"><code>refund</code></a> package is used extensively throughout the book by Kokoszka and Reimnerr.</p>
<div id="first-steps" class="section level3">
<h3>First Steps</h3>
<p>The synthetic data in the figure above were generated by a Wiener, Brownian Motion process, which for the purposes of this post, is just a convenient way to generate a variety of reasonable looking curves. We suppose that the data points shown represent noisy observations generated by a smooth curve f(t). We estimate this curve with the model: <span class="math inline">\(y_{i} = f(t_{i}) + \epsilon_{i}\)</span> where the <span class="math inline">\(\epsilon_{i}\)</span> are normally distributed with mean 0 and variance <span class="math inline">\(\sigma^{2}\)</span>.</p>
<p>Notice that the measurement times are randomly selected within the 100 day window and not uniformly spaced.</p>
<pre class="r"><code>set.seed(999)
n_obs <- 80
time_span <- 100
time <- sort(runif(n_obs,0,time_span))
Wiener <- cumsum(rnorm(n_obs)) / sqrt(n_obs)
y_obs <- Wiener + rnorm(n_obs,0,.05)</code></pre>
<p>Remember that the task ahead is to represent the entire curve of infinitely many points and not just the handful of observed values. Here is where the linear algebra comes in. The curve is treated as a vector in an infinite dimensional vector space, and what we want is something that will serve as a basis for this curve projected down into the subspace where the measurements live. The standard way to do this for non-periodic data is to construct a <a href="https://en.wikipedia.org/wiki/B-spline">B-spline</a> basis. (B-splines or basis splines are splines designed to have properties that make them suitable for representing vectors.) The code that follows is mostly <em>borrowed</em> from Jiguo Cao’s Youtube Video Course<span class="math inline">\(^{3}\)</span> which I very highly recommend for anyone just starting with FDA. In his first five videos, Cao explains B-splines and the placement of knots in great detail and derives the formula used in the code to calculates the number of basis elements from the number of knots and the order of the splines.</p>
<p>Note that we are placing the knots at times equally spaced over the 100 day time span.</p>
<pre class="r"><code>times_basis = seq(0,time_span,1)
knots = c(seq(0,time_span,5)) #Location of knots
n_knots = length(knots) #Number of knots
n_order = 4 # order of basis functions: cubic bspline: order = 3 + 1
n_basis = length(knots) + n_order - 2;
basis = create.bspline.basis(c(min(times_basis),max(times_basis)),n_basis,n_order,knots)
n_basis</code></pre>
<pre><code>## [1] 23</code></pre>
<p>and there are 23 basis vectors.</p>
<p>Next, we use the function <code>eval.basis()</code> to evaluate the basis functions at the times where our data curve was observed The matrix <code>PHI</code> contains the values of the 23 basis functions <span class="math inline">\(\phi_j(t)\)</span> evaluated at 80 points.</p>
<pre class="r"><code>PHI = eval.basis(time, basis)
dim(PHI)</code></pre>
<pre><code>## [1] 80 23</code></pre>
<p>We plot the basis functions and locations of the knots.</p>
<pre class="r"><code>matplot(time,PHI,type='l',lwd=1,lty=1, xlab='time',ylab='basis',cex.lab=1,cex.axis=1)
for (i in 1:n_knots)
{
abline(v=knots[i], lty=2, lwd=1)
}</code></pre>
<p><img src="/2021/05/04/functional-data-analysis-in-r/index_files/figure-html/unnamed-chunk-5-1.png" width="672" /></p>
<p>The plot shows that for interior points, four basis functions contribute to computing the value of any point. The endpoints, however, are computed from a single basis function.</p>
</div>
<div id="estimating-the-basis-coefficients" class="section level3">
<h3>Estimating the Basis Coefficients</h3>
<p>As in ordinary regression, we express the function in terms of the coefficients <span class="math inline">\(c_j\)</span> and basis functions <span class="math inline">\(\phi_j\)</span> using the formula: <span class="math inline">\(f(t) = \sum c_j \phi_j(t)\)</span>. Later we will see how to use built-in <code>fda</code> functions to estimate the coefficients, but now we follow Cao’s lead and calculate everything from first principles.</p>
<p>The following code uses matrix least squares equation <span class="math inline">\(\hat{c} = (\Phi^t\Phi)^{-1} \Phi^{t}y\)</span> to estimate the coefficients.</p>
<pre class="r"><code># Least squares estimate
# estimate basis coefficient
M = ginv(t(PHI) %*% PHI) %*% t(PHI)
c_hat = M %*% Wiener</code></pre>
<p>We compute <span class="math inline">\(\hat{y}\)</span>, the estimates of our observed values, and plot.</p>
<pre class="r"><code>y_hat = PHI %*% c_hat
# Augment data frame for plotting
df <- df %>% mutate(y_hat = y_hat)
p2 <- df %>% ggplot() +
geom_line(aes(x = time, y = Wiener), col = "grey") +
geom_point(aes(x = time, y = y_obs)) +
geom_line(aes(x = time, y = y_hat), col = "red")
p2 + ggtitle("Original curve and least squares estimate") +
xlab("time") + ylab("f(time)")</code></pre>
<p><img src="/2021/05/04/functional-data-analysis-in-r/index_files/figure-html/unnamed-chunk-7-1.png" width="672" />
The gray curve in the plot represents the underlying Brownian motion process, the dots are the observed values (the same as in the first plot), and the red curve represents the least squares “smoothed” estimates.</p>
<p>Now, we work through the matrix calculations to estimate the variance of the noise and the error bars for <span class="math inline">\(\hat{y}\)</span>.</p>
<pre class="r"><code># estimate the variance of noise
## SSE = (Y - Xb)'(Y - Xb)
SSE = t(y_hat-y_obs)%*%(y_hat-y_obs)
sigma2 = SSE/(n_obs-n_basis)
# estimate the variance of the fitted curve
# H is the Hat matrix H
# H = X*inv(X'X)*X``
H = PHI %*% M
varYhat = diag(H %*% H * matrix(sigma2,n_obs,n_obs))
# 95% confidence interval
y_hat025 = y_hat-1.96*sqrt(varYhat)
y_hat975 = y_hat+1.96*sqrt(varYhat)</code></pre>
<p>And, we plot. We have a satisfying smoothed representation of our original curve that looks like it would be and adequate starting point for further study. Note that process of using regression to produce a curve from the basis functions is often referred to as “regression smoothing”</p>
<pre class="r"><code>df <- mutate(df, y_hat025 = y_hat025,
y_hat975 = y_hat975)
#names(df) <- c("time","Wiener","y_hat", "y_hat025", "y_hat975")
p3 <- df %>% ggplot() +
geom_line(aes(x = time, y = Wiener), col = "grey") +
geom_point(aes(x = time, y = y_obs)) +
geom_line(aes(x = time, y = y_hat), col = "red") +
geom_line(aes(x = time, y_hat025), col = "green") +
geom_line(aes(x = time, y_hat975), col = "green")
p3 + ggtitle("Estimated curve with error bars") +
xlab("time") + ylab("f(time)")</code></pre>
<p><img src="/2021/05/04/functional-data-analysis-in-r/index_files/figure-html/unnamed-chunk-9-1.png" width="672" /></p>
<p>We finish for today, by showing how to do the hard work of estimating coefficients and function values with a single line of code using the <code>fda</code> function <code>smooth.basis()</code>. The function takes the arguments <code>argvals</code> the times we want to use for evaluation as a vector (or matrix or array), <code>y</code> the observed values, and <code>fdParobj</code>, an <code>fda</code> object containing the basis elements.</p>
<pre class="r"><code>Wiener_obj <- smooth.basis(argvals = time, y = y_obs, fdParobj = basis)</code></pre>
<p>Here we plot our “hand calculated” curve in red and show the <code>smooth.basis()</code> curve in blue. They are reasonably close, except at the end points, where there is not much data to construct the basis.</p>
<pre class="r"><code>plot(time, Wiener, type = "l", xlab = "time", ylab = "f(time)",
main = "Comparison of fda package and naive smoothing estimates", col = "grey")
lines(time,y_hat,type = "l",col="red")
lines(Wiener_obj, lwd = 1, col = "blue")</code></pre>
<p><img src="/2021/05/04/functional-data-analysis-in-r/index_files/figure-html/unnamed-chunk-11-1.png" width="672" /></p>
<p>Note that we have shown the simplest use of <code>smooth.basis()</code> which is capable of computing penalized regression estimates and more. The <a href="https://www.rdocumentation.org/packages/fda/versions/5.1.9/topics/smooth.basis">examples</a> of using the <code>smooth.basis()</code> function in the <code>fda</code> pdf are extensive and worth multiple blog posts. In general, the pdf level documentation for <code>fda</code> is superb. However, the package lacks vignettes. For a price, the book <em>Functional Data Analysis with R and Matlab</em><span class="math inline">\(^{4}\)</span> supplies the equivalent of several the missing vignettes.</p>
</div>
<div id="next-steps" class="section level3">
<h3>Next Steps</h3>
<p>Once you have a basis representation, what’s next? You may be interested in the following:</p>
<ul>
<li>More exploratory work such as <a href="https://en.wikipedia.org/wiki/Functional_principal_component_analysis">Functional Principal Components Analysis</a>, the analog of principal components analysis.</li>
<li>Clustering curves. See the <a href="https://cran.r-project.org/package=funHDDC">funHDDC</a> package.</li>
<li>Setting up regression models where either the dependent variable, or some of the independent variables, or both are functional objects. See the <a href="https://CRAN.R-project.org/package=refund">refund</a> package and the book by Kokoszka and Reimnerr<span class="math inline">\(^{2}\)</span></li>
<li>Studying the shape of the curves themselves. For example, the shape of a protein concentration curve may convey some clinical meaning. FDA permits studying the velocity and acceleration of curves, offering the possibility of obtaining more information than the standard practice of looking at the area under the curves. You can explore this with the <code>fda</code> package (But be sure to check that the order of your basis functions is adequate to compute derivatives.). See the book by Ramsay, Hooker and Graves<span class="math inline">\(^{4}\)</span>.</li>
<li>Learning what to do when you have sparse data. See the paper by Yao et al. below<span class="math inline">\(^{5}\)</span> and look into the <a href="https://cran.r-project.org/package=fdapace">fdapace</a> package.</li>
<li>Working with two and three dimensional medical images<span class="math inline">\(^{6}\)</span>.</li>
</ul>
<p>I would like to make Functional Data Analysis a regular feature on R Views. If you are working with FDA and would like to post, please let me (<a href="mailto:joseph.rickert@rstudio.com" class="email">joseph.rickert@rstudio.com</a>) know.</p>
</div>
<div id="references" class="section level3">
<h3>References</h3>
<div id="books" class="section level4">
<h4>Books</h4>
<ul>
<li><span class="math inline">\(^{2}\)</span>Kokoszka, P. and Reimherr, M. (2017). <em>Introduction to Functional Data Analysis</em>. CRC.</li>
<li><span class="math inline">\(^{1}\)</span>Ramsay, J.O. and Silverman, B.W. (2005). <em>Functional Data Analysis</em>. Springer.</li>
<li><span class="math inline">\(^{4}\)</span>Ramsay, J.0., Hooker, G. and Graves, S. (2009) <em>Functional Data Analysis with R and MATLAB</em> Springer.</li>
</ul>
</div>
<div id="online-resources" class="section level4">
<h4>Online Resources</h4>
<ul>
<li><span class="math inline">\(^{3}\)</span>Cao, J. (2019). <a href="https://www.youtube.com/watch?v=SUp_Nq8NwfE"><em>Functional Data Analysis Course</em></a></li>
<li>Staicu, A. and Park, Y. (2016) <a href="https://www4.stat.ncsu.edu/~staicu/FDAtutorial/"><em>Short Course on Applied Functional Data Analysis</em></a></li>
</ul>
</div>
<div id="recommended-papers" class="section level4">
<h4>Recommended Papers</h4>
<ul>
<li><span class="math inline">\(^{6}\)</span>Sørensen, H. Goldsmith, J. and Sangalli, L. (2013). <a href="https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.5989"><em>An introduction with medical applications fo functional data analysis</em></a> Wiley</li>
<li>Wang, J., Chiou, J. and Müller, H. (2015). <a href="https://arxiv.org/pdf/1507.05135.pdf"><em>Review of Functional Data Analysis</em></a></li>
<li><span class="math inline">\(^{5}\)</span> Yao, F., Müller, H, Wang, J. (2012). <a href="https://anson.ucdavis.edu/~mueller/jasa03-190final.pdf"><em>Functional Data Analysis for Sparse Longitudinal Data</em></a> JASA J100, I 470</li>
</ul>
</div>
</div>
<script>window.location.href='https://rviews.rstudio.com/2021/05/04/functional-data-analysis-in-r/';</script>