---
title: "Integrating genomic mate selection into the NextGen Cassava pipeline"
site: workflowr::wflow_site
output:
  workflowr::wflow_html:
    toc: false
editor_options:
  chunk_output_type: console
---

We saw in [an initial study](https://www.biorxiv.org/content/10.1101/2021.01.05.425443v1), promising results regarding the prediction of genetic variance in cassava crosses. In that study, we used a high-quality validated pedigree-based phasing pipeline for the marker data. That pipeline is considerably more involved and may not be implementable on the entire breeding germplasm.

In this project, I am integrating cross-variance predictions into our current genomics-enabled breeding pipeline leveraging available data. I conduct additional tests to assess whether current Beagle imputed-and-phased data and a basic pedigree-validation step are sufficient.

This project forms the template for genomic mate selection in NextGen Cassava.

[**See the Results here!**](07-Results.html)

We will soon implement in [summer 2021 for planning fall crossing nurseries and with new GS C5 DArTseqLD data](https://wolfemd.github.io/IITA_2021GS/).

# What's new?

1.  Functions previously used in NextGen GS pipeline (`code/gsFunctions.R`) migrate to and receive an upgrade (`code/gmsFunctions.R`) to support genomic mate selection. The `runGenomicPredictions()` function fits genomic mixed-models and produces GEBV/GETGV for existing germplasm as it did previously, but now also returns marker effects and provides a direct input to the new `predictCrosses()` function, which handles predicting mate selection criteria.

2.  Support for a directional dominance model (`modelType="DirDom"`) to incorporate genome-wide homozygosity-effects (inbreeding) into predictions.

3.  Support for selection indices.

4.  Improved version of **predCrossVar** functions at `code/predCrossVar.R`

    -   eventually to be integrated into the R package
    -   faster unless you want to use the Bayesian approach.

5.  Creation of a single-function interface to accomplish parent-wise cross-validation at `code/parentWiseCrossVal.R`. Provides estimates of (selection index) accuracy predicting family means and variances.

6.  Upgraded the "standard" cross-validation function (`runCrossVal()`), used in previous genomic selection analyses (*e.g.* [IITA_2020GS CV](https://wolfemd.github.io/IITA_2020GS/06-Results.html#cross-validation-accuracy), [NRCRI_2021GS CV](https://wolfemd.github.io/NRCRI_2021GS/05-Results.html#Prediction_accuracy), [TARI_2020GS CV](https://wolfemd.github.io/TARI_2020GS/05-Results.html#Prediction_accuracy)), to measure selection index accuracy (via `selInd=` and `SIwts=` arguments) and added a `modelType="DirDom"` option.

# Analysis steps

1.  [Prepare training dataset](01-cleanTPdata.html): Download data from DB, "Clean" and format DB data. Use the standard pipeline to obtain complete breeding trial data for IITA, generate de-regressed BLUPs for downstream analysis.

    -   Copy `gsFunctions.R` from `code/` of most recent NextGen prediction, [NRCRI C3b predicted April 2021](https://wolfemd.github.io/NRCRI_2021GS/).

    -   Reference [previous analysis for IITA (2020)](https://wolfemd.github.io/IITA_2020GS/) in case there are variations.

2.  [Get BLUPs combining all trial data](02-GetBLUPs.html): Combine data from all trait-trials to get BLUPs for downstream genomic prediction. Fit mixed-model to multi-trial dataset and extract BLUPs, de-regressed BLUPs and weights. Include two rounds of outlier removal.

3.  [Validate the pedigree obtained from cassavabase](03-validatePedigree.html): Before setting up a cross-validation scheme for predictions that depend on a correct pedigree, add a basic verification step to the pipeline. Not trying to fill unknown relationships or otherwise correct the pedigree. Assess evidence that relationship is correct, remove if incorrect.

4.  [Preprocess data files](04-PreprocessDataFiles.html): Prepare haplotype and dosage matrices, GRMs, pedigree and BLUPs, genetic map *and* recombination frequency matrix, for use in predictions.

5.  [Extract and process PHG files](08-PHGfiles.html): Extract a VCF file from the PHG `*.db` file produced by Evan Long. Subsequently, prepare haplotype and dosage matrices, GRMs, genetic map *and* recombination frequency matrix, for use in predictions.

6.  [Parent-wise and standard cross-validation](05-CrossValidation.html):

    -   All cross-validation analyses are here.

    -   Features of the new procedures:

        -   Compute parent-wise cross-validation folds using the validated pedigree. Fit models to get marker effects and make subsequent predictions of cross means and (co)variances.

        -   **Models "AD"** (classic BV+DD partition of additive+dominance) and **"DirDom"** (genotypic add+dom partition with genome-wide homozygosity effect).

        -   Include use of selection index weights to compute index accuracy.

        -   Include new models and index predictions in standard fold cross-validation in addition to the parent-wise scheme / function.

    -   Additional analyses:

        -   [**Optimize marker density for speed**](https://wolfemd.github.io/implementGMSinCassava/05-CrossValidation.html#Optimize_marker_density_for_speed): Cross-variance prediction is slow, but significant speed gains can be made by using fewer markers for the predictions. Examine the speed benefit vs. accuracy cost trade-off. [**Results are here**](https://wolfemd.github.io/implementGMSinCassava/07-Results.html#Optimize_marker_density_for_speed).

    -   [ ] ***[Coming soon]*** Run cross-validation using both Beagle- vs. PHG-imputed and phased marker data to compare their quality.

7.  [Genomic predictions](06-GenomicPredictions.html):

    -   

        (A) Standard genomic prediction of individual GEBV and GETGV for all selection candidates using all available data. (B) Predict cross means and variances for genomic mate selection.

    -   Include models "AD" *and* "DirDom"

    -   Include prediction of selection index GEBV/GETGV and $UC^{SI}_{parent}$/$UC^{SI}_{variety}$.

    -   New functions at `gmsFunctions.R` in `code/`

8.  [Results](07-Results.html): Home for plots and summary tables.

# Data and code repository access

[CLICK HERE FOR ACCESS TO THE FULL REPOSITORY](ftp://ftp.cassavabase.org/marnin_datasets/implementGMSinCassava/) (select "Guest" credentials when prompted by the Cassavabase FTP server)

or

[**DOWNLOAD FROM GitHub\***](https://github.com/wolfemd/implementGMSinCassava/)

\*GitHub only hosts files max 50 Mb.

# Guide to key directories and file names

1.  `data/`: raw data (e.g. unimputed SNP data)
2.  `output/`: outputs (e.g. imputed SNP data)
3.  `analysis/`: most code and workflow documented in **.Rmd** files
4.  `docs/`: compiled **.html**, "knitted" from **.Rmd**
5.  `code/`: supporting functions sourced in `analysis/*.Rmd`'s.

**FILES OF INTEREST**: everything is in the `output/` sub-directory ([click here](ftp://ftp.cassavabase.org/marnin_datasets/implementGMSinCassava/output/) and select "Guest" credentials when prompted by the Cassavabase FTP server).

-   **GEBVs for parent selection and GETGVs for variety advancement:**

    -   [download DirDom model genomic predictions](ftp://ftp.cassavabase.org/marnin_datasets/implementGMSinCassava/output/genomicPredictions_ModelDirDom.csv)

    -   [download AD model genomic predictions](ftp://ftp.cassavabase.org/marnin_datasets/implementGMSinCassava/output/genomicPredictions_ModelAD.csv)

-   **Predicted means, variances and usefulness of crosses among top parents:**

    -   [download DirDom model genomic mate predictions](ftp://ftp.cassavabase.org/marnin_datasets/implementGMSinCassava/output/genomicMatePredictions_top121parents_ModelDirDom.csv)

    -   [download AD model genomic mate predictions](ftp://ftp.cassavabase.org/marnin_datasets/implementGMSinCassava/output/genomicMatePredictions_top121parents_ModelAD.csv)

-   Kinship matrices, dosages, haplotype matrix, recombination frequency matrix, genetic map files

genomicPredictions_ModelDirDom.csv genomicMatePredictions_top121parents_ModelDirDom.csv genomicPredictions_ModelAD.csv genomicMatePredictions_top121parents_ModelAD.csv

# Analyses To Do

1.  **[In Progress]** PHG imputed and phased marker data

2.  Improve mate selection accuracy by...

    -   Choice of genetic map? AWC's map? Sex specific maps?
    -   Multi-trait REML models, single-step, heterogeneous error and/or of the other usual possibilities,
    -   and/or a return to the compute intense Bayesian posterior distribution approach used in the MS... might be more attractive with fewer markers to make compute faster

3.  Simulation to explore factors impacting estimate of accuracy

    -   Impact of phasing switch errors
    -   Quality and quantity of data available on parents
    -   Relatedness of parents
    -   What is the relationship between the true accuracy and estimate of accuracy?
