---
title: "IITA 2021 Genomic Prediction and Mate Selection"
site: workflowr::wflow_site
output:
  workflowr::wflow_html:
    toc: false
editor_options:
  chunk_output_type: console
---

-   I recently (Aug 3, 2021) completed thorough testing to develop a protocol for genomic *mate* selection in NextGen Cassava and will implement it here for the first time in practice! Check [here](https://wolfemd.github.io/implementGMSinCassava/) to see the implementation tests / code-base development documented.
-   Subsequently, I have developed my entire code base into an R package [genomicMateSelectR](https://wolfemd.github.io/genomicMateSelectR/index.html). The package is fully documented (in rough draft version), but doesn't yet include a tutorial / vignette.
-   The IITA DArTseqLD report (DCas21_6038), which contains the GS C5 (i.e. progeny of crosses made in 2020, "TMS20F"), was recently (July 19, 2021) released.
-   The imputations of DCas21_6038, genomic predictions and mate selection that follows will leverage [genomicMateSelectR](https://wolfemd.github.io/genomicMateSelectR/index.html) functions and will form the principal example of their use. Install it e.g. `devtools::install_github("wolfemd/genomicMateSelectR", ref = 'master')` .

[**See the Results here!**](07-Results.html)

# August 2021

## Imputation of DCas21_6038

**Steps**:

1.  [Convert DCas21-6038 report to VCF for imputation](convertDCas21_6038_ToVCF.html):
2.  [Impute DCas21-6038](ImputeDCas21_6038.html): with West Africa reference panel merged with additional GS progeny (IITA TMS18)

**Files**: Access on Cassavabase FTP server here, use "Guest" credentials

-   **RefPanel VCF filename:** `chr*_RefPanelAndGSprogeny_ReadyForGP_72719.vcf.gz`\
    ([**Click here for FTP server location**](ftp://ftp.cassavabase.org/marnin_datasets/nextgenImputation2019/ImputationStageIII_72619/))
-   **Imputed filename:** `output/chr*_DCas21_6038_WA_REFimputed.vcf.gz`\
    ([**Click here for FTP server location**](ftp://ftp.cassavabase.org/marnin_datasets/IITA_2021GS/))
-   **Post-impute filtered filename:** `output/chr*_DCas21_6038_WA_REFimputedAndFiltered.vcf.gz`\
    ([**Click here for FTP server location**](ftp://ftp.cassavabase.org/marnin_datasets/IITA_2021GS/))
-   **All chrom. combined, RefPanel and DCas21_6038 VCF merged** `output/AllChrom_RefPanelAndGSprogeny_ReadyForGP_2021Aug08.vcf.gz`\
    ([**Click here for FTP server location**](ftp://ftp.cassavabase.org/marnin_datasets/IITA_2021GS/))

## Preliminary data steps

1.  [Prepare training dataset](01-cleanTPdata.html): Download data from DB, "Clean" and format DB data.

2.  [Get BLUPs combining all trial data](02-GetBLUPs.html): Combine data from all trait-trials to get BLUPs for downstream genomic prediction. Fit mixed-model to multi-trial dataset and extract BLUPs, de-regressed BLUPs and weights. Include two rounds of outlier removal.

3.  [Validate the pedigree obtained from cassavabase](03-validatePedigree.html): Before setting up a cross-validation scheme for predictions that depend on a correct pedigree, add a basic verification step to the pipeline. Not trying to fill unknown relationships or otherwise correct the pedigree. Assess evidence that relationship is correct, remove if incorrect.

4.  [Preprocess data files](04-PreprocessDataFiles.html): Prepare haplotype and dosage matrices, GRMs, pedigree and BLUPs, genetic map *and* recombination frequency matrix, for use in predictions.

## Genomic (mate) predictions

5.  [Parent-wise and standard cross-validation](05-CrossValidation.html): estimate selection index (and component trait) prediction accuracies using the direction-dominance (**DirDom**) model.

    -   Additionally, [check accuracy and similarity of predictions at reduced marker density](https://wolfemd.github.io/IITA_2021GS/05-CrossValidation.html#Optimize_marker_density_for_speed): Cross-variance prediction is slow, but significant speed gains can be made by using fewer markers. Faster predictions will mean more crosses can be predicted and considered.

        -   ***If,*** accuracy and $cor_{preds}(All\_SNPs, Reduced\_Set)$ are similar based on both kinds of cross-validation, proceed to make cross-variance predictions with reduced marker set; possibly use full marker set for cross-mean predictions.

    -   [Click here to see the results](https://wolfemd.github.io/IITA_2021GS/07-Results.html#Prediction_accuracy_estimates)**!**

6.  [Genomic predictions](06-GenomicPredictions.html): First, predict of individual GEBV/GETGV for all selection candidates using all available data and return marker effects for use downstream. Next, Select a top set of candidate parents, for whom we would like to predict cross performances. Finally, predict all pairwise crosses of candidate parents and evaluate them for genomic mate selection. Select the top crosses and plant a crossing nursery with the parents indicated.

    -   [Click here to see the results](https://wolfemd.github.io/IITA_2021GS/07-Results.html#Genomic_Predictions)**!**

7.  [Results and recommendations](07-Results.html): Home for all plots, summary tables, conclusions and recommendations.

## Empricial inputs for simulations

See the updated [version of this analysis here](inputsForSimulationV2.html) and the [original analysis here](inputsForSimulation.html). 

Uses two approaches to empirically estimate the measurement (selection) error associated with different TrialType's / plot configurations. 

First approach, was to fit a multivariate mixed-model with heterogenous error covariances among TrialType. The direct approach allowed calculating the SELIND error variance (y-axis) by $b^T\boldsymbol{R}_{TrialType}b$, where $\boldsymbol{R}_{TrialType}$ is the TrialType-specific estimate of the error covariance matrix and $b$ are the SELIND weights. 

The second approach was to fit univariate mixed-models to each trait in each trial, then compute trial-specific SELIND using the resulting BLUPs. The SELIND GETGV value for all clones based on using _all_ phenotypic data and genomic information was then regressed on each trial's SELIND BLUPs. The mean squared error (mean residual variance) from each regression was extracted and then the average TrialMSE by TrialType was computed.

We need to choose one of these two options, or revise the approach further, for use input for VDP simulations. Note that in simulation, the error variances we input will be divided by the Nrep and Nloc for each stage specific, so even if e.g. UYT has worse error than AYT overall, at the clone-level, UYT would have lower error b/c of more reps and locs.

For downstream usage of the results, see the [breeding scheme simulations here](https://wolfemd.github.io/BreedingSchemeOpt/). 

# Data and code repository access

[CLICK HERE FOR ACCESS TO THE FULL REPOSITORY](ftp://ftp.cassavabase.org/marnin_datasets/IITA_2021GS/)\
(select "Guest" credentials when prompted by the Cassavabase FTP server)

or

[**DOWNLOAD FROM GitHub\***](https://github.com/wolfemd/IITA_2021GS/)

\*GitHub only hosts files max 50 Mb.

# Key directories and file names

1.  `data/`: raw data (e.g. unimputed SNP data)
2.  `output/`: outputs (e.g. imputed SNP data)
3.  `analysis/`: most code and workflow documented in **.Rmd** files
4.  `docs/`: compiled **.html**, "knitted" from **.Rmd**
5.  `code/`: supporting functions sourced in `analysis/*.Rmd`'s.

**FILES OF INTEREST**: everything is in the `output/` sub-directory ([click here](ftp://ftp.cassavabase.org/marnin_datasets/IITA_2021GS/output/) and select "Guest" credentials when prompted by the Cassavabase FTP server).

-   **GEBVs for parent selection and GETGVs for variety advancement:**

    -   [download DirDom model genomic predictions](ftp://ftp.cassavabase.org/marnin_datasets/IITA_2021GS/output/genomicPredictions_full_set_2021Aug09.csv)

-   **Predicted means, variances and usefulness of crosses among top parents:**

    -   [download DirDom model genomic mate predictions](ftp://ftp.cassavabase.org/marnin_datasets/IITA_2021GS/output/genomicMatePredictions_2021Aug10.csv)

-   Kinship matrices, dosages, haplotype matrix, recombination frequency matrix, genetic map files
