---
title: "TARI 2020 GS and related procedures"
site: workflowr::wflow_site
author: "Marnin Wolfe"
output:
  workflowr::wflow_html:
    toc: false
editor_options:
  chunk_output_type: console
---

This repository and website documents all analyses, summary, tables and figures associated with TARI genomic prediction and related procedures (e.g. imputation).

# December Imputations


### DCas20_5629

GS C1. 

Impute with E. Africa Imputation Reference Panel dataset, which can be found on the [Cassavabase FTP server here](ftp://ftp.cassavabase.org/marnin_datasets/nextgenImputation2019/ImputationEastAfrica_StageII_90919/) with names e.g. `chr*_ImputationReferencePanel_StageVI_91119.vcf.gz` with [code/documentation here](https://wolfemd.github.io/NaCRRI_2020GS/).

**Steps**:

1.   [Convert DCas20_5629 report to VCF for imputation](convertDCas20_5629_ToVCF.html):
2.   [Impute DCas20_5629](ImputeDCas20_5629.html): with East Africa reference panel

**Files**:

-   **RefPanel VCF filename:** `chr*_ImputationReferencePanel_StageVI_91119.vcf.gz`
-   **Imputed filename:** `chr*_DCas20_5629_EA_REFimputed.vcf.gz`
-   **Post-impute filtered filename:** `chr*_DCas20_5629_EA_REFimputedAndFiltered.vcf.gz`
-   **Genome-wide dosage matrix format for use in R:**
    -   Imputation Reference Panel: `DosageMatrix_ImputationReferencePanel_StageVI_91119.rds`
    -   DCas20\_5629 with standard post-impute filter: `DosageMatrix_DCas20_5629_EA_REFimputedAndFiltered.rds`

**HOW TO COMBINE DOSAGE MATRICES:** Users will want to combine the genotypes in the imputation reference panel files, with the genotypes in the imputed DArT file. They can have slightly different sets of markers along the columns. Here is a basic example how to combine:

```{r, eval=F}
snps_refpanel<-readRDS("DosageMatrix_ImputationReferencePanel_StageVI_91119.rds")
snps_dcas20_5629<-readRDS("DosageMatrix_DCas20_5629_EA_REFimputedAndFiltered.rds")

snps2keep<-colnames(snps_refpanel)[,colnames(snps_refpanel) %in% colnames(snps_dcas20_5629)]
snps<-bind_rows(snps_refpanel[,snps2keep],
                snps_dcas20_5629[,snps2keep])
```

# Genomic Prediction (Dec. 20 - Jan. 21)

Get TARI TP data from Cassavabase. Use it with imputed data to predict GEBV/GETGV for all samples in the new reports (**DCas20-5629**). 

1. [Prepare training dataset](01-cleanTPdata.html): Download data from DB, "Clean" and format DB data.
    * **[UPDATED Jan. 21, 2021]:** Matches of germplasmName-to-FullSampleName (GBS and DArTseqLD records) improved for TARI. 
    * All subsequent analyses re-run.
2. [Get BLUPs combining all trial data](02-GetBLUPs.html): Combine data from all trait-trials to get BLUPs for downstream genomic prediction. 
    * Fit mixed-model to multi-trial dataset and extract BLUPs, de-regressed BLUPs and weights. Include two rounds of outlier removal.
3. [Check prediction accuracy](03-CrossValidation.html): Evaluate prediction accuracy with cross-validation.
4. [Genomic prediction](04-GetGBLUPs.html): Predict _genomic_ BLUPs (GEBV and GETGV) for all selection candidates using all available data.
5. [Results](05-Results.html): Plots, results and recommendations.
    * 1) Accuracy estimates are most improved relative to previously. I didn't run the precise cross-validation folds so the judgement is based on visual comparison to the Dec. 2020 plot. 
    * 2) DYLD and FYLD are _not_ well predicted and I would not recommend using them based on selection.
    * 3) Expected Genetic Gain: GS C1 and other germplasm in DCAs20_5629 have better predicted GEBV on average than the TP!


**OUTPUT / FILES**: everything is in the `output/` sub-directory. 

-  **GEBVs for parent selection:** `GEBV_TARI_ModelA_2021Jan21.csv`
-  **GETGVs for variety advancement:** `GETGV_TARI_ModelADE_2021Jan21.csv`
-  **Tidy, long-form CSV of predictions, including PEVs:** `genomicPredictions_TARI_2021Jan21.csv`

[**DOWNLOAD FROM CASSAVABASE FTP SERVER**](ftp://ftp.cassavabase.org/marnin_datasets/TARI_2020GS/output/)

or  

[**DOWNLOAD FROM GitHub**](https://github.com/wolfemd/TARI_2020GS/tree/master/output)

# Data availability and reproducibility

The R package **workflowr** was used to document this study reproducibly.

Much of the supporting data *and* output from the analyses documented here are too large for GitHub.

The repository will be mirrored, here: <ftp://ftp.cassavabase.org/marnin_datasets/TARI_2020GS/> with all data.

# Directory structure of this repository

**NOTICE:** `data/` and `output/` are empty on GitHub. Please see <ftp://ftp.cassavabase.org/marnin_datasets/TARI_2020GS/> for access.

1.  `data/`: raw data (e.g. unimputed SNP data)
2.  `output/`: outputs (e.g. imputed SNP data)
3.  `analysis/`: most code and workflow documented in **.Rmd** files
4.  `docs/`: compiled **.html**, "knitted" from **.Rmd**

Supporting functions `code/`

The analyses in the **html** / **Rmd** files referenced above often source R scripts in the `code/` sub-folder. 
