Merge pull request #32 from OpenOmics/documentation

Updating documentation along with a few other minor updates
* Increasing resources for Seurat integration
* Editing generateSummaryScript for multi pipeline to remove the assumption that GEX would always be present
* Improve formatting for multi Seurat QC Summary Report

Author: chenv3

VERSION

@@ -1 +1 @@
-3.4.0
+3.4.1

config/cluster.json

@@ -28,9 +28,9 @@
     },
     "seuratIntegrate": {
         "threads": "8",
-        "mem": "350g",
+        "mem": "1000g",
         "partition": "largemem",
-        "time": "1-00:00:00"
+        "time": "3-00:00:00"
     },
     "shinycodes": {
         "threads": "16",

docs/usage/atac.md

@@ -46,7 +46,7 @@ Each of the following arguments are required. Failure to provide a required argu
 >
 > FastQ Input: One or more FastQ files can be provided. The pipeline does NOT support single-end data. From the command-line, each input file should separated by a space. Multiple input FastQ files per sample can be provided. Globbing is supported! This makes selecting FastQ files easy. Input FastQ files should always be gzipp-ed.
 >
-> ***Example:*** `--input .tests/*.R?.fastq.gz`
+> ***Example:*** `--input .tests/*_R?_fastq.gz`
 >
 >
 > Cell Ranger Input: Cell Ranger output folders can be provided. It is expected that the outs folder is contained within the Cell Ranger output folders, and keep the normal output folder structure. Globbing is supported!
@@ -244,6 +244,76 @@ Each of the following arguments are optional, and do not need to be provided.
 
 ## 3. Example
 
+### 3.1 Basic Pipeline Usage
+
+#### 3.1.1 Single Input Folder
+
+```bash
+# Step 1.) Grab an interactive node,
+# do not run on head node!
+srun -N 1 -n 1 --time=1:00:00 --mem=8gb  --cpus-per-task=2 --pty bash
+module purge
+module load singularity snakemake
+
+# Step 2A.) Dry-run the pipeline
+./cell-seek run --input .tests/*_R?_fastq.gz \
+                  --output /data/$USER/output \
+                  --pipeline atac \
+                  --genome hg38 \
+                  --cellranger 2.2.0 \
+                  --mode slurm \
+                  --dry-run
+
+# Step 2B.) Run the cell-seek pipeline
+# The slurm mode will submit jobs to
+# the cluster. It is recommended running
+# the pipeline in this mode.
+./cell-seek run --input .tests/*_R?_fastq.gz \
+                  --output /data/$USER/output \
+                  --pipeline atac \
+                  --genome hg38 \
+                  --cellranger 2.2.0 \
+                  --mode slurm
+```
+
+#### 3.1.2 Multiple Input Folders
+
+FASTQ files from multiple folders can be provided as input. The paths to different FASTQ files can be provided in a space separated format.
+
+```bash
+# Step 1.) Grab an interactive node,
+# do not run on head node!
+srun -N 1 -n 1 --time=1:00:00 --mem=8gb  --cpus-per-task=2 --pty bash
+module purge
+module load singularity snakemake
+
+# Step 2A.) Dry-run the pipeline
+./cell-seek run --input .tests/*_R?_fastq.gz .tests2/*_R?_fastq.gz \
+                  --output /data/$USER/output \
+                  --pipeline atac \
+                  --genome hg38 \
+                  --cellranger 2.2.0 \
+                  --mode slurm \
+                  --dry-run
+
+# Step 2B.) Run the cell-seek pipeline
+# The slurm mode will submit jobs to
+# the cluster. It is recommended running
+# the pipeline in this mode.
+./cell-seek run --input .tests/*_R?_fastq.gz .tests2/*_R?_fastq.gz \
+                  --output /data/$USER/output \
+                  --pipeline gex \
+                  --genome hg38 \
+                  --cellranger 2.2.0 \
+                  --mode slurm
+```
+
+### 3.2 Run Downstream on Existing Cell Ranger Output
+
+It is possible to use cell-seek to perform the initial downstream analysis on existing Cell Ranger ATAC output. The files are expected to be in the Cell Ranger ATAC outputted format with the outs folder present.  The sample level folders should be provided as the input for cell-seek.
+
+For example, if sample1 was run in Cell Ranger ATAC then sample1/outs/ contains the final pipeline output files, and sample1 should be provided as input to cell-seek.
+
 ```bash
 # Step 1.) Grab an interactive node,
 # do not run on head node!
@@ -252,7 +322,7 @@ module purge
 module load singularity snakemake
 
 # Step 2A.) Dry-run the pipeline
-./cell-seek run --input .tests/*.R?.fastq.gz \
+./cell-seek run --input .tests/*/ \
                   --output /data/$USER/output \
                   --pipeline atac \
                   --genome hg38 \
@@ -264,11 +334,117 @@ module load singularity snakemake
 # The slurm mode will submit jobs to
 # the cluster. It is recommended running
 # the pipeline in this mode.
-./cell-seek run --input .tests/*.R?.fastq.gz \
+./cell-seek run --input .tests/*/ \
                   --output /data/$USER/output \
                   --pipeline atac \
                   --genome hg38 \
                   --cellranger 2.2.0 \
                   --mode slurm
 ```
 
+
+### 3.3 Renaming Samples
+
+It is possible to rename samples to different names from the ones that are used in the FASTQ files. This function can be used to change the sample names to something more informative, or it could be used if FASTQ files names changed for a specific sample.  
+
+In order to use this option, a CSV file should be created. In the CSV there should be a row for each FASTQ file that will be processed. The first column contains the name of the FASTQ file while the second column contains the output sample name. Only the FASTQ files and samples that are listed within the CSV file will be processed.
+
+Cell Ranger expects the FASTQ file to have a format of
+
+`[Sample Name]_S[Sample Number]_L00[Lane Number]_[Read Type]_001.fastq.gz`
+
+or
+
+`[Sample Name]_S[Sample Number]_[Read Type]_001.fastq.gz`
+
+The FASTQ name that is used in the rename CSV file should match the section in the `[Sample Name]` listed above. An example file could be `Sample_ATAC_S1_L001_R1_001.fastq.gz`. In this file, the sample name would be Sample_ATAC. More information about the FASTQ naming format that Cell Ranger expects can be found at the [10x Genomics website](https://www.10xgenomics.com/support/software/cell-ranger-atac/latest/analysis/inputs/specifying-input-fastq-files).
+
+The following is a potential example of a rename CSV file.
+
+```
+FASTQ,Name
+sample1_run1,sample1
+sample2_run1,sample2
+sample3_run1,sample3
+sample3_run2,sample3
+sample4,sample4
+```
+
+Based on this file, the FASTQ files with the name sample1_run1, sample2_run1, sample3_run1, sample3_run2, and sample4 will be processed, and any FASTQ file that does not match these names will not be run. The outputted sample names would be sample1, sample2, sample3, and sample4. The FASTQ files with the name sample3_run1 and sample3_run2 will be processed as sample3. 
+
+
+```bash
+# Step 1.) Grab an interactive node,
+# do not run on head node!
+srun -N 1 -n 1 --time=1:00:00 --mem=8gb  --cpus-per-task=2 --pty bash
+module purge
+module load singularity snakemake
+
+# Step 2A.) Dry-run the pipeline
+./cell-seek run --input .tests/*_R?_fastq.gz \
+                  --output /data/$USER/output \
+                  --pipeline atac \
+                  --genome hg38 \
+                  --cellranger 2.2.0 \
+                  --rename rename.csv \
+                  --mode slurm \
+                  --dry-run
+
+# Step 2B.) Run the cell-seek pipeline
+# The slurm mode will submit jobs to
+# the cluster. It is recommended running
+# the pipeline in this mode.
+./cell-seek run --input .tests/*_R?_fastq.gz \
+                  --output /data/$USER/output \
+                  --pipeline atac \
+                  --genome hg38 \
+                  --cellranger 2.2.0 \
+                  --rename rename.csv \
+                  --mode slurm
+```
+
+### 3.4 Running while Forcing Cell Call
+
+It is possible to force the number of cells that are called by Cell Ranger ATAC. In this situation, Cell Ranger ATAC will call the top X cell barcodes with the highest number of fragments overlapping peaks as cells, where X the number cells that the sample is forced to. This is generally used if the first analysis run appears to do a poor job at estimating the number of cells, and a re-run while adjusting the number of cells in the sample is helpful.
+
+A CSV file needs to be created with the first column containing the name of the sample (the Cell Ranger outputted name) and the second column containing the number of cells to force the sample to. Only the samples included in the CSV file will be run while forcing the cell call. Any other samples that are processed will use the default cell calling algorithm.
+
+The following is an example of a force cells CSV file.
+
+```
+Sample,Cells
+Sample1,3000
+Sample2,5000
+```
+
+Based on this file, Sample1 and Sample 2 will be run while being forced to have 3000 and 5000 cells respectively.
+
+```bash
+# Step 1.) Grab an interactive node,
+# do not run on head node!
+srun -N 1 -n 1 --time=1:00:00 --mem=8gb  --cpus-per-task=2 --pty bash
+module purge
+module load singularity snakemake
+
+# Step 2A.) Dry-run the pipeline
+./cell-seek run --input .tests/*_R?_fastq.gz \
+                  --output /data/$USER/output \
+                  --pipeline atac \
+                  --genome hg38 \
+                  --cellranger 2.2.0 \
+                  --forcecells forcecells.csv \
+                  --mode slurm \
+                  --dry-run
+
+# Step 2B.) Run the cell-seek pipeline
+# The slurm mode will submit jobs to
+# the cluster. It is recommended running
+# the pipeline in this mode.
+./cell-seek run --input .tests/*_R?_fastq.gz \
+                  --output /data/$USER/output \
+                  --pipeline atac \
+                  --genome hg38 \
+                  --cellranger 2.2.0 \
+                  --forcecells forcecells.csv \
+                  --mode slurm
+```