6 Report

6.1 General structure

The slapnap report consists of an executive summary followed by results for each requested outcome.

The executive summary contains:

  • descriptions of outcomes (including how any derived outcomes are generated);
  • descriptive statistics detailing the number of sequences extracted from CATNAP, the number of sequences with complete feature and outcome information, and the number of estimated sensitive and resistant sequences (defined based on sensitivity, estimated sensitivity, and/or multiple sensitivity);
  • a table describing the learners used to predict each outcome;
  • a table of cross-validated prediction performance for each outcome (if cvperf = TRUE);
  • a table of ranked marginal intrinsic prediction performance for each feature group and outcome (if "marg" is included in importance_grp); and
  • a table of ranked conditional intrinsic prediction performance for each feature group and outcome (if "cond" is included in importance_grp).

The rest of the report is organized by outcome. Each of these sections contains descriptive statistics including summaries of the distribution of the outcome (raw and log-transformed) for each bnAb for continuous outcomes and number sensitive/resistant for binary outcomes. Based on the specific options passed to slapnap, the following subsections may also be present:

  • a table of super learner weights (Section 5.3) if an ensemble is used (i.e., multiple learners are requested);
  • cross-validated prediction performance for the fitted learner (or super learner), if cvperf="TRUE": figures showing cross-validated prediction performance (all outcomes), cross-validated receiver operating characteristic (ROC) curves (binary outcomes), and cross-validated predicted probabilities of resistance (binary outcomes); and
  • variable importance (if importance_ind and/or importance_grp is specified): intrinsic importance (group and/or individual) and/or predictive importance.

Finally, if group intrinsic importance is requested, then the variable groups are displayed in a section immediately preceding the references.

6.2 Example reports

Here we include several example reports and the slapnap container run commands that generated them.

6.2.1 Single antibodies

The following code evaluates binary sensitivity (defined as the indicator that IC\(_{80} < 1\)) for VRC01 using a super learner that includes all three learner types, each with multiple tuning parameter values, and with different variable screening techniques. We also request marginal group and individual intrinsic importance and individual predictive importance. If running this command locally, change docker_output_directory to the path to the folder where the output is to be saved.

See the report

sudo docker run \
    -d \
    -v docker_output_directory:/home/output \
    -e nab="VRC01" \
    -e outcomes="ic80;sens" \
    -e binary_outcomes="ic80" \
    -e learners="rf;lasso;xgboost" \
    -e sens_thresh="1" \
    -e var_thresh="0;4;8" \
    -e nfolds="5" \
    -e cvtune="TRUE" \
    -e cvperf="TRUE" \
    -e importance_grp="marg" \
    -e importance_ind="marg;pred" \
    -e return="report" \
    slapnap/slapnap

The next code chunk evaluates binary sensitivity (defined as the indicator that IC\(_{50} < 50\)) for 10-1074 using a super learner that includes all three learner types, each with multiple tuning parameter values. If running this command locally, change docker_output_directory to the path to the folder where the output is to be saved.

See the report

sudo docker run \
    -d
    -v docker_output_directory:/home/output/ \
    -e nab="10-1074" \
    -e outcomes="sens" \
    -e learners="rf;lasso;xgboost" \
    -e sens_thresh="50" \
    -e nfolds="5" \
    -e cvtune="TRUE" \
    -e cvperf="TRUE" \
    -e return="report" \
    slapnap/slapnap

6.2.2 Multiple antibodies

The following code evaluates binary sensitivity outcomes for a combination antibody using a super learner that includes all three learner types, each with multiple tuning parameter values, and with different variable screening techniques. If running this command locally, change docker_output_directory to the path to the folder where output is to be saved.

See the report.

docker run \
  -d \
  -v docker_output_directory:/home/output \
  -e learners="rf;lasso;xgboost" \
  -e cvperf="TRUE" \
  -e cvtune="TRUE" \
  -e nab="10-1074;PG9" \
  -e outcomes="estsens;multsens" \
  -e sens_thresh="1" \
  -e var_thresh="0;4" \
  -e return="report" \
  -e nfolds="5" \
  slapnap/slapnap