Guided-demo: Exploring 'ddmore' R functionality with the warfarin model

UseCase3 : PKPD model for warfarin population pharmacokinetics and PCA

This example script is intended to illustrate how to use the 'ddmore' R package to perform a M&S workflow using the DDMoRe Standalone Execution Environment (SEE).

The following steps are implemented in this workflow:

To run a task, select wiht the coursor any code lines you wish to execute and press CTRL+R+R in your keyboard. An HTML file containing the commands in this file and associated output will be provided to allow the user to compare the results

Initialisation

Clear workspace and set working directory under 'models' folder

Clear workspace and set working directory under 'UsesCasesDemo' project

rm(list=ls(all=F))
mydir <- file.path(Sys.getenv("MDLIDE_WORKSPACE_HOME"),"UseCasesDemo")
setwd(mydir)

Set name of .mdl file and dataset for future tasks

uc <- "UseCase3"
datafile <- "warfarin_conc_pca.csv"
mdlfile <- paste0(uc,".mdl")

Create a new folder to be used as working directory and where results will be stored

wd <- file.path(mydir,uc)
dir.create(wd)

Copy the dataset and the .mdl file available under “models” into the working directory

file.copy(file.path(mydir,"models", datafile),wd)
## [1] TRUE
file.copy(file.path(mydir,"models",mdlfile),wd)
## [1] TRUE

Set the working directory.

setwd(file.path(mydir,uc))

The working directory needs to be set differently when knitr R package is used to spin the file

library(knitr)
opts_knit$set(root.dir = file.path(mydir,uc)) 

List files available in working directory

list.files()
## [1] "UseCase3.mdl"          "warfarin_conc_pca.csv"

Introduction to 'ddmore' R package

View objects within the .mdl file

Use 'ddmore' function getModelObjects() to retrieve model object(s) from an existing .mdl file. This function reads the MDL in an .mdl file and parses the MDL code for each MDL Object into an R list of objects of appropriate types with names corresponding to the MDL Object names given in the file.

myMDLObj <- getMDLObjects(mdlfile)
length(myMDLObj)
## [1] 4
names(myMDLObj)
## [1] "warfarin_PKPD_turnover_dat"  "warfarin_PKPD_turnover_par" 
## [3] "warfarin_PKPD_turnover_mdl"  "warfarin_PKPD_turnover_task"

Use 'ddmore' function getDataObjects() to retrieve only data object(s) from an existing .mdl file This function returns a list of Parameter Object(s) from which we select the first element Hoover over the variable name to see its structure

myDataObj <- getDataObjects(mdlfile)[[1]]

Use 'ddmore' function getParameterObjects() to retrieve only parameter object(s) from an existing .mdl file

myParObj <- getParameterObjects(mdlfile)[[1]]

Use 'ddmore' function getModelObjects() to retrieve only model object(s) from an existing .mdl file.

myModObj <- getModelObjects(mdlfile)[[1]]

Use 'ddmore' function getTaskPropertiesObjects() to retrieve only task properties object(s) from an existing .mdl file

myTaskObj <- getTaskPropertiesObjects(mdlfile)[[1]]

Exploratory Data Analysis

Recall that getDataObjects only reads the MDL code from the .mdl file. Use 'ddmore' function readDataObj() to create an R object from the MDL data object.

myData <- readDataObj(myDataObj)

Let's look at the first 6 lines of the data set

head(myData)
##   ID TIME   WT AGE  SEX AMT DVID  DV MDV
## 1  1  0.0 66.7  50 male 100    0  NA   1
## 2  1  0.0 66.7  50 male  NA    2  NA   1
## 3  1  0.5 66.7  50 male  NA    1 0.0   0
## 4  1  1.0 66.7  50 male  NA    1 1.9   0
## 5  1  2.0 66.7  50 male  NA    1 3.3   0
## 6  1  3.0 66.7  50 male  NA    1 6.6   0

Extract only observation records

myEDAData<-myData[myData$MDV==0,]

Open an R window to record and access all your plots

windows(record=TRUE) 

Now plot the data using xyplot from the lattice library

plot1 <- xyplot(DV~TIME,groups=ID,data=myEDAData,subset=DVID==1,type="b",ylab="Conc. (mg/L)",xlab="Time (h)")
print(plot1)

plot of chunk unnamed-chunk-15

plot2 <- xyplot(DV~TIME|ID,data=myEDAData,subset=DVID==1,type="b",layout=c(3,4),ylab="Conc. (mg/L)",xlab="Time (h)",scales=list(relation="free"))
print(plot2)

plot of chunk unnamed-chunk-15 plot of chunk unnamed-chunk-15 plot of chunk unnamed-chunk-15

plot3 <- xyplot(DV~TIME,groups=ID,data=myEDAData,subset=DVID==2,type="b",ylab="PCA",xlab="Time (h)")
print(plot3)

plot of chunk unnamed-chunk-15

plot4 <- xyplot(DV~TIME|ID,data=myEDAData,subset=DVID==2,type="b",layout=c(3,4),ylab="PCA",xlab="Time (h)",scales=list(relation="free"))
print(plot4)

plot of chunk unnamed-chunk-15 plot of chunk unnamed-chunk-15 plot of chunk unnamed-chunk-15

Export the results in a PDF file

pdf(paste0(uc,"_EGA.pdf"))
 print(plot1)
 print(plot2)
 print(plot3)
 print(plot4)
dev.off()
## rj.GD 
##     2

Model Development

ESTIMATE model parameters using Monolix

The ddmore “estimate” function translates the contents of the .mdl file to a target language and then estimates parameters using the target software. After estimation, the output is converted to a Standardised Output object which is saved in a .SO.xml file.

Translated files and Monolix output will be returned in the ./Monolix subfolder. The Standardised Output object (.SO.xml) is read and parsed into an R object called “mlx” of (S4) class “StandardOutputObject”.

mlx <- estimate(mdlfile, target="MONOLIX", subfolder="Monolix")
## -- Fri Dec 11 03:50:03 2015
## New
## Submitted
## Job 9ee42c3e-e971-4d7f-8689-b5ab8eadf70b progress:
## Running [ ............ ]
## Importing Results
## Copying the result data back to the local machine for job ID 9ee42c3e-e971-4d7f-8689-b5ab8eadf70b...
## From C:\Users\zparra\AppData\Local\Temp\Rtmpkfi8Jd\DDMORE.jobfd049194167 to D:/SEE-1.2-SNAPSHOT-10DEC2015/MDL_IDE/workspace/UseCasesDemo/UseCase3/Monolix
## Done.
## 
## 
## The following elements were parsed successfully:
##       ToolSettings
##       RawResults
##       TaskInformation
##       Estimation:PopulationEstimates
##       Estimation:PrecisionPopulationEstimates
##       Estimation:IndividualEstimates
##       Estimation:Residuals
##       Estimation:Predictions
##       Estimation:Likelihood
## 
## Completed
## -- Fri Dec 11 03:54:09 2015
slotNames(mlx)
## [1] "ToolSettings"     "RawResults"       "TaskInformation" 
## [4] "Estimation"       "ModelDiagnostic"  "Simulation"      
## [7] "OptimalDesign"    ".pathToSourceXML"

The ddmore “LoadSOObj” reads and parsed existing Standardise Output Objects mlx <- LoadSOObject(“Monolix/UseCase3.SO.xml”) The ddmore function “getPopulationParameters” extracts the Population Parameter values from an R object of (S4) class “StandardOutputObject” and returns the estimates. See documentation for getPopulationParameters to see other arguments and settings for this function.

parameters_mlx <- getPopulationParameters(mlx, what="estimates")$MLE
print(parameters_mlx)
##        POP_V    BETA_V_WT     POP_PCA0       POP_KA       POP_CL 
##      7.92329      1.00000     96.54103      1.45270      0.13633 
##   BETA_CL_WT      POP_TEQ      POP_C50     POP_EMAX     POP_TLAG 
##      0.75000     13.17685      1.15507      1.00000      0.95372 
##        PPV_V     PPV_PCA0       PPV_KA       PPV_CL      PPV_TEQ 
##      0.14805      0.05347      1.06406      0.25932      0.06318 
##      PPV_C50     PPV_EMAX     PPV_TLAG ETA_V_ETA_CL      RUV_ADD 
##      0.42122      0.00000      0.01000      0.17202      0.42894 
##     RUV_PROP       RUV_FX 
##      0.04595      3.81603
print(getPopulationParameters(mlx, what="precisions"))
## $MLE
##       Parameter      MLE      SE    RSE
## 1    BETA_CL_WT  0.75000 0.00000   0.00
## 2     BETA_V_WT  1.00000 0.00000   0.00
## 3  ETA_V_ETA_CL  0.17202 0.20124 116.99
## 4       POP_C50  1.15507 0.09305   8.06
## 5        POP_CL  0.13633 0.00644   4.72
## 6      POP_EMAX  1.00000 0.00000   0.00
## 7        POP_KA  1.45270 0.35984  24.77
## 8      POP_PCA0 96.54103 1.13877   1.18
## 9       POP_TEQ 13.17685 0.30360   2.30
## 10     POP_TLAG  0.95372 0.00373   0.39
## 11        POP_V  7.92329 0.23736   3.00
## 12      PPV_C50  0.42122 0.05997  14.24
## 13       PPV_CL  0.25932 0.03425  13.21
## 14     PPV_EMAX  0.00000 0.00000   0.00
## 15       PPV_KA  1.06406 0.18620  17.50
## 16     PPV_PCA0  0.05347 0.01026  19.18
## 17      PPV_TEQ  0.06318 0.03200  50.64
## 18     PPV_TLAG  0.01000 0.00000   0.00
## 19        PPV_V  0.14805 0.02371  16.01
## 20      RUV_ADD  0.42894 0.06155  14.35
## 21       RUV_FX  3.81603 0.24483   6.42
## 22     RUV_PROP  0.04595 0.01064  23.16
print(getEstimationInfo(mlx))
## $Likelihood
## $Likelihood$LogLikelihood
## [1] -1082.72
## 
## $Likelihood$IndividualContribToLL
##    Subject ICtoLL
## 1        1 -48.94
## 2        2 -27.41
## 3        3 -47.03
## 4        4 -35.85
## 5        5 -40.31
## 6        6 -31.98
## 7        7 -43.30
## 8        8 -44.85
## 9        9 -55.00
## 10      10 -28.55
## 11      12 -41.67
## 12      13 -34.91
## 13      14 -45.87
## 14      15 -36.08
## 15      16 -39.99
## 16      17 -28.98
## 17      18 -26.10
## 18      19 -25.90
## 19      20 -27.70
## 20      21 -25.97
## 21      22 -29.94
## 22      23 -31.07
## 23      24 -25.44
## 24      25 -39.81
## 25      26 -28.53
## 26      27 -25.24
## 27      28 -30.95
## 28      29 -26.30
## 29      30 -28.07
## 30      31 -27.92
## 31      32 -26.83
## 32      33 -26.23
## 
## $Likelihood$InformationCriteria
## $Likelihood$InformationCriteria$AIC
## [1] 2199.44
## 
## $Likelihood$InformationCriteria$BIC
## [1] 2224.36
## 
## 
## 
## $Messages
## list()

Perform model diagnostics for the base model using Xpose functions (graphs are exported to PDF)

There is currently a bug with as.xpdb and Monolix in UseCase3. Therefore, goodness-of-fit plots are created manually from the standardised output. Later in the script, Xpose functionality on UseCase3 is shown for NONMEM.

# #' Use 'ddmore' function as.xpdb() to create an Xpose database object from
# #' the standardised output object, regardless of target software used for estimation.
# #' Users can then call xpose functions directly.  
# mlx.xpdb<-as.xpdb(mlx,datafile)
# #' We can then call Xpose functions referencing this mlx.xpdb object as the input. 
# #' Perform some basic goodness of fit (graphs are exported to PDF file)
# pdf("GOF_MLX.pdf")
# basic.gof(mlx.xpdb,by="DVID",subset="DVID==1")
# ind.plots(mlx.xpdb,groups="dvid")
# dev.off()
myXPDB <- merge(myEDAData, sapply(mlx@Estimation@Predictions$data, function(x) as.numeric(as.character(x))))
plot(ggplot() + ggtitle("Warfarin concentration") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==1,], aes(x=PRED, y=DV)) +
                xlab("Population predictions") + ylab("Observations") )

plot of chunk GOF for Monolix Estimation

plot(ggplot() + ggtitle("PCA level") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==2,], aes(x=PRED, y=DV)) +
                xlab("Population predictions") + ylab("Observations") )

plot of chunk GOF for Monolix Estimation

plot(ggplot() + ggtitle("Warfarin concentration") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==1,], aes(x=IPRED, y=DV)) +
                xlab("Individual predictions") + ylab("Observations") )

plot of chunk GOF for Monolix Estimation

plot(ggplot() + ggtitle("PCA level") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==2,], aes(x=IPRED, y=DV)) +
                xlab("Individual predictions") + ylab("Observations") )

plot of chunk GOF for Monolix Estimation

Export results to a PDF file

pdf("GOF_MLX.pdf")
plot(ggplot() + ggtitle("Warfarin concentration") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==1,], aes(x=PRED, y=DV)) +
                xlab("Population predictions") + ylab("Observations") )
plot(ggplot() + ggtitle("PCA level") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==2,], aes(x=PRED, y=DV)) +
                xlab("Population predictions") + ylab("Observations") )
plot(ggplot() + ggtitle("Warfarin concentration") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==1,], aes(x=IPRED, y=DV)) +
                xlab("Individual predictions") + ylab("Observations") )
plot(ggplot() + ggtitle("PCA level") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==2,], aes(x=IPRED, y=DV)) +
                xlab("Individual predictions") + ylab("Observations") )
dev.off()
## rj.GD 
##     2

SAEM Estimation with NONMEM

NM <- estimate(mdlfile, target="NONMEM", subfolder="NONMEM")
## -- Fri Dec 11 03:54:12 2015
## New
## Submitted
## Job 90ae2189-851e-4590-86e7-988b236e7f8d progress:
## Running [ ............................................................................ ]
## Importing Results
## Copying the result data back to the local machine for job ID 90ae2189-851e-4590-86e7-988b236e7f8d...
## From C:\Users\zparra\AppData\Local\Temp\Rtmpkfi8Jd\DDMORE.jobfd0194b87 to D:/SEE-1.2-SNAPSHOT-10DEC2015/MDL_IDE/workspace/UseCasesDemo/UseCase3/NONMEM
## Done.
## 
## 
## The following elements were parsed successfully:
##       RawResults
##       TaskInformation
##       Estimation:PopulationEstimates
##       Estimation:IndividualEstimates
##       Estimation:Residuals
##       Estimation:Predictions
##       Estimation:Likelihood
## 
## The following MESSAGEs were raised during the job execution:
##  minimization_successful: 1
##  covariance_step_run: 0
##  rounding_errors: 0
##  estimate_near_boundary: 0
##  s_matrix_singular: 0
##  nmoutput2so_version: This SOBlock was created with nmoutput2so version 4.5.11
## 
## Completed
## -- Fri Dec 11 04:19:39 2015

Load previous results NM <- LoadSOObject(“NONMEM/UseCase3.SO.xml”) Results from NONMEM should be comparable with results from MONOLIX

parameters_nm <- getPopulationParameters(NM, what="estimates")
print(getPopulationParameters(NM, what="estimates",block="structural"))
## $MLE
##     POP_CL      POP_V     POP_KA   POP_TLAG   POP_PCA0    POP_C50 
##  0.1349840  8.0012800  1.6895600  0.9757150 96.6301000  1.1704100 
##    POP_TEQ   RUV_PROP    RUV_ADD     RUV_FX   POP_EMAX BETA_CL_WT 
## 12.9179000  0.0589389  0.2718310  3.7974800  1.0000000  0.7500000 
##  BETA_V_WT 
##  1.0000000
prin(parameters_nm)
## Error in eval(expr, envir, enclos): could not find function "prin"
print(parameters_mlx)
##        POP_V    BETA_V_WT     POP_PCA0       POP_KA       POP_CL 
##      7.92329      1.00000     96.54103      1.45270      0.13633 
##   BETA_CL_WT      POP_TEQ      POP_C50     POP_EMAX     POP_TLAG 
##      0.75000     13.17685      1.15507      1.00000      0.95372 
##        PPV_V     PPV_PCA0       PPV_KA       PPV_CL      PPV_TEQ 
##      0.14805      0.05347      1.06406      0.25932      0.06318 
##      PPV_C50     PPV_EMAX     PPV_TLAG ETA_V_ETA_CL      RUV_ADD 
##      0.42122      0.00000      0.01000      0.17202      0.42894 
##     RUV_PROP       RUV_FX 
##      0.04595      3.81603

Covariance step cannot be requested in the current version of the Framework

#getPopulationParameters(NM, what="precisions") 

print(getEstimationInfo(NM))
## $Likelihood
## $Likelihood$Deviance
## [1] 288.7353
## 
## 
## $Messages
## $Messages$Info
## $Messages$Info$minimization_successful
## [1] "1"
## 
## $Messages$Info$covariance_step_run
## [1] "0"
## 
## $Messages$Info$rounding_errors
## [1] "0"
## 
## $Messages$Info$estimate_near_boundary
## [1] "0"
## 
## $Messages$Info$s_matrix_singular
## [1] "0"
## 
## $Messages$Info$nmoutput2so_version
## [1] "This SOBlock was created with nmoutput2so version 4.5.11"

Xpose diagnostics using NONMEM output

nm.xpdb<-as.xpdb(NM,datafile)
## 
## Removed dose rows in rawData+Predictions slot of SO to enable merge with Residuals data.
## 
## Residuals data does not currently contain dose rows in output from Nonmem executions.

Perform some basic goodness of fit (graphs are exported to PDF file)

print(basic.gof(nm.xpdb,subset="DVID==1"))

plot of chunk unnamed-chunk-22

print(basic.gof(nm.xpdb,subset="DVID==2"))

plot of chunk unnamed-chunk-22

print(ind.plots(nm.xpdb,subset="DVID==1"))

plot of chunk unnamed-chunk-22 plot of chunk unnamed-chunk-22

print(ind.plots(nm.xpdb,subset="DVID==2"))

plot of chunk unnamed-chunk-22 plot of chunk unnamed-chunk-22

Export results to a PDF file

pdf("GOF_NM.pdf")
 print(basic.gof(nm.xpdb,subset="DVID==1"))
 print(basic.gof(nm.xpdb,subset="DVID==2"))
 print(ind.plots(nm.xpdb,subset="DVID==1"))
 print(ind.plots(nm.xpdb,subset="DVID==2"))
dev.off()
## rj.GD 
##     2

Change estimation method to FOCEI (for speed)

MDL Objects can be manipulated from R to change for example the estimation algorithm

myTaskProperties <- getTaskPropertiesObjects(mdlfile)[[1]]
myNewTaskProperties <- myTaskProperties
myNewTaskProperties@ESTIMATE$algo <- "focei"

Assembling the new MOG. Note that we reuse the data and model from the previous run.

myNewerMOG <- createMogObj(dataObj = getDataObjects(mdlfile)[[1]], 
        parObj = getParameterObjects(mdlfile)[[1]], 
        mdlObj = getModelObjects(mdlfile)[[1]], 
        taskObj = myNewTaskProperties)

We can then write the MOG back out to an .mdl file.

mdlfile.FOCEI <- paste0(uc,"_FOCEI.mdl")
writeMogObj(myNewerMOG,mdlfile.FOCEI)

Test estimation using this new MOG.

NM.FOCEI <- estimate(mdlfile.FOCEI, target="PsN", subfolder="NONMEM_FOCEI")
## -- Fri Dec 11 04:20:27 2015
## New
## Submitted
## Job 60a66f40-504f-419b-8b3a-47755110f0b9 progress:
## Running [ .............. ]
## Importing Results
## Copying the result data back to the local machine for job ID 60a66f40-504f-419b-8b3a-47755110f0b9...
## From C:\Users\zparra\AppData\Local\Temp\Rtmpkfi8Jd\DDMORE.jobfd06d41672e to D:/SEE-1.2-SNAPSHOT-10DEC2015/MDL_IDE/workspace/UseCasesDemo/UseCase3/NONMEM_FOCEI
## Done.
## 
## 
## The following elements were parsed successfully:
##       RawResults
##       TaskInformation
##       Estimation:PopulationEstimates
##       Estimation:IndividualEstimates
##       Estimation:Residuals
##       Estimation:Predictions
##       Estimation:Likelihood
## 
## The following MESSAGEs were raised during the job execution:
##  minimization_successful: 1
##  covariance_step_run: 0
##  rounding_errors: 0
##  hessian_reset: 0
##  zero_gradients: 0
##  final_zero_gradients: 0
##  estimate_near_boundary: 0
##  s_matrix_singular: 0
##  significant_digits: 3.3
##  nmoutput2so_version: This SOBlock was created with nmoutput2so version 4.5.11
## 
## Completed
## -- Fri Dec 11 04:25:14 2015

Load previous results NM.FOCEI <- LoadSOObject(“NONMEM_FOCEI/UseCase3_FOCEI.SO.xml”) Results from NONMEM should be comparable to previous results

print(getPopulationParameters(NM.FOCEI,  what="estimates"))
## $MLE
##       POP_CL        POP_V       POP_KA     POP_TLAG     POP_PCA0 
##    0.1358490    7.9603300    1.2124600    0.8715500   96.6267000 
##      POP_C50      POP_TEQ     RUV_PROP      RUV_ADD       RUV_FX 
##    1.1690800   13.0684000    0.0571574    0.3543920    3.7549700 
##       PPV_CL ETA_CL_ETA_V        PPV_V       PPV_KA     PPV_PCA0 
##    0.2614470    0.2284570    0.1456670    0.8810330    0.0529037 
##      PPV_C50      PPV_TEQ     POP_EMAX   BETA_CL_WT    BETA_V_WT 
##    0.4392740    0.1020490    1.0000000    0.7500000    1.0000000 
##     PPV_TLAG     PPV_EMAX 
##    0.1000000    0.0000000
print(parameters_nm)
## $MLE
##       POP_CL        POP_V       POP_KA     POP_TLAG     POP_PCA0 
##    0.1349840    8.0012800    1.6895600    0.9757150   96.6301000 
##      POP_C50      POP_TEQ     RUV_PROP      RUV_ADD       RUV_FX 
##    1.1704100   12.9179000    0.0589389    0.2718310    3.7974800 
##       PPV_CL ETA_CL_ETA_V        PPV_V       PPV_KA     PPV_PCA0 
##    0.2677810    0.2388480    0.1454550    0.9999950    0.0544000 
##      PPV_C50      PPV_TEQ     POP_EMAX   BETA_CL_WT    BETA_V_WT 
##    0.4543790    0.1071420    1.0000000    0.7500000    1.0000000 
##     PPV_TLAG     PPV_EMAX 
##    0.1000000    0.0000000

Xpose diagnostics using NONMEM output

nmfocei.xpdb<-as.xpdb(NM.FOCEI,"warfarin_conc_pca.csv")
## 
## Removed dose rows in rawData+Predictions slot of SO to enable merge with Residuals data.
## 
## Residuals data does not currently contain dose rows in output from Nonmem executions.

Basic diagnostics for NONMEM fit.

pdf("GOF_NM_FOCEI.pdf")
 ind.plots(nmfocei.xpdb,subset="DVID==1",main="Individual plots (Warfarin concentration)")
 basic.gof(nmfocei.xpdb,subset="DVID==1",main="Goodness of fit (Warfarin concentration)")
 ind.plots(nmfocei.xpdb,subset="DVID==2",main="Individual plots (PCA level)")
 basic.gof(nmfocei.xpdb,subset="DVID==2",main="Goodness of fit (PCA level)")
 parm.hist(nmfocei.xpdb)
dev.off()
## rj.GD 
##     2

Run the bootstrap using PsN (takes several hours)

The ddmore “bootstrap.PsN” function is a wrap up function that calls Bootstrap PsN functionality using as input an MDL file that will be translated to NMTRAN as first step. Additional PsN arguments can be specified under the “bootstrapOptions” attribute. After task execution, the output from PsN is converted to a Standardised Output object which is saved in a .SO.xml file. Translated files and PsN output will be returned in the ./Bootstrap subfolder

bootstrapResults <- bootstrap.PsN(mdlfile.FOCEI, samples=20, seed=123456,
        bootstrapOptions=" -no-skip_minimization_terminated -threads=2",
        subfolder="Bootstrap", plot=TRUE)
## -- Fri Dec 11 04:25:33 2015
## New
## Submitted
## Job e7a8dcf4-7850-4823-9b90-924d13e306be progress:
## Running [ ............................................................................................................................................................... ]
## Importing Results
## Copying the result data back to the local machine for job ID e7a8dcf4-7850-4823-9b90-924d13e306be...
## From C:\Users\zparra\AppData\Local\Temp\Rtmpkfi8Jd\DDMORE.jobfd0402c631c to D:/SEE-1.2-SNAPSHOT-10DEC2015/MDL_IDE/workspace/UseCasesDemo/UseCase3/Bootstrap
## Done.
## 
## 
## The following elements were parsed successfully:
##       RawResults
##       TaskInformation
##       Estimation:PopulationEstimates
##       Estimation:PrecisionPopulationEstimates
##       Estimation:IndividualEstimates
##       Estimation:Residuals
##       Estimation:Predictions
##       Estimation:Likelihood
## 
## The following WARNINGs were raised during the job execution:
##  bootstrap_parameter_scale: The parameters PPV_CL, ETA_CL_ETA_V, PPV_V, PPV_KA, PPV_TLAG, PPV_PCA0, PPV_EMAX, PPV_C50 and PPV_TEQ were requested on the sd/corr scale but are given on the var/cov scale in all bootstrap results.
## 
## The following MESSAGEs were raised during the job execution:
##  minimization_successful: 1
##  covariance_step_run: 0
##  rounding_errors: 0
##  hessian_reset: 0
##  zero_gradients: 0
##  final_zero_gradients: 0
##  estimate_near_boundary: 0
##  s_matrix_singular: 0
##  significant_digits: 3.3
##  nmoutput2so_version: This SOBlock was created with nmoutput2so version 4.5.11
## 
## Completed
## -- Fri Dec 11 05:18:43 2015
## Warning: NAs introduced by coercion
## [[1]]

plot of chunk Bootstrap

## 
## [[2]]
## NULL
## 
## [[3]]

plot of chunk Bootstrap

## 
## [[4]]
## NULL
## 
## [[5]]
## NULL

Load results from a bootstrap previously performed bootstrapResults <- LoadSOObject(“Bootstrap/UseCase3_FOCEI.SO.xml”) Export bootstrap histograms to a pdf

pdf(paste0(uc,"_Bootstrap.pdf"))
 print(boot.hist(results.file = file.path("Bootstrap",paste0("raw_results_",uc,"_FOCEI.csv")),
                incl.ids.file = file.path("Bootstrap","included_individuals1.csv")))
## [[1]]
## 
## [[2]]
## NULL
## 
## [[3]]
## 
## [[4]]
## NULL
## 
## [[5]]
## NULL
dev.off()
## rj.GD 
##     2

Extract parameter estimates and precision from bootstrap results.

print(getPopulationParameters(bootstrapResults, what="estimates"))
## $MLE
##       POP_CL        POP_V       POP_KA     POP_TLAG     POP_PCA0 
##    0.1358490    7.9603300    1.2124600    0.8715500   96.6267000 
##      POP_C50      POP_TEQ     RUV_PROP      RUV_ADD       RUV_FX 
##    1.1690800   13.0684000    0.0571574    0.3543920    3.7549700 
##       PPV_CL ETA_CL_ETA_V        PPV_V       PPV_KA     PPV_PCA0 
##    0.2614470    0.2284570    0.1456670    0.8810330    0.0529037 
##      PPV_C50      PPV_TEQ     POP_EMAX   BETA_CL_WT    BETA_V_WT 
##    0.4392740    0.1020490    1.0000000    0.7500000    1.0000000 
##     PPV_TLAG     PPV_EMAX 
##    0.1000000    0.0000000 
## 
## $Bootstrap
##                 Parameter         Mean       Median
## POP_CL             POP_CL  0.137047600  0.137482500
## POP_V               POP_V  7.930065000  7.920835000
## POP_KA             POP_KA  1.569206000  1.353525000
## POP_TLAG         POP_TLAG  0.923327200  0.870910000
## POP_PCA0         POP_PCA0 96.450510000 96.290750000
## POP_C50           POP_C50  1.145913000  1.141605000
## POP_TEQ           POP_TEQ 13.067060000 13.083750000
## RUV_PROP         RUV_PROP  0.067744650  0.061786350
## RUV_ADD           RUV_ADD  0.315010200  0.370528000
## RUV_FX             RUV_FX  3.665427000  3.644740000
## POP_EMAX         POP_EMAX  1.000000000  1.000000000
## BETA_CL_WT     BETA_CL_WT  0.750000000  0.750000000
## BETA_V_WT       BETA_V_WT  1.000000000  1.000000000
## PPV_CL             PPV_CL  0.071250310  0.068391900
## ETA_CL_ETA_V ETA_CL_ETA_V  0.007102150  0.007152175
## PPV_V               PPV_V  0.017355460  0.017311800
## PPV_KA             PPV_KA  0.783082100  0.614757000
## PPV_TLAG         PPV_TLAG  0.010000000  0.010000000
## PPV_PCA0         PPV_PCA0  0.002916878  0.003054515
## PPV_EMAX         PPV_EMAX  0.000000000  0.000000000
## PPV_C50           PPV_C50  0.184644800  0.174678000
## PPV_TEQ           PPV_TEQ  0.010818520  0.009639125

Extract the information regarding the precision intervals

print(getPopulationParameters(bootstrapResults, what="intervals")$Bootstrap)
##       Parameter         Mean       Median        Perc_5      Perc_95
## 1    BETA_CL_WT  0.750000000  0.750000000  0.7500000000  0.750000000
## 2     BETA_V_WT  1.000000000  1.000000000  1.0000000000  1.000000000
## 3  ETA_CL_ETA_V  0.007102150  0.007152175 -0.0037175420  0.018796990
## 4       POP_C50  1.145913000  1.141605000  0.9936810000  1.298133000
## 5        POP_CL  0.137047600  0.137482500  0.1247593000  0.149671800
## 6      POP_EMAX  1.000000000  1.000000000  1.0000000000  1.000000000
## 7        POP_KA  1.569206000  1.353525000  0.6859680000  3.844895000
## 8      POP_PCA0 96.450510000 96.290750000 94.2281400000 99.001950000
## 9       POP_TEQ 13.067060000 13.083750000 12.4600100000 13.973450000
## 10     POP_TLAG  0.923327200  0.870910000  0.7539044000  1.458906000
## 11        POP_V  7.930065000  7.920835000  7.4689180000  8.327610000
## 12      PPV_C50  0.184644800  0.174678000  0.1157103000  0.264162000
## 13       PPV_CL  0.071250310  0.068391900  0.0347221000  0.120176000
## 14     PPV_EMAX  0.000000000  0.000000000  0.0000000000  0.000000000
## 15       PPV_KA  0.783082100  0.614757000  0.2237865000  2.312617000
## 16     PPV_PCA0  0.002916878  0.003054515  0.0007134338  0.004935598
## 17      PPV_TEQ  0.010818520  0.009639125  0.0014858640  0.022215080
## 18     PPV_TLAG  0.010000000  0.010000000  0.0100000000  0.010000000
## 19        PPV_V  0.017355460  0.017311800  0.0057963080  0.027671000
## 20      RUV_ADD  0.315010200  0.370528000  0.0030990000  0.499102400
## 21       RUV_FX  3.665427000  3.644740000  3.0351200000  4.061885000
## 22     RUV_PROP  0.067744650  0.061786350  0.0261761200  0.142483900

VPC of model

Before running the VPC with PsN we must update the (initial) values in the MDL Parameter Object MLE estimates from previous step can be used

structuralPar <- getPopulationParameters(NM.FOCEI, what="estimates",block='structural')$MLE
variabilityPar <- getPopulationParameters(NM.FOCEI, what="estimates",block='variability')$MLE

In the current version of the SO standard, we need to manually update parameter names for correlation and covariance parameters to match the SO with the MDL. This will not be needed in future releases. The SO object returned from NONMEM has parameter ETA_CL_ETA_V. This needs to be renamed to conform to model Correlation name OMEGA

variabilityNames <- names(myParObj@VARIABILITY)
names(variabilityPar)[names(variabilityPar)=="ETA_CL_ETA_V"] <- grep("OMEGA",variabilityNames,value=T)
names(variabilityPar)
## [1] "PPV_CL"   "OMEGA"    "PPV_V"    "PPV_KA"   "PPV_PCA0" "PPV_C50" 
## [7] "PPV_TEQ"  "PPV_TLAG" "PPV_EMAX"

Update the parameter object using the ddmore “updateParObj” function. This function updates an R object of (S4) class “parObj”. The user chooses which block to update, what items within that block, and what to replace those items with. NOTE: that updateParObj can only update attributes which ALREADY EXIST in the MDL Parameter Object for that item. This ensures that valid MDL is preserved.

myParObj <- getParameterObjects(mdlfile)[[1]]
myParObjUpdated <- updateParObj(myParObj,block="STRUCTURAL",
        item=names(structuralPar),
        with=list(value=structuralPar))
myParObjUpdated <- updateParObj(myParObjUpdated,block="VARIABILITY",
        item=names(variabilityPar),
        with=list(value=variabilityPar))

Check that the appropriate initial values have been updated to the MLE values from the previous fit.

print(myParObjUpdated@STRUCTURAL)
## $POP_CL
## $POP_CL$value
## [1] "0.135849"
## 
## $POP_CL$lo
## [1] "0.01"
## 
## $POP_CL$hi
## [1] "1"
## 
## 
## $POP_V
## $POP_V$value
## [1] "7.96033"
## 
## $POP_V$lo
## [1] "0.01"
## 
## $POP_V$hi
## [1] "20"
## 
## 
## $POP_KA
## $POP_KA$value
## [1] "1.21246"
## 
## $POP_KA$lo
## [1] "0.01"
## 
## $POP_KA$hi
## [1] "24"
## 
## 
## $POP_TLAG
## $POP_TLAG$value
## [1] "0.87155"
## 
## $POP_TLAG$lo
## [1] "0.01"
## 
## $POP_TLAG$hi
## [1] "24"
## 
## 
## $POP_PCA0
## $POP_PCA0$value
## [1] "96.6267"
## 
## $POP_PCA0$lo
## [1] "0.01"
## 
## $POP_PCA0$hi
## [1] "200"
## 
## 
## $POP_EMAX
## $POP_EMAX$value
## [1] "1"
## 
## $POP_EMAX$lo
## [1] "0"
## 
## $POP_EMAX$fix
## [1] "true"
## 
## 
## $POP_C50
## $POP_C50$value
## [1] "1.16908"
## 
## $POP_C50$lo
## [1] "0.01"
## 
## $POP_C50$hi
## [1] "10"
## 
## 
## $POP_TEQ
## $POP_TEQ$value
## [1] "13.0684"
## 
## $POP_TEQ$lo
## [1] "0.01"
## 
## $POP_TEQ$hi
## [1] "100"
## 
## 
## $BETA_CL_WT
## $BETA_CL_WT$value
## [1] "0.75"
## 
## $BETA_CL_WT$fix
## [1] "true"
## 
## 
## $BETA_V_WT
## $BETA_V_WT$value
## [1] "1"
## 
## $BETA_V_WT$fix
## [1] "true"
## 
## 
## $RUV_PROP
## $RUV_PROP$value
## [1] "0.0571574"
## 
## $RUV_PROP$lo
## [1] "0"
## 
## 
## $RUV_ADD
## $RUV_ADD$value
## [1] "0.354392"
## 
## $RUV_ADD$lo
## [1] "1.0E-4"
## 
## 
## $RUV_FX
## $RUV_FX$value
## [1] "3.75497"
## 
## $RUV_FX$lo
## [1] "0"
print(myParObjUpdated@VARIABILITY)
## $PPV_CL
## $PPV_CL$value
## [1] "0.261447"
## 
## $PPV_CL$type
## [1] "sd"
## 
## 
## $PPV_V
## $PPV_V$value
## [1] "0.145667"
## 
## $PPV_V$type
## [1] "sd"
## 
## 
## $PPV_KA
## $PPV_KA$value
## [1] "0.881033"
## 
## $PPV_KA$type
## [1] "sd"
## 
## 
## $PPV_TLAG
## $PPV_TLAG$value
## [1] "0.1"
## 
## $PPV_TLAG$type
## [1] "sd"
## 
## $PPV_TLAG$fix
## [1] "true"
## 
## 
## $PPV_PCA0
## $PPV_PCA0$value
## [1] "0.0529037"
## 
## $PPV_PCA0$type
## [1] "sd"
## 
## 
## $PPV_EMAX
## $PPV_EMAX$value
## [1] "0"
## 
## $PPV_EMAX$type
## [1] "sd"
## 
## $PPV_EMAX$fix
## [1] "true"
## 
## 
## $PPV_C50
## $PPV_C50$value
## [1] "0.439274"
## 
## $PPV_C50$type
## [1] "sd"
## 
## 
## $PPV_TEQ
## $PPV_TEQ$value
## [1] "0.102049"
## 
## $PPV_TEQ$type
## [1] "sd"
## 
## 
## $OMEGA
## $OMEGA$parameter
## [1] "[ETA_CL,ETA_V]"
## 
## $OMEGA$value
## [1] "0.228457"
## 
## $OMEGA$type
## [1] "corr"

A bug in the writeMogObj function means that for now, we must manually add the square bracket around the OMEGA value to signify that this is a vector (of length 1).

myParObjUpdated@VARIABILITY$OMEGA$value<-paste0("[",myParObjUpdated@VARIABILITY$OMEGA$value,"]")

Assembling the new MOG. Note that we reuse the data, model and tasks from the previous run.

myVPCMOG <- createMogObj(dataObj = getDataObjects(mdlfile)[[1]], 
        parObj = myParObjUpdated, 
        mdlObj = getModelObjects(mdlfile)[[1]], 
        taskObj = getTaskPropertiesObjects(mdlfile)[[1]])

We can then write the MOG back out to an .mdl file.

mdlfile.VPC <- paste0(uc,"_VPC.mdl")
writeMogObj(myVPCMOG,mdlfile.VPC)

Similarly as above, ddmore “VPC.PsN” function can be used to run a VPC using PsN as target tool

vpcFiles <- VPC.PsN(mdlfile.VPC,samples=20, seed=12345,
        vpcOptions ="-n_simulation=10 -stratify_on=DVID -auto_bin=7,10:5,8",
        subfolder="VPC", plot=FALSE) 
## -- Fri Dec 11 05:19:08 2015
## New
## Submitted
## Job b39eebe0-563b-44be-9d43-4278b1d2b948 progress:
## Running [ ..... ]
## Importing Results
## Copying the result data back to the local machine for job ID b39eebe0-563b-44be-9d43-4278b1d2b948...
## From C:\Users\zparra\AppData\Local\Temp\Rtmpkfi8Jd\DDMORE.jobfd0632c6e95 to D:/SEE-1.2-SNAPSHOT-10DEC2015/MDL_IDE/workspace/UseCasesDemo/UseCase3/VPC
## Done.
## 
## 
## The following elements were parsed successfully:
##       RawResults
##       TaskInformation
##       Simulation
## 
## The following MESSAGEs were raised during the job execution:
##  nmoutput2so_version: This SOBlock was created with nmoutput2so version 4.5.11
## 
## Completed
## -- Fri Dec 11 05:20:53 2015

To replay the visualisation using information from the VPC SO file

pdf(paste0(uc,"_VPC.pdf"))
 print(xpose.VPC(vpc.info= file.path("./VPC",vpcFiles@RawResults@DataFiles$PsN_VPC_results$path),
        vpctab= file.path("./VPC",vpcFiles@RawResults@DataFiles$PsN_VPC_vpctab$path),
        main="VPC warfarin PK/PD"))
dev.off()
## rj.GD 
##     2

Simulation using simulx

The mlxR package has been developed to visualize and explore models that are encoded in MLXTRAN or PharmML The ddmore function as.PharmML translates an MDL file (extension .mdl) to its PharmML representation. The output file (extension .xml) is saved in the working directory.

myPharmML <- as.PharmML(mdlfile)

Use parameter values from the FOCEI estimation

parValues <- getPopulationParameters(NM.FOCEI, what="estimates")$MLE

Simulate for the typical weight of 70. Recall that logtWT = log(WT/70).

p <- c(parValues,WT=70)

Parameter values used in simulation

print(p)
##       POP_CL        POP_V       POP_KA     POP_TLAG     POP_PCA0 
##    0.1358490    7.9603300    1.2124600    0.8715500   96.6267000 
##      POP_C50      POP_TEQ     RUV_PROP      RUV_ADD       RUV_FX 
##    1.1690800   13.0684000    0.0571574    0.3543920    3.7549700 
##       PPV_CL ETA_CL_ETA_V        PPV_V       PPV_KA     PPV_PCA0 
##    0.2614470    0.2284570    0.1456670    0.8810330    0.0529037 
##      PPV_C50      PPV_TEQ     POP_EMAX   BETA_CL_WT    BETA_V_WT 
##    0.4392740    0.1020490    1.0000000    0.7500000    1.0000000 
##     PPV_TLAG     PPV_EMAX           WT 
##    0.1000000    0.0000000   70.0000000

Simulate for a dose of 100mg given at time 0 into the GUT (oral administration)

adm <- list(target='GUT', time = 0, amount = 100)

Simulate PK parameters for individuals

ind <- list(name = c('TLAG','KA','CL','V','TEQ','C50','PCA0','EMAX','RUV_ADD','RUV_PROP'))

Simulate predicted (CC) and observed concentration values (CP_obs), predicted (PCA) and observed PCA (PCA_obs)

f1   <- list( name = c('CC'), time = seq(0,to=50,by=1))
y1   <- list( name = c('CP_obs'), time = c(0, 0.5, 1, 2, 3, 4, 6, 8, 12, 24, 36, 48))
f2   <- list( name = c('PCA'), time = seq(0,to=50,by=1))
y2   <- list( name = c('PCA_obs'), time = c(0, 0.5, 1, 2, 3, 4, 6, 8, 12, 24, 36, 48))

Simulate 12 subjects

g <- list( size = 12, level = 'individual',  treatment = adm)

Call simulx

res  <- simulx(model = myPharmML,
               parameter = p,
               group = g,
               output = list(ind,f1,y1,f2,y2))

Simulated parameter values for each individual

print(res$parameter)
##    id      TLAG        KA         CL         V      TEQ       C50
## 1   1 0.9390741 3.2136590 0.15959040  8.707718 11.39175 0.9207872
## 2   2 0.8422733 1.1029870 0.11030410  7.088070 14.44512 0.8348897
## 3   3 0.8775504 1.2299001 0.13609051  7.968212 12.45376 0.8958099
## 4   4 0.9394945 1.3169485 0.14968116  8.402206 12.69506 0.3783062
## 5   5 0.9306461 0.4970874 0.23372535 10.770224 13.10528 1.3938582
## 6   6 0.9112111 1.3498420 0.06911586  5.462826 14.09802 1.4817312
## 7   7 0.8476004 7.0209773 0.17050939  9.034787 11.54422 1.3555894
## 8   8 0.8350809 0.8540427 0.16498140  8.870399 12.06297 0.8458807
## 9   9 1.0466890 1.8511792 0.11706498  7.326935 12.81070 0.9213280
## 10 10 0.7703135 0.5560324 0.12438234  7.578674 11.32738 0.7337858
## 11 11 0.8922969 1.9446077 0.11767241  7.348093 12.11423 1.0363576
## 12 12 0.9336776 3.3269154 0.27503063 11.792393 13.20521 2.2006066
##         PCA0 EMAX  RUV_ADD  RUV_PROP
## 1   97.47892    1 0.354392 0.0571574
## 2  102.10275    1 0.354392 0.0571574
## 3   98.51237    1 0.354392 0.0571574
## 4   98.50812    1 0.354392 0.0571574
## 5   98.26013    1 0.354392 0.0571574
## 6   94.20892    1 0.354392 0.0571574
## 7   92.38522    1 0.354392 0.0571574
## 8  108.38262    1 0.354392 0.0571574
## 9   93.42404    1 0.354392 0.0571574
## 10  91.24791    1 0.354392 0.0571574
## 11  94.15236    1 0.354392 0.0571574
## 12  85.83057    1 0.354392 0.0571574

Plot simulated results

plot(ggplot() + 
                geom_line(data=res$CC, aes(x=time, y=CC, colour=id)) +
                geom_point(data=res$CP_obs, aes(x=time, y=CP_obs, colour=id)) +
                xlab("time (h)") + ylab("concentration") )

plot of chunk unnamed-chunk-47

plot(ggplot() + 
                geom_line(data=res$PCA, aes(x=time, y=PCA, colour=id)) +
                geom_point(data=res$PCA_obs, aes(x=time, y=PCA_obs, colour=id)) +
                xlab("time (h)") + ylab("PCA") )

plot of chunk unnamed-chunk-47

Simulate 1000 subjects - with simulx this is a QUICK process!

g <- list( size = 1000, level = 'individual',  treatment = adm)

Call simulx

res.1000  <- simulx(model =myPharmML,
                    parameter = p,
                    group = g,
                    output = list(ind,f1,y1,f2,y2))

Plot of predicted concentrations band defines the percentile bands displayed level = range of values to examine (in %) 100 = full range of values number = number of bins within the level range. Plot of observed concentrations (with residual error)

plot(prctilemlx(res.1000$CP_obs,band=list(number=9, level=90)))

plot of chunk unnamed-chunk-49

Table of the same information

print(prctilemlx(res.1000$CP_obs,band=list(number=10, level=100), plot=F)$y)
##    time         0%        10%        20%        30%        40%         50%
## 1   0.0 -0.7943102 -0.5411631 -0.4030056 -0.2588491 -0.1651892 -0.04301055
## 2   0.5 -1.1407029 -0.5121234 -0.3701462 -0.2840363 -0.1192569 -0.02390538
## 3   1.0 -0.4969948  0.2428773  0.5532469  0.9832390  1.4152406  1.78935726
## 4   2.0  0.7878573  3.4787804  5.5923146  7.1146174  7.7628533  8.42971716
## 5   3.0  1.9357840  5.6132992  8.1478567  9.1851813 10.1617389 11.01166664
## 6   4.0  3.2902030  7.4059393  8.8475207  9.8911873 10.4074092 11.17976184
## 7   6.0  4.4749049  8.6041339  9.8332233 10.4012872 10.8167300 11.49067973
## 8   8.0  6.6236774  8.9132697  9.6915842 10.1654226 10.6028932 11.24245659
## 9  12.0  5.7057313  8.3684366  9.3124787  9.7994092 10.3293659 10.91458845
## 10 24.0  5.6859355  6.8664061  7.3834959  8.0675386  8.5373697  8.94585642
## 11 36.0  3.9862815  5.7019753  6.3906805  6.6150657  6.9258977  7.30970224
## 12 48.0  2.9712512  4.4675605  4.9727326  5.4130295  5.5351355  5.97223927
##            50%         60%        70%        80%        90%       100%
## 1  -0.04301055  0.02930436  0.1129057  0.2378287  0.4918361  0.9509325
## 2  -0.02390538  0.06605478  0.1954569  0.2801420  0.4177937  0.7934998
## 3   1.78935726  2.27542874  2.8779963  3.4288642  5.4544104 10.2749696
## 4   8.42971716 10.08341707 10.8937459 11.7556072 13.0623037 15.5156744
## 5  11.01166664 11.58803539 12.7289988 13.4198577 14.3906201 17.6131805
## 6  11.17976184 11.72414935 12.2096197 13.0564194 14.4133570 17.3102667
## 7  11.49067973 12.06895402 12.6457645 13.4482203 14.1068994 16.2003516
## 8  11.24245659 11.89256809 12.5693496 13.2304906 14.8164367 18.5109838
## 9  10.91458845 11.46175028 11.9754483 12.6353011 13.7303985 16.1306043
## 10  8.94585642  9.68013085 10.1072668 10.9094926 12.1408460 15.4106155
## 11  7.30970224  7.73993028  8.3209238  9.2344638 10.0653038 11.4158105
## 12  5.97223927  6.41374640  7.0223320  7.5909294  8.1644975 10.1699796

Plot of observed PCA levels (with residual error)

prctilemlx(res.1000$PCA_obs,band=list(number=9, level=90))

plot of chunk unnamed-chunk-51

Table of the same information

prctilemlx(res.1000$PCA_obs,band=list(number=10, level=100), plot=F)$y
##    time        0%      10%      20%      30%      40%      50%      50%
## 1   0.0 78.886681 87.82899 90.90291 92.62164 93.97363 95.73894 95.73894
## 2   0.5 81.139403 87.84769 90.46926 93.26250 94.81464 96.09246 96.09246
## 3   1.0 83.004743 88.41841 90.89430 93.09540 94.14955 95.82519 95.82519
## 4   2.0 71.499541 84.90606 86.31837 88.14775 90.13779 92.28487 92.28487
## 5   3.0 73.809634 79.70562 82.47656 84.64605 86.55429 87.80066 87.80066
## 6   4.0 71.746081 76.24626 78.37131 79.87106 82.22602 83.81598 83.81598
## 7   6.0 60.046815 69.53938 71.24001 73.41818 74.94483 75.82286 75.82286
## 8   8.0 53.639832 59.93491 64.04692 65.90527 68.07903 69.50034 69.50034
## 9  12.0 44.979350 49.94768 52.88275 54.53808 55.16919 56.69634 56.69634
## 10 24.0 22.404738 27.60070 30.23495 32.23561 34.11376 35.99847 35.99847
## 11 36.0 12.173463 16.96826 20.11554 22.16172 23.95749 25.49071 25.49071
## 12 48.0  6.479997 11.37846 13.77394 15.96643 17.62889 19.05863 19.05863
##         60%       70%       80%       90%      100%
## 1  97.65994  99.60682 102.96546 105.66851 116.08861
## 2  97.29237 100.08916 103.29404 106.56176 109.37240
## 3  98.17188  99.99052 102.34054 104.18541 109.11324
## 4  93.76701  95.47013  97.91977 101.14859 106.86414
## 5  88.78722  90.27454  91.99684  94.27959 100.28890
## 6  85.11681  86.64643  88.03768  91.55541  98.21299
## 7  77.20712  79.64653  82.36513  84.43030  90.02125
## 8  70.93514  72.90443  74.77549  77.79630  85.67809
## 9  58.39634  60.47409  62.95756  65.71264  72.57540
## 10 37.84091  38.69796  41.23929  44.36509  55.40904
## 11 27.12519  28.37454  30.66158  34.71703  48.49802
## 12 20.27684  22.90872  25.62683  30.05279  43.59824