Guided-demo: Exploring 'ddmore' R functionality with the warfarin model

UseCase3 : PKPD model for warfarin population pharmacokinetics and PCA

This example script is intended to illustrate how to use the 'ddmore' R package to perform a M&S workflow using the DDMoRe Standalone Execution Environment (SEE).

The following steps are implemented in this workflow:

To run a task, select with the cursor any code lines you wish to execute and press CTRL+R+R in your keyboard. An HTML file containing the commands in this file and associated output will be provided to allow the user to compare the results

Initialisation

Clear workspace and set working directory under 'models' folder

Clear workspace and set working directory under 'UsesCasesDemo' project

rm(list=ls(all=FALSE))
mydir <- file.path(Sys.getenv("MDLIDE_WORKSPACE_HOME"),"UseCasesDemo")
setwd(mydir)

Set name of .mdl file and dataset for future tasks

uc <- "UseCase3"
datafile <- "warfarin_conc_pca.csv"
mdlfile <- paste0(uc,".mdl")

Create a new folder to be used as working directory and where results will be stored

wd <- file.path(mydir,uc)
dir.create(wd)
## Warning in dir.create(wd):
## 'C:\SEE\MDL_IDE\workspace\UseCasesDemo\UseCase3' already exists

Copy the dataset and the .mdl file available under “models” into the working directory

file.copy(file.path(mydir,"models", datafile),wd)
## [1] TRUE
file.copy(file.path(mydir,"models",mdlfile),wd)
## [1] TRUE

Set the working directory.

setwd(file.path(mydir,uc))

The working directory needs to be set differently when knitr R package is used to spin the file

library(knitr)
opts_knit$set(root.dir = file.path(mydir,uc)) 

List files available in working directory

list.files()
##  [1] "Bootstrap"              "GOF_MLX.pdf"           
##  [3] "GOF_NM.pdf"             "GOF_NM_FOCEI.pdf"      
##  [5] "Monolix"                "NONMEM"                
##  [7] "NONMEM_FOCEI"           "UseCase3.mdl"          
##  [9] "UseCase3.xml"           "UseCase3_Bootstrap.pdf"
## [11] "UseCase3_EGA.pdf"       "UseCase3_FOCEI.mdl"    
## [13] "UseCase3_tr.txt"        "UseCase3_VPC.mdl"      
## [15] "UseCase3_VPC.pdf"       "VPC"                   
## [17] "warfarin_conc_pca.csv"

Introduction to 'ddmore' R package

View objects within the .mdl file

Use 'ddmore' function getModelObjects() to retrieve model object(s) from an existing .mdl file. This function reads the MDL in an .mdl file and parses the MDL code for each MDL Object into an R list of objects of appropriate types with names corresponding to the MDL Object names given in the file.

myMDLObj <- getMDLObjects(mdlfile)
length(myMDLObj)
## [1] 4
names(myMDLObj)
## [1] "warfarin_PKPD_turnover_dat"  "warfarin_PKPD_turnover_par" 
## [3] "warfarin_PKPD_turnover_mdl"  "warfarin_PKPD_turnover_task"

Use 'ddmore' function getDataObjects() to retrieve only data object(s) from an existing .mdl file. This function returns a list of Parameter Object(s) from which we select the first element. Hoover over the variable name to see its structure

myDataObj <- getDataObjects(mdlfile)[[1]]

Use 'ddmore' function getParameterObjects() to retrieve only parameter object(s) from an existing .mdl file

myParObj <- getParameterObjects(mdlfile)[[1]]

Use 'ddmore' function getModelObjects() to retrieve only model object(s) from an existing .mdl file.

myModObj <- getModelObjects(mdlfile)[[1]]

Use 'ddmore' function getTaskPropertiesObjects() to retrieve only task properties object(s) from an existing .mdl file

myTaskObj <- getTaskPropertiesObjects(mdlfile)[[1]]

Exploratory Data Analysis

Recall that getDataObjects only reads the MDL code from the .mdl file. Use 'ddmore' function readDataObj() to create an R object from the MDL data object.

myData <- readDataObj(myDataObj)

Let's look at the first 6 lines of the data set

head(myData)
##   ID TIME   WT AGE  SEX AMT DVID  DV MDV
## 1  1  0.0 66.7  50 male 100    0  NA   1
## 2  1  0.0 66.7  50 male  NA    2  NA   1
## 3  1  0.5 66.7  50 male  NA    1 0.0   0
## 4  1  1.0 66.7  50 male  NA    1 1.9   0
## 5  1  2.0 66.7  50 male  NA    1 3.3   0
## 6  1  3.0 66.7  50 male  NA    1 6.6   0

Extract only observation records

myEDAData<-myData[myData$MDV==0,]

Open an R window to record and access all your plots

windows(record=TRUE) 

Now plot the data using xyplot from the lattice library

plot1 <- xyplot(DV~TIME,groups=ID,data=myEDAData,subset=DVID==1,type="b",ylab="Conc. (mg/L)",xlab="Time (h)")
print(plot1)

plot of chunk unnamed-chunk-15

plot2 <- xyplot(DV~TIME|ID,data=myEDAData,subset=DVID==1,type="b",layout=c(3,4),ylab="Conc. (mg/L)",xlab="Time (h)",scales=list(relation="free"))
print(plot2)

plot of chunk unnamed-chunk-15 plot of chunk unnamed-chunk-15 plot of chunk unnamed-chunk-15

plot3 <- xyplot(DV~TIME,groups=ID,data=myEDAData,subset=DVID==2,type="b",ylab="PCA",xlab="Time (h)")
print(plot3)

plot of chunk unnamed-chunk-15

plot4 <- xyplot(DV~TIME|ID,data=myEDAData,subset=DVID==2,type="b",layout=c(3,4),ylab="PCA",xlab="Time (h)",scales=list(relation="free"))
print(plot4)

plot of chunk unnamed-chunk-15 plot of chunk unnamed-chunk-15 plot of chunk unnamed-chunk-15

Export the results in a PDF file

pdf(paste0(uc,"_EGA.pdf"))
 print(plot1)
 print(plot2)
 print(plot3)
 print(plot4)
dev.off()
## png 
##   2

Model Development

ESTIMATE model parameters using Monolix

The ddmore “estimate” function translates the contents of the .mdl file to a target language and then estimates parameters using the target software. After estimation, the output is converted to a Standardised Output object which is saved in a .SO.xml file.

Translated files and Monolix output will be returned in the ./Monolix subfolder. The Standardised Output object (.SO.xml) is read and parsed into an R object called “mlx” of (S4) class “StandardOutputObject”.

mlx <- estimate(mdlfile, target="MONOLIX", subfolder="Monolix")
## -- Wed Aug 17 09:15:24 2016
## New
## Submitted
## Job 6aca1815-7e6f-4b9f-a4bf-b3c9ba450f15 progress:
## Running [ ........... ]
## Importing Results
## Copying the result data back to the local machine for job ID 6aca1815-7e6f-4b9f-a4bf-b3c9ba450f15...
## From C:\Users\smith_mk\AppData\Local\Temp\4\RtmpYF4SFO\DDMORE.job35645c067db0 to C:/SEE/MDL_IDE/workspace/UseCasesDemo/UseCase3/Monolix
## Done.
## 
## 
## The following main elements were parsed successfully:
##   ToolSettings
##   RawResults
##   Estimation::PopulationEstimates::MLE
##   Estimation::PrecisionPopulationEstimates::MLE
##   Estimation::IndividualEstimates::Estimates
##   Estimation::IndividualEstimates::RandomEffects
##   Estimation::Residuals::ResidualTable
##   Estimation::Predictions
##   Estimation::OFMeasures::IndividualContribToLL
##   Estimation::OFMeasures::InformationCriteria
##   Estimation::OFMeasures::LogLikelihood
## 
## Completed
## -- Wed Aug 17 09:19:13 2016
slotNames(mlx)
## [1] "ToolSettings"     "RawResults"       "TaskInformation" 
## [4] "Estimation"       "ModelDiagnostic"  "Simulation"      
## [7] "OptimalDesign"    ".pathToSourceXML"

The ddmore “LoadSOObj” reads and parsed existing Standardise Output Objects

mlx <- LoadSOObject(“Monolix/UseCase3.SO.xml”)

The ddmore function “getPopulationParameters” extracts the Population Parameter values from an R object of (S4) class “StandardOutputObject” and returns the estimates. See documentation for getPopulationParameters to see other arguments and settings for this function.

parameters_mlx <- getPopulationParameters(mlx, what="estimates")$MLE
print(parameters_mlx)
##      POP_V  BETA_V_WT   POP_PCA0     POP_KA     POP_CL BETA_CL_WT 
##    7.98458    1.00000   96.75179    1.49863    0.13491    0.75000 
##    POP_TEQ    POP_C50   POP_EMAX   POP_TLAG      PPV_V   PPV_PCA0 
##   12.93442    1.17894    1.00000    0.94859    0.14295    0.05327 
##     PPV_KA     PPV_CL    PPV_TEQ    PPV_C50   PPV_EMAX   PPV_TLAG 
##    0.91570    0.26079    0.09618    0.43003    0.00000    0.10000 
##  CORR_CL_V    RUV_ADD   RUV_PROP     RUV_FX 
##    0.25136    0.26460    0.06113    3.83279
print(getPopulationParameters(mlx, what="precisions"))
## $MLE
##     Parameter      MLE      SE   RSE
## 1  BETA_CL_WT  0.75000 0.00000  0.00
## 2   BETA_V_WT  1.00000 0.00000  0.00
## 3   CORR_CL_V  0.25136 0.19164 76.24
## 4     POP_C50  1.17894 0.09633  8.17
## 5      POP_CL  0.13491 0.00635  4.71
## 6    POP_EMAX  1.00000 0.00000  0.00
## 7      POP_KA  1.49863 0.45379 30.28
## 8    POP_PCA0 96.75179 1.14680  1.19
## 9     POP_TEQ 12.93442 0.32448  2.51
## 10   POP_TLAG  0.94859 0.05363  5.65
## 11      POP_V  7.98458 0.22842  2.86
## 12    PPV_C50  0.43003 0.06087 14.15
## 13     PPV_CL  0.26079 0.03389 13.00
## 14   PPV_EMAX  0.00000 0.00000  0.00
## 15     PPV_KA  0.91570 0.22958 25.07
## 16   PPV_PCA0  0.05327 0.01043 19.58
## 17    PPV_TEQ  0.09618 0.02556 26.58
## 18   PPV_TLAG  0.10000 0.00000  0.00
## 19      PPV_V  0.14295 0.02259 15.80
## 20    RUV_ADD  0.26460 0.04758 17.98
## 21     RUV_FX  3.83279 0.23353  6.09
## 22   RUV_PROP  0.06113 0.00948 15.51
print(getEstimationInfo(mlx))
## $OFMeasures
## $OFMeasures$LogLikelihood
## $OFMeasures$LogLikelihood[[1]]
## [1] -1056.23
## 
## 
## $OFMeasures$IndividualContribToLL
##    Subject ICtoLL
## 1        1 -48.67
## 2        2 -26.56
## 3        3 -44.31
## 4        4 -35.73
## 5        5 -38.11
## 6        6 -30.87
## 7        7 -43.83
## 8        8 -44.56
## 9        9 -54.85
## 10      10 -28.10
## 11      12 -41.71
## 12      13 -33.04
## 13      14 -44.36
## 14      15 -35.24
## 15      16 -39.16
## 16      17 -29.81
## 17      18 -24.23
## 18      19 -26.13
## 19      20 -27.10
## 20      21 -25.64
## 21      22 -29.29
## 22      23 -30.99
## 23      24 -24.00
## 24      25 -39.25
## 25      26 -26.21
## 26      27 -24.70
## 27      28 -29.80
## 28      29 -25.05
## 29      30 -27.35
## 30      31 -26.00
## 31      32 -25.80
## 32      33 -25.77
## 
## $OFMeasures$InformationCriteria
## $OFMeasures$InformationCriteria$AIC
## [1] 2146.46
## 
## $OFMeasures$InformationCriteria$BIC
## [1] 2171.37
## 
## 
## 
## $Messages
## list()

Perform model diagnostics for the base model using Xpose functions (graphs are exported to PDF)

There is currently a bug with as.xpdb and Monolix in UseCase3. Therefore, goodness-of-fit plots are created manually from the standardised output. Later in the script, Xpose functionality on UseCase3 is shown for NONMEM.

# #' Use 'ddmore' function as.xpdb() to create an Xpose database object from
# #' the standardised output object, regardless of target software used for estimation.
# #' Users can then call xpose functions directly.  
# mlx.xpdb<-as.xpdb(mlx,datafile)
# #' We can then call Xpose functions referencing this mlx.xpdb object as the input. 
# #' Perform some basic goodness of fit (graphs are exported to PDF file)
# pdf("GOF_MLX.pdf")
#  basic.gof(mlx.xpdb,by="DVID",subset="DVID==1")
#  basic.gof(mlx.xpdb,by="DVID",subset="DVID==2")
#  ind.plots(mlx.xpdb,subset="DVID==1")
#  ind.plots(mlx.xpdb,subset="DVID==2")
# dev.off()
myXPDB <- merge(myEDAData, apply(mlx@Estimation@Predictions@data, 2, function(x) as.numeric(x)))

plot(ggplot() + ggtitle("Warfarin concentration") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==1,], aes(x=PRED, y=DV)) +
                xlab("Population predictions") + ylab("Observations") )

plot of chunk GOF for Monolix Estimation

plot(ggplot() + ggtitle("PCA level") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==2,], aes(x=PRED, y=DV)) +
                xlab("Population predictions") + ylab("Observations") )

plot of chunk GOF for Monolix Estimation

plot(ggplot() + ggtitle("Warfarin concentration") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==1,], aes(x=IPRED, y=DV)) +
                xlab("Individual predictions") + ylab("Observations") )

plot of chunk GOF for Monolix Estimation

plot(ggplot() + ggtitle("PCA level") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==2,], aes(x=IPRED, y=DV)) +
                xlab("Individual predictions") + ylab("Observations") )

plot of chunk GOF for Monolix Estimation

Export results to a PDF file

pdf("GOF_MLX.pdf")
 plot(ggplot() + ggtitle("Warfarin concentration") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==1,], aes(x=PRED, y=DV)) +
                xlab("Population predictions") + ylab("Observations") )
plot(ggplot() + ggtitle("PCA level") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==2,], aes(x=PRED, y=DV)) +
                xlab("Population predictions") + ylab("Observations") )
 plot(ggplot() + ggtitle("Warfarin concentration") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==1,], aes(x=IPRED, y=DV)) +
                xlab("Individual predictions") + ylab("Observations") )
 plot(ggplot() + ggtitle("PCA level") +
                geom_abline() +
                geom_point(data=myXPDB[myXPDB$DVID==2,], aes(x=IPRED, y=DV)) +
                xlab("Individual predictions") + ylab("Observations") )
 dev.off()
## png 
##   2

SAEM Estimation with NONMEM

By default, a covariance step is not run when estimating in NONMEM. To see how it can be requested, see UseCase1_1.mdl

NM <- estimate(mdlfile, target="NONMEM", subfolder="NONMEM")
## -- Wed Aug 17 09:19:18 2016
## New
## Submitted
## Job 35ddf9c6-8834-4b41-af8f-10667d762f5c progress:
## Running [ ....................................................................................................... ]
## Importing Results
## Copying the result data back to the local machine for job ID 35ddf9c6-8834-4b41-af8f-10667d762f5c...
## From C:\Users\smith_mk\AppData\Local\Temp\4\RtmpYF4SFO\DDMORE.job35641adc6837 to C:/SEE/MDL_IDE/workspace/UseCasesDemo/UseCase3/NONMEM
## Done.
## 
## 
## The following main elements were parsed successfully:
##   RawResults
##   Estimation::PopulationEstimates::MLE
##   Estimation::IndividualEstimates::Estimates
##   Estimation::IndividualEstimates::RandomEffects
##   Estimation::Residuals::ResidualTable
##   Estimation::Predictions
##   Estimation::OFMeasures::Deviance
## 
## The following MESSAGEs were raised during the job execution:
##  estimation_successful: 1
##  covariance_step_run: 0
##  rounding_errors: 0
##  estimate_near_boundary: 0
##  s_matrix_singular: 0
##  nmoutput2so_version: This SOBlock was created with nmoutput2so version 4.5.27
## 
## Completed
## -- Wed Aug 17 09:53:46 2016

Load previous results

NM <- LoadSOObject(“NONMEM/UseCase3.SO.xml”)

Results from NONMEM should be comparable with results from MONOLIX

parameters_nm <- getPopulationParameters(NM, what="estimates")
print(getPopulationParameters(NM, what="estimates",block="structural"))
## $MLE
##     POP_CL      POP_V     POP_KA   POP_TLAG   POP_PCA0    POP_C50 
##   0.134944   8.002120   1.501590   0.971861  96.589100   1.172950 
##    POP_TEQ   POP_EMAX BETA_CL_WT  BETA_V_WT 
##  12.921600   1.000000   0.750000   1.000000
print(parameters_nm)
## $MLE
##     POP_CL      POP_V     POP_KA   POP_TLAG   POP_PCA0    POP_C50 
##  0.1349440  8.0021200  1.5015900  0.9718610 96.5891000  1.1729500 
##    POP_TEQ   RUV_PROP    RUV_ADD     RUV_FX   POP_EMAX BETA_CL_WT 
## 12.9216000  0.0601880  0.2653420  3.7856500  1.0000000  0.7500000 
##  BETA_V_WT     PPV_CL  CORR_CL_V      PPV_V     PPV_KA   PPV_TLAG 
##  1.0000000  0.2675610  0.2289230  0.1452610  0.9031980  0.1000000 
##   PPV_PCA0   PPV_EMAX    PPV_C50    PPV_TEQ 
##  0.0542843  0.0000000  0.4534200  0.1066040
print(parameters_mlx)
##      POP_V  BETA_V_WT   POP_PCA0     POP_KA     POP_CL BETA_CL_WT 
##    7.98458    1.00000   96.75179    1.49863    0.13491    0.75000 
##    POP_TEQ    POP_C50   POP_EMAX   POP_TLAG      PPV_V   PPV_PCA0 
##   12.93442    1.17894    1.00000    0.94859    0.14295    0.05327 
##     PPV_KA     PPV_CL    PPV_TEQ    PPV_C50   PPV_EMAX   PPV_TLAG 
##    0.91570    0.26079    0.09618    0.43003    0.00000    0.10000 
##  CORR_CL_V    RUV_ADD   RUV_PROP     RUV_FX 
##    0.25136    0.26460    0.06113    3.83279
print(getEstimationInfo(NM))
## $OFMeasures
## $OFMeasures$Deviance
## $OFMeasures$Deviance[[1]]
## [1] 281.5268
## 
## 
## 
## $Messages
## $Messages$Info
## $Messages$Info$estimation_successful
## [1] "1"
## 
## $Messages$Info$covariance_step_run
## [1] "0"
## 
## $Messages$Info$rounding_errors
## [1] "0"
## 
## $Messages$Info$estimate_near_boundary
## [1] "0"
## 
## $Messages$Info$s_matrix_singular
## [1] "0"
## 
## $Messages$Info$nmoutput2so_version
## [1] "This SOBlock was created with nmoutput2so version 4.5.27"

Xpose diagnostics using NONMEM output

nm.xpdb<-as.xpdb(NM,datafile)
## 
## Removed dose rows in rawData+Predictions slot of SO to enable merge with Residuals data.
## 
## Residuals data does not currently contain dose rows in output from Nonmem executions.

Perform some basic goodness of fit (graphs are exported to PDF file)

print(basic.gof(nm.xpdb,subset="DVID==1"))

plot of chunk unnamed-chunk-21

print(basic.gof(nm.xpdb,subset="DVID==2"))

plot of chunk unnamed-chunk-21

print(ind.plots(nm.xpdb,subset="DVID==1"))

plot of chunk unnamed-chunk-21 plot of chunk unnamed-chunk-21

print(ind.plots(nm.xpdb,subset="DVID==2"))

plot of chunk unnamed-chunk-21 plot of chunk unnamed-chunk-21

Export results to a PDF file

pdf("GOF_NM.pdf")
 print(basic.gof(nm.xpdb,subset="DVID==1"))
 print(basic.gof(nm.xpdb,subset="DVID==2"))
 print(ind.plots(nm.xpdb,subset="DVID==1"))
 print(ind.plots(nm.xpdb,subset="DVID==2"))
dev.off()
## png 
##   2

Change estimation method to FOCEI (for speed)

MDL Objects can be manipulated from R to change for example the estimation algorithm

myTaskProperties <- getTaskPropertiesObjects(mdlfile)[[1]]
myNewTaskProperties <- myTaskProperties
myNewTaskProperties@ESTIMATE$algo <- "focei"

Assembling the new MOG. Note that we reuse the data and model from the previous run.

myNewerMOG <- createMogObj(dataObj = getDataObjects(mdlfile)[[1]], 
        parObj = getParameterObjects(mdlfile)[[1]], 
        mdlObj = getModelObjects(mdlfile)[[1]], 
        taskObj = myNewTaskProperties)

We can then write the MOG back out to an .mdl file.

mdlfile.FOCEI <- paste0(uc,"_FOCEI.mdl")
writeMogObj(myNewerMOG,mdlfile.FOCEI)

Test estimation using this new MOG.

By default, a covariance step is not run when estimating in PsN. To see how it can be requested, see UseCase1_1.mdl

NM.FOCEI <- estimate(mdlfile.FOCEI, target="PsN", subfolder="NONMEM_FOCEI")
## -- Wed Aug 17 09:54:11 2016
## New
## Submitted
## Job dfa115de-bd60-4712-bf69-9cd1bbe9d7d0 progress:
## Running [ ..................... ]
## Importing Results
## Copying the result data back to the local machine for job ID dfa115de-bd60-4712-bf69-9cd1bbe9d7d0...
## From C:\Users\smith_mk\AppData\Local\Temp\4\RtmpYF4SFO\DDMORE.job35641d9e2948 to C:/SEE/MDL_IDE/workspace/UseCasesDemo/UseCase3/NONMEM_FOCEI
## Done.
## 
## 
## The following main elements were parsed successfully:
##   RawResults
##   Estimation::PopulationEstimates::MLE
##   Estimation::IndividualEstimates::Estimates
##   Estimation::IndividualEstimates::RandomEffects
##   Estimation::Residuals::ResidualTable
##   Estimation::Predictions
##   Estimation::OFMeasures::Deviance
## 
## The following WARNINGs were raised during the job execution:
##  estimation_successful: 0
##  rounding_errors: 1
## 
## The following MESSAGEs were raised during the job execution:
##  covariance_step_run: 0
##  hessian_reset: 0
##  zero_gradients: 0
##  final_zero_gradients: 0
##  estimate_near_boundary: 0
##  s_matrix_singular: 0
##  significant_digits: 2.5
##  nmoutput2so_version: This SOBlock was created with nmoutput2so version 4.5.27
## 
## Completed
## -- Wed Aug 17 10:01:15 2016

Load previous results

NM.FOCEI <- LoadSOObject(“NONMEM_FOCEI/UseCase3_FOCEI.SO.xml”)

Results from NONMEM should be comparable to previous results

print(getPopulationParameters(NM.FOCEI,  what="estimates"))
## $MLE
##     POP_CL      POP_V     POP_KA   POP_TLAG   POP_PCA0    POP_C50 
##  0.1351820  8.0157500  1.4425500  0.9744370 96.6398000  1.1781200 
##    POP_TEQ   RUV_PROP    RUV_ADD     RUV_FX   POP_EMAX BETA_CL_WT 
## 12.9771000  0.0613936  0.2576460  3.7970600  1.0000000  0.7500000 
##  BETA_V_WT     PPV_CL  CORR_CL_V      PPV_V     PPV_KA   PPV_TLAG 
##  1.0000000  0.2629690  0.2553860  0.1418130  0.8786240  0.1000000 
##   PPV_PCA0   PPV_EMAX    PPV_C50    PPV_TEQ 
##  0.0535504  0.0000000  0.4449220  0.1060570
print(parameters_nm)
## $MLE
##     POP_CL      POP_V     POP_KA   POP_TLAG   POP_PCA0    POP_C50 
##  0.1349440  8.0021200  1.5015900  0.9718610 96.5891000  1.1729500 
##    POP_TEQ   RUV_PROP    RUV_ADD     RUV_FX   POP_EMAX BETA_CL_WT 
## 12.9216000  0.0601880  0.2653420  3.7856500  1.0000000  0.7500000 
##  BETA_V_WT     PPV_CL  CORR_CL_V      PPV_V     PPV_KA   PPV_TLAG 
##  1.0000000  0.2675610  0.2289230  0.1452610  0.9031980  0.1000000 
##   PPV_PCA0   PPV_EMAX    PPV_C50    PPV_TEQ 
##  0.0542843  0.0000000  0.4534200  0.1066040

Xpose diagnostics using NONMEM output

nmfocei.xpdb<-as.xpdb(NM.FOCEI,"warfarin_conc_pca.csv")
## 
## Removed dose rows in rawData+Predictions slot of SO to enable merge with Residuals data.
## 
## Residuals data does not currently contain dose rows in output from Nonmem executions.

Basic diagnostics for NONMEM fit.

pdf("GOF_NM_FOCEI.pdf")
 ind.plots(nmfocei.xpdb,subset="DVID==1",main="Individual plots (Warfarin concentration)")
 basic.gof(nmfocei.xpdb,subset="DVID==1",main="Goodness of fit (Warfarin concentration)")
 ind.plots(nmfocei.xpdb,subset="DVID==2",main="Individual plots (PCA level)")
 basic.gof(nmfocei.xpdb,subset="DVID==2",main="Goodness of fit (PCA level)")
 parm.hist(nmfocei.xpdb)
dev.off()
## png 
##   2

Run the bootstrap using PsN (takes several hours)

The ddmore “bootstrap.PsN” function is a wrap up function that calls Bootstrap PsN functionality using as input an MDL file that will be translated to NMTRAN as first step. Additional PsN arguments can be specified under the “bootstrapOptions” attribute. After task execution, the output from PsN is converted to a Standardised Output object which is saved in a .SO.xml file. Translated files and PsN output will be returned in the ./Bootstrap subfolder

bootstrapResults <- bootstrap.PsN(mdlfile.FOCEI, samples=20, seed=123456,
        bootstrapOptions=" -no-skip_minimization_terminated -threads=2",
        subfolder="Bootstrap", plot=TRUE)
## -- Wed Aug 17 10:01:24 2016
## New
## Submitted
## Job 1bb0bde3-2cdf-41f8-b860-0ee65b763c3d progress:
## Running [ ..................................................................................................................................................................................................... ]
## Importing Results
## Copying the result data back to the local machine for job ID 1bb0bde3-2cdf-41f8-b860-0ee65b763c3d...
## From C:\Users\smith_mk\AppData\Local\Temp\4\RtmpYF4SFO\DDMORE.job356426291aee to C:/SEE/MDL_IDE/workspace/UseCasesDemo/UseCase3/Bootstrap
## Done.
## 
## 
## The following main elements were parsed successfully:
##   RawResults
##   Estimation::PopulationEstimates::MLE
##   Estimation::PopulationEstimates::OtherMethodBootstrap
##   Estimation::PrecisionPopulationEstimates::OtherMethodBootstrap
##   Estimation::IndividualEstimates::Estimates
##   Estimation::IndividualEstimates::RandomEffects
##   Estimation::Residuals::ResidualTable
##   Estimation::Predictions
##   Estimation::OFMeasures::Deviance
## 
## The following WARNINGs were raised during the job execution:
##  estimation_successful: 0
##  rounding_errors: 1
##  bootstrap_parameter_scale: The parameters PPV_CL, CORR_CL_V, PPV_V, PPV_KA, PPV_TLAG, PPV_PCA0, PPV_EMAX, PPV_C50 and PPV_TEQ were requested on the sd/corr scale but are given on the var/cov scale in all bootstrap results.
## 
## The following MESSAGEs were raised during the job execution:
##  covariance_step_run: 0
##  hessian_reset: 0
##  zero_gradients: 0
##  final_zero_gradients: 0
##  estimate_near_boundary: 0
##  s_matrix_singular: 0
##  significant_digits: 2.5
##  nmoutput2so_version: This SOBlock was created with nmoutput2so version 4.5.27
## 
## Completed
## -- Wed Aug 17 11:07:14 2016
## Warning: NAs introduced by coercion
## [[1]]

plot of chunk Bootstrap

## 
## [[2]]
## NULL
## 
## [[3]]

plot of chunk Bootstrap

## 
## [[4]]
## NULL
## 
## [[5]]
## NULL

Load results from a bootstrap previously performed

bootstrapResults <- LoadSOObject(“Bootstrap/UseCase3_FOCEI.SO.xml”)

Export bootstrap histograms to a pdf

pdf(paste0(uc,"_Bootstrap.pdf"))
 print(boot.hist(results.file = file.path("Bootstrap",paste0("raw_results_",uc,"_FOCEI.csv")),
                incl.ids.file = file.path("Bootstrap","included_individuals1.csv")))
## [[1]]
## 
## [[2]]
## NULL
## 
## [[3]]
## 
## [[4]]
## NULL
## 
## [[5]]
## NULL
dev.off()
## png 
##   2

Extract parameter estimates and precision from bootstrap results.

print(getPopulationParameters(bootstrapResults, what="estimates"))
## $MLE
##     POP_CL      POP_V     POP_KA   POP_TLAG   POP_PCA0    POP_C50 
##  0.1351820  8.0157500  1.4425500  0.9744370 96.6398000  1.1781200 
##    POP_TEQ   RUV_PROP    RUV_ADD     RUV_FX   POP_EMAX BETA_CL_WT 
## 12.9771000  0.0613936  0.2576460  3.7970600  1.0000000  0.7500000 
##  BETA_V_WT     PPV_CL  CORR_CL_V      PPV_V     PPV_KA   PPV_TLAG 
##  1.0000000  0.2629690  0.2553860  0.1418130  0.8786240  0.1000000 
##   PPV_PCA0   PPV_EMAX    PPV_C50    PPV_TEQ 
##  0.0535504  0.0000000  0.4449220  0.1060570 
## 
## $Bootstrap
##             Parameter         Mean       Median
## BETA_CL_WT BETA_CL_WT  0.750000000  0.750000000
## BETA_V_WT   BETA_V_WT  1.000000000  1.000000000
## CORR_CL_V   CORR_CL_V  0.007199522  0.006883430
## POP_C50       POP_C50  1.151196000  1.148855000
## POP_CL         POP_CL  0.136623900  0.137295500
## POP_EMAX     POP_EMAX  1.000000000  1.000000000
## POP_KA         POP_KA  1.443438000  1.363440000
## POP_PCA0     POP_PCA0 96.420120000 96.291750000
## POP_TEQ       POP_TEQ 13.017580000 13.014600000
## POP_TLAG     POP_TLAG  0.975505100  0.937320000
## POP_V           POP_V  7.972322000  7.936970000
## PPV_C50       PPV_C50  0.187449400  0.175308500
## PPV_CL         PPV_CL  0.073605600  0.070859800
## PPV_EMAX     PPV_EMAX  0.000000000  0.000000000
## PPV_KA         PPV_KA  0.582861200  0.524732000
## PPV_PCA0     PPV_PCA0  0.002901588  0.003102005
## PPV_TEQ       PPV_TEQ  0.011000350  0.010031490
## PPV_TLAG     PPV_TLAG  0.010000000  0.010000000
## PPV_V           PPV_V  0.017470070  0.017315800
## RUV_ADD       RUV_ADD  0.260336600  0.290060000
## RUV_FX         RUV_FX  3.690415000  3.676410000
## RUV_PROP     RUV_PROP  0.066349200  0.062824150

Extract the information regarding the precision intervals

print(getPopulationParameters(bootstrapResults, what="intervals")$Bootstrap)
##     Parameter         Mean       Median        Perc_5      Perc_95
## 1  BETA_CL_WT  0.750000000  0.750000000  0.7500000000  0.750000000
## 2   BETA_V_WT  1.000000000  1.000000000  1.0000000000  1.000000000
## 3   CORR_CL_V  0.007199522  0.006883430 -0.0035638115  0.016768880
## 4     POP_C50  1.151196000  1.148855000  0.9962851000  1.302692000
## 5      POP_CL  0.136623900  0.137295500  0.1248119000  0.149181600
## 6    POP_EMAX  1.000000000  1.000000000  1.0000000000  1.000000000
## 7      POP_KA  1.443438000  1.363440000  0.7458111000  2.546774000
## 8    POP_PCA0 96.420120000 96.291750000 94.1757800000 98.984920000
## 9     POP_TEQ 13.017580000 13.014600000 12.3902000000 13.944720000
## 10   POP_TLAG  0.975505100  0.937320000  0.7932797000  1.421386000
## 11      POP_V  7.972322000  7.936970000  7.5082720000  8.410979000
## 12    PPV_C50  0.187449400  0.175308500  0.1184288000  0.267163000
## 13     PPV_CL  0.073605600  0.070859800  0.0371696200  0.121153300
## 14   PPV_EMAX  0.000000000  0.000000000  0.0000000000  0.000000000
## 15     PPV_KA  0.582861200  0.524732000  0.0638871300  1.633529000
## 16   PPV_PCA0  0.002901588  0.003102005  0.0006702533  0.004997395
## 17    PPV_TEQ  0.011000350  0.010031490  0.0017042410  0.020658330
## 18   PPV_TLAG  0.010000000  0.010000000  0.0100000000  0.010000000
## 19      PPV_V  0.017470070  0.017315800  0.0079833970  0.026591480
## 20    RUV_ADD  0.260336600  0.290060000  0.0030990000  0.390462500
## 21     RUV_FX  3.690415000  3.676410000  3.0342190000  4.130044000
## 22   RUV_PROP  0.066349200  0.062824150  0.0412749200  0.116884100

VPC of model

Before running the VPC with PsN we must update the (initial) values in the MDL Parameter Object MLE estimates from previous step can be used

structuralPar <- getPopulationParameters(NM.FOCEI, what="estimates",block='structural')$MLE
variabilityPar <- getPopulationParameters(NM.FOCEI, what="estimates",block='variability')$MLE

Update the parameter object using the ddmore “updateParObj” function. This function updates an R object of (S4) class “parObj”. The user chooses which block to update, what items within that block, and what to replace those items with. NOTE: that updateParObj can only update attributes which ALREADY EXIST in the MDL Parameter Object for that item. This ensures that valid MDL is preserved.

myParObj <- getParameterObjects(mdlfile)[[1]]
myParObjUpdated <- updateParObj(myParObj,block="STRUCTURAL",
        item=names(structuralPar),
        with=list(value=structuralPar))
myParObjUpdated <- updateParObj(myParObjUpdated,block="VARIABILITY",
        item=names(variabilityPar),
        with=list(value=variabilityPar))

Check that the appropriate initial values have been updated to the MLE values from the previous fit.

print(myParObjUpdated@STRUCTURAL)
## $POP_CL
## $POP_CL$value
## [1] "0.135182"
## 
## $POP_CL$lo
## [1] "0.01"
## 
## $POP_CL$hi
## [1] "1"
## 
## 
## $POP_V
## $POP_V$value
## [1] "8.01575"
## 
## $POP_V$lo
## [1] "0.01"
## 
## $POP_V$hi
## [1] "20"
## 
## 
## $POP_KA
## $POP_KA$value
## [1] "1.44255"
## 
## $POP_KA$lo
## [1] "0.01"
## 
## $POP_KA$hi
## [1] "24"
## 
## 
## $POP_TLAG
## $POP_TLAG$value
## [1] "0.974437"
## 
## $POP_TLAG$lo
## [1] "0.01"
## 
## $POP_TLAG$hi
## [1] "24"
## 
## 
## $POP_PCA0
## $POP_PCA0$value
## [1] "96.6398"
## 
## $POP_PCA0$lo
## [1] "0.01"
## 
## $POP_PCA0$hi
## [1] "200"
## 
## 
## $POP_EMAX
## $POP_EMAX$value
## [1] "1"
## 
## $POP_EMAX$lo
## [1] "0"
## 
## $POP_EMAX$fix
## [1] "true"
## 
## 
## $POP_C50
## $POP_C50$value
## [1] "1.17812"
## 
## $POP_C50$lo
## [1] "0.01"
## 
## $POP_C50$hi
## [1] "10"
## 
## 
## $POP_TEQ
## $POP_TEQ$value
## [1] "12.9771"
## 
## $POP_TEQ$lo
## [1] "0.01"
## 
## $POP_TEQ$hi
## [1] "100"
## 
## 
## $BETA_CL_WT
## $BETA_CL_WT$value
## [1] "0.75"
## 
## $BETA_CL_WT$fix
## [1] "true"
## 
## 
## $BETA_V_WT
## $BETA_V_WT$value
## [1] "1"
## 
## $BETA_V_WT$fix
## [1] "true"
print(myParObjUpdated@VARIABILITY)
## $PPV_CL
## $PPV_CL$value
## [1] "0.262969"
## 
## 
## $PPV_V
## $PPV_V$value
## [1] "0.141813"
## 
## 
## $PPV_KA
## $PPV_KA$value
## [1] "0.878624"
## 
## 
## $PPV_TLAG
## $PPV_TLAG$value
## [1] "0.1"
## 
## $PPV_TLAG$fix
## [1] "true"
## 
## 
## $CORR_CL_V
## $CORR_CL_V$value
## [1] "0.255386"
## 
## 
## $RUV_PROP
## $RUV_PROP$value
## [1] "0.0613936"
## 
## $RUV_PROP$lo
## [1] "0"
## 
## 
## $RUV_ADD
## $RUV_ADD$value
## [1] "0.257646"
## 
## $RUV_ADD$lo
## [1] "1.0E-4"
## 
## 
## $PPV_PCA0
## $PPV_PCA0$value
## [1] "0.0535504"
## 
## 
## $PPV_EMAX
## $PPV_EMAX$value
## [1] "0"
## 
## $PPV_EMAX$fix
## [1] "true"
## 
## 
## $PPV_C50
## $PPV_C50$value
## [1] "0.444922"
## 
## 
## $PPV_TEQ
## $PPV_TEQ$value
## [1] "0.106057"
## 
## 
## $RUV_FX
## $RUV_FX$value
## [1] "3.79706"
## 
## $RUV_FX$lo
## [1] "0"

Assembling the new MOG. Note that we reuse the data, model and tasks from the previous run.

myVPCMOG <- createMogObj(dataObj = getDataObjects(mdlfile)[[1]], 
        parObj = myParObjUpdated, 
        mdlObj = getModelObjects(mdlfile)[[1]], 
        taskObj = getTaskPropertiesObjects(mdlfile)[[1]])

We can then write the MOG back out to an .mdl file.

mdlfile.VPC <- paste0(uc,"_VPC.mdl")
writeMogObj(myVPCMOG,mdlfile.VPC)

Similarly as above, ddmore “VPC.PsN” function can be used to run a VPC using PsN as target tool

vpcFiles <- VPC.PsN(mdlfile.VPC,samples=20, seed=12345,
        vpcOptions ="-n_simulation=10 -stratify_on=DVID -auto_bin=7,10:5,8",
        subfolder="VPC", plot=TRUE) 
## -- Wed Aug 17 11:07:31 2016
## New
## Submitted
## Job b451e40e-84c7-4c1b-b86f-b6d361f5c63c progress:
## Running [ ... ]
## Importing Results
## Copying the result data back to the local machine for job ID b451e40e-84c7-4c1b-b86f-b6d361f5c63c...
## From C:\Users\smith_mk\AppData\Local\Temp\4\RtmpYF4SFO\DDMORE.job35647796758 to C:/SEE/MDL_IDE/workspace/UseCasesDemo/UseCase3/VPC
## Done.
## 
## 
## The following main elements were parsed successfully:
##   RawResults
##   SimulationSimulationBlock
##   SimulationSimulationBlock
## 
## The following MESSAGEs were raised during the job execution:
##  nmoutput2so_version: This SOBlock was created with nmoutput2so version 4.5.27
## 
## Completed
## -- Wed Aug 17 11:08:33 2016

plot of chunk VPC

To replay the visualisation using information from the VPC SO file

pdf(paste0(uc,"_VPC.pdf"))
 print(xpose.VPC(vpc.info= file.path("./VPC",vpcFiles@RawResults@DataFiles$PsN_VPC_results$path),
        vpctab= file.path("./VPC",vpcFiles@RawResults@DataFiles$PsN_VPC_vpctab$path),
        main="VPC warfarin PK/PD"))
dev.off()
## png 
##   2

Simulation using simulx

The mlxR package has been developed to visualize and explore models that are encoded in MLXTRAN or PharmML. The ddmore function as.PharmML translates an MDL file (extension .mdl) to its PharmML representation. The output file (extension .xml) is saved in the working directory.

myPharmML <- as.PharmML(mdlfile)

Use parameter values from the FOCEI estimation

parValues <- getPopulationParameters(NM.FOCEI, what="estimates")$MLE

Simulate for the typical weight of 70. Recall that logtWT = log(WT/70).

p <- c(parValues,WT=70)

Parameter values used in simulation

print(p)
##     POP_CL      POP_V     POP_KA   POP_TLAG   POP_PCA0    POP_C50 
##  0.1351820  8.0157500  1.4425500  0.9744370 96.6398000  1.1781200 
##    POP_TEQ   RUV_PROP    RUV_ADD     RUV_FX   POP_EMAX BETA_CL_WT 
## 12.9771000  0.0613936  0.2576460  3.7970600  1.0000000  0.7500000 
##  BETA_V_WT     PPV_CL  CORR_CL_V      PPV_V     PPV_KA   PPV_TLAG 
##  1.0000000  0.2629690  0.2553860  0.1418130  0.8786240  0.1000000 
##   PPV_PCA0   PPV_EMAX    PPV_C50    PPV_TEQ         WT 
##  0.0535504  0.0000000  0.4449220  0.1060570 70.0000000

Simulate for a dose of 100mg given at time 0 into the GUT (oral administration)

adm <- list(type=1, time = 0, amount = 100)

Simulate PK parameters for individuals

ind <- list(name = c('TLAG','KA','CL','V','TEQ','C50','PCA0','EMAX','RUV_ADD','RUV_PROP'))

Simulate predicted (CC) and observed concentration values (CP_obs), predicted (PCA) and observed PCA (PCA_obs)

f1   <- list( name = c('CC'), time = seq(0,to=50,by=1))
y1   <- list( name = c('CP_obs'), time = c(0, 0.5, 1, 2, 3, 4, 6, 8, 12, 24, 36, 48))
f2   <- list( name = c('PCA'), time = seq(0,to=120,by=2))
y2   <- list( name = c('PCA_obs'), time = c(0, 12, 24, 48, 72, 96, 120))

Simulate 12 subjects

g <- list( size = 12, level = 'individual',  treatment = adm)

Call simulx

res  <- simulx(model = myPharmML,
               parameter = p,
               group = g,
               output = list(ind,f1,y1,f2,y2))

Simulated parameter values for each individual

print(res$parameter)
##    id      TLAG        KA         CL        V      TEQ       C50      PCA0
## 1   1 1.1323894 1.1942263 0.15857875 6.837156 11.01329 1.1536902  94.00342
## 2   2 1.0147612 1.1103107 0.19392626 7.939794 13.15347 1.4554122 114.80873
## 3   3 0.9613819 1.0283501 0.16964671 6.955385 13.80335 1.1697797  91.23117
## 4   4 1.0167673 0.7744517 0.09205881 8.301527 14.73398 0.7800345  87.21103
## 5   5 1.0023340 0.8202940 0.13116580 7.597050 15.75156 1.1045815  96.86056
## 6   6 0.9157719 1.9618982 0.14591592 7.677061 16.43562 1.1065397  86.47693
## 7   7 1.0842102 0.6396269 0.13567591 7.784544 16.88159 1.1832323  98.90922
## 8   8 0.8455430 4.7760510 0.12768732 9.170121 11.44998 1.3184570  94.77142
## 9   9 1.0001449 4.6606875 0.13154539 6.516251 12.22601 0.9252285  99.76612
## 10 10 0.9967407 0.7074261 0.17379011 8.290396 12.65554 1.7784335  94.27248
## 11 11 0.9598640 3.2863203 0.13133276 8.842932 13.46176 1.9457872  90.95717
## 12 12 0.9506991 0.4433956 0.09825114 8.532447 11.99961 0.5269569  91.93521
##    EMAX  RUV_ADD  RUV_PROP
## 1     1 0.257646 0.0613936
## 2     1 0.257646 0.0613936
## 3     1 0.257646 0.0613936
## 4     1 0.257646 0.0613936
## 5     1 0.257646 0.0613936
## 6     1 0.257646 0.0613936
## 7     1 0.257646 0.0613936
## 8     1 0.257646 0.0613936
## 9     1 0.257646 0.0613936
## 10    1 0.257646 0.0613936
## 11    1 0.257646 0.0613936
## 12    1 0.257646 0.0613936

Plot simulated results

plot(ggplot() + 
                geom_line(data=res$CC, aes(x=time, y=CC, colour=id)) +
                geom_point(data=res$CP_obs, aes(x=time, y=CP_obs, colour=id)) +
                xlab("time (h)") + ylab("concentration") )

plot of chunk unnamed-chunk-44

plot(ggplot() + 
                geom_line(data=res$PCA, aes(x=time, y=PCA, colour=id)) +
                geom_point(data=res$PCA_obs, aes(x=time, y=PCA_obs, colour=id)) +
                xlab("time (h)") + ylab("PCA") )

plot of chunk unnamed-chunk-44

Simulate 1000 subjects - with simulx this is a QUICK process!

g <- list( size = 1000, level = 'individual',  treatment = adm)

Call simulx

res.1000  <- simulx(model =myPharmML,
                    parameter = p,
                    group = g,
                    output = list(ind,f1,y1,f2,y2))

Plot prediction intervals with prctilemlx. band defines the percentile bands displayed:

plot(prctilemlx(res.1000$CP_obs,band=list(number=9, level=90)))

plot of chunk unnamed-chunk-46

Table of the same information

print(prctilemlx(res.1000$CP_obs,band=list(number=10, level=100), plot=FALSE)$y)
##    time         0%        10%         20%         30%         40%
## 1   0.0 -0.5443035 -0.3174430 -0.22589504 -0.16472848 -0.06266723
## 2   0.5 -0.5977669 -0.2991036 -0.20065297 -0.09780454 -0.05166224
## 3   1.0 -0.4536572 -0.1730467 -0.03650321  0.03957093  0.09183161
## 4   2.0  1.7320581  4.8991873  6.56786563  7.63302151  8.44363137
## 5   3.0  3.5211390  7.3870345  8.95958132  9.44199904 10.07830197
## 6   4.0  4.0446315  8.6767898  9.61107294 10.10229837 10.67418590
## 7   6.0  5.1627945  8.9797556  9.64749832  9.99402479 10.36228615
## 8   8.0  6.7811357  9.0894443  9.66403709 10.03070612 10.64445448
## 9  12.0  7.4207940  8.5446299  9.13168064  9.58720788 10.05503276
## 10 24.0  5.2688630  6.5802651  7.29823621  7.80071610  8.36580159
## 11 36.0  4.4997663  5.5761877  6.08065430  6.41076846  6.77531984
## 12 48.0  3.1062515  4.3642886  4.87224532  5.16296991  5.31446526
##             50%          50%         60%         70%        80%        90%
## 1  -0.031144141 -0.031144141  0.03903172  0.08502052  0.1528819  0.3213090
## 2  -0.005521923 -0.005521923  0.03588673  0.10896727  0.1986732  0.3082639
## 3   0.221152565  0.221152565  0.41037884  0.88030800  1.9814785  3.1804951
## 4   9.183903308  9.183903308  9.84880510 10.68257083 11.4073912 13.5103420
## 5  10.854837789 10.854837789 11.45688344 12.25891484 13.1948643 14.3168272
## 6  11.056933587 11.056933587 11.61721509 12.34392740 13.1533247 14.2067226
## 7  10.877591838 10.877591838 11.48336646 11.69347391 12.7126478 13.9727309
## 8  11.214260229 11.214260229 11.60371597 11.98116148 12.5617202 13.2979970
## 9  10.439687010 10.439687010 10.66334884 10.94807585 11.4197547 12.3835767
## 10  8.818412899  8.818412899  9.36467661  9.70083786 10.0599983 10.5678788
## 11  7.082950208  7.082950208  7.35447864  7.59173569  8.2282351  8.9009271
## 12  5.753082086  5.753082086  6.24844717  6.68445710  7.2034741  7.6290636
##          100%
## 1   0.7000266
## 2   0.4654444
## 3   7.6699587
## 4  17.4699190
## 5  17.7815776
## 6  17.1296911
## 7  15.2976692
## 8  17.2326988
## 9  14.7968166
## 10 13.4317797
## 11 10.9510366
## 12  9.5303938

Prediction intervals for PCA levels

prctilemlx(res.1000$PCA_obs,band=list(number=9, level=90))

plot of chunk unnamed-chunk-48

Table of the same information

prctilemlx(res.1000$PCA_obs,band=list(number=10, level=100), plot=FALSE)$y
##   time        0%      10%      20%      30%      40%      50%      50%
## 1    0 85.031369 89.33802 92.15097 93.75648 94.78710 96.12971 96.12971
## 2   12 44.413279 50.30158 53.05034 54.41901 57.04001 58.53595 58.53595
## 3   24 24.157832 28.16860 31.05894 32.84331 35.12129 36.80478 36.80478
## 4   48  8.342804 13.04605 14.99230 17.19885 18.97980 20.39869 20.39869
## 5   72  6.210773 11.81055 15.17071 16.67079 18.13947 19.36680 19.36680
## 6   96  7.321311 14.30502 18.44570 19.48578 21.81371 23.70586 23.70586
## 7  120 10.179995 17.36432 20.38432 25.47962 28.71472 31.50686 31.50686
##        60%      70%       80%       90%      100%
## 1 98.03421 98.87241 101.01294 102.49137 114.29658
## 2 60.06396 61.56861  62.59081  65.75645  71.72720
## 3 38.07178 38.79190  40.80382  44.57329  47.56097
## 4 21.69728 23.34305  25.81019  28.48592  33.01694
## 5 21.22017 24.39782  26.14018  30.04210  41.36763
## 6 26.66115 30.28920  33.56722  40.02711  52.47710
## 7 35.16103 38.34368  40.50379  52.95883  66.17523