Guided-demo: Exploring 'ddmore' R package functionality of time to event models

UseCase14 : Time to event model for exact and right censored information

This example script is intended to illustrate how to use the 'ddmore' R package to perform a M&S workflow using the DDMoRe Standalone Execution Environment (SEE).

The following steps are implemented in this workflow:

To run a task, select with the cursor any code lines you wish to execute and press CTRL+R+R in your keyboard. An HTML file containing the commands in this file and associated output will be provided to allow the user to compare the results

Initialisation

Clear workspace and set working directory under 'UsesCasesDemo' project

rm(list=ls(all=FALSE))
mydir <- file.path(Sys.getenv("MDLIDE_WORKSPACE_HOME"),"UseCasesDemo")
setwd(mydir)

Create a working directory under 'models' folder where results are stored

uc<-"UseCase14"
datafile <- "warfarin_TTE_exact.csv"
mdlfile <- paste0(uc,".mdl")

Create a new folder to be used as working directory and where results will be stored

wd <- file.path(mydir,uc)
dir.create(wd)

Copy the dataset and the .mdl file available under “models” into the working directory

file.copy(file.path(mydir,"models", datafile),wd)
## [1] TRUE
file.copy(file.path(mydir,"models",mdlfile),wd)
## [1] TRUE

Set the working directory.

setwd(file.path(mydir,uc))

The working directory needs to be set differently when knitr R package is used to spin the file

library(knitr)
opts_knit$set(root.dir = file.path(mydir,uc)) 

List files available in working directory

list.files()
## [1] "UseCase14.mdl"          "warfarin_TTE_exact.csv"

Introduction to 'ddmore' R package

View objects within the .mdl file

Use 'ddmore' function getMDLObjects() to retrieve model object(s) from an existing .mdl file. This function reads the MDL in an .mdl file and parses the MDL code for each MDL Object into an R list of objects of appropriate types with names corresponding to the MDL Object names given in the file.

myMDLObj <- getMDLObjects(mdlfile)
length(myMDLObj)
## [1] 4
names(myMDLObj)
## [1] "warfarin_T2E_exact_dat"  "warfarin_T2E_exact_par" 
## [3] "warfarin_T2E_exact_mdl"  "warfarin_T2E_exact_task"

Use 'ddmore' function getDataObjects() to retrieve only data object(s) from an existing .mdl file. This function returns a list of Parameter Object(s) from which we select the first element. Hover over the variable name to see its structure

myDataObj <- getDataObjects(mdlfile)[[1]]

Use 'ddmore' function getParameterObjects() to retrieve only parameter object(s) from an existing .mdl file

myParObj <- getParameterObjects(mdlfile)[[1]]

Use 'ddmore' function getModelObjects() to retrieve only model object(s) from an existing .mdl file.

myModObj <- getModelObjects(mdlfile)[[1]]

Use 'ddmore' function getTaskPropertiesObjects() to retrieve only task properties object(s) from an existing .mdl file

myTaskObj <- getTaskPropertiesObjects(mdlfile)[[1]]

Exploratory Data Analysis

Recall that getDataObjects only reads the MDL code from the .mdl file. Use 'ddmore' function readDataObj() to create an R object from the MDL data object.

myData <- readDataObj(myDataObj)

Let's look at the first 6 lines of the data set

head(myData)
##   ID TIME TRT AMT RATE ADDL II     WT DVID DV MDV REP     CPX   PCAX
## 1  1    0   0   1   -2   27  1 72.065    3  0   0   1 0.00000 96.572
## 2  1   28   0   0    0    0  0 72.065    3  0   0   1 0.00000 96.572
## 3  2    0   1   1   -2   27  1 71.471    3  0   0   1 0.00000 95.398
## 4  2   28   1   0    0    0  0 71.471    3  0   0   1 0.77845 75.223
## 5  3    0   1   1   -2   27  1 69.330    3  0   0   1 0.00000 94.227
## 6  3    4   1   0    0    0  0 69.330    3  1   0   1 0.51216 58.072
##     INRX    ICL     IV    ITABS    ITLAG IF1  IPCA0 IEMAX    IC50    ITEQ
## 1 1.0000 2.9581  9.075 0.014635 0.109280 0.0 96.572    -1 1.39470 0.45693
## 2 1.0000 2.9581  9.075 0.014635 0.109280 0.0 96.572    -1 1.39470 0.45693
## 3 1.0000 2.7709  8.709 0.030355 0.034677 2.5 95.398    -1 3.27190 0.53506
## 4 1.2682 2.7709  8.709 0.030355 0.034677 2.5 95.398    -1 3.27190 0.53506
## 5 1.0000 2.9963 10.192 0.042503 0.085843 2.5 94.227    -1 0.83837 0.51051
## 6 1.6226 2.9963 10.192 0.042503 0.085843 2.5 94.227    -1 0.83837 0.51051
##   NEVT
## 1    1
## 2    1
## 3    1
## 4    1
## 5    1
## 6    1

Extract only observation records

myEDAData<-myData[myData$MDV==0,]
head(myEDAData)
##   ID TIME TRT AMT RATE ADDL II     WT DVID DV MDV REP     CPX   PCAX
## 1  1    0   0   1   -2   27  1 72.065    3  0   0   1 0.00000 96.572
## 2  1   28   0   0    0    0  0 72.065    3  0   0   1 0.00000 96.572
## 3  2    0   1   1   -2   27  1 71.471    3  0   0   1 0.00000 95.398
## 4  2   28   1   0    0    0  0 71.471    3  0   0   1 0.77845 75.223
## 5  3    0   1   1   -2   27  1 69.330    3  0   0   1 0.00000 94.227
## 6  3    4   1   0    0    0  0 69.330    3  1   0   1 0.51216 58.072
##     INRX    ICL     IV    ITABS    ITLAG IF1  IPCA0 IEMAX    IC50    ITEQ
## 1 1.0000 2.9581  9.075 0.014635 0.109280 0.0 96.572    -1 1.39470 0.45693
## 2 1.0000 2.9581  9.075 0.014635 0.109280 0.0 96.572    -1 1.39470 0.45693
## 3 1.0000 2.7709  8.709 0.030355 0.034677 2.5 95.398    -1 3.27190 0.53506
## 4 1.2682 2.7709  8.709 0.030355 0.034677 2.5 95.398    -1 3.27190 0.53506
## 5 1.0000 2.9963 10.192 0.042503 0.085843 2.5 94.227    -1 0.83837 0.51051
## 6 1.6226 2.9963 10.192 0.042503 0.085843 2.5 94.227    -1 0.83837 0.51051
##   NEVT
## 1    1
## 2    1
## 3    1
## 4    1
## 5    1
## 6    1

Open an R window to record and access all your plots

windows(record=TRUE) 

Plot Kaplan-Meier survival curve

library(survival)
kmfit <- survfit(formula=Surv(TIME,DV==1)~1, data=myData)

plot1 <- plot(kmfit, xlab="Time(days)", ylab="Survival Probability", conf.int=F)

plot of chunk unnamed-chunk-15

print(plot1)
## $x
## [1] 28
## 
## $y
## [1] 0.44

Plot Kaplan-Meier survival curve stratified by Treatment

kmfit.TRT <- survfit(formula=Surv(TIME,DV==1)~TRT, data=myData)

trt <- unique(myData$TRT)
plot2 <- plot(kmfit.TRT, xlab="Time(days)", ylab="Survival Probability", conf.int=F,
          col=c(1:length(myData$TRT))) 
        legend("bottomleft", legend= paste("TRT",unique(myData$TRT)) ,
          text.col=c(1:length(myData$TRT)),
          bty="n")

plot of chunk unnamed-chunk-16

print(plot2)
## $x
## [1] 28 28 28 28
## 
## $y
## [1] 0.4782609 0.4838710 0.3500000 0.4230769

Export results to a PDF file

pdf(paste0(uc,"_EGA.pdf"))
 print(plot1)
## $x
## [1] 28
## 
## $y
## [1] 0.44
 print(plot2)
## $x
## [1] 28 28 28 28
## 
## $y
## [1] 0.4782609 0.4838710 0.3500000 0.4230769
dev.off()
## png 
##   2

Model Development

ESTIMATE model parameters using Monolix

The ddmore “estimate” function translates the contents of the .mdl file to a target language and then estimates parameters using the target software. After estimation, the output is converted to a Standardised Output object which is saved in a .SO.xml file.

Translated files and Monolix output will be returned in the ./Monolix subfolder. The Standardised Output object (.SO.xml) is read and parsed into an R object called “mlx” of (S4) class “StandardOutputObject”.

mlx <- estimate(mdlfile, target="MONOLIX", subfolder="Monolix")
## -- Tue Aug 16 16:32:25 2016
## New
## Submitted
## Job 6617edf8-3dce-49d4-8662-b34ae905fe34 progress:
## Running [ ............. ]
## Importing Results
## Copying the result data back to the local machine for job ID 6617edf8-3dce-49d4-8662-b34ae905fe34...
## From C:\Users\smith_mk\AppData\Local\Temp\4\RtmpYF4SFO\DDMORE.job356470a1f5f to C:/SEE/MDL_IDE/workspace/UseCasesDemo/UseCase14/Monolix
## Done.
## 
## 
## The following main elements were parsed successfully:
##   ToolSettings
##   RawResults
##   Estimation::PopulationEstimates::MLE
##   Estimation::PrecisionPopulationEstimates::MLE
##   Estimation::IndividualEstimates::Estimates
##   Estimation::IndividualEstimates::RandomEffects
##   Estimation::OFMeasures::IndividualContribToLL
##   Estimation::OFMeasures::InformationCriteria
##   Estimation::OFMeasures::LogLikelihood
## 
## Completed
## -- Tue Aug 16 16:36:48 2016
slotNames(mlx)
## [1] "ToolSettings"     "RawResults"       "TaskInformation" 
## [4] "Estimation"       "ModelDiagnostic"  "Simulation"      
## [7] "OptimalDesign"    ".pathToSourceXML"

The ddmore “LoadSOObj” reads and parsed existing Standardise Output Objects

mlx <- LoadSOObject(“Monolix/UseCase14.SO.xml”)

The ddmore function “getPopulationParameters” extracts the Population Parameter values from an R object of (S4) class “StandardOutputObject” and returns the estimates. See documentation for getPopulationParameters to see other arguments and settings for this function.

parameters_mlx <- getPopulationParameters(mlx, what="estimates")$MLE
print(parameters_mlx)
## POP_BTATRT  POP_HBASE 
##    0.03009    9.88372
print(getPopulationParameters(mlx, what="precisions"))
## $MLE
##    Parameter     MLE      SE    RSE
## 1 POP_BTATRT 0.03009 0.10203 339.07
## 2  POP_HBASE 9.88372 2.14151  21.67
print(getEstimationInfo(mlx))
## $OFMeasures
## $OFMeasures$LogLikelihood
## $OFMeasures$LogLikelihood[[1]]
## [1] -255.165
## 
## 
## $OFMeasures$IndividualContribToLL
##     Subject ICtoLL
## 1         1  -0.76
## 2         2  -0.78
## 3         3  -3.69
## 4         4  -3.89
## 5         5  -3.64
## 6         6  -3.58
## 7         7  -0.80
## 8         8  -3.88
## 9         9  -0.76
## 10       10  -4.18
## 11       11  -4.22
## 12       12  -3.98
## 13       13  -0.85
## 14       14  -3.53
## 15       15  -4.30
## 16       16  -0.76
## 17       17  -0.80
## 18       18  -4.04
## 19       19  -4.17
## 20       20  -4.19
## 21       21  -0.80
## 22       22  -4.17
## 23       23  -3.74
## 24       24  -0.78
## 25       25  -0.78
## 26       26  -0.85
## 27       27  -0.80
## 28       28  -3.80
## 29       29  -3.59
## 30       30  -0.76
## 31       31  -4.28
## 32       32  -0.85
## 33       33  -0.78
## 34       34  -3.68
## 35       35  -0.80
## 36       36  -4.05
## 37       37  -4.28
## 38       38  -4.19
## 39       39  -3.69
## 40       40  -3.98
## 41       41  -3.66
## 42       42  -0.85
## 43       43  -0.80
## 44       44  -3.66
## 45       45  -3.69
## 46       46  -3.92
## 47       47  -0.85
## 48       48  -0.85
## 49       49  -0.78
## 50       50  -4.33
## 51       51  -3.67
## 52       52  -0.76
## 53       53  -0.85
## 54       54  -0.78
## 55       55  -0.76
## 56       56  -4.11
## 57       57  -3.89
## 58       58  -3.93
## 59       59  -0.76
## 60       60  -4.12
## 61       61  -3.61
## 62       62  -0.78
## 63       63  -0.78
## 64       64  -3.61
## 65       65  -0.85
## 66       66  -3.77
## 67       67  -0.78
## 68       68  -0.78
## 69       69  -3.77
## 70       70  -3.95
## 71       71  -0.78
## 72       72  -3.61
## 73       73  -3.95
## 74       74  -3.64
## 75       75  -3.91
## 76       76  -0.78
## 77       77  -4.11
## 78       78  -0.76
## 79       79  -4.15
## 80       80  -0.78
## 81       81  -4.19
## 82       82  -4.02
## 83       83  -0.78
## 84       84  -0.80
## 85       85  -0.78
## 86       86  -0.76
## 87       87  -0.76
## 88       88  -4.21
## 89       89  -4.26
## 90       90  -4.07
## 91       91  -3.74
## 92       92  -3.61
## 93       93  -0.85
## 94       94  -4.04
## 95       95  -0.85
## 96       96  -3.74
## 97       97  -4.18
## 98       98  -4.27
## 99       99  -0.76
## 100     100  -0.85
## 
## $OFMeasures$InformationCriteria
## $OFMeasures$InformationCriteria$AIC
## [1] 514.33
## 
## $OFMeasures$InformationCriteria$BIC
## [1] 519.54
## 
## 
## 
## $Messages
## list()

SAEM Estimation with NONMEM

By default, a covariance step is not run when estimating in NONMEM. To see how it can be requested, see UseCase1_1.mdl

NM <- estimate(mdlfile, target="NONMEM", subfolder="NONMEM")
## -- Tue Aug 16 16:36:48 2016
## New
## Submitted
## Job 0796e4c8-dbed-47cf-9b23-dbf11e77faff progress:
## Running [ ... ]
## Importing Results
## Copying the result data back to the local machine for job ID 0796e4c8-dbed-47cf-9b23-dbf11e77faff...
## From C:\Users\smith_mk\AppData\Local\Temp\4\RtmpYF4SFO\DDMORE.job3564667f6715 to C:/SEE/MDL_IDE/workspace/UseCasesDemo/UseCase14/NONMEM
## Done.
## 
## 
## The following main elements were parsed successfully:
##   RawResults
##   Estimation::PopulationEstimates::MLE
##   Estimation::IndividualEstimates::Estimates
##   Estimation::OFMeasures::Deviance
## 
## The following MESSAGEs were raised during the job execution:
##  estimation_successful: 1
##  covariance_step_run: 0
##  rounding_errors: 0
##  estimate_near_boundary: 0
##  s_matrix_singular: 0
##  nmoutput2so_version: This SOBlock was created with nmoutput2so version 4.5.27
## 
## Completed
## -- Tue Aug 16 16:37:50 2016

Load previous results

NM <- LoadSOObject(“NONMEM/UseCase14.SO.xml”)

The ddmore function “getPopulationParameters” extracts the Population Parameter values from an R object of (S4) class “StandardOutputObject” and returns the estimates. See documentation for getPopulationParameters to see other arguments and settings for this function.
Results from NONMEM should be comparable with results from MONOLIX

parameters_nm <- getPopulationParameters(NM, what="estimates")
print(parameters_nm)
## $MLE
##  POP_HBASE POP_BTATRT 
##  9.8717300  0.0305249
print(parameters_mlx)
## POP_BTATRT  POP_HBASE 
##    0.03009    9.88372
print(getEstimationInfo(NM))
## $OFMeasures
## $OFMeasures$Deviance
## $OFMeasures$Deviance[[1]]
## [1] 510.33
## 
## 
## 
## $Messages
## $Messages$Info
## $Messages$Info$estimation_successful
## [1] "1"
## 
## $Messages$Info$covariance_step_run
## [1] "0"
## 
## $Messages$Info$rounding_errors
## [1] "0"
## 
## $Messages$Info$estimate_near_boundary
## [1] "0"
## 
## $Messages$Info$s_matrix_singular
## [1] "0"
## 
## $Messages$Info$nmoutput2so_version
## [1] "This SOBlock was created with nmoutput2so version 4.5.27"

Xpose diagnostics using NONMEM output

There is a bug in the generation of xpose database

# #+ GOF for NONMEM Estimation
# nm.xpdb<-as.xpdb(NM,datafile)
#
# #' Perform some basic goodness of fit. There is no IIV in the model
# print(parm.hist(nm.xpdb, inclZeroWRES=TRUE))

Change estimation method to FOCEI (for speed)

MDL Objects can be manipulated from R to change for example the estimation algorithm

myTaskProperties <- getTaskPropertiesObjects(mdlfile)[[1]]
myNewTaskProperties <- myTaskProperties
myNewTaskProperties@ESTIMATE$algo <- "foce"

Assembling the new Modelling Object Group (MOG). Note that we reuse the data, parameters and model from the MOG.

myNewerMOG <- createMogObj(dataObj = getDataObjects(mdlfile)[[1]], 
        parObj = getParameterObjects(mdlfile)[[1]], 
        mdlObj = getModelObjects(mdlfile)[[1]], 
        taskObj = myNewTaskProperties)

We can then write the MOG back out to an .mdl file.

mdlfile.FOCE <- paste0(uc,"_FOCE.mdl")
writeMogObj(myNewerMOG,mdlfile.FOCE)

Test estimation using this new MOG in NONMEM via PsN

By default, a covariance step is not run when estimating in PsN. To see how it can be requested, see UseCase1_1.mdl

NM.FOCE <- estimate(mdlfile.FOCE, target="PsN", subfolder="NONMEM_FOCE")
## -- Tue Aug 16 16:37:58 2016
## New
## Submitted
## Job c1f6548f-4ee3-453f-a15d-9eb65746609f progress:
## Running [ .. ]
## Importing Results
## Copying the result data back to the local machine for job ID c1f6548f-4ee3-453f-a15d-9eb65746609f...
## From C:\Users\smith_mk\AppData\Local\Temp\4\RtmpYF4SFO\DDMORE.job356415b273d3 to C:/SEE/MDL_IDE/workspace/UseCasesDemo/UseCase14/NONMEM_FOCE
## Done.
## 
## 
## The following main elements were parsed successfully:
##   RawResults
##   Estimation::PopulationEstimates::MLE
##   Estimation::IndividualEstimates::Estimates
##   Estimation::OFMeasures::Deviance
## 
## The following MESSAGEs were raised during the job execution:
##  estimation_successful: 1
##  covariance_step_run: 0
##  rounding_errors: 0
##  hessian_reset: 0
##  zero_gradients: 0
##  final_zero_gradients: 0
##  estimate_near_boundary: 0
##  s_matrix_singular: 0
##  significant_digits: 4.2
##  nmoutput2so_version: This SOBlock was created with nmoutput2so version 4.5.27
## 
## Completed
## -- Tue Aug 16 16:38:40 2016

The ddmore “LoadSOObj” reads and parsed existing Standardise Output Objects

NM.FOCE <- LoadSOObject(“NONMEM_FOCE/UseCase14_FOCE.SO.xml”)

Results from NONMEM should be comparable to previous results

print(getPopulationParameters(NM.FOCE,  what="estimates")$MLE)
##  POP_HBASE POP_BTATRT 
##  9.8839000  0.0300856
print(parameters_nm)
## $MLE
##  POP_HBASE POP_BTATRT 
##  9.8717300  0.0305249
print(parameters_mlx)
## POP_BTATRT  POP_HBASE 
##    0.03009    9.88372

Run the bootstrap using PsN

The ddmore “bootstrap.PsN” function is a wrap up function that calls Bootstrap PsN functionality using as input an MDL file that will be translated to NMTRAN as first step. Additional PsN arguments can be specified under the “bootstrapOptions” attribute. After task execution, the output from PsN is converted to a Standardised Output object which is saved in a .SO.xml file. Translated files and PsN output will be returned in the ./Bootstrap subfolder

bootstrapResults <- bootstrap.PsN(mdlfile.FOCE, samples=20, seed=12345,
        bootstrapOptions=" -no-skip_minimization_terminated -threads=2",
        subfolder="Bootstrap", plot=TRUE)
## -- Tue Aug 16 16:38:40 2016
## New
## Submitted
## Job 4a2bf03c-bfef-4620-8149-b875cd67d5ea progress:
## Running [ .... ]
## Importing Results
## Copying the result data back to the local machine for job ID 4a2bf03c-bfef-4620-8149-b875cd67d5ea...
## From C:\Users\smith_mk\AppData\Local\Temp\4\RtmpYF4SFO\DDMORE.job3564158b382b to C:/SEE/MDL_IDE/workspace/UseCasesDemo/UseCase14/Bootstrap
## Done.
## 
## 
## The following main elements were parsed successfully:
##   RawResults
##   Estimation::PopulationEstimates::MLE
##   Estimation::PopulationEstimates::OtherMethodBootstrap
##   Estimation::PrecisionPopulationEstimates::OtherMethodBootstrap
##   Estimation::IndividualEstimates::Estimates
##   Estimation::OFMeasures::Deviance
## 
## The following MESSAGEs were raised during the job execution:
##  estimation_successful: 1
##  covariance_step_run: 0
##  rounding_errors: 0
##  hessian_reset: 0
##  zero_gradients: 0
##  final_zero_gradients: 0
##  estimate_near_boundary: 0
##  s_matrix_singular: 0
##  significant_digits: 4.2
##  nmoutput2so_version: This SOBlock was created with nmoutput2so version 4.5.27
## 
## Completed
## -- Tue Aug 16 16:40:02 2016
## [[1]]

plot of chunk Bootstrap

## 
## [[2]]
## NULL
## 
## [[3]]

plot of chunk Bootstrap

## 
## [[4]]
## NULL
## 
## [[5]]
## NULL

Load results from a bootstrap previously performed

bootstrapResults <- LoadSOObject(“Bootstrap/UseCase14_FOCE.SO.xml”)

Export bootstrap histograms to a pdf

pdf(paste0(uc,"_Bootstrap.pdf"))
 print(boot.hist(results.file = file.path("Bootstrap",paste0("raw_results_",uc,"_FOCE.csv")),
                incl.ids.file = file.path("Bootstrap","included_individuals1.csv")))
## [[1]]
## 
## [[2]]
## NULL
## 
## [[3]]
## 
## [[4]]
## NULL
## 
## [[5]]
## NULL
dev.off()
## png 
##   2

Extract parameter estimates and precision from bootstrap results.

print(getPopulationParameters(bootstrapResults, what="estimates"))
## $MLE
##  POP_HBASE POP_BTATRT 
##  9.8839000  0.0300856 
## 
## $Bootstrap
##             Parameter       Mean     Median
## POP_BTATRT POP_BTATRT 0.04451292 0.00471935
## POP_HBASE   POP_HBASE 9.96834000 9.87130500

Extract the information regarding the precision intervals

print(getPopulationParameters(bootstrapResults,what="intervals")$Bootstrap)
##    Parameter       Mean     Median   Perc_5    Perc_95
## 1 POP_BTATRT 0.04451292 0.00471935 0.004000  0.2523397
## 2  POP_HBASE 9.96834000 9.87130500 6.965948 12.9629100

VPC of model

Simulation of TTE models is not implemented in the current version.

Simulation using simulx

The mlxR package has been developed to visualize and explore models that are encoded in MLXTRAN or PharmML. The ddmore function as.PharmML translates an MDL file (extension .mdl) to its PharmML representation. The output file (extension .xml) is saved in the working directory.

myPharmML <- as.PharmML(mdlfile)

Use parameter values from the FOCEI estimation

parValues <- getPopulationParameters(NM.FOCE, what="estimates")$MLE

Add the treatment covariate

p1 <- c(parValues, TRT = 1)
p2 <- c(parValues, TRT = 4)

Parameter values used in simulation

print(p1)
##  POP_HBASE POP_BTATRT        TRT 
##  9.8839000  0.0300856  1.0000000
print(p2)
##  POP_HBASE POP_BTATRT        TRT 
##  9.8839000  0.0300856  4.0000000

Simulate events

h <- list(name='HAZ', time=seq(0, 60, by=1))
e <- list(name='Y', time=0)

Simulate 100 subjects within each treatment group

g1 <- list( size = 100, level = 'longitudinal',  parameter = p1)
g2 <- list( size = 100, level = 'longitudinal',  parameter = p2)

Call simulx

res <- simulx(model = myPharmML,
               group = list(g1,g2),
               output = list(h,e))

Kaplan-Meyer plot of the simulated time-to-event data

print(kmplotmlx(res$Y, level=0.9))

plot of chunk unnamed-chunk-35