R Under development (unstable) (2026-03-08 r89578 ucrt) -- "Unsuffered Consequences" Copyright (C) 2026 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > library(testthat) > library(PatientLevelPrediction) > test_check("PatientLevelPrediction") Internet: TRUE attempting to download GiBleed trying URL 'https://raw.githubusercontent.com/OHDSI/EunomiaDatasets/main/datasets/GiBleed/GiBleed_5.3.zip' Content type 'application/zip' length 6861852 bytes (6.5 MB) ================================================== downloaded 6.5 MB attempting to extract and load: D:\temp\2026_03_09_15_30_17_16713\Rtmp6HcNzx/GiBleed_5.3.zip to: D:\temp\2026_03_09_15_30_17_16713\Rtmp6HcNzx/GiBleed_5.3.sqlite Cohorts created in table main.cohort No cdm database id entered so using cdmDatabaseSchema - if cdmDatabaseSchema is the same for multiple different databases, please use cdmDatabaseId to specify a unique identifier for the database and version Connecting using SQLite driver Constructing the at risk cohort Executing SQL took 0.0146 secs Fetching cohorts from server Loading cohorts took 0.0256 secs Constructing features on server Executing SQL took 0.078 secs Fetching data from server Fetching data took 0.555 secs Fetching outcomes from server Loading outcomes took 0.0292 secs Use timeStamp: TRUE Creating save directory at: D:\temp\2026_03_09_15_30_17_16713\Rtmp6HcNzx\saveLoc6738479674bb/Test Currently in a tryCatch or withCallingHandlers block, so unable to add global calling handlers. ParallelLogger will not capture R messages, errors, and warnings, only explicit calls to ParallelLogger. (This message will not be shown again this R session) Patient-Level Prediction Package version 6.6.0 Study started at: 2026-03-09 15:33:48.102059 AnalysisID: Test AnalysisName: Testing analysis TargetID: 1 OutcomeID: 3 Cohort size: 1800 Covariates: 75 Creating population Outcome is 0 or 1 Population created with: 1800 observations, 1800 unique subjects and 355 outcomes Population created in 0.0551 secs seed: 12 Creating a 25% test and 75% train (into 3 folds) random stratified split by class Data split into 449 test cases and 1351 train cases (451, 450, 450) Data split in 2.4 secs Train Set: Fold 1 451 patients with 89 outcomes - Fold 2 450 patients with 89 outcomes - Fold 3 450 patients with 89 outcomes 75 covariates in train data Test Set: 449 patients with 88 outcomes Removing 2 redundant covariates Removing 0 infrequent covariates Normalizing covariates Tidying covariates took 2.64 secs Train Set: Fold 1 451 patients with 89 outcomes - Fold 2 450 patients with 89 outcomes - Fold 3 450 patients with 89 outcomes 73 covariates in train data Test Set: 449 patients with 88 outcomes Running Cyclops Done. GLM fit status: OK Creating variable importance data frame Prediction took 0.206 secs Time to fit model: 1.32 secs Removing infrequent and redundant covariates and normalizing Removing infrequent and redundant covariates covariates and normalizing took 0.756 secs Prediction took 0.206 secs Prediction done in: 1.71 secs Calculating Performance for Test ============= AUC 71.47 95% lower AUC: 65.41 95% upper AUC: 77.53 AUPRC: 35.74 Brier: 0.14 Eavg: 0.02 Calibration in large- Mean predicted risk 0.1897 : observed risk 0.196 Calibration in large- Intercept 0.3216 Weak calibration intercept: 0.3216 - gradient:1.2182 Hosmer-Lemeshow calibration gradient: 1.23 intercept: -0.06 Average Precision: 0.37 Calculating Performance for Train ============= AUC 72.44 95% lower AUC: 69.00 95% upper AUC: 75.87 AUPRC: 39.39 Brier: 0.14 Eavg: 0.02 Calibration in large- Mean predicted risk 0.1976 : observed risk 0.1976 Calibration in large- Intercept 0.174 Weak calibration intercept: 0.174 - gradient:1.1421 Hosmer-Lemeshow calibration gradient: 1.21 intercept: -0.04 Average Precision: 0.39 Calculating Performance for CV ============= AUC 67.81 95% lower AUC: 64.14 95% upper AUC: 71.49 AUPRC: 31.81 Brier: 0.14 Eavg: 0.02 Calibration in large- Mean predicted risk 0.1979 : observed risk 0.1976 Calibration in large- Intercept 0.1826 Weak calibration intercept: 0.1826 - gradient:1.149 Hosmer-Lemeshow calibration gradient: 1.17 intercept: -0.03 Average Precision: 0.32 Time to calculate evaluation metrics: 0.459 secs Calculating covariate summary @ 2026-03-09 15:33:57.25295 This can take a while... Creating binary labels Joining with strata calculating subset of strata 1 calculating subset of strata 2 calculating subset of strata 3 calculating subset of strata 4 Restricting to subgroup Calculating summary for subgroup TestWithNoOutcome Restricting to subgroup Calculating summary for subgroup TrainWithNoOutcome Restricting to subgroup Calculating summary for subgroup TrainWithOutcome Restricting to subgroup Calculating summary for subgroup TestWithOutcome Aggregating with labels and strata Finished covariate summary @ 2026-03-09 15:34:00.783472 Time to calculate covariate summary: 3.53 secs Run finished successfully. Saving PlpResult Creating directory to save model plpResult saved to ..\D:\temp\2026_03_09_15_30_17_16713\Rtmp6HcNzx\saveLoc6738479674bb/Test\plpResult runPlp time taken: 12.7 secs Use timeStamp: TRUE Diagnosing impact of minTimeAtRisk in populationSettings Outcome is 0 or 1 Population created with: 1800 observations, 1800 unique subjects and 355 outcomes Population created in 0.0618 secs Outcome is 0 or 1 Population created with: 1800 observations, 1800 unique subjects and 355 outcomes Population created in 0.0707 secs Outcome is 0 or 1 Population created with: 1800 observations, 1800 unique subjects and 355 outcomes Population created in 0.0792 secs Outcome is 0 or 1 Population created with: 1800 observations, 1800 unique subjects and 355 outcomes Population created in 0.0598 secs Outcome is 0 or 1 Population created with: 1800 observations, 1800 unique subjects and 355 outcomes Population created in 0.0906 secs Outcome is 0 or 1 Population created with: 1800 observations, 1800 unique subjects and 355 outcomes Population created in 0.0705 secs Outcome is 0 or 1 Population created with: 1800 observations, 1800 unique subjects and 355 outcomes Population created in 0.0535 secs Saving diagnosePlp to D:\temp\2026_03_09_15_30_17_16713\Rtmp6HcNzx\saveLoc6738479674bb/Test/diagnosePlp.rds Outcome is 0 or 1 Population created with: 1800 observations, 1800 unique subjects and 355 outcomes Population created in 0.0555 secs seed: 12 Creating a 25% test and 75% train (into 3 folds) random stratified split by class Data split into 449 test cases and 1351 train cases (451, 450, 450) Data split in 2.47 secs seed: 12 Creating a 25% test and 75% train (into 3 folds) random stratified split by class Data split into 449 test cases and 1351 train cases (451, 450, 450) Data split in 2.48 secs Use timeStamp: TRUE Creating save directory at: D:\temp\2026_03_09_15_30_17_16713\Rtmp6HcNzx\saveLoc6738479674bb/tinyResults/tinyFit Patient-Level Prediction Package version 6.6.0 Study started at: 2026-03-09 15:34:11.657467 AnalysisID: tinyFit AnalysisName: Study details TargetID: 1 OutcomeID: 3 Cohort size: 865 Covariates: 2 Creating population Outcome is 0 or 1 Population created with: 865 observations, 865 unique subjects and 262 outcomes Population created in 0.0531 secs seed: 123 Creating a 25% test and 75% train (into 3 folds) random stratified split by class Data split into 215 test cases and 650 train cases (217, 217, 216) Data split in 2.7 secs Train Set: Fold 1 217 patients with 66 outcomes - Fold 2 217 patients with 66 outcomes - Fold 3 216 patients with 65 outcomes 2 covariates in train data Test Set: 215 patients with 65 outcomes Removing 0 redundant covariates Removing 0 infrequent covariates Normalizing covariates Tidying covariates took 2.47 secs Train Set: Fold 1 217 patients with 66 outcomes - Fold 2 217 patients with 66 outcomes - Fold 3 216 patients with 65 outcomes 2 covariates in train data Test Set: 215 patients with 65 outcomes Running Cyclops Done. GLM fit status: OK Creating variable importance data frame Prediction took 0.431 secs Time to fit model: 1.41 secs Removing infrequent and redundant covariates and normalizing Removing infrequent and redundant covariates covariates and normalizing took 0.784 secs Prediction took 0.19 secs Prediction done in: 1.75 secs Calculating Performance for Test ============= AUC 63.68 95% lower AUC: 57.26 95% upper AUC: 70.10 AUPRC: 40.36 Brier: 0.20 Eavg: 0.01 Calibration in large- Mean predicted risk 0.3018 : observed risk 0.3023 Calibration in large- Intercept -0.118 Weak calibration intercept: -0.118 - gradient:0.8335 Hosmer-Lemeshow calibration gradient: 0.90 intercept: 0.03 Average Precision: 0.41 Calculating Performance for Train ============= AUC 64.99 95% lower AUC: 61.50 95% upper AUC: 68.48 AUPRC: 41.44 Brier: 0.19 Eavg: 0.00 Calibration in large- Mean predicted risk 0.3031 : observed risk 0.3031 Calibration in large- Intercept 0.0121 Weak calibration intercept: 0.0121 - gradient:1.0174 Hosmer-Lemeshow calibration gradient: 1.02 intercept: -0.00 Average Precision: 0.41 Calculating Performance for CV ============= AUC 63.91 95% lower AUC: 59.57 95% upper AUC: 68.24 AUPRC: 39.42 Brier: 0.20 Eavg: 0.04 Calibration in large- Mean predicted risk 0.3031 : observed risk 0.3031 Calibration in large- Intercept -0.0779 Weak calibration intercept: -0.0779 - gradient:0.8911 Hosmer-Lemeshow calibration gradient: 1.02 intercept: 0.01 Average Precision: 0.39 Time to calculate evaluation metrics: 0.324 secs Run finished successfully. Saving PlpResult Creating directory to save model plpResult saved to ..\D:\temp\2026_03_09_15_30_17_16713\Rtmp6HcNzx\saveLoc6738479674bb/tinyResults/tinyFit\plpResult runPlp time taken: 9.26 secs Warning: PredictionDistribution not available for survival models No databaseRefId specified so using schema as unique database identifier Warning: Unable to coerce hyperParamSearch to data.frame; storing empty table for compatibility. Connecting using SQLite driver All or some PLP result tables do not exist, tables being recreated Deleting existing tables Creating PLP results tables Executing SQL took 0.037 secs PLP result migration being applied Migrating data set Migrator using SQL files in PatientLevelPrediction Connecting using SQLite driver Creating migrations table Executing SQL took 0.00517 secs Migrations table created Executing migration: Migration_1-store_version.sql Executing SQL took 0.00429 secs Saving migration: Migration_1-store_version.sql Executing SQL took 0.00361 secs Migration complete Migration_1-store_version.sql Executing migration: Migration_2-add_hyperparameter_settings.sql Executing SQL took 0.00908 secs Saving migration: Migration_2-add_hyperparameter_settings.sql Executing SQL took 0.0036 secs Migration complete Migration_2-add_hyperparameter_settings.sql Closing database connection Updating version number Connecting using SQLite driver Executing SQL took 0.00458 secs Connecting using SQLite driver Adding new model settings Inserting data took 0.0117 secs Connecting using SQLite driver All or some PLP result tables do not exist, tables being recreated Deleting existing tables Creating PLP results tables Executing SQL took 0.0328 secs PLP result migration being applied Migrating data set Migrator using SQL files in PatientLevelPrediction Connecting using SQLite driver Creating migrations table Executing SQL took 0.00471 secs Migrations table created Executing migration: Migration_1-store_version.sql Executing SQL took 0.00423 secs Saving migration: Migration_1-store_version.sql Executing SQL took 0.00358 secs Migration complete Migration_1-store_version.sql Executing migration: Migration_2-add_hyperparameter_settings.sql Executing SQL took 0.00751 secs Saving migration: Migration_2-add_hyperparameter_settings.sql Executing SQL took 0.00359 secs Migration complete Migration_2-add_hyperparameter_settings.sql Closing database connection Updating version number Connecting using SQLite driver Executing SQL took 0.00519 secs Connecting using SQLite driver Adding new model settings Inserting data took 0.00947 secs Adding new model settings Inserting data took 0.0086 secs Adding new model settings Inserting data took 0.00877 secs Adding new model settings Inserting data took 0.00833 secs Adding new model settings Inserting data took 0.0106 secs Adding new model settings Inserting data took 0.00849 secs Adding new model settings Inserting data took 0.00872 secs Connecting using SQLite driver All or some PLP result tables do not exist, tables being recreated Deleting existing tables Creating PLP results tables Executing SQL took 0.0344 secs PLP result migration being applied Migrating data set Migrator using SQL files in PatientLevelPrediction Connecting using SQLite driver Creating migrations table Executing SQL took 0.00407 secs Migrations table created Executing migration: Migration_1-store_version.sql Executing SQL took 0.00498 secs Saving migration: Migration_1-store_version.sql Executing SQL took 0.00384 secs Migration complete Migration_1-store_version.sql Executing migration: Migration_2-add_hyperparameter_settings.sql Executing SQL took 0.00733 secs Saving migration: Migration_2-add_hyperparameter_settings.sql Executing SQL took 0.00388 secs Migration complete Migration_2-add_hyperparameter_settings.sql Closing database connection Updating version number Connecting using SQLite driver Executing SQL took 0.00466 secs Connecting using SQLite driver Adding new hyperparameter settings Inserting data took 0.00914 secs Hyperparameter setting exists Connecting using SQLite driver All or some PLP result tables do not exist, tables being recreated Deleting existing tables Creating PLP results tables Executing SQL took 0.0352 secs PLP result migration being applied Migrating data set Migrator using SQL files in PatientLevelPrediction Connecting using SQLite driver Creating migrations table Executing SQL took 0.00368 secs Migrations table created Executing migration: Migration_1-store_version.sql Executing SQL took 0.00391 secs Saving migration: Migration_1-store_version.sql Executing SQL took 0.00346 secs Migration complete Migration_1-store_version.sql Executing migration: Migration_2-add_hyperparameter_settings.sql Executing SQL took 0.00712 secs Saving migration: Migration_2-add_hyperparameter_settings.sql Executing SQL took 0.00379 secs Migration complete Migration_2-add_hyperparameter_settings.sql Closing database connection Updating version number Connecting using SQLite driver Executing SQL took 0.0044 secs Connecting using SQLite driver Adding TAR Executing SQL took 0.00686 secs tarId: 1 Adding cohort target Inserting data took 0.009 secs Adding cohort target Inserting data took 0.00877 secs tId: 1 Adding cohort outcome Inserting data took 0.00865 secs Adding cohort outcome Inserting data took 0.00885 secs oId: 2 Adding new population settings Inserting data took 0.00797 secs popSetId: 1 Adding new covariate settings Inserting data took 0.00816 secs covSetId: 1 Adding new model settings Inserting data took 0.00868 secs modSetId: 1 Adding new plp data settings Inserting data took 0.00881 secs plpDataSetId: 1 Adding new feature_engineering settings Inserting data took 0.00871 secs FESetId: 1 Adding new sample settings Inserting data took 0.00836 secs sampleSetId: 1 Adding new tidy covariates settings Inserting data took 0.00797 secs tidySetId: 1 Adding new split settings Inserting data took 0.00867 secs splitId: 1 Adding new hyperparameter settings Inserting data took 0.00827 secs hyperparameterSetId: 2 Executing SQL took 0.00404 secs modelDesignId: 1 TAR exists tarId: 1 json in jsons:TRUE Cohort target exists in cohort_definition with cohort id1 Cohort target exists in cohorts with cohort id1 tId: 1 json in jsons:TRUE Cohort outcome exists in cohort_definition with cohort id2 Cohort outcome exists in cohorts with cohort id2 oId: 2 Population settings exists popSetId: 1 Covariate setting exists covSetId: 1 Model setting exists modSetId: 1 Split setting exists plpDataSetId: 1 feature engineering setting exists FESetId: 1 sample setting exists sampleSetId: 1 tidy covariates setting exists tidySetId: 1 Adding new split settings Inserting data took 0.00925 secs splitId: 2 Adding new hyperparameter settings Inserting data took 0.00873 secs hyperparameterSetId: 3 Executing SQL took 0.00365 secs modelDesignId: 2 TAR exists tarId: 1 json in jsons:TRUE Cohort target exists in cohort_definition with cohort id1 Cohort target exists in cohorts with cohort id1 tId: 1 json in jsons:TRUE Cohort outcome exists in cohort_definition with cohort id2 Cohort outcome exists in cohorts with cohort id2 oId: 2 Population settings exists popSetId: 1 Covariate setting exists covSetId: 1 Model setting exists modSetId: 1 Split setting exists plpDataSetId: 1 feature engineering setting exists FESetId: 1 sample setting exists sampleSetId: 1 tidy covariates setting exists tidySetId: 1 Split setting exists splitId: 1 Hyperparameter setting exists hyperparameterSetId: 2 modelDesignId: 1 Prediction done in: 0.00048 secs Creating directory to save model New best performance 0.5 with param: lambda=1 Fit best model on whole training set Calculating covariate summary @ 2026-03-09 15:34:28.892108 This can take a while... calculating subset of strata 1 Restricting to subgroup Calculating summary for subgroup Aggregating with no labels or strata Finished covariate summary @ 2026-03-09 15:34:29.65921 Time to calculate covariate summary: 0.768 secs Calculating covariate summary @ 2026-03-09 15:34:29.673118 This can take a while... calculating subset of strata 1 Restricting to subgroup Calculating summary for subgroup Aggregating with no labels or strata Finished covariate summary @ 2026-03-09 15:34:30.502271 Time to calculate covariate summary: 0.83 secs Calculating covariate summary @ 2026-03-09 15:34:30.522561 This can take a while... Creating binary labels calculating subset of strata 1 calculating subset of strata 2 Restricting to subgroup Calculating summary for subgroup WithOutcome Restricting to subgroup Calculating summary for subgroup WithNoOutcome Aggregating with only labels or strata Finished covariate summary @ 2026-03-09 15:34:32.158267 Time to calculate covariate summary: 1.64 secs Restricting to subgroup Calculating summary for subgroup variance needs to be >= 0 variance should be of class:numericvariance should be of class:integer seed should be of class:numericseed should be of class:NULLseed should be of class:integer threads should be of class:numericthreads should be of class:integer lowerLimit should be of class:numericlowerLimit should be of class:integer upperLimit should be of class:numericupperLimit should be of class:integer upperLimit needs to be >= 3 variance needs to be >= 0 variance should be of class:numericvariance should be of class:integer seed should be of class:numericseed should be of class:NULLseed should be of class:integer threads should be of class:numericthreads should be of class:integer lowerLimit should be of class:numericlowerLimit should be of class:integer upperLimit should be of class:numericupperLimit should be of class:integer upperLimit needs to be >= 3 testFraction should be of class:numerictestFraction should be of class:integer testFraction needs to be >= 0 -1 * testFraction needs to be > -1 trainFraction should be of class:numerictrainFraction should be of class:integer -1 * trainFraction needs to be >= -1 trainFraction needs to be > 0 splitSeed should be of class:numericsplitSeed should be of class:integer splitSeed should be of class:numericsplitSeed should be of class:integer nfold should be of class:numericnfold should be of class:integer nfold should be of class:numericnfold should be of class:integer Invalid type setting. Pick from: 'stratified','time','subject' type should be of class:character type should be of class:character seed: 52583 seed: 52583 Creating a 30% test and 70% train (into 4 folds) random stratified split by class Data split into 59 test cases and 141 train cases (36, 36, 35, 34) seed: 52583 Creating a 20% test and 80% train (into 4 folds) random stratified split by class Data split into 99 test cases and 401 train cases (101, 100, 100, 100) seed: 52583 Creating a 20% test and 40% train (into 4 folds) random stratified split by class Data split into 99 test cases and 201 train cases (51, 51, 50, 49) 200 were not used for training or testing seed: 52583 Creating 20% test and 80% train (into 4 folds) stratified split at 2011-02-05 Data split into 100 test cases and 400 train samples (101, 101, 101, 97) seed: 52583 Creating 20% test and 40% train (into 4 folds) stratified split at 2011-02-05 Data split into 100 test cases and 204 train samples (51, 51, 51, 51) 196 were not used for training or testing seed: 52583 seed: 52583 Creating a 20% test and 80% train (into 4 folds) stratified split by subject Data split into 40 test cases and 160 train cases (41, 41, 39, 39) seed: 52583 Creating a 25% test and 75% train (into 3 folds) stratified split by subject Data split into 52 test cases and 148 train cases (52, 48, 48) Evaluating survival model at time: 365 days C-statistic: 0.39291 (0.226-0.56) E-statistic: 0.026553307531677 E-statistic 90%: 0.0519477918330355 Warning: PredictionDistribution not available for survival models Time to calculate evaluation metrics: 0.501 secs Setting levels: control = 0, case = 1 Warning: Cannot compute AUC: outcomeCount has only one class Warning: Cannot compute AUC: outcomeCount has only one class Warning: Cannot compute AUPRC: outcomeCount has only one class Warning: Cannot compute Average Precision: outcomeCount has no positive class Calculating Performance for Validation ============= Warning: Cannot compute AUC: outcomeCount has only one class AUC NA 95% lower AUC: NA 95% upper AUC: NA Warning: Cannot compute AUPRC: outcomeCount has only one class AUPRC: NA Brier: 0.34 Eavg: 0.51 Calibration in large- Mean predicted risk 0.5135 : observed risk 0 Calibration in large- Intercept -26.5661 Weak calibration intercept: -26.5661 - gradient:0 Hosmer-Lemeshow calibration gradient: 0.00 intercept: -0.00 Warning: Cannot compute Average Precision: outcomeCount has no positive class Average Precision: NA Warning: Number of positives is zero Time to calculate evaluation metrics: 0.108 secs Column names of coefficients are not correct Column types of coefficients are not correct Column types of coefficients are not correct intercept should be of class:numeric mapping should be of class:charactermapping should be of class:function targetId should be of class:numerictargetId should be of class:NULL outcomeId should be of class:numericoutcomeId should be of class:NULL populationSettings should be of class:NULLpopulationSettings should be of class:populationSettings restrictPlpDataSettings should be of class:NULLrestrictPlpDataSettings should be of class:restrictPlpDataSettings covariateSettings should be of class:listcovariateSettings should be of class:NULLcovariateSettings should be of class:covariateSettings predict risk probabilities using predictGlm Prediction took 0.188 secs Prediction done in: 0.195 secs databaseDetails should be of class:databaseDetails databaseDetails should be of class:databaseDetails databaseDetails should be of class:databaseDetails No cdm database name entered so using cdmDatabaseSchema threshold needs to be >= 0 threshold should be of class:numeric threshold needs to be < 1 starting to map the columns and rows finished MapCovariates starting to map the columns and rows finished MapCovariates plpData size estimated to use 0GBs of RAM starting toSparseM starting to map the columns and rows finished MapCovariates toSparseM non temporal used plpData size estimated to use 0GBs of RAM finishing toSparseM toSparseM took 1.85891389846802 secs starting to map the columns and rows finished MapCovariates Warning: Imputation is experimental and may have bugs, please report any issues on the GitHub repository. Warning: Imputation is experimental and may have bugs, please report any issues on the GitHub repository. Warning: Imputation is experimental and may have bugs, please report any issues on the GitHub repository. missingThreshold needs to be > 0 Warning: Imputation is experimental and may have bugs, please report any issues on the GitHub repository. missingThreshold should be of class:numeric Warning: Imputation is experimental and may have bugs, please report any issues on the GitHub repository. missingThreshold needs to be < 1 Warning: Imputation is experimental and may have bugs, please report any issues on the GitHub repository. method should be mean or median Warning: Imputation is experimental and may have bugs, please report any issues on the GitHub repository. addMissingIndicator should be of class:logical Warning: Imputation is experimental and may have bugs. Please report any issues on the GitHub repository. Warning: Imputation is experimental and may have bugs. Please report any issues on the GitHub repository. method should be pmm Warning: Imputation is experimental and may have bugs. Please report any issues on the GitHub repository. missingThreshold needs to be > 0 Warning: Imputation is experimental and may have bugs. Please report any issues on the GitHub repository. missingThreshold should be of class:numeric Warning: Imputation is experimental and may have bugs. Please report any issues on the GitHub repository. missingThreshold needs to be < 1 Warning: Imputation is experimental and may have bugs. Please report any issues on the GitHub repository. Warning: Imputation is experimental and may have bugs. Please report any issues on the GitHub repository. addMissingIndicator should be of class:logical Warning: Imputation is experimental and may have bugs, please report any issues on the GitHub repository. Imputing missing values with simpleImputer using: mean and missing threshold: 0.3 Calculating missingness in data Found 2 features with missing values Imputation done in time: 3.92 secs Applying imputation to test data with simpleImputer using method: mean and missing threshold: 0.3 Imputation done in time: 3.2 secs Warning: Imputation is experimental and may have bugs, please report any issues on the GitHub repository. Imputing missing values with simpleImputer using: mean and missing threshold: 0.95 Calculating missingness in data Found 31 features with missing values Imputation done in time: 3.28 secs Applying imputation to test data with simpleImputer using method: mean and missing threshold: 0.95 Imputation done in time: 2.76 secs Warning: Imputation is experimental and may have bugs, please report any issues on the GitHub repository. Imputing missing values with simpleImputer using: mean and missing threshold: 0.3 Calculating missingness in data Found 2 features with missing values Imputation done in time: 3.1 secs Warning: Imputation is experimental and may have bugs. Please report any issues on the GitHub repository. Imputing missing values with iterativeImputer using: pmm, missing threshold: 0.3 and method settings: 111 Calculating missingness in data Found 2 features with missing values Imputation iteration: 1 Imputing variable: fakeMissingVariable Imputation done in time: 6.1 secs Applying imputation to test data with iterativeImputer using method: pmm and missing threshold: 0.3 Imputing variable: fakeMissingVariable PMM predict for variable fakeMissingVariable (covariateId=666): missing=32, donors=64, k=1, time=0.223 secs Imputation done in time: 5.44 secs Warning: Imputation is experimental and may have bugs. Please report any issues on the GitHub repository. Applying imputation to test data with iterativeImputer using method: pmm and missing threshold: 0.3 Imputation done in time: 2.96 secs Warning: Imputation is experimental and may have bugs. Please report any issues on the GitHub repository. Applying imputation to test data with iterativeImputer using method: pmm and missing threshold: 0.8 Imputing variable: bmi PMM predict for variable bmi (covariateId=100): missing=2, donors=2, k=3, time=0.271 secs Imputation done in time: 4.8 secs Warning: Imputation model only has intercept. It does not fit the data well Use timeStamp: TRUE Creating save directory at: D:\temp\2026_03_09_15_30_17_16713\Rtmp6HcNzx\saveLoc6738479674bb/lcc/learningCurve_1 Patient-Level Prediction Package version 6.6.0 Study started at: 2026-03-09 15:35:33.503546 AnalysisID: learningCurve_1 AnalysisName: Study details TargetID: 1 OutcomeID: 3 Cohort size: 1800 Covariates: 75 Creating population Outcome is 0 or 1 Population created with: 1767 observations, 1767 unique subjects and 355 outcomes Population created in 0.0646 secs seed: 70165 Creating a 20% test and 60% train (into 2 folds) random stratified split by class Data split into 353 test cases and 1061 train cases (531, 530) 353 were not used for training or testing Data split in 2.59 secs Train Set: Fold 1 531 patients with 107 outcomes - Fold 2 530 patients with 106 outcomes 75 covariates in train data Test Set: 353 patients with 71 outcomes Removing 2 redundant covariates Removing 0 infrequent covariates Normalizing covariates Tidying covariates took 2.65 secs Train Set: Fold 1 531 patients with 107 outcomes - Fold 2 530 patients with 106 outcomes 73 covariates in train data Test Set: 353 patients with 71 outcomes Running Cyclops Done. GLM fit status: OK Creating variable importance data frame Prediction took 0.256 secs Time to fit model: 1.09 secs Removing infrequent and redundant covariates and normalizing Removing infrequent and redundant covariates covariates and normalizing took 0.799 secs Prediction took 0.218 secs Prediction done in: 1.83 secs Calculating Performance for Test ============= AUC 70.65 95% lower AUC: 63.83 95% upper AUC: 77.47 AUPRC: 32.86 Brier: 0.14 Eavg: 0.01 Calibration in large- Mean predicted risk 0.1994 : observed risk 0.2011 Calibration in large- Intercept 0.1296 Weak calibration intercept: 0.1296 - gradient:1.0968 Hosmer-Lemeshow calibration gradient: 1.09 intercept: -0.02 Average Precision: 0.35 Calculating Performance for Train ============= AUC 74.41 95% lower AUC: 70.83 95% upper AUC: 77.99 AUPRC: 40.35 Brier: 0.14 Eavg: 0.03 Calibration in large- Mean predicted risk 0.2008 : observed risk 0.2008 Calibration in large- Intercept 0.1786 Weak calibration intercept: 0.1786 - gradient:1.1495 Hosmer-Lemeshow calibration gradient: 1.25 intercept: -0.04 Average Precision: 0.41 Calculating Performance for CV ============= AUC 69.66 95% lower AUC: 65.74 95% upper AUC: 73.58 AUPRC: 32.83 Brier: 0.14 Eavg: 0.03 Calibration in large- Mean predicted risk 0.2012 : observed risk 0.2008 Calibration in large- Intercept 0.2838 Weak calibration intercept: 0.2838 - gradient:1.2355 Hosmer-Lemeshow calibration gradient: 1.24 intercept: -0.05 Average Precision: 0.31 Time to calculate evaluation metrics: 0.356 secs Run finished successfully. Saving PlpResult Creating directory to save model plpResult saved to ..\D:\temp\2026_03_09_15_30_17_16713\Rtmp6HcNzx\saveLoc6738479674bb/lcc/learningCurve_1\plpResult runPlp time taken: 9.19 secs Creating save directory at: D:\temp\2026_03_09_15_30_17_16713\Rtmp6HcNzx\saveLoc6738479674bb/lcc/learningCurve_2 Patient-Level Prediction Package version 6.6.0 Study started at: 2026-03-09 15:35:42.701184 AnalysisID: learningCurve_2 AnalysisName: Study details TargetID: 1 OutcomeID: 3 Cohort size: 1800 Covariates: 75 Creating population Outcome is 0 or 1 Population created with: 1767 observations, 1767 unique subjects and 355 outcomes Population created in 0.0607 secs seed: 70165 Creating a 20% test and 70% train (into 2 folds) random stratified split by class Data split into 353 test cases and 1237 train cases (619, 618) 177 were not used for training or testing Data split in 2.54 secs Train Set: Fold 1 619 patients with 124 outcomes - Fold 2 618 patients with 124 outcomes 75 covariates in train data Test Set: 353 patients with 71 outcomes Removing 2 redundant covariates Removing 0 infrequent covariates Normalizing covariates Tidying covariates took 2.68 secs Train Set: Fold 1 619 patients with 124 outcomes - Fold 2 618 patients with 124 outcomes 73 covariates in train data Test Set: 353 patients with 71 outcomes Running Cyclops Done. GLM fit status: OK Creating variable importance data frame Prediction took 0.227 secs Time to fit model: 1.55 secs Removing infrequent and redundant covariates and normalizing Removing infrequent and redundant covariates covariates and normalizing took 0.779 secs Prediction took 0.21 secs Prediction done in: 1.78 secs Calculating Performance for Test ============= AUC 69.52 95% lower AUC: 62.68 95% upper AUC: 76.35 AUPRC: 30.92 Brier: 0.14 Eavg: 0.01 Calibration in large- Mean predicted risk 0.1987 : observed risk 0.2011 Calibration in large- Intercept 0.0621 Weak calibration intercept: 0.0621 - gradient:1.0373 Hosmer-Lemeshow calibration gradient: 0.90 intercept: 0.02 Average Precision: 0.33 Calculating Performance for Train ============= AUC 74.47 95% lower AUC: 71.11 95% upper AUC: 77.83 AUPRC: 40.88 Brier: 0.14 Eavg: 0.02 Calibration in large- Mean predicted risk 0.2005 : observed risk 0.2005 Calibration in large- Intercept 0.1642 Weak calibration intercept: 0.1642 - gradient:1.1391 Hosmer-Lemeshow calibration gradient: 1.26 intercept: -0.04 Average Precision: 0.41 Calculating Performance for CV ============= AUC 70.17 95% lower AUC: 66.55 95% upper AUC: 73.79 AUPRC: 33.71 Brier: 0.14 Eavg: 0.02 Calibration in large- Mean predicted risk 0.2004 : observed risk 0.2005 Calibration in large- Intercept 0.2634 Weak calibration intercept: 0.2634 - gradient:1.2179 Hosmer-Lemeshow calibration gradient: 1.24 intercept: -0.05 Average Precision: 0.33 Time to calculate evaluation metrics: 0.363 secs Run finished successfully. Saving PlpResult Creating directory to save model plpResult saved to ..\D:\temp\2026_03_09_15_30_17_16713\Rtmp6HcNzx\saveLoc6738479674bb/lcc/learningCurve_2\plpResult runPlp time taken: 9.57 secs Finished in 19 secs. type should be minmax or robust type should be minmax or robust type should be minmax or robust type should be minmax or robust settings$clip needs to be a boolean Starting min-max normalization of continuous features Finished min-max normalization of continuous features in 0.832 secs Applying min-max normalization of continuous features to test data Finished min-max normalization of continuous features in 0.599 secs Starting robust normalization of continuous features Finished robust normalization in 0.933 secs Applying robust normalization of continuous features to test data Finished robust normalization in 0.678 secs Starting robust normalization of continuous features Finished robust normalization in 0.922 secs Applying robust normalization of continuous features to test data Finished robust normalization in 0.626 secs Starting min-max normalization of continuous features Finished min-max normalization of continuous features in 0.837 secs Applying min-max normalization of continuous features to test data Finished min-max normalization of continuous features in 0.601 secs Starting robust normalization of continuous features Finished robust normalization in 0.918 secs Applying robust normalization of continuous features to test data Finished robust normalization in 0.79 secs 1 needs to be a boolean "tertet" needs to be a boolean 1 needs to be >= 2 "tertet" needs to be >= 1 2 needs to be <= 1 "tertet" needs to be <= 1 1 needs to be > 2 0 needs to be > 0 "tertet" needs to be > 1 2 needs to be < 1 0 needs to be < 0 "tertet" needs to be < 1 NULL cannot be empty "dsdsds" should be of class:double "dsdsds" should be dsds or double Creating save directory at: D:\temp\2026_03_09_15_30_17_16713\Rtmp6HcNzx\file67383c892941 file does not exist Column names of data.frame(a = 1:2, b = 1:2) are not correct Column types of data.frame(a = 1:2, b = 1:2) are not correct includeAllOutcomes should be of class:logical includeAllOutcomes should be of class:logical firstExposureOnly should be of class:logical firstExposureOnly should be of class:logical washoutPeriod should be of class:numericwashoutPeriod should be of class:integer washoutPeriod needs to be >= 0 removeSubjectsWithPriorOutcome should be of class:logical removeSubjectsWithPriorOutcome should be of class:logical priorOutcomeLookback should be of class:numericpriorOutcomeLookback should be of class:integer priorOutcomeLookback needs to be >= 0 requireTimeAtRisk should be of class:logical requireTimeAtRisk should be of class:logical minTimeAtRisk should be of class:numericminTimeAtRisk should be of class:integer minTimeAtRisk needs to be >= 0 riskWindowStart should be of class:numericriskWindowStart should be of class:integer riskWindowEnd should be of class:numericriskWindowEnd should be of class:integer startAnchor should be of class:character endAnchor should be of class:character minFraction needs to be >= 0 minFraction should be of class:numericminFraction should be of class:integer removeRedundancy should be of class:logical removeRedundancy should be of class:logical normalize should be of class:logical normalize should be of class:logical Removing infrequent and redundant covariates and normalizing Removing infrequent and redundant covariates covariates and normalizing took 0.643 secs recal initial baseline hazard: 0.8 recal initial offset: 0.15 recal initial timepoint: 220 recal initial baseline hazard: 0.85 recal initial offset: -0.05 recal initial timepoint: 240 recal initial baseline hazard: 0.9 recal initial offset: 0 recal initial timepoint: 365 runSplitData should be of class:logical runSampleData should be of class:logical runFeatureEngineering should be of class:logical runPreprocessData should be of class:logical runModelDevelopment should be of class:logical runCovariateSummary should be of class:logical numberOutcomestoNonOutcomes should be of class:numericnumberOutcomestoNonOutcomes should be of class:integer numberOutcomestoNonOutcomes needs to be > 0 sampleSeed should be of class:numericsampleSeed should be of class:integer type should be of class:character plpData object At risk concept ID: Outcome concept ID(s): plpData object summary At risk cohort concept ID: Outcome concept ID(s): People: Outcome counts: Creating directory to save model Computing covariate prevalence Fitting outcome model(s) Generating covariates Generating cohorts Generating outcomes Warning: restrictPlpDataSettings are not the same in models and validationDesign, using from design restrictPlpDataSettings not set in design, using model's targetId should be of class:numerictargetId should be of class:integer outcomeId should be of class:numericoutcomeId should be of class:integer targetId should be of class:numerictargetId should be of class:integer outcomeId should be of class:numericoutcomeId should be of class:integer populationSettings should be of class:populationSettings recalibrate should be of class:characterrecalibrate should be of class:NULL runCovariateSummary should be of class:logical [ FAIL 0 | WARN 0 | SKIP 133 | PASS 899 ] ══ Skipped tests (133) ═════════════════════════════════════════════════════════ • On CRAN (129): 'test-LightGBM.R:18:3', 'test-LightGBM.R:54:3', 'test-LightGBM.R:75:3', 'test-UploadToDatabase.R:202:3', 'test-UploadToDatabase.R:276:3', 'test-UploadToDatabase.R:377:3', 'test-UploadToDatabase.R:424:3', 'test-UploadToDatabase.R:488:3', 'test-UploadToDatabase.R:543:3', 'test-UploadToDatabase.R:651:3', 'test-andromedahelperfunctions.R:19:3', 'test-compatModelSettings.R:248:3', 'test-compatModelSettings.R:303:3', 'test-compatModelSettings.R:367:3', 'test-cyclopsModels.R:159:3', 'test-cyclopsModels.R:216:3', 'test-cyclopsModels.R:233:3', 'test-cyclopsModels.R:293:3', 'test-cyclopsModels.R:305:3', 'test-cyclopsModels.R:342:3', 'test-dataSplitting.R:129:3', 'test-dataSplitting.R:232:3', 'test-dataSplitting.R:426:3', 'test-diagnostic.R:60:3', 'test-diagnostic.R:111:3', 'test-evaluation.R:17:3', 'test-evaluation.R:27:3', 'test-existingModel.R:18:3', 'test-existingModel.R:74:3', 'test-existingModel.R:124:3', 'test-extractData.R:17:3', 'test-extractData.R:43:3', 'test-extractData.R:62:3', 'test-extractData.R:93:3', 'test-extractData.R:112:3', 'test-extractData.R:156:3', 'test-extractData.R:185:3', 'test-extractData.R:204:3', 'test-extractData.R:231:3', 'test-featureEngineering.R:40:3', 'test-featureEngineering.R:56:3', 'test-featureEngineering.R:84:3', 'test-featureEngineering.R:130:3', 'test-featureEngineering.R:158:3', 'test-featureEngineering.R:203:3', 'test-featureEngineering.R:248:3', 'test-featureEngineering.R:316:3', 'test-featureEngineering.R:343:3', 'test-featureImportance.R:19:3', 'test-featureImportance.R:45:3', 'test-fitting.R:20:3', 'test-fitting.R:32:3', 'test-formatting.R:103:3', 'test-helperfunctions.R:33:3', 'test-imputation.R:261:3', 'test-imputation.R:282:3', 'test-imputation.R:333:3', 'test-learningCurves.R:35:3', 'test-learningCurves.R:50:3', 'test-learningCurves.R:74:3', 'test-multiplePlp.R:86:3', 'test-multiplePlp.R:132:3', 'test-plotting.R:21:3', 'test-plotting.R:60:3', 'test-plotting.R:74:3', 'test-plotting.R:91:3', 'test-plotting.R:126:3', 'test-plotting.R:148:3', 'test-plotting.R:155:3', 'test-plotting.R:165:3', 'test-plotting.R:174:3', 'test-plotting.R:182:3', 'test-plotting.R:199:3', 'test-plotting.R:208:3', 'test-plotting.R:218:3', 'test-plotting.R:225:3', 'test-population.R:263:3', 'test-population.R:471:3', 'test-prediction.R:18:3', 'test-prediction.R:39:3', 'test-prediction.R:78:3', 'test-preprocessingData.R:56:3', 'test-preprocessingData.R:118:3', 'test-rclassifier.R:18:3', 'test-rclassifier.R:51:3', 'test-rclassifier.R:71:3', 'test-rclassifier.R:134:3', 'test-recalibration.R:61:3', 'test-runPlpHelpers.R:18:3', 'test-runPlpHelpers.R:32:3', 'test-sampling.R:91:3', 'test-sampling.R:132:3', 'test-sampling.R:168:3', 'test-saveloadplp.R:31:3', 'test-saveloadplp.R:46:3', 'test-saveloadplp.R:87:3', 'test-saveloadplp.R:171:3', 'test-sklearnClassifier.R:19:3', 'test-sklearnClassifier.R:43:3', 'test-sklearnClassifier.R:54:3', 'test-sklearnClassifier.R:117:3', 'test-sklearnClassifier.R:132:3', 'test-sklearnClassifier.R:164:3', 'test-sklearnClassifier.R:201:3', 'test-sklearnClassifier.R:237:3', 'test-sklearnClassifier.R:281:3', 'test-sklearnClassifier.R:301:3', 'test-sklearnClassifierSettings.R:19:3', 'test-sklearnClassifierSettings.R:37:3', 'test-sklearnClassifierSettings.R:46:3', 'test-sklearnClassifierSettings.R:81:3', 'test-sklearnClassifierSettings.R:94:3', 'test-sklearnClassifierSettings.R:122:3', 'test-sklearnJson.R:39:3', 'test-sklearnJson.R:57:3', 'test-sklearnJson.R:78:3', 'test-sklearnJson.R:99:3', 'test-sklearnJson.R:117:3', 'test-sklearnJson.R:136:3', 'test-validation.R:44:3', 'test-validation.R:68:3', 'test-validation.R:201:3', 'test-validation.R:226:3', 'test-validation.R:241:3', 'test-validation.R:262:3', 'test-validation.R:287:3', 'test-validation.R:315:3', 'test-validation.R:345:3', 'test-validation.R:368:3' • only run on CI (4): 'test-UploadToDatabase.R:39:3', 'test-UploadToDatabase.R:92:3', 'test-UploadToDatabase.R:113:3', 'test-UploadToDatabase.R:163:3' [ FAIL 0 | WARN 0 | SKIP 133 | PASS 899 ] > > proc.time() user system elapsed 133.06 49.42 163.71