R Under development (unstable) (2024-08-17 r87027 ucrt) -- "Unsuffered Consequences" Copyright (C) 2024 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > library(testthat) > library(rmcfs) Loading required package: rJava ####################### # rmcfs version 1.3.6 # ####################### If used please cite the following paper: M. Draminski, J. Koronacki (2018), rmcfs: An R Package for Monte Carlo Feature Selection and Interdependency Discovery, Journal of Statistical Software, vol 85(12), 1-28, doi:10.18637/jss.v085.i12. > test_check("rmcfs") X1 X2 X3 X4 X5 X6 X7 1 0.2917485 0.05007404 0.63722513 0.81284287 0.24768797 0.10302293 0.42749840 2 0.5093811 0.73081669 0.87650853 0.55987782 0.70211720 0.04760038 0.44939897 3 0.2447372 0.32487502 0.39269819 0.47413384 0.33249650 0.24936386 0.97692750 4 0.4719058 0.22835946 0.03222088 0.37995215 0.39201457 0.31186902 0.04527567 5 0.9985390 0.73319132 0.51511243 0.02804263 0.42547346 0.57599450 0.94067911 6 0.1561214 0.35289464 0.15437972 0.96770362 0.09978626 0.29131655 0.88521533 7 0.2748429 0.13611602 0.45229359 0.43218403 0.97692007 0.86964367 0.89450875 8 0.9001874 0.07358815 0.55434004 0.36547454 0.45782693 0.68718863 0.93914107 9 0.6887313 0.41446280 0.47208878 0.79889817 0.62377673 0.57009568 0.31589888 10 0.9346415 0.28162283 0.03732576 0.89862017 0.07954690 0.54044162 0.36858871 X8 X9 X10 1 0.29601284 0.65794311 0.302052374 2 0.08734771 0.01663852 0.244205297 3 0.87093554 0.70830950 0.591712025 4 0.35389467 0.68097155 0.772607348 5 0.90273537 0.27080171 0.393339856 6 0.57123117 0.16006018 0.748494918 7 0.61079152 0.58840529 0.659639413 8 0.20231058 0.17915699 0.115445700 9 0.38494244 0.16901436 0.136330316 10 0.20096750 0.77777717 0.005234786 X7 X8 X9 X10 A1 A2 B1 B2 C1 C2 class 60 0.2188540 0.362559234 0.81554060 0.3450043 0 0 B B 0 0 B 61 0.3035803 0.088113909 0.99780583 0.2324606 0 0 0 0 C C C 62 0.7999598 0.908288863 0.47630133 0.2763702 0 0 0 0 0 0 C 63 0.5125571 0.925099193 0.95238666 0.9719202 0 0 0 0 0 0 C 64 0.9607326 0.283837598 0.76852704 0.3717665 0 0 0 0 C C C 65 0.1372308 0.709634187 0.82737850 0.3071373 0 0 0 0 0 0 C 66 0.9580729 0.366068508 0.01305201 0.9536664 0 0 0 0 C C C 67 0.7301433 0.001147732 0.19792949 0.7288264 0 0 0 0 0 0 C 68 0.1828031 0.754164543 0.07285060 0.7480717 0 0 0 0 C C C 69 0.4406439 0.112393832 0.93446333 0.2075628 0 0 0 0 C C C 70 0.6025564 0.653734100 0.69801836 0.9467070 0 0 0 0 C C C class: 'data.frame' size: 70 x 17Checking input data... Exporting params... Exporting input data... Running MCFS-ID... ################################################## ##### dmLab 2.3.6 [2024.08.18] ##### ################################################## Created by Michal Draminski [michal.draminski@ipipan.waw.pl] http://www.ipipan.eu/staff/m.draminski/ Polish Academy of Sciences - Institute of Computer Science ################################################## *************************************************** *** MCFS-ID Cutoff Permutation Experiment #1/3 *** *************************************************** Loading header: 'input.adh'... Loading data: 'input.csv'... 70 objects and 17 attributes to load... Done MEMORY Status - free: 0.01G used: 0.52G total: 0.01G max: 0.53G Pearson's correlation of shuffled decision: -0.2000 Nominal target detected - using J48 model MCFS-ID param: ID-Graph is ON MCFS-ID param: balance classes is AUTO Classes = ["A", "B", "C"], Sizes = [40, 20, 10], classSizeRatio = 0.25, balanceValue = 1.0 Calculation of DecisionValuesTable... Starting MCFS-ID Procedure: projectionSize(m) = 4, projections(s) = 200, splits(t) = 5 Start time: Mon Aug 19 12:02:19 CEST 2024 Running: 2 threads... [ ] 0% Time: 00:00 ETA: --:-- [======> ] 10% Time: 00:00 ETA: --:-- [=============> ] 20% Time: 00:00 ETA: --:-- [====================> ] 30% Time: 00:00 ETA: --:-- [===========================> ] 40% Time: 00:00 ETA: --:-- [==================================> ] 50% Time: 00:00 ETA: --:-- [=========================================> ] 60% Time: 00:00 ETA: --:-- [================================================> ] 70% Time: 00:00 ETA: --:-- [=======================================================> ] 80% Time: 00:00 ETA: --:-- [==============================================================> ] 90% Time: 00:00 ETA: --:-- [=====================================================================>] 100% Time: 00:00 ETA: --:-- [=====================================================================>] 100% Time: 00:00 1000 trees built within 0.64 s. Prediction Summary on a Random Subsample (st): Accuracy = 51.14% WeightedAccuracy = 32.65% Cutoff RI (based on linear regression angle) = 0.0129476 Cutoff RI (based on k-means clustering) = 0.0129476 Cutoff RI (based on mean cutoff value) = 0.0111472 Important attributes (based on mean cutoff value) = 6 *************************************************** *** MCFS-ID Cutoff Permutation Experiment #2/3 *** *************************************************** Loading header: 'input.adh'... Loading data: 'input.csv'... 70 objects and 17 attributes to load... Done MEMORY Status - free: 0.01G used: 0.51G total: 0.02G max: 0.53G Pearson's correlation of shuffled decision: 0.0099 Nominal target detected - using J48 model MCFS-ID param: ID-Graph is ON MCFS-ID param: balance classes is AUTO Classes = ["A", "B", "C"], Sizes = [40, 20, 10], classSizeRatio = 0.25, balanceValue = 1.0 Calculation of DecisionValuesTable... Starting MCFS-ID Procedure: projectionSize(m) = 4, projections(s) = 200, splits(t) = 5 Start time: Mon Aug 19 12:02:20 CEST 2024 Running: 2 threads... [ ] 0% Time: 00:00 ETA: --:-- [======> ] 10% Time: 00:00 ETA: --:-- [=============> ] 20% Time: 00:00 ETA: --:-- [====================> ] 30% Time: 00:00 ETA: --:-- [===========================> ] 40% Time: 00:00 ETA: --:-- [==================================> ] 50% Time: 00:00 ETA: --:-- [=========================================> ] 60% Time: 00:00 ETA: --:-- [================================================> ] 70% Time: 00:00 ETA: --:-- [=======================================================> ] 80% Time: 00:00 ETA: --:-- [==============================================================> ] 90% Time: 00:00 ETA: --:-- [=====================================================================>] 100% Time: 00:00 ETA: --:-- [=====================================================================>] 100% Time: 00:00 1000 trees built within 0.25 s. Prediction Summary on a Random Subsample (st): Accuracy = 49.98% WeightedAccuracy = 33.77% Cutoff RI (based on linear regression angle) = 0.0287323 Cutoff RI (based on k-means clustering) = 0.0256081 Cutoff RI (based on mean cutoff value) = 0.0204526 Important attributes (based on mean cutoff value) = 5 *************************************************** *** MCFS-ID Cutoff Permutation Experiment #3/3 *** *************************************************** Loading header: 'input.adh'... Loading data: 'input.csv'... 70 objects and 17 attributes to load... Done MEMORY Status - free: 0.01G used: 0.51G total: 0.02G max: 0.53G Pearson's correlation of shuffled decision: -0.0399 Nominal target detected - using J48 model MCFS-ID param: ID-Graph is ON MCFS-ID param: balance classes is AUTO Classes = ["A", "B", "C"], Sizes = [40, 20, 10], classSizeRatio = 0.25, balanceValue = 1.0 Calculation of DecisionValuesTable... Starting MCFS-ID Procedure: projectionSize(m) = 4, projections(s) = 200, splits(t) = 5 Start time: Mon Aug 19 12:02:20 CEST 2024 Running: 2 threads... [ ] 0% Time: 00:00 ETA: --:-- [======> ] 10% Time: 00:00 ETA: --:-- [=============> ] 20% Time: 00:00 ETA: --:-- [====================> ] 30% Time: 00:00 ETA: --:-- [===========================> ] 40% Time: 00:00 ETA: --:-- [==================================> ] 50% Time: 00:00 ETA: --:-- [=========================================> ] 60% Time: 00:00 ETA: --:-- [================================================> ] 70% Time: 00:00 ETA: --:-- [=======================================================> ] 80% Time: 00:00 ETA: --:-- [==============================================================> ] 90% Time: 00:00 ETA: --:-- [=====================================================================>] 100% Time: 00:00 ETA: --:-- [=====================================================================>] 100% Time: 00:00 1000 trees built within 0.233 s. Prediction Summary on a Random Subsample (st): Accuracy = 51.67% WeightedAccuracy = 34.29% Cutoff RI (based on linear regression angle) = 0.0438692 Cutoff RI (based on k-means clustering) = 0.0360147 Cutoff RI (based on mean cutoff value) = 0.0249331 Important attributes (based on mean cutoff value) = 4 ************************** *** MCFS-ID Experiment *** ************************** Loading header: 'input.adh'... Loading data: 'input.csv'... 70 objects and 17 attributes to load... Done MEMORY Status - free: 0.01G used: 0.51G total: 0.02G max: 0.53G Nominal target detected - using J48 model MCFS-ID param: ID-Graph is ON MCFS-ID param: balance classes is AUTO Classes = ["A", "B", "C"], Sizes = [40, 20, 10], classSizeRatio = 0.25, balanceValue = 1.0 Calculation of DecisionValuesTable... Starting MCFS-ID Procedure: projectionSize(m) = 4, projections(s) = 200, splits(t) = 5 Start time: Mon Aug 19 12:02:20 CEST 2024 Running: 2 threads... [ ] 0% Time: 00:00 ETA: --:-- [======> ] 10% Time: 00:00 ETA: --:-- [=============> ] 20% Time: 00:00 ETA: --:-- [====================> ] 30% Time: 00:00 ETA: --:-- [===========================> ] 40% Time: 00:00 ETA: --:-- [==================================> ] 50% Time: 00:00 ETA: --:-- [=========================================> ] 60% Time: 00:00 ETA: --:-- [================================================> ] 70% Time: 00:00 ETA: --:-- [=======================================================> ] 80% Time: 00:00 ETA: --:-- [==============================================================> ] 90% Time: 00:00 ETA: --:-- [=====================================================================>] 100% Time: 00:00 ETA: --:-- [=====================================================================>] 100% Time: 00:00 1000 trees built within 0.157 s. Prediction Summary on a Random Subsample (st): Accuracy = 79.44% WeightedAccuracy = 68.97% Cutoff RI (based on linear regression angle) = 0.0168928 Cutoff RI (based on k-means clustering) = 0.3882650 Cutoff RI (based on mean cutoff value) = 0.0168928 Important attributes (based on mean cutoff value) = 6 *** Calculation of cutoff RI (based on permutations) *** Max RI (raw data) = 0.7200021 Max RI (after permutations) = [0.023476448, 0.037551068, 0.048657004] Anderson-Darling normality test p-value = 0.6064343 Confidence Interval: 0.0052131 ; 0.0679098 Cutoff RI (based on permutations) = 0.0679098 Important attributes (based on permutations) = 6 *** Calculation of cutoff ID *** Anderson-Darling normality test p-value = 0.5987068 Confidence Interval: 1.1080772 ; 15.0552321 Cutoff ID (based on permutations) = 15.0552321 *** Final Important attributes (based on permutations) = 6 *** MCFS-ID Processing is done. Time: 1.5 s. *** Reading results... Done. ##### MCFS-ID result (s = auto, t = 5, m = auto) ##### Target feature: 'class' Top 6 features: position attribute RI 1 A2 0.7200021 2 A1 0.6843487 3 B2 0.4490495 4 B1 0.3882650 5 C2 0.2902232 6 C1 0.2447448 ################################# Cutoff values: method minRI size minID criticalAngle 0.01689289 7 NA kmeans 0.38826501 4 NA permutations 0.06790986 6 15.05523 mean 0.01689289 6 NA ################################# Confusion matrix obtained on randomly selected (st) datasets: Confusion Matrix: A B C A 13417 530 193 B 1789 4942 339 C 1446 892 1702 TPR (sensitivity/recall): TPR 1 94.9 % 2 69.9 % 3 42.1 % Accuracy: 79.4 % wAccuracy: 69 % ################################# MCFS-ID execution time: 2 secs method minRI size minID 1 criticalAngle 0.01689289 7 NA 2 kmeans 0.38826501 4 NA 3 permutations 0.06790986 6 15.05523 4 mean 0.01689289 6 NA [1] 6 projection distance commonPart mAvg beta1 1 30 1.000 1 0 0 2 40 0.750 1 0 0 3 50 0.500 1 0 0 4 60 0.125 1 0 0 5 70 0.250 1 0 0 6 80 0.375 1 0 0 7 90 0.125 1 0 0 8 100 0.000 1 0 0 9 110 0.125 1 0 0 10 120 0.000 1 0 0 11 130 0.250 1 0 0 12 140 0.125 1 0 0 13 150 0.000 1 0 0 14 160 0.000 1 0 0 15 170 0.000 1 0 0 16 180 0.125 1 0 0 17 190 0.125 1 0 0 18 200 0.250 1 0 0 position attribute projections classifiers nodes RI 12 1 A2 50 0.8680000 0.8680000 0.720002100 11 2 A1 52 0.8576923 0.8576923 0.684348700 14 3 B2 62 0.8645161 0.8645161 0.449049530 13 4 B1 50 0.8600000 0.8600000 0.388265000 16 5 C2 50 0.8360000 0.8360000 0.290223180 15 6 C1 49 0.7591836 0.7591836 0.244744840 4 7 X4 43 0.3441860 0.4744186 0.016892891 8 8 X8 50 0.2840000 0.4480000 0.013558985 6 9 X6 46 0.2652174 0.4086956 0.012915965 1 10 X1 52 0.3038461 0.4384615 0.012872161 3 11 X3 45 0.3555556 0.6088889 0.012565950 5 12 X5 58 0.1896552 0.2896552 0.006976671 9 13 X9 44 0.1636364 0.1909091 0.006783004 2 14 X2 52 0.1769231 0.2500000 0.005961394 7 15 X7 46 0.1304348 0.2043478 0.004842349 10 16 X10 59 0.1288136 0.2067797 0.004818176 position edge_a edge_b weight 1 1 B2 C2 28.105164 2 2 B2 C1 21.325325 3 3 A2 B1 16.477222 4 4 A2 B2 15.275316 5 5 A1 B1 10.628960 6 6 B1 C2 10.463650 7 7 A1 B2 7.718126 8 8 A1 C1 7.698044 9 9 B1 C1 7.210217 10 10 A1 C2 5.916451 11 11 A2 C1 4.707914 12 12 X8 X2 2.997086 13 13 X3 X2 2.993449 14 14 A2 C2 2.974209 15 15 X3 X10 2.755518 16 16 C1 X6 2.300357 17 17 B2 X6 2.233918 18 18 B1 X1 2.225656 19 19 B1 A1 2.133333 20 20 X4 X2 2.060968 21 21 C2 X4 1.931806 22 22 X6 X7 1.919075 23 23 X3 X4 1.876069 24 24 X1 X5 1.874010 25 25 X5 X3 1.864367 26 26 X3 X7 1.862037 27 27 X3 X6 1.825284 28 28 X3 X1 1.775435 29 29 X1 X3 1.724768 30 30 C1 X4 1.626060 31 31 X2 X6 1.566419 32 32 X5 X1 1.539928 33 33 X8 X7 1.515524 34 34 C2 X5 1.515325 35 35 C1 X1 1.500005 36 36 X3 X9 1.494811 37 37 X6 X10 1.457399 38 38 C1 X5 1.445549 39 39 X1 X10 1.437541 40 40 X2 X8 1.431766 41 41 X6 X5 1.430042 42 42 X4 X3 1.424474 43 43 X10 X1 1.419026 44 44 C2 X1 1.358736 45 45 C2 B2 1.356040 46 46 A2 X3 1.353772 47 47 B1 X6 1.334239 48 48 X2 X3 1.316596 49 49 C1 X8 1.307927 50 50 X1 X9 1.265500 Selected 6 nodes and 16 edges. Selected 6 nodes and 12 edges. [ FAIL 0 | WARN 2 | SKIP 8 | PASS 0 ] ══ Skipped tests (8) ═══════════════════════════════════════════════════════════ • On CRAN (7): 'test-io.R:7:3', 'test-man.build.idgraph.R:5:3', 'test-man.plot.idgraph.R:5:3', 'test-mcfs.M5.R:7:3', 'test-mcfs.R:7:3', 'test-mcfs.R:36:3', 'test-mcfs.mode2.R:7:3' • empty test (1): 'test-man.mcfs.R:4:1' [ FAIL 0 | WARN 2 | SKIP 8 | PASS 0 ] > > proc.time() user system elapsed 16.06 1.64 5.46