R Under development (unstable) (2024-08-17 r87027 ucrt) -- "Unsuffered Consequences"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> library(testthat)
> library(rmcfs)
Loading required package: rJava
#######################
# rmcfs version 1.3.6 #
#######################
If used please cite the following paper:
M. Draminski, J. Koronacki (2018),
rmcfs: An R Package for Monte Carlo Feature Selection and Interdependency Discovery,
Journal of Statistical Software, vol 85(12), 1-28, doi:10.18637/jss.v085.i12.
> test_check("rmcfs")
X1 X2 X3 X4 X5 X6 X7
1 0.2917485 0.05007404 0.63722513 0.81284287 0.24768797 0.10302293 0.42749840
2 0.5093811 0.73081669 0.87650853 0.55987782 0.70211720 0.04760038 0.44939897
3 0.2447372 0.32487502 0.39269819 0.47413384 0.33249650 0.24936386 0.97692750
4 0.4719058 0.22835946 0.03222088 0.37995215 0.39201457 0.31186902 0.04527567
5 0.9985390 0.73319132 0.51511243 0.02804263 0.42547346 0.57599450 0.94067911
6 0.1561214 0.35289464 0.15437972 0.96770362 0.09978626 0.29131655 0.88521533
7 0.2748429 0.13611602 0.45229359 0.43218403 0.97692007 0.86964367 0.89450875
8 0.9001874 0.07358815 0.55434004 0.36547454 0.45782693 0.68718863 0.93914107
9 0.6887313 0.41446280 0.47208878 0.79889817 0.62377673 0.57009568 0.31589888
10 0.9346415 0.28162283 0.03732576 0.89862017 0.07954690 0.54044162 0.36858871
X8 X9 X10
1 0.29601284 0.65794311 0.302052374
2 0.08734771 0.01663852 0.244205297
3 0.87093554 0.70830950 0.591712025
4 0.35389467 0.68097155 0.772607348
5 0.90273537 0.27080171 0.393339856
6 0.57123117 0.16006018 0.748494918
7 0.61079152 0.58840529 0.659639413
8 0.20231058 0.17915699 0.115445700
9 0.38494244 0.16901436 0.136330316
10 0.20096750 0.77777717 0.005234786
X7 X8 X9 X10 A1 A2 B1 B2 C1 C2 class
60 0.2188540 0.362559234 0.81554060 0.3450043 0 0 B B 0 0 B
61 0.3035803 0.088113909 0.99780583 0.2324606 0 0 0 0 C C C
62 0.7999598 0.908288863 0.47630133 0.2763702 0 0 0 0 0 0 C
63 0.5125571 0.925099193 0.95238666 0.9719202 0 0 0 0 0 0 C
64 0.9607326 0.283837598 0.76852704 0.3717665 0 0 0 0 C C C
65 0.1372308 0.709634187 0.82737850 0.3071373 0 0 0 0 0 0 C
66 0.9580729 0.366068508 0.01305201 0.9536664 0 0 0 0 C C C
67 0.7301433 0.001147732 0.19792949 0.7288264 0 0 0 0 0 0 C
68 0.1828031 0.754164543 0.07285060 0.7480717 0 0 0 0 C C C
69 0.4406439 0.112393832 0.93446333 0.2075628 0 0 0 0 C C C
70 0.6025564 0.653734100 0.69801836 0.9467070 0 0 0 0 C C C
class: 'data.frame' size: 70 x 17Checking input data...
Exporting params...
Exporting input data...
Running MCFS-ID...
##################################################
##### dmLab 2.3.6 [2024.08.18] #####
##################################################
Created by Michal Draminski [michal.draminski@ipipan.waw.pl]
http://www.ipipan.eu/staff/m.draminski/
Polish Academy of Sciences - Institute of Computer Science
##################################################
***************************************************
*** MCFS-ID Cutoff Permutation Experiment #1/3 ***
***************************************************
Loading header: 'input.adh'...
Loading data: 'input.csv'...
70 objects and 17 attributes to load... Done
MEMORY Status - free: 0.01G used: 0.52G total: 0.01G max: 0.53G
Pearson's correlation of shuffled decision: -0.2000
Nominal target detected - using J48 model
MCFS-ID param: ID-Graph is ON
MCFS-ID param: balance classes is AUTO
Classes = ["A", "B", "C"], Sizes = [40, 20, 10], classSizeRatio = 0.25, balanceValue = 1.0
Calculation of DecisionValuesTable...
Starting MCFS-ID Procedure: projectionSize(m) = 4, projections(s) = 200, splits(t) = 5
Start time: Mon Aug 19 12:02:19 CEST 2024
Running: 2 threads...
[ ] 0% Time: 00:00 ETA: --:--
[======> ] 10% Time: 00:00 ETA: --:--
[=============> ] 20% Time: 00:00 ETA: --:--
[====================> ] 30% Time: 00:00 ETA: --:--
[===========================> ] 40% Time: 00:00 ETA: --:--
[==================================> ] 50% Time: 00:00 ETA: --:--
[=========================================> ] 60% Time: 00:00 ETA: --:--
[================================================> ] 70% Time: 00:00 ETA: --:--
[=======================================================> ] 80% Time: 00:00 ETA: --:--
[==============================================================> ] 90% Time: 00:00 ETA: --:--
[=====================================================================>] 100% Time: 00:00 ETA: --:--
[=====================================================================>] 100% Time: 00:00
1000 trees built within 0.64 s.
Prediction Summary on a Random Subsample (st):
Accuracy = 51.14%
WeightedAccuracy = 32.65%
Cutoff RI (based on linear regression angle) = 0.0129476
Cutoff RI (based on k-means clustering) = 0.0129476
Cutoff RI (based on mean cutoff value) = 0.0111472
Important attributes (based on mean cutoff value) = 6
***************************************************
*** MCFS-ID Cutoff Permutation Experiment #2/3 ***
***************************************************
Loading header: 'input.adh'...
Loading data: 'input.csv'...
70 objects and 17 attributes to load... Done
MEMORY Status - free: 0.01G used: 0.51G total: 0.02G max: 0.53G
Pearson's correlation of shuffled decision: 0.0099
Nominal target detected - using J48 model
MCFS-ID param: ID-Graph is ON
MCFS-ID param: balance classes is AUTO
Classes = ["A", "B", "C"], Sizes = [40, 20, 10], classSizeRatio = 0.25, balanceValue = 1.0
Calculation of DecisionValuesTable...
Starting MCFS-ID Procedure: projectionSize(m) = 4, projections(s) = 200, splits(t) = 5
Start time: Mon Aug 19 12:02:20 CEST 2024
Running: 2 threads...
[ ] 0% Time: 00:00 ETA: --:--
[======> ] 10% Time: 00:00 ETA: --:--
[=============> ] 20% Time: 00:00 ETA: --:--
[====================> ] 30% Time: 00:00 ETA: --:--
[===========================> ] 40% Time: 00:00 ETA: --:--
[==================================> ] 50% Time: 00:00 ETA: --:--
[=========================================> ] 60% Time: 00:00 ETA: --:--
[================================================> ] 70% Time: 00:00 ETA: --:--
[=======================================================> ] 80% Time: 00:00 ETA: --:--
[==============================================================> ] 90% Time: 00:00 ETA: --:--
[=====================================================================>] 100% Time: 00:00 ETA: --:--
[=====================================================================>] 100% Time: 00:00
1000 trees built within 0.25 s.
Prediction Summary on a Random Subsample (st):
Accuracy = 49.98%
WeightedAccuracy = 33.77%
Cutoff RI (based on linear regression angle) = 0.0287323
Cutoff RI (based on k-means clustering) = 0.0256081
Cutoff RI (based on mean cutoff value) = 0.0204526
Important attributes (based on mean cutoff value) = 5
***************************************************
*** MCFS-ID Cutoff Permutation Experiment #3/3 ***
***************************************************
Loading header: 'input.adh'...
Loading data: 'input.csv'...
70 objects and 17 attributes to load... Done
MEMORY Status - free: 0.01G used: 0.51G total: 0.02G max: 0.53G
Pearson's correlation of shuffled decision: -0.0399
Nominal target detected - using J48 model
MCFS-ID param: ID-Graph is ON
MCFS-ID param: balance classes is AUTO
Classes = ["A", "B", "C"], Sizes = [40, 20, 10], classSizeRatio = 0.25, balanceValue = 1.0
Calculation of DecisionValuesTable...
Starting MCFS-ID Procedure: projectionSize(m) = 4, projections(s) = 200, splits(t) = 5
Start time: Mon Aug 19 12:02:20 CEST 2024
Running: 2 threads...
[ ] 0% Time: 00:00 ETA: --:--
[======> ] 10% Time: 00:00 ETA: --:--
[=============> ] 20% Time: 00:00 ETA: --:--
[====================> ] 30% Time: 00:00 ETA: --:--
[===========================> ] 40% Time: 00:00 ETA: --:--
[==================================> ] 50% Time: 00:00 ETA: --:--
[=========================================> ] 60% Time: 00:00 ETA: --:--
[================================================> ] 70% Time: 00:00 ETA: --:--
[=======================================================> ] 80% Time: 00:00 ETA: --:--
[==============================================================> ] 90% Time: 00:00 ETA: --:--
[=====================================================================>] 100% Time: 00:00 ETA: --:--
[=====================================================================>] 100% Time: 00:00
1000 trees built within 0.233 s.
Prediction Summary on a Random Subsample (st):
Accuracy = 51.67%
WeightedAccuracy = 34.29%
Cutoff RI (based on linear regression angle) = 0.0438692
Cutoff RI (based on k-means clustering) = 0.0360147
Cutoff RI (based on mean cutoff value) = 0.0249331
Important attributes (based on mean cutoff value) = 4
**************************
*** MCFS-ID Experiment ***
**************************
Loading header: 'input.adh'...
Loading data: 'input.csv'...
70 objects and 17 attributes to load... Done
MEMORY Status - free: 0.01G used: 0.51G total: 0.02G max: 0.53G
Nominal target detected - using J48 model
MCFS-ID param: ID-Graph is ON
MCFS-ID param: balance classes is AUTO
Classes = ["A", "B", "C"], Sizes = [40, 20, 10], classSizeRatio = 0.25, balanceValue = 1.0
Calculation of DecisionValuesTable...
Starting MCFS-ID Procedure: projectionSize(m) = 4, projections(s) = 200, splits(t) = 5
Start time: Mon Aug 19 12:02:20 CEST 2024
Running: 2 threads...
[ ] 0% Time: 00:00 ETA: --:--
[======> ] 10% Time: 00:00 ETA: --:--
[=============> ] 20% Time: 00:00 ETA: --:--
[====================> ] 30% Time: 00:00 ETA: --:--
[===========================> ] 40% Time: 00:00 ETA: --:--
[==================================> ] 50% Time: 00:00 ETA: --:--
[=========================================> ] 60% Time: 00:00 ETA: --:--
[================================================> ] 70% Time: 00:00 ETA: --:--
[=======================================================> ] 80% Time: 00:00 ETA: --:--
[==============================================================> ] 90% Time: 00:00 ETA: --:--
[=====================================================================>] 100% Time: 00:00 ETA: --:--
[=====================================================================>] 100% Time: 00:00
1000 trees built within 0.157 s.
Prediction Summary on a Random Subsample (st):
Accuracy = 79.44%
WeightedAccuracy = 68.97%
Cutoff RI (based on linear regression angle) = 0.0168928
Cutoff RI (based on k-means clustering) = 0.3882650
Cutoff RI (based on mean cutoff value) = 0.0168928
Important attributes (based on mean cutoff value) = 6
*** Calculation of cutoff RI (based on permutations) ***
Max RI (raw data) = 0.7200021
Max RI (after permutations) = [0.023476448, 0.037551068, 0.048657004]
Anderson-Darling normality test p-value = 0.6064343
Confidence Interval: 0.0052131 ; 0.0679098
Cutoff RI (based on permutations) = 0.0679098
Important attributes (based on permutations) = 6
*** Calculation of cutoff ID ***
Anderson-Darling normality test p-value = 0.5987068
Confidence Interval: 1.1080772 ; 15.0552321
Cutoff ID (based on permutations) = 15.0552321
*** Final Important attributes (based on permutations) = 6
*** MCFS-ID Processing is done. Time: 1.5 s. ***
Reading results...
Done.
##### MCFS-ID result (s = auto, t = 5, m = auto) #####
Target feature: 'class'
Top 6 features:
position attribute RI
1 A2 0.7200021
2 A1 0.6843487
3 B2 0.4490495
4 B1 0.3882650
5 C2 0.2902232
6 C1 0.2447448
#################################
Cutoff values:
method minRI size minID
criticalAngle 0.01689289 7 NA
kmeans 0.38826501 4 NA
permutations 0.06790986 6 15.05523
mean 0.01689289 6 NA
#################################
Confusion matrix obtained on randomly selected (st) datasets:
Confusion Matrix:
A B C
A 13417 530 193
B 1789 4942 339
C 1446 892 1702
TPR (sensitivity/recall):
TPR
1 94.9 %
2 69.9 %
3 42.1 %
Accuracy: 79.4 %
wAccuracy: 69 %
#################################
MCFS-ID execution time: 2 secs
method minRI size minID
1 criticalAngle 0.01689289 7 NA
2 kmeans 0.38826501 4 NA
3 permutations 0.06790986 6 15.05523
4 mean 0.01689289 6 NA
[1] 6
projection distance commonPart mAvg beta1
1 30 1.000 1 0 0
2 40 0.750 1 0 0
3 50 0.500 1 0 0
4 60 0.125 1 0 0
5 70 0.250 1 0 0
6 80 0.375 1 0 0
7 90 0.125 1 0 0
8 100 0.000 1 0 0
9 110 0.125 1 0 0
10 120 0.000 1 0 0
11 130 0.250 1 0 0
12 140 0.125 1 0 0
13 150 0.000 1 0 0
14 160 0.000 1 0 0
15 170 0.000 1 0 0
16 180 0.125 1 0 0
17 190 0.125 1 0 0
18 200 0.250 1 0 0
position attribute projections classifiers nodes RI
12 1 A2 50 0.8680000 0.8680000 0.720002100
11 2 A1 52 0.8576923 0.8576923 0.684348700
14 3 B2 62 0.8645161 0.8645161 0.449049530
13 4 B1 50 0.8600000 0.8600000 0.388265000
16 5 C2 50 0.8360000 0.8360000 0.290223180
15 6 C1 49 0.7591836 0.7591836 0.244744840
4 7 X4 43 0.3441860 0.4744186 0.016892891
8 8 X8 50 0.2840000 0.4480000 0.013558985
6 9 X6 46 0.2652174 0.4086956 0.012915965
1 10 X1 52 0.3038461 0.4384615 0.012872161
3 11 X3 45 0.3555556 0.6088889 0.012565950
5 12 X5 58 0.1896552 0.2896552 0.006976671
9 13 X9 44 0.1636364 0.1909091 0.006783004
2 14 X2 52 0.1769231 0.2500000 0.005961394
7 15 X7 46 0.1304348 0.2043478 0.004842349
10 16 X10 59 0.1288136 0.2067797 0.004818176
position edge_a edge_b weight
1 1 B2 C2 28.105164
2 2 B2 C1 21.325325
3 3 A2 B1 16.477222
4 4 A2 B2 15.275316
5 5 A1 B1 10.628960
6 6 B1 C2 10.463650
7 7 A1 B2 7.718126
8 8 A1 C1 7.698044
9 9 B1 C1 7.210217
10 10 A1 C2 5.916451
11 11 A2 C1 4.707914
12 12 X8 X2 2.997086
13 13 X3 X2 2.993449
14 14 A2 C2 2.974209
15 15 X3 X10 2.755518
16 16 C1 X6 2.300357
17 17 B2 X6 2.233918
18 18 B1 X1 2.225656
19 19 B1 A1 2.133333
20 20 X4 X2 2.060968
21 21 C2 X4 1.931806
22 22 X6 X7 1.919075
23 23 X3 X4 1.876069
24 24 X1 X5 1.874010
25 25 X5 X3 1.864367
26 26 X3 X7 1.862037
27 27 X3 X6 1.825284
28 28 X3 X1 1.775435
29 29 X1 X3 1.724768
30 30 C1 X4 1.626060
31 31 X2 X6 1.566419
32 32 X5 X1 1.539928
33 33 X8 X7 1.515524
34 34 C2 X5 1.515325
35 35 C1 X1 1.500005
36 36 X3 X9 1.494811
37 37 X6 X10 1.457399
38 38 C1 X5 1.445549
39 39 X1 X10 1.437541
40 40 X2 X8 1.431766
41 41 X6 X5 1.430042
42 42 X4 X3 1.424474
43 43 X10 X1 1.419026
44 44 C2 X1 1.358736
45 45 C2 B2 1.356040
46 46 A2 X3 1.353772
47 47 B1 X6 1.334239
48 48 X2 X3 1.316596
49 49 C1 X8 1.307927
50 50 X1 X9 1.265500
Selected 6 nodes and 16 edges.
Selected 6 nodes and 12 edges.
[ FAIL 0 | WARN 2 | SKIP 8 | PASS 0 ]
══ Skipped tests (8) ═══════════════════════════════════════════════════════════
• On CRAN (7): 'test-io.R:7:3', 'test-man.build.idgraph.R:5:3',
'test-man.plot.idgraph.R:5:3', 'test-mcfs.M5.R:7:3', 'test-mcfs.R:7:3',
'test-mcfs.R:36:3', 'test-mcfs.mode2.R:7:3'
• empty test (1): 'test-man.mcfs.R:4:1'
[ FAIL 0 | WARN 2 | SKIP 8 | PASS 0 ]
>
> proc.time()
user system elapsed
16.06 1.64 5.46