About the bsnsing package

The bsnsing package provides functions for building a decision tree classifier and making predictions. It solves the two-class and multi-class classification problems under the supervised learning paradigm. While building a decision tree, bsnsing uses a Boolean rule involving multiple variables to split a node. Each split rule is identified by solving an optimization model that minimizes misclassification as well as complexity. Compared to other decision tree learners that only generate single-variable splits, bsnsing gives more interpretable trees while scoring a high predictive accuracy.

Install bsnsing from Github

install.packages('devtools')
library(devtools)
install_github("profyliu/bsnsing")
library(bsnsing)

Binary classification

Load the Glaucoma data

data("GlaucomaMVF", package = "ipred") 
n <- nrow(GlaucomaMVF)
set.seed(2018)
train_index <- sample(1:n, round(0.5 * n))
test_index <- setdiff(1:n, train_index)

Out-of-the-box run

bs <- bsnsing(Class ~ ., data = GlaucomaMVF, subset = train_index)
Warning in bsnsing.default(x, y, ...): The cplexAPI is not installed. The
opt.solver is set to 'lpSolve' instead.
summary(bs)
Call: summary.bsnsing(object = bs)
Response coding: normal ==> 1   glaucoma ==> 0
Decision Tree:
-------------
Node 0: class = 0 w/ prob 0.5059, prop 1.0000, n1 42, n0 43
Split by (phcn > 0.0435 | emd > 0.461 | clv > 5.4)
      *Node 1: class = 0 w/ prob 0.9545, prop 0.5176, n1 2, n0 42
      *Node 2: class = 1 w/ prob 0.9756, prop 0.4824, n1 40, n0 1
-------------
The tree has 1 splits (0 fractional), 2 leaf nodes. 
Confusion matrix:
      actual
fitted  0  1
     0 42  2
     1  1 40
Accuracy: 0.9647
# Predict the test cases
pred <- predict(bs, GlaucomaMVF[test_index, ], type = 'class')
table(pred, actual = GlaucomaMVF[test_index, 'Class'])
          actual
pred       glaucoma normal
  glaucoma       37      8
  normal          5     35

Customize parameters

bs <- bsnsing(Class ~ ., data = GlaucomaMVF, subset = train_index, stop.prob = 0.99)
Warning in bsnsing.default(x, y, ...): The cplexAPI is not installed. The
opt.solver is set to 'lpSolve' instead.
summary(bs)
Call: summary.bsnsing(object = bs)
Response coding: normal ==> 1   glaucoma ==> 0
Decision Tree:
-------------
Node 0: class = 0 w/ prob 0.5059, prop 1.0000, n1 42, n0 43
Split by (phcn > 0.0435 | emd > 0.461 | clv > 5.4)
      Node 1: class = 0 w/ prob 0.9545, prop 0.5176, n1 2, n0 42
      Split by (lora <= 48.07036)
            *Node 3: class = 1 w/ prob 1.0000, prop 0.0235, n1 2, n0 0
            *Node 4: class = 0 w/ prob 1.0000, prop 0.4941, n1 0, n0 42
      Node 2: class = 1 w/ prob 0.9756, prop 0.4824, n1 40, n0 1
      Split by (at > 0.575 | eag < 2.455)
            *Node 5: class = 1 w/ prob 1.0000, prop 0.4706, n1 40, n0 0
            *Node 6: class = 0 w/ prob 1.0000, prop 0.0118, n1 0, n0 1
-------------
The tree has 3 splits (0 fractional), 4 leaf nodes. 
Confusion matrix:
      actual
fitted  0  1
     0 43  0
     1  0 42
Accuracy: 1.0000
table(pred = predict(bs, GlaucomaMVF[test_index, ], type = 'class'), actual = GlaucomaMVF[test_index, 'Class'])
          actual
pred       glaucoma normal
  glaucoma       37      7
  normal          5     36

Multi-class classification

The iris data

n <- nrow(iris)
set.seed(2018)
train_index <- sample(1:n, round(0.5 * n))
test_index <- setdiff(1:n, train_index)

Run bsnsing out-of-the-box

bs <- bsnsing(Species ~ ., data = iris, subset = train_index)
Warning in bsnsing.default(x, yi, ...): The cplexAPI is not installed. The
opt.solver is set to 'lpSolve' instead.

Warning in bsnsing.default(x, yi, ...): The cplexAPI is not installed. The
opt.solver is set to 'lpSolve' instead.

Warning in bsnsing.default(x, yi, ...): The cplexAPI is not installed. The
opt.solver is set to 'lpSolve' instead.
summary(bs[[1]])  # display the first tree
Call: summary.bsnsing(object = bs[[1]])
Response coding: setosa ==> 1   all.other ==> 0
Decision Tree:
-------------
Node 0: class = 0 w/ prob 0.6933, prop 1.0000, n1 23, n0 52
Split by (Petal.Length <= 2.45)
      *Node 1: class = 1 w/ prob 1.0000, prop 0.3067, n1 23, n0 0
      *Node 2: class = 0 w/ prob 1.0000, prop 0.6933, n1 0, n0 52
-------------
The tree has 1 splits (0 fractional), 2 leaf nodes. 
Confusion matrix:
      actual
fitted  0  1
     0 52  0
     1  0 23
Accuracy: 1.0000
summary(bs[[2]])  # display the second tree
Call: summary.bsnsing(object = bs[[2]])
Response coding: versicolor ==> 1   all.other ==> 0
Decision Tree:
-------------
Node 0: class = 0 w/ prob 0.6400, prop 1.0000, n1 27, n0 48
Split by (Petal.Length > 4.75 | Petal.Length < 2.45)
      *Node 1: class = 0 w/ prob 0.9600, prop 0.6667, n1 2, n0 48
      *Node 2: class = 1 w/ prob 1.0000, prop 0.3333, n1 25, n0 0
-------------
The tree has 1 splits (0 fractional), 2 leaf nodes. 
Confusion matrix:
      actual
fitted  0  1
     0 48  2
     1  0 25
Accuracy: 0.9733
summary(bs[[3]])  # display the third tree
Call: summary.bsnsing(object = bs[[3]])
Response coding: virginica ==> 1   all.other ==> 0
Decision Tree:
-------------
Node 0: class = 0 w/ prob 0.6667, prop 1.0000, n1 25, n0 50
Split by (Petal.Width < 1.75)
      *Node 1: class = 0 w/ prob 0.9615, prop 0.6933, n1 2, n0 50
      *Node 2: class = 1 w/ prob 1.0000, prop 0.3067, n1 23, n0 0
-------------
The tree has 1 splits (0 fractional), 2 leaf nodes. 
Confusion matrix:
      actual
fitted  0  1
     0 50  2
     1  0 23
Accuracy: 0.9733
# Confusion matrix on the test set
table(pred = predict(bs, iris[test_index, ], type = 'class'), actual = iris[test_index, 'Species'])
            actual
pred         setosa versicolor virginica
  setosa         27          0         0
  versicolor      0         22         3
  virginica       0          1        22