Mathematical optimization is at the heart of many machine learning algorithms.
My research aims at advancing machine intelligence via state-of-the-art optimization techniques and their innovative applications.
The bsnsing (pronounced 'B-sensing', for Boolean Sensing) package provides functions for building a decision tree classifier and making predictions. It solves the two-class and multi-class classification problems under the supervised learning paradigm. While building a decision tree, bsnsing uses a Boolean rule involving multiple variables to split a node. Each split rule is identified by solving an optimization model that minimizes misclassification as well as complexity. Compared to other decision tree learners that seek single-variable splits in a greedy fashion, bsnsing's approach is more holistic and produces highly interpretable and accurate trees.
This is an active, work-in-progress project.
Install the bsnsing package from GitHub (https://github.com/profyliu/bsnsing)
Check out a quick demo in R.
A two-dimensional example: how bsnsing learns a pattern via rectilinear splits. In the plot, the ring-shaped data pattern is effectively learned by multivariate splits. Splits 5 to 11 would not be possible by the traditional single-variable splits.
Most optimization-based machine learning methods assume a known functional form between the input and output variables and then find the best-fit parameters for the assumed function. The new question is: Can we let the machines to learn more freely? In other words, can we NOT specify a functional form, but let the computer figure one out and fit the parameters?