Since starting my own AI/machine learning research lab over a year ago, I have published 24 technical papers and 4 books, in addition to articles on Data Science Central. Here are the most popular ones listed in random order and a short summary. doing. You can access it from this blog.
Feature clustering
We describe how to perform clustering (also known as unsupervised classification) on feature sets, as opposed to traditional clustering of observational data. Cross-correlation between features is used as a similarity metric in order to retain one representative feature in each feature group. The goal is to reduce the number of features with minimal loss in terms of predictability. This is an alternative to PCA, but without combining features into meaningless predictors. Finally, we apply this method to the generation of synthetic data. See article #21 and related article #23 for a discussion of how to randomly remove up to 80% of observations without losing predictive power.
Data Synthesis: GAN vs Copula
Synthetic data has many uses. Companies use it for data augmentation to enhance existing training sets, balance datasets (fraud detection, clinical trials), and reduce algorithmic bias. In my case, I use it for benchmarking different algorithms. Case studies in my articles include insurance and medical datasets. The focus is on tabular data, replicability, and parameter fine-tuning for optimal performance. We detail two methods, generative adversarial networks (GANs) and copulas, and show when GANs give good results. I also cover this material in my classes and books on this topic. See article 20.
Gradient descent without math in Python
Almost all machine learning algorithms require some optimization technique to find a good solution. This is what all neural networks ultimately do. The generic term for these techniques is gradient descent. But it comes with all sorts of flavors like stochastic gradient descent and swarm optimization. In my article, I describe a very intuitive approach, recreating the path a raindrop takes to hit the ground uphill and reach the bottom of a valley. There is no learning rate, no parameters, and no mathematics in this technique. So it applies to pure data where there is no mathematical function to minimize. In addition to contour levels, it shows how to compute orthogonal trajectories and process hundreds of starting points at once to create cool videos. See article #17 and related article #23, which describes a smart grid search algorithm for hyperparameter optimization.
Cloud Regression: The Swiss Army Knife of Optimization
In this article, we discuss a new method that blends all regression-related techniques under the same umbrella: linear, logistic, lasso, and ridge. For logistic regression, we show how to replace the logit function with a parameterless, data-driven version. A method is first defined for a non-response dataset where all features play the same role. In this case, the dataset is just a point cloud, hence the name of the technique. This leads to unsupervised regression. In some cases, however, it is possible to solve classification problems using closed-form formulas without assigning points to clusters. This solution involves Lagrangian multipliers and gradient descent. Lets you compute confidence intervals for predicted values without using statistical terms. I use the concept of confidence level, which is more intuitive to non-statisticians. All standard regression techniques are only specific cases. See article #10 and related article #16 on multivariate interpolation. The latter describes a hybrid additive-multiplicative algorithm to get the best of both.
A gentle introduction to linear algebra
This was my first article on the topic, featuring a new and simpler approach to solving linear and matrix algebra problems related to AI and machine learning. It focuses on simplicity and provides a light presentation without watering down the content for beginners who are overwhelmed with math. Quite the opposite, we go so far as to discuss double-integral Brownian motion and autoregressive processes without eigenvalues or terminology. We also discover anomalous continuous time series that fill the entire domain. The goal is to share new models with potential applications such as fintech. See article 5. Once you’ve mastered this material, move on to more advanced articles #18 and #19, which deal with chaotic dynamic systems.
Simple alternative to XGBoost
When I worked on credit card fraud detection at Visa, I designed a binning technique to process the combined features using many small overlapping decision trees. This methodology was originally developed around 2002 and has since been extended to address natural language processing problems. Since then it has been constantly improved. This ensemble technique was originally devised by the team that created XGBoost. So in my article you may discover some unique tricks, best practices and rules of thumb, especially regarding algorithmic simplification. In particular, it blends simplified logistic regression with many decision trees. As with my other articles, it comes with a Python implementation, also available on GitHub. Describe the use case. Choose and optimize your article titles for maximum impact. It contains many elements found in applications such as ChatGPT. See article 11.
About the author
Vincent Granville is a pioneering data scientist, machine learning expert, founder of MLTechniques.com, co-founder of Data Science Central (acquired by TechTarget in 2020) and former VC-funded executive, author, and patent owner. Vincent’s past corporate experience includes Visa, Wells Fargo, eBay, NBC, Microsoft, CNET and InfoSpace. Vincent is also a former postdoc at the University of Cambridge and the National Institute of Statistical Sciences (NISS).
Posted in Vincent number theory journal, Journal of the Royal Statistical Society (Series B), and IEEE Transactions on Pattern Analysis and Machine IntelligenceHe is also the author of “Intuitive Machine Learning and Explainable AI”, available here. He lives in Washington state and enjoys researching stochastic processes, dynamical systems, experimental mathematics, and probabilistic number theory.
