Tabular Foundation Models
Intro I have been working with models for tabular data a bit professionally and as a hobby. I did notice attempts over time to use neural nets to somehow beat decision tree based models. However, they all pretty much at most were on par with decision tree based models, even in regimes with millions of samples. A recent post by Gaël Varoquaux caught my attention stating “Tabular foundation models have gone from a promising idea to a production-ready technology”. Notable, as he is one of the creators of scikit-learn - a tool I consider an essential work horse of data science. ...