Machine Learning - RSS Feed

notes, thoughts, and practice of applied machine learning

Latest articles

Better than BERT: Pick your best model

Have you ever had to sort through HuggingFace to find your best model ? There are over 54,000 models on HuggingFace! So it’s not an easy task. Most people just choose the most popular model–and this is usually BERT. Or some BERT variant. Bert was created by Google, so it must be good. But is BERT the really best choice for...

Is your layer over-trained ? (part 2)

Say you are training a Deep Neural Network (DNN), and you see your model is over-trained. Or just not performing well. Is there a way to detect which layer is actually over-trained? In this post, we will show how to use the open-source weightwatcher tool to answer this. WeightWatcher is an open-source, data-free diagnostic tool for...

Fantastic Measures of Generalization — That Actually Work (part 1)

In the next few posts, I am going to discuss how to use the generalization metrics included in the open-source weightwatcher tool. The goal is to develop a general-purpose tool can that you can use, among other things, to predict (tends in) the test accuracy of a Deep Neural Network — without access to the test data — or even training data! ...

Model Monitoring with WeightWatcher: Data-Free DEEP LEARNING Diagnostics

Weighwatcher is an open-source Model Monitoring tool that provides Data-Free Diagnostics for production-quality Deep Neural Networks (DNNs). It. can tell you if your model is over-trained or over-parameterized. And it can it tell you which layers are over-trained or under-trained (over-parameterized). And all without needing training or test data. ...

How to tell if you have trained your Model with enough data ?

Deep Neural Networks (DNN) require a lot of training data. Even fine-tuning a model can require a lot. A LOT. So how can you know if you have used enough? For Computer Vision (CV) models, you can always look at the test error. But what about fine-tuning large, transformer models like BERT or GPT ? What is the best metric to evaluate your...

is your model overtrained ?

Are your models over-trained ? The weightwatcher tool can detect the signatures of overtraining in specific layers of a pre/trained Deep Neural Networks. In the Figure above, fig (a) is well trained, whereas fig (b) may be over-trained. That orange spike on the far right is the tell-tale clue; it’s what we call a Correlation Trap. Weightwatcher...

Simpson’s Paradox and Deep Learning Metrics with Weightwatcher

What is WeightWatcher ? The WeightWatcher ( github site here ) tool is an open-source python package that can be used to predict the test accuracy of a series similar of Deep Neural Network (DNN) — without peeking at the test data. WeightWatcher is based on research done in collaboration with UC Berkeley on the foundations of Deep...

Why WeightWatcher Works

I am frequently asked, why does weightwatcher work ? The weightwatcher tool uses power law fits to model the eigenvalue density of weight matrices of any Deep Neural Network (DNN). The average power-law exponent is remarkably well correlated with the test accuracy when changing the number of layers and/or fine-tuning the hyperparameters....

WeightWatcher: Empirical Quality Metrics for Deep Neural Networks

We introduce the weightwatcher (ww) , a python tool for computing quality metrics of trained, and pretrained, Deep Neural Networks pip install weightwatcher Here is an example with pretrained VGG11 from pytorch (ww works with keras models also): import weightwatcher as ww import torchvision.models as models model = models.vgg11(pretrained=True)...

Towards a new Theory of Learning: Statistical Mechanics of Deep Neural Networks

Introduction For the past year or two, we have talked a lot about how we can understand the properties of Deep Neural Networks by examining the spectral properties of the layer weight matrices . Specifically, we can form the correlation matrix , and compute the eigenvalues . By plotting the histogram of the eigenvalues...

Discover, share and read the best on the web

Follow RSS Feeds, Blogs, Podcasts, Twitter searches, Facebook pages, even Email Newsletters! Get unfiltered news feeds or filter them to your liking.

Get Inoreader
Inoreader - Follow RSS Feeds, Blogs, Podcasts, Twitter searches, Facebook pages, even Email Newsletters!