#ConvolutionalNeuralNetworks (#CNNs in short) are immensely useful for many #imageProcessing tasks and much more...

Yet you sometimes encounter some bits of code with little explanation. Have you ever wondered about the origins of the values for image normalization in #imagenet ?

• Mean: [0.485, 0.456, 0.406] (for R, G and B channels respectively)
• Std: [0.229, 0.224, 0.225]

Strangest to me is the need for a three-digits precision. Here, after finding the origin of these numbers for MNIST and ImageNet, I am testing if that precision is really important : guess what, it is not (so much) !

if interested in more details, check-out https://laurentperrinet.github.io/sciblog/posts/2024-12-09-normalizing-images-in-convolutional-neural-networks.html

How a stubborn #computerscientist accidentally launched the #deeplearning boom
"You’ve taken this idea way too far," a mentor told Prof. Fei-Fei Li, who was creating a new image #dataset that would be far larger than any that had come before: 14 million images, each labeled with one of nearly 22,000 categories. Then in 2012, a team from Univ of Toronto trained a #neura network on #ImageNet, achieving unprecedented performance in image recognition, dubbed #AlexNet.
https://arstechnica.com/ai/2024/11/how-a-stubborn-computer-scientist-accidentally-launched-the-deep-learning-boom/ #AI

#AI heroic stories and underpaid labour:

"The project was saved when Li learned about Amazon Mechanical Turk, a crowdsourcing platform Amazon had launched a couple of years earlier. "

#Imagenet....How a stubborn computer scientist accidentally launched the deep learning boom - Ars Technica
https://arstechnica.com/ai/2024/11/how-a-stubborn-computer-scientist-accidentally-launched-the-deep-learning-boom/

New #AI Research: Simplified Continuous-time Consistency Models (#sCM)

Key findings:
• #OpenAI's new approach matches leading #diffusion models' quality using only 2 sampling steps
• 1.5B parameter model generates samples in 0.11 seconds on single #GPU
• Achieves ~50x wall-clock speedup compared to traditional methods
• Uses less than 10% of typical sampling compute while maintaining quality

Technical highlights:
• Simplifies theoretical formulation of continuous-time consistency models
• Successfully scaled to 1.5B parameters on #ImageNet at 512×512 resolution
• Demonstrates consistent performance scaling with teacher diffusion models
• Enables real-time generation potential for images, audio, and video

Learn more: https://openai.com/index/simplifying-stabilizing-and-scaling-continuous-time-consistency-models/

Re-reading 'On the genealogy of machine learning datasets: A critical history of ImageNet' by @alexhanna. So clear the LLM debacle goes back to the start of the DL boom; it's data fetish, flat universalism, social illiteracy & contempt for workers https://journals.sagepub.com/doi/full/10.1177/20539517211035955
#AI #datasets #Imagenet #resistingAI

Exploring ImageNet's influence in digital humanities and art curation:
The Curator's Machine project delves into how this dataset shapes relationships in the art world. By analyzing its absence of 'art' classification, lack of historical context, and texture vs. outline focus, this research bridges art history, coding, and digital humanities.
by @databasecultures

#DigitalHumanities #DigitalArtHistory #ImageNet
https://dahj.org/article/why-so-many-windows

A DNN Optimizer that Improves over AdaBelief by Suppression of the Adaptive Stepsize Range

Guoqiang Zhang, Kenta Niwa, W. Bastiaan Kleijn

Action editor: Rémi Flamary.

https://openreview.net/forum?id=VI2JjIfU37

Efficient Inference With Model Cascades

Luzian Lebovitz, Lukas Cavigelli, Michele Magno, Lorenz K Muller

Action editor: Yarin Gal.

https://openreview.net/forum?id=obB415rg8q

A simple, efficient and scalable contrastive masked autoencoder for learning visual representations

https://openreview.net/forum?id=pjdxPts6er

Learned Thresholds Token Merging and Pruning for Vision Transformers

Maxim Bonnaerens, Joni Dambre

Action editor: Mathieu Salzmann.

https://openreview.net/forum?id=WYKTCKpImz

Optimizing Learning Rate Schedules for Iterative Pruning of Deep Neural Networks

Shiyu Liu, Rohan Ghosh, John Chong Min Tan, Mehul Motani

Action editor: Mingsheng Long.

https://openreview.net/forum?id=nGW2Hotpq3

Foiling Explanations in Deep Neural Networks

Snir Vitrack Tamam, Raz Lapid, Moshe Sipper

Action editor: Jakub Tomczak.

https://openreview.net/forum?id=wvLQMHtyLk

Synthetic Data from Diffusion Models Improves ImageNet Classification

https://openreview.net/forum?id=DlRsoxjyPm

Can we please, as people who work with large public #datasets, start using torrents? I am just simply trying to find an old version of the #ImageNet Object Detection from Video dataset, and all of the links are broken! For multiple years! Another ImageNet dataset I’m downloading is downloading at 500KB/s.

People have clearly been looking for and using these datasets, and now I need to retrain something and I’m without them. We need to band together and start a torrent tracker for datasets so that we don’t need to rely on one website to download from. With proper permission from the dataset owners of course…

I’m so committed I might buy my own domain and start hosting a torrent tracker. Anyone interested?

Contrastive Attraction and Contrastive Repulsion for Representation Learning

Huangjie Zheng, Xu Chen, Jiangchao Yao et al.

Action editor: Yanwei Fu.

https://openreview.net/forum?id=f39UIDkwwc

Efficient Inference With Model Cascades

https://openreview.net/forum?id=obB415rg8q

#DeepLearning is fun sometimes, especially when you play with #ImageNet...

Here is one result of (our modified version of) ResNet which gives a wrong answer compared to the ground truth label, yet it is visually accurate.

A warning for us all that the objective is not just to reach the highest accuracy, more to better understand what is going on...

This was a result obtained by Emmanuel Daucé from Aix Marseille Université, in a joint work with @jnjer and myself.

Supervised Knowledge May Hurt Novel Class Discovery Performance

ZIYUN LI, Jona Otholt, Ben Dai, Di Hu, Christoph Meinel, Haojin Yang

Action editor: Vikas Sindhwani.

https://openreview.net/forum?id=oqOBTo5uWD

Training Vision-Language Transformers from Captions

https://openreview.net/forum?id=xLnbSpozWS