Lightly

Changes
* Add MoCoV2 ImageNet benchmarks.
* Make KNN feature normalization optional.
* Implement W-MSE Loss and Transform (thanks johnsutor).
* Update generated specs with datasource expiration.

Models
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [DCL: Decoupled Contrastive Learning, 2021](https://arxiv.org/abs/2110.06848)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022](https://link.springer.com/chapter/10.1007/978-3-031-16788-1_4)
- [I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023](https://arxiv.org/abs/2301.08243)
- [MAE: Masked Autoencoders Are Scalable Vision Learners, 2021](https://arxiv.org/abs/2111.06377)
- [MSN: Masked Siamese Networks for Label-Efficient Learning, 2022](https://arxiv.org/abs/2204.07141)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [PMSN: Prior Matching for Siamese Networks, 2022](https://arxiv.org/abs/2210.07277)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [SimMIM: A Simple Framework for Masked Image Modeling, 2021](https://arxiv.org/abs/2111.09886)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022](https://arxiv.org/pdf/2207.06167.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)
- [TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022](https://arxiv.org/pdf/2206.10698.pdf)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022](https://arxiv.org/abs/2105.04906)
- [VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022](https://arxiv.org/abs/2210.01571)

Changes
* Add MMCR loss and transform in https://github.com/lightly-ai/lightly/pull/1446. Thanks to johnsutor!
* Update README to reflect correct import for LightlyDataset in https://github.com/lightly-ai/lightly/pull/1437. Thanks to dnth!
* Fix MoCoV2 transform parameters in https://github.com/lightly-ai/lightly/pull/1441
* Set model to eval for benchmark knn and linear classification in https://github.com/lightly-ai/lightly/pull/1444
* Remove extra mean calculation in VICRegLoss in https://github.com/lightly-ai/lightly/pull/1450. Thanks to RylanSchaeffer for pointing out the issue!
* Fix MultiCropTransform crop_max_scales check in https://github.com/lightly-ai/lightly/pull/1454. Thanks to Djoels for pointing out the issue!

Models
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [DCL: Decoupled Contrastive Learning, 2021](https://arxiv.org/abs/2110.06848)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022](https://link.springer.com/chapter/10.1007/978-3-031-16788-1_4)
- [I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023](https://arxiv.org/abs/2301.08243)
- [MAE: Masked Autoencoders Are Scalable Vision Learners, 2021](https://arxiv.org/abs/2111.06377)
- [MSN: Masked Siamese Networks for Label-Efficient Learning, 2022](https://arxiv.org/abs/2204.07141)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [PMSN: Prior Matching for Siamese Networks, 2022](https://arxiv.org/abs/2210.07277)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [SimMIM: A Simple Framework for Masked Image Modeling, 2021](https://arxiv.org/abs/2111.09886)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022](https://arxiv.org/pdf/2207.06167.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)
- [TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022](https://arxiv.org/pdf/2206.10698.pdf)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022](https://arxiv.org/abs/2105.04906)
- [VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022](https://arxiv.org/abs/2210.01571)

Changes
- Typecheck part of models
- Polish benchmarks page
- Update specs

Models
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [DCL: Decoupled Contrastive Learning, 2021](https://arxiv.org/abs/2110.06848)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022](https://link.springer.com/chapter/10.1007/978-3-031-16788-1_4)
- [I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023](https://arxiv.org/abs/2301.08243)
- [MAE: Masked Autoencoders Are Scalable Vision Learners, 2021](https://arxiv.org/abs/2111.06377)
- [MSN: Masked Siamese Networks for Label-Efficient Learning, 2022](https://arxiv.org/abs/2204.07141)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [PMSN: Prior Matching for Siamese Networks, 2022](https://arxiv.org/abs/2210.07277)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [SimMIM: A Simple Framework for Masked Image Modeling, 2021](https://arxiv.org/abs/2111.09886)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022](https://arxiv.org/pdf/2207.06167.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)
- [TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022](https://arxiv.org/pdf/2206.10698.pdf)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022](https://arxiv.org/abs/2105.04906)
- [VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022](https://arxiv.org/abs/2210.01571)

Changes
- Typecheck part of utils
- Add ability to download the new reportv2.json
- Update specs

Models
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [DCL: Decoupled Contrastive Learning, 2021](https://arxiv.org/abs/2110.06848)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022](https://link.springer.com/chapter/10.1007/978-3-031-16788-1_4)
- [I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023](https://arxiv.org/abs/2301.08243)
- [MAE: Masked Autoencoders Are Scalable Vision Learners, 2021](https://arxiv.org/abs/2111.06377)
- [MSN: Masked Siamese Networks for Label-Efficient Learning, 2022](https://arxiv.org/abs/2204.07141)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [PMSN: Prior Matching for Siamese Networks, 2022](https://arxiv.org/abs/2210.07277)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [SimMIM: A Simple Framework for Masked Image Modeling, 2021](https://arxiv.org/abs/2111.09886)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022](https://arxiv.org/pdf/2207.06167.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)
- [TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022](https://arxiv.org/pdf/2206.10698.pdf)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022](https://arxiv.org/abs/2105.04906)
- [VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022](https://arxiv.org/abs/2210.01571)

Changes
- Add `period` argument to `CosineWarmupScheduler` 1413, thanks to JannikWirtz!

Models
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [DCL: Decoupled Contrastive Learning, 2021](https://arxiv.org/abs/2110.06848)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022](https://link.springer.com/chapter/10.1007/978-3-031-16788-1_4)
- [I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023](https://arxiv.org/abs/2301.08243)
- [MAE: Masked Autoencoders Are Scalable Vision Learners, 2021](https://arxiv.org/abs/2111.06377)
- [MSN: Masked Siamese Networks for Label-Efficient Learning, 2022](https://arxiv.org/abs/2204.07141)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [PMSN: Prior Matching for Siamese Networks, 2022](https://arxiv.org/abs/2210.07277)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [SimMIM: A Simple Framework for Masked Image Modeling, 2021](https://arxiv.org/abs/2111.09886)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022](https://arxiv.org/pdf/2207.06167.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)
- [TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022](https://arxiv.org/pdf/2206.10698.pdf)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022](https://arxiv.org/abs/2105.04906)
- [VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022](https://arxiv.org/abs/2210.01571)

Changes
- Small improvements to documentation
- Added types to embedding scripts (kudos to agpeshal!)
- Fix: Handle OPTIONS method for `lightly-serve` script

Models
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021](https://arxiv.org/abs/2103.03230)
- [Bootstrap your own latent: A new approach to self-supervised Learning, 2020](https://arxiv.org/abs/2006.07733)
- [DCL: Decoupled Contrastive Learning, 2021](https://arxiv.org/abs/2110.06848)
- [DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021](https://arxiv.org/abs/2104.14294)
- [FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022](https://link.springer.com/chapter/10.1007/978-3-031-16788-1_4)
- [I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023](https://arxiv.org/abs/2301.08243)
- [MAE: Masked Autoencoders Are Scalable Vision Learners, 2021](https://arxiv.org/abs/2111.06377)
- [MSN: Masked Siamese Networks for Label-Efficient Learning, 2022](https://arxiv.org/abs/2204.07141)
- [MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019](https://arxiv.org/abs/1911.05722)
- [NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021](https://arxiv.org/pdf/2104.14548.pdf)
- [PMSN: Prior Matching for Siamese Networks, 2022](https://arxiv.org/abs/2210.07277)
- [SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020](https://arxiv.org/abs/2002.05709)
- [SimMIM: A Simple Framework for Masked Image Modeling, 2021](https://arxiv.org/abs/2111.09886)
- [SimSiam: Exploring Simple Siamese Representation Learning, 2020](https://arxiv.org/abs/2011.10566)
- [SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022](https://arxiv.org/pdf/2207.06167.pdf)
- [SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020](https://arxiv.org/abs/2006.09882)
- [TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022](https://arxiv.org/pdf/2206.10698.pdf)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022](https://arxiv.org/abs/2105.04906)
- [VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022](https://arxiv.org/abs/2210.01571)