Más contenido relacionado
La actualidad más candente (20)
Similar a 【DL輪読会】Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers (16)
Más de Deep Learning JP (20)
【DL輪読会】Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers
- 1. 1
DEEP LEARNING JP
[DL Papers]
http://deeplearning.jp/
“Scale Efficiently: Insights from Pre-training and Fine-
tuningTransformers” (ICLR2022)
Okimura Itsuki, Matsuo Lab, M1
- 3. 1 書誌情報
タイトル: Scale Efficiently: Insights from Pre-training and Fine-tuning
Transformers
出典: ICLR2022 https://openreview.net/pdf?id=f2OYVDyfIB
著者: Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira
Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish
Vaswani, Donald Metzler (Google Research & Deepmind)
選んだ理由:べき乗則の再検討みたいなもので気になった
3
- 17. DEEP LEARNING JP
[DL Papers]
“Grokking: Generalization Beyond Overfitting on Small
Algorithmic Datasets” (ICLR 2021 workshop)
Okimura Itsuki, Matsuo Lab, B4
http://deeplearning.jp/