| Item type |
SIG Technical Reports(1) |
| 公開日 |
2019-07-17 |
| タイトル |
|
|
タイトル |
Toward Training a Large 3D Cosmological CNN with Hybrid Parallelization |
| タイトル |
|
|
言語 |
en |
|
タイトル |
Toward Training a Large 3D Cosmological CNN with Hybrid Parallelization |
| 言語 |
|
|
言語 |
eng |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
機械学習 |
| 資源タイプ |
|
|
資源タイプ識別子 |
http://purl.org/coar/resource_type/c_18gh |
|
資源タイプ |
technical report |
| 著者所属 |
|
|
|
Tokyo Institute of Technology／Lawrence Livermore National Laboratory |
| 著者所属 |
|
|
|
Lawrence Livermore National Laboratory |
| 著者所属 |
|
|
|
University of Illinois at Urbana-Champaign／Lawrence Livermore National Laboratory |
| 著者所属 |
|
|
|
Lawrence Berkeley National Laboratory |
| 著者所属 |
|
|
|
Lawrence Berkeley National Laboratory |
| 著者所属 |
|
|
|
RIKEN Center for Computational Science／Tokyo Institute of Technology |
| 著者所属 |
|
|
|
University of Illinois at Urbana-Champaign |
| 著者所属 |
|
|
|
Lawrence Berkeley National Laboratory |
| 著者所属 |
|
|
|
Lawrence Livermore National Laboratory |
| 著者所属(英) |
|
|
|
en |
|
|
Tokyo Institute of Technology / Lawrence Livermore National Laboratory |
| 著者所属(英) |
|
|
|
en |
|
|
Lawrence Livermore National Laboratory |
| 著者所属(英) |
|
|
|
en |
|
|
University of Illinois at Urbana-Champaign / Lawrence Livermore National Laboratory |
| 著者所属(英) |
|
|
|
en |
|
|
Lawrence Berkeley National Laboratory |
| 著者所属(英) |
|
|
|
en |
|
|
Lawrence Berkeley National Laboratory |
| 著者所属(英) |
|
|
|
en |
|
|
RIKEN Center for Computational Science / Tokyo Institute of Technology |
| 著者所属(英) |
|
|
|
en |
|
|
University of Illinois at Urbana-Champaign |
| 著者所属(英) |
|
|
|
en |
|
|
Lawrence Berkeley National Laboratory |
| 著者所属(英) |
|
|
|
en |
|
|
Lawrence Livermore National Laboratory |
| 著者名 |
Yosuke, Oyama
Naoya, Maruyama
Nikoli, Dryden
Peter, Harrington
Jan, Balewski
Satoshi, Matsuoka
Marc, Snir
Peter, Nugent
Brian, Van Essen
|
| 著者名(英) |
Yosuke, Oyama
Naoya, Maruyama
Nikoli, Dryden
Peter, Harrington
Jan, Balewski
Satoshi, Matsuoka
Marc, Snir
Peter, Nugent
Brian, Van Essen
|
| 論文抄録 |
|
|
内容記述タイプ |
Other |
|
内容記述 |
We report our preliminary work on large-scale training of a 3D convolutional neural network model for cosmological analyses of dark matter distributions. Previous work showed promising results for predicting cosmological parameters using CNNs trained on a large-scale parallel computing platform. However, due to its weak scaling nature, there exists a trade-off of training performance and prediction accuracy. This paper extends the existing work for better prediction accuracy and performance by exploiting finer-grained parallelism in distributed convolutions. We show significant improvements using the latest complex cosmological dataset with a huge model that was previously unfeasible due to its memory pressure. We achieve 1.42 PFlop/s on a single training task with a mini-batch size of 128 by using 512 Tesla V100 GPUs. Our results imply that the state-of-the-art deep learning case study can be further advanced with HPC-based algorithms. |
| 論文抄録(英) |
|
|
内容記述タイプ |
Other |
|
内容記述 |
We report our preliminary work on large-scale training of a 3D convolutional neural network model for cosmological analyses of dark matter distributions. Previous work showed promising results for predicting cosmological parameters using CNNs trained on a large-scale parallel computing platform. However, due to its weak scaling nature, there exists a trade-off of training performance and prediction accuracy. This paper extends the existing work for better prediction accuracy and performance by exploiting finer-grained parallelism in distributed convolutions. We show significant improvements using the latest complex cosmological dataset with a huge model that was previously unfeasible due to its memory pressure. We achieve 1.42 PFlop/s on a single training task with a mini-batch size of 128 by using 512 Tesla V100 GPUs. Our results imply that the state-of-the-art deep learning case study can be further advanced with HPC-based algorithms. |
| 書誌レコードID |
|
|
収録物識別子タイプ |
NCID |
|
収録物識別子 |
AN10463942 |
| 書誌情報 |
研究報告ハイパフォーマンスコンピューティング（HPC）
巻 2019-HPC-170,
号 8,
p. 1-8,
発行日 2019-07-17
|
| ISSN |
|
|
収録物識別子タイプ |
ISSN |
|
収録物識別子 |
2188-8841 |
| Notice |
|
|
|
SIG Technical Reports are nonrefereed and hence may later appear in any journals, conferences, symposia, etc. |
| 出版者 |
|
|
言語 |
ja |
|
出版者 |
情報処理学会 |
IPSJ-HPC19170008.pdf (678.3 kB)
