Meta-learning sparse implicit neural representations

J Lee, J Tack, N Lee, J Shin - Advances in Neural …, 2021 - proceedings.neurips.cc
Implicit neural representations are a promising new avenue of representing general signals
by learning a continuous function that, parameterized as a neural network, maps the domain …

Pruning via iterative ranking of sensitivity statistics

S Verdenius, M Stol, P Forré - arxiv preprint arxiv:2006.00896, 2020 - arxiv.org
With the introduction of SNIP [arxiv: 1810.02340 v2], it has been demonstrated that modern
neural networks can effectively be pruned before training. Yet, its sensitivity criterion has …

Scheduling hyperparameters to improve generalization: From centralized sgd to asynchronous sgd

J Sun, Y Yang, G Xun, A Zhang - ACM Transactions on Knowledge …, 2023 - dl.acm.org
This article studies how to schedule hyperparameters to improve generalization of both
centralized single-machine stochastic gradient descent (SGD) and distributed asynchronous …

Critical parameters for scalable distributed learning with large batches and asynchronous updates

S Stich, A Mohtashami, M Jaggi - … Conference on Artificial …, 2021 - proceedings.mlr.press
It has been experimentally observed that the efficiency of distributed training with stochastic
gradient (SGD) depends decisively on the batch size and—in asynchronous …

Performance of physical-informed neural network (PINN) for the key parameter inference in Langmuir turbulence parameterization scheme

F **u, Z Deng - Acta Oceanologica Sinica, 2024 - Springer
The Stokes production coefficient (E 6) constitutes a critical parameter within the Mellor-
Yamada type (MY-type) Langmuir turbulence (LT) parameterization schemes, significantly …

Эмоциональный анализ данных социальных сетей с использованием кластерной вероятностной нейронной сети с параллелизмом данных

СД Старлин, НИ Ченталир - Научно-технический вестник …, 2024 - ntv.elpub.ru
Аннотация Социальные сети содержат огромное количество данных, которые
используются различными организациями для изучения эмоций, мыслей и мнений …

Efficient DNN training based on backpropagation parallelization

D **ao, C Yang, W Wu - Computing, 2022 - Springer
Pipeline parallelism is an efficient way to speed up the training of deep neural networks
(DNNs) by partitioning the model and pipelining the training process across a cluster of …

Understanding the impact of data parallelism on neural network classification

S Starlin **i, DN Chenthalir Indra - Optical Memory and Neural Networks, 2022 - Springer
Social Networks have become a platform to express each moment of a person via texts in
widespread. With the help of a lot of words, ideas, thoughts and good memories are shared …

A chaos theory approach to understand neural network optimization

M Sasdelli, T Ajanthan, TJ Chin… - 2021 Digital Image …, 2021 - ieeexplore.ieee.org
Despite the complicated structure of modern deep neural network architectures, they are still
optimized with algorithms based on Stochastic Gradient Descent (SGD). However, the …

Federated Learning Optimization Algorithm Based on Dynamic Client Scale

L Wang, W Feng, R Luo - … Networking for Quality, Reliability, Security and …, 2023 - Springer
Federated learning methods typically learn models from the local iterative updates of a large
number of clients. The interest in the impact of client quantity on the training dynamics of …