The shape of learning curves: a review
Learning curves provide insight into the dependence of a learner's generalization
performance on the training set size. This important tool can be used for model selection, to …
performance on the training set size. This important tool can be used for model selection, to …
The statistical mechanics of learning a rule
TLH Watkin, A Rau, M Biehl - Reviews of Modern Physics, 1993 - APS
A summary is presented of the statistical mechanical theory of learning a rule with a neural
network, a rapidly advancing area which is closely related to other inverse problems …
network, a rapidly advancing area which is closely related to other inverse problems …
Hidden progress in deep learning: Sgd learns parities near the computational limit
There is mounting evidence of emergent phenomena in the capabilities of deep learning
methods as we scale up datasets, model sizes, and training times. While there are some …
methods as we scale up datasets, model sizes, and training times. While there are some …
Optimal errors and phase transitions in high-dimensional generalized linear models
Generalized linear models (GLMs) are used in high-dimensional machine learning,
statistics, communications, and signal processing. In this paper we analyze GLMs when the …
statistics, communications, and signal processing. In this paper we analyze GLMs when the …
[หนังสือ][B] Statistical mechanics of learning
A Engel - 2001 - books.google.com
Learning is one of the things that humans do naturally, and it has always been a challenge
for us to understand the process. Nowadays this challenge has another dimension as we try …
for us to understand the process. Nowadays this challenge has another dimension as we try …
Learning by on-line gradient descent
M Biehl, H Schwarze - Journal of Physics A: Mathematical and …, 1995 - iopscience.iop.org
We study on-line gradient-descent learning in multilayer networks analytically and
numerically. The training is based on randomly drawn inputs and their corresponding …
numerically. The training is based on randomly drawn inputs and their corresponding …
Using machine learning to generate novel hypotheses: Increasing optimism about COVID-19 makes people less willing to justify unethical behaviors
How can we nudge people to not engage in unethical behaviors, such as hoarding and
violating social-distancing guidelines, during the COVID-19 pandemic? Because past …
violating social-distancing guidelines, during the COVID-19 pandemic? Because past …
Rethinking generalization requires revisiting old ideas: statistical mechanics approaches and complex learning behavior
CH Martin, MW Mahoney - arxiv preprint arxiv:1710.09553, 2017 - arxiv.org
We describe an approach to understand the peculiar and counterintuitive generalization
properties of deep neural networks. The approach involves going beyond worst-case …
properties of deep neural networks. The approach involves going beyond worst-case …
Learning a rule in a multilayer neural network
H Schwarze - Journal of Physics A: Mathematical and General, 1993 - iopscience.iop.org
The problem of learning from examples in multilayer networks is studied within the
framework of statistical mechanics. Using the replica formalism we calculate the average …
framework of statistical mechanics. Using the replica formalism we calculate the average …
Learning and generalization in a two-layer neural network: The role of the Vapnik-Chervonvenkis dimension
M Opper - Physical review letters, 1994 - APS
Bounds for the generalization ability of neural networks based on Vapnik-Chervonenkis
(VC) theory are compared with statistical mechanics results for the case of the parity …
(VC) theory are compared with statistical mechanics results for the case of the parity …