This comprehensive treatment of network information theory and its applications provides
the first unified coverage of both classical and recent results. With an approach that …

This report reviews the conceptual and theoretical links between Granger causality and
directed information theory. We begin with a short historical tour of Granger causality …

Consider a control problem with a communication channel connecting the observer of a
linear stochastic system to the controller. The goal of the controller is to minimize a quadratic …

The notion of information pervades informal descriptions of biological systems, but formal
treatments face the problem of defining a quantitative measure of information rooted in a …

Four estimators of the directed information rate between a pair of jointly stationary ergodic
finite-alphabet processes are proposed, based on universal probability assignments. The …

In this paper, we introduce a fundamental principle for optimal communication over general
memoryless channels in the presence of noiseless feedback, termed posterior matching …

The feedback capacity of additive stationary Gaussian noise channels is characterized as
the solution to a variational problem in the noise power spectral density. When specialized …

This work develops a new method for estimating and optimizing the directed information rate
between two jointly stationary and ergodic stochastic processes. Building upon recent …

We consider capacity of discrete-time channels with feedback for the general case where
the feedback is a time-invariant deterministic function of the output samples. Under the …

We investigate the role of directed information in portfolio theory, data compression, and
statistics with causality constraints. In particular, we show that directed information is an …