Transfer Entropy Sample Clauses

Transfer Entropy. ‌ Although Mutual Information provides a very useful measure of information sharing be- tween two time series, it is a symmetric metric and, thus, does not provide any directional information. In the financial domain, it is usually very useful to be able to determine not only the amount of shared information between the two vectors, but also the direction of influence, i.e., which one of the two “drives” the other. For example, if there are two market players, namely A and B, it is important to know if A is correlated to B. However, it is also important (maybe even more important) to know that, for the sake of argument, A affects B. This may mean that if A’s price rises, then B’s price will also rise, whereas the opposite might not hold (i.e., “when B’s price rises, A’s price will rise too”). |x Transfer Entropy is an asymmetrical information transfer measure that can provide the required directivity. Unfortunately, Transfer Entropy is quite demanding with respect to computation resources since it requires a cubic (O(n3)) memory and time complexity. Formally, Transfer Entropy between two discrete random variables X and Y (from Y to X) can be defined as: Y →X xn+1∈X xn∈X yn∈Y TE = X X X x(x x+0 n , x , y ) log .p(xn+1|xn, yn) Σ, n+1 n | | n p(x ) where x(xx+0, xx, yn) is the joint probability, and p(xn+1 xn, yn), p(xn+1 xn) are conditional probabilities. ∗ As in Mutual Information, the main challenge lies in maintaining the probability functions. In the case of histogram approximation, the complexity is related to the number of bins used (as in Mutual Information). For example, if 1000 bins are chosen as the size of the histogram, then the joint probability of a single pair amount to 4Gb of memory (10003 4 (number of int for the count of each cell)).

Transfer Entropy