Deep learning and attention models Sample Clauses

Deep learning and attention models. Accurately predicting the behavior of a customer with specific characterizations, after certain life events or with a unique set of preferences, can quickly become very broad and complex. In theory, every small event could trigger a client to get actively involved in their financial situation, for example a simple discussion with a relative at a birthday party. This complexity and vast amount of (different) data sets, tend to be handled best by neural networks (Xxxxxxxx, 1982). A neural network is a set of multiple interconnected layers of neurons, inspired by the human brain, which can be trained to represent data at high levels of abstraction (LeCun, Xxxxxx, and Xxxxxx, 2015). Neurons are artificial units, transmitting signals to the next neuron. Influenced by input from neurons in the previous layer or the input data, these neurons output a level of activity. By use of backpropagation, this network of neurons can be trained to recognize patterns and are able to predict the outcome of new data instances. Backpropagation is the gradient based learning method applied in the training of most neural networks. Due to the differentiable activation functions, the network is able to back propagate the contribution of each neuron to the error of the training instance. This is used to update the weights of each of the neurons in order to achieve higher accuracy in the next iteration. Required is a known output matched to the input, therefore backpropagation is mostly used in supervised learning tasks. Backpropagation was first described by XxXxx, Boser, et al. (1989) and is considered the accelerator neural networks needed to be further developed into an applicable algorithm. Deep learning is the collection of neural networks with multiple layers of neurons. Given the sequential nature of the problem at hand (sequences of events), recurrent neural networks seem best suited. This specific type of a neural network architecture allow previous output of a unit to influence the next input and are therefore able to account for events in the past when predicting the next event. For example, RNN’s are widely used in text translation, where the output (the translation) not only depends on a single word, but on the sequence of words. General XXX’s however suffer from the difficulty of learning dependencies over time. Due to the gradient based backpropagation algorithm, it gets harder to train the weights of the recurrent layers when the number of layers is increased (i.e. t...