Probabilistic and Statistical Inference Group
Electrical and Computer Engineering
University of Toronto
The PSI group develops new computational machine learning tools and theoretical frameworks for analyzing large-scale data sets, with applications in molecular biology, computer vision, and sensory processing. The group is led by Brendan J. Frey in the department of Electrical and Computer Engineering, and collaborates closely with colleagues in computer science and molecular biology. The research focus of members of the PSI group is on introducing principled algorithms that reveal hidden variables and efficiently take into account the structural knowledge that is critical in most real-world applications.
Current Research Highlight
Cumulative distribution networks: A novel graphical modelling framework
Traditional graphical models express a decomposition of a joint distribution as a product of functions of subsets of variables. However, in some situations, such as when modeling uncertainties in ratings data, it is more natural to think of each function as providing evidence about possible orderings of its variables. We formalize this by introducing a new type of graphical model called a `cumulative distribution network' (CDN), which uses a product of functions of subsets of variables to describe the joint cumulative distribution. The conditional independence properties of CDNs are quite different from standard graphical models. Unlike Bayesian networks, Markov networks and factor graphs, in a CDN the marginal distribution of any variable can be computed in constant time whilst allowing for complex conditional dependencies. To compute conditional cumulative distributions in the network, we have developed the derivative-sum-product (DSP) algorithm in which messages correspond to local conditional CDFs obtained from computing derivatives of the joint cumulative distribution. The CDN framework has been applied to the problem of ranking players in multiplayer games, document retrieval and collaborative prediction for ratings data.
4-variable example illustrating the CDN. Suppose we wish to simultaneously represent marginal independence between X1 and X4, but also conditional dependence between X1 and X3 given X2, and conditional dependence between X2 and X4 given X3. This cannot be properly modelled using a Bayesian network or a Markov network. In the former case, directed cycles are necessarily introduced (top), producing an invalid Bayesian network. In the latter case (middle), edges must be introduced to express conditional dependence and these spoil the independence of X1 and X4. However, a CDN can represent this distribution (bottom).
References:
J.C. Huang and B.J. Frey. (2008) Cumulative distribution networks and the derivative-sum-product algorithm. Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence (UAI). (Runner-up for Best Student Paper Award) [PDF]