download report - Sapienza

Scientific Report 2007-2009
Theoretical physics
T14. Optimization problems and message passing algorithms
Optimization problems are widespread in scientific disciplines.
The goal is typically to found the minimum of
a given cost function defined in terms of a large number
N of variables. In physical terms it corresponds to the
computation of a ground state configuration. The problem
may become very hard when the interacting terms
in the cost function are in competition, i.e. the model
is frustrated. In recent years our group has developed
many statistical physics tools that allow to perform analytical
computations in these models, even directly at
zero temperature, such as to probe the structure of the
ground states of the model.
Among optimization problems, a quite general class
is formed by Constraint Satisfaction Problems (CSP)
where a set is given of M = αN constraints, that must
be satisfied by a proper assignment of the N variables.
We have been able to solve this kind of models in the
case where the constraints are generated independently,
which actually correspond to defining the model on a
random graph. Under this hypothesis, the Bethe approximation
turns out to work in a certain range of model
parameters (i.e. in the equivalent of the paramagnetic
phase). When it fails, the replica symmetry need to be
broken and we have obtained the solutions up to one level
of replica symmetry breaking, that actually provide the
exact answer for many well-known CSP.
α d,+ α d α c α s
Figure 1: Phase transitions in the structure of solutions to
random k-SAT problems [1].
While studying the structure of the space of solutions
to random CSP we have uncovered many different phase
transitions. In Figure 1 we show a schematic picture
of how the structure of solution changes, e.g. forming
clusters, while increasing the ratio α of constraints per
variable. The picture is from Ref. [1] where we have
presented the most general solution to important random
CSP, like satisfiability and coloring. The random
k-satisfiability problem has been further examined and
solved in great detail in Ref. [2].
An important role of the phase transitions uncovered
in this kind of problems relies on the fact that solving
algorithms are typically affected by the drastic changes
taking place at these thresholds: stochastic local search
algorithms (like Monte Carlo) should get stuck at the
dynamical threshold α d , while Belief Propagation (BP)
works up to the condensation threshold α c and finally
Survey Propagation (SP) should be able to go beyond
α c and get closer to the satisfiability threshold α s .
The last two algorithms, BP and SP, are so-called
Message Passing Algorithms (MPA) and are extremely
efficient for making probabilistic inference on random
graphs. These algorithms work by sending messages between
the nodes of the graphs that represent the variables
and the constraints in the problem. Messages leaving
a node are updated according to the incoming messages
to that node. For this reason the algorithm is easy
to implement, fast to use and possibly distributed.
2
3
1
4
0
5
9
6
8
7
Figure 2: Examples of node ranking by counting loops [3].
A very important aspect of MPA that we have started
to investigate recently is their use on non-random graphs,
that is graphs with many short loops and topological motifs.
An interesting example is given by the problem of
ranking graphs nodes, i.e. to uncover which nodes are
the most important in the graph topology (a straightforward
application being the ranking of web pages). In
Ref. [3] we have introduced a new MPA that ranks nodes
depending on how many loops pass through that node.
Typical rankings are shown in Fig. 2. The performances
we obtain are comparable with those of widely used algorithms,
like PageRank and betweenness centrality.
The effectiveness of our analytical approach to optimization
problems is that we can deeply understand the
physical origin of their hardness and thus explain why
solving algorithms may fail to find solution to this problem.
In Ref. [4] we have been able to solve analytically a
stochastic search algorithm, which is based on BP and a
decimation procedure. The analytical solution perfectly
coincides with the outcome of the numerical algorithm
and predicts a fail of the searching procedure due to the
existence of a phase transition in the space of solutions.
The resulting picture is somehow counter-intuitive:
reducing the problem by fixing a certain fraction of
variables does not simplify the problem, but rather
makes it harder to solve.
References
1. F. Krzakala et al., PNAS 104, 10318 (2007).
2. A. Montanari et al., J. Stat. Mech., P04004 (2008).
3. V. Van Kerrebroeck et al., Phys. Rev. Lett. 101, 098701
(2008).
4. F. Ricci-Tersenghi et al., J. Stat. Mech., P09001 (2009).
Authors
F. Ricci-Tersenghi, E. Marinari, G. Parisi, V. Van Kerrebroeck
1
0
2
3
1
4
0
5
9
6
8
7
Sapienza Università di Roma 37 Dipartimento di Fisica