download report - Sapienza
download report - Sapienza
download report - Sapienza
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
Scientific Report 2007-2009<br />
Theoretical physics<br />
T14. Optimization problems and message passing algorithms<br />
Optimization problems are widespread in scientific disciplines.<br />
The goal is typically to found the minimum of<br />
a given cost function defined in terms of a large number<br />
N of variables. In physical terms it corresponds to the<br />
computation of a ground state configuration. The problem<br />
may become very hard when the interacting terms<br />
in the cost function are in competition, i.e. the model<br />
is frustrated. In recent years our group has developed<br />
many statistical physics tools that allow to perform analytical<br />
computations in these models, even directly at<br />
zero temperature, such as to probe the structure of the<br />
ground states of the model.<br />
Among optimization problems, a quite general class<br />
is formed by Constraint Satisfaction Problems (CSP)<br />
where a set is given of M = αN constraints, that must<br />
be satisfied by a proper assignment of the N variables.<br />
We have been able to solve this kind of models in the<br />
case where the constraints are generated independently,<br />
which actually correspond to defining the model on a<br />
random graph. Under this hypothesis, the Bethe approximation<br />
turns out to work in a certain range of model<br />
parameters (i.e. in the equivalent of the paramagnetic<br />
phase). When it fails, the replica symmetry need to be<br />
broken and we have obtained the solutions up to one level<br />
of replica symmetry breaking, that actually provide the<br />
exact answer for many well-known CSP.<br />
α d,+ α d α c α s<br />
Figure 1: Phase transitions in the structure of solutions to<br />
random k-SAT problems [1].<br />
While studying the structure of the space of solutions<br />
to random CSP we have uncovered many different phase<br />
transitions. In Figure 1 we show a schematic picture<br />
of how the structure of solution changes, e.g. forming<br />
clusters, while increasing the ratio α of constraints per<br />
variable. The picture is from Ref. [1] where we have<br />
presented the most general solution to important random<br />
CSP, like satisfiability and coloring. The random<br />
k-satisfiability problem has been further examined and<br />
solved in great detail in Ref. [2].<br />
An important role of the phase transitions uncovered<br />
in this kind of problems relies on the fact that solving<br />
algorithms are typically affected by the drastic changes<br />
taking place at these thresholds: stochastic local search<br />
algorithms (like Monte Carlo) should get stuck at the<br />
dynamical threshold α d , while Belief Propagation (BP)<br />
works up to the condensation threshold α c and finally<br />
Survey Propagation (SP) should be able to go beyond<br />
α c and get closer to the satisfiability threshold α s .<br />
The last two algorithms, BP and SP, are so-called<br />
Message Passing Algorithms (MPA) and are extremely<br />
efficient for making probabilistic inference on random<br />
graphs. These algorithms work by sending messages between<br />
the nodes of the graphs that represent the variables<br />
and the constraints in the problem. Messages leaving<br />
a node are updated according to the incoming messages<br />
to that node. For this reason the algorithm is easy<br />
to implement, fast to use and possibly distributed.<br />
2<br />
3<br />
1<br />
4<br />
0<br />
5<br />
9<br />
6<br />
8<br />
7<br />
Figure 2: Examples of node ranking by counting loops [3].<br />
A very important aspect of MPA that we have started<br />
to investigate recently is their use on non-random graphs,<br />
that is graphs with many short loops and topological motifs.<br />
An interesting example is given by the problem of<br />
ranking graphs nodes, i.e. to uncover which nodes are<br />
the most important in the graph topology (a straightforward<br />
application being the ranking of web pages). In<br />
Ref. [3] we have introduced a new MPA that ranks nodes<br />
depending on how many loops pass through that node.<br />
Typical rankings are shown in Fig. 2. The performances<br />
we obtain are comparable with those of widely used algorithms,<br />
like PageRank and betweenness centrality.<br />
The effectiveness of our analytical approach to optimization<br />
problems is that we can deeply understand the<br />
physical origin of their hardness and thus explain why<br />
solving algorithms may fail to find solution to this problem.<br />
In Ref. [4] we have been able to solve analytically a<br />
stochastic search algorithm, which is based on BP and a<br />
decimation procedure. The analytical solution perfectly<br />
coincides with the outcome of the numerical algorithm<br />
and predicts a fail of the searching procedure due to the<br />
existence of a phase transition in the space of solutions.<br />
The resulting picture is somehow counter-intuitive:<br />
reducing the problem by fixing a certain fraction of<br />
variables does not simplify the problem, but rather<br />
makes it harder to solve.<br />
References<br />
1. F. Krzakala et al., PNAS 104, 10318 (2007).<br />
2. A. Montanari et al., J. Stat. Mech., P04004 (2008).<br />
3. V. Van Kerrebroeck et al., Phys. Rev. Lett. 101, 098701<br />
(2008).<br />
4. F. Ricci-Tersenghi et al., J. Stat. Mech., P09001 (2009).<br />
Authors<br />
F. Ricci-Tersenghi, E. Marinari, G. Parisi, V. Van Kerrebroeck<br />
1<br />
0<br />
2<br />
3<br />
1<br />
4<br />
0<br />
5<br />
9<br />
6<br />
8<br />
7<br />
<strong>Sapienza</strong> Università di Roma 37 Dipartimento di Fisica