The.Algorithm.Design.Manual.Springer-Verlag.1998

Parallel Algorithms
pruning algorithm can easily save over 99.99% of the work, thus dwarfing any benefits of parallel
brute-force search. Alpha-beta can be parallelized, but not easily, and speedups are typically
limited to a factor of six or so regardless of how many processors you have.
● Parallel algorithms are tough to debug - Unless your problem can be decomposed into several
independent jobs, the different processors will have to communicate with each other in order to
end up with the correct final result. Unfortunately, the non-deterministic nature of this
communication makes parallel programs notoriously difficult to debug. Perhaps the best example
is Deep Blue, the world-champion chess computer. Although it beat Kasparov, over the years it
has lost several games in embarrassing fashion due to bugs, mostly associated with its extensive
parallelism.
I recommend considering parallel processing only after repeated attempts at solving the problem
sequentially prove too slow. Even then, I would restrict attention to algorithms that parallelize the
problem by partitioning the input into distinct tasks, where no communication is needed between the
processors, except in collecting the final results. Such large-grain, naive parallelism can be simple
enough to be readily implementable and debuggable, because it really reduces to producing a good
sequential implementation. Still, there can be pitfalls in this approach, as discussed in the war story
below.
Next: War Story: Going Nowhere Up: Combinatorial Search and Heuristic Previous: War Story:
Annealing Arrays
Algorithms
Mon Jun 2 23:33:50 EDT 1997
file:///E|/BOOK/BOOK/NODE100.HTM (2 of 2) [19/1/2003 1:29:41]