Free - Suif - Stanford University

Jeffrey D. Ullman
Stanford University

2
• Programming is easier if the run-time system
“garbage-collects” – makes space belonging to
unusable data available for reuse.
• Java does it; C does not.
• But stack allocation in C gets some of the advantage.

1. Speed – low overhead for garbage collector.
2. Little program interruption.
• Many collectors shut down the program to hunt for
garbage.
3. Locality – data that is used together is placed
together on pages, cache-lines.
• If the garbage collector moves data, you have some
choices where you put things.
3

4
• There is a root set of data that is a-priori
reachable.
• Example: In Java, root set = static class variables
plus variables on run-time stack.
• Reachable data: root set plus anything
referenced by something reachable.
• Question: Why doesn’t the notion of
“reachability” make sense for C? Why is it
OK for Java?

5
• Things requiring space are “objects.”
• Available space is in a heap – large area
managed by the run-time system.
• Allocator finds space for new objects.
• Space for an object is a chunk.
• Garbage collector finds unusable objects,
returns their space to the heap, and maybe
moves objects around in the heap.

6
Free List
. . .
Object 1 Object 2
Object 3

7
1. Chunks are either allocated or free.
2. Free chunks are linked in some sort of
available space list.
• Why do the links not waste space?
3. Chunks record their length so you know where
the next chunk begins.
• Seems to waste space. Why needed?
4. Adjacent free chunks may not be combined
into one larger chunk.
• Why would we let that happen?

8
1. Via the free-space list.
2. Following references in the objects that
appear in the allocated chunks.
3. In order through the heap.
• That’s why it is useful to have the length of each
chunk in the chunk itself.

• Typically a program needs chunks of various
sizes.
• Fragmentation problem: free chunks become
small, disconnected pieces, so you can’t find
space for a large object.
• Allocation by best fit requires examination of
the entire free list.
• First fit is more efficient, but can give an object
a much larger chunk than it needs.
9

10
• Option: keep many free lists, each with an
exponentially growing range of sizes.
• Then first-fit can’t waste more than 50% of the
space.
• If there are few classes, there will be few sizes of free
chunks, so maybe no waste.
• More extreme: allow only a fixed set of sizes,
e.g., Fibonacci numbers.
• Allocation is trivial since there is no decision.
• But usually wastes space.

11
Garbage Collectors
Reference-
Counters
Trace-
Based

12
• The simplest (but imperfect) method is to
give each object a reference count = number
of references to this object.
• Assumes objects have no internal references or
these can be excluded.
• Initially, object has one reference.
• If reference count becomes 0, object is
garbage and its space becomes available.

Integer i = new Integer(10);
• Integer object is created with RC = 1.
• Consider execution of j = k; (j, k are Integer
references.)
• Object referenced by j has RC--.
• Object referenced by k has RC++.
13

• If an object reaches RC=0, its chunk is garbagecollected
(added to the free list), and the
references within that object effectively
disappear.
• Follow these references and decrement RC in
the objects reached.
• That may result in more objects with RC=0,
leading to recursive collection.
14

15
Root
Object
We remove
this link during
program
execution
A(1)
B(2)
E.g., object B has
reference count 2,
from A and D
C(1)
D(2)
E(1)

16
Root
Object
Reference count
for A drops to zero
A(0)
B(2)
C(1)
D(2)
E(1)

17
Root
Object
So remove A.
Reference count for C drops
to zero, wile the count for
B drops to 1.
B(1)
C(0)
D(2)
E(1)

18
Root
Object
Remove C. Reference count
for D drops to one
B, D, and E are
garbage, but their
reference counts
are all > 0. They
never get collected.
D(1)
B(1)
E(1)

19
Garbage Collectors
Reference-
Counters
Trace-
Based
Stop-the-World
Short-Pause
Mark-and-
Sweep
Mark-and-
Compact
Basic

• Mark: Starting from the root set, search all
references to discover reachable objects.
• Sweep : Return all unreached objects to the free
list.
20

Free = not holding an object; available for
allocation.
Unreached = Holds an object, but has not yet
been reached from the root set.
Unscanned = Reached from the root set, but
its references not yet followed.
Scanned = Reached and references followed.
21

1. Assume all objects in Unreached state.
2. Start with the root set. Put them in state
Unscanned.
3. while Unscanned objects remain do
examine one of these objects;
make its state be Scanned;
add all referenced objects to Unscanned
list if they were previously Unreached;
end;
22

23
• Place all objects still in the Unreached state into
the Free state.
• Question: how do we find all the chunks in the
heap?
• Place all objects in the Scanned state into the
Unreached state.
• To prepare for the next mark-and-sweep.

24
Garbage Collectors
Reference-
Counters
Trace-
Based
Stop-the-World
Short-Pause
Mark-and-
Sweep
Mark-and-
Compact
Basic
Baker’s

25
• Problem: The basic algorithm takes time
proportional to the heap size.
• Because you must visit all objects to see if they are
Unreached.
• Baker’s algorithm keeps a list of all allocated
chucks of memory, as well as the Free list.

26
• Key change: In the sweep, look only at the
list of allocated chunks.
• Those that are not marked as Scanned are
garbage and are moved to the Free list.
• Those in the Scanned state are put in the
Unreached state.
• For the next collection.
• Question: What is the downside to linking
the allocated chunks?

27
Garbage Collectors
Reference-
Counters
Trace-
Based
Stop-the-World
Short-Pause
Mark-and-
Sweep
Mark-and-
Compact
Basic Baker’s Basic

• Compact = move reachable objects to
contiguous memory.
1. Locality – fewer pages or cache-lines needed
to hold the active data.
2. Avoids fragmentation – after GC, the free
space is one large chunk, so there is space to
store large objects.
28

29
1. Mark reachable objects as before.
2. Create a table (hash?) from reached chunks
to new locations for the objects in those
chunks.
3. Scan chunks from low end of heap.
4. Maintain pointer free that counts how much
space is used by reached objects so far.
• That is, free points to the first byte that has not
been used for the reached objects so far.

30
5. Move all reached objects to their new
locations, and also retarget all references in
those objects to the new locations.
• Use the table of new locations.
6. Retarget root references.

31
Consider first chunk,
but it is free, so skip
over it
free

32
Enter into table that
this is where the
first object will go
Consider first object
free
New address is
current value of “free”

33
Consider next chunk,
but it is free
free
Advance “free” to
next available byte

34
Consider next chunk
free
New address is current “free”;
enter that into table

35
free
Advance “free” to
the next available byte

36
Eventually advance to
the next used chunk
free
New address is current “free”;
enter that into table

37
free
Advance “free” to
the next available byte

38
Change internal pointer to
second object to point to new
location of second object
Start the Compact phase.
Move first object
to its new location
free

39
Change internal pointer to
third object to point to new
location of third object
Move second object
to its new location
free

40
Change internal pointer to
first object to point to new
location of first object
Move third object
to its new location
free

41
Garbage Collectors
Reference-
Counters
Trace-
Based
Stop-the-World
Short-Pause
Mark-and-
Sweep
Mark-and-
Compact
Basic Baker’s Basic Cheney’s

• Uses 2 heaps.
• Allocate space in one.
• When the first is full, copy reachable objects
to the second heap.
• Then, swap roles.
42

43
• To copy: maintain table of new locations for
the reachable objects.
• As soon as an object is reached, give it the
next available bytes in the second heap.
• As you scan objects, adjust their references to
point to second heap.
• Trick: scan recursively, starting with the root
objects, so by the time you finish with an object,
all the objects it references have their new
locations.

Object scan(Object: O) {
select new location o for O in the other heap,
and enter that location into table;
for (each reference r in O to an object P)
if (exists entry in the table for P)
r = that entry;
else
r = scan(P);
return o;
}
44

45
• There is an analogy between program memory
management and file-system disk
management:
• Heap :: volume or partition
• Object :: file
• Cache line or page :: track or cylinder

• A file system can also benefit from compaction,
for faster file access, even if there is no need for
GC.
• Question: can you garbage collect a file system?
• Modern distributed file systems replicate files in
several places for availability and fault
tolerance.
• Consider a replicated volume holding many
files.
• Each copy of the volume holds the same files, but
the location within the volume is unimportant.
46

• Because the volume is replicated, we can take
one copy “off-line” and compact it using the
basic compaction algorithm.
• Other copies are still available for reads and
writes.
• After the compacted copy of the volume comes
back on-line, create within it copies of any files
written to the other volume-copies while it was
off-line.
47

48
Garbage Collectors
Reference-
Counters
Trace-
Based
Stop-the-World
Short-Pause
Mark-and-
Sweep
Mark-and-
Compact
Incremental
Partial
Basic Baker’s Basic Cheney’s

1. Incremental – run garbage collection in
parallel with mutation (execution of the
program).
2. Partial – stop the mutation, but only briefly,
to garbage collect a part of the heap.
49

50
• OK to mark garbage as reachable.
• Not OK to GC a reachable object.
• If a reference r within a Scanned object is
mutated to point to an Unreached object,
the latter may be garbage-collected anyway.
• Subtlety: How can a reference be made to point
to an Unreached object?
• Answer: a new object O be created during the
marking, and a scanned object may refer to O.

51
• Intercept every write of a reference in a
scanned object.
• Place the new object referred to on the
Unscanned list.
• A trick: protect all pages containing Scanned
objects.
• A hardware interrupt will invoke the fixup.

52
Garbage Collectors
Reference-
Counters
Trace-
Based
Stop-the-World
Short-Pause
Mark-and-
Sweep
Mark-and-
Compact
Incremental
Partial
Basic Baker’s Basic Cheney’s
Generational

53
• “Most objects die young.”
• But those that survive one GC are likely to survive
many.
• Tailor GC to spend more time on regions of the
heap where objects have just been created.
• Gives a better ratio of reclaimed space per unit time.

54
• We collect one part(ition) of the heap.
• The target set.
• Other objects are the stable set.
• We maintain for each partition a remembered
set of those objects in the stable set that refer
to objects in the target set.
• Write barriers can be used to maintain the
remembered set.

55
• To collect a part of the heap:
1. Add the remembered set for that partition to the
root set.
2. Do a reachability analysis (mark) as before.
• Note the resulting Scanned set may include
garbage. Why?

56
In the remembered set
The target
partition
Not reached from
the root set
Stable set

57
• Divide the heap into partitions P 0 , P 1 , …
• Each partition holds older objects than the one before
it.
• Create new objects in P 0 , until it fills up.
• Garbage collect P 0 only, and move the reachable
objects to P 1 .
• When P 1 fills, garbage collect P 0 and P 1 , and put
the reachable objects in P 2 .
• In general: When P i fills, collect P 0 , P 1 , … ,P i and
put the reachable objects in P i +1 .

58
Garbage Collectors
Reference-
Counters
Trace-
Based
Stop-the-World
Short-Pause
Mark-and-
Sweep
Mark-and-
Compact
Incremental
Partial
Basic Baker’s Basic Cheney’s
Generational
Train

59
• Problem with generational GC:
1. Occasional total collection (last partition).
2. Long-lived objects move many times.
• Train algorithm useful for long-lived objects.
• Replaces higher-numbered partitions in
generational GC.
• I.e., first partition is not part of the “train,” and is used for
new objects just as in the previous algorithm.

60
Train 1
Car 11
Car 12
Car 13
Train 2
.
.
.
Car 21
Car 22
. . .
Car 2k
Train n
Car n1
Car n2

61
• There can be any number of trains, and each
train can have any number of cars.
• You need to decide on a policy that gives a
reasonable number of each.
• Objects entering the system can:
• Be placed in the last car of the last train,
• Or start a new car at the end of the last train,
• Or even start a new train (which becomes the last).

62
• Each car has a remembered set of references
from
1. Higher-numbered trains and
2. Higher-numbered cars of the same train.
• Important: since we only garbage-collect first
cars and first trains, we never need to worry
about “forward” references (to later trains or
later cars of the same train).

63
1. Collect the first car of the first train.
• Like we collect P 0 in generational GC.
2. Collect the entire first train if there are no
references from the root set or other trains.
• This is how we find and eliminate large, cyclic
garbage structures.

1. Do a partial collection as before, using
every other car/train as the stable set.
2. Move all Scanned (reachable) objects of
the first car somewhere else.
3. Get rid of the car.
64

65
• If object o has a reference from another train,
pick one such train and move o to that train.
• Same car as reference, if possible, else make new
car.
• If references only from root set or first train,
move o to another car of first train, or create
new car.

66
• The problem: it is possible that when collecting
the first car, nothing is garbage.
• We may then create a new last car of the first
train with the same objects as the old first car.
• Situation is rare, but possible, requiring:
1. A cyclic, but nongarbage structure in the first train.
2. A mutator that changes references to the first train
with perfect timing so that we never see anything
in the remembered sets for the first car from
outside the first train.

67
• If that happens, we go into panic mode, which
requires that:
1. If a reference to any object in the first train is
rewritten, we make the new reference a “dummy”
member of the root set.
2. During panic-mode GC, if we encounter a
reference from the dummy “root set,” we move
the referenced object to another train.
• Eventually, all reachable objects will leave the
first train, leaving cyclic garbage, and we can
delete the first train.

68
Garbage Collectors
Reference-
Counters
Trace-
Based
Stop-the-World
Short-Pause
Mark-and-
Sweep
Mark-and-
Compact
Incremental
Partial
Basic Baker’s Basic Cheney’s
Generational
Train