Lecture 6: Tomasulo Algorithm – register renaming and tag-based ...

Lecture 6: Tomasulo Algorithm –register renaming and tag-baseddependence checkTomasulo design, big example1

Scoreboarding MeritEnable out-of-order execution and reduces stallcycles caused by RAW dependencesUse register result status table to detect WAW and RAWdependence between a newly fetched instruction and pendinginstructionsRecord dependence into the FU status of scoreboard (in formsof Qj, Qk and Rj, Rk)Keep an instruction waiting if it has dependences with pendinginstructionsWhen an inst writes result, wake up dependent instructions bymatching the inst’s FU index with Qj and Qk of every FU statusentry2

Good Dynamic Scheduling NeedsMoreBetter handling of WAR and WAW dependences Scoreboarding: (1) Stalls issuing when WAW is detected; (2)Delay writing results when WAR is detectedIs it necessary to enforce WAR and WAW dependences?Better handling of structure hazardsWhy stall pipeline when two instructions go to the same FU?(Particular a problem for memory/integer instructions)Better pipeline efficiencyTwo extra cycles between the EXs of two dependentinstructions – Need data forwardingMore ILP beyond a basic blockNeed speculative execution, branch predictions, and dynamicmemory disambiguation3

Tomasulo AlgorithmHistory:1966: scoreboarding in CDC6600, implementinglimited dynamic schedulingThree years later: Tomasulo in IBM 360/91,introducing register renaming and reservation stationNow appearing in todays Dec Alpha, SGI MIPS, SUNUltraSparc, Intel Pentium, IBM PowerPC and others indifferent formsapted from UCB CS252 S98, Copyright 1998 USB4

What Tomasulo ProvidesBetter handling of WAR and WAW dependences Use register renaming to remove WAR and WAW dependences –No stalls or delays anymoreBetter handling of structure hazardsMultiple reservation stations per FU – instruction is assigned toa reservation stationBetter pipeline efficiencyOne extra (instead of two) between EXs of two dependentinstructionsDynamic memory disambiguationEnforce dependence between stores and loadsDistributed scheduling logicRegister and RS status no longer centralized5

Three Stages of Tomasulo Algorithm1. Issue—get instruction from FP Op Queue Condition: a free RS at the required FU Actions: (1) decode the instruction; (2) allocate a RS; (3)do source register renaming; (4) do dest registerrenaming; (5) read register file; (6) dispatch the decodedand renamed instruction to the RS entry2. Execution—operate on operands (EX) Condition: At a given FU, At lease one instruction is ready Action: select a ready instruction and send it to the FU3. Write result—finish execution (WB) Condition: At a given FU, some instruction finishes FUexecution Actions: (1) the FU writes the result onto the CommonData (store inst writes to memory instead); (2) the tag(RS index) is broadcast to all other RS to wake uppotential waiting instructions; (3) update register status;(4) de-allocate the RSdapted from UCB CS252 S986

Reservation Station ComponentsOp—Operation to perform in the unit (e.g., + or –)Vj, Vk—Value of Source operands Store buffers has V field, result to be storedQj, Qk—Reservation stations producing sourceregisters Note: No ready flags as in Scoreboard; Qj,Qk=0=> ready Store buffers only have Qi for RS producing resultA – to hold memory address for load andstore instructoinsBusy—Indicates reservation station or FU is busyHow are dependences represented in reservationstation?dapted from UCB CS252 S98B8

Register Renaming in TomasuloRegister Result Status Table: Indicates whichRS will write each register, if one exists. Blankwhen no pending instructions that will writethat register.Is very similar to the one in scoreboardingPerforms register renaming to remove name(WAR and WAW) dependencesModern term: register alias table (sometimesregister renaming table)9

Register Renaming in TomasuloSource Renaming: rename source register tomapped RS index (if renamed)Destination renaming: record the renamingfrom the destination register to allocated RSindex; used in future renamingExample:LD F2,45(R3) => LD (LD1),45(R3)MULT F0, F2, F4 => MULT (MULT1), (LD1), F4Key difference from that in scoreboarding: use RSindex as tag of the operands10

Renaming Implementation0- - - …F0 F2 F4LD F2,45(R3) => LD (LD1), 45(R3)PdRd Rs RtRenaming1- LD1 - …F0 F2 F4MULT F0, F2, F4 => MULT (MULT1),(LD1), F42 Mult1 LD1 -F0 F2 F4…11PsPtFor every inst:1. Translate sourceregisters ifrenamed2. Record therenaming for dest

Selection of Instruction for ExecutionOnly “ready” instructions can join thecompetitionThere is a select logic to selectinstructions for FU execution Some policy may be used, e.g. age basedNon-ready instructions can be “wakenup” during writeback of its parent inst12

Writeback and Common DataBusNormal data bus: data + destination (“go to”bus)Common data bus: data + source (“come from”bus) 64 bits of data + 4 bits of source index (tag) Does the broadcast to every instruction in theflyChild instructions do tag matching andupdate their ready bits and value fields (ifthe tag matches theirs)Register update involves tag matchingapted from UCB CS252 S98, Copyright 1998 USB13

Code ExampleLD F6,34(R2)LD F2,45(R3)MULTI F0,F2,F4SUBD F8,F6,F2DIVD F10,F0,F6ADD F6,F8,F2LD1SUBDADDLD2MULTIDIVDOperation latencies: load/store 2 cycles,Add/sub 2 cycles, Mult 10 cycles, divide 40 cycle14

What to Observe1. Whether some instructions can be issued=> pay attention to (1) RS (or load/store buffer)allocation (decode, RS allocation, source registerrenaming, dispatch); (2) change to registerstatus (dest renaming)2. Whether some instruction can be selected forexecution (for every FU)=> Pay attention to instruction status change;the inst will finish in a given number of cycle3. Whether some instruction is finishing execution=> Pay attention to instruction status change;the inst may write its result the next cycle4. Whether some instruction is writing result=> Pay attention to (1) wakeup of thedependent instructions; (2) register statuschange; (3) RS de-allocation15

Tomasulo Example Cycle 0Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 Load1 NoLD F2 45+ R3 Load2 NoMULTDF0 F2 F4 Load3 NoSUBD F8 F6 F2DIVD F10 F0 F6ADDDF6 F8 F2Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No0 Add2 No0 Add3 No0 Mult1 No0 Mult2 NoRegister result statusClock F0 F2 F4 F6 F8 F10 F12 ... F300 FUapted from UCB CS252 S98, Copyright 1998 USB16

Tomasulo Example Cycle 1Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 Load1 No Yes 34+R2LD F2 45+ R3 Load2 NoMULTDF0 F2 F4 Load3 NoSUBD F8 F6 F2DIVD F10 F0 F6ADDDF6 F8 F2Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No0 Add2 NoAdd3 No0 Mult1 No0 Mult2 NoRegister result statusClock F0 F2 F4 F6 F8 F10 F12 ... F301 FU Load1apted from UCB CS252 S98, Copyright 1998 USB17

Tomasulo Example Cycle 2Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 Load1 Yes 34+R2LD F2 45+ R3 2 Load2 Yes 45+R3MULTDF0 F2 F4 Load3 NoSUBD F8 F6 F2DIVD F10 F0 F6ADDDF6 F8 F2Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No0 Add2 NoAdd3No0 Mult1 No0 Mult2 NoRegister result statusClock F0 F2 F4 F6 F8 F10 F12 ... F302 FU Load2 Load1apted from UCB CS252 S98, Copyright 1998 USB18

Tomasulo Example Cycle 3Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 Load1 Yes 34+R2LD F2 45+ R3 2 Load2 Yes 45+R3MULTDF0 F2 F4 3 Load3 NoSUBD F8 F6 F2DIVD F10 F0 F6ADDDF6 F8 F2Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No0 Add2 NoAdd3No0 Mult1 Yes MULTD R(F4) Load20 Mult2 NoRegister result statusClock F0 F2 F4 F6 F8 F10 F12 ... F303 FU Mult1 Load2 Load1• Note: registers names are removed (“renamed”) in ReservationStations• Load1 completing; what is waiting for Load1?apted from UCB CS252 S98, Copyright 1998 USB19

Tomasulo Example Cycle 4Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 Load2 Yes 45+R3MULTDF0 F2 F4 3 Load3 NoSUBD F8 F6 F2 4DIVD F10 F0 F6ADDDF6 F8 F2Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 Yes SUBD M(34+R2) Load20 Add2 NoAdd3No0 Mult1 Yes MULTD R(F4) Load20 Mult2 NoRegister result statusClock F0 F2 F4 F6 F8 F10 F12 ... F304 FU Mult1 Load2 M(34+R2) Add1• Load2 completing; what is waiting for it?apted from UCB CS252 S98, Copyright 1998 USB20

Tomasulo Example Cycle 5Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTDF0 F2 F4 3 Load3 NoSUBD F8 F6 F2 4DIVD F10 F0 F6 5ADDDF6 F8 F2Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk2 Add1 Yes SUBD M(34+R2) M(45+R3)0 Add2 NoAdd3No10 Mult1 Yes MULTDM(45+R3) R(F4)0 Mult2 Yes DIVD M(34+R2) Mult1Register result statusClock F0 F2 F4 F6 F8 F10 F12 ... F305 FU Mult1 M(45+R3) M(34+R2) Add1 Mult2apted from UCB CS252 S98, Copyright 1998 USB21

Tomasulo Example Cycle 6Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTDF0 F2 F4 3 Load3 NoSUBD F8 F6 F2 4DIVD F10 F0 F6 5ADDDF6 F8 F2 6Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk1 Add1 Yes SUBD M(34+R2) M(45+R3)0 Add2 Yes ADDD M(45+R3) Add1Add3No9 Mult1 Yes MULTDM(45+R3) R(F4)0 Mult2 Yes DIVD M(34+R2) Mult1Register result statusClock F0 F2 F4 F6 F8 F10 F12 ... F306 FU Mult1 M(45+R3) Add2 Add1 Mult2apted from UCB CS252 S98, Copyright 1998 USB22

Tomasulo Example Cycle 7Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTDF0 F2 F4 3 Load3 NoSUBD F8 F6 F2 4 7DIVD F10 F0 F6 5ADDDF6 F8 F2 6Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 Yes SUBD M(34+R2) M(45+R3)0 Add2 Yes ADDD M(45+R3) Add1Add3No8 Mult1 Yes MULTDM(45+R3) R(F4)0 Mult2 Yes DIVD M(34+R2) Mult1Register result statusClock F0 F2 F4 F6 F8 F10 F12 ... F307 FU Mult1 M(45+R3) Add2 Add1 Mult2• Add1 completing; what is waiting for it?apted from UCB CS252 S98, Copyright 1998 USB23

Tomasulo Example Cycle 8Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTDF0 F2 F4 3 Load3 NoSUBD F8 F6 F2 4 7 8DIVD F10 F0 F6 5ADDD F6 F8 F2 6Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No2 Add2 Yes ADDD M()-M() M(45+R3)0 Add3 No7 Mult1 Yes MULTD M(45+R3) R(F4)0 Mult2 Yes DIVD M(34+R2) Mult1Register result statusClock F0 F2 F4 F6 F8 F10 F12 ... F308 FU Mult1 M(45+R3) Add2 M()-M() Mult2apted from UCB CS252 S98, Copyright 1998 USB24

Tomasulo Example Cycle 9Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTDF0 F2 F4 3 Load3 NoSUBD F8 F6 F2 4 7 8DIVD F10 F0 F6 5ADDDF6 F8 F2 6Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No1 Add2 Yes ADDD M()–M() M(45+R3)0 Add3 No6 Mult1 Yes MULTDM(45+R3) R(F4)0 Mult2 Yes DIVD M(34+R2) Mult1Register result statusClock F0 F2 F4 F6 F8 F10 F12 ... F309 FU Mult1 M(45+R3) Add2 M()–M() Mult2apted from UCB CS252 S98, Copyright 1998 USB25

Tomasulo Example Cycle 10Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTDF0 F2 F4 3 Load3 NoSUBD F8 F6 F2 4 7 8DIVD F10 F0 F6 5ADDDF6 F8 F2 6 10Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No0 Add2 Yes ADDD M()–M() M(45+R3)0 Add3 No5 Mult1 Yes MULTDM(45+R3) R(F4)0 Mult2 Yes DIVD M(34+R2) Mult1Register result statusClock F0 F2 F4 F6 F8 F10 F12 ... F3010 FU Mult1 M(45+R3) Add2 M()–M() Mult2• Add2 completing; what is waiting for it?apted from UCB CS252 S98, Copyright 1998 USB26

Tomasulo Example Cycle 11Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTDF0 F2 F4 3 Load3 NoSUBD F8 F6 F2 4 7 8DIVD F10 F0 F6 5ADDD F6 F8 F2 6 10 11Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No0 Add2 No0 Add3 No4 Mult1 Yes MULTD M(45+R3) R(F4)0 Mult2 Yes DIVD M(34+R2) Mult1Register result statusClock F0 F2 F4 F6 F8 F10 F12 ... F3011 FU Mult1 M(45+R3) (M-M)+M() M()ŠM() Mult2• Write result of ADDD here vs. scoreboard?apted from UCB CS252 S98, Copyright 1998 USB27

Tomasulo Example Cycle 12Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTD F0 F2 F4 3 Load3 NoSUBDF8 F6 F2 4 7 8DIVDF10 F0 F6 5ADDDF6 F8 F2 6 10 11Reservation Stations S1 S2 RS for j RS for kTime NameBusyOp Vj Vk Qj Qk0 Add1 No0 Add2 No0 Add3 No3 Mult1 Yes MULTDM(45+R3) R(F4)0 Mult2 Yes DIVD M(34+R2) Mult1Register result statusClock F0 F2 F4 F6 F8 F10 F12 ... F3012 FU Mult1 M(45+R3) (M-M)+M()M()–M()Mult2apted from UCB CS252 S98, Copyright 1998 USB28

Tomasulo Example Cycle 13Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTDF0 F2 F4 3 Load3 NoSUBD F8 F6 F2 4 7 8DIVD F10 F0 F6 5ADDDF6 F8 F2 6 10 11Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No0 Add2 NoAdd3No2 Mult1 Yes MULTDM(45+R3) R(F4)0 Mult2 Yes DIVD M(34+R2) Mult1Register result statusClock F0 F2 F4 F6 F8 F10 F12 ... F3013 FU Mult1 M(45+R3) (M–M)+M() M()–M() Mult2apted from UCB CS252 S98, Copyright 1998 USB29

Tomasulo Example Cycle 14Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTDF0 F2 F4 3 Load3 NoSUBD F8 F6 F2 4 7 8DIVD F10 F0 F6 5ADDDF6 F8 F2 6 10 11Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No0 Add2 No0 Add3 No1 Mult1 Yes MULTDM(45+R3) R(F4)0 Mult2 Yes DIVD M(34+R2) Mult1Register result statusClock F0 F2 F4 F6 F8 F10 F12 ... F3014 FU Mult1 M(45+R3) (M–M)+M() M()–M() Mult2apted from UCB CS252 S98, Copyright 1998 USB30

Tomasulo Example Cycle 15Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTDF0 F2 F4 3 15 Load3 NoSUBD F8 F6 F2 4 7 8DIVD F10 F0 F6 5ADDDF6 F8 F2 6 10 11Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No0 Add2 NoAdd3No0 Mult1 Yes MULTDM(45+R3) R(F4)0 Mult2 Yes DIVD M(34+R2) Mult1Register result statusClock F0 F2 F4 F6 F8 F10 F12 ... F3015 FU Mult1 M(45+R3) (M–M)+M() M()–M() Mult2• Mult1 completing; what is waiting for it?apted from UCB CS252 S98, Copyright 1998 USB31

Tomasulo Example Cycle 16Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTDF0 F2 F4 3 15 16 Load3 NoSUBD F8 F6 F2 4 7 8DIVD F10 F0 F6 5ADDDF6 F8 F2 6 10 11Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No0 Add2 NoAdd3No0 Mult1 No40 Mult2 Yes DIVD M*F4 M(34+R2)Register result statusClock F0 F2 F4 F6 F8 F10 F12 ... F3016 FU M*F4 M(45+R3) (M–M)+M() M()–M() Mult2• Note: Just waiting for divideapted from UCB CS252 S98, Copyright 1998 USB32

Tomasulo Example Cycle 55Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTDF0 F2 F4 3 15 16 Load3 NoSUBD F8 F6 F2 4 7 8DIVD F10 F0 F6 5ADDDF6 F8 F2 6 10 11Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No0 Add2 NoAdd3No0 Mult1 No1 Mult2 Yes DIVD M*F4 M(34+R2)Register result statusClock F0 F2 F4 F6 F8 F10 F12 ... F3055 FU M*F4 M(45+R3) (M–M)+M() M()–M() Mult2apted from UCB CS252 S98, Copyright 1998 USB33

Tomasulo Example Cycle 56Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTDF0 F2 F4 3 15 16 Load3 NoSUBD F8 F6 F2 4 7 8DIVD F10 F0 F6 5 56ADDDF6 F8 F2 6 10 11Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No0 Add2 NoAdd3No0 Mult1 No0 Mult2 Yes DIVD M*F4 M(34+R2)Register result statusClock F0 F2 F4 F6 F8 F10 F12 ... F3056 FU M*F4 M(45+R3) (M–M)+M() M()–M() Mult2• Mult 2 completing; what is waiting for it?apted from UCB CS252 S98, Copyright 1998 USB34

Tomasulo Example Cycle 57Instruction status Execution WriteInstruction j k Issue complete Result Busy AddressLD F6 34+ R2 1 3 4 Load1 NoLD F2 45+ R3 2 4 5 Load2 NoMULTDF0 F2 F4 3 15 16 Load3 NoSUBD F8 F6 F2 4 7 8DIVD F10 F0 F6 5 56 57ADDDF6 F8 F2 6 10 11Reservation Stations S1 S2 RS for j RS for kTime Name Busy Op Vj Vk Qj Qk0 Add1 No0 Add2 NoAdd3No0 Mult1 No0 Mult2 NoRegister result statusClock F0 F2 F4 F6 F8 F10 F12 ... F3057 FU M*F4 M(45+R3) (M–M)+M() M()–M() M*F4/M• Again, in-oder issue,out-of-order execution, completionapted from UCB CS252 S98, Copyright 1998 USB35

The Use of TagIn Tomasulo, RS index is used as tag (tag isa modern term).Tag is a unique identifier for a pendingregister resultTag decouples the register result from thearchitectural register specifierTag removes WAR and WAWdependences without changing RAWdependencesIt is not necessary to use RS index as tag!36

Machine CorrectnessE(D,P) = E(S,P) ifE(D,P) and E(S,P) execute the same set ofinstructionsFor any inst i, i receives the outputs in E(D,P)of its parents in E(S,P)In E(D,P) any register or memory wordreceives the output of inst j, where j is thelast instruction writes to the register ormemory word in E(S,P)RAW and WAW dependences are notnecessary for a correct execution!37

Tomasulo SummaryReservations stations: Increases effective register number Distributes scheduling logicRegister renaming: Avoids WAR and WAWdependenceTag + Data broadcasting for waking up childinstructionsPros: can be effectively combined withspeculative executionCons: CDB broadcasting adds one-cycle delay(addressed in modern instruction scheduling)apted from UCB CS252 S98, Copyright 1998 USB38