Introduction to the DirectX 9 Shader Models - Nvidia
Introduction to the DirectX 9 Shader Models - Nvidia
Introduction to the DirectX 9 Shader Models - Nvidia
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
Precision Pitfalls<br />
• An IEEE-like fp16 is 1 sign bit, 5 bits of exponent<br />
and 10 bits of mantissa<br />
• Think of mantissa as tick marks on a ruler<br />
• The Exponent is <strong>the</strong> length of ruler<br />
– +/-1024 ticks, no matter what, so .1% precision across<br />
whatever range you have<br />
Inches<br />
Feet