30.04.2014 Views

Introduction to the DirectX 9 Shader Models - Nvidia

Introduction to the DirectX 9 Shader Models - Nvidia

Introduction to the DirectX 9 Shader Models - Nvidia

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

Precision Pitfalls<br />

• An IEEE-like fp16 is 1 sign bit, 5 bits of exponent<br />

and 10 bits of mantissa<br />

• Think of mantissa as tick marks on a ruler<br />

• The Exponent is <strong>the</strong> length of ruler<br />

– +/-1024 ticks, no matter what, so .1% precision across<br />

whatever range you have<br />

Inches<br />

Feet

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!