Calibration of Dial Gauges - ITTC

ITTC – RecommendedProceduresSample Work InstructionsCalibration of Dial Gauges7.6-02-05Page 1 of 11Effective Date2002Revision00CONTENTSPURPOSEWORK INSTRUCTION1. Introduction2. Item and condition of calibration2.1 Items and Main Tools of Calibration2.2 Calibration Conditions3. Technical requirements and calibration method3.1 Exterior3.2 Relationship Among Parts3.3 Relative positions of point hand and dial plate3.4 Width of Indicator Needle End and Dial Plate Lines3.5 Roughness of Measuring Head Working Surface3.6 Diameter of clamping sleeve3.7 Measuring force3.8 Hysteresis of Indication3.9 Effect on indication from radial force of survey rod3.10 Error of indication3.11 Error of Return4 Treatment of calibration result and calibration period4.1 Treatment of Calibration Result4.2 Calibration PeriodAPPENDIXCalibration method for additional pointsSourceVerification regulation of dial gauge (reading in 0.01 mm and 0.001 mm)[Issued on Feb. 14, 1996 and put into effect since June 1, 1996 by National Technical Bureau - JJG 34—96, NationalMeasuring Verification Regulation of People’s Republic of China]EditedApprovedQuality Systems Group of 23 rd ITTC 23 rd ITTC 2002DateDate

ITTC – RecommendedProceduresSample Work InstructionsCalibration of Dial Gauges7.6-02-05Page 2 of 11Effective Date2002Revision00Calibration of Dial GaugesPURPOSEThis working instruction can be applied tothe verification of new dial gauges or to thecalibration of dial gauges in service and afterrepair, having a scale division of 0.01 mm,measuring ranges 0~3、0~5、0~10 mm and witha scale division of 0.001 mm and measuringranges between 0~1、0~2、0~3、0~5mm.WORK INSTRUCTION1. IntroductionDial gauge is a metrological tool whichuses the rack gear or the level gear operation totransfer the line displacement of the survey rodinto an angular one of the pointer hand. Itsmain usage is to measure the size、the shapeand the position error of products. The outlinesof the dial gauges are shown in Figure 1 (readingin 0.01mm) and in Figure 2 (reading in0.001mm).Fig. 1 Dial gauge (with the reading in0.01mm)Where, 1 dial gauge body; 2 dial gauge ring;3 revolution point hand; 4 dial plate; 5 pointhand (needle); 6 clamping sleeve; 7 surveyrod; 8 measuring head.2. Item and condition of calibration2.1 Items and Main Tools of CalibrationThe items and the main tools of the calibrationfor dial gauges are listed in Table 1.2.2 Calibration ConditionsBefore the calibration the dial gauge mustbe kept in the room for temperature equilibriumand by the time of the calibration theroom temperature must meet the requirementsin Table 2.Fig.2 Dial gauge (with the reading in0.001mm)Where, 1 dial gauge body; 2 dial gauge ring;3 dial plate; 4 revolution point hand; 5 pointhand; 6 clamping sleeve; 7 survey rod; 8measuring head; 9 ear ring.

ITTC – RecommendedProceduresSample Work InstructionsCalibration of Dial Gauges7.6-02-05Page 3 of 11Effective Date2002Revision00Table 1No Item Main calibration toolsType of calibrationNew In Afterproduct servicerepair1 Exterior ------ + + +2 Relationship among Raster calibration instrument for dial + + +partsgauge or dial gauge calibrator3 Relative positions of Tool microscope + + +pointer hand and dialplate4 Width of pointer handend and dial plate lines5 Roughness of measuringhead working surfaceTool microscope + - +Sample block for comparison of surfaceroughness or measuring apparatusof surface roughness+ - +First class + - +6 Diameter of clampingsleeve7 Measuring force Force gauge + + +8 Hysteresis of indication Rigid mounting for dial gauge, leveling+ + +working platform9 Effect on indication Half cylinder side block, rigid mounting+ + +from radial force offor dial gauge, working platformsurvey rodwith reinforcement10 Error of indication Raster calibration instrument for dial + + +gauge or dial gauge calibrator11 Error of return+ + +Note: “+”means calibration must be done; “-“ means calibration may not be done3. Technical requirements and calibrationmethod3.1 ExteriorRequirement: The watch glass of the dialgauge should be transparent and clear withoutany air bubbles and obvious scores.Table 2RoomtemperatureTemperatureChangeper hourTime forTemperaturebalance20±10 ℃ ≯ 1℃ ≥ 2h

ITTC – RecommendedProceduresSample Work InstructionsCalibration of Dial Gauges7.6-02-05Page 4 of 11Effective Date2002Revision003.1.1 The lines of the dial plate should appearclear and straight to the eye, withoutany breaks and uneven thickness. Themeasuring head should not have any mechanicaldamage, rust spots or obviousscoring. There should be no paint drops,poor chromium plating, burrs or any otherfault which affects the exterior quality.The name of the manufacturer or thetrademark, the scale division value and theproduction serial number should bemarked on the dial gauge.A dial gauge in service and after repairshould not have an exterior fault which affectsits accuracy.3.1.2 Calibration method: By eyesight.3.2 Relationship Among Parts3.2.1 Requirements3.2.1.1 The dial gauge ring should be steadilymovable. Its stopping should be reliable.The clearance between ring and bodyshould not have obvious looseness .3.2.1.2 The movement of the survey rod andthe return of the needle should be steadyand nimble without any phenomena likejumping, seizure, and retarded motion.3.2.1.3 The pointer hand should be tightlyfixed on the shaft. It should not be loosewhen the survey rod is moved.3.2.1.4 The survey rod should be freely movableand must not be seized after theclamping sleeve has been positioned andfastened on the dial gauge.3.2.2 Calibration method: Testing andobservation.3.3 Relative positions of point hand and dialplate3.3.1 Requirements3.3.1.1 The zero line of the dial plate shouldbe adjusted till it coincidences with theshaft direction of the survey rod, when thesurvey rod of the dial gauge locates at thefree position. This position of the needleshould meet the requirement in Table 3Table 3 (scale division)Scale mark Pointer hand locatestype of dial plate on upper-left ofzero scale mark50 4~12100 8~25200 16~503.3.1.2 The survey rod running distance of thedial gauge should exceed the end point ofthe working running distance. The exceedingrunning distance should meet therequirement written in Table 4.Table 4TypeReadingin0.01mmReadingin0.001mmMeasuringrange(mm)Scalemarktypeof dialplate(scaledivision)Runningdistancebeyondendpoint(mm)0~3 50 ≥ 0.300~5, 100 ≥ 0.500~100~1, 100 ≥ 0.050~2, 200 ≥ 0.100~30~5

ITTC – RecommendedProceduresSample Work InstructionsCalibration of Dial Gauges7.6-02-05Page 5 of 11Effective Date2002Revision003.3.1.3 When the revolving needle points toany of the integral revolutions the positionof the needle should meet the requirementsin Table 5.Table 5 (scale division)TypeReading in 0.01mmReading in 0.001mmPoint hand deviationfrom the zero position≤15≤203.3.1.4 The direction of the needle end shouldbe the same as the one of the dial plate line,without any deflection seen by eyesight.The distance between the upper surface ofthe needle end and the dial plate shouldn’texceed the values in Table 6.3.3.1.5 The end of the needle should cover30~80% length of the short scale mark.Table 6Type3.3.1 Calibration Method:.The distance between the upper surface ofthe needle end and the dial plate can beobserved by eyesight. If there is any dispute,it can be calibrated by use of a toolmicroscope with the lens at five-timesmagnification. The lens can be focused onthe upper surface of the needle and the dialplate respectively, the values can be obtainedby reading with the final motion deviceor can be read through an additionaldial gauge with an accuracy of 0.0 1mm.The difference of these two readings is thedistance between the upper surface of theneedle end and the plane of the dial plate.Readingin0.01mmReadingScale mark Distance betweentypeof dial plate pointer hand end(scale division)upper surfaceand dial plate50, 100 0.9mm100 0.9mm200 0.7mm3.4 Width of Indicator Needle End and DialPlate Lines3.4.1 Requirements: The values in Table 7should not be exceeded.3.4.2 Calibration methodThe calibration can be made by use of atool microscope. At least three lines shouldbe selectively calibrated.Table 7TypeMeasuredrange (mm)Scale mark typeof dial plate(scale division)Diameter of dialgauge ringWidth of pointer handend and dial platelines (mm)Reading in 0~3 50 42mm 0.10~0.200.01mm 0~5, 0~10 100 >42mm 0.15~0.25Reading in 0~1, 0~2, 10042mm 0.10~0.200.001mm 0~3, 0~5200>42mm>42mm0.15~0.250.10~0.20

ITTC – RecommendedProceduresSample Work InstructionsCalibration of Dial Gauges7.6-02-05Page 6 of 11Effective Date2002Revision003.5 Roughness of Measuring Head WorkingSurface3.5.1 Requirements: The values in Table 8should not be exceeded.Table 8 (μm)Material ofmeasuring headRoughness ofmeasuring headworking surfaceSteel Hard alloy GemR a 0.1 R a 0.2 R a 0.053.5.2 Calibration method: It may be calibratedusing the comparing method with asample block for the comparison of the surfaceroughness. If there is any dispute, a measuringapparatus of surface roughness can be used forcalibration.3.6 Diameter of clamping sleeve3.6.1 Requirement: The diameter should be0φ8 + −0.015mm.3.6.2 Calibration method: The calibration ofthe clamping sleeve diameter can be madeby use of a first class dial gauge.3.7 Measuring force3.7.1 Requirement: The measuring forceshould not exceed the values in Table 9.Table 9 (N)TypeMeasuredrange (mm)MaximummeasuredforceChange of measuredforce in one directionrunning distanceChange of measured forcein go and return directionat same pointReading in 0~3, 0~5, 1.5 0.5 0.40.01mm 0~10Reading in 0~1, 0~2 1.5 0.4 0.40.001mm 0~3, 0~5 1.5 0.5 0.53.7.2 Calibration Method: The calibrationshould be carried out at three positions, thebeginning、the mid and the end position ofthe working running distance for the dialgauge using a force gauge with the scaledivision not bigger than 0.1 N. After theforward direction calibration is over, let theneedle continue going for 5~10 scale divisions,then the return direction calibrationcan be carried out.The maximum force measured in the forwardrunning direction is taken as themaximum measured force of the dialgauge. The difference of the maximum andthe minimum forces measured in one directionare taken as the change of themeasured force in one direction. The differencebetween the measured forces in theforward and the return direction, at thecalibration points, can be taken as thechange of the measured force in forwardand return directions at the same point. Allof these should not exceed the values inTable 9.

ITTC – RecommendedProceduresSample Work InstructionsCalibration of Dial Gauges7.6-02-05Page 7 of 11Effective Date2002Revision00Table 10 (μm)Type Measurementrange (mm)Indication hysteresis Effect on indication from radialforce of survey rodZero class First class Zero class First classReading in 0~3, 0~5, 0~10 3 5 3 50.01mmReading in 0~1, 0~2, 0~3 0.3 0.5 0.3 0.50.001mm 0~5 0.4 0.5 0.4 0.53.8 Hysteresis of Indication3.8.1 Requirement: The hysteresis of indicationfor the dial gauge should not exceedthe values in Table 10.3.8.2 Calibration Method: The dial gaugeshould be clamped on a rigid mounting tomake the shaft line of the survey rod verticalto the leveling plane. The survey rodmust be lifted five times when the needlehas been adjusted until it points to thescale mark at the beginning, the mid andthe end positions of the range respectively.The difference between the maximum andthe minimum values of five readings eachare taken as the indication hysteresis of therelevant point. The indication hystereses atthe above mentioned three positions shouldnot exceed the values in Table 10.3.9 Effect on indication from radial force ofsurvey rod3.9.1 Requirement: The values in Table 10shouldn’t be exceeded.3.9.2 Calibration Method: The dial gaugeshould be clamped on a rigid mounting tomake the shaft line of the survey rod verticalto the working platform with reinforcement.One piece of the half cylinderside block with a radius of 10 mm (appendageof the measuring block) should bepositioned between the measuring headand the working platform. The dial gaugeat the zero position of the working rangeshould be adjusted till the point near themaximum position of the side block cylindersurface is touched. The side blockshould be moved twice for each of fourpositions, the front, the back, the left andthe right of the gauge along the directionvertical to the generating line of the sideblock respectively. The maximum value(the turning point) should be read eachtime when the measuring head touches themaximum point of the side block. The differenceof the maximum and the minimumamong eight readings should not exceedthe values in Table 10. Such a calibrationshould be performed at the middle and theend positions of the measurement range.3.10 Error of indication3.10.1 Requirements: The indication errors of0.01 and 0.001mm reading dial gaugesshould not exceed the values in Tables 11and 12 respectively.

ITTC – RecommendedProceduresSample Work InstructionsCalibration of Dial Gauges7.6-02-05Page 8 of 11Effective Date2002Revision003.10.2 Calibration Method3.10.2.1 Dial gauges with in 0.01 mm readingscan be calibrated by use of a raster calibrationinstrument for dial gauges or a calibratorfor dial gauges with 0.01mm readings.Dial gauges with 0.001 mm readingscan be calibrated by use of a raster calibrationinstrument for dial gauges or a calibratordial gauges with 0.001 mm readings.Other methods can be used for the calibration,the accuracy of which should not belower.3.10.2.2 The zero positioning of the calibrationinstrument (or the calibrator) and of thedial gauge should be adjusted at the beginningof the calibration. The indicationerror of the dial gauge with a 0.01 mmreading should be calibrated at the pointsof every 10-scale division interval in thedirection of the forward and the returntravel The indication error of dial gaugeswith a 0.001 mm reading should be calibratedat the points of every 50-scale divisioninterval in the direction of the forwardand the return travel The relevant error ofevery point can be read from the dialgauge after the needle has been shifted tothe scale divisions of the instrument, tillthe end of the working running distance.Then the survey rod can be compressed forthe needle turning 10-scale division andthe calibration can be carried out in the returndirection.During the whole calibration process thedirection of travel of the survey rod mustnot be changed right in the middle and anyadjustment must not be done on the calibrateddial gauge and on the calibrating instrument(or the calibrator). Whether theerror in the return travels positive or negativecan be determined in the same way asthat for the forward travel running distance.Table 11AccuracyclassReturn error(μm)Indication error of 0.01mm reading dial gauge (μm)Optional Optional Working running distance0.1mm 1mm 0~3mm 0~5mm 0~10mm0 5 8 10 12 14 31 7 10 15 18 20 5Table 12Measured Indication error of 0.001mm reading dial gauge (μm) Return error (μm)range (mm) Optional 0.2mm Working running distanceZero class First class Zero class First class Zero class First class0~1 3.0 3.5 4 5 2.0 2.50~2 3.0 3.5 5 6 2.0 2.50~3 3.5 4.0 7 8 2.5 3.00~5 3.5 4.0 8 9 2.5 3.0

ITTC – RecommendedProceduresSample Work InstructionsCalibration of Dial Gauges7.6-02-05Page 9 of 11Effective Date2002Revision003.10.2.1 The indication error of the measuringrange of the dial gauge can be determinedfrom the difference of the maximum andthe minimum values of the errors at eachcalibrated point within the measuring range.3.10.2.2 The indication error of a dial gaugewith a reading in 0.01 mm at the conditionof optional 1 mm can be determined fromthe difference between the maximum andthe minimum of the errors measured in eachsection of the forward travel0~1、1~2、2~3、3~4、mm.The indication error of a dial gaugewith0.001 mm readings at the condition ofoptional 0.2 mm can be determined fromthe difference between the maximum andthe minimum of the errors measured in eachsection of the direction of forwardtravel0~0.2、0.2~0.4、0.4~0.6、mm.3.10.2.3 The indication error of the dial gaugewith 0.01 mm readings within the range ofoptional 0.1 mm can be determined fromthe difference of the errors measured at twooptional adjacent points in the direction offorward travel.3.10.2.4 The calibration of the indication errorfor an in service dial gauge with 0.01 mmreadings by use of the 0.01 mm reading dialgauge calibrator can be carried out on thebasis of 20-scale division interval. After thiscalibration is over, if the indication errors ofthe working running distance and the optional1mm are close to the values in Table11 and the indication error of the adjacentpoints exceeds the values of the optional0.1mm indication error in Table 11, calibrationof additional points should be carriedout. This method is shown in the Appendix.3.11 Error of Return3.11.1 Requirement: The return error of thedial gauge should not exceed the valuesgiven in Tables 11 and 12.3.11.2 Calibration method: After the calibrationof the indication error, the maximum ofthe differences of the errors between theforward and the return directions of travelcan be taken as return error at the samepoint .Table 13 shows the data treatment of theindication and the return errors of the dialgauge with the reading in 0.01 mm.The indication error of the measuringrange :10-(-3)=13μmThe indication error within the range of theoptional 0.1mm:4-(-1)=5μmThe indication error within the range of theoptional 1mm: the indication error withinthe range of 2~3 mm is at maximum.10-(-1)=11μmThe return error: 2-(-1)=3μm4 Treatment of calibration result andcalibration period4.1 Treatment of Calibration ResultFor the dial gauges that meet the requirementsof this working instruction during calibration,a calibration report will be supplied;For the ones which do not meet the requirementsa notice of the calibration result will besupplied.4.2 Calibration PeriodThe calibration period can be determinedupon the service condition, but not longer thanone-year.

ITTC – RecommendedProceduresSample Work InstructionsCalibration of Dial Gauges7.6-02-05Page 10 of 11Effective Date2002Revision00Table 13Distance ofCalibrated point (scale division)travel(mm) 0 10 20 30 40 50 60 70 80 90 0Error (μm)0~1 Forward 0 0 0 +4 +6 +2 +2 0 0 +1 +1Return +1 +2 +1 +5 +5 +4 +4 +1 0 +1 +11~2 Forward +1 0 +1 +2 +3 +6 +5 +4 +2 +2 -1Return +1 +1 +3 +4 +5 +6 +6 +3 +3 +1 +22~3 Forward -1 +4 +2 +4 +8 +8 (+10) +8 +8 +6 +6Return +2 +4 +3 +4 +8 +9 +11 +8 +7 +7 +63~4 Forward +6 +2 -1 -1 0 0 -1 0 -1 -1 -2Return +6 +1 +1 0 +1 0 +2 0 +1 0 -24~5 Forward -2 (-3) -3 -3 -1 -1 -2 -2 -1 -2 -1Return -2 -2 -2 -2 -1 -1 -2 0 0 0 0APPENDIXCalibration method for additional points1. Usage range: The calibration of the indicationerror and the return error for the dial gaugewith the reading in 0.01mm in service.2. Equipment for calibration: Calibrator fordial gauges.3. Technical requirements: The values in Table11 should not be exceeded.4. Calibration method: The calibration of theindication error for the dial gauge with thereading in 0.01 mm in service can be carriedout on the basis of 20-scale division interval inthe directions of the go and the return distance.After the calibration, if the indication errors ofthe measuring range and the optional 1 mm areclose to values in Table 11, the calibration ofthe additional points in the area of the maximum,the minimum errors in the forward distanceof travel and the maximum, the minimumerrors of the optional 1 mm should be carriedout respectively.During the calibration of the additionalpoints the clamping position of the calibrateddial gauge should be kept the same as before.The calibrator should be adjusted to the positions0.1 mm before and after the point of themaximum error in the go running distance forthe additional points calibration respectively. Ifthere is an error in an additional point that isbigger than the original maximum, this error ofthe additional point should be taken as maximumerror.Similarly, the calibration of the additionalpoints should be carried out at the positions0.1mm before and after the point of the minimumerror in the forward distance of travel ofthe calibrated dial gauge. If there is an error inan additional point which is smaller than theoriginal minimum, this error of the additionalpoint should be taken as minimum error.The same method should be used for thecalibration of the additional points in the area

ITTC – RecommendedProceduresSample Work InstructionsCalibration of Dial Gauges7.6-02-05Page 11 of 11Effective Date2002Revision00of the maximum and the minimum errors forthe optional 0.1 mm.If the indication error of the adjacent pointsexceeds values of the indication error for theoptional 0.1 mm in Table 11, a calibration ofthe middle point between these adjacent pointsshould be carried out.Table 14 shows the data treatment of the indicationerror and the return error for a dial gaugewith the reading in 0.01 mm in service.The indication error of the measuring range:11-(-9)=20μmThe indication error within the range of the optional1mm: 11-0=11μmThe indication error within the range of the optional0.1 mm: 11-5=6μm (at the position0.7~0.8mm of the first revolution)The return error: 0-(-4)=4μmTable 14Distance Calibrated point (scale division) Optional 1mm Optional Whole distanceof travel 0 20 40 60 80 00.1mm error(mm)Error (μm)0~1 Go 0 +2 +3 (+8) +5 +1 8R 1 +1 +3 +5 +9 +6 +3 (11)1~2 Go +1 +3 +4 +6 +6 +7 620R +3 +5 +5 +6 +5 +82~3 Go +7 -1 +1 -1 -1 -1 86Return errorR +8 +2 +3 0 0 -13~4 Go -2 -4 -3 -3 -1 -3 3R -1 -1 -2 -3 0 -14~5 Go -3 (-9) -4 -4 -3 -3 64R -1 -6 0 -3 0 -1Calibration of additional points Maximum Minimum Optional0.1mmDistance of travel (mm) 0~1 4~5 2~3Calibrated point (scale division) 50 60 70 10 20 30 0 10 20Indication error (μm) +5 +8 +1 -5 -9 -6 +7 +3 -11Note: 1 means the return distance of travel.