data consistency, completeness and cleaning - The INCLEN Trust

More documents

Recommendations

Info

Checking for Invalid Numeric Values• The techniques for checking invalid numeric data are quite differentfrom the techniques used with character data– Examine minimum and maximum values for each numeric variable– Internal consistency methods; if we see that most of the data values fallwithin a certain range of values, then any values that fall far enough outsidethe range may be data errors– Run a univariate analysis, focusing especially on• Number of non-missing observations, number of observation not equal to zero andthe number of observation greater than zero are of most interest at this stage• Extremes shows the five lowest and five highest values for numeric variables• Quantiles• Mean• Standard deviation to decide on constitute reasonable cutoffs for low and high datavalue• Range• Graphic displays: a stem-and leaf plot, a box plot and a normal probability plot• Check the medical records for the extreme values and write a note to thedata center about the findings to help in further cleaning of these data
Dates: Hospitalization…..(1)• We can create a variablefrom subtracting thedate of discharge fromdate of admission, andcall it totalhospitalization 1• This variable will detectany wrong data entryfor dates such as casenumber 6014
Page 8 and 9: Another Approach:Using Frequencies
Page 10 and 11: Checking for Invalid Character Valu
Page 12 and 13: Correcting Invalid Character Values
Page 16 and 17: Dates: Hospitalization…..(2)• W
Page 18: ACKNOWLEDGEMENTWe thank• Prof. Do

data consistency, completeness and cleaning - The INCLEN Trust

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?