Introduction to handwritten signature verification - ACE Electoral ...

Introduction to HandwrittenSignature VerificationDave FentonUniversity of OttawaSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 1/53

Handwritten signature verificationPresentation overview (altered to remove allsignatures):• Goal, applications and assumptions• Basic concepts in biometrics• Experimental setup• Technical difficulties posed by HSV• Past research and the current state of the art• Overview of my researchSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 2/53

Goal of HSV• To verify a person’s identity based on the way inwhich he/she signs his/her name• Two types of system:• Offline systems use static features (thesignature image)• Online systems use dynamic features (thetime series)• Written passwords are also under considerationSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 3/53

ApplicationsPrincipal application: reduce fraud in financialtransactions• Cannot rely on sales staff to visually verifysignatures on credit card receipts• Occasional acceptances of forgeries are allowable• Rejections of valid signatures may irritatevaluable customers• To date, used mostly for electronic signature ofbusiness documents (hash function protectsdocument against alteration)SPOT presentation, University of Ottawa, 29 Oct 2004 – p. 4/53

Document verification• Apply hash function to document to generate hash code• If signature is valid, encrypt hash with signer’s private key• Recipient decodes received hash using public key• If document has been altered, hashes don’t matchSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 5/53

ApplicationsSecondary application: access security for buildingsor mobile computing devices• For building security, it would not be tolerable toaccept forgeries• HSV would have to be combined with on-sitesecurity staff or other biometric/password/PINsystems• Already used on some laptops and PDAsSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 6/53

Naïve assumptions• A person signs his or her name consistently eachtime• All signatures contain enough steady features tobe reliably verified• A forger cannot perfectly imitate the dynamicfeatures of a signature• All a user’s passwords can be replaced by his/hersignatureSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 7/53

Example of consistency – staticPassword: “Prejunife”21 July 2003 2 Sept 2003 24 Sep 2003SPOT presentation, University of Ottawa, 29 Oct 2004 – p. 8/53

Example of consistency – dynamic50Y Velocity (cm/s)0−50500 0.5 1 1.5 2 2.5 30−50500 0.5 1 1.5 2 2.5 30−50500 0.5 1 1.5 2 2.5 30−50500 0.5 1 1.5 2 2.5 30−50500 0.5 1 1.5 2 2.5 30−50500 0.5 1 1.5 2 2.5 30−500 0.5 1 1.5 2 2.5 3Time (normalized)SPOT presentation, University of Ottawa, 29 Oct 2004 – p. 9/53

Example of inconsistency – staticPassword: “Ingusions”12 Jan 2004 18 Mar 2004 27 Sep 2004SPOT presentation, University of Ottawa, 29 Oct 2004 – p. 10/53

Example of inconsistency – dynamic50Y Velocity (cm/s)0−500 0.5 1 1.5 2 2.5500−500 0.5 1 1.5 2 2.5500−500 0.5 1 1.5 2 2.5500−500 0.5 1 1.5 2 2.5500−500 0.5 1 1.5 2 2.5500−500 0.5 1 1.5 2 2.5Time (normalized)SPOT presentation, University of Ottawa, 29 Oct 2004 – p. 11/53

Example of forger ability – staticPassword: “Prejunife”SPOT presentation, University of Ottawa, 29 Oct 2004 – p. 12/53

Example of forger ability – dynamic50Y Velocity (cm/s)0−500 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5500−500 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5200−200 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5500−500 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5200−200 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5500−500 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5Time (normalized)SPOT presentation, University of Ottawa, 29 Oct 2004 – p. 13/53

More realistic assumptions• Most signers sign their names consistently• Most signatures contain enough steady featuresto be reliably verified• Most forgers cannot reproduce a signature wellenough to defeat a good verifier• It is more difficult to forge both the static anddynamic features of a signature than just thestatic featuresSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 14/53

Fallout from broken assumptions• It may not be possible to verify all signaturesreliably• For any signature, there will probably exist askilled forger who can forge it competently• Serious consideration must be given to passwords• confidential• easily replaced if template compromised• can exert some control over the length(quality of features)• can request the signer to write legiblySPOT presentation, University of Ottawa, 29 Oct 2004 – p. 15/53

Handwritten signature verificationPresentation overview:• Goal, applications and assumptions• Basic concepts in biometrics• Experimental setup• Technical difficulties posed by HSV• Past research and the current state of the art• Overview of my researchSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 16/53

Basics of biometricsPhysical v. behavioural biometrics:• A physical biometric makes use of a fixedcharacteristic of the body (e.g. fingerprints, irispatterns, retina patterns, hand geometry, facialfeatures)• The most accurate methods are usually perceivedas too intrusive.• A behavioural biometric makes use of personalbehaviours which are assumed to be almostinvariant (e.g. voice, handwriting, typing, gait)• Perceived as less intrusive, but less accurate thanphysical biometricsSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 17/53

Two stages• 1. Enrolment:• a user’s signature characteristics are learnedfrom a small number of input samples. Theresulting information is called the template.• Typically, 3 – 5 signatures are used• 2. Verification or recognition:• For verification, a candidate signature iscompared to the template of a single signer.• For recognition, the candidate signature mustbe compared against many templates.SPOT presentation, University of Ottawa, 29 Oct 2004 – p. 18/53

FRR and FAR• Two error rates are specified:• The False Rejection Rate (FRR) is the rate atwhich valid signatures are rejected.• The False Acceptance Rate (FAR) is the rateat which forged signatures are accepted asvalid.• In many cases, low FRR implies high FAR, andvice-versa• Current state of the art: FRR and FAR sum to2 – 5%. Actual numbers may be even worse!SPOT presentation, University of Ottawa, 29 Oct 2004 – p. 19/53

ROC curves• Most verifiers have a single numerical output. Ifthe output level is above a decision threshold, thesignature is accepted as valid, otherwise it isrejected.• In this case, the FRR and FAR can both beplotted against the decision threshold in areceiver operating characteristic (ROC) curve.SPOT presentation, University of Ottawa, 29 Oct 2004 – p. 20/53

Example ROC curves10.9False Acceptance Rate (FAR)False Rejection Rate (FRR)Error rate0.80.70.60.50.40.30.20.100 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5Decision thresholdSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 21/53

Example ROC curves10.9False Acceptance Rate (FAR)False Rejection Rate (FRR)Error rate0.80.70.60.50.40.30.20.1Equal Error Rate00 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5Decision thresholdSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 22/53

Types of forgery• Random. A random forgery is simply anotherperson’s valid signature.• Simple. The forger spells the name correctly, butwrites in his own style.• Skilled, or Knowledgeable. The forger tries tofully reproduce all the shapes and dynamics ofthe original signature. In this study, forgers areshown MPEG movies of the original signature.• The training set consists of a few valid signaturesand many random forgeries.• After training, the verifier is tested against all 3types of forgery.SPOT presentation, University of Ottawa, 29 Oct 2004 – p. 23/53

Genuine samplesPassword: “Taximotels”19 Sep 2003 16 Oct 2003 6 Nov 2003SPOT presentation, University of Ottawa, 29 Oct 2004 – p. 24/53

ForgeriesPassword: “Taximotels”Random Simple KnowledgeableSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 25/53

Motivations for research• Despite company claims, error rates are high, andneed improvement• For computing devices with pen inputs (PDAs,tablet PCs), automatic signature verification is asensible technology• Signatures are already a widely accepted meansof identificationSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 26/53

Handwritten signature verificationPresentation overview:• Goal, applications and assumptions• Basic concepts in biometrics• Experimental setup• Technical difficulties posed by HSV• Past research and the current state of the art• Overview of my researchSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 27/53

Data collection• Signatures collected using an InterlinkElectronics ePad-ink (100 Hz samp freq)• Captures X & Y position, pressure, time stamp• Data collection program written in C++• Data protected by PGPdiskSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 28/53

Data collection• Two levels of volunteer:• Level 1: one signing session• Level 2: three signing sessions• Each volunteer contributes:• 10 samples of genuine signature• 10 samples of genuine password• Simple forgeries of 2 signatures and 2passwords• Knowledgeable forgeries of a signature and apasswordSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 29/53

¦ © ¦¡ ¥¨¡¥ ¦¥¡¦£ ¤¥ ¦ ©¤§¨ ¢¡ ¦ £¥¦¨ ¤Enrolment¦¨¦ ©¨ ¦ ¥ ¦¨ £¥¦ ¦¨ £¥¦Acquire valid signatures:• Operational systems typically collect 3 – 5genuine signatures; academic systems up to 20• Some use warping and interpolation schemes to“create” extra valid signaturesSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 31/53

& . /( &&- ! &* + &' ()##" !!& , & #, & #& ## $%EnrolmentPreprocess:• Concatenate strokes into single time sequence• Render invariant to:• translation: subtract X & Y means• rotation: force linear regression line to be horizontal• scale: may normalize based on box size or signal power• Normalization of duration is not carried out at this stageSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 32/53

68> 6 F5 6G@ >>E 6 9 61>B C >5 81?5 6@5A16;;:45 6 7 8 94123 09 8 > 6D 68> 356;D 68> 356;> 356;8; 4

NPV N ^M N_X VV] N Q NIVZ [ VM PIWM NXMYINSSRLM N O P QLIJK HQ P V N\ NPV KMNS\ NPV KMNSV KMNSPS LTUEnrolmentSelect features:• With function features, typically use the same features foreach signer• Not all discrete features are equally informative• Cost used for feature selection is usually error rate;classifier dependent• Sequential forward/backward searchSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 34/53

nu f i fanr s ne haoe fpeqafkkjde f g h id`abci h n ffhn f ve fwp nt fhn cefkt fhn cefkn cefkhk dlmEnrolmentCreate template:• Best performance so far: keep raw data ofmultiple signatures• Bad practice, from security perspective• Template may also include list of features tokeep, best classifier to use, decision thresholdsSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 35/53

• { ‚ †{| ƒ‡€ †} ‡y~ƒƒˆ‰ ‚} yVerification‚yƒ | ‡€ ~†} ‡y~ƒƒ‹ ‡Œ†}‚ ~}ƒ‚‚ ƒ{}~ƒŠ ~~ ‚ ”Ž ‚| ‡€†}Œ~{} ‡‚ {}~ƒ Ž ~y’ ~ ~ “ Š ”}~ƒ ‹ ‡€Š|„~ ‚ Ž ~‘~ | ~Œ† ~ ‚}‚ ~†}‡y~ƒƒ…} ~ ‚ ~ |} Ž yz{ x ~Œ† ‚|} ~ ‚ ~ ‚ {}~€| |„ x € ƒyz{y• Initial steps of verification are same as enrolment• Only selected features need to be extractedSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 36/53

—˜ š› œ – ž—© ¥ž¨š¢š œ¯œ±²› œ¡¬œœª¤Verification³ ¤« œ—¡ š ¥ž ¤› ¥—œ¡¡š ¡¥ž© ¥ª¤› ¤› ¥—œ¡¡ª œœ›¡— œ¡›§ š ¥ž¤› ›Ÿ¦¨œ› ¡œ› œ—œ ¬¨œ ° œœ› ® œ¤›¥—œ¡¡£›œ œ—˜Ÿ ›Ÿš–¡ š¢žœ­«œš› ª¤ ¬œ² ¥ ¬Ÿ ¡• Template ID is acquired at same time ascandidate signature• Designated by swipe card, PIN number, etc.• Template is usually stored in central database, butmay also be held on swipe cardSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 37/53

´µ·Ä Å ¾¹ »µÇ üƸÀÊ » ¾ ºVerification¾Ñ· ¾ ·¸¿ ¸Ã¼ É ºµ ¹ õº¿¿¸¿Ã¼Â¹ õº¿¿Ç ÃÈ»¹¾ º¹¿¾¾·¹º¿¾¸Ã¼Â»¹»ÈºÆ º»Ê º ºµ ¾ º ½»·¹¾·¹º¿Æ º» Î ¸ºÍº¾¹ º ÌÁ¹ ºÂ¹Ãµº¿¿´µ·º ½¸ ½¾ º » ¾·¹º ¸¹ »¼¸À¿»¼ µÐ à ʽ ¿¾ ºÈ¾ º ËÉÏ Ð¹º¿¸¹ º ¾ ºÈÂÊ »• Many different classifiers have been tried• May have to combine results from multipleclassifiers• Will be covered in more detail laterSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 38/53

Handwritten signature verificationPresentation overview:• Goal, applications and assumptions• Basic concepts in biometrics• Experimental setup• Technical difficulties posed by HSV• Past research and the current state of the art• Overview of my researchSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 39/53

Technical difficulties• Physiology of handwriting is not well understood• Signers not motivated to sign in a careful,invariant manner• Training sets are sparse and badly imbalanced(few valid signatures)• No knowledgeable forgeries available for training• Short, variable signatures often easily forgedSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 40/53

Technical difficulties• No standard database of signatures and forgeries:• every researcher uses a different set ofamateur forgeries• some researchers test only against randomforgeries• FAR is ill-defined; a low error rate may reflectthe forgers’ lack of skill rather than theverifier’s abilitySPOT presentation, University of Ottawa, 29 Oct 2004 – p. 41/53

Technical work-arounds• Disqualify certain signers during enrolment• Allow multiple signing attempts• Allow probationary period with relaxedacceptance criteria (collect more trainingsignatures)• Use passwords with a certain minimum lengthSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 42/53

Handwritten signature verificationPresentation overview:• Goal, applications and assumptions• Basic concepts in biometrics• Experimental setup• Technical difficulties posed by HSV• Past research and the current state ofthe art• Overview of my researchSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 43/53

Past research• Research has been underway for several decades.Peak activity in mid-1980s to mid-1990s.• Early ideas are still good performers because theyhave fewer control parameters• Between 30 – 60% of forgeries can be detectedby a basic time verifierSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 44/53

Time verifierGenuine signatures20015010050Total Signing TimeAccepted by time verifierRejected by time verifierSample mean = 3.58Sample deviation = 1.5300 2 4 6 8 10 12 14 16 18 20Simple forgeries15010050Sample mean = 4.46Sample deviation = 1.6000 2 4 6 8 10 12 14 16 18 20Knowledgeable forgeries10050Sample mean = 6.11Sample deviation = 2.7400 2 4 6 8 10 12 14 16 18 20Time (seconds)SPOT presentation, University of Ottawa, 29 Oct 2004 – p. 45/53

Classifiers• Euclidean distance• weighted linear metrics• regional correlation• dynamic time warping (DTW)• neural networks• hidden Markov models (HMMs)• Bayesian belief netSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 46/53

Features used• Function features (usually position, velocity andpressure)• Vectors of discrete features• Features calculated within sliding window (e.g.centre of mass, torque)• Wavelet coefficients• LPC coefficients• Walsh transform of pen-up/pen-down signal• Pre-defined strokes (HMMs)SPOT presentation, University of Ottawa, 29 Oct 2004 – p. 47/53

State of the art• In academic studies, more complicated verifiersoften achieve better results than simple verifiers• However, in field use, simple verifiers like DTWoften outperform everything else:• few adjustable parameters• with normalization, can set a single decisionthreshold for all signers• Best verifier in public contest: DTW with5-signature templateSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 49/53

State of the art• Most sophisticated verifier: Plamondon’sSign@metric solution• discrete parametric verifier• physiological delta-lognormal verifier• static feature verifier• claimed performance: error rate of 0.0003%among 86,500 people!!• Other companies that did not take part in publiccontest: CIC, Cyber-SIGN, SoftPro, WondernetSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 50/53

Handwritten signature verificationPresentation overview:• Goal, applications and assumptions• Basic concepts in biometrics• Experimental setup• Technical difficulties posed by HSV• Past research and the current state of the art• Overview of my researchSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 51/53

My research• Classifier comparison (DTW, NN, SVM,weighted distance metric)• Techniques to mitigate imbalance of training data• Re-open debate on the use of passwords• Data analysis across signing sessions• Feature selection algorithm that gives preferentialtreatment to features that are most likely to bestable• Use of support vector machineSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 52/53

Questions?To volunteer: please e-mail d.fenton@ieee.orgSPOT presentation, University of Ottawa, 29 Oct 2004 – p. 53/53