14 1. INTRODUCTIONspecific scenario <strong>of</strong> applying s<strong>of</strong>t biometrics for human frontal-to-side re-identification. We thenchange gear and deviate from security related applications to the more commercially oriented application<strong>of</strong> employing s<strong>of</strong>t biometrics in quantifying and predicting female facial aesthetics. Theabove approaches are then complemented by a more practical automatic s<strong>of</strong>t biometric classificationtool that we present. Finally, motivated by human acceptance issues, we proceed to provide ausability study relating to s<strong>of</strong>t biometrics.1.1 Achievements and structure <strong>of</strong> the dissertationWe proceed with an explicit description <strong>of</strong> the structure <strong>of</strong> the thesis, and the introduction <strong>of</strong>the scenarios / applications <strong>of</strong> interest in each chapter.Chapter 2 - S<strong>of</strong>t biometrics: characteristics, advantages and related workIn Chapter 2 we <strong>of</strong>fer general considerations related to s<strong>of</strong>t biometrics. Firstly in Section 2.1we introduce a candidate list <strong>of</strong> traits and furthermore proceed to portray pertinent advantages andlimitations in Section 2.2. We then identify in Section 2.3 previous work on s<strong>of</strong>t biometric traits.Chapter 3 - Bag <strong>of</strong> facial s<strong>of</strong>t biometrics for human identificationChapter 3 considers the case where a SBS can distinguish between a set <strong>of</strong> traits (categories),which set is large enough to allow for the classification that achieves human identification. Theconcept <strong>of</strong> person identification based on s<strong>of</strong>t biometrics originates in the way humans performface recognition. Specifically human minds decompose and hierarchically structure complex problemsinto fractions and those fractions into further sub-fractions, see [Ley96], [Sim96]. Consequentlyface recognition performed by humans is the division <strong>of</strong> the face in parts, and subsequentclassification <strong>of</strong> those parts into categories. Those categories can be naturally <strong>of</strong> physical, adheredor behavioral nature and their palette includes colors, shapes or measurements, what we refer tohere as s<strong>of</strong>t biometrics. The key is that each individual can be categorized in terms <strong>of</strong> such characteristics,by both humans or by image processing algorithms. Although features such as hair,eye and skin color, facial hair and shape, or body height and weight, gait, cloth color and humanmetrology are generally non distinctive, a cumulative combination <strong>of</strong> such features providesan increasingly refined and explicit description <strong>of</strong> a human. SBSs for person identification haveseveral advantages over classical biometric systems, as <strong>of</strong> non intrusiveness, computational andtime efficiency, human compliance, flexibility in pose- and expression-variance and furthermorean enrolment free acquirement in the absence <strong>of</strong> consent and cooperation <strong>of</strong> the observed person.S<strong>of</strong>t biometrics allow for a reduced complexity determination <strong>of</strong> an identity. At the same timethough, the named reduced computational complexity comes with restrictions on the size <strong>of</strong> anauthentication group. It becomes apparent that a measure <strong>of</strong> performance must go beyond theclassical biometric equal error rate <strong>of</strong> the employed detectors and include a different and newparametrization. Our general interest here is to provide insightful mathematical analysis <strong>of</strong> reliability<strong>of</strong> general s<strong>of</strong>t biometric systems, as well as to concisely describe the asymptotic behavior <strong>of</strong>pertinent statistical parameters that are identified to directly affect performance. Albeit its asymptoticand mathematical nature, the approach aims to provide simple expressions that can yieldinsight into handling real life surveillance systems.In Chapter 3, Section 3.1 introduces the operational setting <strong>of</strong> a SBS. In this setting we elaborateon pertinent factors, such as those <strong>of</strong> the authentication group, traits, traits instances, overallcategories and their interrelations. We then proceed in Section 3.5.1 to introduce and explain the
15event <strong>of</strong> collision, which is <strong>of</strong> significant character when employing SBSs for person identification.Furthermore we introduce the number <strong>of</strong> effective categories F which is later identified as animportant parameter related to collision, and is shown to directly affect the overall performance <strong>of</strong>an SBS.Section 3.5.2 analyzes the statistical distribution and mean <strong>of</strong>F and furthermore Section 3.5.3<strong>of</strong>fers an insight regarding the bounds <strong>of</strong> the statistical behavior <strong>of</strong>F over large populations. Thesebounds address the following practical question: if more funds are spent towards increasing thequality <strong>of</strong> an SBS, then what reliability gains do we expect to see? The answer and further intuitionon the above bounds are provided in Section 3.5.3.1, the pro<strong>of</strong>s can be found in the Appendix 3.In Section 3.5.4 we examine the influence <strong>of</strong> algorithmic estimation errors and give an exampleon the overall performance <strong>of</strong> a realistic SBS. We improve the performance by a study <strong>of</strong> thedistribution between population in the overall categories, see Section 3.5.4.1. We then proceed inSection 3.5.4.2 to elaborate on the human compliant aspect <strong>of</strong> s<strong>of</strong>t biometrics in re–identification,hereby specifically on the quantification <strong>of</strong> traits and on the human interaction view <strong>of</strong> an SBS.Chapter 4 - Search pruning in video surveillance systemsIn Chapter 4 we explore the application using s<strong>of</strong>t-biometric related categorization-based pruningto narrow down a large search.In recent years we have experienced an increasing need to structure and organize an exponentiallyexpanding volume <strong>of</strong> images and videos. Crucial to this effort is the <strong>of</strong>ten computationallyexpensive task <strong>of</strong> algorithmic search for specific elements placed at unknown locations inside largedata sets. To limit computational cost, s<strong>of</strong>t biometrics pruning can be used, to quickly eliminate aportion <strong>of</strong> the initial data, an action which is then followed by a more precise and complex searchwithin the smaller subset <strong>of</strong> the remaining data. Such pruning methods can substantially speed upthe search, at the risk though <strong>of</strong> missing the target due to classification errors, thus reducing theoverall reliability. We are interested in analyzing this speed vs. reliability trade<strong>of</strong>f, and we focuson the realistic setting where the search is time-constrained and where, as we will see later on, theenvironment in which the search takes place is stochastic, dynamically changing, and can causesearch errors. In our setting a time constrained search seeks to identify a subject from a large set<strong>of</strong> individuals. In this scenario, a set <strong>of</strong> subjects can be pruned by means <strong>of</strong> categorization that isbased on different combinations <strong>of</strong> s<strong>of</strong>t biometric traits such as facial color, shapes or measurements.We clarify that we limit our use <strong>of</strong> “pruning the search” to refer to the categorization andsubsequent elimination <strong>of</strong> s<strong>of</strong>t biometric-based categories, within the context <strong>of</strong> a search withinlarge databases (Figure 4.1). In the context <strong>of</strong> this work, the elimination or filtering our <strong>of</strong> theemployed categories is based on the s<strong>of</strong>t biometric characteristics <strong>of</strong> the subjects. The pruneddatabase can be subsequently processed by humans or by a biometric such as face recognition.Towards analyzing the pruning behavior <strong>of</strong> such SBSs, Chapter 4 introduces the concept <strong>of</strong>pruning gain which describes, as a function <strong>of</strong> pruning reliability, the multiplicative reduction<strong>of</strong> the set size after pruning. For example a pruning gain <strong>of</strong> 2 implies that pruning managed tohalve the size <strong>of</strong> the original set. Section 4.5.1 provides average case analysis <strong>of</strong> the pruning gain,as a function <strong>of</strong> reliability, whereas Section 4.5 provides atypical-case analysis, <strong>of</strong>fering insighton how <strong>of</strong>ten pruning fails to be sufficiently helpful. In the process we provide some intuitionthrough examples on topics such as, how the system gain-reliability performance suffers withincreasing confusability <strong>of</strong> categories, or on whether searching for a rare looking subject rendersthe search performance more sensitive to increases in confusability, than searching for commonlooking subjects.In Section 4.6.1 we take a more practical approach and present nine different s<strong>of</strong>t biometric
- Page 1: FACIAL SOFT BIOMETRICSMETHODS, APPL
- Page 5: AcknowledgementsThis thesis would n
- Page 8: 6hair, skin and clothes. The propos
- Page 11 and 12: 97 Practical implementation of soft
- Page 13 and 14: 11Notations used in this workE : st
- Page 15: 13Chapter 1IntroductionTraditional
- Page 19 and 20: 17ric. In Section 6.6 we employ the
- Page 21 and 22: 19Chapter 2Soft biometrics: charact
- Page 23 and 24: 21is the fusion of soft biometrics
- Page 25 and 26: 23plied on low resolution grey scal
- Page 27 and 28: 25Chapter 3Bag of facial soft biome
- Page 29 and 30: 27In this setting we clearly assign
- Page 31 and 32: 29Table 3.1: SBSs with symmetric tr
- Page 33 and 34: 31corresponding to p(n,ρ). Towards
- Page 35 and 36: the same category (all subjects in
- Page 37 and 38: 3.5.2 Analysis of interference patt
- Page 39 and 40: an SBS by increasing ρ, then what
- Page 41 and 42: 39Table 3.4: Example for a heuristi
- Page 43 and 44: 41for a given randomly chosen authe
- Page 45 and 46: 43Chapter 4Search pruning in video
- Page 47 and 48: 45Figure 4.1: System overview.SBS m
- Page 49 and 50: 472.52rate of decay of P(τ)1.510.5
- Page 51 and 52: 49to be the probability that the al
- Page 53 and 54: 51The following lemma describes the
- Page 55 and 56: 534.5.1 Typical behavior: average g
- Page 57 and 58: 55n = 50 subjects, out of which we
- Page 59 and 60: 5710.950.9pruning Gain r(vt)0.850.8
- Page 61 and 62: 59for one person, for trait t, t =
- Page 63 and 64: 61Chapter 5Frontal-to-side person r
- Page 65 and 66: 63Figure 5.1: Frontal / gallery and
- Page 67 and 68:
6510.90.80.7Skin colorHair colorShi
- Page 69 and 70:
6710.90.80.70.6Perr0.50.40.30.20.10
- Page 71 and 72:
69Chapter 6Soft biometrics for quan
- Page 73 and 74:
71raphy considerations include [BSS
- Page 75 and 76:
73Figure 6.3: Example image of the
- Page 77 and 78:
75A direct way to find a relationsh
- Page 79 and 80:
77- Pearson’s correlation coeffic
- Page 81 and 82:
79shown to have a high impact on ou
- Page 83 and 84:
81Chapter 7Practical implementation
- Page 85 and 86:
834) Eye glasses detection: Towards
- Page 87 and 88:
857.2 Eye color as a soft biometric
- Page 89 and 90:
87Table 7.5: GMM eye color results
- Page 91 and 92:
89and office lights, daylight, flas
- Page 93 and 94:
917.5 SummaryThis chapter presented
- Page 95 and 96:
93Chapter 8User acceptance study re
- Page 97 and 98:
95Table 8.1: User experience on acc
- Page 99 and 100:
97scared of their PIN being spying.
- Page 101 and 102:
99Table 8.2: Comparison of existing
- Page 103 and 104:
101ConclusionsThis dissertation exp
- Page 105 and 106:
103Future WorkIt is becoming appare
- Page 107 and 108:
105Appendix AAppendix for Section 3
- Page 109 and 110:
107- We are now left withN −F = 2
- Page 111 and 112:
109Appendix BAppendix to Section 4B
- Page 113 and 114:
111Blue Green Brown BlackBlue 0.75
- Page 115 and 116:
113Appendix CAppendix for Section 6
- Page 117 and 118:
115Appendix DPublicationsThe featur
- Page 119 and 120:
117Bibliography[AAR04] S. Agarwal,
- Page 121 and 122:
119[FCB08] L. Franssen, J. E. Coppe
- Page 123 and 124:
121[Ley96] M. Leyton. The architect
- Page 125 and 126:
123[RN11] D. Reid and M. Nixon. Usi
- Page 127 and 128:
125[ZG09] X. Zhang and Y. Gao. Face
- Page 129:
2Rapporteurs:Prof. Dr. Abdenour HAD
- Page 132 and 133:
Biométrie faciale douce 2Les terme
- Page 134 and 135:
Biométrie faciale douce 4une perso
- Page 136 and 137:
Couleur depeauCouleur descheveuxCou
- Page 138 and 139:
Biométrie faciale douce 8Nous nous
- Page 140 and 141:
Biométrie faciale douce 103. Proba
- Page 142 and 143:
Biométrie faciale douce 12l’entr
- Page 144 and 145:
Biométrie faciale douce 14Figure 6
- Page 146 and 147:
Biométrie faciale douce 16pages 77
- Page 148 and 149:
Reviewers:Prof. Dr. Abdenour HADID,
- Page 150 and 151:
3hair, skin and clothes. The propos
- Page 152 and 153:
person in the red shirt”. Further
- Page 154 and 155:
7- Not requiring the individual’s
- Page 156 and 157:
9Probability of Collision10.90.80.7
- Page 158 and 159:
11the color FERET dataset [Fer11] w
- Page 160 and 161:
13Table 2: Table of Facial soft bio
- Page 162 and 163:
15Chapter 1PublicationsThe featured
- Page 164 and 165:
17Bibliography[ACPR10] D. Adjeroh,
- Page 166 and 167:
19[ZESH04] R. Zewail, A. Elsafi, M.