Semantic Annotation for Process Models: - Department of Computer ...
Semantic Annotation for Process Models: - Department of Computer ... Semantic Annotation for Process Models: - Department of Computer ...
142 CHAPTER 8. QUALITY EVALUATION OF THE METHOD semantic annotation approaches and tools. Maynard in [102] identified some requirements for ontology-based annotation tools such as expected functionality, interoperability, usability, accessibility, scalability and reusability. Usually, criteria and metrics for performance evaluation, such as precision, recall and F-measure [154], are defined for the evaluation of semi-automatic or automatic semantic annotations by using information extraction techniques. However, the evaluation is mainly for the semantic annotation of textual contents. The model features are certainly not concerned, but they are very important in our case of the semantic annotation of business process models. Moreover, those metrics are not sufficient for ontology-based information extraction, because the distinction between right and wrong is less obvious [102]. We do not apply any information extraction techniques in our system and the current prototype of the annotation tool mainly supports manual annotation. We have chosen SEQUAL that has been widely and successfully applied in the information modeling area, to evaluate the proposed approach. Also SEQUAL shares the same theoretical foundation with our semantic annotation framework. According to the semiotic quality categories, the quality analysis has been made at both the metamodel level (GPO and the PSAM specifications) and model level (the PSAM model instances and Pro-SEAT), which make up our semantic annotation method.
Chapter 9 Validation of Applicability In this chapter, we validate the applicability of annotation results derived from the proposed method. The validation is undertaken through an application scenario, where a new IS solution is modeled through selecting and reusing annotated process model fragments. The validation is supposed to check if the annotation method can facilitate process knowledge management, which is the general application objective of this work. Quality evaluation for Chapter 8 is further analyzed based on the discussion of the application results. 9.1 Validation Design A walkthrough scenario of a process knowledge management application is deployed in this chapter based on the semantic annotation results from the exemplar studies of Chapter 7. In the scenario, the annotated process models (PM A , PM B1 and PM B2 ) are IS solutions presenting logistics process knowledge of different organizations, and a new IS problem (an integrated sales delivery solution for their cooperative business) is supposed to be dealt with by reusing and integrating some of the solutions which can achieve goals of the new system. The annotation results which we got from the exemplar studies are the source data consumed by the application scenario. The annotation goals set in section 8.2.2 concretize a general application objective of our annotation method – to facilitate process knowledge management. The validation is therefore to measure the goals. Based on the goals, a set of requirements are derived. The requirements are implemented through a set of SWRL (Semantic Web Rule Language Combining OWL and RuleML) [202] rules and queries. The rules and queries are executed on the annotation results. The validation is thus to check how the returned query results fulfill those requirements in the context of the application scenario. In addition, semantic validity and semantic completeness of the annotation models are analyzed in the validation. The implementation tool of the application is Protégé-OWL with the SWRL editor and the Jess rule engine 1 . Accuracy of mapping between ontologies and models may affect the validation results. Therefore, we assume that the ontology-based annotation is accomplished by 1 http://herzberg.ca.sandia.gov/jess/ 143
- Page 112 and 113: 92 CHAPTER 5. GOAL ANNOTATION
- Page 114 and 115: 94 CHAPTER 6. PRO-SEAT (PROCESS SEM
- Page 116 and 117: 96 CHAPTER 6. PRO-SEAT (PROCESS SEM
- Page 118 and 119: 98 CHAPTER 6. PRO-SEAT (PROCESS SEM
- Page 120 and 121: 100 CHAPTER 6. PRO-SEAT (PROCESS SE
- Page 122 and 123: 102 CHAPTER 6. PRO-SEAT (PROCESS SE
- Page 124 and 125: 104 CHAPTER 7. EXEMPLAR STUDIES AND
- Page 126 and 127: 106 CHAPTER 7. EXEMPLAR STUDIES AND
- Page 128 and 129: 108 CHAPTER 7. EXEMPLAR STUDIES AND
- Page 130 and 131: 110 CHAPTER 7. EXEMPLAR STUDIES AND
- Page 132 and 133: 112 CHAPTER 7. EXEMPLAR STUDIES AND
- Page 134 and 135: 114 CHAPTER 7. EXEMPLAR STUDIES AND
- Page 136 and 137: 116 CHAPTER 7. EXEMPLAR STUDIES AND
- Page 138 and 139: 118 CHAPTER 7. EXEMPLAR STUDIES AND
- Page 140 and 141: 120 CHAPTER 7. EXEMPLAR STUDIES AND
- Page 142 and 143: 122 CHAPTER 7. EXEMPLAR STUDIES AND
- Page 144 and 145: 124 CHAPTER 7. EXEMPLAR STUDIES AND
- Page 146 and 147: 126 CHAPTER 7. EXEMPLAR STUDIES AND
- Page 149 and 150: Chapter 8 Quality Evaluation of the
- Page 151 and 152: 8.2. SETTING FOR THE QUALITY EVALUA
- Page 153 and 154: 8.2. SETTING FOR THE QUALITY EVALUA
- Page 155 and 156: 8.3. QUALITY ANALYSIS 135 to those
- Page 157 and 158: 8.3. QUALITY ANALYSIS 137 annotatio
- Page 159 and 160: 8.3. QUALITY ANALYSIS 139 • G1 -
- Page 161: 8.5. SUMMARY 141 3. Semantic annota
- Page 165 and 166: 9.1. VALIDATION DESIGN 145 - RE3.3
- Page 167 and 168: 9.2. SWRL FORMULATION 147 Table 9.1
- Page 169 and 170: 9.2. SWRL FORMULATION 149 RE3.2 RE3
- Page 171 and 172: 9.3. APPLICABILITY VALIDATION IN AN
- Page 173 and 174: 9.3. APPLICABILITY VALIDATION IN AN
- Page 175 and 176: 9.3. APPLICABILITY VALIDATION IN AN
- Page 177 and 178: 9.3. APPLICABILITY VALIDATION IN AN
- Page 179 and 180: 9.4. DISCUSSION ON RESULTS OF THE V
- Page 181 and 182: 9.4. DISCUSSION ON RESULTS OF THE V
- Page 183: Part IV Synopsis 163
- Page 186 and 187: 166 CHAPTER 10. CONCLUSIONS AND FUT
- Page 188 and 189: 168 CHAPTER 10. CONCLUSIONS AND FUT
- Page 190 and 191: 170 CHAPTER 10. CONCLUSIONS AND FUT
- Page 193 and 194: Appendix A BPMN This chapter presen
- Page 195 and 196: A.3. CONNECTING OBJECTS 175 Figure
- Page 197 and 198: A.6. BPMN META-MODEL TREE IN METIS
- Page 199 and 200: Appendix B EEML 2005 The language v
- Page 201 and 202: B.2. RECOURSES MODELING DOMAIN 181
- Page 203 and 204: Appendix C SCOR SCOR — Supply Cha
- Page 205 and 206: C.2. LEVEL 2 TOOLKIT 185 Level 1 Me
- Page 207 and 208: C.3. LEVEL 3 PROCESS ELEMENTS 187 F
- Page 209 and 210: Appendix D Algorithm for Semi-Autom
- Page 211 and 212: 191 Algorithm 1 Goal Annotation Alg
Chapter 9<br />
Validation <strong>of</strong> Applicability<br />
In this chapter, we validate the applicability <strong>of</strong> annotation results derived from the<br />
proposed method. The validation is undertaken through an application scenario, where<br />
a new IS solution is modeled through selecting and reusing annotated process model<br />
fragments. The validation is supposed to check if the annotation method can facilitate<br />
process knowledge management, which is the general application objective <strong>of</strong> this work.<br />
Quality evaluation <strong>for</strong> Chapter 8 is further analyzed based on the discussion <strong>of</strong> the<br />
application results.<br />
9.1 Validation Design<br />
A walkthrough scenario <strong>of</strong> a process knowledge management application is deployed<br />
in this chapter based on the semantic annotation results from the exemplar studies <strong>of</strong><br />
Chapter 7. In the scenario, the annotated process models (PM A , PM B1 and PM B2 )<br />
are IS solutions presenting logistics process knowledge <strong>of</strong> different organizations, and a<br />
new IS problem (an integrated sales delivery solution <strong>for</strong> their cooperative business) is<br />
supposed to be dealt with by reusing and integrating some <strong>of</strong> the solutions which can<br />
achieve goals <strong>of</strong> the new system.<br />
The annotation results which we got from the exemplar studies are the source data<br />
consumed by the application scenario. The annotation goals set in section 8.2.2 concretize<br />
a general application objective <strong>of</strong> our annotation method – to facilitate process<br />
knowledge management. The validation is there<strong>for</strong>e to measure the goals. Based on the<br />
goals, a set <strong>of</strong> requirements are derived. The requirements are implemented through<br />
a set <strong>of</strong> SWRL (<strong>Semantic</strong> Web Rule Language Combining OWL and RuleML) [202]<br />
rules and queries. The rules and queries are executed on the annotation results. The<br />
validation is thus to check how the returned query results fulfill those requirements in<br />
the context <strong>of</strong> the application scenario. In addition, semantic validity and semantic<br />
completeness <strong>of</strong> the annotation models are analyzed in the validation. The implementation<br />
tool <strong>of</strong> the application is Protégé-OWL with the SWRL editor and the Jess rule<br />
engine 1 .<br />
Accuracy <strong>of</strong> mapping between ontologies and models may affect the validation results.<br />
There<strong>for</strong>e, we assume that the ontology-based annotation is accomplished by<br />
1 http://herzberg.ca.sandia.gov/jess/<br />
143
Change language