Workshop - Data collection, management, sharing - DRIVE C2X

making cooperative systems cooperate

DRIVE C2X @ DITCM, Helmond, NL 

Data collection, management, sharing 

Logging & data collection: 

Security & privacy: 

Testing & validation: 

Moderator: 

Ilja Radusch, Fraunhofer FOKUS 

Thomas Benz, PTV 

Bart Netten, TNO 

Helena Gellerman, SAFER/Chalmers 

05.07.2012|2 

DRIVE C2X @ DITCM – Data collection, management, sharing 

project start date: 01.01.2011 | end date: 31.12.2013

Packet count analysis 

Measured on a single ITS Gateway 

05.07.2012|3 DRIVE C2X @ DITCM – Data collection, management, sharing 


Lessons learned from validation testing 

Flexibility of the architecture 

• Both participate and observe 

• Adapt quickly 

Standardisation of ITS G5 

• Hardware reuse 

• Multiple project can coexist 

Standard tests are crucial 

• Fast analysis and feedback to developers 

• Short development cycles 



DRIVE C2X @ DITCM 


Data Definition, Collection and 

(Pre-)Assessment 

July 05, 2012 

Ilja Radusch 

Fraunhofer FOKUS

Context 

Using 



What is Data? 

Measurands 

• Direct (raw) sensor values 

• Derived (preprocessed) measures 

• Derived from several sensors 

• Can be filtered or combined 

• Events 

• e.g. “crash” 

• Needs trigger criteria 

• Self-reported measures 

• Derived from e.g. questionnaires 

• Situational variables 

• e.g. weather condition 

• Require instrumentation in source 

code! 

Performance Indicators (PI) 

• Provide measurable parameters to 

prove or confute hypotheses 

• Each PI has denominator, e.g. “per 

time interval”, “per distance” 

• Hypothesis: “XYZ will decrease travel 

time.” 

• PI is travel time (per month) 

• PI are calculated from measured data 

• Size of PI data is significantly less 

then measure data 

05.07.2012|7 



Predicament of Data Collection 

• Optimum: Collect over the shortest possible time exactly those 

measurands necessary to calculate all performance indicators 

• Caveat 1: Definition of research questions and hypotheses and 

implementation of reference system work in parallel 

• Hence knowledge about necessary PI arrives “too late” for proper 

instrumentation in components 

• Vice versa: knowledge about components for proper definition of 

hypotheses only available after implementation 

05.07.2012|8 



Solution: 

Combined top-down, bottom-up approach 

• Top-down: Specification of performance indicators 

• Defining what measurands should be collected 

• Bottom-up: Developers instrument their components a-priori 

• Defining what measurands can be collected 

• Iterative checks that measurands that should be collected in fact can be 

collected 

• Dynamic log profiles filter out unnecessary measurands at run-time 

05.07.2012|9 



IVS Data 

Sources 

05.07.2012|10 



Integer Measurand 

Enum Measurand 

05.07.2012|11 



Data Collection – Software Logging 

• Defined measurands can be combined to log tupels (e.g.: the 

log tupel “position” is combined of “latitude” and 

“longitude”) 

• Custom API interface is generated for each log tupel to 

instrument components 

• E.g. log_newLogProfile(int value) 

• Strings are logged separately 

• All other values are transformed into integers for efficient 

processing later on 

05.07.2012|12 



Data Collecting System 

ITS Station 

Test Management Centre 

Test operator client 

Use 

cases 

OSGi 

Monitoring data 

proxy 

Live data 

Monitoring 

PoTi 

LDM 

Logging & monitoring data 

Testing Unit 

Logging & monitoring data 

ITEF Tool 

Measurand Spec 

Test Scenario 

Monitoring Spec 

Assessment Rules 

Test Site 

Measurand Storage 

Test Scenario 

Selection 

Quick Validation 

Pre-Assessment of 

Data Quality 

NWT 

VAPI 

Log station 

PI Database 

05.07.2012|13 



Quick Validation 

Automated (Pre-)Assessment of Data Quality 

Assessment Criteria 

(rules defined in XML) 

Upload via web interface 

Assessment 

Computation 

Compiled Report 

Calculation of 

Performance Indicators 

Upload via Java-API 

05.07.2012|14 



XML Syntax – Assessment Criteria 

 

Stations that delivered no data. 

 

 

 

 

 

Stations that delivered data. 

 

 

 

 

05.07.2012|15 



Context 

Using 

05.07.2012|16 





Data security and privacy 

July 05, 2012 

Thomas Benz 

PTV

Contents 

• Definitions 

• Information Flow 

• DRIVE C2X Context 

05.07.2012|18 



Definitions (1) 

• Privacy 

• Wikipedia states: 

• “Privacy is the ability of an individual or group to seclude 

themselves or information about themselves and thereby reveal 

themselves selectively.” 

• “Privacy is sometimes related to anonymity, the wish to remain 

unnoticed or unidentified in the public realm.” 

• Requirement: Reveal only such personal data that is required for 

analysis (e.g. age, gender) 

05.07.2012|19 



Definitions (2) 

• Security 

• Wikipedia again: 

• “Security is the degree of protection against danger, damage, 

loss, and crime.” 

• “Security as a form of protection are structures and processes 

that provide or improve security as a condition.” 

• Requirements: 

• Protect data against (internal and external) adversaries 

• Keep data “tamper-proof” 

05.07.2012|20 



Information Flow 

• General Requirement: 

• Provide privacy and data security over the information flow, e.g. 

Source: PRECIOSA Deliverable D1, „V2X Privacy Issue Analysis“, Project-Id: IST-224201, DG INFSO, FP7, 2008 

05.07.2012|21 



DRIVE C2X Context (1) 

• Controlled tests: 

• Test subjects informed and aware of test situation 

• “Informed consent” holds for shorter test periods 

• Confined test track => no need for “track hiding” 

• Number of test subjects allows for aggregation 

05.07.2012|22 



DRIVE C2X Context (2) 

• Naturalistic tests: 

• Test subjects informed but sometimes maybe unaware of test 

situation 

• “Informed consent” may not be enough 

• Real-world road networks, real-life driving tracks => time and 

position potentially privacy critical, privacy measures required: 

• Partial tracks (cut-off start and ending) 

• Link to personal data restricted (only partial personal data like 

age) 

• Dilute exact timing (e.g. only classes of start time, only weekday 

instead of exact date) 

05.07.2012|23 





Testing & Validation @ DITCM 

Date 05.07.2012 

Bart Netten 

TNO

System Validation @ DITCM 

• Role of system validation 

• System Test Site – DITCM 

• First results of system testing and validation 

• Observed phenomena 

• First experiences 

05.07.2012|25 



DRIVE-C2X Process 

Development 

System 

Testing 

Field Testing 

Evaluation 

PoTi 

CAM 

HMI 

SVW 

OW 

AEV 

VIS VIS 

VIS 

VIS 

RIS 

TMC 

Integration 

System Test 

& Validation 

System 

Test 

Site 

(STS) 

D 

S 

F 

E 

I 

Fin 

Validation 

D 

S 

F 

E 

I 

Fin 

Analysis 

Evaluation 

05.07.2012|26 



Objectives for Validation @ STS 

• Validate that systems are ready for roll-out to National Test Sites for FOT 

• Validate DRIVE-C2X reference system in vehicle and road side 

stations 

and TS or partner specific systems 

• Validate monitoring and 

• Motto “Validate once, use everywhere” 

• Testing, Verification & Validation: 

• Technical testing of integrated systems 

• Performance of positioning, communication, … 

• Functional testing of applications 

• Interoperability testing 

05.07.2012|27 



Evaluation of Field Testing 

• DRIVE-C2X tools to support testing & evaluation: 

• Monitoring tools for visual checks during testing 

• Logging on ITS stations for post-analysis and evaluation 

• Processing of log data to derive performance indicators 

Assumption: Log data is correct 

• Need objective external measurement system (ground truth) to also 

validate log data 

05.07.2012|28 



29 

DITCM Architecture & Drive C2X extensions 

Traffic Control 

Center 

Roadside ITS 

Roadside ITS 

station Roadside ITS 

station 

station 

Central ITS 

Central ITS 

Station 

Station 

Roadside ITS 

Roadside ITS 

station 

station 

51 cameras 16 ITS Gateway 

2 traffic light 

controllers 

05.07.2012|29 



STS Validation process 

DRIVE C2X 

DITCM 

Live during test runs 

• Visual validation of HMI events during 

test runs 

• Evaluate DRIVE-C2X scenario control 

and monitoring tools 

• Monitoring technical performance 

• Track DRIVE-C2X & normal 

vehicles 

• Analyse communication 

Post analyses 

• Decode and verify log files per 

component and per ITS station 

• Verify and validate events and 

interaction of ITS stations 

• Derive performance indicators per 

function 

• Verify and validate log data 

• Align DRIVE-C2X log data 

• Location of vehicles & events 

• Timing of HMI & events 

05.07.2012|30 



Monitoring 

05.07.2012|31 



Time offsets and communication delays 

ITS Stations received at road side 

Delay time [msec] 

Road side reception time - CAM generation time 

Vehicle stations 

05.07.2012|32 



Effect of processing load(?) on logging 

ITS station 

Logging time 

Generation time of CAM 

05.07.2012|33 



Effects of timing on function evaluation 

DENM for Approaching Emergency Vehicle 

Color indicates status of sending / receiving DENMs 


Log time stamp 

05.07.2012|34 



Effects of timing on function evaluation 

DENM for Approaching Emergency Vehicle 

Color indicates status of sending / receiving DENMs 


Log time stamp – corrected to road side measured offsets 

05.07.2012|35 



Validation of functions 

In-Vehicle Signage – speed limits 

Driving direction 

05.07.2012|36 



First experiences 

• Comparison with external measurements identify technical flaws in the 

implementation of components and logging 

• System performance and uncertainties need to be incorporated in the 

evaluations 

• Consequences of any remaining flaws on functions and performance, and 

their impact on evaluation, need to be fully assessed 

05.07.2012|37

Workshop - Data collection, management, sharing - DRIVE C2X

Create successful ePaper yourself

Delete template?

Save as template?