Geodatabase Versioning, Replication and History - Northeast Arc ...

Geodatabase Versioning, Replication 

and Archiving (History) 

NEARC 2007 ESRI Technical Session 

ESRI , Boston 

2007 ESRI NEARC 1

Brief Overview 

• Geodatabases 

– Types of geodatabases 

– What is a geodatabase 

• Editing environments 

2007 ESRI NEARC 2

Defining the geodatabase (GDB) 

• Container of spatial & attribute data 

– Collection of geographic datasets 

• Native data structure for ArcGIS 

• Provides the ability to: 

– Leverage data relationships 

– Enforce data integrity 

– Create intelligent features 

Vectors 

Networks 

Surveys 

Addresses 

Annotation 

3D Objects 

Attribute 

Topology 

Dimensions 

Geodatabase 

Terrain 

Cadastral 

Cartography 

Images 

CAD 

2007 ESRI NEARC 3

Geodatabase options at ArcGIS 9.2 

• MS Access-based personal GDB 

• File GDB 

• ArcSDE GDB - 3 editions: 

– Personal, Workgroup: SQL Server Express 

– Enterprise: DB2, Informix, 

Oracle, SQL Server, PostgreSQL (9.3) 

ArcGIS Server 

ArcGIS Desktop 

Personal 

(Access) 

File 

File 

Personal 

ArcSDE 

Workgroup 


Enterprise 

Enterprise 


Capacity 

Number of users 

2007 ESRI NEARC 4

Enterprise ArcSDE Geodatabase 

Required software environment 

ArcObjects 


DBMS 



Operating system 

2007 ESRI NEARC 5 

A

Overview 



– Non versioned Editing 

– Versioned Editing 

– Move to base Editing 

2007 ESRI NEARC 6

Editing options 

• Non-versioned editing (9.2) 

– Edit the base table directly 

– Supports short transaction workflows 

– Edits immediately available upon save 

• Versioned editing 

– Available since 8.0 

– Supports long transaction workflows 

– Lineage of change maintained by SDE tables 

• Move to base (9.2) 

– If non Default version, preserves any edits to a feature class 

– Upon save or post to DEFAULT, edits are moved to the base table 

2007 ESRI NEARC 7

Overview 






2007 ESRI NEARC 8

Introducing Non–Versioned Editing 

New@9.2 

• Preserves only last set of edits made to a Feature Class 

• Suggested for Non-ESRI client interaction 

• Uses the underlying database transaction model 

– Edits immediately available upon save 

• Simple data only 

– Points, lines, polygons, annotation, relationship classes 

– NO Topology, Geometric Networks…etc 

etc 

• No Archiving, Geodatabase Replication 

• No Conflict Detection 

2007 ESRI NEARC 9

Editing in a non-versioned environment 

• Versioning tab - Edit with no undo/redo ability 

• Start Edit Session 

• Exclusive database lock when updating feature 

– Lock held until edit session is saved or rolled back 

• Hourglass 

lingers; does not time out 

• Save changes - commit in database 

• Stop editing with outstanding - rollback changes 

Blocked 

2007 ESRI NEARC 10

No undo/redo 

• Edits between save operations are a single transaction 

– Committed (upon save) 

– Or rolled back (upon stop editing without saving) 

Undo/redo not available during 

edit session 

Stop editing – commit or rollback 

Three edits in 

current edit session 

2007 ESRI NEARC 11

Workflow considerations 

• Concurrency issues must be identified 

– Design workflow to attempt to minimize blocking 

• To spatially distribute your editors 

– Blocking related to DBMS isolation levels 

– Insert hyperlink to concurrency & locking 

• No conflicts 

– Minimize lost updates by designing workflow 

• Integrating non-ESRI applications 

– Edits saved immediately to base tables 

– Allows for easy integration with non-ESRI applications 

– Can use DBMS behavior to enforce integrity 

• Compress not necessary 

Editing 

workflow 

Business 

workflow 

2007 ESRI NEARC 12

IT Integration 

• Easy to implement 

• No A & D tables 

– Edits immediately saved to base tables 

• DBMS behavior is easy to implement 

• Easy IT integration 

– Non-ESRI applications see edits in base tables 

3 rd party 

applications 

Only see the base table: 

Do not understand contents 

of A & D tables 

Base table 

ArcGIS editing 

applications 

Base & delta tables: 

Can read and edit 

versioned data 

2007 ESRI NEARC 13

Overview 





• What is versioning 

• Repository and Feature Class tables 

• Workflow considerations 

• Reconcile and conflict resolution 

• Post 

• Compress 


2007 ESRI NEARC 14

Introducing: Versioned Editing 

• Intended for Multi-Editor Environment 

• Long transactions are enabled by 

– Repository tables, Application locks, Application logic 

– Edits stored in delta tables 

• Simple & Complex data 


– Topology, Geometric Networks, etc. 

• Facilitates Archiving & Geodatabase Replication 

• Conflict Detection 

2007 ESRI NEARC 15

What versioning offers 

• Long transactions 

– Isolate work across multiple sessions 

• Save, reconnect, continue edits 

• E.g., Edits do not impact others 

• Model what-if 

scenarios 

– Simulate situations with versions 

• E.g., Disaster event planning 

• Workflow management 

– Create versions for stages 

• History 

• E.g., Land development 

– Save snapshot of database state 

• E.g., Wildlife habitat monitoring 

Public 

Analyst 

Public 

Farmland 

Flood 

Residential 

Year 1 Year 2 

A 

2007 ESRI NEARC 16

What is Versioning 

• Storing edits without affecting the original 

• An alternative view of the database that has: 

– an owner 

– a description 

– a permission 

– a parent version 

• Versions are not affected by 

changes occurring 

in other versions of the database 

• Version: 

– A window into a database 

– A view into a database 

– A snapshot of a geodatabase 

Default 

QA 

2007 ESRI NEARC 17

Feature class tables in the database 




DBMS 


A

Versioning system tables 

• ArcSDE repository maintains geodatabase properties 

Defines different 

versions 

Lineage of 

state_id’s 

States at which 

tables are edited 

Properties of 

individual states 

Tables tracked by 

registration_id 

Compress 

information 

2007 ESRI NEARC 19

Versioned Editing Terminology 

• State 

– Unit of work 

– A NUMBER per edit 

– Made up of multiple transactions 

on the DB level 

• Lineage 

– Set of states belonging to a version 

• State tree 

– Logical map of states 

• Version 

• Visualization of a Lineage 

• State_Lineages Table 

– Textual pointer to a state 

– Represents entire lineage of edits 

VERSION_NAME 

State (base) 

0 

State 

1 

State 

2 

LINEAGE 


A

Editing example: User versioning tables 

Riverside W 

# 

# 

Deletes table 

u Flags rows as deleted 

Adds table 

u Stores new rows 

Business table 

Riverside NE 

# 

Riverside SE 

# 

# 

# 

u Stores base data 

u State 0 is implied 

D77 

SDE_STATE_ID SDE_DELETES_ROW_ID DELETED_AT 

A77 

SHAPE NAME 

CELL_TOWERS 

Unique object ID 

u Constant for all 

versions of a row 

OBJECTID SDE_STATE_ID 

Database state 

u Identifies when 

row was added 

SHAPE NAME OBJECTID SDE_STATE_ID 

1 Riverside NE 1 0 

2 Riverside SE 2 0 

3 Riverside W 3 0 

2007 ESRI NEARC 21

Editing example: INSERT 

D77 


Riverside W 

# # 

# 

Riverside NE 

# 

# 

Riverside SE 

# 

# 

# 

• Add point 

• Add row to A table 

NAME is NULL 

• State ID is 

incremented 

• Edits always an 

INSERT or DELETE 

A77 


4 4 1 

CELL_TOWERS 





2007 ESRI NEARC 22

Editing example: DELETE 

Riverside W 

# # 

# 

Riverside NE 

# 

# 

# 

• Delete point 

• Add row to D table 

• State ID is 

incremented 

D77 


0 2 2 

A77 

Created at 

state 

Deleted at 

state 


4 4 1 

CELL_TOWERS 





2007 ESRI NEARC 23

Editing example: UPDATE 

Riverside W 

# 

Riverside Mid 

# 

# 

Riverside NE 

# 

# 

# 

• Add attribute 

• Both edits occur 

at same new state 

•Add row to A table 

•Add row to D table 

• OBJECTID is same 

• SHAPE is same 

D77 


0 2 2 

1 4 3 

A77 


4 4 1 

4 Riverside Mid 4 3 

CELL_TOWERS 





2007 ESRI NEARC 24

Editing example: Save 

Riverside W 

# 

# 

Riverside Mid 

# 

Riverside NE 

# 

# 

# 

D77 


0 2 3 

1 4 3 

• Save edits 

• Unneeded rows 

removed from 

delta tables 

• States renumbered 

to most recent ID 

•DELETED_AT 

updated to 3 

A77 


4 4 1 

4 Riverside Mid 4 3 

CELL_TOWERS 





2007 ESRI NEARC 25


• Edit DEFAULT 

– Simplest to manage 

– Deal with conflicts immediately 

• Two – tier version tree 

– DEFAULT represents master 

– Deal with conflicts upon reconcile 

• Two – tier version tree – Variation 

– Protect DEFAULT with a proxy 

– Deal with conflicts upon reconcile 

DEFAULT 

DEFAULT 

DEFAULT 

Represents a version 


A

Example: business cycle workflow 

DEFAULT 

History 

Version 

Daily Edits 

Long term project 

Represents a version 


A

Merging changes – reconcile & post 

• ArcMap includes 

methods for merging 

versions 

– Reconciles with and 

posts changes to target 

version 

• Depends on workflow 

– Tools available to 

resolve conflicts 

Edit (child) version 

Edit (child) version 

Reconcile pulls changes 

from target into edit version 

Target (parent) version 

Target (parent) version 

Post pushes changes 

from edit into target version 


A

Versioning – reconcile procedure 

• Functionality in ArcSDE geodatabases 

• Incorporates edits from parent (target) version to 

child (edit) version 

– Can reconcile against any ancestor version 

– Performed in an edit session 

– Can undo changes 

– Must be done before a post operation 

• Detects differences & discovers conflicts 

• Must be the only user currently 

editing version to execute 

Plan1 

Child 

Current 

Parent 

A 

2007 ESRI NEARC 29

Conflict resolution dialog 

• Provides ability to visualize & manage conflicts from both the 

target & edit versions 

– Interactive dynamic display of each version 

– Common ancestor state is also viewable 

2007 ESRI NEARC 30

Post 

• Synchronizes current edit session with target version 

– Modifications saved to target; two versions now identical 

– Must be done in an edit session 

– Performed after reconciling with another version 

– Cannot be undone 

• Applying changes to target version 

– Not necessary to post; optional operation 

• Depends on workflow 

Current 

Parent 

Plan1 

Child 

A 

2007 ESRI NEARC 31

How often should you compress 

• Performs many functions 

– Move records from delta tables to Base 

• Depends on the amount of editing activity 

– Not unreasonable to compress every night 

• Executed as the “SDE” user 

• compress_log provides information on compress 

• ArcGIS clients can remain connected at 9x 

– ArcMap users can continue editing 

2007 ESRI NEARC 32

Overview 






2007 ESRI NEARC 33

Move to Base Editing 

9.2 

• New Functionality at ArcGIS 9.2 

• Versioned behavior if Feature Class in non-DEFAULT 

version 

• Suggested for Non-ESRI client interaction 

• Simple data only 


• No Archiving, Geodatabase Replication 

• No conflict detection 

• Supports undo/redo 

• Better IT integration with versioned editing 

2007 ESRI NEARC 34

Editing DEFAULT version 

• Start editing DEFAULT version 

• Edits stored in A and D tables 

– Each edit receives state ID 

– Enables undo/redo capability 

ZIP_CODES 

Base table 

(state 0) 

No edits have occurred 

(state 0) 

A40 

During edit session 

(edits in A- and D-tables) 

D40 

2007 ESRI NEARC 35

Move to Base Editing 

• DEFAULT version references new state 

– Last state of edit session 

• Edits referenced by DEFAULT moved to base table 

– Edits now at state 0 

ZIP_CODES 

OID 2—deleted 

Save edit session 

(edits now in base tables) 

A40 

D40 

OID 11—new feature, no attribute 

OID 5—updated feature geometry 

OID 12—new feature, new attribute 

2007 ESRI NEARC 36


• IT integration 

– Use DBMS behavior to enforce integrity 

– Use geodatabase behavior to aid in ArcGIS editing 

• Versioning behavior on non DEFAULT versions 

• No conflict detection on DEFAULT 

– Option: Use surrogate for conflicts before posting to DEFAULT 

• Compress necessary 

2007 ESRI NEARC 37

Summary: ArcSDE geodatabase editing options 

Non-versioned 

editing 

Versioned 

editing with 

move to base 

Versioned 

editing 

Editable 

data types 

Simple 

feature classes 

Simple 

feature classes 

All data types 

Supported 

workflows 

Simple 

Simple & advanced 

with versions 

Archiving /replication 

not supported 

Simple & advanced 

with versions 

Archiving & replication 

supported 

DBMS 

transaction “type” 

Short 

(single edit session) 

Long 

(multiple edit session) 

Long 

(multiple edit session) 

Supports 

undo/redo 

No 

Yes 

Yes 

Feature class 

tables involved 

Base table 

Base & possibly 

delta tables 

Delta tables 

Supports 

archiving & replication 

No 

No 

Yes 

Supports DBMS 

data integrity 

features 

Yes 

Yes - when editing 

DEFAULT version; 

after save 

No 

No – when editing 

other versions 

2007 ESRI NEARC 38

Archiving 

2007 ESRI NEARC 39

Understanding change in the geodatabase 

• Ability to maintain information 

on how objects exist 

throughout time 

– Only required if demanded 

by your workflow 

– Example: Analyze how a specific 

feature has changed over 

given period of time 

• Answer queries about 

how the data has 

changed over time 

2007 ESRI NEARC 40

Common historical queries 

• What did the feature look like at time y 

– Features change over time; can time be 

moved back to a certain point 

• How has the feature changed over time 

– Is maintaining each change important 

• What was located here at time y 

– Compare location at different times to 

examine the change over time 

RO 

RO 

R-1- 

65 

R-1-65 

R-1-65 

R-1-65 

Jan 1, 2001 

Jan 1, 2001 August 31, 2004 

C-2 

R-1-65 

R-1-65 

What was 

here on 

Jan 1, 2001 

T I M E 

2007 ESRI NEARC 41

Pre 9.2: History through versions 

• Ability to capture history was based 

on creating transactional versions 

• Benefits 

– Point in time 

– No data duplication 

• Limitations 

– Versions pin state tree 

• Will prevent full compress 

• Fewer versions is better 

• Remove when no longer needed 

– Can not unregister as versioned and 

maintain history 

2007 ESRI NEARC 42

9.2: Geodatabase archiving 

• Preserve change to data 

• Persistent history 

RO 

R-1-65 

January 1 st , ‘01 

– Change maintained 

separately in archive class 

R-1-65 

• Archive class can persist if object 

unregistered as versioned 

• Requirements vary 

– Which object needs archiving 

RO 

July 16 th , ‘02 

– What granularity of change 

is needed 

R-1-65 

C-2 

August 31 st , ‘04 

R-1-65 

2007 ESRI NEARC 43

Archiving overview 

• Select classes to archive 

– Supports full geodatabase 

data model 

• Register as versioned 

– Do not use move to base 

• Enable archiving 

– Archive class is created 

• Edit data 

– Edits saved or posted to 

DEFAULT will be archived 

2007 ESRI NEARC 44

Archive class 

• New feature class created 

– Built when archiving enabled 

– Not visible in ArcCatalog 

• Business table 

– Same schema as enabled class 

• Any schema changes on enabled 

class will affect archive class 

– Includes attributes for timestamp 

• Archive class is not registered as versioned 

• All rows part of DEFAULT lineage copied to archive class 

– All archived with same timestamp 

PARCELS 

PARCELS_H 

2007 ESRI NEARC 45

Archive class attributes 

• GDB_FROM_DATE 

– Time the feature was added to the original class 

• Action: New or updated feature 

– All features from DEFAULT are archived upon activation 

• GDB_TO_DATE 

– Timestamp when the feature was retired from the original class 

• Action: Deleted or updated feature 

– All current features maintain timestamp of Dec 31, 9999 

• GDB_ARCHIVE_OID 

– Maintains unique ID for each record 

2007 ESRI NEARC 46

Archive time 

• ArcGIS represents time as the transaction time 

– Transaction time: System time the edit was saved 

– Valid time: Real-world time that the event occurred 

– Example: A parcel was sold on April 6, 2006; due to backlog, the new 

owner was not entered until April 30, 2006 

• Valid time and transaction time are rarely the same 

– Option: Add user maintained column to enabled class to maintain valid 

time 

• Feature archived in the order of edit operations 

– Transaction time may be out of order from real world event 

– Example: Work orders saved out of sequence are archived out of 

sequence 

2007 ESRI NEARC 47

Editing workflow: Save every edit 

• Scenario: Every edit operation must be archived 

• Need to update the archive class on every edit operation 

– Each edit preserved with timestamp of save operation 

– Used when you need to preserve every edit 

• Example: Change ownership of a parcel several times—need to preserve each 

individual edit to the parcel 

• Version options 

– Edit the DEFAULT version 

• Save every edit 

2007 ESRI NEARC 48

Example 1: Add new feature 

• Editing the DEFAULT version 

• Create new feature 

• Save edit 

– New feature added 

to archive class 

– Timestamp of FROM date 

is time of save 

– Timestamp of TO date 

is Dec 31, 9999 

5808 Jurupa Ave 

*No Site Address* 


6024 William St 


Start editing PARCELS 







6015 Anita Pl 

Create new feature 

Save in the DEFAULT 

version 

PARCELS_H 


6015 Anita Pl 

ObjectID 71565 was added 

A 

2007 ESRI NEARC 49

Example 2: Update feature 

• Update feature and save 

• Feature retired 

– Existed between FROM and TO date 

• New feature added 

– New row 

• New GDB_ARCHIVE_OID 

– Same OBJECTID 


Editing PARCELS 






6015 Anita Pl 






6027 Sheppard St 

Update attribute 

Save in the DEFAULT 

version 

PARCELS_H 


6015 Anita Pl 

Update added to archive 

with new GDB_ARCHIVE_OID 

A 

2007 ESRI NEARC 50

Example 3: Delete feature 

• Delete feature and save 

• Feature retired 

– Existed between FROM and TO time 

– View archive class after 

TO time—will not 

see feature 

Editing PARCELS 









6015 Anita Pl 





Save edit to the 

DEFAULT version 

PARCELS_H 


6015 Anita Pl 

Feature was retired 

at GDB_TO_DATE 

A 

2007 ESRI NEARC 51

Editing workflow: Save group of edits 

• Scenario: Groups of edit operations can be archived 

– Archive class records changes saved/posted with one timestamp 

– Individual feature modification not preserved—final representation 

preserved 

– Example: Census tracks may change every year 

• Each change not archived; only archive once completed 

• Version options 

– Edit the DEFAULT version 

• Save after logical group of edits 

– Edit child version 

• Reconcile and post after logical group is completed 

2007 ESRI NEARC 52

Example 4: Make multiple edits and save 




6015 Anita Pl 


1) No parcel 

2) Add parcel 

• Make several edits to same feature 

• Save to DEFAULT 

– Save trims intermediate states 

– Preserve the last representation of feature in the 

archive class 


6015 Anita Pl 


3) Update attribute 



6015 Anita Pl 




6015 Anita Pl 




4) Split feature 

5) Update attribute 

Archive class preserves 

the split and new 

attribute—no 

intermediate steps are 

archived 




6007 Anita Pl 

6015 Anita Pl 

6007 Anita Pl 

6015 Anita Pl 

Save edits 

A 

2007 ESRI NEARC 53

Historical version 

• Read only view of geodatabase 

– Transactional versions 

are used for editing 

• References particular moment 

• Historical marker 

– Created by user 

– References specific time 

– Like a bookmark 

• Specific date and time 

– Pick date and time from calendar 

2007 ESRI NEARC 54

Answering common historical questions 

• What did the feature class look like at time y 

– Connect to historical version 

– Use calendar or historical marker to examine data at a specific point-in-time 

Historical marker: Original 

Historical marker: T10 

2007 ESRI NEARC 55

Answering common historical questions 

• How has the feature class 

changed over time 

• Add the archive class 

– Identify archive class 

– Select feature 

• Investigate all the representations 

of that feature 

– Query definition 

• Time range to view features 


Time 





A 

2007 ESRI NEARC 56

Disable archiving 

• If required, you can disable archiving 

– Necessary if unregistering feature class as versioned 

• Option: Preserve or delete the archive class 

– Preserve as a standalone feature class 

• Re-enable enable archiving a new archive class is created 

• Does not preserve link to old archive class 

– Delete the archive class 

Preserve 

Delete 

2007 ESRI NEARC 57

Workflow benefits 

• Archived features stored without impacting active features 

• Ability to enable on all geodatabase vector data 

• Offers ability to archive only objects of interest 

– May not be required for some feature classes 

• Standard tools used for feature historical queries 

– Identify 

– Select 

– Query definition 

2007 ESRI NEARC 58

Benefits 

• Using DEFAULT historical marker may provide better performance 

– Does not access versioning repository tables or delta tables 

• Opportunity to unversion the data and/or disable archiving 

– Archive class no longer linked to original feature class 

– Preserve archive class as long as needed 

ZONING 

C-2 

RO 

ZONING_H 

R-1-65 

R-1-65 

R-1-65 

2007 ESRI NEARC 59


• ArcGIS represents time as the transaction time 

– Valid time and transaction time are rarely the same 

– Pay attention to order of work performed 

• Transaction time can become out of sync with event sequence 

– Option: Add user maintained date column to feature class 

• Archive from DEFAULT version 

• Must determine granularity 

– Archive every transaction: Save every edit to DEFAULT 

– Archive logical transactions: Determine when to save/post 

to DEFAULT 

2007 ESRI NEARC 60

Distributed Geodatabase 

(Geodatabase Replication) 

2007 ESRI NEARC 61

Geodatabase Replication 

• Allows you to distribute copies of data across 2 or more 

geodatabases 

• You can edit the databases independently and 

synchronize them as needed 

• Released at 9.2 - Builds upon disconnected editing from 

earlier releases (8.3) 

2007 ESRI NEARC 62

Geodatabase Replication – Use Cases 

• Mobile Users and Field Crews who need to be disconnected 

from the network 

• Users who need to maintain copies of data at different 

organizational levels (city, county, state) 

• Users who want to maintain copies of data at different 

geographic facilities 

• Users who need to distribute work to contractors 

• Production and publication geodatabases 

2007 ESRI NEARC 63

Distributed Geodatabase Use Cases 

Regional Offices 

Mobile Users 

Multiple levels 

Production / Publication 

2007 ESRI NEARC 64

Geodatabase Replication - LAN and WAN 

• LAN - Use connections to your local geodatabases 

• WAN - Use ArcGIS Server and geodata web services to access 

remote geodatabases 

• All geodatabase replication workflows are supported in both 

environments 

2007 ESRI NEARC 65

Geodatabase Replication Workflows 

• Workflows can involve Multi- 

User geodatabases and single 

user geodatabases 

Multi-user user to Multi-user 

user 

– Multi-user user geodatabase – 

Multi-user user ArcSDE 

geodatabase accessed locally 

or remotely through ArcGIS 

server 

Enterprise 

Enterprise 

– Single user geodatabase – 

Personal ArcSDE, file 

geodatabase or personal 

geodatabase on a local 

machine 

Multi-user user to Single User 

Enterprise 

Personal Geodatabase, 

File Geodatabase or 

Peronal ArcSDE 

2007 ESRI NEARC 66

Distributing Data 

• Geodatabase Replication can be 

used in conjunction with other 

data distribution techniques 

• Example: 

• Use geodatabase replication to 

synchronize changes between 

offices 

• Use mobile ADF for field workers 

with light weight mobile devices 

• Use geodatabase replication for 

field workers who need ArcGIS 

desktop or engine in the field 

0DLQ2IILFH 

5HSOLFDWLRQ 

)LHOG2IILFH 

: HE 

6HUYLFHV 

0RELOH$' ) 5HSOLFDWLRQ 

0RELOH$' ) 

2007 ESRI NEARC 67

Database Replication vs. Distributed Geodatabase 

ArcSDE technology 



Data managment 

RDBMS 


Replication 

Purpose: Data Distribution 

1-way, 2-way 

all feature classes 

some feature classes 

portions of feature classes 

history 

RDBMS/Storage 

Replication 

Purpose: HA (High 

Availability), Data 

Distribution 

full Geodatabase 

OS/Storage 

2007 ESRI NEARC 68

DBMS Replication with Geodatabases 

• Geodatabase replication does not use DBMS replication 

• Requirements and limitations of DBMS replication 

– Requires knowledge of how the geodatabase\ArcSDE system tables 

work 

– No tools provided in ArcGIS to support it 

– Limited support for cross DBMS replication 

– Does not support or has limited support for complex geodatabase data 

types and limited filters to define the data to replicate 

• Advantages of DBMS replication 

– Can work with non-versioned data 

– Can replicate entire database 

– Can be configured to provide synchronous replication 

2007 ESRI NEARC 69

Geodatabase Replication Types 

Check out / 

Check in 

Parent 

Once Only 

Child 

Two Way 

Parent 

Multiple Times 

Child 

One Way 

Parent 

Multiple Times 

Child 

2007 ESRI NEARC 70

Check out / Check in Replication 

• Disconnected editing - ArcGIS 8.3 to 9.1 

• Child replica can be hosted in a Personal Geodatabase, File 

Geodatabase or ArcSDE Geodatabase (only ArcSDE can host the 

Parent) 

Check In 

Personal GDB 

Enterprise 


Check-out 

Check In 

Personal SDE 

Check-out 

Check In 

Check -out 

File GDB 

2007 ESRI NEARC 71

One Way Replication 

• Child replica is considered read-only 

• No system versions on the child replica 

• Choose between 2 model types: 

– Full – Supports complex types (Geometric Networks and Topologies) and 

requires the child replicas data to be versioned 

– Simple – Child replica’s data is simple and does not need to be versioned 

Web users 

Enterprise 


ArcIMS 

ArcGIS 

Server 

Read-only 


2007 ESRI NEARC 72

Two Way Replication 

• Requires ArcSDE geodatabases and versioned data 

• Can use 2 way replication with personal ArcSDE instead of check 

out/check in replication 

Enterprise 


Enterprise 


Personal SDE 

Personal SDE 

2007 ESRI NEARC 73

Geodatabase Replication - Data Requirements 

• You must have write access to the data 

• All data must be registered as versioned without the option to move m 

edits to base 

• Two way and One way replicas 

– Each dataset must have a Globalid column 

– All spatial data must be stored in a high precision spatial reference rence 

2007 ESRI NEARC 74

Replica Creation – What Data Types can be 

Replicated 

• In general, if it supports the full versioning data model it can be 

replicated 

• Data Types 

– Fully supported – Simple Feature Classes and Tables, Geometric 

Networks, Topologies, Representations, Relationship Classes and 

Annotation 

– Source data can be replicated -Terrains and Network datasets 

– Copied during creation – Raster datasets and Raster catalogs 

– Not Supported - Survey datasets, Schematics datasets, Locators can 

not be replicated 

2007 ESRI NEARC 75

Maintaining Object Identity 

• GlobalID columns 

– Based on GUIDs / UUID technology 

– Introduced in ArcGIS 9.0 

{9DFACA0A-982F-4175-80E7-B553378D9E6D} 

– GlobalIDs are system maintained (like ObjectIDs) 

– Add Globalid command in ArcCatalog (9.2) adds the columns at 

the feature dataset and standalone feature class/table level 

• Can be added to versioned data 

• File Geodatabase and Personal Geodatabase require schema only 

– Sample provided to delete a globlalId column 

– Differs from columns of type GUID 

2007 ESRI NEARC 76

Replica Creation - Versioning 

• Edits made to the replica versions 

are synchronized 

• Like extending the version tree to 

span multiple 

databases 

• 2 way and 1 way replicas 

– Parent replica version can be a named 

version or default 

– Child replica version is always default 

• Check-out replicas 

– Parent and child replica versions are 

named versions 

ArcSDE GDB 

Default 

RV 

Parent 

replica 

version 

Default 

Child 

replica 

version 

V2 

ArcSDE GDB 

2007 ESRI NEARC 77

Replica Creation – Defining data to replicate 

• Filters and Relationship 

classes are used to define 

the data to replicate 

• Filters are applied first 

– Spatial – A geometry used 

to define the area to 

replicate 

– Selections – Selection sets 

on feature classes and 

tables 

– QueryDef – Definition 

queries applied to individual 

feature classes and tables 

• Additional rows are then 

added if they are related to 

the rows in the filter 

– Relationship classes are 

applied in a single direction 

and in an optimal order 

More options available when creating replica in 

ArcMap 

2007 ESRI NEARC 78

Replica Synchronization 

• Applying data changes across replicas 

• Replicas can be Synchronized in either both 

directions or just a single direction 

• Synchronization is based on exchanging messages 

and is fault tolerant 

• Uses versioning 

2007 ESRI NEARC 79

Replica Synchronization - Message Exchange 

• Synchronizations are performed using message exchange 

• Messages types: 

– Data Change messages 

• Includes the data changes to synchronize 

• By default all changes since the last acknowledgement are included 

– Acknowledgement messages 

• Acknowledges that previously sent changes have been received by the 

relative replica 

2007 ESRI NEARC 80

Connected Synchronization 

• Connected Synchronization 

– Can be performed when all replicas accessible on the network (LAN or WAN) 

– Always connected or intermittently connected 

– System manages message exchange and always acknowledges 

– Can send changes in one direction or both directions in one operation 

– Example: to synchronize, use the synchronize wizard in ArcCatalog 

Synchronize 

2007 ESRI NEARC 81

Disconnected Synchronization 

• Disconnected Synchronization 

– Performed when replicas are not on the same network 

– Message exchange is performed by the end user 

– Operations are performed by export, file transfer and import 

• Example: to synchronize, export changes to a delta XML file, transfer 

the file (ftp, CD through the mail, etc.), have the file imported d on the 

relative replica when it arrives 

Transfer 

2007 ESRI NEARC 82


• At any time a replica can be either a Data Sender or a Data 

Receiver 

– Data Sender sends Data Change messages to the Data Receiver 

– Data Receiver accepts Data Change messages from the Data Sender 

and sends Acknowledgement messages back to the Data Sender 

• Switch Roles 

– Data Sender sends a final Data Change message that includes 

instructions to switch roles 

2007 ESRI NEARC 83


• Acknowledgement messages 

– Not required, but recommended after each receive 

– If acknowledgement messages are not received by the Data Sender: 

• Previous changes are re-sent by default 

• Data Senders geodatabase can become large due to accumulated changes 

that are saved for error recovery 

• Switching roles and then sending data changes is another way to 

acknowledged messages 

2007 ESRI NEARC 84

2007 ESRI NEARC 85

Replica Synchronization Examples 

• Example 1 – Connected Synchronization with a 2 way replica 

Connected 

Synchronization 

• Example 2 – Disconnected Synchronization with a 2 way replica 

Sender 

Receiver 

Transfer 

A 

Transfer 

A 

2007 ESRI NEARC 86

Replica Reconcile Policies 

• Three policies: 

– Favor the database – if there is a conflict, the value currently in the 

replica database is applied 

– Favor the imported changes – if there is a conflict, the value being 

imported is applied 

– Manual 

• When a conflict occurs, the replica is marked as in conflict. While in 

conflict it can received changes but not send changes 

• To take it out of a conflict state, you need to later manually reconcile and 

post the synchronization version with the replica version 

• When synchronizing in both directions at once, manual can not 

be used as a synchronization policy 

• Support for object and attribute level reconciles on 

synchronization 

2007 ESRI NEARC 87

Manual and Automated Systems 

• Manual 

– Replication supports manual operations through wizards and 

geoprocessing tools in ArcCatalog and ArcMap 

– Example: A field worker connecting his lap top to a LAN or WAN 

and clicking synchronize. 

• Automated 

– Operations such as synchronizations and check-out replica 

creation are set up to happen on a regular basis 

– Example: A geoprocessing model exported to python and set to 

run in the windows scheduler 

– Recommended 

2007 ESRI NEARC 88

What Features are Synchronized 

• 2 way and 1 way replicas 

– Only changes that satisfy the filters and the relationship class criteria 

are synchronized 

– Example: Features moved within, into or out of a replica geometry y filter 

are synchronized. Features edited outside the replica geometry are a 

not 

synchronized 

– Edited rows related to rows in the replica are synchronized if they t 

belong to a table / feature class with a schema only filter 

• All edits are synchronized for check-out replicas 

2007 ESRI NEARC 89

Working through errors 

• System is designed to stay consistent 

• If the system fails during a synchronization, it is rolled back to it’s 

previous state 

• If a data changes message is lost in a disconnected system, the 

next message will contain changes from the lost message and any 

new changes 

• Replica log can be used to get error information about a 

synchronization 

2007 ESRI NEARC 90

Schema synchronization 

• Applying schema changes 

– Separate process from data synchronization 

– Compare replica schemas 

– Apply changes to match 

• Workflow 

– Avoid schema changes 

to prevent issues 

– Periodically check 

schemas 

– Apply system 

wide schema changes 

Region A 

Apply schema change 

National 

Apply schema change 

Make schema change 

Region B 

District 1 

District 2 

A 

2007 ESRI NEARC 91

Working with Schema Changes 

• Fault tolerant 

– In most cases synchronization will still execute successfully 

even if each replica makes schema changes 

– Example: If a field has been dropped, synchronization skips that 

field 

• Tools to apply schema changes across replicas 

– Subset of schema changes can be applied 

– Recommend applying schema changes across all replicas before 

synchronizing 

2007 ESRI NEARC 92

Applying Schema Changes 

• Two step process 

– Compare replica schemas 

• Compare directly in a connected environment 

• Export replica schema to XML in a disconnected environment, send to 

relative replica and compare 

– Import the schema differences 

• Schema changes can be listed in different ways: 

– Optional – you can decide whether or not to apply the change 

– Can’t t Apply – these can’t t be applied 

2007 ESRI NEARC 93

Schema Changes that can be Applied 

Add 

Change 

Drop 

Field 

Y 

Y (domains) 

Y 

Domain 

Y 

Y 

Y 

Table/FC 

Y 

Y (Domain, 

Add/drop field) 

Y 

Geometric network 

N 

N 

Y 

Topology 

N 

N 

Y 

Feature dataset 

N 

N 

Y 

Relationship class 

N 

Y (add/drop field, 

domain) 

Y 

2007 ESRI NEARC 94

Geodatabase Replication – API 

• Basic creation and synchronizations using geoprocessing models 

require little or no coding 

• Write code when… 

– Integrate geodatabase replication into larger applications 

• Example: integrate synchronization with reconcile service 

– Extend the replica creation and synchronization process 

– Methods and properties not exposed through the UI or geoprocessing 

tools 

• Recommended synchronization order 

• Browse changes before synchronization 

• Advanced replica creation options 

• Export and Import version differences 

2007 ESRI NEARC 95

Geodatabase Replication – API 

2007 ESRI NEARC 96

Geodatabase Replication Enhancements for 9.3 

• Replication New Functionality 

– Extend one way replication to file 

geodatabase and personal 

geodatabase 

• Replication Synchronization 

– Better support for synchronization 

with large delta XML files (files in 

excess of 1 Gig) 

– Improved logging for cases where 

synchronization is not successful 

UC 2007 2007 Tech ESRI Sessions NEARC 97 

97

Replica Creation – Performance Tips 

• Options in the UI 

– Re-use schema – If the schema already exists, replica creation can 

skip copying the schema. For check-out replicas only. 

– Schema only – Creates a replica of only schema for cases where you 

plan on only adding new data. For check-out replicas only. 

– Register existing data – Registers a replica without copying data. 

Useful for cases where the data is loaded before replica creation. 

NOTE: Globalids must match across geodatabases 

– Replicate related data – Can choose to turn it off to avoid processing 

at creation time 

• Open all feature classes and tables to replicate for creating the 

replica 

2007 ESRI NEARC 98

Example Automation Technique 

• Create a model with just the synch GP tool 

• fill in the parameters with the right info and save the model 

• export the model to a python file 

• add the command line to execute the python script to the windows 

scheduler 

2007 ESRI NEARC 99

ESRI Educational Solutions for Geodatabase 

Versioning, Replication, and Historical Archiving 

Danvers 1/17 

Johnstown 3/11 

Hartford 3/24 

Danvers 11/26, 2/26 

*Additional training locations and dates at http://training.esri.com 

2007 ESRI NEARC 100

Geodatabase Versioning, Replication and History - Northeast Arc ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?