Outline of High Speed Ingestion with Data Partitioning in Content ...

© Copyright 2007 EMC Corporation. All rights reserved.4Direct Path Loading• Oracle Direct Path Load (ODPL) allows object meta-data tobe loaded significantly faster than conventionaltransactional methods– By-passes normal transaction logic– By-passes Oracle server, formats pages, stores directly to file– 100’s of millions per day can easily be ingested• Caveats:– Indexes made unusable on tables by ODPL– Indexes need to be rebuilt• Data can come from ascii files

© Copyright 2007 EMC Corporation. All rights reserved.5Other “traditional” limitations of Direct Path Loading• Many of the tables that had to be generated have lots ofContent Server internal columns– Subject to change in a release– Not documented– Example: data ticket• Direct Path loading provided back-door that wouldn’t allowBOF customizations to be applied to objects

© Copyright 2007 EMC Corporation. All rights reserved.6Getting around the ODPL limitations• Model Simplifications– Lightweight objects:• Allows us to avoid loading dm_sysobject_s & dm_sysobject_r– External store objects• Stored where the application decides. Avoids having to reverse engineerdatatickets• Data Partitioning:– ODPL can be run on schema-identical partition on separate offlinetable– Once Indexes rebuilt the offline partition can be swapped withcorresponding empty online one with little overhead– Requires, potentially only a few minutes of application downtime

© Copyright 2007 EMC Corporation. All rights reserved.7Database layout for exampl normal sysobjectDm_sysobject_s and _rDmi_object_typeDmr_content_s & _rAn object’smeta-data isspread overseveral tablesjoined byr_object_idDm_document_sMytype_sMyparent_type_s

© Copyright 2007 EMC Corporation. All rights reserved.8Lightweight objects consume less space per object• Each Lightweight objects will nothave a corresponding separateset of entries in:– Dm_sysobject_s & r– Dm_document_s– And the parent custom typetables• Light weight objects share thosewith other light weight objectsDmi_object_typeDmr_content_s & _rMytype_s

© Copyright 2007 EMC Corporation. All rights reserved.9Background: Leveraging RDBMS Partitioning• Scheme• Hash partitioning vs. Range Partitioning• Local Indexes

© Copyright 2007 EMC Corporation. All rights reserved.10Database Management Challenges at a Billion ObjectsLarge underlying DCTM tableCommon operationsbecome slow and difficult:-Index creation-Statistics update

© Copyright 2007 EMC Corporation. All rights reserved.11Range Partition on D6.5 partition_id0 < ID < 10P0One Table spaceper partition10 ≤ ID < 2020 ≤ ID < 30P2P31,000 ≤ ID < 1,100P1011,100 ≤ ID < MAXVALUE

© Copyright 2007 EMC Corporation. All rights reserved.12Partitioning Scheme• Partitioning on partition_id for the tables associated with sysobjects:– Dm_sysobject_s, dm_sysobject_r, dm_document_s– dmi_object_type,– Dmr_content_s and dmr_content_r– Aspect tables– Acl tables– Your custom type tables• Partition id is an integer value

© Copyright 2007 EMC Corporation. All rights reserved.13Some Advantages of Partitioning• Indexes can be managed on partition basis (“Local”)• Statistics computed on partition basis• Partition-only lookups fast• Partition exchange for high speed ingest• Compression of cold partitions• Mass drop for partition aligned data

© Copyright 2007 EMC Corporation. All rights reserved.14Partition Key Index lookupsSELECT …. FROM … WHERE partition_id = ‘3’Database can take advantage of key tolimit search to single partition

© Copyright 2007 EMC Corporation. All rights reserved.15Non-Partition Key Index lookupsSELECT …. FROM … WHERE OBJECT_NAME = ‘FOO’Database must iterate through eachpartition to locate results

© Copyright 2007 EMC Corporation. All rights reserved.16“Global” indexes span the entire set of partitionsSELECT …. FROM … WHERE OBJECT_NAME = ‘FOO’Faster lookup’s than Non-partition KeyLocal ones, but maintenance cost is high

© Copyright 2007 EMC Corporation. All rights reserved.17Index notes• A partitioned set of Documentum tables will only leveragelocal indexes– Partition exchange can’t be done with a table that has a global index• Limits on partitions:– Oracle : 64K – 1 partitions per table/index– SQL Server 2005: 1000 partitions per table/index• Non-partition key lookups will get more expensive as thenumber of partitions grows– However, parallel index scans over partitions possible

© Copyright 2007 EMC Corporation. All rights reserved.18Range vs. Hash PartitioningID = 1ID = 2ID = 3Consecutive Objects mapped to same partition in range

© Copyright 2007 EMC Corporation. All rights reserved.19Hash PartitioningID = 1ID = 2ID = 3Consecutive Objects to different partitions Less Complicated Table Setup and Maintenance However, poor data management qualities

© Copyright 2007 EMC Corporation. All rights reserved.20Partition Exchange ExampleDmr_content_rDmr_content_r_offlinePartition 1Num recs = 1,000Partition 1Num recs = 0Partition 2Num recs = 0Partition 2Num recs = 0Partition 3Num recs = 0Partition 3Num recs = 0Online version of tableContent Server = UP

© Copyright 2007 EMC Corporation. All rights reserved.21Direct Path Loading into offline tableDmr_content_rDmr_content_r_offlinePartition 1Num recs = 1,000Partition 1Num recs = 0Partition 2Num recs = 0Partition 2Num recs = 10MDirect Path LoadPartition 3Num recs = 0Partition 3Num recs = 0Online version of tableContent Server = UP

© Copyright 2007 EMC Corporation. All rights reserved.22Indexes rebuilt in offline tableDmr_content_rDmr_content_r_offlinePartition 1Num recs = 1,000Partition 1Num recs = 0Partition 2Num recs = 0Partition 2Num recs = 10MIndex RebuildPartition 3Num recs = 0Partition 3Num recs = 0Online version of tableContent Server = UP

© Copyright 2007 EMC Corporation. All rights reserved.23Content Server needs to be brought downDmr_content_rDmr_content_r_offlinePartition 1Num recs = 1,000Partition 1Num recs = 0Partition 2Num recs = 0Partition 2Num recs = 10MPartition 3Num recs = 0Partition 3Num recs = 0Online version of tableContent Server = DOWN

Partition ExchangeDmr_content_rDmr_content_r_offlinePartition 1Num recs = 1,000Partition 1Num recs = 0Partition 2Num recs = 10MPartition 3Num recs = 0Online version of tableContent Server = UP© Copyright 2007 EMC Corporation. All rights reserved.Partition 2Num recs = 0Partition 3Num recs = 0SQL> ALTER TABLE DMR_CONTENT_SEXCHANGE PARTITION 2 WITH TABLEDMR_CONTENT_S _offline INCLUDINGINDEXES WITHOUT VALIDATION;Table altered.24

© Copyright 2007 EMC Corporation. All rights reserved.25Content Server can be brought back upDmr_content_rDmr_content_r_offlinePartition 1Num recs = 1,000Partition 1Num recs = 0Partition 2Num recs = 10MPartition 2Num recs = 0Partition 3Num recs = 0Partition 3Num recs = 0Online version of tableContent Server = DOWN UP

© Copyright 2007 EMC Corporation. All rights reserved.26Other notes for partition exchange• Indexes associated with partitioned tables will be ‘local’indexes• Content server will generate scripts to exchange a partitionwith some in an offline tables– Apply method determines the tables involved and generates a script• The content must be shutdown before (and during) theexecution of the scripts• Note: dmi_object_type should be ranged partitioned tosupport this (currently not done by default)– A new getObjectWithOptions() method will be provided to allow thecaller to supply the object type and partition id in order to makefetch inexpensive– Otherwise a fetch will require a local index lookup per partition

© Copyright 2007 EMC Corporation. All rights reserved.27Best Use-cases for partitions• Ingest of legacy data for online system• Massive ingest of few input streams in short load window• Mass drop of partition aligned data