Apress.Expert.Oracle.Database.Architecture.9i.and.10g.Programming.Techniques.and.Solutions.Sep.2005
CHAPTER 15 ■ ■ ■ Data Loading and Unloading In this chapter, we will discuss data loading and unloading—in other words, how to get data into and out of an Oracle database. The main focus of the chapter is on the following bulk data loading tools: • SQL*Loader (pronounced “sequel loader”): This is still a predominant method for loading data. • External tables: This is a new feature with Oracle9i and above that permits access to operating system files as if they were database tables and, in Oracle 10g and above, even allows for the creation of operating system files as extracts of tables. In the area of data unloading, we’ll look at two techniques: • Flat file unload: The flat file unloads will be custom developed implementations, but will provide you with a result that is portable to other types of systems (even a spreadsheet). • Data Pump unload: Data Pump is a binary format proprietary to Oracle and is accessible via the Data Pump tool and external tables. SQL*Loader SQL*Loader (SQLLDR) is Oracle’s high-speed, bulk data loader. It is an extremely useful tool used to get data into an Oracle database from a variety of flat file formats. SQLLDR can be used to load enormous amounts of data in an amazingly short period of time. It has two modes of operation: • Conventional path: SQLLDR will employ SQL inserts on our behalf to load data. • Direct path: SQLLDR does not use SQL in this mode; it formats database blocks directly. The direct path load allows you to read data from a flat file and write it directly to formatted database blocks, bypassing the entire SQL engine, undo generation and, optionally, redo generation at the same time. Parallel direct path load is among the fastest ways to go from having no data to a fully loaded database. We will not cover every single aspect of SQLLDR. For all of the details, refer to the Oracle Utilities manual, which dedicates seven chapters to SQLLDR in Oracle 10g. The fact that it is 649
650 CHAPTER 15 ■ DATA LOADING AND UNLOADING covered in seven chapters is notable, since every other utility, such as DBVERIFY, DBNEWID, and LogMiner get one chapter or less. For complete syntax and all of the options, I will refer you to the Oracle Utilities manual, as this chapter is intended to answer the “How do I . . .?” questions that a reference manual does not address. It should be noted that the Oracle Call Interface (OCI) allows you to write your own direct path loader using C, with Oracle 8.1.6 Release 1 and onward. This is useful when the operation you want to perform is not feasible in SQLLDR, or when seamless integration with your application is desired. SQLLDR is a command-line tool (i.e., it’s a separate program). It is not an API or anything that can be “called from PL/SQL,” for example. If you execute SQLLDR from the command line with no inputs, it gives you the following help: [tkyte@desktop tkyte]$ sqlldr SQL*Loader: Release 10.1.0.4.0 - Production on Sat Jul 16 10:32:28 2005 Copyright (c) 1982, 2004, Oracle. All rights reserved. Usage: SQLLDR keyword=value [,keyword=value,...] Valid Keywords: userid -- ORACLE username/password control -- control file name log -- log file name bad -- bad file name data -- data file name discard -- discard file name discardmax -- number of discards to allow (Default all) skip -- number of logical records to skip (Default 0) load -- number of logical records to load (Default all) errors -- number of errors to allow (Default 50) rows -- number of rows in conventional path bind array or between direct path data saves (Default: Conventional path 64, Direct path all) bindsize -- size of conventional path bind array in bytes (Default 256000) silent -- suppress messages during run (header,feedback,errors,discards,partitions) direct -- use direct path (Default FALSE) parfile -- parameter file: name of file that contains parameter specifications parallel -- do parallel load (Default FALSE) file -- file to allocate extents from skip_unusable_indexes -- disallow/allow unusable indexes or index partitions (Default FALSE) skip_index_maintenance -- do not maintain indexes, mark affected indexes as unusable (Default FALSE) commit_discontinued -- commit loaded rows when load is discontinued (Default FALSE) readsize -- size of read buffer (Default 1048576) external_table -- use external table for load; NOT_USED, GENERATE_ONLY, EXECUTE
- Page 644 and 645: CHAPTER 13 ■ PARTITIONING 599 imp
- Page 646 and 647: CHAPTER 13 ■ PARTITIONING 601 OLT
- Page 648 and 649: CHAPTER 13 ■ PARTITIONING 603 5 s
- Page 650 and 651: CHAPTER 13 ■ PARTITIONING 605 Sur
- Page 652 and 653: CHAPTER 13 ■ PARTITIONING 607 On
- Page 654 and 655: CHAPTER 13 ■ PARTITIONING 609 Row
- Page 656 and 657: CHAPTER 13 ■ PARTITIONING 611 So,
- Page 658 and 659: CHAPTER 13 ■ PARTITIONING 613 Aud
- Page 660 and 661: CHAPTER 14 ■ ■ ■ Parallel Exe
- Page 662 and 663: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 664 and 665: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 666 and 667: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 668 and 669: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 670 and 671: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 672 and 673: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 674 and 675: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 676 and 677: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 678 and 679: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 680 and 681: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 682 and 683: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 684 and 685: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 686 and 687: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 688 and 689: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 690 and 691: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 692 and 693: CHAPTER 14 ■ PARALLEL EXECUTION 6
- Page 696 and 697: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 698 and 699: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 700 and 701: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 702 and 703: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 704 and 705: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 706 and 707: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 708 and 709: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 710 and 711: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 712 and 713: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 714 and 715: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 716 and 717: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 718 and 719: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 720 and 721: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 722 and 723: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 724 and 725: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 726 and 727: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 728 and 729: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 730 and 731: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 732 and 733: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 734 and 735: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 736 and 737: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 738 and 739: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 740 and 741: CHAPTER 15 ■ DATA LOADING AND UNL
- Page 742 and 743: CHAPTER 15 ■ DATA LOADING AND UNL
650<br />
CHAPTER 15 ■ DATA LOADING AND UNLOADING<br />
covered in seven chapters is notable, since every other utility, such as DBVERIFY, DBNEWID,<br />
<strong>and</strong> LogMiner get one chapter or less. For complete syntax <strong>and</strong> all of the options, I will refer<br />
you to the <strong>Oracle</strong> Utilities manual, as this chapter is intended to answer the “How do I . . .?”<br />
questions that a reference manual does not address.<br />
It should be noted that the <strong>Oracle</strong> Call Interface (OCI) allows you to write your own direct<br />
path loader using C, with <strong>Oracle</strong> 8.1.6 Release 1 <strong>and</strong> onward. This is useful when the operation<br />
you want to perform is not feasible in SQLLDR, or when seamless integration with your application<br />
is desired. SQLLDR is a comm<strong>and</strong>-line tool (i.e., it’s a separate program). It is not an API<br />
or anything that can be “called from PL/SQL,” for example.<br />
If you execute SQLLDR from the comm<strong>and</strong> line with no inputs, it gives you the following<br />
help:<br />
[tkyte@desktop tkyte]$ sqlldr<br />
SQL*Loader: Release 10.1.0.4.0 - Production on Sat Jul 16 10:32:28 2005<br />
Copyright (c) 1982, 2004, <strong>Oracle</strong>. All rights reserved.<br />
Usage: SQLLDR keyword=value [,keyword=value,...]<br />
Valid Keywords:<br />
userid -- ORACLE username/password<br />
control -- control file name<br />
log -- log file name<br />
bad -- bad file name<br />
data -- data file name<br />
discard -- discard file name<br />
discardmax -- number of discards to allow<br />
(Default all)<br />
skip -- number of logical records to skip (Default 0)<br />
load -- number of logical records to load (Default all)<br />
errors -- number of errors to allow (Default 50)<br />
rows -- number of rows in conventional path bind array or<br />
between direct path data saves<br />
(Default: Conventional path 64, Direct path all)<br />
bindsize -- size of conventional path bind array in bytes (Default 256000)<br />
silent -- suppress messages during run<br />
(header,feedback,errors,discards,partitions)<br />
direct -- use direct path<br />
(Default FALSE)<br />
parfile -- parameter file: name of file that contains parameter specifications<br />
parallel -- do parallel load<br />
(Default FALSE)<br />
file -- file to allocate extents from<br />
skip_unusable_indexes -- disallow/allow unusable indexes or index partitions<br />
(Default FALSE)<br />
skip_index_maintenance -- do not maintain indexes, mark affected indexes as unusable<br />
(Default FALSE)<br />
commit_discontinued -- commit loaded rows when load is discontinued (Default FALSE)<br />
readsize -- size of read buffer (Default 1048576)<br />
external_table -- use external table for load; NOT_USED, GENERATE_ONLY, EXECUTE