Scalable Performance Data Server ® 3.0 - sasCommunity.org
Scalable Performance Data Server ® 3.0 - sasCommunity.org
Scalable Performance Data Server ® 3.0 - sasCommunity.org
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>Scalable</strong> <strong>Performance</strong><strong>Data</strong> <strong>Server</strong> ® <strong>3.0</strong>Alan GormleyCopyright © 2000 SAS EMEA
Design Goals of SPD <strong>Server</strong><strong>3.0</strong>! <strong>Performance</strong>! Web Enabled for E-intelligence!! 100% compatibility with existing SASapplications! Increase reliability/availability ! 24/7! Application benefits06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 2Copyright © 2000 SAS EMEA
Product History! Officially announced at SUGI 21(Chicago) - Apr1996! Release 1.0 Production (Sun) - Nov1996! Release 1.1 Production (Sun) - Feb1997! Release 1.2 Production (Sun) - Jun1997! Release 1.3 Beta (IBM AIX) - Jul1997! Release 1.4 Beta (NT) - Sep1997Continued...06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 3Copyright © 2000 SAS EMEA
Product History! Release 2.0 Beta (Sun, NT, HP-UX, AIX)- Nov1997! Release 2.0 (Sun, NT, HP) - Mar1998! Release 2.1 Beta (IBM AIX) - Aug1998! Release 2.1 (Sun, NT, HP) - Aug1998! Release 2.1 64 bit HP - Mar 1999! Release 2.1 TS020 - all platforms Sep06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 4Copyright © 2000 SAS EMEA1999Continued...
Product History! Continuing product enhancements tomeet real-world customerrequirements...! Release 2.2 TS020 wide-mode64-bit (HP) - production 1/2000! Release 2.2 64-bit (Compaq Tru64) -production 1/2000! Release 2.2 64-bit (Sun Solaris 7) -production 1/200006.07.00 High <strong>Performance</strong> Storage for E-Intelligence 5Copyright © 2000 SAS EMEA
Large Just Gets BiggerUnder 10 GBSmall10 GB - 50 GBMedium50 GB - 500 GBLarge500 GB - 10 TBVery Large(VLDB)Over 10 TBExtremely Large(ELDB)06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 6Copyright © 2000 SAS EMEA
Technology InfrastructureParallelism and ScalabilityHigh-speed interconnectNodes<strong>Performance</strong>CPUsSystemBUSDiskcontrollers...SMP nodesI/Osubsystems... ...High-speed interconnectClustered SMPLocaldisk...Massively parallelprocessors (MPP)Symmetric multi-processing(SMP)Storage Capacity06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 7Copyright © 2000 SAS EMEA
Parallelism and Scalability! Symmetric Multi-Processing! So-called “shared everything” architecture! All CPUs have access to commonmemory/IO! Traditional process becomes process +threads! Process == resource ownership(eg.Memory, IO)! Threads == code execution context == workunits06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 8Copyright © 2000 SAS EMEA
Parallelism and Scalability! Non-Uniform Memory Accessarchitectures! Many vendors have NUMA offerings! Composed of multi-CPU board-level blocks! Blocks connected via high-speed fabric! Looks to application like SMP architecturebut…! Applications/admins can make intelligentuse of config06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 10Copyright © 2000 SAS EMEA
SPD <strong>Server</strong> Client/<strong>Server</strong>ModelSPD Name<strong>Server</strong>(spdsnsrv)SETINITLibname‘xyz’registerdomaindataserverlocationlibnamerequestdomaindatarequestSPD <strong>Data</strong><strong>Server</strong>(spdsserv)libname domain‘xyz’launchesuserproxyprocessuserverificationUserTableSAS Libname EngineSASSPDSSAS ClientcommunicationSPD Proxy<strong>Server</strong>(spdsbase)<strong>Data</strong>06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 12Copyright © 2000 SAS EMEA
SPD <strong>Server</strong> Client/<strong>Server</strong>ModelODBC,JDBC,htmSQLClientSPD Name<strong>Server</strong>SETINIT(spdsnsrv)Libname‘xyz’registerdomaincommunicationdataserverlocationlibnamerequestdomaindatarequestSPD <strong>Data</strong><strong>Server</strong>(spdsserv)libname domain‘xyz’launchesuserproxyprocessuserverificationUserTableSPD SNet<strong>Server</strong>(spdssnet)communicationSPD Proxy<strong>Server</strong>(spdsbase)<strong>Data</strong>06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 13Copyright © 2000 SAS EMEA
SPD <strong>Server</strong> <strong>3.0</strong> Features! V8 Compliant! Admin Gui! Automatic Aging! SMP Enablement! New Enhanced bitmapindexing! VERY large tables! SQL pass-through! ASYNC parallelism! 64-bit edition!! Formats(SAS and User)! Incremental backup andrestore! Extensive security -Access Control Lists(ACLs)! Row level integrity! ODBC/JDBC/htmSQL! SQL C-runtimeDLL/shared library API! Dynamic refresh of<strong>Server</strong> parameters.06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 14Copyright © 2000 SAS EMEA
Features of SPD <strong>Server</strong> <strong>3.0</strong>Plug-compatibility! Provide all functionality of BASE Enginefor V8! New SASSPDS Engine! Support various data types! <strong>Data</strong>, indexes, catalogs, utility files (MDDBsetc.)LIBNAME! VariousspdlibloadSASSPDStechniques‘domain’ SERV=host.portUSER=username PROMPT=YES|PASSWD=password;! Procedures (COPY, APPEND), DATA Step,Pass-through LOAD and COPYPROC APPEND BASE=spdlib.table DATA=saslib.newdata;RUN;06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 15Copyright © 2000 SAS EMEA
Features of SPD <strong>Server</strong> <strong>3.0</strong>Exploit SMP Parallelism! WHERE-clause evaluation06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 16Copyright © 2000 SAS EMEA! Where paybacks are evident! Index updates! Doesn’t require the index deletion prior toupdates! Index Creation! Multiple indexes are created simultaneously! Sorting! PROC SORT, BY Group processing (autosort)
Features of SPD <strong>Server</strong> <strong>3.0</strong>Exploit SMP Parallelism! Full table scan! Uses multiple file partitions! SQL SELECT with GROUP BY! Includes aggregate function support! Very fast!!! Loading/copying of table! APPEND/COPY procedures, SQL passthroughload and copy06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 17Copyright © 2000 SAS EMEA
Features of SPD <strong>Server</strong> <strong>3.0</strong>Enhanced Bitmap Indexing! Improves creation of index segments! Faster where clause evaluation usingpre-evaluation and costing of indexsegments! Removes low cardinality restrictions fromprevious releases06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 18Copyright © 2000 SAS EMEA
Features of SPD <strong>Server</strong> <strong>3.0</strong>Large File Support! Eliminate file size restrictions! Dynamic file partitioning! Component Files06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 19Copyright © 2000 SAS EMEA
Features of SPD <strong>Server</strong> <strong>3.0</strong>Name <strong>Server</strong>s! Reduce maintenance requirements! Eliminate impacts on server distribution06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 20Copyright © 2000 SAS EMEA
Features of SPD <strong>Server</strong> <strong>3.0</strong>SPD SQL Pass-through! Better optimisation control! Full SQL capability on the server! New SQL verbs COPY and LOADprovide parallel copying and loading oftables! Parallel execution of Grouped-SelectQueries! Asynchronous statement execution06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 21Copyright © 2000 SAS EMEA
Features of SPD <strong>Server</strong> <strong>3.0</strong>SPD SQL Pass-throughproc sql;connect to sasspds (dbq='tmp' host='gerhds2'serv='5131’user='master' passwd='spds21');execute (Begin Async Operation) by sasspds;execute ( create table salesnorthas select * from saleswhere region = 'North' ) by sasspds;execute ( create table saleseastas select * from saleswhere region = ‘East' ) by sasspds;execute ( create table salessouthas select * from saleswhere region = 'South' ) by sasspds;execute ( create table saleswestas select * from saleswhere region = 'West' ) by sasspds;execute (End Async Operation) by sasspds;disconnect from sasspds;quit;06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 22Copyright © 2000 SAS EMEA
Features of SPD <strong>Server</strong> <strong>3.0</strong>Incremental Backup and Restore! Reduces backup times! Conserves valuable system resources! Faster incremental backups encouragesmore frequent use that will minimisepossible loss or corruption of data06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 24Copyright © 2000 SAS EMEA
Features of SPD <strong>Server</strong> <strong>3.0</strong>Row Level Integrity! Concurrent updates by multiple users onthe same table! All user share one process, but run in aseparate thread environment. This givesmore efficient dispatching and loadbalancing.06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 25Copyright © 2000 SAS EMEA
Features of SPD <strong>Server</strong> <strong>3.0</strong>ODBC/JDBC/htmSQL - SPDSSNET! Accesses the <strong>Data</strong> <strong>Server</strong> using anODBC/JDBC/htmSQL serverSPDSSNET! Transparent access to the <strong>Scalable</strong><strong>Performance</strong> <strong>Data</strong> <strong>Server</strong> using thestandard ODBC/JDBC/htmSQL driversprovided by the SAS/IntrNet product06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 26Copyright © 2000 SAS EMEA
Features of SPD <strong>Server</strong> <strong>3.0</strong>Automatic Aging Rollout! Automatic creation and maintenance ofaged/time based generations.! Automatic deletion of oldest generation.! Access can be via the rollout cluster or aparticular generation.! Very useful and common need in DW06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 27Copyright © 2000 SAS EMEA
Features of SPD <strong>Server</strong> <strong>3.0</strong>Automatic Aging Rollout! Access via simple options:! Create MAXGEN=nnn table option, SPDSGENDmacro variable! Normal accesses == whole rollout cluster! NUMGEN= option access cluster members! Access by relative generation number! Access by absolute generation number%LET SPDSGEND=1/1/1999;DATA SPDS.CLUSTERED(MAXGEN=6);SET SOMELIB.GEN1;RUN;PROC APPEND DATA=SOMELIB.GEN2 OUT=SPDS.CLUSTERED(NUMGEN=+1); RUN;PROC PRINT DATA=SPDS.CLUSTERED(NUMGEN=1); RUN;06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 28Copyright © 2000 SAS EMEA
Automatic Aging123456JanFebMarAprMayJun7Jul8Aug9Sep10Oct11Nov12Dec06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 29Copyright © 2000 SAS EMEA
Automatic Aging123456JanFebMarAprMayJun7Jul8Aug9Sep10Oct11Nov12Dec13Jan06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 30Copyright © 2000 SAS EMEA
Automatic Aging123456JanFebMarAprMayJun7Jul8Aug9Sep10Oct11Nov12Dec13Jan06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 31Copyright © 2000 SAS EMEA
Features of SPD <strong>Server</strong> <strong>3.0</strong>Support for user formats! Can be used in SQL group-by or inwhere clauses or SQL statement.! You have to use SAS to define the procformat style formats and store it in SPDS.06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 32Copyright © 2000 SAS EMEA
Migration from V2 to V3 ofSPDS! On disk structure different, cannot be shared! You must copy data from V2 to V3! You can convert “in-place” using spdsconv utility! Copy: Pros/Cons! Can continue to access data from SAS V6 and V8! Requires 2x disk space to keep both! Conversion: Pros/Cons! Saves disk space. Converts only MDF component! No longer can access data from SAS V6! Can’t preserve V2 indexes but V3 better anyway06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 33Copyright © 2000 SAS EMEA
Migration from V2 to V3 ofSPDS! Password <strong>Data</strong>base must be converted! Create a new one and “import” an “exported”version of the password database for V2! New SAS engine must be used.! We do provide a compatibility engine toaccess V2 SPDS server.! V2 ACL files must be converted using theconversion utility.06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 34Copyright © 2000 SAS EMEA
Conclusion! Very high performance store for detail datawhich you can get from Web logs, your DW! Use SPDS group by to create summarizedtables.! Support for Version 8 SAS.! Many new RAS features! Proven track record in the field!!!!! <strong>Scalable</strong> architecture, size and speed!! High ROI06.07.00 High <strong>Performance</strong> Storage for E-Intelligence 35Copyright © 2000 SAS EMEA
Change language