Migration Planning Guide: IBM Rational ClearCase to Perforce SCM

Perforce Software’s migration tool for ClearCase migrations is “front door” with respect to extractionfrom ClearCase, and “back door” with respect to importing into Perforce. Being front door on theClearCase side enables the tool to work across many ClearCase versions, while being back dooron the Perforce side allows it to generate the most accurate representations of ClearCase history interms of Perforce journal records.Custom ScriptingA detailed migration sometimes involves custom scripting due to differences in ClearCase usage andany oddities in a particular data set.Hardware Capacity PlanningHardware capacity planning may be impacted significantly with DHI migrations. An SCM system with,for example, 12 years of history would require more hardware (more disk space, more RAM, faster CPUsand I/O subsystems, etc.) than one with no history. If you import 12 years of detailed history, yournew Perforce system will initially require as much hardware as if it had been in operation for 12 years.Pros of a Detailed History Import• Most complete approach. Detailed history imports can transfer the most historical detail,including branching history, from ClearCase to Perforce.• After the migration, comprehensive historical research and “merge forensics” can be done inPerforce without the need for going back to ClearCase (we recommmend keeping one userlicense in ClearCase as a backup).• The ability to view file history with Perforce’s powerful visualization tools like Time-Lapse Viewcan shed new light on the evolution of source code and help increase understanding of thechanges over time.• There is an increased benefit for systems integrated with version control. For example, themeaning of the linkage between a set of files originally modified in ClearCase and an issue fromyour issue tracking system can be maintained. Code review tools such as SmartBear CodeCollaborator will have greater context.• Unlike Perforce, ClearCase does not validate the integrity of versioned file contents usingchecksums. File corruption, for example, because of disk failures, goes undetected. 1 Oncehistorical data is in Perforce, it will gain the benefit of checksum verification of contents of allrevisions, which improves IP provenance.Cons of a Detailed History Import• Detailed history import tools have a variety of limitations and technical caveats because of thepotential complexity of ClearCase environments, including unusual patterns (or even corruption)in the data. So-called evil-twin elements or branching scenarios created by misconfigured configspecs can be difficult to follow.• Complex migrations can mean potential schedule and budget risks for the migration project.Baseline and Branch Import (BBI)The BBI strategy provides a lightweight migration alternative that is more sophisticated than the Tipsapproach and avoids the technical complexity and schedule and budget risks of detailed historyimports.BBI is a generic from-anything-to-Perforce process, and has been used to migrate to Perforce froma variety of SCM systems, including IBM Rational ClearCase, Borland StarTeam, Merant PVCS,1. ClearCase does have a “checkvob” utility that can detect and fix some forms of metadata corruption. However, this utility does not detect data container corruption, andthus the contents of versioned files cannot be audited.7

Subversion, Mercurial, CVS, Microsoft Visual Source Safe, AccuRev, and even unsophisticated SCM“systems” like a set of network drives with directories named for releases.With the BBI approach, the “interesting history” to be imported is specified using a branch diagramthat shows the baselines (snapshots of a directory structure at a specified point in time) and majorbranching operations. For example, Figure 1 represents a software product. The baselines (blue dots)indicate the “interesting versions” that are to be imported. The arrows indicate branching operationsthat affect an entire branch. In this example, a 2.0-Rel branch has been created, and four patcheswere created from that branch. Two of those four patches have been merged back to MAIN. The BBIprocess imports all the baselines, records that the two patches were merged with resulting updatesto MAIN, and tracks the two unmerged patches. After migrating, you can use Perforce to completethose merges.Delivered(Released)p1 p2 p3 p4 p12.0 RelBranch3.0 RelBranchMAINBranch1.0 2.0 2.1 3.0 4.02.1 (new features)+ patch p1 merged2.1 (new features)+ patch p2 mergedFigure 1: Sample baseline and branch diagramImporting the branching operations enables Perforce to select common ancestors when merging, soyou can resume branching activities after migration. The BBI process imports branching operations ata high level, capturing the sum of merge operations. For example, in Figure 1, the merge of p2 backto MAIN most likely consisted of a series of merges by several developers. The individual file mergesare not tracked, but the sum of the results of the merges (file adds, edits, and deletes) are tracked.The imported baseline represents the point in time when the merge of p2 is complete.The intent of this approach is to bring over just enough branching history to answer key questionssuch as what did Release 2.0 look like, where was this file branched from, and what files do I need inmy workspace to start maintenance work on Release 2.3? The BBI approach preserves file contentsat key points and preserves enough branching history so that the switch to Perforce can happen atany point in the release cycle, rather than just at “convenient points” in the schedule (which tend tobe hard to find).After conversion, Perforce contains the history of your software product viewable using the RevisionGraph tool, and the history is similar to what would have been recorded if development was doneusing Perforce to begin with. Detailed data is discarded: you know what your product looked like atRelease 1.0 and Release 2.0, for example, but hundreds of check-ins between those baselines arediscarded, as are the user ID, date, time, and check-in comments.Accurate diagrams are essential for planning a BBI migration. Ideally, release engineers can quicklydraw an accurate branch history for each software product to be imported. If that is not the case,such information can be extracted by exploring ClearCase manually. After the diagram is drawn andverified, it is translated into the Perforce commands that re-create the baselines in Perforce. The firstMigration Planning Guide: IBM Rational ClearCase to Perforce SCM 8

4. Terminology and ConceptsThe following sections map ClearCase concepts to their Perforce equivalents.VOBs and DepotsA Perforce depot is roughly analogous to a ClearCase VOB (which stands for “versioned objectbase”). VOBs and depots both appear as top-level directories to users, and both store a set of files.One VOB or depot must exist before any file can be versioned.A VOB is a container for versioned file contents and metadata relating to those versioned files. APerforce depot contains only the contents of versioned files. All metadata is stored in a database onthe Perforce server.When mapping VOBs to depots, consider the following:• Unlike files in VOBs, files in Perforce can be branched to other depots. VOBs in ClearCase areeffectively islands of code.• Perforce manages binary files efficiently, enabling you to manage all your digital assets.ClearCase sites avoid storing large number of binaries because of performance considerations.For example, a software product might consist of source code, software products built from thatsource code, and various released configurations of software products. A separate depot mightbe assigned for each, for example, //gizmo, //gizmo-build, and //gizmo-release.• Perforce depot names should be kept short. //Engineering is OK, but //Eng is better.//Eng-AdvancedTechnologyGroup is a bit long, so //Eng-ATG is better.Regions and ProtectionsIn ClearCase, network registry regions are used to segregate VOBs. A region sees only a subset ofall VOBs in a ClearCase installation.To achieve similar segregation in Perforce, you use the protections table. Users are assigned to differentPerforce groups. Access is then managed at the group level by assigning permissions to groups.VOB Servers and the Perforce ServerIn ClearCase environments, you might have multiple VOB server processes, possibly distributedamong multiple VOB server machines. With Perforce, a single Perforce Server can support an entireinstallation. 3 The Perforce Server process, P4D, runs on a single machine, is frugal with systemresources, and is much less demanding than ClearCase. One P4D process can scale to supportextremely large environments (for example, 10,000+ users) on a single server using enterprise-gradeserver machines.One of the first steps in any migration is to set up Perforce hardware. It is common to allocate two orthree identical server machines to Perforce to achieve high availability and disaster recovery goals.A typical configuration is two servers (a primary and a hot spare) in a primary data center and a thirdserver (warm spare) in another data center located far from the primary data center.Operating System SelectionThe primary factor in selecting an operating platform for Perforce is the platform that the local ITgroup is most comfortable supporting. In mixed Windows/*nix environments, Unix platforms are almostinvariably selected for ClearCase, because of case sensitivity reasons and because of its reliance on3. Deploying multiple Perforce Server instances within an enterprise is possible and common. For the purposes of this document, we consider only the single-server-perenterpriseapproach because that best suits most ClearCase migration scenarios.Migration Planning Guide: IBM Rational ClearCase to Perforce SCM 10

effectively monodirectional (Unix Ê Windows) file system mounts to make VOB data accessible toclients on both Unix and Windows.You can configure the Perforce server on Windows or *nix in multiplatform environments. Only a TCPconnection is needed between clients and servers. You can configure the case-sensitivity behaviorof the Perforce Server independently of the platform.Registry and License ServersPerforce does not require license or registry server processes or hardware to support it. A simple0.5KB license file on the Perforce Server machine composes the entire Perforce license mechanism.For large environments with lots of turnover, Perforce provides alternative licensing schemes thatsupply extra licenses for growth.Release Servers and InstallationWith ClearCase, to ensure users run client software that is compatible with the current version of theserver, you must manage server and client versions carefully. To aid in ensuring consistency, someClearCase installations deploy a release server, which is a defined network resource from which allusers are expected to download correct versions of client software. This approach provides a morescalable alternative to making sure everyone has the installation CD available.With Perforce, all client components (and the server) install in minutes over the Web. More importantly,Perforce clients and the server have a very flexible forward- and backward-compatible relationship,because of a version-aware client/server protocol. Users can generally run client versions that areolder or newer than the server. Client programs simply hide or disable features that require newerversions of the server, and new server versions rarely require client upgrades.For Windows sites with a large number of users, Perforce supports a centrally configured, automateddeployment of Perforce. You can use such sites to ensure that users download consistent, trustedversions of software that are supported by IT and/or release engineering staff. However, becauseof extreme compatibility and extreme ease of installation, maintaining such areas is much less of arequirement than in ClearCase environments.View Servers, Protecting Unversioned and Checked-Out FilesPerforce stores all metadata in a database on the central server, managed by the P4D process. 4A ClearCase View Server process has no equivalent in Perforce, and administrators don’t need toallocate and configure View Server machines.When dynamic views are used, View Server machines contain the contents of checked-out andunversioned-view private files. Some organizations back up view storage areas regularly to protectagainst the loss of such files. If protecting checked-out and unversioned files is a priority, you willneed to back up client machines.Some organizations devise a process to protect unversioned and checked-out workspace files inPerforce, such as locating workspaces on network drives that are backed up. During Perforce training,users are advised to avoid keeping files checked out for too long, using sandbox branches if needed.ClearCase MultiSite versus Perforce ProxiesIf your ClearCase environment relies on MultiSite, you will find the Perforce Proxy mechanism simpleand effective. Perforce proxies cache the contents of versioned files at remote sites, greatly reducingdependency on the WAN.4. Some GUI programs temporarily cache metadata in running processes, but such information is not persisted.11

Proxies do not cache any metadata, thereby ensuring that there is a single source of state informationand eliminating the need for branch mastership, scheduling batch replication, and so on.Replicated VOB servers generally run on server machines that are similar to the primary server, thusrequiring similar-tiered data centers. Because of the significant investment in licenses, hardware,and administrative overhead, MultiSite installations are used only where major development centersexist, and they are of little use to small, geographically diverse teams.By contrast, Perforce Proxy servers are lightweight programs that can run on desktop-grade hardware,even in enterprise environments. Proxy servers do not require high-performance hardware or largeamounts of disk space, so they can be deployed anywhere that a few developers gather. In somecases, individual users deploy Perforce Proxy instances in small offices without IT support. Thereare no additional costs or licenses required to deploy a Perforce Proxy server.ClearCase Views with Perforce WorkspacesThe term “workspace” is familiar to both ClearCase and Perforce users: it is where developersmanage files under version control on their local machines.With ClearCase, it is typical for a developer to maintain several workspaces, called “views.” Developerswho are working on multiple branches typically use a different view for each activity, working in oneview at a time. For example, a developer might maintain a joe_user_main_dev view with a configspec selecting /main/LATEST versions and a separate joe_user_rel_2.3 view selecting/main/REL2.3/LATEST versions. 5A Perforce “client specification” defines a workspace and determines the files from the server thatare visible in the workspace. Branches in Perforce appear to the user as fully-populated directorytrees. For example, the server might contain a directory named MAIN and another directory namedREL2.3. A developer might have a joe_user_dev workspace, that includes both MAIN and REL2.3directories. The developer can work in both activities at the same time.In Perforce, only user files are stored on the local disk. All metadata, including information about thename and contents of a user’s workspace, resides on the server.Label StrategiesBoth ClearCase and Perforce provide labels, which identify the versions of files that constitute abaseline. For many ClearCase users, labels are mandatory. Applying labels is time-consuming, oftenaccounting for 30 percent or more of the time associated with creating stable builds.In Perforce, labels are just one way of reproducing baselines. Changelists accomplish the samegoal in ways that are less taxing on the build process and faster and easier to reference than a label.Each Perforce check-in generates a unique changelist number that reflects the state of the repositoryat a point in time. Any changelist can be used to describe the state of every file in the repository,even though it affects only a small subset of the repository.Branches in Perforce are represented as directories, making it easy to combine branches andchangelist numbers to represent a baseline. Alternately, labels can refer to changelist numbers limitedto an identified scope in the server, where the scope is typically a particular branch.5. This is an oversimplication since a typical config spec is several lines or more.Migration Planning Guide: IBM Rational ClearCase to Perforce SCM 12

Unified Change Management (UCM)UCM adds a layer of process to ClearCase. Out of the box, Perforce provides atomic changelistsand jobs, which correlate to UCM functionality that enables ClearCase developers to group files andconnect them to an activity description. UCM also provides guidance on common software promotionmodels and branching strategies. Perforce’s open architecture accommodates various enterprise-widedevelopment methodologies and SCM customizations.Migration Technical DetailsPerforce and ClearCase have very different internal representations and models of parallel development,branching, and merging. Both support parallel development, but you should be aware of the differences.Evil TwinsDescribes the problem in which two elements with the same name appear in different branches.For example, say you have a MAIN, DEV_A, and DEV_B branches, with each of the DEV branchesparented directly from MAIN. In DEV_A branch, a developer does a ‘ct mkelem’ to create a new fileelement, foo.c. Independently in DEV_B branch, a developer does the same thing —does a ‘ct mkelem’to create a new file element, foo.c. Someone then does a ‘ct findmerge’ to make file that originatedon DEV_A appear on MAIN. Later, someone does another ‘ct findmerge’ intending to merge changesfrom MAIN to DEV_B, including the new foo.c merged to MAIN earlier from DEV_A.Which is the real “foo.c” in DEV_B, the one that originated in DEV_A, or the one that originated inDEV_B? It’s not clear. One is identified as the correct file, and the other is the evil twin. One mightexpect them to be branch-relations of the same element, but they’re not related in the ClearCasedatabase. As far as ClearCase is concerned, they’re completely independent elements, referencedin its database by different OIDs (object identifiers).The findmerge completes successfully, and you would have two foo.c’s in the branch, but the operatingsystem permits only one foo.c to appear in the directory. The one that shows in your view is deterministic,but not obvious to most users. ClearCase provides no warning about this problem. Situations are evenworse when there are evil twin directories.This is one of the more insidious complexities of ClearCase. ClearCase admins aware of this potentiallyconfusing scenario sometimes put in “Evil Twin Detection” and “Evil Twin Prevention” triggers. Whenmigrating data from ClearCase, evil twins are murky history that should not (and cannot) be migratedinto Perforce. We detect instances of evil twins, manually select the correct element from the pair,and use ‘ct rmelem’ to eliminate the evil twin.In Perforce, the same filename can be created independently in two branches. However, Perforceenables you to resolve the situation the first time you merge the branches together. The file historiescan be combined, instead of having to kill off an evil twin and lose its history.Symlinks on WindowsThrough use of Multi-Version File System (MVFS), ClearCase supports symlinks on Windows indynamic views. Perforce does not have a custom filesystem and does not support symlinks on platformsthat do not natively support it. If you use symlinks on Windows, you must decide how to handle themwhen planning a migration.Perforce allows symlinks to be versioned. When a file of type ‘symlink’ is brought into a Perforceworkspace on a Windows machine, it appears as a text file containing the path of the target file. Forexample, using a Linux workspace, you issue the ‘ln –s hello.hpp hello.h’ and the file hello.h isa symlink pointing to hellol.hpp. In Perforce, if you sync the hello.h symlink to a Windows workspace,13