WorkShop Teamware: Solutions Guide: - Remote Software Development Using the Configuring Tool

# Remote Software Development Using the Configuring Tool |

Software projects are sometimes developed by groups in which all the developers cannot share a common file system. Developers in these groups may be separated geographically and/or electronically. This separation can further complicate the already complex task of coordinating the work of development groups. Configuring tool is exceptionally well suited to assist groups to coordinate the flow of code developed concurrently under these circumstances.

This topic discusses how a group of developers at a U.S. software company successfully use Configuring to coordinate concurrent compiler development between the United States and Russia.

## Scenario

The U.S. company has contracted with a group of Russian computer scientists in Moscow to assist with compiler development. Most new development is done at the company's U.S. facility, while the Russian group focuses largely on fixing bugs. As a result of this arrangement, both groups are often working simultaneously on the same files.

The compiler development group decided that Configuring provides the means by which they can keep work at both sites synchronized and manage the transfer of files and merging of changes made simultaneously at both sites.

## Implementation

This section describes the process the compiler group uses to coordinate remote development between the U.S. and Russian sites. The first section discusses how the group decided on a remote development strategy and the second section discusses tactics they considered for transferring data between sites.

### The Remote Development Process

After analyzing the situation in great detail, the group came to the following conclusions. These conclusions formed the basis of their final remote development process.

#### Maintain Equivalent Source Bases at Both Sites

In order to coordinate their work the group realized that it was necessary for both sites to contain complete source hierarchies and for the hierarchies to be kept as synchronized as possible. They recognized that it is important to update each site with changes made by the other site as frequently as possible.

#### Use Bridge Workspaces for Data Transfer

In order to make it quicker and easier for developers to access the changes made by the other site, the group decided that each local site treat the other (remote) site as if it were (hierarchically) another local developer. To accomplish this they created bridge workspaces at each site at the same hierarchical level as the other developers. Bridge workspaces are regular Configuring workspaces that are dedicated exclusively to transferring data between the sites. Bridge workspaces are used to:

- Bring over updates from the integration workspace in preparation for shipping them to the other site

- Put back updates to the integration workspace from the other site

The primary goal is to keep both bridge workspaces synchronized.

    Click for closeup.

#### Figure 12 Workspace Hierarchies With Bridge Workspaces

#### Update Each Site Sequentially

For the sake of reliability, the group realized that data must move between the sites sequentially -- only one site is updated at a time. At first glance, it seemed seductively more efficient to have both sites ship their updates to the other site at the same time. Upon further analysis they determined that this method does not ensure that changes are propagated correctly.

To maintain consistency, Configuring enforces a single-writer model -- only one transaction that alters data can proceed in a workspace at a time. Concurrency is achieved by the developers working concurrently in their own workspaces and then updating a central integration workspace sequentially, one at a time. From Configuring's perspective, both Bridge workspaces in this model function together as a single virtual workspace.

If both sites execute bringover transactions to their Bridge workspaces simultaneously, it is equivalent to executing two simultaneous bringover transactions from two different workspaces into a single workspace. This breaks the Configuring single-writer paradigm; under normal (non-remote) circumstances Configuring does not permit this to happen.

    Click for closeup.

#### Figure 13 A Remote Update Cycle

Consider the typical update cycle pictured in Figure 13. At the conclusion of Step and Step the Bridge workspaces at both sites must be identical. The putback transaction in Step (A) is exactly the same as if it were executed directly from the Russian Bridge workspace and the putback transaction in Step (A) is exactly the same as if it were executed directly from the U.S. Bridge workspace -- from a Configuring perspective, both Bridge workspaces function together as a single workspace.

### Transferring File Changes Between Sites

After the group determined the fundamental model they would use, they had to decide how they intended to transfer the updates between the two sites. After analyzing the situation they realized that there is an inherent trade-off -- the less information they transferred between sites, the more additional overhead was required.

After careful evaluation they determined that they had the following three options:

#### Table 0-1 Data Transfer Options

Option | Description
1 | Transfer the entire Bridge workspace between the sites.
2 | Transfer a workspace that contains only the files that have changed (files that are updated, newly created or renamed).
3 | Transfer only: Differences (created using the `diff` command) created in the SCCS history files that have changed The entire SCCS histories of files that were newly created A list of the names of files that had been renamed A workspace "checksum" created using the Freezepointing command that is used to verify that the two Bridge workspaces are identical after each update

The group decided that it was crucial for them to be able to transfer data via email -- this eliminated Option 1. They felt it important to reduce as much as possible the amount of data they send between the sites, even at the added cost of additional administrative work at both sites -- for this reason they chose Option 3.

The following three sections describe the three different options in some detail and present advantages and disadvantages of each.

#### Option 1

Transferring the entire Bridge workspace makes the process virtually the same as if the Russian site's Bridge workspace was actually a child of the U.S. Integration workspace (and vice versa). With this option, except for the transit delay, Configuring is used normally.

    Click for closeup.

#### Figure 14 Transferring the Entire Bridge Workspace (Option 1)

  1. The Russian bridge workspace is updated from the integration workspace (A), copied to tape and shipped to the U.S. site (B).

> Note - As mentioned in the previous section it is crucial that the two Bridge workspaces stay synchronized. After it has been updated with changes from the Integration workspace, the Russian Bridge workspace must not be altered until it is in turn updated with changes from U.S. The Configuring access control facility can be used to lock the workspace against any changes.

  2. The update is received at the U.S. site and the Russian Bridge workspace is extracted.

  3. Using the Configuring reparenting facility, the newly arrived Russian Bridge workspace is reparented to the U.S. Bridge workspace.1

    Note that this step is necessary even if the names of the workspaces are the same at both sites because the fully qualified path names of the parent workspaces are probably different both sites. Configuring stores the names of a workspace's parent and child workspaces as fully qualified path names.

  4. The U.S. bridge workspace is updated with the work of Russian developers using the putback transaction (C).

    At this point the U.S. Bridge workspace is identical to the current Russian Bridge workspace. After the update is complete, the copy of the Russian workspace can be deleted.

  5. The information in the newly updated bridge workspace is putback into the U.S. integration workspace (D).

    The information is now available to the U.S. developers.

  6. Reverse the previous steps to update the Russian site with changes made in the U.S.

#### Option 2

Transfer a workspace that contains only the recent updates to the Bridge workspace.

The advantage of this method over Option 1 is that you can greatly reduce the amount of data you transfer between sites. This reduction in data size might enable you to transfer updates via email, saving time and expense over other methods such as shipping tapes.

The disadvantage of this method over Option 1 is that more human intervention is required. A few additional steps are added to the process and people are required to do more work.

    Click for closeup.

#### Figure 15 Transferring Updated, Created or Renamed Files (Option 2)

  1. A Reference workspace is created as a child of the Russian Bridge workspace (A).

    This workspace is used to preserve the original state of the Bridge workspace.

  2. The Russian Bridge workspace is updated from the integration workspace (B).

  3. The Bringover transaction (with the Preview option selected) is used to compare the Bridge workspace to the Reference workspace.

    This produces a list of files that have been updated, created or renamed.

  4. The files that are listed as updated, created, or renamed are brought over into a newly created Transfer workspace (C).

    The Transfer workspace can be compressed and encoded using standard UNIX utilities such as `compress`, `uuencode`, and `tar`. The resulting file can then be sent via email to the U.S. site.

  5. The Transfer workspace is emailed to the U.S. where it is uncompressed (D).

  6. The Transfer workspace is reparented to the U.S. Bridge workspace.

    At this point the contents of the U.S. Bridge workspace are still identical to the Russian Bridge workspace before its most recent update.

  7. The contents of the Transfer workspace are put back into the Bridge workspace (E).

    At this point the U.S. Bridge workspace is identical to the current Russian Bridge workspace. The Transfer workspace can be deleted after the update is complete.

  8. The contents of the U.S. Bridge workspace are put back to the integration workspace (F).

#### Option 3

It is possible to reduce even further the amount of data transferred between sites. There are however, costs for this reduction:

- The process is significantly more complicated.
- More human intervention is required.
- The potential for data corruption increases.

    Click for closeup.

#### Figure 16 Transferring Only `diff`s of Changed SCCS History Files (Option 3)

  1. A Reference workspace is created as a child of the Russian Bridge workspace (A).

    This workspace is used to preserve the original state of the Bridge workspace.

  2. The Russian Bridge workspace is updated from the integration workspace (B).

  3. The Bringover transaction (with the Preview option selected) is used to compare the Bridge workspace to the Reference workspace.

    This produces a list of files that have been updated, created, or renamed.

  4. The TeamWare Freezepointing utility is used to create a "checksum" from the Russian Bridge workspace.

    This checksum is used at the U.S. site to confirm that the U.S. and Russian Bridge workspaces are identical after they are updated.

  5. The `diff` utility is used to determine the differences between SCCS history files in the Bridge workspace and the Reference workspace.

    The SCCS history file of each file that has been updated is compared to its previous state in the Reference workspace. The output of the `diff` utility is applied (using the `patch` utility) to the same files in the U.S. Bridge workspace to recreate the current state of the Russian Bridge workspace.

  6. The data required to make the U.S. Bridge workspace identical to the Russian Bridge workspace is packaged into an email file (compressed, encoded, etc.) (C) and mailed to the U.S. site (D).

    The information required for this process includes:

  - The `diff` results for all updated files

  - The entire SCCS history files of any newly created files

  - A list of all files that have been renamed

  - The workspace checksum created in Step 4

  7. The U.S. Bridge workspace is updated using the information sent from Russia (E).

  - As required, files are renamed using the `mv` command so that they correctly match the new names in the Russian Bridge workspace.

  - The `patch` utility is used to apply the `diff` output to each updated file.

  - History files of newly created files are copied into the appropriate directories.

  8. The updated U.S. Bridge workspace is verified to be identical to the Russian Bridge workspace.

    Freezepointing is used to create a freezepoint file from the U.S. Bridge workspace. The U.S. freezepoint file is compared to the Russian freezepoint file; if they match, then the update was performed correctly.

  9. The contents of the U.S. Bridge workspace are put back to the integration workspace (F).

### Final Notes

- As in the case of most Configuring use, frequent, disciplined updating has been found to be helpful. The more out of sync the two sites become, the more conflicts must be merged. This group has found that by updating frequently, conflict problems can be resolved with occasional email communication.

- Remember that it is crucial that the Bridge workspaces at both sites stay synchronized.

- After a Bridge workspace is updated with changes from the integration workspace, that Bridge workspace must not be altered until it is in turn updated with changes from the other site.

- After being updated from the other site, the Bridge workspaces at both sites must be identical.

- You can use Configuring's Access Control facility to ensure that the contents of the Bridge workspaces are not inadvertently changed between update cycles.

- You can used Configuring's Notification facility to automatically inform another site by email when changes of interest are made to the source base. This can alert developers to expect changes in areas in which they are working.

- Configuring does not use time information to determine whether files have changed. This means that time zone changes and slight discrepancies between system clocks do not effect Configuring's ability to determine whether files have changed.

1 You can save a step in the process by simply replacing the U.S. bridge workspace with the Russian bridge workspace, and then reparenting the Russian bridge workspace directly to the integration workspace. However, by circumventing the putback transaction, Configuring cannot protect you from losing any data that may have been inadvertently copied to the U.S. bridge workspace. In addition, the Configuring history facility will not record the transactions for later inspection.