RCUG 6 Deploying The RootCause Workspace

From OC Systems Wiki!
Revision as of 15:02, 20 June 2018 by Ern (talk | contribs)
Jump to: navigation, search

Next Previous Index Top

RootCause User Guide

rcc-9

Deploying the RootCause Workspace


RootCause performs root cause analysis of problems at the user's site, in the production system. This chapter discusses how to run traces on a remote computer.

Installing The RootCause Agent

RootCause has two components: the RootCause Console and the RootCause Agent. The Agent is the subset of RootCause that is used to run RootCause traces on a remote computer. The Console is the full product that allows one to define and view traces as well as run them (that is, the Console also includes the Agent).

If you're just "trying out" the deploy process on a single computer, your Console installation can also serve as the remote installation. If you really are deploying RootCause to a remote site you will want to install just the RootCause Agent subset there, and enable remote execution using RootCause agent licenses.

Follow the installation directions in "RootCause Agent Installation" to install the RootCause Agent on the remote computer; this only needs to be done once per remote computer.

After this is installed on the remote computer, you deploy RootCause trace definitions to the remote RootCause Agent and get back files that contain the logged trace data.

The Agent installation does not have its own license--the license is delivered with the deployed probes. Instead, you will obtain one or more agent license keys from OC Systems and append them to the file

$APROBE/licenses/agent_license.dat

in your RootCause Console installation. Agent licenses will be automatically copied into the deployed workspace (see Deploying A RootCause Workspace"). See "Licensing" for a more in-depth discussion on licensing issues.

Building a "Traceable" Application

Production systems often have their applications stripped of debug and symbol information to save space and increase security. The program runs fine in the field without this information. However, meaningful tracing information cannot be obtained without this debug and symbol information.

RootCause solves this problem by allowing one to define the trace on a version of the application that contains debug and symbol information and then to run that trace against a version of the same application from which this debug information has been removed by the "strip" command. RootCause will "remember" the necessary debug and symbol information for the stripped application.

The presence or absence of debug and symbol information should be the only difference between the development and production versions. You should define your traces based on the exact same object code on which the traces will be run. This is a requirement if you wish to trace source lines or access local data values. So, to be assured of reliable traces regardless of what you select, you must develop your traces against the program you will be shipping, and then strip (using a "strip" command) the executable files prior to distribution. RootCause has been designed so that you can define your traces on the application in which debug information is available, and then run that application under RootCause either with or without that information present in the execution environment.

When you perform the RootCause deploy operation (see "Deploying A RootCause Workspace") to create the packaged workspace for use in the production environment, you check the module(s) that will be stripped in the remote environment. RootCause collects the necessary information for the trace from the local non-stripped module and writes it to an ADI file. The ADI file is then automatically and invisibly included in the deployed RootCause workspace. The deployed workspace now contains sufficient symbolic information to allow RootCause to run on the stripped application in the production environment.

RootCause ADI files carry the symbolic information necessary to perform RootCause traces, so that information may be stripped from the executables themselves. But there remains the issue of how to develop useful traces on an application built for delivery: must you change your build process in order to accommodate RootCause? In some cases, yes.

If you simply wish to trace the functions executed by your program, you do not need debug information; all you need is symbols, which are generally present until you perform a strip command. So, for example, the initial tracing of the "pi_demo" program from Chapter 5, "RootCause Demo" would work fine even if the program were compiled with optimize and with no debug information.

If you wish to select parameters and data values as part of your trace, the executable or shared module in which the log operation is taking place must be compiled with a debug (-g) flag. If you usually build your application with optimize (-O) and other flags that affect the generated code (e.g., -DNDEBUG) then you must do this on the version to be traced.

Note that specifying -g with -O may reduce the effect of optimization, but this compromise is necessary in order to have access to accurate source-level information with which to develop your traces. However, if you cannot use -g on your delivered programs, you can still develop traces by restricting the information you want to log. Also, note that only those files that you desire to trace need be compiled with -g.

If you simply wish to log parameter values on function entry, you can build the probes to do this against a debug version of your program and they will probably work against an optimized, non-debug version of the program. However, inlining of function calls may eliminate some or all points at which the probes would be executed.

Regardless of how you build your application, you can generally get to data within it by generating custom probes that don't require debug information.

Please contact support@ocsystems.com with any questions you have. We feel confident you can get the trace you want without greatly altering your development practices or sacrificing performance in your production environment.

Deploying A RootCause Workspace

When you have built and tested the traces and probes, and you want to apply them to an application that exists at (or will be shipped to) a remote site, you are ready to deploy the workspace containing those traces and probes. This workspace will be your "flight recorder" for the application at the remote site. To generate the deploy (.dply) file:

  1. Confirm that you're developing and testing with the same build of the application you'll be shipping. See "Building a "Traceable" Application".
  2. Develop and test your traces locally. When you see the information you think you'll need at the remote site, then you're ready to deploy.
  3. Enable the verify predefined UAL in the Workspace Browser window. This checks the correspondence between the program and modules on the remote system with those in the local (formatting) environment, and alerts the user when an incompatibility is detected.
  4. Click on the Deploy button in the main Workspace Browser window. This will display the Deploy Dialog.
  5. In the Deploy Dialog, enter a file name for your deploy file. This file will be created by RootCause, and it will contain the trace definitions for your current workspace.
  6. Click the ADI Files tab in the dialog. In the ADI files tab, you may specify which, if any, of the modules for which you have traces and probes have been stripped of symbol information. RootCause will generate an ADI file that contains sufficient symbolic information for the traces and probes to work with the stripped executable at the remote site. Do not generate ADI for any of the system-provided modules and shared libraries; only modules and libraries that you have explicitly stripped. If you have not explicitly stripped an executable or library, then you do not need to generate any ADI files for them.
  7. In the Deploy window, click "OK". RootCause will attempt to create the dply file. For example, if you're using the pi_demo.aws workspace created in Chapter 5, "RootCause Demo", then this will create pi_demo.dply.

    The .dply file will also contain the license needed to run RootCause on the remote computer. When the .dply file is created, the file $APROBE/licenses/agent_license.dat mentioned in see "Installing The RootCause Agent" is automatically included. If the license file is not found, or does not contain a license, an error dialog will appear. If you see this, you can click "Yes" to proceed with deploying, or "No" if you wish to investigate getting a valid Agent license.

  8. Transfer the .dplyfile over to the remote computer (be sure to specify binary mode if you use ftp).

Registering a Deployed Workspace

  1. On the remote computer open an xterm or other command shell.
  2. Register the application on the remote computer using the rootcause register command. For example:

    rootcause register pi_demo.dply

    This creates a workspace in the current directory (or a path specified with the -w option) and registers the workspace with the application at the same path as on the Console side. If the application is at a different path, the new (local) path must be specified with the -x option before the deploy file name. Similarly, each dynamic module that is at a differen tlocation from where it was when deployed must be specified with a -m option. For example:

    rootcause register
    -x /opt/demos/bin/pi_demo.exe
    -m /opt/demos/lib/libpi.so
    pi_demo.dply

  3. Enable RootCause on the remote computer in the environment where the application will be run, using the rootcause_on command. Note that this needs to be done in a parent process of the shell that will run the application(s) so that the application(s) will inherit the value of the environment variables it sets.

Collecting Data At The Remote Site

  1. Run the application(s) as you normally would.
  2. When you want to collect and examine the RootCause trace data, execute the rootcause collect command on the remote computer. For example: rootcause collect -x $APROBE/demo/RootCause/C++/pi_demo
    Many executables may be collected at once by listing them on the collect command line. The applicable files in each registered workspace will be compressed into a single .clct file.

Formatting and Viewing the Remotely-Collected Data

  1. Transfer the .clct file back to the local computer where the RootCause Console is installed (be sure to specify binary mode if you use ftp).
  2. In the RootCause GUI, click on the Decollect button in the main window. This will display the Decollect Data Dialog. If the RootCause GUI is not currently running, you may open the collect file when starting the GUI, for example: rootcause open file.clct
  3. In the Decollect Data Dialog, enter the name of the .clct file and the destination directory into which the file will be expanded. The destination directory should be empty as the decollect operation will expand a number of files into this directory. Then click "OK".

    This will create a decollection, a directory, with suffix .dclct, containing the workspaces collected from the remote computer. It will then proceed with the Open Decollection operation, which will show a Trace Index Dialog for the newest data in the decollection.

  4. Select the event(s) you wish to view, or use Select Data Files to change the data that was indexed.
  5. Format the data into a Trace Display. If you used the verify predefined UAL as suggested in "Deploying A RootCause Workspace", you'll see a TEXT node identifying any mismatches that may cause formatting problems.
  6. From the Trace Display for the decollected data, you can use Add Data Files To Display to add in the Decollected RootCause Log file and other data.



Copyright 2006-2017 OC Systems, Inc.

Next Previous Index Top