AUG 2 Aprobe Basics

From OC Systems Wiki!
Revision as of 18:17, 4 May 2017 by Swn (talk | contribs) (Created page with "<div><div> [aprobe-6.html [Next]] [aprobe-4.html [Previous]] [aprobe-1.html [Top]] [aprobe-3.html [Contents]] [aprobe-13.html [Index]] Aprobe </div> = Aprobe Basics = ---...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

[aprobe-6.html [Next]] [aprobe-4.html [Previous]] [aprobe-1.html [Top]] [aprobe-3.html [Contents]] [aprobe-13.html [Index]]

Aprobe

Aprobe Basics


Overview of Aprobe

This section gives a quick overview of Aprobe. We suggest that you read this section and then try the on-line examples in [../examples/evaluate/README $APROBE/examples/evaluate] ([../ada_examples/evaluate/README or in $APROBE/ada_examples/evaluate] if you're using Ada) to get a more complete overview of what Aprobe is and what it can do.

Aprobe is a very sophisticated patching tool. It applies the patches (we call them probes) into the memory image of the executing program as it starts execution.

Probes are like parameterless subprograms (that is, functions or procedures) that are written in the ANSI C programming language. Calls to the probes are patched into the user-specified locations of the memory image of your executable program. Probes typically collect and log some data about the program's environment for later analysis, although, since the full C language can be used, probes can do just about anything including altering the execution of the program. Rather than being linked into a program at compilation time or link time, both the probes and the calls to them are patched into the executable program when the program is loaded for execution by the aprobe command.

The Aprobe product includes an Application Programming Interface (API), documented in the header file $APROBE/include/aprobe.h, which provides support to your probes for many activities, such as patching and collecting data from your application.

Aprobe also includes a preprocessor that will automatically generate the correct C code and the necessary calls to the [aprobe-6.html#MARKER-9-624 Aprobe API] from your probes. The preprocessor is called [aprobe-9.html#MARKER-9-987 apc], and we refer to files containing preprocessor directives as APC files. The examples in this book generally use these preprocessor directives, since they make using the API very simple. See [aprobe-6.html#MARKER-9-479 "The Aprobe Preprocessor Directives"].

Some Definitions

There are a number of terms that we use when discussing Aprobe. Some of them may be familiar to you, and some we intend to have a specific meaning. The following few paragraphs describe these terms, which are shown in boldface.

The terms application, target application, target program, user program, executable and target executable are used interchangeably to mean a program you'll investigate using Aprobe.

For our purposes, an application may be either Java or native (or a combination of both). A Java application consists primarily of a Java classes loaded by an external Java Virtual Machine (JVM). By contrast, a native application consists by compiled object files linked by you (or a third party) which will run only on a specific processor and operating system. Aprobe does support Java, as described below, but is primarily targeted to native applications.

A native application usually consists of an executable object file linked by you (or a third party) and one or more shared libraries (e.g., libc.so) which are loaded by the target operating system when the executable is invoked and which executes on the target machine. Variables in the target program are referred to as target variables, registers as target registers, etc.

The target executable and shared libraries are all load modules (or object modules, or just modules). They consist of a number of functions (or subprograms or routines, or methods -- the term varies with the language in which the target program is written), and these functions call one another.

As the target program executes, it may consist of one or more parallel threads of execution which share memory within a single process.

Aprobe is a tool which allows you to instrument or patch (that is, modify the machine instructions of) any application with probes. These probes may interact with the target program at the start and end of the program (program probes), the creation and termination of threads (thread probes) and anywhere in functions (function probes) to do just about anything.

One of the things your probe can do, for which Aprobe has special support, is logging data which means recording raw data at runtime (while the program is running). This logged data is then formatted--processed, filtered, and generally made more readable -- after the program completes.

Each probe is described in ANSI C (the standard, modern definition of the C language, but not C ), augmented with Aprobe preprocessor directives. The C code with these directives is called APC. The APC is translated to C by the apc preprocessor, which then invokes the target C compiler already present on your system. The C code generated by the preprocessor contains calls to the Aprobe API -- the Application Programming Interface to the Aprobe runtime, which is contained in a shared library. Special syntax in the APC allows the C code in a probe to distinguish between target variables and probe variables, which are those identifiers defined within the APC itself.

The Tools

There are three primary tools involved in the Aprobe process described above: [aprobe-9.html#MARKER-9-987 apc], [aprobe-9.html#MARKER-9-1075 aprobe], and [aprobe-9.html#MARKER-9-1035 apformat]. The [aprobe-9.html#MARKER-9-987 apc] command compiles the source code of your probes, written in ANSI C and containing Aprobe preprocessor directives, to produce a User Action Library (UAL). The [aprobe-9.html#MARKER-9-1075 aprobe] command loads the executable into memory, applies the patches from the specified UALs and then runs the patched executable. The [aprobe-9.html#MARKER-9-1035 apformat] command reads the data logged by the UALs and formats it into a human-readable form.

Java

Java probes use this same process, aided by the [aprobe-9.html#MARKER-9-1061 apjava] program which applies aprobe to the java executable itself, and a special "xmj" file which specifies what Java classes contain the Java probes. See [aprobe-8.html#MARKER-9-910 Chapter 5, "Writing Java Probes"] for all Java-specific information.

Ada

At the low level, Aprobe applies to Ada programs just as to C and C programs. However, Ada requires special handling of variable names, scoping and exceptions, for which Aprobe provides explicit syntax and API calls. Ada support is closely tied to the compiler with which the program was built. At this writing, Aprobe provides support for the GNAT Ada compiler on Solaris, and the PowerAda Ada compiler on Linux and AIX.

A Simple Example

This example illustrates the capabilities of Aprobe. The example contains a small C application program, and an APC file, along with the commands and output produced by running the program with and without Aprobe. The source code for this example is found in $APROBE/examples/evaluate/2.fib.

There are a number of other illustrative examples delivered with Aprobe. You'll find them under the examples directory ($APROBE/examples and $APROBE/ada_examples). We suggest that, after reading this example, you try running those examples. After running them, you will have a very good idea of what Aprobe can do, so you can start applying it to your own software.

Aprobe can investigate most programs, even those without source code. On the next page is a short C language program called fib.c. This program calculates the number of Fibonocci numbers specified on the command line and prints them on a single line. Its fib routine is highly recursive. Note that Aprobe works on any C, C , or Ada code built with any supported compiler; there is really nothing special about this code.

 #include <stdlib.h>
 #include <stdio.h>
 int NumIterations;
 
 int fib( int n )
 {
   if ( n < 2 )
     return( 1 );
   else
     return( fib( n - 1 )   fib( n - 2 ) );
 }
 
 int main( int argc, char **argv )
 {
   int i;
 
   NumIterations = 0;
   if (argc > 1)
   {
     NumIterations = atoi( argv[1] );
     for( i = 0; i < NumIterations; i   )
       printf( "%d ", fib ( i ) );
   }
   else
   {
     printf("Specify a positive integer argument.\n");
   }
 
   printf( "\n" );
 
   return( 0 );
 }
 

Example 2-1. fib.c

We want to set two probes on our fib program, so we write the APC file fib.apc file shown on the facing page.


probe thread
{
  int NumFibCalls = 0;

  probe "fib"
  {
    on_entry
    {
      NumFibCalls  ;
    }
  }

  probe "main"
  {
    on_exit
    {
      log("For NumIterations = ", $NumIterations);
      log("the number of calls to fib = ",
         NumFibCalls);
    }
  }
}

Example 2-2. fib.apc

This APC file probes both the main function and the fib function. (Don't worry about the "probe thread" for now -- it's always there.) This example counts the number of times the fib function is called. Also, main's input parameter and fib's execution count are logged upon exit from the main. Note that the APC preprocessor, [aprobe-9.html#MARKER-9-987 apc], automatically figures out the type of NumFibCalls (a variable in the probe) and NumIterations (a variable in the program being probed), and it logs the right amount of data and how to format that data. Aprobe can do this with more than just integers and simple types; if NumIterations were a struct, Aprobe would log the entire structure and format each of the fields correctly! The $ before an identifier such as NumIterations indicates a reference to the target program's name space, as distinct from variables in the current APC file. These can be assigned to one another, and mixed within any expression.

This APC file can contain arbitrary C code. For example, instead of logging the data, we could print it to standard output. However, logging (to a file) is generally much more useful.

Now we will see how to compile and execute the probed program:

First compile the C code, saving debugging information. This will produce an executable called fib.

cc -g -o fib fib.c

Try running fib on its own. It will print out some numbers.

fib 10
1 1 2 3 5 8 13 21 34 55

Compile the APC code. This will produce a UAL (User Action Library) that can be applied to the fib executable.

apc fib.apc -x fib -o fib.ual

Run fib with the UAL we just produced. The aprobe command will load fib, apply the (probes from the) UAL, and then run the program.

aprobe -u fib.ual fib 10
1 1 2 3 5 8 13 21 34 55

Notice that fib's behavior is unchanged. Let's see the data logged by our probe:

apformat fib For input parameter = 10
the number of calls to fib = 276

Notice that the original fib program remains unaltered. We could also probe fib with a different UAL (or multiple UALs) to gather totally different information.

The vast majority of your probes' text (APC) will consist of C code, not Aprobe directives. The examples that are shown in this manual, however, will not have much C code since our purpose is to teach Aprobe rather than to teach C.

Using Aprobe

As the previous example shows, the process of using Aprobe normally consists of these four steps:

  1. [#MARKER-9-358 Write Probe in APC],
  2. [#MARKER-9-361 Compile Probe into UAL],
  3. [#MARKER-9-371 Run Program Probe],
  4. [#MARKER-9-381 Format Logged Data], if any.

Write Probe in APC

  1. Write your probe(s) in the ANSI C programming language, using your favorite editor, and add the Aprobe specific directives (e.g. on_exit) where needed to indicate when your probes should be executed. You may put as many probes as you wish into a single probe source file. The source file is called an APC file. In the example above, the APC file is the fib.apc file.

The Aprobe preprocessor directives within your APC source file allow you to reference the various entities declared inside the executable application program (e.g., NumIterations above) using the actual names that were used in the executable's own source program, provided that the program was built for debugging (typically compiled with the -g flag). These entities may include any variables, functions, expressions, objects and overloaded names declared in the executable.

Compile Probe into UAL

  1. Use the [aprobe-9.html#MARKER-9-987 apc] command, which is the Aprobe compiler, to preprocess and compile the APC files. The Aprobe compiler automatically expands any preprocessor directives in your APC file by using the debug information that can usually be found in the executable to be probed. After preprocessing, it automatically invokes the target C compiler and produces a single object file called the User Action Library (UAL) as its output.

In the example above, we compiled fib.apc with the following line:

apc fib.apc -x fib -o fib.ual

This apc command compiled the APC input file fib.apc and used the executable file fib (to obtain debug information) and produced the output file fib.ual. The Aprobe compiler (apc) preprocesses the APC file to replace its directives with corresponding C constructs and with calls to the Aprobe runtime. The preprocessed file is then passed to the target C compiler to be compiled. The resulting object code is then linked with the Aprobe runtime and any other object code that you specified, and placed in the specified object file which is called a User Action Library (UAL). You can have as many probes as you like, even for the same executable. You can also include other object files that were not necessarily compiled by the Aprobe compiler. Multiple probes of the same location are additive; one does not replace another.

Run Program Probe

  1. Use the [aprobe-9.html#MARKER-9-1075 aprobe] command to run your executable program along with the probe(s) you just compiled. In the above example we ran fib together with the compiled probes in fib.ual with the command:

aprobe -u fib.ual fib 10 This loaded fib into memory, patched the probes from fib.ual into the in-memory image of fib, and then executed the newly probed image of fib. Notice that the parameter 10 was passed to fib. In this example, we applied one UAL, but Aprobe can apply many UALs at once, not just one. If your probes use the logging features of Aprobe (as opposed to just printing data to the standard output), the data is logged to an [aprobe-10.html#MARKER-9-1325 APD file] (or files). The fib example did log data, so while it was running with aprobe, the APD files named fib.apd and fib-0.apd were created.

Format Logged Data

  1. The [aprobe-9.html#MARKER-9-1035 apformat] command performs the formatting of any data that was logged into the APD files. Logging only happens while probed executable is still running, but formatting is typically done after the executable is no longer running. Apformat needs the same target executable and UALs that collected the logged data, so that it can properly format the data.

In the above example we formatted the APD files with the command: apformat fib This read and automatically formatted the data from the fib.apd and fib-0.apd files, and it displayed the results. It also used information in the original fib executable and the fib.ual to do so. Formatting is governed by the log statments in your UAL's probes. You can eliminate the apformat step by specifying -if on the [aprobe-9.html#MARKER-9-1075 aprobe] command-line, which causes data to be immediately formatted at the point of the log statement (while the probed executable is still running).

Predefined Probes

The directory $APROBE/probes contains APC of some predefined probes. Among these are: coverage (for test coverage), memwatch (for heap memory usage), profile and statprof (for program execution time profiling), and trace (for execution tracing). These probes were already compiled into UALs, and are ready to use simply by naming them in an aprobe command, like this:


  aprobe -u trace.ual -p -g fib 10

(where -p indicates that -g is a parameter to the trace.ual probe, not to aprobe itself, as described in [aprobe-9.html#MARKER-9-1075 Appendix A, "aprobe"]).

These probes are summarized below, and fully described in [aprobe-12.html#MARKER-9-1899 "Predefined Probes and Libraries Reference" in Appendix D].

coverage.ual

records the execution of each known source line in the specified functions of the executable program and produces a report indicating which lines in each function were not executed. See [aprobe-12.html#MARKER-9-1978 "Test Coverage Probe: coverage.ual"].

events.ual

provides flexible and powerful event-based performance analysis by measuring time between any two user-defined events, and reporting the results in a tablular format. See [aprobe-12.html#MARKER-9-2031 "Performance Probe: events.ual"].

profile.ual

monitors the selected function calls in the executable program and displays execution time statistics about those functions to help indicate where and when the program spends its CPU time. See [aprobe-12.html#MARKER-9-2058 "Profile Probe: profile.ual"].

trace.ual

traces the execution of the selected functions within an application program to record when, and in which order, and by which thread, each function was invoked. See [aprobe-12.html#MARKER-9-2117 "Trace Probe: trace.ual"]. The trace.ual predefined probe is used in the on-line example [../examples/evaluate/4.predefined/README $APROBE/examples/evaluate/4.predefined].

memwatch.ual

detects heap memory leaks within an application program by monitoring the allocations and deallocations of heap memory. See [aprobe-12.html#MARKER-9-2173 "Memory Watch Probe: memwatch.ual"].

info.ual

prints information about the executable program and/or an APD file. It can be used with either [aprobe-9.html#MARKER-9-1075 aprobe] or [aprobe-9.html#MARKER-9-1035 apformat] to display the symbol names within either the executable and its shared libraries or the UALs and threads recorded in an APD file. This probe is used to implement the [aprobe-9.html#MARKER-9-1052 apinfo] and [aprobe-9.html#MARKER-9-1110 apsymbols] commands. See [aprobe-12.html#MARKER-9-2205 "Info Probe: info.ual"].

'Other probes
'
The directory $APROBE/ual_lib contains a number of other probes, and more are available from OC Systems. Contact support@ocsystems.com.
'quick_gui.ual'

a library of functions useful when writing your own probes. It provides a simpler way to create a Java GUI. The interface is defined in [../include/quick_gui.h $APROBE/include/quick_gui.h], and it contains histogram and XY-plot graph support, as well pop-up Yes/No and message dialogs. Use of this library is illustrated by the $APROBE/examples/learn/visualize_data example. See [aprobe-12.html#MARKER-9-2220 "quick_gui Library"].

Built-in Operations

As shown in the [#MARKER-9-344 Example 2-2. "fib.apc"], probes are easy to create and use. Probes are easy to write because the Aprobe compiler's APC preprocessor recognizes some special keywords, such as probe and log, to simplify and add power to your probes. Probes can be as simple, or as complex, as you like. If you want to get fancy, Aprobe has a library of built-in operations to help you query or act upon the state of the program's execution or of your probes. The interface to this library is the Aprobe API, defined by the C header file $APROBE/include/aprobe.h which is documented in [aprobe-11.html#MARKER-9-1351 Appendix C, "Aprobe API Reference"]. Portions of the API are demonstrated in some of the on-line examples, and some of the more useful operations are described in [aprobe-6.html#MARKER-9-624 "The Aprobe API" in Chapter 3], including:

  • [aprobe-6.html#MARKER-9-630 "Stub Support"],
  • [aprobe-6.html#MARKER-9-636 "Stack Trace Support"],
  • [aprobe-6.html#MARKER-9-642 "Exception Support"].
  • [aprobe-6.html#MARKER-9-661 "Time Support"],
  • [aprobe-6.html#MARKER-9-673 "Periodic Action Support"], and
  • [aprobe-6.html#MARKER-9-680 "Tracing Support"].

Summary

  1. Aprobe allows you to see, and alter, what's going on in a program.
  2. Aprobe does not require you to have, change, nor recompile the original source of your program.
  3. The executable program remains totally unchanged.
  4. Aprobe patches your probes into selected places within your program. You can do almost anything from a probe including:
    1. Logging data collected from the running program, in a minimally invasive manner.
    2. Altering data in the running program.
    3. Catching (handling) exceptions occurring in the program.
    4. Simulating various kinds of errors from which your program might have to recover.
  5. There are three main tools used in the Aprobe process:
    1. [aprobe-9.html#MARKER-9-987 apc] - This command compiles the source code of your probes into a User Action Library (UAL) file. Probes are written in ANSI C and containing Aprobe preprocessor directives.
    2. [aprobe-9.html#MARKER-9-1075 aprobe] - This command loads the executable program into memory, applies the patches from the specified UALs, and then runs the patched executable.
    3. [aprobe-9.html#MARKER-9-1035 apformat] - This command formats the data logged by the UALs into a human-readable form.
  6. Aprobe supports probing Java as well as native object-code programs.

[aprobe-6.html [Next]] [aprobe-4.html [Previous]] [aprobe-1.html [Top]] [aprobe-3.html [Contents]] [aprobe-13.html [Index]]

Copyright 2006-2017 OC Systems, Inc.