Difference between revisions of "Demand Predefined Probe"

From OC Systems Wiki!
Jump to: navigation, search
m (The Demand Framework)
m (The Demand Framework)
Line 15: Line 15:
 
See [[AUG_Tools_Reference#apdemand|apdemand]] for an overview of <code>apdemand</code>.
 
See [[AUG_Tools_Reference#apdemand|apdemand]] for an overview of <code>apdemand</code>.
  
The <code>demand<code> UAL needs to be included in Aprobe runs by <code>-u demand</code>,
+
The <code>demand</code> UAL needs to be included in Aprobe runs by <code>-u demand</code>,
 
because it is a pre-defined UAL whose logic you can ignore.
 
because it is a pre-defined UAL whose logic you can ignore.
  

Revision as of 21:15, 1 June 2018

Demand Action: demand.ual

(Since 4.4.4a)

The predefined probe demand.ual is part of a framework for "demanding" action from another probe from the command-line at nun-time, independent of what the probed program might be doing. This is an advanced feature, but can be powerful in the right circumstances.

The demand framework is integrated into a number of predefined probes (coverage, memstat, memwatch, profile, statprof) and ready to use. The demand framework can be added to custom probes as well.

The Demand Framework

The demand framework consists of the demand.h include file, the demand.ual probe, and the apdemand command-line script.

The apdemand command is your way of requesting action. See apdemand for an overview of apdemand.

The demand UAL needs to be included in Aprobe runs by -u demand, because it is a pre-defined UAL whose logic you can ignore.

The ap_OfferDemand() macro is invoked in the pre-defined UAL apc files. You only need to look at the macro to write custom APC which offers your own actions on-demand. The macro is in $APROBE/include/demand.h, and an example of its use is at the end of $APROBE/probes/profile.apc.

Demand Framework Example

A probe can be enabled for the demand framework by including the demand.h file and then instantiating the ap_OfferDemand() macro to define actions which can be demanded from the command line. The ap_OfferDemand() macro defines the action name and a callback to perform the action.

Here is an example of that:

 #include "demand.h"
 
 ...

 void MyActionCallback(FILE *spool)
 {
    // write to the demand spool file to provide status 
    fprintf(spool "starting my action...\n");

    // performs the action
    DoSnapshot();

    // write to the demand spool file to provide status
    fpritnf(spool, "dons my action\n");
 }

 ap_OffserDemand(MyActionCallback, "myprobe action1");
 

To use the demand framework with an enabled probe you will add demand.ual to the Approve command line:

<mowiki>
aprobe -u my probe -u demand  MyApp.exe
</nowiki>

To send action commands to the probe use apdemand form the command line:

 apdemand action1
 

Actions On-Demand From Predefined UALs

Several predefined UALs offer snapshots and other actions on user demand:

  • coverage.ual
  • memwatch.ual
  • memstat.ual
  • profile.ual
  • statprof.ual

Demand snapshots are logged like snapshots you can trigger by naming trigger functions in the config files, but demand needs no internal triggers. Instead, the trigger is a command you can enter from any shell using apdemand. This may be useful for long-running applications or applications that don't terminate and for which there is no good snapshot location or if you don't know where to place triggers.


Remote Control is an advanced use of demand.ual to affect a remote realtime application while it runs, with Aprobe and uals, of course. The demand ual is an Aprobe predefined probe which you just include in the ual list. To go with it, you design probes which test switches you plan to flip while the application runs. Here is model apc:

To affect optional probe output, such as logging, printing, and writing to files, you put the output code inside 'if (OptionalOutputWanted) { ... }', and you declare ap_BooleanT OptionalOutputWanted = TRUE or FALSE for the normal state. You code a callback function

 void FlipOptionalOutput(FILE *spool) {
    OptionalOutputWanted = ! OptionalOutputWanted;
    fprintf(spool, "OptionalOutputWanted is %d\n", OptionalOutputWanted);
 }

and register your callback function with a macro for the apdemand script

 ap_OfferDemand(FlipOptionalOutput, "invert OptionalOutput")

then trigger callbacks while the application runs. This topic is about Remote Control, so you plan to trigger callbacks remotely. Aprobe's apdemand script supports your plan. From a separate, usually remote command shell, entering "apdemand ..." dynamically updates a "demand.cfg" file. It is created by the ap_OfferDemand macros. It is periodically read

by the demand.ual probe within the running application. When it sees an update, it triggers your callbacks. The apdemand script also manages an output "spool" file that keeps you informed about what is happening and what actions are available. Continuing our example then, entering

 apdemand nal

triggers callbacks which contain "nal" in the registration string. You can have many switches and callback functions, each registered with a meaningful phrase in quotes. You can trigger one at a time, or several at once, by choosing a unique string fragment, or a string fragment which occurs in several registrations. Planning ahead is useful.

This model can also be used to affect application output, such as sending messages, printing, and writing to files, when the application already tests switches. Your apc coding differences are minor, using $ in your probe references to an application boolean instance.

A callback fprintf to spool is recommended. FILE *spool is passed to all callback functions. The file already has a spool header which identifies the callback by its registration string. The spool file is closed with a trailer when the callback function returns. Then apdemand sees the closing trailer and pages the spool file.

To set a number, such as a multiplier or a limit, instead of a boolean, you echo the value to a work file before triggering your callback function by apdemand. Your callback function opens the work file and reads the input with fscanf. The input value can replace or bump the controlled number. You can also read a string from a work file, being careful about the size of the area into which you put the string. A string pointer is safer to modify, with the input string copied to a malloc area. You can write scripts which echo and invoke apdemand. Callbacks can also set numbers and strings in the application by echo plus demand, using $ before an application name.

In summary, remote controls are your design: boolean, numeric, and string data are your choices, you register phrases which apdemand searches for a callback function to trigger, and you write confirming feedback from your callback function. You can write spool output from callback functions which don't change anything, and thereby design remote queries of collected probe data, or application state data. Using probes in this way, you do not interface manually with the application process. Instead, your manual interface is with files in a remote working directory. Your interface to change a file can be ssh commands to the application's server, or commands in a remote window on the server, or commands in any window onto shared disk, whatever works for you.




Usage

Using statprof is straightforward - there is no configuration file or GUI to worry about. All options can be set by command line options (either at runtime or format-time).

Statprof UAL parameters

statprof.ual is specified on the aprobe command line or in an APO file as described in Command Line. The specific options are:


aprobe  -u statprof.ual [-p "[-a] [-c] [-h] [-i interval] [-l] [-o filename] [-r] [-s size] [-S signum] [-z] 
    [module name]]" your_program

where:

-a
This performs a profile on the whole address space. This uses the smallest cell size possible, though it will generally be rather coarse. Be careful to set the APD file size large enough to hold all the data.
-c
This performs a coarse profile to see overall usage in shared libraries. This uses a very large "cell" size which allows you to determine which shared library was active when the kernel tick occurred but not which routine.
-i
This sets the profile timer interval. The default value is 10 milliseconds. Valid values range from 10 milliseconds to 999 milliseconds. (Since 4.4.9).
-l
This displays a breakdown of time within a function by line number as well as by function. The bigger the cell size the less reliable it will be but this can give useful information. This is a format-time option.
-o filename
This outputs comma-delimited data to the given filename. This is suitable for input into a spreadsheet or similar program. Note that the line breaks are PC-based rather than Unix-based as this seems to be more generally accepted by programs we've tested with.
-r
This will reset profile counts for each new snapshot. Without this option profile information will be cumulative for the start of the test.
-s
This sets the number of bytes used for each sampling cell. It's unlikely that you would modify this but if you have a huge application and limited memory you may wish to do so. Basically the larger the cell size, the smaller the amount of storage required to hold the sampling table but it's more likely that the output data will be incorrect. By default two instructions are mapped to a cell which has nearly a 100% chance of both instructions belonging to the same routine due to the padding usually used by compilers. But if there are four instructions for the same cell, there's a chance that some of the instructions are for one function and some for another. However, when formatted they are treated as one address (obviously) so wrong information can result.
-S
This specifies a signal which should force a snapshot. The probe will register a signal handler for this signal so it should be one that is compatible with he application being tested. (Since 4.4.9).
-z
This displays symbols or modules which have no associated CPU: Normally just the names of the routines (or modules) that have matches in the sampling table are displayed. This switch will also display the list of all other routines (or modules). This is a format-time option.

If you do not provide a module name, the application will be profiled.

At format time, the -l, -o, -r and -z options may be provided if they were not specified at runtime.

Statprof Memory And APD Usage

Using statprof requires some consideration for the memory and APD file sizes. Memory usage and APD file size is dependent on the number of sampling "cells" needed to record the profile counts. Memory space must be allocated by statprof to hold a counter (2 bytes per counter) for each sampling cell the cell counters must fit into a single APD file. Additionally, the APD files must not wrap around or the snapshot data will be lost.

Profiling more of the application requires more cells. Increasing the sampling cell size (with -s) will reduce the number of sampling cells needed at the expense of granularity. The -c option requires the fewest sampling cells. The -a option requires the most. On Linux the counters are allocated using mmap. On AIX the counters are allocated using malloc.

As and example, suppose you are profiling an application module whose text size is 10 MB. Roughly speaking, you will need about 1.25 MB (= 10 MB / 8) for the counters at runtime. As for APD files, they must be at least 1.25 MB in size. Use the Aprobe -s option to control individual APD file sizes and the Aprobe -n option to control the number of APD files in the ring.

Here is a sample command that might be used to achieve this:

 aprobe -u statprof -s 2000000 -n 10 myapp,exe
 

This example sets the APD file size to 2 MB and number to 10 which would allow for 10 snapshots during the test run.

Here is a sample command that might be used to profile a larger module but limit memory usage:

 aprobe -u statprof  -p "-s 32" -s 2000000 -n 10 myapp,exe
 

Note that there are two -s options in use here. In this case the cell size was set to 32 bytes. This allows profiling a module whose size if 4 times that of the previous example using the same memory footprint. This cells size is probably good enough to distinguish functions but not lines in the functions.

Statprof API

Users can control the behavior of the statprof probe by calls from within their own probes. The API is defined by $APROBE/include/statprof.h. The functions exported by statprof.ual are:

ap_Statprof_Enable

Enables collection of statprof data.
ap_Statprof_Disable

Disables collection of statprof data.
ap_Statprof_Snapshot

Takes a snapshot of current statprof data.