Memstat Predefined Probe

Statistical Memory Analysis Probe: memstat.ual

The memstat.ual predefined probe uses statistical sampling to monitor memory usage to help you detect heap memory leaks.

Heap memory leaks may be identified by tracking the invocations of all known heap manager functions that allocate and deallocate portions of the heap memory, and by matching each such heap memory allocation with its corresponding deallocation. Rather than tracking all allocations and deallocations, which can cause performance problems, memstat.ual will sample allocations and deallocations over a longer period of time.

Snapshots may be taken at any time while the program is running to save the present state of the tracked heap data. This allows the allocation state data that was recorded at various times to be compared to each other and to give an indication of how much heap memory was consumed at those moments. Also, when associated with particular actions of the application program, snapshots can help locate heap usage problems.

The collected data, including any snapshots, are saved in an APD file so that they can be viewed using apformat after the program has terminated. Reports are produced showing the time, size, and point of allocation for all items that were still allocated at the time of the snapshot.

Assumptions

This probe assumes that all requests for allocations and deallocations of dynamic storage are made through calls to discrete run-time heap manager functions (e.g., malloc, free, etc.).

There can be any number of these allocation or deallocation functions, and some functions (e.g., realloc) may do both allocations and deallocations during the same call. Additional heap manager functions may be specified in addition to the default ones by adding your own probes to those provided by memwatch.ual.

Background

Typically, the heap is a large repository of unused memory available for dynamic storage that is controlled by a heap manager. All requests for more memory (especially for dynamically sized objects) needed by a running application program are made to the heap manager. The heap manager carves the heap into smaller portions and allocates those portions upon request. In a well behaved application program, when those allocated portions of heap memory are no longer needed, they are deallocated by returning them to the heap manager for later reuse.

A heap memory leak is an unused and possibly inaccessible portion of heap memory that was allocated but was not subsequently deallocated. If an allocated portion of heap is no longer being used, such as when the only pointer to it goes out of scope or is overwritten, then that portion has probably "leaked." Such heap leaks gradually erode the available heap memory, which may lead to disastrous results when memory runs out.

A true "heap leak" is hard to detect without language and compiler support, because often a program allocates large amounts of data and intentionally keeps it around until program completion. Without tracking every variable to which a heap address is assigned, it is impossible to know when all pointers to a given location are lost. And even if this were possible, there may be other cases where memory should be freed even though it is still potentially accessible.

The memwatch probe helps the user identify heap leaks by recording all allocations and deallocations, and keeping a running total of total allocated memory, and allowing the user to analyze this data to determine if memory usage is appropriate. The reports produced by memwatch are designed to help identify potential leaks by grouping allocations by size, age, and point of allocation.

Usage

This probe is applied at run time using probe] as described in Memstat UAL Parameters below. The configuration file (see Memstat Configuration File controls the amount and kinds of data collected by this probe.

While most of the probes described in this Appendix provide a point-and-click interface for setting configuration options, the memstat probe currently does not. The default configuration is generally acceptable, and the heap manager functions to probe are hard-coded into the probe itself.

Memstat UAL Parameters

memstat.ual is specified on the aporbe] command line or in an APO file as described in Command Line. The specific options are:

 aprobe -u memwatch.ual    [-p [-h] [-v] [-g]  [-c config_filename]]  your_program
 

where:

-c config_filename

specifies that the name of the probe configuration file will follow immediately after -c. The default file name is your_program.memstat.cfg, where your_program is replaced with the name of your executable program. For example, if your executable program is called "wilbur.exe", then the default file name would be "wilbur.exe.memwatch.cfg".

-h

produces brief help text.

-v

means verbose mode, which produces additional progress messages.

Memstat Configuration File

The example below shows the default memstat configuration file.

PROBE CONFIGURATION FILE FOR MEMSTAT VERSION 1.0.0


// 1. Runtime options:
//
// 1.1. DepthOfCallChain <num>
//    Controls the maximum number of entries in each call chain recorded
//    by this probe.
//
// 1.1. PeriodicStatsInterval <num> (default 1)
//    Sets the number of seconds to be used as an interval for data summary.  
//
//    For example:
//
//    PeriodicStatsInterval 1
//
//    Will report allocation statistics every second.  
// 
//    PeriodicStatsInterval of 0 turns off reporting periodic statistics.
//
// 1.2. Verbose TRUE | FALSE
//
//    Will turn on reporting of periodic intervals expiration.
//
// 1.3. FreeCacheSize <num> (default 0)
//
//    Defines a number of objects deallocated with a call to free()
//    that this probe will hold on to before giving them back to the heap.
//    Larger FreeCacheSize  values make it detection of double deallocations
//    more likely, increasing the memory consumption by the application
//    at the same time.
//
// 1.4. Options to control sampling and size filtering
//
// 1.4.1. SamplingRatio <num> (default 100)
//
//    Defines the ratio of the number of allocations to the number of samples.
//    For instance, specifying 1000 means that only one in a 1000 allocations
//    are actually sampled (but see the filtering criteria below). Specifying
//    a ratio of 1 means that every allocation is recorded.
//
// 1.4.2. MaxItemsToTrack <num> (default 100000)
//
//    When sampling is on (e.g. the ratio is > 1), this contains the maximum
//    number of objects we will track. If an object is not entered into the
//    table, a corresponding deallocation or reallocation will not be logged.
//    If the table size is reached, you will get a single error message
//    informing you of this; further allocations will be recorded once there
//    is free space. Note that setting this to 0 has the same behavior as
//    disabling sampling.
//
// 1.4.3. FilterOutBelowSize <num> (default is 0)
//
//    When sampling is on, ignore the allocation if it is smaller than the
//    specified size.
//
// 1.4.3. FilterInAboveSize <num> (default is UINT_MAX == 4294967295U)
//
//    When sampling is on, add this allocation to the current list if it is
//    greater than the specified size regardless of how many objects are in
//    the table. Use this option carefully since it could lead to very large
//    active object tables if you set it too low.
//
// 1.5. AllocationModule modulename
//    For Solaris and Linux you can override the use of libc.so as the 
//    module for locating the malloc and free routines. For instance, if you
//    use the libmapmalloc.so library, you would use:
//
// AllocationModule "libmapmalloc.so"
//
//    This option is ignored (with a warning) for other platforms.
//
// 1.6. TrackingEnabledInitially TRUE | FALSE
//
//    Will turn on or off tracking at program start-up.  Default is TRUE.
//
//
//
// 2. Formattime options:
// 2.1. Reports To Produce (DisplayReports)
//    DisplayReports \
//       ReportAllAllocations \
//       ReportPeriodicStats \
//       ReportDoubleDeallocations \
//       ReportAllocationPoints \
//       ReportDeallocationPoints \
//       PlotPeriodicStats \
//       ShowTimeSliceSummary \
//       ReportAnalysis
//
// 2.1.1. ReportAllAllocations
//    At runtime memstat probe collects information about ALL allocations 
//    made by your application.  This includes the address of 
//    allocated/deallocated memory its size, location (traceback) of 
//    the allocation/deallocation point.  By default this data will be
//    summarized for you in several reports that groups the data
//    by allocation/deallocation points.  Sometimes you may want to see
//    the raw allocation/deallocation data as it was recorded at runtime.
//    Selecting ReportAllAllocations report and reformatting the data
//    will accomplish that.  Be aware of large amounts of data this report
//    may produce.
//
// 2.1.2. Report Periodic Statistics (ReportPeriodicStats)
//    Choosing this report will print the memory allocation statistics
//    collected at periodic intervals.  This report will not have any affect
//    unless PeriodicStatsInterval runtime option (-s <num> on the command line)
//    was used at runtime.  This report may be useful in seeing how
//    the allocation activity relates to the time line.
//
// 2.1.3. PlotPeriodicStats
//    Choosing this report will result in a plot of all periodic statistics
//    collected at runtime (see 2.1.2 ReportPeriodicStats).
//
// 2.1.4. ReportDoubleDeallocations
//    This option is on by default.  When the option is on you will be
//    notified of all deallocations for which an allocation has not been
//    seen yet.  In some rare circumstances it is possible to have legitimate
//    deallocations of memory for which aprobe has not detected an allocation.
//    In all of these cases the deallocation is done by some system
//    libraries and usually could be ignored.  In particular their is initial
//    allocation done by a Java VM which happens when Aprobe has probes
//    disabled and they can be safely ignored. The double deallocations
//    coming from the user code are almost always legitimate problems, which
//    should be addressed. 
//    There are actually two double deallocation cases that are detected by
//    this probe: one is when a memory is deallocated, for which have not
//    seen a single allocation, and two is the case when we see memory
//    being deallocated after having been deallocated already.
//    The error message for the first one is:
//       "freeing memory that has not been allocated"
//    The error message for the second one is:
//       "freeing memory that has been deallocated"
// 
//    ReportDoubleDeallocations only controls reporting of the first type
//    of double deallocation error, since the second one should never be
//    ignored.
//
// 2.1.5. ReportAllocationPoints
//    This option is on by default.  It is the central report produced
//    by this probe.  This report produces a summary of allocations 
//    grouped by allocation points.
//
// 2.1.6. ReportDeallocationPoints
//    This option is on by default.  This report produces the summary
//    of all deallocation points known for each allocation point.
//    It is very useful when you try to understand the data flow between
//    the allocation and deallocation points.
//
// 2.1.7. ShowTimeSliceSummary
//    This option summarizes the outstanding allocations over the lifetime
//    of the application. Each allocation is placed into a slice based on the
//    allocation time. The allocation time slice is specified with the
//    TimeSliceDuration option. The MaxTimeSlices controls the maximum number
//    of these slices that are displayed. Note that the TimeSliceDuration may
//    be modified dynamically to allow more samples to fit into the array.
//
// 2.1.8. ReportAnalysis
//    This option presents the results of the analysis performed on the data
//    to attempt to identify which allocation points you should look at first.
//    This option is on by default.
//
// 2.2 Allocation Points
//    This probe records Allocation Points with tracebacks to describe where
//    a given memory allocation was made.  Labels to help identify these
//    tracebacks, as well as special filters to only include those tracebacks
//    which you are interested in monitoring, are available.
//
// 2.2.1 AllocPoint (Allocation Points)
//    You can give meaningful names to symbolic tracebacks produced by this 
//    probe with the AllocPoint directive.  The AllocPoint keyword has to be
//    followed on the same line by the name/comment you would like to associate 
//    with a given traceback, and the traceback itself has to be provided
//    on the lines following the AllocPoint directive.  The tracebacks have
//    to start with keywords: "==>" or "Filter".  For example:
//
//   AllocPoint "Known leak #200"
//   Filter extern:"malloc()" in "MSVCRT.dll"
//   ==> extern:"spaghetti()"
//
// 2.2.2 Filtering Allocation Points
//    Allocation Points can also be excluded from recording, by providing
//    a matching traceback to filter; the syntax for specifying exclusive
//    filters is the same as that for additive filters, but with the keyword
//    "REMOVE" in place of "Filter":
//
//   REMOVE extern:"malloc()" in "MSVCRT.dll"
//   ==> extern:"spaghetti()"
//
// 2.3. StartTime <Seconds> /StopTime <Seconds>
// 
//    These options allow for selective formatting of data collected
//    between the two intervals specified with StartTime and StopTime options.
//
//    Example:
//    StartTime 10
//    StopTime 20
//
//    The above options will cause your report to only include data
//    during seconds 10 - 20 of your program run.
//    You can specify these options on the command line with
//    arguments -t <num> and -e <num> to memstat.
//
// 2.4. ShowFreedAllocationsInSummary
//    By default the summary table(s) do not show allocation points where
//    all of the allocations have been freed. Setting this option to TRUE
//    will include such freed allocations in the summary.
//
// 2.5. MaxTimeSlices
//    This is used with the ShowTimeSliceSummary option to control the maximum
//    slices we hold. Be careful about making this number too high because it
//    will directly influence the amount of memory required to format the
//    data.
//
// 2.6. TimeSliceDuration
//    This is used with the ShowTimeSliceSummary option to control the size of
//    each slice. Using a smaller number allows a finer granularity but may
//    move more allocations into the last time slice if MaxTimeSlices is
//    reached.
//
// 2.7. DisplayDeltaTimes
//    This option is on by default. It displays each allocation, free, etc.
//    as a delta in seconds from the beginning of the program. If this is
//    turned off (DisplayDeltaTimes FALSE) it will display the actual times.
//
// 2.8. OutputFile
//    This keyword sets the output file(s) that the reports are directed to.
//    By default all of the reports go to standard out. The option can set
//    a directory that all files go to or can set individual files. Any files
//    that are not specified will go to stdout. Note that the directory must
//    already be created and must be writable. The Directory option cannot
//    appear with any of the individual file options - the first option found
//    will take precedence.
//
//    The sub-options are:
//
//    Directory "Name"     - specify an output directory. Three files will be
//                           created of the form xxx.memstat.yyy.txt where 
//                           xxx is the application name and yyy is "summary",
//                           "details" and "traceback" for the three files.
//
//    SummaryFile "Name"   - specify the summary output filename. All of the
//                           summarized tables and header information goes into
//                           this file.
//
//    DetailsFile "Name"   - specify the details output filename. All of the
//                           detailed allocation data and periodic data goes
//                           into this file.
//
//    TracebackFile "Name" - specify the traceback output filename. All of the
//                           tracebacks that were found go into this file.
//
// 2.9. PlotAllocation ID
//    This will plot an individual graph for the given traceback ID showing
//    the outstanding allocation size over time. You may have multiple instances
//    of this keyword. Note that traceback IDs will only remain the same
//    between one apformat and the next if the same set of APD files is used
//    and the start and stop times are used.
//
// 2.10. ShowZeroGrowthIds
//     By default, the analysis summary does not display entries for
//     allocation points that exhibited zero growth. Turning this option on
//     will display those entries.
//
// 2.11. AnalysisPercentage
//     The percentage of the formatted data used for analysis. Setting this
//     to a higher value than the default (80) will look at earlier data in
//     the run with an increased risk of marking initial allocations as growth.
//
// 2.12. DisplayAnalysisWithTracebacks
//     This option displays any analysis results with the traceback as well
//     as in the table. Note that if you have requested a detailed report you
//     will see tracebacks for analyzed allocations twice - once with the
//     analysis data and once in the full list of tracebacks. This option is
//     on by default.
//
// 2.13. RuntimeStartTime
//     By default tracking of object creation occurs from the start of the
//     application. By setting this option to a value (in seconds) you can
//     delay the tracking, speeding up initialization. For many applications,
//     the allocations made during initialization are not representative of the
//     steady-state behavior and are already filtered out by the analysis
//     algorithms. There is no optimal value for this - it depends entirely
//     on your application.
//
// 2.14. MaxDisplayTimeSlices
//     This sets the number of time slices you see presented if the
//     interval summary report is enabled. If this is less than the number of
//     slices recorded, you will see the last 'n' slices.
//
// 2.15. TimeSliceFields
//     This sets the fields displayed in the interval summary report.
//     It is followed by one or more of the following sub-options. You can
//     specify this option multiple times if desired.
//
//        AllocSizeField      : Size of allocations in this interval
//        FreedSizeField      : Size of frees in this interval
//        DeltaSizeField      : Size difference in this interval
//        TotalSizeField      : Total outstanding size
//        TotalDeltaSizeField : Delta outstanding size from last interval
//        NumAllocsField      : Number of allocations in this interval
//        NumFreesField       : Number of frees in this interval
//        DeltaNumField       : Number difference in this interval
//        TotalNumField       : Total outstanding number
//        TotalDeltaNumField  : Delta outstanding number from last interval
//
// 2.16 LimitAllocPoints
//     This limits the number of allocation points included in the reports.
//     It is followed by an integer value which specifies the maximum 
//     number of allocation points to summarize.  The detailed report is
//     not affected by this setting.
//
// 2.17 LimitAllocPercentage
//     This limits the number of allocation points included in the reports
//     to the number of allocation points displaying the user defined
//     percentage of the total number of bytes leaked.  It is followed
//     by an integer value indicating the limiting percentage of the total
//     bytes leaked.  This setting does not affect the growth analysis 
//     report, or the detailed report.
//
//     Ex: LimitAllocPercentage 85 - limits the report to those allocation
//         points contributing to 85 percent of the total bytes leaked.

PeriodicStatInterval

This must be followed by an unsigned integer value that specifies the interval in seconds between runtime statistics reporting. The value 0 will disable reporting. 0 is the default.

Verbose

This must be followed by TRUE or FALSE. The value TRUE enables reporting of sampling interval expirations. The default is FALSE.

FreeCacheSize

This must be followed by an unsigned integer. This value defines a number of objects deallocated with a call to free() that this probe will hold on to before giving them back to the heap. Larger FreeCacheSize values make it detection of double deallocations more likely, increasing the memory consumption by the application at the same time. The default value is 0.

SamplnigRatio

This must be followed by an unsigned integer create than 0. This value defines the ratio of the number of allocations to the number of samples. For instance, specifying 1000 means that only one in a 1000 allocations are actually sampled (but see the filtering criteria below). Specifying a ratio of 1 means that every allocation is recorded. The default is 100.

MaxItemsToTrack

This must be followed by an unsigned integer. When sampling is on (e.g. the ratio is > 1), this value specifies the maximum number of objects we will track. If an object is not entered into the table, a corresponding deallocation or reallocation will not be logged. If the table size is reached, you will get a single error message informing you of this; further allocations will be recorded once there is free space. Note that setting this to 0 has the same behavior as disabling sampling. the default value is 100000.

FilterOutBelowSize

This must be followed by an unsigned integer. When sampling is on, allocations smaller than this value in size will be ignored. The default value is 0 (no filtering).

FilterInAboveSize

This must be followed by an unsigned integer. When sampling is on, an allocation will be added to the current list if it is greater than the specified size regardless of how many objects are in the table. Use this option carefully since it could lead to very large active object tables if you set it too low. The default value is 4294967295.

AllocationModule

This must be followed by a module name in quotes. For Solaris and Linux you can override the use of libc.so as the module for locating the malloc and free routines. For instance, if you use the libmapmalloc.so library, you would use:

 AllocationModule "libmapmalloc.so"
 

This option is ignored (with a warning) for other platforms. The default value is "libc.so".

TrackingEnabledInitially

This must be followed by TRUE or FALSE. When FALSE tracking will be turned off at program start-up. The default is TRUE.

DisplayReports

This must be followed by a list of report names from the list below. The default is ReportDoubleDeallocations ReportAllocationPoints ReportAnalysis. The reports are as follows:

ReportAllAllocations

At runtime memstat probe collects information about ALL allocations made by your application. This includes the address of allocated/deallocated memory its size, location (traceback) of the allocation/deallocation point. By default this data will be summarized for you in several reports that groups the data by allocation/deallocation points. Sometimes you may want to see the raw allocation/deallocation data as it was recorded at runtime. Selecting ReportAllAllocations report and reformatting the data will accomplish that. Be aware of large amounts of data this report may produce.

Report Periodic Statistics (ReportPeriodicStats)

Choosing this report will print the memory allocation statistics collected at periodic intervals. This report will not have any affect unless PeriodicStatsInterval runtime option (-s <num> on the command line) was used at runtime. This report may be useful in seeing how the allocation activity relates to the time line.

PlotPeriodicStats

Choosing this report will result in a plot of all periodic statistics collected at runtime (see 2.1.2 ReportPeriodicStats).

ReportDoubleDeallocations

This option is on by default. When the option is on you will be notified of all deallocations for which an allocation has not been seen yet. In some rare circumstances it is possible to have legitimate deallocations of memory for which aprobe has not detected an allocation. In all of these cases the deallocation is done by some system libraries and usually could be ignored. In particular their is initial allocation done by a Java VM which happens when Aprobe has probes disabled and they can be safely ignored. The double deallocations coming from the user code are almost always legitimate problems, which should be addressed. There are actually two double deallocation cases that are detected by this probe: one is when a memory is deallocated, for which have not seen a single allocation, and two is the case when we see memory being deallocated after having been deallocated already. The error message for the first one is: "freeing memory that has not been allocated," and the error message for the second one is: "freeing memory that has been deallocated." ReportDoubleDeallocations only controls reporting of the first type of double deallocation error, since the second one should never be ignored.

ReportAllocationPoints

This option is on by default. It is the central report produced by this probe. This report produces a summary of allocations grouped by allocation points.

ReportDeallocationPoints

This option is on by default. This report produces the summary of all deallocation points known for each allocation point. It is very useful when you try to understand the data flow between the allocation and deallocation points.

// // 2.1.7. ShowTimeSliceSummary // This option summarizes the outstanding allocations over the lifetime // of the application. Each allocation is placed into a slice based on the // allocation time. The allocation time slice is specified with the // TimeSliceDuration option. The MaxTimeSlices controls the maximum number // of these slices that are displayed. Note that the TimeSliceDuration may // be modified dynamically to allow more samples to fit into the array. // // 2.1.8. ReportAnalysis // This option presents the results of the analysis performed on the data // to attempt to identify which allocation points you should look at first. // This option is on by default. //

IndexCallChains.

This must be followed by the value TRUE or FALSE. The default is TRUE. When this is set to TRUE, each unique traceback is denoted by a unique identification number, and the formatted report tables refer to this ID number rather than the entire traceback. This makes the tables easier to read. A separate list will be reported to show each traceback and its ID number. When this is set to FALSE, the full tracebacks are displayed in the formatted output.

DisplayReports.

This may be followed by any one or more of the following values: NoREPORT, AgeREPORT, or SizeREPORT. The default is both AgeREPORT and SizeREPORT. AgeREPORT causes the formatted output to contain a report showing the outstanding allocations sorted by age, starting with the oldest. SizeREPORT causes the formatted output to contain a report showing the outstanding allocations sorted by size, starting with the biggest. If neither value is specified, then no report is omitted. NoReport is just a placeholder, it does nothing except to specify that you want at least the summary report.

StartGUI

This must be followed by the value TRUE or FALSE. The default is FALSE. The value TRUE indicates that the heap allocation graphs should be shown when the target program runs, even if -g wasn't specified on the command-line. A FALSE value is overridden by the -g command-line option.

FILTER

This must be followed by all the lines of a traceback. Normally, a traceback consists of multiple lines: the first line has the name of a called function, and each subsequent line begins with an arrow and names the immediate caller of the previous line's function. By default, all allocations are recorded by the memwatch predefined probe. But a filter allows you to limit the memwatch probe's recording to only those allocations whose call chain matches all the lines in the filter.

You may specify multiple filters at once. The traceback you need to specify for each filter can be easily obtained by cut-and-paste from a previously formatted output.

Notice that the traceback almost always occupies several lines, and that the "==>" parts of the traceback are necessary at the beginning of each additional line. The following example shows the correct syntax:

FILTER extern:"malloc()" in "libc.so"
==> extern:"calloc()" 0x0044 in "libc.so"
==> extern:"getcwd()" 0x0130 in "libc.so"
==> extern:"::getCurDir(void)" at line 86 (t.cc)
==> extern:"main()" at line 348 (xpdf.cc)
==> extern:"_start()" 0x00dc

Configuration of Snapshots

The memwatch probe configuration file allows you to specify the name of some functions for which snapshots are to be automatically taken. This is done with lines beginning with the keyword SNAPSHOT.

Each SNAPSHOT line must specify a particular function as described above. The remainder of the SNAPSHOT line contains pairs, where each pair has a special identifier keyword followed by its own associated value. These pairs give supplementary information about the snapshot.

ON - This optional special identifier must be followed by the value ENTRY or EXIT. These signify that the snapshot is to be taken, respectively, on entry to the function, or upon exit from the function. The default is ON ENTRY.

IS - This optional special identifier must be followed by an arbitrarily long string enclosed within "" quotation marks. It specifies a textual description that is to be logged with the snapshot.

Memory Usage Monitor

When memwatch.ual is invoked with the UAL -g parameter, the Aprobe Memwatch Probe GUI window comes up. By default, this GUI shows two graphs providing information about the amount and rate of heap memory allocations.

In addition to the graphs, there are buttons to control the graphs, and there is a button that allows a snapshot of the current allocation data to be taken interactively.

The graphs display a record of the actual heap activity. The top graph displays the size (number of bytes) of outstanding heap memory allocated over time. Outstanding means that the heap memory is still allocated and has not yet been deallocated. The bottom graph displays the number of allocations that took place during each time interval. The time interval length (in seconds) is shown below.

You can manually zoom in to a selected portion of each graph by dragging the mouse while holding down the CONTROL key on the keyboard. Similarly, you can shift each graph horizontally by dragging the mouse sideways while holding down the SHIFT key on the keyboard. The ResumeUpdates button will restore the graphs back to their normal scale and positions, and resume updating them.

Use the Snapshot button while the target program is running to take snapshots of the current state of recorded heap allocations.

Use the Close button to close the probe and its GUI.

The graphs in this runtime GUI show actual heap memory usage. These graphs are updated periodically, at user-specified intervals. Many allocations and deallocations usually occur within each interval, between updates, so the range of heap sizes is shown for each interval. The HighWaterMark shows the highest value of heap size that was every recorded.

The heap allocation and deallocation events can be examined in more detail, later, by formatting the recorded data to produce reports. The formatted reports will list the data, sorted by age, size, or both. Each snapshot will be reported separately, and each will list all outstanding allocations (allocations which have not be deallocated yet) and all new deallocations since the previous (if any) snapshot was taken.

Memwatch API

You can control the behavior of the memwatch probe by calls from within their own probes. The API for the memwatch probe is defined by [../include/memwatch.h $APROBE/include/memwatch.h]. Some of the functions exported by memwatch.ual are:

ap_Memwatch_Allocation

Record that a heap allocation event has occurred.

ap_Memwatch_Deallocation
Record that a heap deallocation event has occurred.

ap_Memwatch_Reallocation

Record that a heap reallocation event has occurred (which both a deallocation and an allocation event at once).

ap_Memwatch_DoSnapshot

Takes a snapshot of current heap allocations.

Memwatch Performance Issues

The additional execution time caused by the memwatch probe is small (except for snapshot overhead, discussed below). This is because the memwatch probe only instruments a few specific functions, and the probe is tiny compared to the functions themselves (which are usually very computationally intensive).

The memwatch data requires quite a bit of memory. The amount of memory required is proportional to the number of unique tracebacks found among the allocation and deallocation events. This will reduce the memory available to your application program. So, if your application program is close to the process or system memory limit, this could cause its allocations to fail, which could even kill the application.

All the memwatch data that was collected is logged to an APD file when a snapshot is taken. A snapshot is taken either by interactively clicking the GUI button, by calling ap_Memwatch_DoSnapShot(), or by default at program exit. You can see that each snapshot may consume many megabytes, and it can take some time for the probe to write all this data to APD files on disk. The probe shares the same process as your application program, so we suggest that you take a snapshot only if the delay caused by writing out this data will not change your program's behavior.