Overview of the Log and Trace Analyzer

Standardized data collection is one of the core technology requirements identified by the Hyades project. As a first exploiter of the above technologies, the Log and Trace Analyzer, which is targeted for developers and support personnel, has been developed based on current Eclipse tools.

An interface has been designed which provides a single point of operation to deal with logs and traces produced by various components of a deployed system. Linking these two sets of tooling (tracing and logging) helps bridge the gap between problem determination and debugging of applications and middleware. By capturing and correlating end-to-end events in the distributed stack of a customer application, this tool allows for a more structured analysis of distributed application problems. The tool also makes it easier and faster for a person to debug and resolve problems within a system.

The objective for this release was to implement a single point of contact for log browsing, analysis, correlation and symptom database exploitation, covering users of Hyades technology as well as allowing plug-in of ISV applications.

Determining the root cause of a problem in a solution that consists of a collection of products can be difficult. All products produce problem determination data, such as trace records, log records and messages. However, the problem determination data cannot be easily correlated across different products and products on different servers. Each product's problem determination data can only provide a view through a small window into the overall solution problem. Timestamps are not sufficient: they are not granular enough and often clocks are not often sufficiently synchronized between servers. All of these problems make the job of problem isolation (that is, determining which server, which product, and what the root cause of the problem was) very difficult, and this complexity increases with the complexity and size of a solution.

The Log and Trace Analyzer, which enables you to import various log files as well as symptom databases against which log files can be analyzed and correlated, decrease this complexity. The core problem in problem isolation in today's solutions is that problem determination data between products is not correlated, that is, you cannot easily determine the relationship of events captured by one product with the events captured by another. The Logging tools address this problem by now allowing you to import and analyze log files (or trace files) from multiple products, as well as to determine the relationship between the events captured by these products (correlation).

Correlation is the process of analyzing and determining a set of related events, based on a set of rules that are used to interpret the data contained in the events. The Logging tools already provide several ways of correlating information within a single log file, such as ordering the events in a log (for example, by time stamp) or filtering the events (displaying only those events with a specific thread ID).

Correlation plug-ins typically provide the ability to relate data in two different logs, using a very specific set of rules to analyze and correlate the data. The data used by the correlation plug-ins to determine relationships is referred to as correlation data.

Most types of correlation can be classified as either sequence correlation or associative correlation.

Sequence correlation orders a set of events, using a specific set of values and rules to order those values. Ordering a set of events by time stamp is an example of sequence correlation.

Associative correlation groups a set of events that have equivalent or related information, using a specific set of values and rules to interpret those values. Associative correlation can be used to associate two events, group a set of events together, or associate two groups of events. Showing only the events with the same thread ID is an example of associative correlation.

Finally, correlation types can be used together to provide a complete picture. For example, when grouping a set of events together you typically also order the events in the group.

The Log and Trace Analyzer provides, as part of the base implementation, several correlation methods for events in a single product log:

Sequential correlation: the ability to sort the events in a log by various fields contained in the events (for example, time stamp)
Associative correlation: the ability to filter the events displayed in a log by the values in various fields contained in the events (for example, thread ID)

Related reference
Log view