Configuration file structure

A configuration file contains a collection of contexts that are to be run by a Hyades Generic Log Adapter runtime instance. Each context describes how to process a single log file. Because the configuration file is able to contain many of these contexts, the adapter is capable of processing many log files at the same time.

Each context within the log file is constituted of a series of components. Each component has a role to play in processing a log file.

Context

The context describes the ordered grouping of components as they are chained together for log file processing. Each log file has a separate context, and multiple contexts can be defined in a single configuration. The Generic Log Adaptor provides one context:

A context has the following properties:

Property Type Behavior
Description string A text description for the context. This value is used only by the Log editor.
Unique Id string A unique identifier for this component. The context refers to this string to identify which component instance to use.
Is Continuous Operation boolean Whether or not this log file should be monitored for changes based upon the Maximum Idle Time and Pause Interval properties.
Maximum Idle Time integer (milliseconds) The maximum amount of time the context will wait for a log file to change before shutting itself down.
Pause Interval integer (milliseconds) The amount of time the context waits after reaching the end of the log file before trying to read again.

Sensor

The sensor provides the mechanism to read the content for processing. The Generic Log Adapter provides the following sensors:

A sensor has the following properties:

Property Type Behavior
Description string A text description for the sensor. This value is used only by the Log editor.
Unique Id string A unique identifier for this component. The sensor refers to this string to identify which component instance to use.
Maximum Blocking integer The maximum amount of data the sensor reads before forwarding to the extractor for processing. For the SingleOSFileSensor, this is the number of lines.
Convertor string This property, which applies to the SingleOSFileSensor only, is a command line for a preprocessing instruction to run before starting the sensor.
Directory string The directory location of the file to be read by the SingleOSFileSensor.
File Name string The name of the file to be read by the SingleOSFileSensor.
Shell string The UNIX shell to run the converter command in. Used only on UNIX platforms.

Extractor

The extractor takes a collection of input lines provided by the sensor and separates them into message boundaries. The Generic Log Adapter provides the following extractors:

The following properties are provided as part of the configuration for the extractor:

Property Type Behavior
Description string A text description for the extractor. This value is used only by the Log editor.
Unique Id string A unique identifier for this component. The extractor refers to this string to identify which component instance to use.
Contains Line Breaks boolean Indicates whether the messages in the template file span multiple lines.
Replace Line Breaks boolean Replace the line breaks in this file by another string segment. The replacement is made before the extractor searches for Start Pattern or End Pattern.
Line Break Symbol string The string segment to replace line breaks with. This is used only if Replace Line Breaks is true.
Include Start Pattern boolean Whether to include the StartPattern as part of the message to be forwarded to the next component. If the value is false, the matching portion of the message is stripped.
Start Pattern string or regular expression The string segment that identifies the start of a message. You can always use the RegularExpressionExtractor class. However, if your start pattern is a simple string, the SimpleExtractor class incurs less overhead.
Include End Pattern boolean Whether to include End Pattern as part of the message to be forwarded to the next component. If the value is false, the matching portion of the message is stripped.
End Pattern string or regular expression The string segment that identifies the end of a message. If End Pattern is not specified, then the message is delimited by the next Start Pattern.

Parser

The parser takes the messages that have been delimited by the extractor and builds a set of mappings of string values to the data structure which is being specified within the editor (Common Base Event 1.0.1). The parser has two phases of execution:

  1. The global processing phase, where a set of global regular expressions are executed against the message provided by the extractor.
  2. The attribute processing phase, where specific sets of substitution rules are executed to determine the value to be assigned to each attribute specified in the Log Editor.

The following global properties are specified as part of the parser:

Property Type Behavior
Description string A text description for the parser. This value is used only by the Log editor.
Unique Id string A unique identifier for this component. The parser refers to this string to identify which component instance to use.
Separator Token string or regular expression A token, or a regular expression that describes a series of tokens, that breaks the message up into attribute-value pairs.

Assume each new line of your log file contains an attribute and its value. You could specify a Separator Token of \n. The advantage of breaking the message up is that the string segments can be referred to in the attribute processing phase.
Designation Token string or regular expression A token, or a regular expression that describes a series of tokens, that breaks the string segments created by the Separator Token into attribute-value pairs. For example, if each attribute and value are separated by a = sign , and the value of Designation Token is = , the attribute and its value can be referred to in the attribute processing phase.

The individual attributes of the Common Base Event have the following properties:

Property Type Behavior
Use Previous Match Substitution As Default boolean If the current set of Substitution Rules for this attribute returns a positive match, then store the match value so that it can be used as a default to process the next record if there is no positive match.

SubstitutionRules are run as part of the attribute processing phase. They are processed in order from top to bottom. Once a rule is a match, no further rules are run against this property. If no match is found, then no value is placed in the attribute.

SubstitutionRules contain the following properties:

Property Type Behavior
Match regular expression The regular expression that identifies the value to be applied to this attribute if a match occurs in the scope of the string segment being processed by this Substitution Rule. If you do not provide a regular expression, the rule is considered a match.
Positions name-value attribute pairs Positions let you refer to the name-value attribute pairs created during the global processing phase. A Positions string can refer to many attributes by name using the following grammar:

$h('Attr1')@@$h('Attr2')...

$h('Attrn') identifies a previously created attribute name.

@@ is the separator between the attributes.

In the Match property, you can add the separator to indicate which regular expressions should be applied to which attribute value. The Substitute property can refer to these attribute atoms using $1, $2, and so on..
Substitute string The value to place in the attribute. This can be a string or it can refer to the regular expression in the Match property. The string segments that are extracted by the Match property can be referred to using the groupings in the regular expression. $1 indicates the first grouping, $2 the second, and so on.
Use built in function checkbox
  • Use built-in function when selected for an attribute, if none of the previous substitution rules, if any exist, are matched, will cause the Generic Log Adapter runtime to generate a value for that attribute using its own internal functions. This feature is only supported the followingn CBE attributes:
    • localInstanceId
    • globalInstanceId
    • componentIdentification.location
    • componentIdentification.locationType
    		•

    Formatter

    The formatter takes the mappings of attributes to their values provided by the parser and builds the correct Java object instance. The adapter runtime contains a single formatter that creates Common Base Event objects conforming to the Common Base Event version 1.0.1 specification.

    The formatter contains the following properties:

    Property Type Behavior
    Unique Id string A unique identifier for this component. The formatter refers to this string to identify which component instance to use.
    Description string A text description for the formatter. This value is used only by the Log Editor.

    Outputter

    The outputter externalizes the resulting Common Base Event records provided by the formatter. Outputters provide, or wrap, the mechanism for storing the final outputs of the context. The Hyades Generic Log Adapter provides the following outputters:

    Properties of the outputter:

    Property Type Behavior
    Unique Id string A unique identifier for this component. The outputter refers to this string to identify which component instance to use.
    Description string A text description for the outputter. This value is used only by the Log Editor.
    Outputter Type enumeration The type of outputter this is.

    Outputter types and required properties:

    Outputter type Required properties
    StandardOutOutputter None
    SingleFileOutputter directory, fileName
    LoggingAgentOutputter agentName - name given to the logging agent when it is created
    waitUntilLoggingTime(optional)- the time the outputter will wait for the user to attach and start monitoring the agent before it starts writing to the agent. If this property is not specified, the outputter will not wait for the agent to be monitored before writing to it.
    NotificationOutputter resourceURL - URL of the Notification Sink which will receive the events

    Related concepts
    Generic Log Adapter editor

    Related tasks
    Developing an end-to-end transform solution
    Creating an adapter file
    Deleting an adapter configuration file
    Testing an adapter configuration file with the editor test views

    Related reference
    Regular expression grammar
    Generic Log Adapter editor icons  

    (C) Copyright IBM Corporation 2000, 2004. All Rights Reserved.