Regular expressions are used in many places in the Network Configuration Manager (NCM). Because of technical and historical reasons, NCM uses a few different flavors of regular expressions for different functionality. In this post, I would like to explain the differences and provide references to more detailed documentation.

 

What Are Regular Expressions

Informally, a regular expression is a search pattern (sequence of characters) used mainly for string pattern matching and string finding. The term originated in formal language theory where regular expressions are exactly defined and related to other concepts of the same power (regular languages, finite automata, etc.). Although the current tools still use the term 'regular expression' (regex), it would be more precise to call them something like 'search patterns'; the expressive power of the search engine is, in fact, often much higher than that of the original regular expressions. (Many modern tools support patterns that are even more powerful than context-free languages.).

 

Regular Expressions for Comparison Criteria

When you compare two config files for changes, you may want to ignore some of them. (Examples: certificates, last modification time.) Defining comparison criteria enables you to filter out of comparison results lines that you do not need NCM to evaluate; this saves both processing time and, more importantly, makes the review of compared files easier.

 

Comparison-Criteria.PNG

 

The regular expressions you create and enable in the settings (Settings -> NCM Settings -> Comparison Criteria) are used throughout NCM — for example, in performing scheduled jobs — wherever the software needs to compare config files as part of its work. A few criteria that are often used come with NCM out of the box; you can also define your own. The lines you typically want to ignore are often informational comments like when was the configuration changed last.

 

NCM uses the well-known diff tool (from the GNU Diffutils package) for config file comparisons. As you can find in Diffutils documentation, diff uses grep-style regular expressions; then it's not difficult to learn from GNU grep manual that the default regex flavour for grep are basic regular expressions (BRE). Please note that the diff program works line by line -- you cannot define a regex that would match multiple lines.

 

Let's review some implications of BRE being the regex kind used in NCM comparison criteria.

  • Meta-characters ‘?’, ‘+’, ‘{’, ‘|’, ‘(’, and ‘)’ lose their special meaning; you must use the backslashed versions ‘\?’, ‘\+’, ‘\{’, ‘\|’, ‘\(’, and ‘\)’. In other words, you will use backslashes to give certain characters special meaning, while many other regex implementations use backslashes to take the special meaning away.
  • You have to use '%%' in the pattern when you want to match '%'.

 

More details on BRE syntax can be found e.g. on this page.

 

Device Templates

The parameter RegEx may be added to certain commands to recognize the string that is received when the command is complete. For example, if the command is complete when the device responds with System Characteristic, then you must add the following attribute to the command: RegEx="System Characteristic".

 

Another example: If you login on a device and must switch user context to execute a command, resulting in a different command prompt, use the following example to guide you when switching context and recognizing the new command prompt:

<Command Name=”Reset” Value=”appropriateSwitchContextCommands” RegEx=”newPrompt”/>

(You should also specify that the template logic should run in CLI mode: <Command Name=”MenuBased” Value=”false”/>)

 

NCM uses the Microsoft VBScript Regular Expression 5.5 engine to parse the value of the RegEx parameter. This engine implements Perl-style regular expressions with some limitations that you will probably never notice. Please see the details on this page.

 

Using Regular Expressions for Compliance Checks

Policy reports help ensure device configurations conform to both internal business practices and federal regulations, such as Sarbanes-Oxley Act (SOX), Health Insurance Portability and Accountability (HIPAA), and Computer Inventory of Survey Plans (CISP). Policy reports scan configuration files and report any discovered rule violations. For example, a rule requires configurations not to include the read-only community string public. You can run a report on your configuration files, and then display any configurations that violate the rule.

 

If you want to search the device config for a text block, or by a regular expression, you should use the Advanced Config Search. Compliance checks use the Microsoft .NET Framework regex engine. The syntax is described in Appendix B of the NCM Administrator Guide (http://www.solarwinds.com/documentation/orionNCM/docs/orionNCMAdministratorGuide.pdf) and also in Regular Expression Language - Quick Reference and Using Regular Expressions with .NET - C# and Visual Basic.

 

A Basic Example

By Tim Nelson.

When you create an advanced rule with multiple patterns, it simply takes the true/false of the patterns and then performs an AND operator on them to see if any are violated based on the overall criteria, not the individual rule criteria. Second, you can use parentheses to make rules inclusive of each other. This is needed when you want conditional statements with dependencies. Think of it as a creative way around if-then statements. So to explain this in better detail let’s cover what will and will not match in a policy.

 

If you create an advanced rule and you want to do a conditional based on bandwidth, you can say the following (keep in mind you should test ALL rules with the single regex you want to match before creating a complex multi-line rule, or use a regex tester):

 

       Must contain “string”: bandwidth 1536

and    Must not contain “regex”: class ipp1[^\r\n]

or     Must not containt “string”: bandwidth 1536

 

In order for this to actually work the top two rules must be contained within parentheses to make them inclusive of each other, i.e. one requires the other. What we are actually doing with the middle pattern is matching anything BUT “class ipp1\r” or “class ipp1\n” which is why we used, Must not contain. Lastly, all of this is wrapped within a container of “Must not contain” for rule violation, this is what actually decides if the rule is violated or not.

 

Policy-Create-Rule.PNG

 

What you will notice when this violates is that the “bandwidth 1536” is matched within the config, because you put “must contain” and it matched, NCM will add a plus sign where ever there is a pattern match. You must keep this in mind as you write rules, as without being the creator of the rule, this can become difficult to parse through.

 

NCM will show the first line on a multi-line match as the violation, so for end-user readability we can create rules that identify the exact line in error and therefore have more granular reporting. You should keep in mind that if your standards change in your network environment, you will have to be able to update these rules. So it is best to fully document your logic for every advanced rule you create.