Netomata Software Documentation -- Version 0.10.x

Programs

ncg -- Netomata Config Generator

Synopsys

ncg [ -v | --verbose ] [ -k | --keys ] [ -d | --dump ] [ -E errfile | --error-dump errfile ] [ -L libdir | --libdir libdir ] [ -V | --version ] [ -h | --help ] file [...]

Description

ncg generates config files for networks that are described by neto format file(s).

ncg parses the file(s) (in neto format) specified by the file ... argument(s) to build an internal model of a network, and then generates a config file for each node in the resulting network model which has a ncg_output tag defined for it (which specifies the file where the node's config should be written to, creating or overwriting the file as necessary), using the template file (in ncg format) specified by the node's ncg_template tag.

To read from standard input, specify "-" as the filename.

Options

-v or --verbose
Enable verbose mode, causing ncg to emit comments about what it is doing as it proceeds.
-k or --keys
Print a list of all keys defined within the model, then exit without generating any config files.
-d or --dump
Print the fully-parsed data structure defined by the model, then exit without generating any config files. The dump is in neto format, the same as ncg's input file(s).
-E errfile or --error-dump errfile
Specify the basename of a set of files to dump detailed error context to, if an error occurs while generating a config. It's possible that more than one file will be created as the error-handling process unwinds; the files will be named errfile.0, errfile.1, and so forth. The deepest and most specific error will generally be found in errfile.0, while the other error files may provide more context to help you understand how you got to the point where that deepest error occurred.
-L libdir or --libdir libdir
Specify the location of the Netomata libraries (the directory containing the netomata.rb file and related subdirectories), overriding the NETOMATA_LIB environment variable
-V or --version
Print the program's version number, then exit.
-h or --help
Print a help message, then exit.

Environment

NETOMATA_LIB
Name of directory containing Netomata libraries (file netomata.rb and associated subdirectories). If not specified, defaults to /usr/local/lib/netomata/lib.

As a Ruby program, ncg also looks for the standard Ruby environment variables (see ruby(1) man page).

Bugs

Please report any bugs you find by email to bugs@netomata.com.

History

ncg was originally written in 2008 by Brent Chapman of Netomata, Inc.

Copyright

Copyright (C) 2009, 2010 Netomata, Inc. All Rights Reserved.

See Also

ncg file format, neto file format.

Version

This documentation is for ncg version 0.10.x.

File Formats

ncg file -- Netomata Config Generator (ncg) template files

Description

Netomata Config Generator (NCG) template files (which are commonly named with the extension .ncg) are used by ncg as templates for generating config files (or subsections of those files) for various network devices and services, using the Embedded Ruby (ERB) mechanism (which is sometimes also referred to as "eRuby").

Format

In a nutshell, ncg files are text files with embedded Ruby (ERB) statements to control the final output when the file is interpreted. ERB is much like PHP and other embedded templating languages.

To use ERB, you embed Ruby language syntax into your text file, and mark the Ruby language segments like this:

<% ... Ruby code ... %>
so that the ERB interpreter can recognize them and apply them.

The ERB interpreter as used for ncg files recognizes the following variations of these tags:

<% Ruby code %>

The Ruby code is executed, but nothing is output.

<%= Ruby code %>

The Ruby code is executed, and its resulting value is output in place of the whole <%= Ruby code %> block.

<%# Comment %>

The block is treated as a comment and ignored. (This can be very useful in testing, to temporarily disable a certain block of Ruby code.)

<%% or %%>

Replaced in the output with '<%' or '%>', respectively.

Examples

Here's how you might use a template to generate an /etc/network/interfaces file for an Ubuntu or Debian Linux server which has 4 Ethernet interfaces on 4 different networks. Interfaces are "eth0" through "eth3", and for each interface "ethN":
  • The host's address is 192.168.N.17
  • The netmask is 255.255.255.0
  • The broadcast address is 192.168.N.255
  • The gateway address is only defined on eth0, and is 192.168.0.1
Here's the template:
<% 
my_hostid = 17
(0..3).each do |i|
-%>
auto eth<%= i %>
interface eth<%= i %> inet static
    address 192.168.<%= i %>.<%= my_hostid %>
    netmask 255.255.255.0
<% if (i == 0) then -%>
    gateway 192.168.<%= i %>.1
<% end -%>
    broadcast 192.168.<%= i %>.255
<%# blank line %>
<% end -%>
When this template is processed by ERB, the result is:
auto eth0
interface eth0 inet static
    address 192.168.0.17
    netmask 255.255.255.0
    gateway 192.168.0.1
    broadcast 192.168.0.255

auto eth1
interface eth1 inet static
    address 192.168.1.17
    netmask 255.255.255.0
    broadcast 192.168.1.255

auto eth2
interface eth2 inet static
    address 192.168.2.17
    netmask 255.255.255.0
    broadcast 192.168.2.255

auto eth3
interface eth3 inet static
    address 192.168.3.17
    netmask 255.255.255.0
    broadcast 192.168.3.255

Usage Notes

Passing Data to Templates

Data is passed to templates through Ruby instance variables; in Ruby, instance variable names begin with a single "@" symbol).

The instance variables may be of any class, but are usually of either class "Netomata::Node" or the fundamental Ruby "String" class.

Currently, templates must agree with whatever invokes them on what variables are passed, and what the class of each of those variables is. There is no automatic checking of variable presence or type. If a template requires a variable by a particular name and/or of a particular class, then the template needs to explicitly check that it has been passed what it expects, and raise an exception if it discovers a problem.

For example, here is a template to configure an interface on a Cisco router/switch. It expects that it has been passed a variable named @interface, which is a Netomata::Node and has subkeys of "name", "target", and "active" which are strings.

<%
    template_arg("@interface")
    template_arg_node_key("@interface", "name")
    template_arg_node_key("@interface", "target")
    template_arg_node_key("@interface", "active")
-%>
interface <%= @interface["name"] %>
 description IPMI <%= @interface["target"] %>
 switchport access vlan 48
 switchport mode access
<%= apply_idiom("shutdown", @interface, {"@active" => @interface["active"]}) %>
!

In the above example, you can also see how to apply other templates ("idioms" are templates that the ncg program expects to find somewhere in (...)!idioms relative to the current node; see the documentation for apply_idiom() elsewhere in this document) from within a template, and how to pass arguments to these other templates.

Helper Methods

In addition to the standard Ruby library classes and methods, there are a handful of "helper" methods available for use within ncg files.

apply_idiom(idiom, node [ , vars])

Looks for a node named node[(...)!idioms!idiom] (i.e., checks node and each of its ancestor nodes in turn for a sub-node named "idioms" which itself has a sub-sub-node named "idiom"), and applies it with optional vars passed as variables.

Variables to pass are specified as a Ruby hash, with the following syntax:

{ "@name" => @var [, ...] }
So, for example, each of the following would be a valid vars argument to apply_idiom() (assuming the referenced variables exist):
{"@vlan" => @vlan}
{"@active" => @vlan["active"]}
{"@target" => @target, "@vlan" => @vlan}

buildkey(part1 [ , part2 [ ...]])

Returns a key built by combining all the partN arguments, using "!" as a separator. For example:

buildkey("a", "b", "c")
returns
"a!b!c"

compare_parts(a, b)

Compares two interface- or version-style names, and determines what order they should sort in. This is similar to the Ruby core String#<=> operator, but it works in a way that is more useful for comparing multi-part strings such as interface and subinterface names, version numbers, and so forth.

Returns:

  • -1 if a should sort before b
  • 0 if a and b are equivalent
  • 1 if b should sort before a

To compare the arguments a and b, they are first broken into parts. Each part, in turn, is the longest-possible string of either:

  • letters [a-zA-Z]
  • digits [0-9]
  • anything else (i.e., a sequence of 1 or more non-alphanumeric characters)

Then, the corresponding parts of each argument are compared in turn, until an answer is determined, subject to the following rules:

  • If both parts are integers, they are compared numerically (so that "2" is less than "10", even though alphabetically "10" would sort before "2")
  • Otherwise, both parts are compared as text strings (case sensitive).

If the corresponding parts of both arguments are equivalent by these rules, then the next pair of corresponding parts is considered, and so forth, until an answer is determined.

For example, consider a list of interfaces; sorted normally (not using compare_parts, their order would be:

  • FastEthernet1
  • FastEthernet10
  • FastEthernet10.10
  • FastEthernet10.2
  • FastEthernet2

compare_parts would break each of these down into the following parts (shown by spaces) for comparison:

  • FastEthernet 1
  • FastEthernet 10
  • FastEthernet 10 . 10
  • FastEthernet 10 . 2
  • FastEthernet 2

So, sorted using compare_parts for the comparison, the list would be:

  • FastEthernet1
  • FastEthernet2
  • FastEthernet10
  • FastEthernet10.2
  • FastEthernet10.10

This is a more "natural" order for a list of interfaces and subinterfaces.

emit_header([prefix [ , before [ , after ]]])

Returns a string that is a multi-line header suitable for placing as a comment at the beginning of a generated config file, describing what tool was used to generate the config, what target key it was generated for, what date it was generated on, what user, hostname, and directory name it was generated for, and what arguments the generating program was invoked with.

prefix, if defined, is prepended to every line of the generated header. It should be whatever marks the rest of the line as a comment in the generated config file.

before, if defined, is used as the very first line of the generated header. It should be whatever marks the beginning of a multi-line comment in the generated config file. prefix is not applied to before, so before should include prefix if necessary.

after, if defined, is used as the very last line of the generated header. It should be whatever marks the end of a multi-line comment in the generated config file. prefix is not applied to after, so after should include prefix if necessary.

For example, to generate headers suitable for a shell script, you would do

emit_header("# ", nil, nil)
which would return a header of the form
# line 1
# line 2
# ...

To generate headers for a C-style config, you would do

emit_header(" *  ", "/*", " */")
which would return a header of the form
/*
 * line 1
 * line 2
 * ...
 */

filename_to_key(filename)

Converts filename to a key, by

  1. converting ".." elements to "(..)"
  2. converting "." elements to "(.)"
  3. replacing "/" with "!"
For example:
filename_to_key("../a/b/c/./d")
returns
"(..)!a!b!c!(.)!d"

ip_union(ip_a, ip_b)

ip_a, ip_b, and the result are all IP addresses represented as strings. The result is the bitwise OR of the two arguments ip_a and ip_b. For example:

ip_union("10.5.0.0", "0.0.16.34") => "10.5.16.34"
ip_union("10.5.1.0", "0.0.16.34") => "10.5.17.34"

relative_filename(basename, filename)

Expands file name filename relative to another file or directory name basename, returning the expanded filename. For example:

relative_filename("/a/b/c", "d") => "/a/b/d"
relative_filename("/a/b/c", "d/e") => "/a/b/d/e"

Note: as a special case, if filename begins with "/", it is treated as an absolute filename and returned as-is, not relative to basename. For example:

relative_filename("/a/b/c", "/d/e") => "/d/e"

template_arg(name [ , expected_class ])

This method can be used at the beginning of a template to check whether a needed argument has been passed to the template as an instance variable (see the section elsewhere in this document on Passing Data to Templates). Raises a Ruby ArgumentError (which typically aborts the program) if name (which is passed to this method as a quoted string, i.e. "@interface") doesn't exist. Also raises a Ruby ArgumentError if the variable exists, but isn't of the expected Ruby class (Netomata::Node, by default).

For example, the following would check that a node named "@interfaces" had been passed into a template, and raise an error if it hadn't:

<%
    template_arg("@interfaces")
-%>

template_arg_node_key(name, key [ , expected_class ])

This method can be used at the beginning of a template to check whether a needed argument key is defined within a node that has been passed to the template as an instance variable (see the section elsewhere in this document on Passing Data to Templates). Raises a Ruby ArgumentError (which typically aborts the program) if any of the following tests fail:

  • name (which is passed to this method as a quoted string, i.e. "@interface") exists
  • name is a node (Ruby class Netomata::Node)
  • name has a key named key
  • name[key] is of the expected Ruby class (String, by default)

For example, the following would check that a node named "@vlan" had been passed into a template, and that it had a key named "id" of Ruby class String, raising an error if it hadn't:

<%
    template_arg_node_key("@vlan", "id")
-%>

Netomata::IPAddr methods

Ruby class Netomata::IPAddr is an extension of the standard Ruby IPAddr class. In addition to all the methods defined in the Ruby IPAddr class, the Netomata::IPAddr class adds a set of methods for obtaining IP netmasks in various formats.

ipaddr.netmask

Returns ipaddr's netmask as a string in canonical format (dotted-quad for IPv4, and ':'-separated hex for IPv6).

For example:

  • Netomata::IPAddr.new("198.102.244.34/24").netmask -> "255.255.255.0"
  • Netomata::IPAddr.new("dead:beef:0123:4567::cafe:babe/80").netmask -> "ffff:ffff:ffff:ffff:ffff:0000:0000:0000"

ipaddr.netmask_as_bits

Returns the number of significant bits in the netmask of ipaddr.

For example:

  • Netomata::IPAddr.new("172.16.32.16/252").netmask_as_bits -> 30
  • Netomata::IPAddr.new("198.102.244.34/24").netmask_as_bits -> 24

Netomata::IPAddr.netmask_as_bits(netmask)

Returns the number of significant bits in netmask, which is expressed in any of the following forms:

  • string in IPv4 dotted-quad notation (i.e., "255.255.252.0")
  • string in IPv6 colon-separated-hex notation (i.e., "ffff:ffff::")
  • string in IPv4 or IPv6 CIDR notation (i.e., "/24" or "/40")
  • string as number of bits (i.e., "30")
  • integer as number of bits (i.e., 27)

For example:

  • Netomata::IPAddr.netmask_as_bits("255.255.255.252") -> 30
  • Netomata::IPAddr.netmask_as_bits("ffff:ffff::") -> 32
  • Netomata::IPAddr.netmask_as_bits("/24") -> 24
  • Netomata::IPAddr.netmask_as_bits("18") -> 18
  • Netomata::IPAddr.netmask_as_bits(12) -> 12

ipaddr.netmask_as_cidr

Returns the netmask of ipaddr as a string in CIDR notation.

For example:

  • Netomata::IPAddr.new("10.5.16.0/255.255.255.252").netmask_as_cidr -> "/30"
  • Netomata::IPAddr.new("198.102.244.34/24").netmask_as_cidr -> "/24"
  • Netomata::IPAddr.new("cafe:babe:1234::/ffff:ffff::").netmask_as_cidr -> "/32"

Netomata::IPAddr.netmask_as_cidr(netmask)

Returns a netmask as a string in CIDR notation, from input netmask, which is expressed in any of the following forms:

  • string in IPv4 dotted-quad notation (i.e., "255.255.252.0")
  • string in IPv6 colon-separated-hex notation (i.e., "ffff:ffff::")
  • string in IPv4 or IPv6 CIDR notation (i.e., "/24" or "/40")
  • string as number of bits (i.e., "30")
  • integer as number of bits (i.e., 27)

For example:

  • Netomata::IPAddr.netmask_as_cidr("255.255.255.252") -> "/30"
  • Netomata::IPAddr.netmask_as_cidr("ffff:ffff:ffff::") -> "/48"
  • Netomata::IPAddr.netmask_as_cidr("/24") -> "/24"
  • Netomata::IPAddr.netmask_as_cidr("18") -> "/18"
  • Netomata::IPAddr.netmask_as_cidr(12) -> "/12"

ipaddr.netmask_as_ipv4_quads

Returns the netmask of ipaddr as a string in "dotted quad" (IPv4) notation.

For example:

  • Netomata::IPAddr.new("10.5.16.0/255.255.255.252").netmask_as_ipv4_quads -> "255.255.255.252"
  • Netomata::IPAddr.new("198.102.244.34/24").netmask_as_ipv4_quads -> "255.255.255.0"
  • Netomata::IPAddr.new("cafe:babe:1234::/ffff:ffff::").netmask_as_ipv4_quads -> nil

Netomata::IPAddr.netmask_as_ipv4_quads(netmask)

Returns a netmask as a string in CIDR notation, from input netmask, which is expressed in any of the following forms:

  • string in IPv4 dotted-quad notation (i.e., "255.255.252.0")
  • string in IPv4 or IPv6 CIDR notation (i.e., "/24" or "/40")
  • string as number of bits (i.e., "30")
  • integer as number of bits (i.e., 27)

netmask must be an IPv4 netmask (i.e., 0 to 32 bits). If netmask is not a valid IPv4 netmask, then an error is raised.

For example:

  • Netomata::IPAddr.netmask_as_ipv4_quads(30) -> "255.255.255.252"
  • Netomata::IPAddr.netmask_as_ipv4_quads("/28") -> "255.255.255.240"

End-of-line Handling

Normally, if '<% Ruby code %>' or '<%= Ruby code %>' appears as a line in a template, then after the Ruby code is processed, there will still be a "newline" character in the template (the newline that followed the '<% Ruby code %>' block), and ERB will copy this into the output. This can lead to unexpected results. For example, consider the following template:
foo
<% 3.times do %>
x
<% end %>
baz
This will result in the following output, with what many would consider to be extraneous newlines:
foo

x

x

x

baz
To eliminate the extraneous newlines, simply change the closing tag for the embedded Ruby blocks from '%>' to '-%>'. That will cause the ERB interpreter to suppress the newline that would otherwise immediately follow the closing tag:
foo
<% 3.times do -%>
x
<% end -%>
baz
Resulting in the following output, without the extraneous newlines:
foo
x
x
x
baz

Alternate ERB Syntax Not Supported

Some ERB documentation speaks of a syntax whereby lines begining with '%' are also treated as Ruby code (i.e., a line '% Ruby code' is treated the same as a line '<% Ruby code %>'). This syntax is not supported in ncg files.

Embedded Ruby (ERB) References

Documentation and examples for ERB can be found at

History

ncg was originally written in 2008 by Brent Chapman of Netomata, Inc.

Bugs

Please report any you find by email to bugs@netomata.com.

Author

Brent Chapman of Netomata, Inc.

Copyright

Copyright (C) 2009, 2010 Netomata, Inc. All Rights Reserved.

See Also

ncg program, neto file format, neto_table file format.

Version

This documentation is for ncg version 0.10.x.

neto file -- Netomata network description

Description

Netomata network description files (which are commonly named with the extension .neto) are used as input by Netomata tools such as ncg, and describe a particular network configuration.

Structure

A neto file yields a tree-structured description of a network. The tree is made up of nodes, keys, values, and strings.

node

A node contains a list of key/value pairs.

value

A value can either be a node or a string.

key

A key is a string that provides the name for a particular value. keys consist of a string of any printable character except whitespace, '#' (which denotes a comment), "!" (which is the separator for composite keys), "(" or ")" (which are used to mark special keys, as described below), and "{" or "}" (which are used to mark metadata keys, as described below).

string

A string is any string of printable characters.

A string containing [%= ... %] is treated as an ERB (Embedded Ruby) block. See the full discussion of ERB blocks below.

(Note that parsing in general, and string parsing in particular, is currently very simple-minded; for example, anything from '#' to end-of-line is going to be ignored, as is leading and trailing whitespace. A future release will include quoting and escape mechanisms to work around these limitations.)

Special Keys

keys may be simple keys (i.e., not containing a "!" separator), or composite keys (a series of simple keys separated by "!").

composite keys that begin with a "!" are absolute keys that are interpreted relative to the root of the current tree of nodes. composite keys that don't begin with a "!" are relative keys that are interepreted relative to the current node in the tree.

Selector Keys

simple keys of the form ( selector ) are selector keys which do various magic with the tree of nodes and keys. The following selector keys are available:

(.)

Maps to the current node (just like "." as part of a UNIX file path). So, an absolute key of the form "!a!b!c!(.)" is the same as "!a!b!c"; if your tree is set up such that "!a!b!c" is a valid path, and the current node is "!a!b!c", then (.) relative to the current node will map to node "!a!b!c". Note that "!a!(.)!b!c", "!a!b!(.)!c", and "!a!(.)!b!(.)!c" will also all map to "!a!b!c".

(..)

Maps to the parent node of the current node (just like ".." as part of a UNIX file path). So, an absolute key of the form "!a!b!c!(..)" is the same as "!a!b"; if your tree is set up such that "!a!b!c" is a valid path, and the current node is "!a!b!c", then (..) relative to the current node will map to node "!a!b"

(...)

Maps to the ancestor of the current node which has keys that match whatever follows to the right of (...). For example, if you have a tree with nodes having keys:

!
!a
!a!b
!a!b!c
!a!b!c!d
!a!b!c!d!e
!a!b!c!x
!a!b!c!x!y
!a!b!c!x!y!z

and you are at node !a!b!c!x!y!z, then the "(...)" in relative key (...)!d!e is going to refer to node !a!b!c (which is the closest ancestor to !a!b!c!x!y!z that has sub-nodes !d!e), and thus (...)!d!e is going to refer to !a!b!c!d!e

(key=value[,key=value,...])

Maps to the child node of the current node having key/value pairs that match all of the key/value pairs specified in the selector. For example, if you have a tree with nodes and keys like so:

! (node)
!a (node)
!a!b1 (node)
!a!b1!k = "v1" (value)
!a!b1!m = "x1" (value)
!a!b2 (node)
!a!b2!k = "v2" (value)
!a!b2!m = "x2" (value)

then "!a!(k=v1)" is going to map to node "!a!b1" and "!a!(k=v2)" is going to map to node "!a!b2". Similarly, "!a!(k=v1)!m" is going to map to the string "x1" (same as "!a!b1!m"), and "!a!(k=v2)!m" is going to map to the string "x2" (same as "!a!b2!m").

Anonymous Keys

In addition to the selector keys defined above, there are also special selector keys which create and operate on anonymous keys. Anonymous keys are simply numbered keys which the software creates as requested, for example when building a list of items. Anonymous keys are all of the form "@NNNNNNNNN" (i.e., "@" followed by a number).

The following selector keys operate only on anonymous keys:

(<)

Maps to the minimum anonymous key of the node. I.e., if the node has anonymous keys "@000000001", "@000000002", and "@000000003" (regardless of any non-anonymous keys it may have), then (<) will map to "@000000001".

(>)

Maps to the maximum anonymous key of the node. I.e., if the node has anonymous keys "@000000001", "@000000002", and "@000000003" (regardless of any non-anonymous keys it may have), then (>) will map to "@000000003".

(+)

Maps to the successor anonymous key of the node, which is whatever would come next after the node's current maximum anonymous key of the node. I.e., if the node has anonymous keys "@000000001", "@000000002", and "@000000003" (regardless of any non-anonymous keys it may have), then (+) will map to "@000000004". This is often used to create a new node with a new anonymous key.

All of the regular selector keys defined above also operate on anonymous keys.

You should never refer to an item directly by an anonymous key, as the anonymous key for that object might be different the next time the software runs; instead, you should use selector keys such as (key=value) to identify the node by some relevant characteristic of it. For example, if you have a list of VLANs stored under !vlans, each with an "id" and "name" attribute defined, you could identify the node for VLAN 3 (the "Lab" VLAN) by "!vlans!(id=3)" or "!vlans!(name=Lab)", rather than "!vlans!@000000009", because the "@000000009" might not refer to the same VLAN entry the next time the program runs.

Metadata Keys

Metadata keys are simple keys (i.e., not containing a "!" separator) of the form "{METADATA_KEY}". They may not be a part of a composite key (a series of simple keys separated by "!").

The following metadata keys are available:

{FILENAME}

Maps to the full name of the file currently being processed.

{BASENAME}

Maps to the basename of the file currently being processed (i.e., the last part of the filename, with all the preceding directory names removed; for example, if {FILENAME} is "sample/templates/cisco.neto", then {BASENAME} will be simply "cisco.neto").

{DIRNAME}

Maps to the directory name of the file currently being processed (i.e., everything except for the last part of the filename; for example, if {FILENAME} is "sample/templates/cisco.neto", then {DIRNAME} will be "sample/templates").

ERB Blocks

A string containing [%= ... %] is treated as an ERB (Embedded Ruby) block. The block is processed through the ERB interpreter as if it were a small file containing just the block with [%= ... %] changed to <%= ... %>. The value placed in the tree for such a string is the ERB interpreter's output, not the ERB block itself.

When the ERB block is executed, the following arguments are made available to the running code as Ruby instance variables:

@target
The current node (a Ruby object of class Netomata::Node)
@target_key
The key that identifies the current node (a Ruby object of class String).

Note that each [%= ... %] block is run through ERB separately, as if each was a separate little file. This is not how ERB is normally used; ERB is normally applied on a whole-file basis, essentially as a preprocessor for the entire file. This difference from normal behavior is why these blocks are marked with [%= ... %] rather than the usual ERB <%= ... %>.

For more information about ERB, as well as about helper methods that are available, see the ncg file format documentation. Note that ERB files, as discussed in that documentation, are a mix of Ruby code (identified in <% ... %> segments) intermixed with template text; ERB blocks, on the other hand, are independent, self-contained blocks of Ruby code, without any template text. Keep in mind that each [%= ... %] ERB block is treated as if it were a separate file, and thus, some of the complex examples in the ncg file format documentation that show Ruby control logic spread across multiple <%= ... %> statements aren't applicable for ERB blocks.

Format

A neto file consists of a series of lines. Lines that end with '\' are treated as if they continued on the next line; before processing continues, the current line is joined with the next, and the '\' and embedded newline characters are removed. Comments begin with a '#' symbol, and proceed to the end of the line. Leading and trailing whitespace on a line are ignored, as are blank lines. Once stripped of comments, leading whitespace, and trailing whitespace, each line must match one of the following formats:

key = string

Sets key to be string in the current node. (If string is empty or "-", then key will not be defined; this provides a way for an ERB block to say "never mind, don't set this key", by returning an empty string or "-" as its result.)

key {

Creates a new (empty) sub-node of the current node, names the new sub-node key, and makes the new sub-node the current node (so that further lines are processed relative to the sub-node, until the matching "}"

key < source_key {

Creates a copy of the node identified by source_key as a new sub-node of the current node, names the new sub-node key, and makes the new sub-node the current node (so that further lines are processed relative to the sub-node, until the matching "}"

}

Ends processing of the current sub-node, and makes the current node whatever it was before processing of the current sub-node began.

include filename [ ... ]

Incorporates a file (or set of files) in neto format into the tree at the current location, as if its contents appeared here in the original file. Each filename can name a specific file, or can be a filename glob pattern (in Ruby Dir.glob format, which is very similar to shell filename glob format), which will be expanded to a sorted list of filenames. If multiple filename arguments are given (either directly, or via a filename glob pattern that expands to multiple filenames), the software behaves as if all those named files had been concatenated together for processing. All filename arguments (whether specified directly or determined as a result of filename glob expansion) are interpreted relative to the current filename (the file in which the include statement appears).

table filename [ ... ]

Incorporates a file (or set of files) in neto_table format into the tree at the current location. Each filename can name a specific file, or can be a filename glob pattern (in Ruby Dir.glob format, which is very similar to shell filename glob format), which will be expanded to a sorted list of filenames. If multiple filename arguments are given (either directly, or via a filename glob pattern that expands to multiple filenames), the software behaves as if all those named files had been concatenated together for processing. All filename arguments (whether specified directly or determined as a result of filename glob expansion) are interpreted relative to the current filename (the file in which the include statement appears).

template filename.ncg

template directives are used to populate the tree with pointers to ERB template files that are referenced elsewhere, and used when ncg begins its config generation pass.

The directive defines two keys within the current node:

name
basename of file (i.e., filename stripped of leading directory path and trailing ".ncg" suffix)
ncg_template
path to file (i.e., the argument to the template directive)

filename.ncg is interpreted relative to the current filename (the file in which the template statement appears). An error is raised if filename.ncg doesn't exist or isn't a file.

Because template works on the current node, rather than creating a new node, you cannot specify multiple filename.ncg arguments (or wildcard arguments) to a template directive; each successive file would simply end up overwriting the name and basename keys defined in the current node. To incorporate multiple templates into the tree, either use multiple template directives or use the template_dir directive.

template_dir dirname

Creates a sub-tree of nodes in the tree at the current location, based on the contents of directory dirname. Depending on the name and type of each entry in directory dirname, the following actions are taken:

file named filename.ncg
A sub-node named filename is created, and a template filename.ncg directive is carried out to populate it. Equivalent to the following:

filename {
    template filename.ncg
}
subdirectory named subdir
A sub-node named subdir is created, and a template_dir subdir directive is carried out (recursively) to populate it. Equivalent to the following:

subdir {
    template_dir subdir
}
Any other file or subdirectory
Any other files or subdirectories are simply skipped.

Example

base_ip         = 10.5.0.0
admin_ip        = [%= ip_union(@target["(...)!base_ip"], "0.0.16.0") %]
syslog_ip       = [%= ip_union(@target["(...)!admin_ip"], "0.0.0.26") %]
syslog_facility = local5
snmp_ip         = [%= ip_union(@target["(...)!admin_ip"], "0.0.0.27") %]
snmp_community  = public
domain          = example.com

table vlans.neto_table

!templates {
    template_dir templates
}

!templates!devices!routers!cisco {
    # add to Cisco router device template
    model       = 4948-10G
    default_route = [%= ip_union(@target["(...)!base_ip"], "0.0.4.9") %]
    domain      = [%= "mgmt." + @target["(...)!domain"] %]
}

devices!(+) < !templates!devices!routers!cisco {
    name        = switch-1
    role        = primary
    # can't put this in the template, because "name" isn't defined there
    ncg_output  = [%= File.join(@target["{DIRNAME}"], \
                                "configs", (@target["name"] + ".config")) %]
}

devices!(+) < !templates!devices!routers!cisco {
    name        = switch-2
    role        = secondary
    # can't put this in the template, because "name" isn't defined there
    ncg_output  = [%= File.join(@target["{DIRNAME}"], \
                                "configs", (@target["name"] + ".config")) %]
}

table interfaces.neto_table

History

ncg was originally written in 2008 by Brent Chapman of Netomata, Inc.

Bugs

Please report any you find by email to bugs@netomata.com.

Author

Brent Chapman of Netomata, Inc.

Copyright

Copyright (C) 2009, 2010 Netomata, Inc. All Rights Reserved.

See Also

ncg program, ncg file format, neto_table file format.

Version

This documentation is for ncg version 0.10.x.

neto_table file -- tabular Netomata network description

Description

Netomata tabular network description files (which are commonly named with the extension .neto_table) are used as input by Netomata tools such as ncg, and describe parts of a particular network configuration. They are a specialized form of input for descriptions that make most sense when represented in tabular form (for example, lists of interfaces on a particular device, and associated parameters for each interface).

Anything in a neto_table file could be expressed in a neto file instead; a neto_table file is simply a more concise and natural form of expressing some types of data.

Structure

A neto_table file consists of three types of lines: comments, header lines, and data lines. Comments begin with '#', and extend to the end of the line. Header lines are lines that begin with special characters (either '@', '+', '<', or '%'). Data lines are any lines that are neither header lines nor comments.

Header lines define a set of actions, which are applied to each data line in turn; the header lines, taken as a set, describe how to convert a given data line into entries in the Netomata tree-structured description of the network.

Lines that end with '\' are treated as if they continued on the next line; before processing continues, the current line is joined with the next, and the '\' and embedded newline characters are removed.

Format

Data Lines

Data lines are broken into fields by strings of one or more tabs. All data lines must have the same number of fields, which must match the number of fields in the '%' header line (which specifies the name of each field). This has some important consequences:

  • You can't have tabs within a field in a data line, since tabs are used to separate one field from the next.
  • You can't have a "blank" or "null" value for any field in a data line, since adjacent tabs will be treated as a single field separator. There has to be something in the field, even if it's a "-" or a word like "UNUSED" or "NULL" or similar.

Header Lines

Header lines take one of the following forms:

% field1 [ field2 ... ]

A '%' line defines the names of the fields in the data lines. As with data lines, the field names are separated by strings of one or more tabs. Each neto_table file must have exactly one '%' line. The number of fields in each data line must match the number of fields named on the '%' line. The name of each field must be unique.

+ key

Creates a new (empty) node, and names it key.

+ key < source_key

Creates a copy of the node named by source_key as new node, and names it key.

@ key = value

Sets key to be value. (If value is a single dash, "-", then key will not be set.)

< key filename [ ... ]

Processes the neto format file(s) named by filename ... in the context of key. This is equivalent to the neto format construct of

key {
    include filename [ ... ]
}

In processing '+', '@', and '<' lines:
  • keys and source_keys can be simple keys or composite keys (see the discussion of simple and composite keys in the Special Keys section of the neto file documentation)
  • keys, source_keys, and values can use the construct '%{field}' to obtain the value of the named field from the current data line.
  • A value containing [%= ... %] is treated as an ERB (Embedded Ruby) block, and is fed through the ERB interpreter once for each data line; the value for such a string is the ERB interpreter's output, not the ERB template itself. See the discussion of ERB Blocks in the neto file format documentation. (Note that '%{field}' string substitution is performed on the ERB block before ERB is invoked, which can be used as a way to pass the value of fields from the current data line into the ERB block.)

Examples

Example 1

Consider the following simple neto_table file, which populates the "!vlans!" section of the data tree with the list of VLANs and their associated parameters for a particular network:
@ !vlans!(+)!id = %{id}
@ !vlans!(id=%{id})!active = %{active}
@ !vlans!(id=%{id})!type = %{type}
@ !vlans!(id=%{id})!vlan_ip = [%= ip_union(@target["(...)!base_ip"], "0.0.%{id}.0") %]
@ !vlans!(id=%{id})!netmask = %{netmask}
@ !vlans!(id=%{id})!name = %{name}
@ !vlans!(id=%{id})!description = %{description}
#
% id    active  type    netmask         name            description
# --    ------  ----    -------         ----            -----------
2       yes     switch  255.255.255.0   ISP-FW          ISP-to-Firewall
3       yes     switch  255.255.255.0   FW-LB           Firewall-to-LoadBal
4       yes     admin   255.255.255.0   LB-Rtr          LoadBal-to-Router
16      yes     admin   255.255.240.0   Management      Management
32      yes     admin   255.255.240.0   Bulk            Bulk
48      yes     admin   255.255.240.0   IPMI            IPMI
81      yes     env     255.255.255.0   Production      Production
82      yes     env     255.255.255.0   Pre-Prod        Pre-Prod
83      yes     env     255.255.255.0   Demo            Demo
84      yes     env     255.255.255.0   Dev             Dev
85      yes     env     255.255.255.0   QA              QA
86      yes     env     255.255.255.0   Test            Test
128     yes     switch  255.255.255.0   Corporate       Corporate
This is equivalent to the following neto file:
# first line of data from table
!vlans!(+)!id = 2
!vlans!(id=2)!active = yes
!vlans!(id=2)!type = switch
!vlans!(id=2)!vlan_ip = [%= ip_union(@target["(...)!base_ip"], "0.0.2.0") %]
!vlans!(id=2)!netmask = 255.255.255.0
!vlans!(id=2)!name = ISP-FW
!vlans!(id=2)!description = ISP-to-Firewall
# second line of data from table
!vlans!(+)!id = 3
!vlans!(id=3)!active = yes
!vlans!(id=3)!type = switch
!vlans!(id=3)!vlan_ip = [%= ip_union(@target["(...)!base_ip"], "0.0.3.0") %]
!vlans!(id=3)!netmask = 255.255.255.0
!vlans!(id=3)!name = FW-LB
!vlans!(id=3)!description = Firewall-to-LoadBal
# third line of data from table
!vlans!(+)!id = 4
!vlans!(id=4)!active = yes
!vlans!(id=4)!type = admin
!vlans!(id=4)!vlan_ip = [%= ip_union(@target["(...)!base_ip"], "0.0.4.0") %]
!vlans!(id=4)!netmask = 255.255.255.0
!vlans!(id=4)!name = LB-Rtr
!vlans!(id=4)!description = LoadBal-to-Router
# fourth line of data from table
!vlans!(+)!id = 16
!vlans!(id=16)!active = yes
!vlans!(id=16)!type = admin
!vlans!(id=16)!vlan_ip = [%= ip_union(@target["(...)!base_ip"], "0.0.16.0") %]
!vlans!(id=16)!netmask = 255.255.240.0
!vlans!(id=16)!name = Management
!vlans!(id=16)!description = Management
# and so forth ...

Example 2

Consider the following more complex neto_table file, which generates the interface entries for two separate switches (named "switch-1" and "switch-2") that make up a redundant switch pair in a web hosting environment. In this example, each host has two data Ethernet ports, one of which is connected to each switch; each host also an IPMI management Ethernet port, with the management port on odd-numbered hosts being connected to switch-1, and that on even-numbered hosts being connected to switch-2. Each switch's base node ("!devices!(name=switch-1)", for example) is presumed to already exist, and contain a "templates!interfaces!..." section with template nodes of various types.
+ !devices!(name=switch-1)!interfaces!(+) < \
        (...)!templates!interfaces!(name=%{type})
@ !devices!(name=switch-1)!interfaces!(>)!name = %{name}
@ !devices!(name=switch-1)!interfaces!(name=%{name})!type = %{type}
@ !devices!(name=switch-1)!interfaces!(name=%{name})!target = %{target1}
@ !devices!(name=switch-1)!interfaces!(name=%{name})!active = %{active1}
#
+ !devices!(name=switch-2)!interfaces!(+) < \
        (...)!templates!interfaces!(name=%{type})
@ !devices!(name=switch-2)!interfaces!(>)!name = %{name}
@ !devices!(name=switch-2)!interfaces!(name=%{name})!type = %{type}
@ !devices!(name=switch-2)!interfaces!(name=%{name})!target = %{target2}
@ !devices!(name=switch-2)!interfaces!(name=%{name})!active = %{active2}
#
% name  type    target1         active1 target2         active2
# ----  ----    ------          ------- ------          -------
Gig1/1  host    host-1a         yes     host-1b         yes
Gig1/2  host    host-2a         yes     host-2b         yes
Gig1/3  host    host-3a         yes     host-3b         yes
Gig1/4  host    host-4a         yes     host-4b         yes
Gig1/25 ipmi    host-1m         yes     host-2m         yes
Gig1/26 ipmi    host-3m         yes     host-4m         yes
This is equivalent to the following neto file:
# first data line, first set of header lines (for switch-1)
!devices!(name=switch-1)!interfaces!(+) < (...)!templates!interfaces!(name=host) {
}
!devices!(name=switch-1)!interfaces!(>)!name = Gig1/1
!devices!(name=switch-1)!interfaces!(name=Gig1/1)!type = host
!devices!(name=switch-1)!interfaces!(name=Gig1/1)!target = host-1a
!devices!(name=switch-1)!interfaces!(name=Gig1/1)!active = yes
# first data line, second set of header lines (for switch-2)
!devices!(name=switch-2)!interfaces!(+) < (...)!templates!interfaces!(name=host) {
}
!devices!(name=switch-2)!interfaces!(>)!name = Gig1/1
!devices!(name=switch-2)!interfaces!(name=Gig1/1)!type = host
!devices!(name=switch-2)!interfaces!(name=Gig1/1)!target = host-1b
!devices!(name=switch-2)!interfaces!(name=Gig1/1)!active = yes
# second data line, first set of header lines (for switch-1)
!devices!(name=switch-1)!interfaces!(+) < (...)!templates!interfaces!(name=host) {
}
!devices!(name=switch-1)!interfaces!(>)!name = Gig1/2
!devices!(name=switch-1)!interfaces!(name=Gig1/2)!type = host
!devices!(name=switch-1)!interfaces!(name=Gig1/2)!target = host-2a
!devices!(name=switch-1)!interfaces!(name=Gig1/2)!active = yes
# second data line, second set of header lines (for switch-2)
!devices!(name=switch-2)!interfaces!(+) < (...)!templates!interfaces!(name=host) {
}
!devices!(name=switch-2)!interfaces!(>)!name = Gig1/2
!devices!(name=switch-2)!interfaces!(name=Gig1/2)!type = host
!devices!(name=switch-2)!interfaces!(name=Gig1/2)!target = host-2b
!devices!(name=switch-2)!interfaces!(name=Gig1/2)!active = yes
# third data line, first set of header lines (for switch-1)
!devices!(name=switch-1)!interfaces!(+) < (...)!templates!interfaces!(name=host) {
}
!devices!(name=switch-1)!interfaces!(>)!name = Gig1/3
!devices!(name=switch-1)!interfaces!(name=Gig1/3)!type = host
!devices!(name=switch-1)!interfaces!(name=Gig1/3)!target = host-3a
!devices!(name=switch-1)!interfaces!(name=Gig1/3)!active = yes
# third data line, second set of header lines (for switch-2)
!devices!(name=switch-2)!interfaces!(+) < (...)!templates!interfaces!(name=host) {
}
!devices!(name=switch-2)!interfaces!(>)!name = Gig1/3
!devices!(name=switch-2)!interfaces!(name=Gig1/3)!type = host
!devices!(name=switch-2)!interfaces!(name=Gig1/3)!target = host-3b
!devices!(name=switch-2)!interfaces!(name=Gig1/3)!active = yes
# fourth data line, first set of header lines (for switch-1)
!devices!(name=switch-1)!interfaces!(+) < (...)!templates!interfaces!(name=host) {
}
!devices!(name=switch-1)!interfaces!(>)!name = Gig1/4
!devices!(name=switch-1)!interfaces!(name=Gig1/4)!type = host
!devices!(name=switch-1)!interfaces!(name=Gig1/4)!target = host-4a
!devices!(name=switch-1)!interfaces!(name=Gig1/4)!active = yes
# fourth data line, second set of header lines (for switch-2)
!devices!(name=switch-2)!interfaces!(+) < (...)!templates!interfaces!(name=host) {
}
!devices!(name=switch-2)!interfaces!(>)!name = Gig1/4
!devices!(name=switch-2)!interfaces!(name=Gig1/4)!type = host
!devices!(name=switch-2)!interfaces!(name=Gig1/4)!target = host-4b
!devices!(name=switch-2)!interfaces!(name=Gig1/4)!active = yes
# fifth data line, first set of header lines (for switch-1)
!devices!(name=switch-1)!interfaces!(+) < (...)!templates!interfaces!(name=ipmi) {
}
!devices!(name=switch-1)!interfaces!(>)!name = Gig1/25
!devices!(name=switch-1)!interfaces!(name=Gig1/25)!type = ipmi
!devices!(name=switch-1)!interfaces!(name=Gig1/25)!target = host-1m
!devices!(name=switch-1)!interfaces!(name=Gig1/25)!active = yes
# fifth data line, second set of header lines (for switch-2)
!devices!(name=switch-2)!interfaces!(+) < (...)!templates!interfaces!(name=ipmi) {
}
!devices!(name=switch-2)!interfaces!(>)!name = Gig1/25
!devices!(name=switch-2)!interfaces!(name=Gig1/25)!type = ipmi
!devices!(name=switch-2)!interfaces!(name=Gig1/25)!target = host-2m
!devices!(name=switch-2)!interfaces!(name=Gig1/25)!active = yes
# sixth data line, first set of header lines (for switch-1)
!devices!(name=switch-1)!interfaces!(+) < (...)!templates!interfaces!(name=ipmi) {
}
!devices!(name=switch-1)!interfaces!(>)!name = Gig1/26
!devices!(name=switch-1)!interfaces!(name=Gig1/26)!type = ipmi
!devices!(name=switch-1)!interfaces!(name=Gig1/26)!target = host-3m
!devices!(name=switch-1)!interfaces!(name=Gig1/26)!active = yes
# sixth data line, second set of header lines (for switch-2)
!devices!(name=switch-2)!interfaces!(+) < (...)!templates!interfaces!(name=ipmi) {
}
!devices!(name=switch-2)!interfaces!(>)!name = Gig1/26
!devices!(name=switch-2)!interfaces!(name=Gig1/26)!type = ipmi
!devices!(name=switch-2)!interfaces!(name=Gig1/26)!target = host-4m
!devices!(name=switch-2)!interfaces!(name=Gig1/26)!active = yes

History

ncg was originally written in 2008 by Brent Chapman of Netomata, Inc.

Bugs

Please report any you find by email to bugs@netomata.com.

Author

Brent Chapman of Netomata, Inc.

Copyright

Copyright (C) 2009, 2010 Netomata, Inc. All Rights Reserved.

See Also

ncg program, ncg file format, neto file format.

Version

This documentation is for ncg version 0.10.x.

Licenses

Netomata Config Generator (ncg) License

This is the LICENSE file from the Netomata Config Generator (ncg) software package:

Netomata Config Generator (ncg) License
=======================================

Copyright (C) 2008, 2009 Netomata, Inc. All Rights Reserved. 

Netomata Config Generator (ncg) is free software: you can redistribute
it and/or modify it under the terms of the GNU General Public License,
version 3, as published by the Free Software Foundation.

Netomata Config Generator (ncg) is distributed in the hope that it
will be useful, but WITHOUT ANY WARRANTY OF ANY KIND (without even
the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE), either express or implied, unless required by applicable
law or agreed to in writing.  See the GNU General Public License,
version 3, for more details.

You should have received a copy of the GNU General Public License,
version 3, along with Netomata Config Generator (ncg), in the file
named "COPYING".  If not, see <http://www.gnu.org/licenses/gpl-3.0.txt>.

For any other questions or comments, please contact us at:

        Netomata, Inc.
        2601C Blanding Ave., #327
        Alameda, CA 94501
        USA

        Web http://www.netomata.com/
        Phone +1 510 355 0123
        Fax +1 510 355 0134
        Email license@netomata.com