Sysdef-files and Config-files

This page describes the grammar and semantics of system-definition files and configuration files and what they are good for.

Contents:
   Introduction
   Details on the System-Definition File
      Name, Location and General Format
      The Subsystem Name
      The VC-path
      Sysdef-file Options
      Redirection
      How AbaPerls Finds the Sysdef-file
   Config-file Overview
      Syntax
      The $SYSDEF directive
   The Subsystem Entry
      Name
      VC-Path
      The Version Specification
      Configuration options
   Config-file Examples

Introduction

System-definition files (sysdef-files for short) and configurations files (config-files for short) are important concepts in AbaPerls. Config-files have existed in AbaPerls for a long time, while sysdef-files are a newer addition.

A system-definition file defines which subsystems your application consists of and how may refer to each other. On each line there are two items: the name of the subsystem and its VC-path. The path can be relative to the location of the sysdef-file; this is very practical when you have a version-before-subsystem structure. One sysdef-file can redirect to another file, so that you have a single file as master if you use subsystem-before-version. The sysdef-file can also include rules how subsystems may refer to each other.

A config-file is used to build and maintain a certain database configuration. A config-file can be free-standing or refer to a system-definition file. A config-file defines which subsystems that are included in a particular configuration or database. For each subsystem the config-file determines:

Configuration options control how AbaPerls load SQL files. DBBUILD saves the configuration options in the database. ABASQL and the update scripts generated by DBUPDGEN read them when operating on the database. One such configuration option is ‑crypt that instructs AbaPerls to add the clause WITH ENCRYPTION when loading stored procedures and other modules in SQL Server.

You use the config-file when you build a database with DBBUILD, and you can also use a config-file as input to DBUPDGEN to build an update script for several subsystems. You also use config-files with SSGREP and SSREPLACE to specify where you want to search, although the interpretation of the file is somewhat different in the latter case.

A config-file may reside in version-control or on disk. When the config-file is under version-control, you refer to it through its VC-path, for instance AbaBos32/$/AbaBos/4.40/AbaSec.cfg. If you specify a directory, AbaPerls looks for config.cfg.

There is a command, LISTCONFIG that reads a config-file and lists the contents of it. This is a useful debug tool.

Details on the System-Definition File

Name, Location and General Format

A sysdef-file is always named SYSTEM.DEFINITION and is stored in the version directory (that is, a directory with the name on the form Major.Middle in the version-control system).

The grammar for a sysdef-file is as follows:

{
[global-option [global-option...] | . global-option] [comment]
[...]
subsys-entry 
[subsys-entry]
[...]
} |
$REDIRECT VC-path [comment]

That is, a file consists of either subsystem entries, possibly preceded by global options, or by a single $REDIRECT command. Details on the elements:

global-option    ::= -[no]restricted_references | -[no]enforce_reforder

subsys-entry     ::= subsys-name VC-path [subsys-option [subsys-option...]] [comment]
                     [subsys-option [subsys-option...] | . subsys-option] [comment]
                     [...]

subsys-name      ::= A-Z[-_A-Z0-9...]

VC-path          ::= absolute-VC-path | relative-VC-path

absolute-VC-path ::= ["]VC-sys::[repository/]$/directory-path["]

relative-VC-path ::= ["]directory-path["]

subsys-option    ::= -[no]allmayref | 
                     -mayrefto subsys-name [subsys-name ...] |
                     -maynotrefto subsys-name [subsys-name ...] 

comment          ::= -- any-text

Or in text: Disregarding blank lines and comments starting with -- each line in the system-definition file contains of two mandatory fields, the subsystem name and the VC-path for the subsystem. As the example indicates, the VC-path must be enclosed in double quotes if it contains spaces. Options can follow directly on same line as the subsystem name or VC-path, or they may appear on the next line.

In the following sections, we will look more close at the components, but as a start, here is an example of a system-definition file.

-- This is an example
-restricted_references
ABAPERLS     data6/$/AbaPerls   
ABAEVENTLOG  AbaEventLog -allmayref
BLANKTEST    "Blank Test" 
KATI         TOFSIE/test/$/AbaPerls/Kati/2.70 -mayrefto BLANKTEST
MQF          "Gateways/MQF Gateway"
EXTRA        Extra
. -mayrefto BLANKTEST KATI

The Subsystem Name

The subsystem name is a string with a leading uppercase letter followed by zero or more characters that must be uppercase letters, digits, hyphen or underscore. The name must not be longer than 30 characters. The name NAMELESS is reserved and cannot be used. Furthermore, the name for AbaPerls as a subsystem should be ABAPERLS and nothing else. The subsystem name must be unique in the file.

The subsystems should come in dependency order. That is, if subsystem B depends on A, A should come before B. Your first subsystem will almost always be ABAPERLS.

A word of warning: once you have labelled subsystems and started to build subsystems, you should be cautious with renaming subsystems, even if the name proves to have embarrassing spelling errors or to be offensive to some users. AbaPerls looks up the sysdef-file per label as discussed below, and if the same subsystem has different names at different labels this is bound to cause confusion. This will be even more emphasised even more in the future, when AbaPerls hopefully supports include between subsystems.

Adding new subsystems is of course permissible, as is removing an old subsystem, once all references to it have been removed.

The VC-path

The VC-path specifies where the SQL directory for the subsystem is located. The path may include a trailing /SQL, but it can be left out. The path must never include /SQL/ as a middle part.

The VC-path can be absolute or relative. A path is absolute if it includes the string $/. In full, an absolute path consists of a designator for the version-control system (VSS or TFS), the repository and the path for the directory within the repository. As when you specify a VC-path, AbaPerls will attempt to fill in the parts that are missing. For instance, in the sample above, the full path for the ABAPERLS subsystem is VSS::S:\Data6\data\/$/AbaPerls/SQL.

When a path is relative, AbaPerls appends the path to the path of the system-definition file. That is, if the example sysdef-file above is in TFS::TOFSIE/test/$/AbaPerls/Test projects/1.00, the VC-path for the subsystem ABAEVENTLOG is TFS::TOFSIE/test/$/AbaPerls/Test projects/1.00/AbaEventLog/SQL and the MQF subsystem is in TFS::TOFSIE/test/$/AbaPerls/Test projects/1.00/Gateways/MQF Gateway/SQL.

If the path includes a space, you need to enclose the path in quotes, no matter you have a relative or an absolute path.

AbaPerls uses the sysdef-file to determine the subsystem given a VC-path. For this reason, AbaPerls verifies that a path unique within the file. More precisely the part after $/ must be unique. That is, you cannot have two subsystems with the path $/a/b/c in two different repositories. (AbaPerls performs the lookup on the path within the repository, since the path to a repository can be expressed in multiple ways.)

If you use version-before-subsystem, save for external subsystems like ABAPERLS, all paths in the sysdef-file can be relative, which means that you don't have to change the file when you move to a new version. You only need to change it to add or remove a subsystem. However, it is conceivable that you use components that have their own life-cycle. For instance, assume that in the middle of the development of 1.00 of the Test Projects, a decision is made to move to a more recent version of the KATI subsystem. In this case, you edit the system-definition file so that the KATI line ends with 2.80 instead. You should be careful to label all your subsystems when you do this, to get a clear division point.

Sysdef-file Options

While the options in the sysdef file are decoded with Getopt::Long like the command-line options, the options in the sysdef-file must be spelt out in full. They are case-sensitive.

Options are global options until there is a line starts with a letter A-Z which marks the first subsystem entry. When you enter options for a subsystem, you can enter them after the VC-Path, or you can enter them on the following line. You can split the options on several lines if you like. If an option takes a parameter, the parameter and the option must be on the same line, though.

The rule is that as long as a line starts with a hyphen or a period it is a line with options. (Lines that start with two hyphens are comments of course.) If a line starts with a period followed by a space and an option, all text following the option is a parameter to that option. This is useful with the options ‑mayrefto and ‑maynotrefto

Currently, all options you can use in the sysdef-file control how subsystems may refer to each other. These rules are enforced whenever a file is loaded and the AbaPerls tool in question reads the sysdef-file. That is, the rules will typically be enforced when a developer loads a new version of a stored procedure with ABASQL, or you build or upgrade a test database at home, but not when you make installations at customer sites.

The default is that a subsystem is permitted to refer to any subsystem on an earlier line in the file, but references to subsystems later in the list are not permitted. The idea is that the sysdef-file mirrors the build order: the first subsystem in the list is the first loaded and so on. You can change this overall behaviour with the two options ‑[no]restricted_references and ‑[no]enforce_reforder.

When ‑restricted_references is in force (the default is ‑norestricted_references), a subsystem may only refer to another subsystem, if it is explicitly permitted through any of the subsystem options ‑allmayref or ‑mayrefto. That is, if you add ‑restricted_references to the top of the file, and do nothing more, subsystems may not refer to each other at all. The reason that you would use ‑restricted_references is that you ship different subsystems to different customers. If customer A gets subsystem Y, but not subsystem X, you should not permit Y to refer to X, even if X comes before Y in the build order.

When ‑enforce_reforder is in force, subsystem references must follow the order in the file. That is, a subsystem is not permitted not refer to a later subsystem, unless there is an explicit permission with ‑mayrefto. This is the default, and you can turn it off with ‑noenforce_reforder. Note that if you do this, you are likely to get errors when you run DBBUILD. If all wrong-order references are stored procedure calls, you can be saved from seeing these errors with LISTERRS if you use the ‑lax option. If you have stored procedures that refer to tables in later subsystems, it is likely that you need to run DBBUILD followed by DBBUILD ‑rebuild to build the database without errors. The option ‑noenforce_reforder exists in case you have a system where references in general are out of order, and you are not in position to fix this for the moment. ‑noenforce_reforder can also be useful when you are experimenting to determine the best order for your subsystems.

Note that you can also use ‑mayrefto to override the order for a specific subsystem.

There are three options you can set on subsystem level. The first of these options is ‑[no]allmayref. When ‑allmayref is in force, all later subsystems in the file are permitted to refer to this subsystem. (Or all subsystems if ‑enforce_reforder is not in force.) The default for ‑allmayref is the reverse to the setting for ‑restricted_references. That is, ‑norestricted_references implies ‑allmayref for all subsystems, whereas ‑restricted_references implies ‑noallmayref for all subsystems.

The option ‑mayrefto lists subsystems this subsystem is explicitly permitted to refer to. Commonly you will want to list multiple subsystems, but you need to be careful and use any of these notations:

EXTRA        Extra -mayrefto "BLANKTEST KATI"
EXTRA        Extra
. -mayrefto BLANKTEST KATI

That is, you either need to put the list in double quotes, or put the option on a line of its own preceded by a single period. As discussed above, this states that the line is one single command-line option.

‑mayrefto and ‑allmayref supplement each other. In the example file above, the subsystem EXTRA may refer to ABAEVENTLOG, BLANKTEST and KATI, but not to ABAPERLS or MQF.

‑mayrefto overrides all other settings, and it's legal to add a subsystem which comes later in the sysdef-file, even if ‑enforce_reforder is in effect.

The option ‑maynotrefto lists subsystem which the subsystem is not permitted to refer to. The format is the same as for ‑mayrefto; that is put the list in quotes or put the option on a separate line preceded by a dot. ‑mayrefto takes precedence over other settings, including ‑allmayref. If the entry for a subsystem A includes both ‑mayrefto and ‑maynotrefto and a subsystem B is included in both lists, that is an error.

If you misspell an option name in the sysdef-file, this renders a warning, but the command will still execute.

Redirection

A system-definition file can redirect to another file, as in this example:

$REDIRECT "TOFSIE/test/$/AbaPerls/Test Projects/1.00"

A $REDIRECT directive cannot be combined with subsystem definitions. That is, save for blank lines and comments, the $REDIRECT directive must be alone in the file. AbaPerls only supports a redirection in one level. If a file A redirects to file B and B redirects to C, this is an error.

As you see from the example, you specify a directory for the $REDIRECT directive. AbaPerls automatically adds SYSTEM.DEFINITION to the path you provide. If AbaPerls is not able to access the file, this is an error.

Note: while $REDIRECT may look like a Preppis directive, it is only understood in the context of sysdef-files, and sysdef-files are not run through Preppis.

How AbaPerls Finds the Sysdef-file

AbaPerls always accesses the sysdef-file through version control. When you use ABASQL, DBBUILD or DBUPDGEN and specify the ‑VC option, AbaPerls searches the VC-path for a component that is on the form major.middle and looks for SYSTEM.DEFINITION in that version directory. That is, if you specify the VC-path as TFSSrv/$/Project/2.20/Alfa/Beta, AbaPerls will attempt to access TFS::TFSSrv:8080/tfs/DefaultCollation/$/Project/2.20/SYSTEM.DEFINITION. If the file is missing, this is not an error, and AbaPerls will let it pass silently. Note however, that if the file exists, but redirects to a file which is not accessible, this is an error.

If you don't specify ‑VC, and neither specify ‑config, AbaPerls will set the ‑VC option implicitly if your current Windows directory is mapped to a directory in some TFS workspace. (AbaPerls never considers working directories for SourceSafe.)

When you specify the ‑config option with DBBUILD and DBUPDGEN, AbaPerls never looks up the system-definition file as described above. Instead the config-file should specify the system-definition file as described below.

If you don't specify a version specification with ‑label, AbaPerls reads the most recent version of the sysdef-file. When you use ‑label, AbaPerls reads the version of the system-definition file that was current when the label was created. When you use ‑label, you can specify a label, a date or a version number as discussed below. However, if you arrive at a system-definition file that redirects, and you have specfied something else than AbaPerls label as the argument to ‑label, you will get an error.

If you do not specify a version with ‑label (or specify LATEST), and you have the file checked out, AbaPerls will attempt to read your local version of the file. Note that AbaPerls does not consider which computer you checked out the file from, but will consider all your checkouts. If you have checked out the file multiple times, it is not deterministic which file AbaPerls attempts to access.

Config-file Overview

Syntax

The grammar for the config-file follows the same pattern as the sysdef-file. Here is the formal grammar.

[$SYSDEF VC-path]
[option [option...] | . option] [comment]
[...]
subsys-entry [comment]
[subsys-entry] [comment]
[...]

Where:

subsys-entry     ::= subsys-name [VC-path] [version-spec] [option [option...]]
                     [option [option...] | . option]
                     [...]

subsys-name      ::= A-Z[-_A-Z0-9...]

VC-path          ::= "" | Absolute-VC-Path

Absolute-VC-path ::= ["]VC-sys::[repository/]$/directory-path["]

version-spec     ::= date | label  | LATEST | NULL

option           ::= -[no]crypt | -environment {DEV | TEST | PROD} |
                     -Macro &macro[=value] | -[no]quoterepl |
                     -site site-id[:site-id...] | -undef &macro 

comment          ::= -- any-text

That is, you first specify the path for the system-definition file, if you have one. Next comes global configuration options, and then follow any number of subsystem entries. A subsystem entry defines the name of the subsystem, its path, its version and configuration options. In difference to the sysdef-file, the set of available options is the same on global level as on subsystem level.

On this page we discuss the grammar of the config-file and the semantics on the subsystem entry. A separate page details the semantics of the various configuration options.

The $SYSDEF directive

When you specify a config-file, AbaPerls never looks up the sysdef-file implicitly from any path, but the only way to access a sysdef-file is through the $SYSDEF directive to get a fully deterministic behaviour for a given config-file.

As with the $REDIRECT directive, you only specify the directory path for the system-definition file, and AbaPerls adds SYSTEM.DEFINITION. You can use a different file name, and you can also use a path on disk. But the only reason to do this is to experiment or prototype. Beware that if you specify the sysdef-file as a disk path, you can only use absolute VC-paths.

If AbaPerls cannot find the file you specified with $SYSDEF, this is an error.

When accessing the sysdef-file, AbaPerls follows these rules to determine which version to use:

  1. If you do not specify ‑label or specify LATEST, AbaPerls will retrieve the most recent version of the sysdef-file. If you have the file checked out, AbaPerls will attempt to access the disk copy as discussed above.
  2. If you specify a label, AbaPerls will retrieve the version that was recent version when that label was set.
  3. If you specify a version by other means, AbaPerls will use the version that was recent when the config-file was checked in.

Using $SYSDEF in config-files to be used with DBBUILD and DBUPDGEN is highly recommendable, as the config-file can inherit the path from the sysdef-file. On the other hand, config-files used with SSGREP or SSREPLACE are loser in nature, and there is little point in using $SYSDEF in such files.

The Subsystem Entry

A subsystem entry defines:

The subsystem entries should follow in dependency order: if subsystem B refers to subsystem A, B should come after A. Normally, in a config-file used for building a database, the first subsystem is ABAPERLS itself. When you use $SYSDEF, the subsystems should normally follow the order in the sysdef-file. (The only reason they would not is that you are experimenting.)

Name

Subsys-name follows the same rules as in the system-definition-file. That, a string of most 30 characters with a leading uppercase followed by zero or more characters that must be uppercase letters, digits, hyphen or underscore. The name must not be longer than 30 characters. The name NAMELESS is reserved and cannot be used. Furthermore, the name for AbaPerls as a subsystem should be ABAPERLS and nothing else.

If the config-file includes a $SYSDEF directive, you can only use names that appears in the sysdef-file. You cannot add subsystems that are not in the sysdef-file.

VC-Path

VC-path specifies the location of the subsystem. It is only optional when you use the $SYSDEF directive; in this case AbaPerls will use the path for the subsystem in the sysdef-file. An empty VC-path, "", has the same effect. You need to use an empty path as a placeholder if you want to specify a version specification or a configuration option for the subsystem.

When specified, the VC-path is an absolute VC-path which follows the same format as in the system-definition file. (You cannot use relative paths.) That is, in full it consists of a designator for the version-control system (VSS or TFS), the repository and the path for the directory within the repository. As always, AbaPerls will attempt to fill in the parts that are missing in the repository specification.

If the config-file includes a $SYSDEF directive, and you yet specify an explicit path for a subsystem, the path in the config-file takes precedence.

When you use a config-file with DBBUILD or DBUPDGEN, you always specify the path to the SQL directory, explicitly or implicitly. That is, AbaPerls tacks on /SQL if the path ends in something else. That is, if you specify TFSSRV/$/Custreg/4.40 in your config-file, DBBUILD and DBUPDGEN will work with TFSSRV/$/Custreg/4.40/SQL as the VC-path. This does not apply to SSGREP and SSREPLACE. They use the paths as given, and they do not have to be limited to SQL directories, as you may want to search client code as well.

The Version Specification

This field can have any of these values:

When you do not specify a version-spec, AbaPerls uses the config-file as a starting point on how to find the version of the directory. If you retrieved the config-file from version-control by specifying ‑label LATEST, then AbaPerls applies LATEST to all directories. In all other cases, AbaPerls will use a labelled version of the directory. If you specified a config-file on disk, or you specified a config-file in version-control without specifying an explicit version of the config-file, AbaPerls use the most recent AbaPerls label (i.e. the format LetterMajor.Middle.Minor) for each subsystem directory. If you instead specify a certain version of the config-file – by label, date or version number – AbaPerls will first translate that version to a point in time, and then for each subsystem find the label that was the most recently created at that occasion.

By default, AbaPerls only considers labels that adhere to the standard AbaPerls label format. You can use the configuration options ‑onlylabel and ‑notlabel to alter these rules.

While the rules for a blank version specification are complex, they are devised to give you what you want. The tool LISTCONFIG is available to let you see what versions you get from a certain config-file.

Note that for test and production databases – as controlled by the configuration option ‑environment – AbaPerls only permits its own standard labels.

With the tools  SSGREP and SSREPLACE, the version specification does not matter, because these tools always retrieve the config-file by version LATEST.

Configuration options

When you build a database with DBBUILD you specify configuration options in a config-file or on the command line. The settings of the options are saved in the database if the AbaPerls system tables are present in the database. Later, when in you invoke ABASQL or an update script generated by DBUPDGEN, they will read these settings and apply them, as if you had specified them on the command line.

The rules for entering configuration options are the same as for options in the sysdef-file. Configuration options that come before the first subsystem entry are global configuration options that apply to all subsystems, unless explicitly overridden for a certain subsystem.

When you enter configuration options for a subsystem, you can enter them after the version specification, or you can enter them on the following line. You can split the options on several lines if you like. As long as a line starts with a hyphen or a period it is a line with configuration options.

If a line starts with a period followed by a space and a configuration option, all text following the configuration option is a parameter to that option. This is necessary when the argument includes space characters or characters that could confuse the option parser. The typical case for this is macro definitions. Thus, this line defines the macro &nisse to have the value use ‑crypt.

. -Macro &nisse=use -crypt

While this line defines &nisse to have the value use and also sets the option ‑crypt.

-Macro &nisse=use -crypt

With the exception of the options ‑onlylabel and ‑notlabel, the configuration options apply only to DBBUILD. ‑onlylabel and ‑notlabel apply also to DBUPDGEN and LISTCONFIG. However, all tools will complain if there are incorrectly specified options in the config-file.

For a description of what each configuration option controls, please see the page Configuration Options.

Config-file Examples

Assume the config-file VSS::Data6/$/Test/Testconfig.cfg:

-- This is a config-file for demonstration purposes.
-Macro &nisse
ABAPERLS   data6/$/AbaPerls    LATEST
TEST       data6/$/test/4.40   -site charlie  -crypt
. -Macro &colname=[space name]
KATI       test2/$/kati/1.0 -undef &nisse

When we build a database from this config-file, this database will contain the subsystems ABAPERLS, TEST and KATI.

For the subsystems ABAPERLS and TEST the macro &nisse is defined with the value of the empty string. The macro will also be defined by default for any new subsystems that we may add later. For KATI, &nisse is not defined, however, as there is an explicit ‑undef option for this subsystem, overriding the setting on database level.

For the subsystem TEST only, the macro &colname is defined with the value [space name]. (Note: in this example, the brackets are not marking any optional entry, but part of the value for the macro &colname.) Furthermore, only in TEST procedures, views, triggers and functions will be obfuscated (‑crypt), and files with the site-id @charlie will be loaded only for the TEST subsystem.

Then the following events takes place:
2011-02-22, 14:40:40 Labelling of Data6/$/test/4.40 with L4.40.0010
2011-02-22, 16:40:10 Labelling of Test2/$/Kati/1.0 with L1.0.0010
2011-02-22, 18:25:21 Labelling of Test2/$/Kati/1.0 with SPECIAL
2011-02-23, 09:13:35 Check-in of Data6/$/Test/Testconfig.cfg, version 7
2011-02-23, 12:35:16 Labelling of Data6/$/Test/4.40 with L4.40.0020
2011-02-22, 14:09:46 Labelling of Test2/$/Kati /1.0 with L1.0.0020

We get a copy of the config-file to C:\temp. W use LISTCONFIG to see what the config-file actually defines. The commands:

listconfig c:\temp\testconfig.cfg
listconfig data6/$/test/testconfig.cfg
both yield:
Subsystem    Label          Version Date                Path
---------    -----          ------- ----                ----
ABAPERLS                            Current             $/abaperls
TEST         L4.40.0020          10 2011-02-23 12:35:16 $/test/4.40
KATI         L1.0.0020           10 2011-02-26 15:55:00 $/kati/1.0

Thus, we get the most recently labelled version of the subsystems, except for ABAPERLS. On the other hand, if we say:

listconfig data6/$/test/testconfig.cfg 7 

We get:

Subsystem   Label          Version Date                Path
---------   -----          ------- ----                ----
ABAPERLS                           Current             $/abaperls
TEST        L4.40.0010           9 2011-02-22 14:40:40 $/test/4.40
KATI        L1.0.0010            9 2011-02-22 16:40:10 $/kati/1.0

Despite we are referring to the same version of Testconfig.cfg in the two cases, we get different results. In the latter example we explicitly stated which version of the config-file we wanted, and thus we got the labelled versions of TEST and KATI that were the most current when Testconfig.cfg was checked in. When we did not specify the version for the config-file, we got what was the most recently labelled at the time of the command. Note also that the label SPECIAL was ignored. To include this label, we would have to add the

If we say

listconfig data6/$/test/testconfig.cfg LATEST 

we get:

Subsystem   Label          Version Date                Path
---------   -----          ------- ----                ----
ABAPERLS                           Current             $/abaperls
TEST                               Current             $/test/4.40
KATI                               Current             $/kati/1.0

Here LATEST applies to all subsystems.

Note: in these examples we have used LATEST as the version specification for ABAPERLS for the matter of the example only. When you build a database which you later intend to upgrade, you should use a blank version specification with ABAPERLS as you do with any other subsystem.