Migration scripts

From VyOS Wiki
Jump to: navigation, search

To make it possible to change configuration syntax without breaking forward compatibility, VyOS uses a syntax migration mechanism. Configuration syntax is versioned, and if current version does not match the version specified in the configuration file to be loaded, a set of migration scripts is run to convert it to the current version.

Syntax versioning

Syntax versioning is modular rather than monolithic, and the syntax of each component is versioned independently.

Versions in the config file

In the configuration files, component syntax versions are stored in the trailing comments at the end of the tile.

For example:

/* Warning: Do not remove the following line. */
/* === vyatta-config-version: "broadcast-relay@1:cluster@1:config-management@1:conntrack@1:conntrack-sync@1:dhcp-relay@2:dhcp-server@5:firewall@5:ipsec@5:l2tp@1:mdns@1:nat@4:ntp@1:pptp@1:qos@1:quagga@3:ssh@1:system@9:vrrp@2:vyos-accel-ppp@1:wanloadbalance@3:webgui@1:webproxy@2:zone-policy@1" === */

/* Release version: 1.2.0-rolling+201904020337 */

In this example, the version of the "dhcp-relay" syntax is 2, while the version of the "firewall" syntax is 5.

If a version is not present, it's assumed to be 0.

Versions on the running system

Current syntax versions are stored as empty files in the /opt/vyatta/etc/config-migrate/current/ directory. The file names follow the same format as versions in the config file, for example, presence of a file named "firewall@5" means the current version of the firewall syntax is 5.

On a running system, the correct trailing comments for the current version can be generated with the /opt/vyatta/sbin/vyatta_current_conf_ver.pl script that comes from vyatta-config-migrate package.

The migration process

Before configuration is loaded, /opt/vyatta/sbin/vyatta_config_migrate.pl script from the vyatta-config-migrate package is run.

If version of a component does not match, scripts from the /opt/vyatta/etc/config-migrate/migrate/ directory are run with the config file path as an argument.

For example, suppose a config with "nat@1" version is loaded on a system where current version is 4. The migration script runner will execute the following scripts in a sequence:

  • /opt/vyatta/etc/config-migrate/migrate/nat/1-to-2
  • /opt/vyatta/etc/config-migrate/migrate/nat/2-to-3
  • /opt/vyatta/etc/config-migrate/migrate/nat/3-to-4

Future plans

Deprecation of the old scripts

VyOS 1.2.0 will be the last release to support configuration files from Vyatta before 6.5 (released in 2012).

Version data storage improvements

The current system is rather clunky, but it has an advantage of treating VyOS built-ins and third party add-ons in the same way and makes it easy for add-ons to include configuration versions and take advantage of the migration mechanism.

One idea is to store all versions of VyOS built-in components in a single JSON file or similar. The question is how to make it possible for third-party components to store their version too?

Moving the scripts out of /opt

New VyOS code goes to /usr/libexec/vyos. We need to design a directory layout for the migration scripts there.

Rewriting the script

The current script is in Perl and is rather messy. It should be redone in Python and with cleaner logic.

Changing the trailing comment syntax

The trailing comments have multiple issues:

  • They have the same syntax as node comments, which is problematic for the parser
  • They are technically not even a part of the config grammar
  • They still refer to Vyatta

Can we improve them? For example:

  • Introduce a different syntax for comments that are to be ignored by the loader, like shell-style single line comments
  • Change the keyword to "vyos-config-version"
  • Use presence of "vyatta-config-version" rather than "vyos-config-version" to tell old and new config apart and use appropriate version extraction logic in the migrator