10. Migrating from LinOTP 2 to LinOTP 3

The jump to version 3 of LinOTP brings several breaking changes. To ensure a smooth migration, please read the following sections.

10.1. Configuration

The .ini files used formerly have been replaced with .cfg files, with most configuration parameters renamed (due to restrictions imposed by the Flask web server). Configuration parameters can now also be set via the process environment, for increased convenience in container-based setups (e.g., Docker). This is done by prefixing their names with LINOTP_.

For example, the database configuration URL previously found in the .ini file as sqlalchemy.url can now be found in the DATABASE_URI setting, and the respective environment variable is LINOTP_DATABASE_URI.

10.1.1. Configuration directory

The settings directory /etc/linotp2 is renamed to /etc/linotp.

10.1.2. Configuration file

Configuration settings will now be loaded from multiple locations. These locations are determined by the value of the LINOTP_CFG environment variable, which may contain a colon-separated list of directory names and file glob expressions. LinOTP checks these names and file glob expressions in the order they occur in LINOTP_CFG, and reads any matching files. File names resulting from the expansion of a glob expression are processed in lexicographic order. Directory names have *.cfg appended to them implicitly. Relative directory names and file glob expressions are taken relative to the application root directory (where the app.py file is found).

If LINOTP_CFG is not set and a file called linotp-cfg-default exists in the application root directory, the content of that file is used as the default value for LINOTP_CFG (this feature is generally used to establish a distribution-specific search order for configuration files). If the file does not exist, linotp.cfg is assumed as the default value.

10.1.3. Configuration file locations for Debian packages (as per linotp-cfg-default)

File path

Description

/usr/share/linotp/linotp.cfg

Distribution defaults - these settings are supplied by the packaging and represent settings that are appropriate for the distribution and packaging format used.

/etc/linotp/linotp.cfg

Central configuration settings.

/etc/linotp/conf.d/*

Configuration snippets can be placed in this directory. The Debian packaging places the database configuration in 01-debian-database.cfg

The order of the locations in the above table matters. Settings in later files in the search order will override settings previously found. For example, if LOGFILE_DIR is defined in the following places:

LOGFILE_DIR=/srv/linotp-logs in the file /etc/linotp/linotp.cfg and

LOGFILE_DIR=/var/log/linotp in the file /usr/share/linotp/linotp.cfg,

then the value that is used is /srv/linotp-logs, because that was the last one encountered.

10.1.4. New configuration settings

Configuration settings have been renamed in LinOTP 3. The existing configuration entries in a LinOTP instance are migrated during the upgrade from version 2 to version 3. You can find an overview of the changes in the following table:

Old .ini name

New .cfg name

Details

sqlalchemy.url

DATABASE_URI

If you are using the Debian packages this setting will be automatically migrated during upgrade

linotpSecretFile

SECRET_FILE

linotpAudit.sql.url

AUDIT_DATABASE_URI

When set to the new default value of SHARED, linotp will write the audit log to DATABASE_URI

linotpAudit.sql.pool_recycle

AUDIT_POOL_RECYCLE

linotpAudit.key.private

AUDIT_PRIVATE_KEY_FILE

linotpAudit.key.public

AUDIT_PUBLIC_KEY_FILE

linotpAudit.error_on_truncation

AUDIT_ERROR_ON_TRUNCATION

linotpadmin.username

ADMIN_USERNAME

linotpadmin.password

ADMIN_PASSWORD

linotpGetotp.active

GETOTP_ENABLED

linotpHelp.url

HELP_URL

linotpTokenModules

TOKEN_MODULES

cache_dir

CACHE_DIR

DATA_DIR/template-cache

Generated Mako templates previously found under cache_dir are now under the new DATA_DIR directory

custom_templates

CUSTOM_TEMPLATES_DIR

sqlalchemy.echo

logging

logger_*

handler_*

formatter_*

LOGGING

See the new logging section of the documentation

service.*

CONTROLLERS

maintenance_verify_client_env_var

MAINTENANCE_VERIFY_CLIENT_ENV_VAR

radius.nas_identifier

RADIUS_NAS_IDENTIFIER

debug

DEBUG (not completely equivalent)

See the developer documentation for details of how this setting has changed

Configuration settings whose names end in _FILE or _DIR contain file or directory names, respectively. These names can be either absolute (starting with a slash) or relative, in which case they are evaluated (at the time of use, not when the configuration is being read) starting from the directory named by the ROOT_DIR configuration setting. This setting must contain an absolute directory name.

10.1.5. Removed settings

The following settings have been removed from the configuration file and are no longer supported. Many of them are only meaningful in the context of the old Pylons-based implementation and have no equivalent in the new Flask-based implementation.

In the default section:
  • unitTest

  • profile

  • smtp_server

  • email_error_from

  • radius.dictfile

  • linotpAudit.sql.highwatermark

  • linotpAudit.sql.lowwatermark

  • linotpAudit.sql.table_prefix

  • linotpNoSessionCheck

  • linotp.uencode_data

  • linotp.DefaultSyncWindow

  • linotp.DefaultOtpLen

  • linotp.DefaultCountWindow

  • linotp.DefaultMaxFailCount

  • linotp.FailCounterIncOnFalsePin

  • linotp.PrependPin

  • linotp.DefaultResetFailCount

  • linotp.splitAtSign

In the server section:
  • use

  • host

  • port

  • ssl_pem

The host, port, and ssl_pem settings are obsoleted by the fact that LinOTP is run productively as a WSGI application, i.e., these settings should be configured in the web server (e.g., Apache) or WSGI application server (e.g., uWSGI). During development they can be set using the Flask development server’s CLI.

In the app:main section:
  • use

  • full_stack

  • static_files

  • who.config_file

  • who.log_file

  • who.log_level

  • beaker.session.key

  • beaker.session.secret

  • licfile

10.2. Database encoding migration

Starting with LinOTP2 there has been support for mysql. Mysql in its early version did not support the today standard utf8 encoding to store non ascii data. Instead mysql did use the ISO latin encoding on top of a python 2 which as well did not fully support utf8. During the years the support for utf8 increased and mysql and the python 2 db driver layer changed the standard encoding to utf8.

Thus there are cases when you migrate your lintop server from an early version to the 3.2 version of linotp & database & database driver, you will see problems with the encoding eg. when looking at the manage ui or in your linotp log files.

To support you with this sitation there is a linotp command available which is

linotp admin fix-db-encoding

which will convert the mysql data and database schema from the ISO-8859-1 encoding to the utf8 standard encoding. The command

linotp admin fix-db-encoding –help

will describe which data in your database will be migrated.

But be careful with this command and apply this command only if you have some log entries which show up hints for a conversion. And as always: do a backup of your data upfront before doing any data changing command.

10.3. Administrator authentication

LinOTP no longer needs to rely on the authentication mechanisms of web servers (such as Apache’s htaccess) for securing access to the management interface and management API.

There is a dedicated realm containing a dedicated resolver for defining administrator accounts. You can add an account to this resolver by executing linotp local-admins add <username> and linotp local-admins password <username>. Further documentation on the linotp local-admins command can be found in the respective man page. You can also add other resolvers to the administrator realm.

The management API authentication is done by calling the http://<linotp-server>/admin/login API endpoint with username and password in the body (see LinOTP’s API documentation). The response returns a JSON Web Token (JWT) which has to be used for all management API calls.

Management API endpoints which modify data or the server state can only be accessed via HTTP POST request (see the full list below). In addition to the JWT for authentication, these endpoints require a X-CSRF-TOKEN field in the header. Its value is provided by the csrf_access_token cookie.

10.4. API endpoints requiring POST requests

Following LinOTP API endpoints are now accessible via HTTP POST requests only. This is to prevent data or server state modification via cross-site request forgery (CSRF).

  • admin/assign

  • admin/copyTokenPin

  • admin/copyTokenUser

  • admin/disable

  • admin/enable

  • admin/init

  • admin/loadtokens

  • admin/losttoken

  • admin/remove

  • admin/reset

  • admin/resync

  • admin/set

  • admin/setPin

  • admin/setValidity

  • admin/tokenrealm

  • admin/unassign

  • admin/unpair

  • maintenance/setLogLevel

  • migrate/backup

  • migrate/restore

  • reporting/delete_all

  • reporting/delete_before

  • system/delConfig

  • system/delPolicy

  • system/delProvider

  • system/delRealm

  • system/delResolver

  • system/importPolicy

  • system/setConfig

  • system/setDefault

  • system/setDefaultProvider

  • system/setDefaultRealm

  • system/setPolicy

  • system/setProvider

  • system/setRealm

  • system/setResolver

  • system/setSupport

  • system/setupSecurityModule

  • tools/import_users

  • tools/migrate_resolver

  • tools/setPassword

  • userservice/activateocratoken

  • userservice/assign

  • userservice/delete

  • userservice/disable

  • userservice/enable

  • userservice/enroll

  • userservice/finishocra2token

  • userservice/reset

  • userservice/resync

  • userservice/setdescription

  • userservice/setmpin

  • userservice/setpin

  • userservice/unassign

  • userservice/webprovision

For detailed information on each endpoint please have a look at the LinOTP API documentation.

10.5. User interface

10.5.1. Audit trail tab of ManageUI

In the Audit trail, the times are expressed in UTC. The Column header expresses this clearly by “Date (UTC)” so we make sure we are all on the same page. If you have any script that uses the column/header to read data from this table, you might need to consider readjusting it.