Enabling metrics from Notakey Authentication server

Latest versions of our Authentication server provide various operational metrics that allow monitoring service usage and creating alerts for anomalies. Metrics are in Prometheus format and available directly from the Notakey Appliance node.

Statistics are aggregated by the stats-agent plugin that has to be configured on one or more nodes. A configuration with a VRRP address for metrics is also possible.

# minimal plugin configuration

ntk cfg set :plugins.\"stats-agent\" '{
   "ports": [
      "9394:9394"
    ],
    "tag": "statsagent"
  }' --json-input

Run ntk plugins update to update all plugin container images, followed by ntk plugins restart.

After the agent is configured you need to enable stats on the Authentication server:

ntk cfg set :nas.custom_params "ENABLE_STATS=1,STATS_HOST=stats-agent"
ntk as restart

Now metrics from the Authentication server will be accessible on http://<internal-ip>:9394/metrics.

Some metrics to measure:

ntk_as_auth_requests_created: Number of created auth requests per service
ntk_as_auth_requests_failed: Number of failed creations of auth requests
ntk_as_auth_requests_signed: Number of signing events for auth requests per service
ntk_as_api_auth_success_total: Number of successful API client authentication requests
ntk_as_api_auth_failures_total: Number of failed API client authentication requests
ntk_as_api_token_auth_fail_total: Token auth validation requests that failed
ntk_as_api_tokens_emitted_total: Number of issued authentication tokens
ntk_as_seats_issued_total: Total number of issued licensed seats
ntk_as_seats_total: Total number of available licensed seats
ntk_as_seat_destroy_count: Number of license seat removal events
ntk_as_seat_create_count: Number of license seat create events
ntk_as_http_duration_seconds_count: Request count of HTTP reqs, per endpoint
ntk_as_http_duration_seconds_sum: Time spent in HTTP reqs in seconds, per endpoint

To configure the stats-agent plugin to use a VRRP IP address use the following commands:

# VRRP service must be configured on every node running stats-agent

# 172.17.0.1 is the default internal bridge IP address, available on every node
ntk cfg set vrrp.track.generic.url 'http://172.17.0.1:9394/metrics'

# Adjust IP address and select different priorities for each node
ntk cfg set vrrp.services.stats  '{
      "prio": "100",
      "vrid": "54",
      "interface": "eth0",
      "address": "10.0.1.97",
      "preempt": "off",
      "track": [
        "generic"
      ]
    }' --json-input

# Any VRRP config changes require restart
ntk vrrp restart

After VRRP configuration, instruct the authentication server to publish stats to the VRRP IP by setting STATS_HOST=10.0.1.97 and restarting the service (ntk as restart).