2021 Update Rollup 1

Release NotesPermanent link for this heading

Fabasoft app.telemetry 2021 UR 1Permanent link for this heading

Full-text search in Research viewPermanent link for this heading

If you are looking for a particular text in the telemetry points within a particular time interval, you can use the new full-text search field in the Research view to identify all the requests containing this specific search string.

Anonymization of personal dataPermanent link for this heading

Specific columns containing personal data (e.g. Loginname, user group) can be automatically anonymized by setting a corresponding “anonymous” flag in the Log Definition. Additionally, the Data Anonymization has to be activated in the Log Pool Properties, where also the number of days is set after which data is anonymized.

The entries for a column to be anonymized are encoded using a random GUID which changes every day. This GUID is written to the database instead of the real entry and the information which is used to decode the GUID into the real value again is stored only for the number of days specified for the anonymization. Therefore, for personal data older than the specified number of days only the GUID can be read, enabling to still distinguish between e.g. different users, but not showing the directly related personal information any more. For younger data the real values as described in the Log Definition are shown enabling you to still read the personal data directly for this specific time period.

If the anonymization is activated for an already existing Log Pool without former anonymization, older data won’t be anonymized and new data arriving after the activation will be stored in a new table with the corresponding anonymization. Personal data contained in the raw data files, in sessions or feedback dialogs is not anonymized.

The anonymized data encoded by the GUID is also considered for the statistics accordingly.

In addition to the anonymization of columns containing personal data, such columns can be hidden altogether from specific users or groups by entering such users and groups in the Log Pool Properties next to “Hide Personal Data from User/Group”.

Support of multiple telemetry services in C/C++ Client ApplicationsPermanent link for this heading

Applications like Fabasoft Native Client process data in the context of multiple services each using their own app.telemetry server. The correct assignment of telemetry data to the right app.telemetry server is now supported by using a common directory per service. Initialize and close your service directories as required and associate your registered applications with the directory context, so that you can send each applications telemetry data to the correct telemetry server also in case of later data recovery processing.

Automatic configuration of service checks for telemetry counterPermanent link for this heading

When registering a telemetry counter, you can automatically configure a corresponding service check for this counter by specifying at least one of two new attributes in the registration:

  • APM_COUNTER_ATTRIBUTE_WARNING_LIMIT: Warning limit for the service check.
  • APM_COUNTER_ATTRIBUTE_ERROR_LIMIT: Error limit for the service check.

Namespace property within Kubernetes Permanent link for this heading

A Kubernetes namespace provides the scope for Pods, Services and Deployments in a Cluster. The namespace is thus an important information where a specific container belongs to. As Fabasoft app.telemetry should know where services belong to, the Fabasoft app.telemetry library attaches this namespace information to the application registration as an application property (apm:namespace). There is currently no documented source available, where the namespace can be read from, but if there is a file named /run/secrets/kubernetes.io/serviceaccount/namespace it contains the namespace and the library reads that file. If the file is not available, the Kubernetes Downward API should be used to provide an environment variable named “APM_NAMESPACE” containing the namespace of the pod. The Fabasoft app.telemetry Library will pass the value to the apm:namespace application property.

Fabasoft app.telemetry 2021Permanent link for this heading

Platform Support for RHEL/CentOS 8Permanent link for this heading

All Fabasoft app.telemetry Services are supported also on RHEL/CentOS 8.

OpenMetricsPermanent link for this heading

The Fabasoft app.telemetry Agent can provide an http endpoint for retrieving Software-Telemetry Based Counters of all connected applications in the OpenMetrics (Draft) format, which is supported e.g. by Prometheus. Enable this feature by providing apptelemetryagent command line options openMetricsBindAddress and openMetricsBindPort.

“gap” column in Software-Telemetry DetailsPermanent link for this heading

The new “gap” column denotes the time between the previous and the current telemetry event, which helps in finding delays in not instrumented parts of the code.

Fabasoft app.telemetry 2020Permanent link for this heading

Database cleanup using Partitioned Tables for PostgreSQLPermanent link for this heading

Automatic cleanup of Log Pool tables and statistics is supported on a daily basis. Since deleting records is at least as costly as the insertion of new records, the Fabasoft app.telemetry 2020 supports the automatic usage of partitioned tables in PostgreSQL (as implemented since PostgreSQL 11). Therefore, every day a new partition table will be created to hold the records of that particular day. Cleanup is performed by simply dropping outdated partitions. There is no migration of old table layout performed, so switching from a non-partitioned table layout to a partitioned table layout requires manual migration of data if necessary. An app.telemetry database connection can either use partitioning for all tables or store all the data in the traditional table layout. But you can create an additional database connection to migrate the log pools by changing the database connection assigned and optionally migrating data.

Linux counters from the /sys filesystemPermanent link for this heading

Linux systems provide status data and performance measures in files under the /sys folder. There are a number of such counters that are not accessible using SNMP, so Fabasoft app.telemetry agents can now access the files in the /sys folder to read the respective counters.

Additional Functions Available in Calculated CountersPermanent link for this heading

Additional functions are available calculation counter values:

  • log … Natural Logarithm
  • log10 .. Logarithm base 10
  • sqrt … Square root

Grafana Support for Software-Telemetry Based CountersPermanent link for this heading

Grafana is a common visualization tool used heavily in container environments. Fabasoft app.telemetry now supports JSON REST queries, that reflect the Software-Telemetry counters in a format compatible with Prometheus data sources. So you can integrate Fabasoft app.telemetry Software-Telemetry counters in your Grafana dashboards. See the REST-API documentation on the product kit for more details.

Support of Dimension Filters in Activity StatisticsPermanent link for this heading

The Sunburst visualization of the Activity Statistics data operated on the unfiltered statistics data regardless of the filters set in the Stream, Time Line or Data Grid view. Now, filters are respected and can even be extended based on the selection within the Sunburst graph. In the Stream and Time Line visualization an additional Dimension “Agent” is supported.

Container SupportPermanent link for this heading

The Fabasoft app.telemetry rpms now support installation within RHEL/CentOS 7 containers. So a Fabasoft app.telemetry server can now be hosted in a container environment e.g. on a RedHat OpenShift Container Platform.

To improve application identification in a container environment, an application is now identified by a random GUID instead of Agent Id and Process Id, which is not unique in a container environment, where the main process of each container is started with pid 1. This change requires the extension of the block header format of the software telemetry data, which makes the data incompatible with older versions of Fabasoft app.telemetry. Therefore, an older Fabasoft app.telemetry Server cannot read requests exported from a version 2020 server. And during the upgrade process, an older Fabasoft app.telemetry Server will not be able to process telemetry data received from already upgraded Fabasoft app.telemetry Agents version 2020. So the preferred upgrade sequence implies to upgrade the Fabasoft app.telemetry Server before the agents.

Fabasoft app.telemetry 2019Permanent link for this heading

Software-Telemetry Based Counter APIPermanent link for this heading

The addition of a counter API based on the Fabasoft app.telemetry Software-Telemetry data transport enables applications to provide insight into their operation with minimal configuration. The Software-Telemetry based counter API also sidesteps limitations of Windows Performance Counters and SNMP by allowing for much greater flexibility to identify counter instances. Check the SDK-Documentation for more information.

Fabasoft app.telemetry 2018 UR 3Permanent link for this heading

Extended Application Registration PropertiesPermanent link for this heading

Applications can register additional metadata about them that empowers operators to gain a better understanding of which specific services were involved in requests. A small set of predefined metadata is automatically provided by the Software-Telemetry API libraries, check the SDK Documentation for more information.

Fabasoft app.telemetry 2018 UR 2Permanent link for this heading

Fabasoft app.telemetry Configuration servicePermanent link for this heading

The new Fabasoft app.telemetry Configuration service manages app.telemetry configuration data such as:

  • The Fabasoft app.telemetry Infrastructure configuration (infra.xml, encryption.key and encryption.pem)
  • User settings
  • The License
  • Request Categories

The files that represent the listed configuration data changed their location, which may require changes to the configuration of backup software to ensure backups remain usable in the future. A running Fabasoft app.telemetry Configuration Service is very important for the correct operation of all Fabasoft app.telemetry services.

Loading of XML and JSON files from Infrastructure ScriptsPermanent link for this heading

The Fabasoft app.telemetry infrastructure scripting gained functions to load and parse Log-Definitions (including forms), and JSON files into JavaScript objects that are directly useable.

Dynamic Java Instrumentation (JVMTI) removedPermanent link for this heading

Dynamic instrumentation of Java Applications using JVMTI has been removed.

Authentication support for mod_auth_openidc / KeycloakPermanent link for this heading

Authentication using the Apache HTTPD module mod_auth_openidc with Keycloak is now supported. With appropriate configuration the Fabasoft app.telemetry Client can also provide autocomplete support for Keycloak roles to simplify the configuration of access permissions within Fabasoft app.telemetry.

Fabasoft app.telemetry 2018Permanent link for this heading

Software-Telemetry Research ViewPermanent link for this heading

The Software-Telemetry Research View implemented in Fabasoft app.telemetry 2018 supports the identification of requests matching complex filter criteria and longer timespans. In contrast to the standard Software-Telemetry Data View, which has been primarily designed for live request view, the Research View handles a dataset of (up to 10000) requests which is calculated on demand and can be sorted appropriately. The calculation of the dataset is done on request, asynchronously and can be interrupted, so the user has full control of the calculation and no more client timeouts may occur during calculation due to long running queries. The compulsory need of specifying the period of time to search in, allows to influence the duration of the query directly and helps avoiding long running queries. The analysis of single requests is available as well in the Research View as in the Data View. Navigation to the Requests from the Request Statistics view and the Request Categories view will lead to the new Research view to support the analysis of larger datasets.

Apache HTTPD Configuration on CentOS 7Permanent link for this heading

In case you have customized the Apache HTTPD configuration for app.telemetry in /etc/httpd/conf.d/apptelemetrywebserver.conf it may be necessary to review the Fabasoft app.telemetry Webserver configuration file in /etc/httpd/conf.d/apptelemetry.conf to reapply your customizations.

Services need to be manually started after the installation on CentOS 7Permanent link for this heading

On CentOS 7 app.telemetry services will no longer be started immediately after an installation to improve support for installations in systems that are not fully running (such as for example during an automated kickstart installation). Since you need to run the /opt/app.telemetry/bin/serversetup.sh script after an installation of a Fabasoft app.telemetry Server you will not notice a difference there. Fabasoft app.telemetry Agent installations on the other hand require a manual start of the app.telemetry Agent (or a system reboot if you prefer that).

Fabasoft app.telemetry 2016Permanent link for this heading

Agent CountersPermanent link for this heading

In order to trigger status events in case of an invalid agent time status, additional counters are available from the “Server Statistics Counter” plugin under the agent object, which reflect the values of the agent view.

The “Time Drift (ms)” counter represents the absolute value of the time difference between host machine of the app.telemetry server and the host machine of the selected app.telemetry agent in milliseconds. In some environments (e.g. when using Kerberos authentication) it is essential to keep the time drift between machines within a small range (< 5 seconds). With the new counter you can set warning or error limits to get informed when the time difference between servers is out of the valid band.

The “RTT (ms)” counter represents the time in milliseconds it takes to send a simple request to the selected agent and receive the answer. The time depends on the quality of the network connection and the load of the systems involved.

SSL Connection information in nginx ModulePermanent link for this heading

In order to track the quality of the SSL encryption, the SSL Version and SSL Cipher property is logged for SSL connections.

In addition the remote port property is reported to allow identifying http connections based on the remote port.

Compare Counter Checks with String ValuesPermanent link for this heading

Some counter (e.g. from SNMP sources) report status values as strings. These strings can be matched with regular expressions to generate a warning or error status.

In the Service Check configuration dialog you can either use the Numeric Ranges to specify a range of critical values as you could do this also in previous versions or you use the new Text Match to specify, which text values should trigger an error. The Pattern is a regular expression to match the value of the counter.

In this example, the counter is reported as an error, if the value is no equal to connected or ok.

The following example will generate a critical status, if the value starts with err.

Deactivate and Activate Dashboards and ChartsPermanent link for this heading

To temporarily avoid the usage of dashboards or dashboard charts, they can now be deactivated by an administrator using the context menu in the configuration mode.

Filter Notifications by source and target statusPermanent link for this heading

To avoid notifications, when the status changes between OK and warning, notifications can now be filtered not only by target status but also by source status.

Thus, if only Critical is selected as from and to status filter, then notifications are only sent, when a selected check, service or service group changes to critical or if it has changed from critical to any other status.

Encryption of sensitive parameters in the InfrastructurePermanent link for this heading

There are several password or passphrase parameters required for service checks, database connections. On order to safely store these parameters they are now encrypted using an RSA key so they are not readable in the infra.xml. The encryption keys are generated automatically by the app.telemetry server. Make sure to create a backup of the key pair encryption.(key|pem) located under /etc/app.telemetry/server/ or C:\ProgramData\Fabasoft app.telemetry\server\ to allow the server to decrypt the messages in case of restoring the infra.xml on another system.

Instrumentation extended in nginx modulePermanent link for this heading

SSL Version and SSL Cipher were added as parameters of the nginx softwaretelemetry module.

Additional customization options in feedback formsPermanent link for this heading

Some additional options have been implemented to allow further customization of your feedback dialogs. Choose your own fonts, define the border radius and shadow of the form and the buttons and hide the copyright text to adapt the feedback dialog to your website design.