Datasource Micrometer Reference Documentation

1. Legal

Copies of this document may be made for your own use and for distribution to others, provided that you do not charge any fee for such copies and further provided that each copy contains this Copyright Notice, whether distributed in print or electronically.

2. Getting Started

If you are getting started with Datasource Micrometer, start by reading this section. It answers the basic “what?”, “how?” and “why?” questions. It includes an introduction to Datasource Micrometer, along with installation instructions.

2.1. Introducing Datasource Micrometer

The Datasource Micrometer provides a Micrometer Tracing instrumentation for JDBC operations.

Currently, this project provides two modules - core instrumentation and its Spring Boot Auto Configuration.

If you are a Micrometer Tracing user but not using Spring Boot, you can directly use the datasource-micrometer module, which doesn’t have a dependency on Spring Framework.
If you are a Spring Boot 3 user, then you can add the datasource-micrometer-spring-boot module to the classpath. Then, it automatically instruments the DataSource and provides a tracing capability on the JDBC operations.

The instrumentation implementation uses datasource-proxy to provide a proxy for JDBC operations.

2.1.1. Background

The Micrometer v1.10.0 introduced a new module, Micrometer Tracing, which abstracts the popular tracers and provides the vendor free APIs for observations. The Spring Boot 3 and its portfolio projects natively support Micrometer Tracing.

In Spring Boot 2.x, the Spring Cloud Sleuth provided the instrumentation to the many components including JDBC operations. It was a central library that provides tracing instrumentation to the Spring Boot applications. However, with the new observability, the responsibility for instrumentation has shifted to the individual component. For example, Spring Framework will provide native instrumentation support for its components. As a result, there will be no Spring Cloud Sleuth for Spring Boot 3.

This Datasource Micrometer project provides instrumentation on the JDBC operations with the Micrometer Tracing to cover what was provided by the Spring Cloud Sleuth. The initial version aims users to smoothly transition from Spring Boot 2.x with Spring Cloud Sleuth to the Spring Boot 3 in JDBC instrumentation. In addition, since the Micrometer Tracing is independent of Spring Framework, the instrumentation is available to the non-spring applications as well.

2.1.2. Modules

datasource-micrometer

This module provides the instrumentation on the JDBC operations. The implementation is provided as a datasource-proxy listener. Non-spring applications can directly use this module.

datasource-micrometer-spring-boot

This module provides an auto-configuration for the Spring Boot 3 applications.

2.2. Installation

2.2.1. Maven and Gradle

datasource-micrometer

Maven

<dependency>
    <groupId>net.ttddyy.observation</groupId>
    <artifactId>datasource-micrometer</artifactId>
    <version>1.0.3</version>
</dependency>

Gradle

dependencies {
    implementation "net.ttddyy.observation:datasource-micrometer:1.0.3"
}

datasource-micrometer-spring-boot

Maven

<dependency>
    <groupId>net.ttddyy.observation</groupId>
    <artifactId>datasource-micrometer-spring-boot</artifactId>
    <version>1.0.3</version>
</dependency>

Gradle

dependencies {
    implementation "net.ttddyy.observation:datasource-micrometer-spring-boot:1.0.3"
}

2.2.2. Setup

datasource-micrometer

To setup the observation for JDBC operations, create a proxy of your DataSource, then register the corresponding tracing observation handlers.

// Register the tracing observation handlers
ObservationRegistry observationRegistry = ObservationRegistry.create();
ObservationConfig observationConfig = observationRegistry.observationConfig();
observationConfig.observationHandler(new ConnectionTracingObservationHandler(tracer));
observationConfig.observationHandler(new QueryTracingObservationHandler(tracer);
observationConfig.observationHandler(new ResultSetTracingObservationHandler(tracer));
//  add other necessary handlers

// Create a DataSource proxy with the observation listener
DataSource dataSource = ...
DataSourceObservationListener listener = new DataSourceObservationListener(observationRegistry);
DataSource instrumented = ProxyDataSourceBuilder.create(dataSource).listener(listener).methodListener(listener).build;

// Use the instrumented DataSource

datasource-micrometer-spring-boot

The auto-configuration class automatically sets up the observation on your DataSource bean.

2.3. Migration from Spring Cloud Sleuth

Datasource Micrometer deliberatively provides similar property names to ease the migration from Spring Cloud Sleuth. Most of the JDBC related properties from spring.sleuth.jdbc and spring.sleuth.jdbc.datasource-proxy map to the jdbc and jdbc.datasource-proxy properties.

Please reference the list of application properties in Spring Cloud Sleuth and Datasource Micrometer.

3. Using Datasource Micrometer

This section goes into more detail about how you should use Datasource Micrometer.

3.1. Types of Observations

The Datasource Micrometer creates Connection, Query, and ResultSet observations.

The Connection observation represents the database connection operations. It is the base observation, as any database access requires a connection. The Query observation provides query execution details, such as execution time, SQL query, bind parameters, etc. The ResultSet observation shows how the operations fetched the data from the query result, including the number of retrieved rows.

To configure these observations, see How to Add Tracing Observation Handlers for JDBC operations.
For Spring Boot, see How to Choose What To Observe.

3.2. Features

3.2.1. HikariCP Support

jdbc.datasource.driver and jdbc.datasource.pool tags are available when the target datasource is a HikariDataSource.

The HikariJdbcObservationFilter provides this feature and this observation filter needs to be registered to the ObservationRegistry.

ObservationRegistry registry = ...
registry.observationConfig().observationFilter(new HikariJdbcObservationFilter());

It is auto configured in datasource-micrometer-spring-boot.

3.2.2. Remote IP and Port

For spans, remote IP and port are retrieved from the datasource url.

3.2.3. Remote Service Name

The datasource name is used as the remote service name in spans. The name is specified when creating a proxy datasource by datasource-proxy.

DataSource instrumented =
    ProxyDataSourceBuilder.create(dataSource)
        .name("myDS")    // Specify datasource name
        .listener(listener)
        .methodListener(listener)
        .build;

For datasource-micrometer-spring-boot, the datasource name is resolved by looking the catalog name at start up (or the connection pool name for Hikari, then fallback to its beanname) by default.

To specify a custom datasource name, see How to Customize the ProxyDataSource Name and How to Customize the ProxyDataSource Creation section.

4. “How-to” Guides

This section provides answers to some common “how do I do that…?” questions. Its coverage is not exhaustive, but it does cover quite a lot.

We are also more than happy to extend this section. If you want to add a “how-to”, send us a pull request.

4.1. datasource-micrometer

4.1.1. How to instrument `DataSource`

The DataSourceObservationListener provides the observation logic. It is implemented as a datasource-proxy listener. Follow the datasource-proxy usage to create a proxied DataSource with the listener. Then adds tracing observation handlers to the ObservationRegistry.

ObservationRegistry observationRegistry = ...
DataSourceObservationListener listener = new DataSourceObservationListener(observationRegistry);
DataSource instrumented = ProxyDataSourceBuilder.create(dataSource).listener(listener).methodListener(listener).build;

4.1.2. How to Add Tracing Observation Handlers for JDBC operations

There are 3 tracing observation handlers that react to the observations from DataSourceObservationListener.

ConnectionTracingObservationHandler
QueryTracingObservationHandler
ResultSetTracingObservationHandler

ObservationRegistry registry = ...
registry.observationConfig().observationHandler(new ConnectionTracingObservationHandler(tracer));
registry.observationConfig().observationHandler(new QueryTracingObservationHandler(tracer);
registry.observationConfig().observationHandler(new ResultSetTracingObservationHandler(tracer));

4.1.3. How to Instrument `ResultSet`

By default, datasource-proxy does not create a proxy for ResultSet. This, in turn, does not instrument the ResultSet. You need to explicitly enable the ResultSet proxy creation. Then, ResultSet get instrumented automatically.

ProxyDataSourceBuilder builder = ProxyDataSourceBuilder.create(dataSource).listener(listener).methodListener(listener);
builder.proxyResultSet();  // enable ResultSet proxy creation
DataSource instrumented = builder.build();

4.1.4. How to Include Bind Parameter Values

Bind parameter values - values from setInt, setString, etc operations on prepared and callable statement - are not tagged to spans by default. The DataSourceObservationListener class has a toggle to enable this. When it is enabled, values are tagged to the query spans as jdbc.params[]

DataSourceObservationListener listener = ...;
listener.setIncludeParameterValues(true);

4.2. datasource-micrometer-spring-boot

4.2.1. How to Disable JDBC Instrumentation

Set the jdbc.datasource-proxy.enabled property to false.

4.2.2. How to Choose What To Observe

Specify jdbc.includes property. By default, the property is set to include(observe) all(CONNECTION, QUERY, FETCH) types.

4.2.3. How to Include the Bind Parameter Values in Spans

Set the jdbc.datasource-proxy.include-parameter-values property to true.

4.2.4. How to Enable and Configure Query Logging

To enable the query logging, set the jdbc.datasource-proxy.query.enable-logging property to true.

jdbc.datasource-proxy.query.enable-logging=true,

# logging configuration
jdbc.datasource-proxy.logging=slf4j,
jdbc.datasource-proxy.query.log-level=DEBUG,
jdbc.datasource-proxy.query.logger-name=my.query-logger,
jdbc.datasource-proxy.multiline=false,

# spring boot log level property
logging.level.my.query-logger=DEBUG,

4.2.5. How to Customize the ProxyDataSource Name

Create a custom DataSourceNameResolver bean. It replaces the default bean, DefaultDataSourceNameResolver.

4.2.6. How to Customize the ProxyDataSource Creation

The ProxyDataSourceBuilderCustomizer beans are automatically called before creating a proxy datasource. This callback API allows you to customize the ProxyDataSourceBuilder.

For example, you can use ProxyDataSourceBuilderCustomizer to specify the datasource proxy name. In turn, it becomes the remote service name of the spans.

@Bean
public ProxyDataSourceBuilderCustomizer myCustomizer(){
    return (builder, dataSource, beanName, dataSourceName) -> {
        builder.name("MyAppDataSource");
    };
}

4.2.7. How to Modify the Query in Span

If you want to modify the query string in the span (high cardinality tag), use ObservationFilter to update the tag value. For example, you could perform query sanitization, truncation, etc.

This approach is generally applicable to modify the tags in span.

ObservationFilter is applied when the observation stops. The modification will not be available for timers created by DefaultMeterObservationHandler since it sets the timers at observation start.

@Bean
public ObservationFilter observationFilter() {
    return (context) -> {
        String tagKey = QueryHighCardinalityKeyNames.QUERY.name();
        KeyValue tag = context.getHighCardinalityKeyValue(tagKey);
        if(tag != null) {
            String query = tag.getValue();

            // ... modify query

            context.addHighCardinalityKeyValue(KeyValue.of(tagKey, modifiedQuery));
        }
        return context;
    };
}

5. Appendix

5.1. Common Spring Boot application properties

Various properties can be specified inside your application.properties file, inside your application.yml file, or as command line switches. This appendix provides a list of common Datasource Micrometer properties and references to the underlying classes that consume them.

Property contributions can come from additional jar files on your classpath, so you should not consider this an exhaustive list. Also, you can define your own properties.

Name Default Description

Name	Default	Description
jdbc.datasource-proxy.enabled	`true`	Whether to enable JDBC instrumentation.
jdbc.datasource-proxy.include-parameter-values	`false`	Whether to tag actual query parameter values.
jdbc.datasource-proxy.json-format	`false`	Use json output for logging query. @see ProxyDataSourceBuilder#asJson()
jdbc.datasource-proxy.logging		Logging to use for logging queries.
jdbc.datasource-proxy.multiline	`true`	Use multiline output for logging query. @see ProxyDataSourceBuilder#multiline()
jdbc.datasource-proxy.query.enable-logging	`false`	Enable logging all queries to the log.
jdbc.datasource-proxy.query.log-level	`DEBUG`	Severity of query logger.
jdbc.datasource-proxy.query.logger-name		Name of query logger.
jdbc.datasource-proxy.slow-query.enable-logging	`false`	Enable logging slow queries to the log.
jdbc.datasource-proxy.slow-query.log-level	`WARN`	Severity of slow query logger.
jdbc.datasource-proxy.slow-query.logger-name		Name of slow query logger.
jdbc.datasource-proxy.slow-query.threshold	`300`	Number of seconds to consider query as slow.
jdbc.excluded-data-source-bean-names		List of DataSource bean names that will not be decorated.
jdbc.includes		Which types of tracing we would like to include.

jdbc.datasource-proxy.enabled

true

Whether to enable JDBC instrumentation.

jdbc.datasource-proxy.include-parameter-values

false

Whether to tag actual query parameter values.

jdbc.datasource-proxy.json-format

false

Use json output for logging query. @see ProxyDataSourceBuilder#asJson()

jdbc.datasource-proxy.logging

Logging to use for logging queries.

jdbc.datasource-proxy.multiline

true

Use multiline output for logging query. @see ProxyDataSourceBuilder#multiline()

jdbc.datasource-proxy.query.enable-logging

false

Enable logging all queries to the log.

jdbc.datasource-proxy.query.log-level

DEBUG

Severity of query logger.

jdbc.datasource-proxy.query.logger-name

Name of query logger.

jdbc.datasource-proxy.slow-query.enable-logging

false

Enable logging slow queries to the log.

jdbc.datasource-proxy.slow-query.log-level

WARN

Severity of slow query logger.

jdbc.datasource-proxy.slow-query.logger-name

Name of slow query logger.

jdbc.datasource-proxy.slow-query.threshold

300

Number of seconds to consider query as slow.

jdbc.excluded-data-source-bean-names

List of DataSource bean names that will not be decorated.

jdbc.includes

Which types of tracing we would like to include.

5.2. Observability Metrics and Spans

5.2.1. Observability - Conventions

Below you can find a list of all GlobalObservationConvention and ObservationConvention declared by this project.

Table 1. ObservationConvention implementations
ObservationConvention Class Name	Applicable ObservationContext Class Name
`net.ttddyy.observation.tracing.ConnectionObservationConvention`	`ConnectionContext`
`net.ttddyy.observation.tracing.QueryObservationConvention`	`QueryContext`
`net.ttddyy.observation.tracing.ResultSetObservationConvention`	`ResultSetContext`

5.2.2. Observability - Metrics

Below you can find a list of all metrics declared by this project.

Connection

Span created when a JDBC connection takes place.

Metric name jdbc.connection. Type timer.

Metric name jdbc.connection.active. Type long task timer.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Micrometer internally uses nanoseconds for the baseunit. However, each backend determines the actual baseunit. (i.e. Prometheus uses seconds)

Fully qualified name of the enclosing class net.ttddyy.observation.tracing.JdbcObservationDocumentation.

All tags must be prefixed with jdbc prefix!

Table 2. Low cardinality Keys
Name	Description
`jdbc.datasource.driver` (required)	Name of the JDBC datasource driver. (HikariCP only)
`jdbc.datasource.pool` (required)	Name of the JDBC datasource pool. (HikariCP only)

Since, events were set on this documented entry, they will be converted to the following counters.

Connection - jdbc connection acquired

When the connection is acquired. This event is recorded right after successful "getConnection()" call.

Metric name jdbc.connection.acquired. Type counter.

Connection - jdbc connection commit

When the connection is committed.

Metric name jdbc.connection.commit. Type counter.

Connection - jdbc connection rollback

When the connection is rolled back.

Metric name jdbc.connection.rollback. Type counter.

Query

Span created when executing a query.

Metric name jdbc.query. Type timer.

Metric name jdbc.query.active. Type long task timer.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Micrometer internally uses nanoseconds for the baseunit. However, each backend determines the actual baseunit. (i.e. Prometheus uses seconds)

Fully qualified name of the enclosing class net.ttddyy.observation.tracing.JdbcObservationDocumentation.

All tags must be prefixed with jdbc prefix!

Table 3. High cardinality Keys
Name	Description
`jdbc.params[%s]` (required)	JDBC query parameter values. (since the name contains `%s` the final value will be resolved at runtime)
`jdbc.query[%s]` (required)	Name of the JDBC query. (since the name contains `%s` the final value will be resolved at runtime)
`jdbc.row-affected` (required)	Result of "executeUpdate()", "executeLargeUpdate()", "executeBatch()", or "executeLargeBatch()" on "Statement". For batch operations, the value is represented as array.

Result Set

Span created when working with JDBC result set.

Metric name jdbc.result-set. Type timer.

Metric name jdbc.result-set.active. Type long task timer.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Micrometer internally uses nanoseconds for the baseunit. However, each backend determines the actual baseunit. (i.e. Prometheus uses seconds)

Fully qualified name of the enclosing class net.ttddyy.observation.tracing.JdbcObservationDocumentation.

All tags must be prefixed with jdbc prefix!

Table 4. High cardinality Keys
Name	Description
`jdbc.row-count` (required)	Number of SQL rows.

5.2.3. Observability - Spans

Below you can find a list of all spans declared by this project.

Connection Span

Span created when a JDBC connection takes place.

Span name connection.

Fully qualified name of the enclosing class net.ttddyy.observation.tracing.JdbcObservationDocumentation.

All tags must be prefixed with jdbc prefix!

Table 5. Tag Keys
Name	Description
`jdbc.datasource.driver` (required)	Name of the JDBC datasource driver. (HikariCP only)
`jdbc.datasource.pool` (required)	Name of the JDBC datasource pool. (HikariCP only)

Table 6. Event Values
Name	Description
`acquired`	When the connection is acquired. This event is recorded right after successful "getConnection()" call.
`commit`	When the connection is committed.
`rollback`	When the connection is rolled back.

Query Span

Span created when executing a query.

Span name query.

Fully qualified name of the enclosing class net.ttddyy.observation.tracing.JdbcObservationDocumentation.

All tags must be prefixed with jdbc prefix!

Table 7. Tag Keys
Name	Description
`jdbc.params[%s]` (required)	JDBC query parameter values. (since the name contains `%s` the final value will be resolved at runtime)
`jdbc.query[%s]` (required)	Name of the JDBC query. (since the name contains `%s` the final value will be resolved at runtime)
`jdbc.row-affected` (required)	Result of "executeUpdate()", "executeLargeUpdate()", "executeBatch()", or "executeLargeBatch()" on "Statement". For batch operations, the value is represented as array.

Result Set Span

Span created when working with JDBC result set.

Span name result-set.

Fully qualified name of the enclosing class net.ttddyy.observation.tracing.JdbcObservationDocumentation.

All tags must be prefixed with jdbc prefix!

Table 8. Tag Keys
Name	Description
`jdbc.row-count` (required)	Number of SQL rows.