symfony-nats-messenger 5.7: reconnect, clusters and a bundle

symfony-nats-messenger 5.7.1 is out. Since the 5.4 post on duplicate protection the transport has become a Symfony bundle, gained a reconnect mode in which a worker polls through a NATS restart instead of exiting, and learned to place streams on chosen servers of a cluster. This post covers those, what to configure, and what to check before upgrading, with a short recap for anyone still on 5.1.

A bundle Flex enables for you (5.5.0)#

The package ships IDCTNatsMessengerBundle. Symfony Flex enables it on install, and it registers the transport factory and the igbinary serializer, so composer require and a DSN are all an application needs:

framework:
  messenger:
    transports:
      orders: 'nats-jetstream://localhost:4222/orders/orders.created'

The services.yaml entries earlier versions asked for keep working and can go, with one exception: the bundle does not register the igbinary_serializer id that an older README suggested. If a transport says serializer: 'igbinary_serializer', point it at the class name before you delete that entry:

serializer: 'IDCT\NatsMessenger\Serializer\IgbinarySerializer'

5.5.0 also turns a pull refused at the connection's subscription limit into a TransportException that names the stream, the consumer and the limit, built on the typed error from php-nats-jetstream-client 2.27.

When the connection goes away#

You now choose who recovers a lost connection.

Default: Symfony recovers#

With the defaults, the operation that runs into a lost connection fails, the next one dials again, a consumer worker exits and its supervisor starts a new one. Symfony's retries and failure transport take care of the message. Since 5.2, ping_after_idle (default 30 seconds) checks a connection that sat idle, or just failed an operation, with a PING before it is used again, and since 5.3 request_timeout (default 10 seconds) bounds the wait for a publish, a synchronous ACK, setup() and the message count.

Reconnect mode (5.7.0)#

options:
  reconnect: true             # the client recovers a lost connection (default: false)
  max_reconnect_attempts: 20  # attempts per recovery (default: the client's 10)

With reconnect: true the NATS client re-dials in the background with backoff, replays its subscriptions and flushes what it buffered, and the transport steps back. An operation that meets a recovery in progress waits for it within its own budget: a request within request_timeout, a pull within max_batch_timeout plus a second, after which it ends as an empty batch. A worker polls through an outage shorter than one recovery cycle instead of exiting; the client's default of 10 attempts gives about 109 seconds against a server that refuses connections. With auto_setup, the transport verifies the stream and consumer again once the client has reconnected, because a memory stream does not survive a server restart.

We measured it with a 53 second docker stop of the server. With the client's default of 10 attempts, every operation waited within its budget and none had to dial. With only 2 attempts the client gave up about 12 seconds in, and every operation after that dialled inline for about 18 seconds and failed. A recovery that gives up surfaces as an exception, not as an empty queue, so the worker ends and its supervisor restarts it.

5.7.1 requires client 2.27.1. With it, an ack() whose write is the first to notice the lost connection goes into the reconnect buffer and returns at once; with 2.27.0 it ran the whole recovery inside the call.

Reconnect mode is meant for long-running workers. Leave it off for web requests that dispatch: once the client has given up, a request that runs into the outage waits for a whole attempt loop.

Logging#

The NATS client now logs through your application's logger: connection lifecycle at info level, errors at error level, under Monolog's messenger channel. The bundle wires it when a logger service exists. In reconnect mode this matters: if a recovery gives up, the client discards the ACKs it buffered during the outage, and the only trace is an error-level line. The messages come back after ack_wait.

5.7.0 also reads a pull that the server ended because it was shutting down, or because the consumer's leader moved, as an empty batch. Before, it read as a missing consumer, and with auto_setup every graceful server stop ended the worker with "Failed to setup NATS stream".

Clusters and stream placement (5.6.0, 5.7.1)#

Two options pin a stream to a JetStream cluster and to tagged servers:

options:
  stream_replicas: 3
  stream_placement_cluster: 'east'
  stream_placement_tags: ['ssd']

On an existing stream each option changes only its own half of the live placement, and leaving both unset keeps the placement the stream already has. NATS refuses a placement change and a replica change in one update, so setup() fails with the steps to stage it.

The README now has a section on running against a cluster. The DSN names one server and the client learns the others from it, so a process that starts while that server is down cannot connect: use a DNS name that resolves to several servers, or a load balancer. On a TLS cluster set tls_required: true next to the nats-jetstream+tls:// scheme, otherwise the client dials the servers it learned without TLS.

Three new examples run in CI against a three-server test cluster. reconnect.php restarts the server under a worker. cluster-failover.php stops the server a worker is connected to and shows every message still arrives. stream-placement.php places, changes and stages a move.

Also since 5.1#

• 5.2.0: a transport whose connection was lost no longer fails every later operation until the process restarts.
• 5.2.1: the recommended IgbinarySerializer no longer stops the worker when Symfony retries a message or sends it to the failure transport, and with retry_handler: nats Symfony's retry no longer multiplies the deliveries.
• 5.2.2: messenger:consume --keepalive no longer stops the worker at the first alarm.
• 5.2.3, a security fix: a tls_verify_peer value the transport does not recognize no longer turns peer verification off.
• 5.4: duplicate protection with DeduplicationIdStamp and deduplicate: true, covered in the 5.4 post.

Upgrading#

composer update idct/symfony-nats-messenger

• From 5.4 or earlier, commit the line Flex adds to config/bundles.php, and check for the igbinary_serializer id described above. If you ever go back to 5.4 or earlier, delete that line yourself, or the application stops with a class-not-found error.
• The NATS client requirement is now ^2.27.1, and psr/log ^3.0 is a direct requirement.
• reconnect and deduplicate are off and the placement options unset by default, so those features change nothing until you turn them on. From 5.1, note that ping_after_idle is on (30 seconds): it adds a PING before a connection that sat idle longer than that is used again.

Requirements: PHP 8.2 or newer, Symfony 7.2 or 8, NATS 2.9 or newer (2.12 for scheduled messages and ! exclusion tags). The suite has 673 unit tests at 99.78% statement coverage, a mutation score of 96.9% and Behat scenarios against plain, TLS and mTLS NATS.

More on the symfony-nats-messenger software page; the full list of changes is in the 5.7.1 changelog.