OApp Operations

This guide covers operational procedures for managing live OApps, with a focus on safely updating ULN configurations without disrupting in-flight messages.

The In-Flight Message Problem

When your OApp is live, messages may be in-flight at any moment—sent from the source chain but not yet delivered on the destination chain. Each message carries the ULN configuration that was active at the time it was sent. If you update the destination chain's receive configuration while messages are in-flight, those messages may become undeliverable.

This race condition is inherent to cross-chain systems. Consider this scenario:

1. User sends a message from Base to Solana when the send config has confirmations = 32
2. While the message is in-flight, you update the receive config on Solana to require confirmations = 64
3. The DVNs verify the message with 32 confirmations (as specified when it was sent)
4. The destination chain rejects the message because the receive config now requires 64 confirmations

The message is now stuck—it cannot be verified because the DVN attestations don't meet the updated requirement.

Config Changes That Can Cause Issues

The following ULN config changes can cause in-flight messages to become undeliverable:

Config Change	Risk
Increasing confirmations on receive side	Messages verified with fewer confirmations will fail
Changing requiredDVNs	Messages verified by the old DVN set won't match
Changing optionalDVNs or optionalDVNThreshold	Messages may not meet the new quorum requirement

tip

Decreasing confirmations or loosening DVN requirements is generally safe—the receive-side verification uses a >= comparison for confirmations, and any superset of DVN attestations will satisfy a smaller requirement.

Safe Configuration Update Procedure

To safely update configurations on live pathways, use the following procedure:

Step 1: Block New Messages on the Pathway

Set the send library to the BlockedMessageLib for all pathways that need configuration updates. This prevents new messages from being sent while you wait for in-flight messages to clear.

Using layerzero.config.ts

Update your config to use the blockedLibrary address (available from endpoint.blockedLibrary()):

connections: [
  {
    from: baseContract,
    to: solanaContract,
    config: {
      // Set to blocked library to prevent new sends
      sendLibrary: "0x...", // BlockedMessageLib address on Base
      // Keep receive config unchanged for now
    },
  },
],

Then run the wire command:

npx hardhat lz:oapp:wire --oapp-config layerzero.config.ts

Using Direct Contract Calls

// Get the blocked library address
address blockedLib = IEndpointV2(endpoint).blockedLibrary();

// Block sends on the pathway
IEndpointV2(endpoint).setSendLibrary(
    oappAddress,
    destinationEid,
    blockedLib
);

Any send() calls on this pathway will now revert with LZ_NotImplemented().

Step 2: Wait for In-Flight Messages to Deliver

Monitor LayerZero Scan to confirm all in-flight messages on the affected pathways have been delivered. You can query messages by OApp address and filter by status.

Look for:

• All messages showing Delivered status
• No messages in Inflight or Confirming status

caution

Do not proceed until all in-flight messages are delivered. Any messages still in-flight when you update the receive config may become permanently stuck.

Step 3: Apply Configuration Changes

Once all messages are delivered, update the ULN configuration as needed.

Updating Send Config

config: {
  sendConfig: {
    ulnConfig: {
      confirmations: BigInt(64), // New confirmation requirement
      requiredDVNs: ["0x..."],   // Updated DVN set
      optionalDVNs: [],
      optionalDVNThreshold: 0,
    },
  },
},

Updating Receive Config

config: {
  receiveConfig: {
    ulnConfig: {
      confirmations: BigInt(64),
      requiredDVNs: ["0x..."],
      optionalDVNs: [],
      optionalDVNThreshold: 0,
    },
  },
},

Run the wire command to apply changes:

npx hardhat lz:oapp:wire --oapp-config layerzero.config.ts

Step 4: Restore the Send Library

After applying the configuration changes, restore the send library to re-enable message sending:

config: {
  // Restore to ULN302 (or your preferred library)
  sendLibrary: "0x...", // SendUln302 address
},

npx hardhat lz:oapp:wire --oapp-config layerzero.config.ts

Using Grace Periods for Receive Library Updates

When migrating between message library versions, you can use the gracePeriod mechanism to allow in-flight messages to complete under the old library while new messages use the new library.

config: {
  receiveLibraryConfig: {
    receiveLibrary: "0x...", // New ReceiveUln302 address
    gracePeriod: BigInt(7200), // ~24 hours on most chains (blocks)
  },
},

During the grace period:

• New messages are verified against the new receive library configuration
• Messages sent before the update can still be verified against the old library configuration
• After the grace period expires, only the new library is valid

info

The gracePeriod is specified in blocks, not seconds. Calculate the appropriate value based on the destination chain's block time.

Setting an Explicit Timeout

For more control, use receiveLibraryTimeoutConfig to set an explicit expiry:

config: {
  receiveLibraryConfig: {
    receiveLibrary: "0x...", // New library
    gracePeriod: BigInt(0),  // No automatic grace period
  },
  receiveLibraryTimeoutConfig: {
    lib: "0x...",           // Old library address
    expiry: BigInt(12345678), // Block number when old library expires
  },
},

Configuration Coordination Checklist

Before updating configurations on a live OApp:

• Audit all pathways: List every source-destination pair that uses the config you're changing
• Check for in-flight messages: Query LayerZero Scan for pending messages on affected pathways
• Plan the update sequence: For bidirectional pathways, you may need to block both directions
• Coordinate DVN changes: If changing DVNs, ensure the new DVNs are operational before updating configs
• Test on testnet first: Validate your config changes on testnet pathways before mainnet
• Communicate downtime: If blocking pathways, notify users of the maintenance window

Troubleshooting Stuck Messages

If messages become stuck due to a configuration mismatch:

1. Identify the mismatch: Compare the message's original send config (visible on LayerZero Scan) with the current receive config
2. Assess options:
   • If the mismatch is minor (e.g., confirmation count), consider temporarily reverting the receive config
   • If messages cannot be recovered, use skip() to clear the pathway

See Debugging Messages for detailed troubleshooting steps.

Summary

Scenario	Recommended Approach
Updating ULN config on live pathway	Block → Wait → Update → Unblock
Migrating message library versions	Use gracePeriod on receive library
Adding/removing DVNs	Block send, wait for drain, update both sides
Decreasing confirmations	Safe to apply directly (backward compatible)
Emergency: stuck messages	Consider reverting config or using skip()