← back to Exo

gossipsub: stop silent message dropping and warn (#1434)

1f242e8eee922262579674473e199062a6d69ec2 · 2026-02-10 18:47:47 +0000 · Jake Hillion

The 15-second publish_queue_duration caused messages in peer queues to
be silently dropped. When events are dropped, workers detect gaps in the
event index sequence and request missing events via the NACK path
(RequestEventLog), but this recovery is inefficient.

Removed the timeout configuration - gossipsub now uses its default
behavior without time-based eviction. If queue buildup is a concern,
queue size should be limited explicitly rather than dropping by timeout.

Split error handling to log AllQueuesFullError as a warning (indicates
peers are unresponsive) while keeping NoPeersSubscribedToTopicError
silent (expected during startup and network partitions).

Test plan:
- CI

Files touched

• M rust/networking/src/swarm.rs
• M src/exo/routing/router.py

Diff

commit 1f242e8eee922262579674473e199062a6d69ec2
Author: Jake Hillion <jake@hillion.co.uk>
Date:   Tue Feb 10 18:47:47 2026 +0000

    gossipsub: stop silent message dropping and warn (#1434)
    
    The 15-second publish_queue_duration caused messages in peer queues to
    be silently dropped. When events are dropped, workers detect gaps in the
    event index sequence and request missing events via the NACK path
    (RequestEventLog), but this recovery is inefficient.
    
    Removed the timeout configuration - gossipsub now uses its default
    behavior without time-based eviction. If queue buildup is a concern,
    queue size should be limited explicitly rather than dropping by timeout.
    
    Split error handling to log AllQueuesFullError as a warning (indicates
    peers are unresponsive) while keeping NoPeersSubscribedToTopicError
    silent (expected during startup and network partitions).
    
    Test plan:
    - CI
---
 rust/networking/src/swarm.rs | 2 --
 src/exo/routing/router.py    | 6 +++---
 2 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/rust/networking/src/swarm.rs b/rust/networking/src/swarm.rs
index a5c87af5..33cf8966 100644
--- a/rust/networking/src/swarm.rs
+++ b/rust/networking/src/swarm.rs
@@ -106,7 +106,6 @@ mod behaviour {
     use crate::{alias, discovery};
     use libp2p::swarm::NetworkBehaviour;
     use libp2p::{gossipsub, identity};
-    use std::time::Duration;
 
     /// Behavior of the Swarm which composes all desired behaviors:
     /// Right now its just [`discovery::Behaviour`] and [`gossipsub::Behaviour`].
@@ -134,7 +133,6 @@ mod behaviour {
         gossipsub::Behaviour::new(
             MessageAuthenticity::Signed(keypair.clone()),
             ConfigBuilder::default()
-                .publish_queue_duration(Duration::from_secs(15))
                 .max_transmit_size(1024 * 1024)
                 .validation_mode(ValidationMode::Strict)
                 .build()
diff --git a/src/exo/routing/router.py b/src/exo/routing/router.py
index 9bba9c71..309f7d0b 100644
--- a/src/exo/routing/router.py
+++ b/src/exo/routing/router.py
@@ -207,10 +207,10 @@ class Router:
                 try:
                     logger.trace(f"Sending message on {topic} with payload {data}")
                     await self._net.gossipsub_publish(topic, data)
-                # As a hack, this also catches AllQueuesFull
-                # Need to fix that ASAP.
-                except (NoPeersSubscribedToTopicError, AllQueuesFullError):
+                except NoPeersSubscribedToTopicError:
                     pass
+                except AllQueuesFullError:
+                    logger.warning(f"All peer queues full, dropping message on {topic}")
 
 
 def get_node_id_keypair(

← 64179c6f Dont save to app directory (#1435)  ·  back to Exo  ·  add log rotation for .exo/exo.log (#1438) 009b43c6 →