pamoja

An SDK for IoT, robotics, and drones

pamojaRev 0.2.0MIT licensepamoja.molex.cloud

Engine surface#

A node should not care which link it got. The reading it took is worth the same whether it leaves over MQTT, over CoAP, over a mesh hop, or into a queue on disk until morning. So every link in pamoja implements one contract: connect, subscribe, send, and, for a link that can deliver, receive. Everything that carries traffic takes that contract rather than a particular link, which is what lets a store, a ladder, or a fault injector work with all of them and with each other. A ladder is a link under that contract too, so a node written against one transport runs over a ladder unchanged.

The fault injector is the clearest case. It is a transport that wraps a transport and fails a set number of the sends passing through it, so the offline path can be exercised without unplugging anything. It composes exactly where a real link does, because as far as the ladder above it is concerned it is a real link.

In Rust the contract is a trait and composition is a move, so the compiler already stops a link being used after it has been handed on. The three bindings have no way to say "any transport", so each holds one handle that dispatches to whichever kind was built, and composing empties the handle rather than leaving it aliasing what now belongs to something else. A link handed to a fault injector or to a ladder is spent, and calling anything on it afterwards throws.

What the example does#

It runs a link that fails its first send, underneath a ladder with a queue, and follows two readings through: the one the failure catches, and the one taken after it.

The gateway and the ladder are two links onto the same in-process broker, so what the subscriber reads is what actually crossed the link rather than something handed to it. The failure is arranged by telling the injector how many sends to refuse, and the depth of the backlog is the ladder's own buffered count, not a tally the example keeps.

It proves:

  • A fault injector sits in the ladder where a plain link would, and the send it refuses comes back as Buffered rather than an error, so the reading is held instead of lost.
  • The reading taken next is buffered too, even though the link would carry it now, and the ladder counts both as queued.
  • A flush forwards the whole backlog and leaves nothing queued behind it.
  • The subscriber reads 20.1 and then 20.4, so the far end sees the readings in the order they were taken, not the order the link became willing to carry them.

Run it#

The example below is a program CI runs on every change, in each language, from a clone of the repository:

Rust

cargo run -p pamoja-examples --example transport

TypeScript

npm --prefix bindings/node run guides -- transport

Python

python bindings/python/guides/transport.py

C#

dotnet run --project bindings/dotnet/samples/Pamoja.Guides -- transport

Rust#

In Rust the contract is two traits in pamoja-core. Transport has connect, subscribe, send, and send_text; Receive adds recv for a link that delivers, which gives Ok(None) once the link has ended. Every call is async and returns pamoja_core::Result, whose Error names the failure. A received Message is a topic and a payload that reads itself with text() and number(). A receive is cancel-safe, so tokio::time::timeout(limit, link.recv()) gives up without losing the message it would have taken. Anything that carries traffic is generic over the traits, and composition is a move: Faulty::new(link, 1) and a ladder's rung(link) take the link by value, so the compiler stops it being used after it has been handed on.

From examples/guides/transport.rs:

Rust
use pamoja_core::{Receive, Transport};
use pamoja_ladder::{Delivery, TransportLadder};
use pamoja_loopback::{Faulty, LoopbackBroker, LoopbackTransport};
use pamoja_sync::MemoryStore;

// Whatever a link is underneath, MQTT, CoAP, or the in-process broker here, it reaches
// the rest of the framework through one trait. Anything that takes a link is generic
// over it, so a node is written once and pointed at whichever link it has.
let broker = LoopbackBroker::new();
let topic = "sensors/1/temperature";
let mut gateway = LoopbackTransport::new(broker.clone());
gateway.connect().await?;
gateway.subscribe(topic).await?;

// The fault injector is itself a transport wrapping a transport, so it composes
// anywhere a link does. This one fails its next send and passes the rest through.
let mut ladder = TransportLadder::new(MemoryStore::new())
    .rung(Faulty::new(LoopbackTransport::new(broker), 1));
ladder.connect().await?;

// The injected failure lands, so the reading is buffered rather than lost.
let first = ladder.send_text(topic, "20.1").await?;
let after_first = ladder.buffered().await.expect("a count");
println!("first reading: {first:?}, {after_first} queued");

// The next reading joins the back of the queue instead of overtaking it, even though
// the link would take it now. Order on the wire is the order the readings were taken.
let second = ladder.send_text(topic, "20.4").await?;
let queued = ladder.buffered().await.expect("a count");
println!("second reading: {second:?}, {queued} queued");

// Flushing forwards the backlog oldest first, and the subscriber sees it in order.
let forwarded = ladder.flush().await?;
let first_out = gateway.recv().await?.expect("a message");
let second_out = gateway.recv().await?.expect("a message");
let earlier = first_out.text().expect("text");
let later = second_out.text().expect("text");
println!("flush forwarded {forwarded}, gateway saw {earlier} then {later}");

TypeScript#

In TypeScript the contract is the Transport class in @pamoja/core. Transport.mqtt(options) and Transport.coap(options) open the network links, Transport.fromHandlers(handlers) wraps one written in JavaScript, and a broker's rung() gives the loopback. connect, subscribe, send, and recv return promises; recv resolves with a TransportMessage, { topic, payload, text?, number? }, or null once the link has ended, and recv(timeoutMs) rejects when the time runs out without losing the next message. Calls on one transport run one at a time. Transport.faulty, Transport.degraded, and a ladder's rung take a transport and empty it: isAvailable turns false, and a call on it rejects.

From bindings/node/guides/transport.ts:

TypeScript
import { Transport } from '@pamoja/core'
import { Delivery, Ladder } from '@pamoja/ladder'
import { LoopbackBroker } from '@pamoja/loopback'
import { Store } from '@pamoja/sync'

const TOPIC = 'sensors/1/temperature'

async function main() {
  // Whatever a link is underneath, MQTT, CoAP, or the in-process broker here, it reaches
  // the rest of the framework as a Transport, driven with the same four calls: connect,
  // subscribe, send, and recv. A node is written once and pointed at whichever link it has.
  const broker = new LoopbackBroker()
  const gateway: Transport = broker.rung()
  await gateway.connect()
  await gateway.subscribe(TOPIC)

  // The fault injector is itself a link wrapping a link, so it composes anywhere one does.
  // This one fails its next send and passes the rest through.
  const ladder = new Ladder(Store.memory())
  await ladder.rung(Transport.faulty(broker.rung(), 1))
  await ladder.connect()

  // The injected failure lands, so the reading is buffered rather than lost.
  const first = await ladder.send(TOPIC, '20.1')
  console.log(`first reading: ${first}, ${await ladder.buffered()} queued`)

  // The next reading joins the back of the queue instead of overtaking it, even though the
  // link would take it now. Order on the wire is the order the readings were taken.
  const second = await ladder.send(TOPIC, '20.4')
  const queued = await ladder.buffered()
  console.log(`second reading: ${second}, ${queued} queued`)

  // Flushing forwards the backlog oldest first, and the subscriber sees it in order.
  const forwarded = await ladder.flush()
  const earlier = (await gateway.recv())!.text!
  const later = (await gateway.recv())!.text!
  console.log(`flush forwarded ${forwarded}, gateway saw ${earlier} then ${later}`)

  return { first, second, queued, forwarded, left: await ladder.buffered(), earlier, later }
}

main()

Python#

In Python it is the same Transport class, in pamoja.core: Transport.mqtt(...), Transport.coap(...), Transport.from_handlers(...), and a broker's rung(). connect, subscribe, send, and recv are coroutines, and recv returns a Message, whose payload is bytes and whose text and number raise ValueError when the payload is not one, or None once the link has ended. asyncio.wait_for(link.recv(), seconds) gives up without losing the next message. Composing empties the handle, is_available turns false, and using it raises PamojaError, as every failed call does.

From bindings/python/guides/transport.py:

Python
import asyncio

from pamoja.core import Transport
from pamoja.ladder import Delivery, Ladder
from pamoja.loopback import LoopbackBroker
from pamoja.sync import Store

TOPIC = "sensors/1/temperature"


async def main() -> None:
    # Whatever a link is underneath, MQTT, CoAP, or the in-process broker here, it reaches
    # the rest of the framework as a Transport, driven with the same four calls: connect,
    # subscribe, send, and recv. A node is written once and pointed at whichever link it has.
    broker = LoopbackBroker()
    gateway: Transport = broker.rung()
    await gateway.connect()
    await gateway.subscribe(TOPIC)

    # The fault injector is itself a link wrapping a link, so it composes anywhere one
    # does. This one fails its next send and passes the rest through.
    ladder = Ladder(Store.memory())
    await ladder.rung(Transport.faulty(broker.rung(), 1))
    await ladder.connect()

    # The injected failure lands, so the reading is buffered rather than lost.
    first = await ladder.send(TOPIC, "20.1")
    print(f"first reading: {first}, {await ladder.buffered()} queued")

    # The next reading joins the back of the queue instead of overtaking it, even though
    # the link would take it now. Order on the wire is the order the readings were taken.
    second = await ladder.send(TOPIC, "20.4")
    queued = await ladder.buffered()
    print(f"second reading: {second}, {queued} queued")

    # Flushing forwards the backlog oldest first, and the subscriber sees it in order.
    forwarded = await ladder.flush()
    earlier = (await gateway.recv()).text
    later = (await gateway.recv()).text
    print(f"flush forwarded {forwarded}, gateway saw {earlier} then {later}")

    return first, second, queued, forwarded, await ladder.buffered(), earlier, later


first, second, queued, forwarded, left, earlier, later = asyncio.run(main())

C##

In C# it is Transport in Pamoja.Core, disposable, with ConnectAsync, SubscribeAsync, SendAsync, and ReceiveAsync, which returns a TransportMessage record, (Topic, Payload) with Text and Number, or null once the link has ended. ReceiveAsync(limit) throws TimeoutException when the time runs out without losing the next message, and calls on one transport run one at a time. MqttTransport.Open and CoapTransport.Open open the network links and Transport.FromHandlers wraps one written in .NET. Transport.Faulty, Transport.Degraded, and a ladder's Rung take a transport and empty it; IsAvailable turns false, and a call on it throws PamojaException, as every failure does.

From bindings/dotnet/samples/Pamoja.Guides/TransportGuide.cs:

C#
const string Topic = "sensors/1/temperature";

// Whatever a link is underneath, MQTT, CoAP, or the in-process broker here, it
// reaches the rest of the framework as a Transport, driven with the same four calls:
// connect, subscribe, send, and receive. A node is written once and pointed at
// whichever link it has.
using var broker = new LoopbackBroker();
using Transport gateway = broker.Rung();
await gateway.ConnectAsync();
await gateway.SubscribeAsync(Topic);

// The fault injector is itself a link wrapping a link, so it composes anywhere one
// does. This one fails its next send and passes the rest through.
using var ladder = new Ladder(Store.Memory());
ladder.Rung(Transport.Faulty(broker.Rung(), 1));
await ladder.ConnectAsync();

// The injected failure lands, so the reading is buffered rather than lost.
Delivery first = await ladder.SendAsync(Topic, "20.1");
Console.WriteLine($"first reading: {first}, {await ladder.BufferedAsync()} queued");

// The next reading joins the back of the queue instead of overtaking it, even
// though the link would take it now. Order on the wire is the order they were
// taken.
Delivery second = await ladder.SendAsync(Topic, "20.4");
int queued = await ladder.BufferedAsync();
Console.WriteLine($"second reading: {second}, {queued} queued");

// Flushing forwards the backlog oldest first, and the subscriber sees it in order.
int forwarded = await ladder.FlushAsync();
TransportMessage earlier = (await gateway.ReceiveAsync())!;
TransportMessage later = (await gateway.ReceiveAsync())!;
Console.WriteLine(
    $"flush forwarded {forwarded}, gateway saw"
    + $" {earlier.Text} then"
    + $" {later.Text}");

Values at a glance#

The contract, call by call:

CallWhat it doesRustTypeScriptPythonC#
connectestablishes the link, which every other call needsconnect().await?await t.connect()await t.connect()await t.ConnectAsync()
subscribeasks for a topic, with + for one level and # for the restsubscribe(topic).await?await t.subscribe(topic)await t.subscribe(topic)await t.SubscribeAsync(topic)
sendpublishes bytes or text to a topicsend(topic, &bytes), send_text(topic, text)await t.send(topic, bytesOrText)await t.send(topic, bytes_or_text)await t.SendAsync(topic, bytesOrText)
receivethe next message on a subscribed topic, or none once the link has endedrecv().await?await t.recv()await t.recv()await t.ReceiveAsync()
receive with a limitthe same, giving up when the time runs out and leaving the next message queuedtimeout(limit, t.recv()).awaitawait t.recv(ms)await asyncio.wait_for(t.recv(), seconds)await t.ReceiveAsync(limit)

The links that keep it:

LinkRustTypeScriptPythonC#
MQTTMqttTransport::new(config)Transport.mqtt(options)Transport.mqtt(...)MqttTransport.Open(options)
CoAPCoapTransport::new(config)Transport.coap(options)Transport.coap(...)CoapTransport.Open(options)
loopbackLoopbackTransport::new(broker)broker.rung()broker.rung()broker.Rung()
your ownimpl TransportTransport.fromHandlers(handlers)Transport.from_handlers(handlers)Transport.FromHandlers(handlers)
a fault injectorFaulty::new(link, failures)Transport.faulty(link, failures)Transport.faulty(link, failures)Transport.Faulty(link, failures)
a degraded linkDegradedLink::new(link)Transport.degraded(link, faults)Transport.degraded(link, ...)Transport.Degraded(link, ...)
a ladderTransportLadder::new(store)new Ladder(store)Ladder(store)new Ladder(store)

Rust also carries Zenoh and a LoRa mesh radio behind the same traits. Your own link writes one in each language, and Simulators has the degraded link.

What a received message holds:

PartRustTypeScriptPythonC#
the topic it was published tomessage.topicmessage.topicmessage.topicmessage.Topic
the payloadmessage.payload, a Vec<u8>message.payload, a Buffermessage.payload, bytesmessage.Payload, a byte[]
the payload as textmessage.text()?message.text, or absentmessage.text, or ValueErrormessage.Text
the payload as a numbermessage.number()?message.number, or absentmessage.number, or ValueErrormessage.Number, or null

When it goes wrong#

What the contract refuses, and what it says:

What happenedThe messageWhat to check
a transport used after it was handed onthis transport was already added to a ladder or a wrapperbuild another, or drive the thing it was handed to
a transport handed on while a call runsthis transport is busy with a callawait the call first
a link used before connect, or after it closedresource is closedconnect first, or again once it has closed
a receive given a limit ran out of timeno message arrived within 250 ms in TypeScript and C#nothing had arrived; the next message waits for the next receive
a fault injector refused a sendtransport error: simulated link failurenothing: that is its job
a payload read as text that is notthe payload is not UTF-8 text, in Pythonread payload, the bytes
a payload read as a number that is notthe payload is not a number, in Pythonread text, or check it first

How each language hands a failure over:

LanguageA failed callA receive out of timeA spent transport
RustErr(pamoja_core::Error): Transport, Closed, Codec, Io, Auth, or UnsupportedErr(Elapsed) from tokio::time::timeouta compile error, since the link was moved
TypeScripta rejected promise with an Errora rejected promisethe same, and isAvailable is false
PythonPamojaErrorasyncio.TimeoutError from asyncio.wait_forthe same, and is_available is false
C#PamojaExceptionTimeoutExceptionthe same, and IsAvailable is false

The mistakes that cost an afternoon:

  • A reading goes nowhere and nothing complains. A link was sent to after it was handed to a ladder. In Rust the compiler catches it; in the bindings the call fails, so do not swallow it.
  • A subscriber hears nothing. It subscribed after the reading went out. A link hears what is published after its subscribe, and an MQTT broker hands it an older message only when that one was retained.
  • A link that listens stops sending. A receive holds the link until a message arrives, and a handle runs one call at a time, so a send on the same handle waits behind it. Receive with a limit in a loop and send between receives, or give a task that listens a link of its own.
  • A message vanished after a timer won a race. In TypeScript and C#, a receive raced against a timer with Promise.race or Task.WhenAny keeps running after it loses, and takes the next message. Pass the limit to the receive instead, which gives up without taking anything.
  • A message arrives and the program reads nonsense. A payload is bytes; text and number only work when the sender wrote text. Agree on the encoding at both ends, or use Codecs.
  • Every send fails in a test, and not in the field. A fault injector is still in the chain. It fails the number of sends it was told to, then passes the rest through.

Where next#

  • Your own link: A link pamoja does not ship, written in your language against the transport contract and composed like any other.
  • Loopback: An in-process transport with topic matching, a fault injector, and outages on demand, for testing with no broker.
  • Transport ladder: Cheapest reachable link first, buffering to a store when every link is down.
  • Also in Transports and testing: MQTT, CoAP, Store and forward, Event bus, Simulators.

Reference#

Edit this page on GitHub