I think this is an interesting idea, and I think it mostly comes down to the question of how much data we want to send through Interledger versus the Internet.

this basically removes the need for a "transport" layer altogether

I think this would be a transport protocol, but one that just happens to depend on BTP+WebSockets+TLS+TCP+IP. (One argument for doing this for now would be that it may take us a while to re-implement QUIC on top of ILP and ILP may not be great at carrying larger amounts of data at the moment. It does seem a bit annoying to need to run a WebSocket server, but it's not much worse than running an SPSP server now.)

if the receiver is a client, they will need to run a hosted WebSocket server with something like localtunnel.

That is quite insecure (just like running an SPSP server through localtunnel), because anyone that can control the localtunnel instance can redirect payments that are meant to go to you. I wouldn't recommend doing this with real money.

The types of data we have are:

• how to regenerate the fulfillment
• minimum acceptable destination amount
• actual destination amount
• total amount received, amount expected, or amount wanted
• application protocol data

The spectrum of options is:

it mostly comes down to the question of how much data we want to send through Interledger versus the Internet

I don't really agree with that assessment. This is a very clever way to setup and execute a payment denominated in the receiver amount even if data is not an important part of the transaction at all.

if the receiver is a client, they will need to run a hosted WebSocket server with something like localtunnel.

I also don't think this is necessarily true

Imagine a network as described above where Alice is the sender, she has an account with conn1 who has an account with conn2 who has an account with Bob (the receiver).

Bob want's to receive a payment from Alice so he establishes a connection with her. (i.e. Bob is the client).

Once they are communicating, Bob notifies Alice that the temporary account between them has an ILP Address of g.bob.alice, and balance of -100. (i.e. Alice owes Bob 100 of Bob's currency)

Alice starts sending ILP packets addressed to g.bob.alice using a condition only she can fulfill. When these get to Bob he forwards them to Alice to fulfill and then forwards the fulfillment on to conn2.

Each packet Bob forwards to Alice moves the account between them closer to 0.

Instead of complex transport protocols the ILP prepare packet from Bob to Alice and the ILP fulfill packet from Alice to Bob could be carried along with all of the data that needs to be exchanged between them.

It's interesting to then think whether you'd put data in the ILP packet at all, and if so what and if so, would Bob forward the same data to Alice or replace it with some other data.

This is a very clever way to setup and execute a payment

Thx! 😊

how much data we want to send through Interledger versus the Internet

Yes! People who have a specific reason to send data over the sender -> conn1 -> conn2 -> receiver path instead of directly over the more efficient sender -> receiver path can still use PSK over ILP, and the connectors will just see ILPv4 packets, and not even notice! :) For instance, if both sender and receiver are clients and running a reverse proxy tunnel is not possible for whatever reason, or if (instead of BTP) they talk a custom protocol to their ILSP (this was the main argument in the ILPv1 days, IIRC). Or if they want to use PSK as a way to hide their identity behind their ILSP. But for the vast majority of micropayment use cases, I think Interledger Full Circle is the simpler and better option.

anyone that can control the localtunnel instance can redirect payments

Not if you terminate TLS on the client, which is possible if the reverse proxy tunnel server uses SNI.

This idea was first described by @emschwartz during the original discussion about transport layer protocols. He used the name "Delayed Key Protocol" (DKP). @BobWay also advocated this option (sender fulfilling their own packets) with me many times over the years. Bob likes his loops, guess he's tired that all the examples end with him. :)

(Perhaps it's interesting to note that the ping protocol uses a sender-fulfilled condition. Although there the loop back to the sender goes over ILP instead of over IP.)

This discussion boils down to one question: Should ILP connections depend on a separate IP connection for data or be self-sufficient?

How do we decide? We could look at a number of criteria we care about and see how either approach scores:

(Note that I'm comparing the best-case scenarios. PSK is still evolving and there is no DKP implemented yet. So I'm using what I know to estimate what PSK and DKP would end up looking like, not what they look like today. Notably, I believe that we're still learning ways to make PSK/PayStream simpler. For DKP, there is a bit of "grass is greener" going on this thread. For example, congestion control is the big source of complexity in PSK and DKP would still require it.)

Ease of implementing ILP

It's probably easier to implement ILP if some of the complexities of sending data can always be assumed to be handled by some parallel IP connection. On the other hand, ILP implementations probably gain some complexity in order to interface with this separate data connection.

In terms of transport layer complexity, it may seem like using IP for data would get rid of a lot of complexity in the transport layer. We don't need to attach any data to the packet. We don't need a shared-secret scheme for generating the fulfillment.

However, I would say: Not so fast! You still need to decide how fast to send ILP packets. It doesn't matter if they contain data or not. I.e. congestion control is required in either case. And since that's probably fifty percent of the complexity of the transport protocol right there, you may not save as much complexity as you might assume.

That said, it is overall simpler to implement ILP if it doesn't support any data features at all.

This round goes to: using IP for data

Ease of use (application-layer protocol designer)

If ILP can handle both data and money, the application-layer protocol designer can choose to use it for both, while still having the option to create a separate IP connection. So it seems like by definition ILP with built-in data is strictly better from the application-layer protocol designer's perspective. Even if they don't want to send their data over ILP, in the worst case they establish a separate parallel IP connection which is always required in the using IP for data scenario.

This round goes to: self-sufficiency

Ease of use (end user)

This is difficult to judge in general, since it depends on the application-layer protocol. But I think we can once again make an argument that self-sufficient ILP will at least never be worse than ILP with IP. And on the flip-side, it will often be simpler (see the localtunnel discussion in this thread.)

This round goes to: self-sufficiency

Robustness

Doing everything over ILP is probably more robust because we have to establish and maintain only the ILP connection rather than both an ILP and IP connection between sender and receiver where either one can fail for different reasons.

This round goes to: self-sufficiency

Latency

In PSK, when a packet gets to the receiver, it is done and can be fulfilled. In DKP, it still has to start the return journey to the sender. One extra hop may not sound like much, but it's likely that ILP hops are roughly geographic (e.g. I use a local payment provider and so does the receiver) and therefore for an international payment the DKP could realistically as much as double the latency. Assuming that in-flight amount is the primary bottleneck, this could as much as half the bandwidth given the same level of risk tolerance.

This round goes to: self-sufficiency

In the end, it doesn't seem like this is even an either-or. Nobody is arguing (afaict) changing the ILP packet or connector behavior. So we can easily implement both transport protocols and compare them in practice.

@michielbdejong If you want to pursue this idea further, I'd be supportive to see where you end up. My gut feeling is that PSK will outperform loops for most use cases, but I might be wrong.

One request: Let's give this transport protocol a name (Full Circle, DKP, or something else). Calling it "BTP" or "no transport layer protocol at all" is just confusing. The sender and receiver are agreeing to use BTP in a certain way to create a return path, there are plenty of questions about connection establishment behavior and implementation questions like congestion control, so let's have a label for this work and the spec that it would ultimately go into.

we can easily implement both transport protocols and compare them in practice

Indeed, I think full-circle and semi-circle will both continue to exists, both on top of ILPv4.

If you want to pursue this idea further, I'd be supportive to see where you end up.

Thanks! l'll discuss with @emschwartz for the time-planning / prioritization of other pending stuff, and create a new PR following your indications.

This idea was first described by @emschwartz during the original discussion about transport layer protocols. He used the name "Delayed Key Protocol" (DKP). @BobWay also advocated this option

Small correction, sorry, the idea for Interledger Full Circle is more related to my previous work on LedgerLoops than to the DKP idea we discussed in the Luxembourg office shortly after I joined. That was more related to how a user of TapFi would have ~30 seconds to "undo" a payment (see explanation in the comment you quoted). Bob is obviously due a large part of the credit as he has thought a lot about circular payments and he came up with the idea of treating the chunked payment balance as just another account, in essence no different from the account between a sender and a connector. And as I alluded to in this PR's description, Evan contributed an improvement: I proposed to him that we could do fungible chunks, that don't belong to a specific payment, but he rightly remarked that routing them in the right direction would be non-trivial. He then came up with the brilliant simplification of using an address "at" the receiver, so that routing is the same for both semi-circle and full-circle ILPv4 packets.

This discussion boils down to one question: Should ILP connections depend on a separate IP connection for data or be self-sufficient?

Interesting point! I can think of four interpretations of that term "self-sufficient":

1. not using IP at all (not so interesting because it seems silly not to use ILP-over-BTP-over-IP for the individual ILPv4 links
2. we could call a layering like BTP > ILP > PSK > SPSP self-sufficient because the middle two layers don't depend on IP (even if BTP and SPSP both do). I think this interpretation is probably what you mean by the term.
3. using a direct IP connection between sender and receiver only during the pre-shared key exchange phase. This may make sense if an IP connection exists during that time, but the receiver or the sender wants to "go dark" once the key has been shared and before the chunked payment phase starts. But if that's the goal, then this ILSP might as well help with that right from the pre-shared key exchange phase, and act as a proxy at the IP level during both phases.
4. "air-gapped PSK + ILP-over-IP", where sender and receiver exchange the shared secret over an "air gap", instead of over IP - for instance using a QR code.
   Even while acknowledging that ILP always depends on IP for the underlying connector-to-connector connectivity, we could still see if a direct IP connection between sender and receiver can be avoided - especially if it can be avoided not only during the PSK chunked payment phase, but also during the pre-shared key exchange phase. I don't think this is what you mean, because we don't usually envision PSK as having an air-gapped setup phase.

congestion control is the big source of complexity in PSK and DKP would still require it

I would say the big source of complexity in PSK is remote decision making about which chunks to accept and which not, but maybe I'm overestimating that.

I would say congestion control is only how the sender decides the timing of next chunks, to achieve optimal throughput. That shouldn't be too hard if you can distinguish between chunks that get rejected due to traffic overload vs. due to insufficient destination amount, but maybe I'm underestimating that.

Even so, only semi-circle ILP requires remote decision making, full-circle ILP does not, so semi-circle is obviously an additional layer of complexity in that respect. And this extra complexity may only be desirable when it has benefits.

Apart from the complexity, we need to consider the end result (in terms of speed and cost of the payment), because the remote decision making used in semi-circle ILP will by definition be at most as good as the local decision making used in full-circle ILP.

ILP with built-in data is strictly better from the application-layer protocol designer's perspective

This point is about the interface between application layer protocols and lower layers. Interledger Full Circle requires setting up a BTP connection, whereas PSK only requires exchanging a secret, which gives application layer protocol designers more options (at least in theory). I think this relates to my second interpretation of the term 'self-sufficient'.

Pushing the dependency on IP to the lowest and highest layer, and keeping the middle layers independent from IP is an interesting goal in itself, and may have advantages if there are many different lowest layers (as was the case in the ILPv1 days), or if there are many highest-level layers. But a downside of having many different application-layer protocols is that we splinter our own market: a http-ilp sender like ilp-curl cannot talk to a web-monetization receiver.

So if we want to have a lot of different application-level protocols, then it makes sense to push the complexity down into the transport layer, and make the lives of these many application-layer protocol designers easier. However, pushing complexity down from application layer to transport layer may lead to a splintering of transport-layer protocols, so we may want to pursue the opposite goal, and try to push complexity up from transport layer to application layer.

I'll think some more about whether it's a good thing or a bad thing to have a lot of different application layer protocols.

[for the end user] self-sufficient ILP will at least never be worse than ILP with IP.
And on the flip-side, it will often be simpler (see the localtunnel discussion in this thread.)

I think this is only possibly true for my third interpretation of 'self-sufficient', where the user has the option to 'go dark' after the setup phase. But then, localtunnel will still be needed for the setup phase. I agree that some users may find a use for semi-circle interledger, but I also think it will be a niche market, a bit like Tor is a niche market for users of the web.

Just like some people will say the web, or even the internet itself, should have been designed like Tor from the start, you could say we need to design Interledger in such a way that the sender and receiver only need to have a direct IP connection during the setup phase, and during the payment phase only an indirect ILP-routed IP connection should be required, but I think it's more sane to make this sort of advanced mechanisms optional, and not encumber the default behavior with this requirement.

So I think full-circle should be the default, and semi-circle should be used in special cases where either the sender or the receiver requires the more complex remote decision making it involves, and both of them support it.

we have to establish and maintain only the ILP connection rather than both an ILP and IP connection between sender and receiver where either one can fail for different reasons.

It's true that semi-circle Interledger over n participants only requires n-1 IP connections, instead of the n connections required by full-circle Interledger.

But we need to acknowledge that usually, when an IP connection fails, this is due to a failure in the last mile - connectivity failures in the internet's backbone are exceedingly rare. If all parties (i.e., the sender, the receiver, and all n-2 connectors) can "connect to the internet", then both semi-circle and full-circle will work. If one of these participants has generic connectivity problems, then neither will work.

It will be rare that both sender and receiver can connect to their ILSP, but not to each other. Also, whenever this would happen (e.g. a sub-oceanic cable is physically damaged), the internet will route around the damage, and find a path that works. Trying to protect against this in the ILP layer would only make sense if the underlying IP layer does a bad job at that, and it comes at the non-trivial cost of complex remote decision making about which chunks the receiver should accept, and which not. I think it's cleaner to assume that the quality of each participant's connection to "the Internet" is unrelated to which IP address that participant is trying to reach.

For an international payment the DKP could realistically as much as double the latency. Assuming that in-flight amount is the primary bottleneck, this could as much as half the bandwidth given the same level of risk tolerance.

Interesting point! That's the best argument in favor of semi-circle ILP I've heard so far. I'll think about that some more, thanks!

ILP always depends on IP for the underlying connector-to-connector connectivity

Nothing in ILP says it must be built on IP. Connectors may eventually want to replace IP links between them with physical lines for performance and reliability reasons.

"air-gapped PSK + ILP-over-IP", where sender and receiver exchange the shared secret over an "air gap", instead of over IP

They could also do Diffie-Hellman over ILP (like when we were working on PSK-ECDH) to generate a shared secret and then start from something like a QR code or static record in a blockchain or identity system. I don't think we should implement TLS over ILP now because we have the luxury of using existing implementations, but it's not the crazies thing to imagine doing later.

I would say congestion control is only how the sender decides the timing of next chunks, to achieve optimal throughput.

The receiver also needs to apply backpressure if the sender is sending more than they want to receive. I think that's actually more complicated than figuring out how much the network can support, because it changes based on the application's usage of the money rather than the observed network conditions.

But a downside of having many different application-layer protocols is that we splinter our own market: a http-ilp sender like ilp-curl cannot talk to a web-monetization receiver.

To me it feels like we're converging on a stack of ILP STREAM (the combined PSK + Payment Sockets) + (Application Layer Protocol TBD) + Payment Pointer + various setup methods that depend on the context. Especially with #409, I think it'll make sense to start all ILP interactions with something like a Payment Pointer.

The receiver also needs to apply backpressure if the sender is sending more than they want to receive. I think that's actually more complicated than figuring out how much the network can support, because it changes based on the application's usage of the money rather than the observed network conditions.

I don't understand this. In a circular payment, if the receiver doesn't want any more money at this time, then they are the next to last hop in the "prepare" phase of the payment. To stop the flow, they would just not prepare the final hop back to the sender. The lack of final prepare would stop the sender from sending the fulfillment back upstream to the receiver by default.

To stop the flow, they would just not prepare the final hop back to the sender.

Ah, actually it's a bit trickier then that. If you're the sender, would you stop sending on any rejection or just on specific error codes? If you look for specific error codes but don't have a way to authenticate a message from the receiver, a connector that returns an F99 application error code (for whatever reason) could get you to stop sending when the receiver actually wants more money.

Also, it seems like this wouldn't work very well for streaming payments or ongoing interactions. Once the receiver has rejected one packet, how do they tell the sender to start sending again if they now want more money?

Once the receiver has rejected one packet, how do they tell the sender to start sending again if they now want more money?

Surely that's pretty easy if there is a direct line of communication open between them?

Surely that's pretty easy if there is a direct line of communication open between them?

If I understand the idea properly, that's only the case in what Michiel is describing as "half circle". With "full circle", the receiver would need to send ILP packets to the sender with some message in them, but there would be no way to authenticate the data and ensure that the message actually came from the receiver.

If I understand the idea properly, that's only the case in what Michiel is describing as "half circle".

The other way around. :) Semi-circle basically amounts to what we call PSK: the sender and receiver need to send coordination signals to one another (for instance, specifying a hard minimum for the destinationAmount), so that the receiver can more or less know what the sender wants to happen with each packet (i.e. accept or reject).

In full-circle, the account between sender and receiver is represented with a BTP connection, so this remote decision making is not needed. The sender sees each packet arrive full-circle, sees how it would update the account between sender and receiver, and can decide to accept or reject it.

It could be that the sender prepares a few chunks in parallel, but the exchange rate is better than expected, or their account balance is depleted more slowly than expected, and then they can decide to reject a chunk which they had prepared themselves.

The sender will also probably reject any chunks whose destination amount is lower than was expected, in the hope of getting a better deal on retry.

@emschwartz maybe the part you're missing is that this BTP connection between sender and receiver would have some sort of balance protocol on top of it, so each time some of the sender's balance is used up (e.g. a chunk of pay-per-byte video gets streamed), the sender's balance would become lower by the cost of whatever service was provided. And each time the sender fulfills one chunk, their balance at the receiver would be increased by the destination amount of that chunk. Just like the sender can send a BTP message to ask their ILSP what their balance is, they could also send such a message to the receiver.

The other way around. :) Semi-circle basically amounts to what we call PSK

Ok, can we please stick to the same terminology then?

Just like the sender can send a BTP message to ask their ILSP what their balance is, they could also send such a message to the receiver.

Would there also be a push notification from the receiver to let the sender know if the balance changes (because the receiver wants more money)?

Would there also be a push notification from the receiver to let the sender know if the balance changes (because the receiver wants more money)?

In my mind, absolutely! This new account is shared relationship just like every other account.

So with BTP, when you want to "send" money to your account counterparty,
1. you credit "his" account on your system
2. you tell him you did with a message
3. he debits "your" account on his system
With this process both connectors keep their balances immediately reconciled.

When the receiver consumes some of your pre-funded balance the same happens in the reverse order
3. he debits "your" account on his system
2. he tells you he did with a message
1. you credit "his" account on your system
Again, both connectors keep their balances immediately reconciled.

It is the nature of the "accounting relationship" that tells the sender what to do next, not an explicit "send more money" message. If Alice is OK with the balance going to zero and the service stopping then she does nothing. If she wants to keep the movie playing then she sends more to top-up her remote balance.

Ok, then there's really very little difference in the information that gets communicated. The question is just whether it goes through Interledger as data attached to the packets, or via an Internet connection.

Just like the sender can send a BTP message to ask their ILSP what their balance is, they could also send such a message to the receiver.

Interesting that an Interledger Service Provider (ILSP) is clearly a connector concept. On the other hand "get balance" is clearly a ledger provided service.

If everything uses the BTP concepts as I described them above, then there is no need for a "get balance" protocol at all. Both connectors always know what the balance is. That is the foundational principle of the "Synchronized Network Accounting Protocol" concept.

At this point it's difficult to discuss this protocol without some more details on how it will work. There already seems to be a lot of confusion about definitions and transport protocols that will be used. (E.g.: @michielbdejong says BTP between sender and receiver but I'm not sure why that is a requirement)

The premise, as I understand it, is that there is an account that exists between the sender and receiver. It may be temporary (as a result of an invoice) or longer lived (as a result of an ongoing service the receiver provides the sender). This account is represented by an ILP address that is part of the receiver's address-space. i.e. All ILP payments to that address are routed to the receiver.

The first step in this protocol is a direct communication channel between the sender and receiver. It's not important how this is established (i.e. either can be the client).

This could be a simple HTTP-based channel although it could also be Websockets or some other data transport. I think that even if it is a stateless protocol such as HTTP there is no need for both parties to run servers. If the sender is the server than the receiver will make a call to them every time a packet arrives. If the receiver is the server then the sender can make a call to them in parallel to sending the ILP packet (polling).

This sender-receiver channel will be used to exchange ILP packets (ILP Prepare from receiver to sender and ILP Fulfill/Reject from sender to receiver) but potentially other messages between the sender and the receiver that are part of this protocol.

Once this channel is established, the receiver provides the sender with the ILP address for their mutual account.

The sender then starts sending payments (ILP packets) to that address via some peer (likely the sender's ILSP).

When the ILP packets arrive at the receiver, they may inspect the body of the packet to determine some meta-data about this payment. Right now it seems like very little data is needed in the ILP packet since the address should be enough for the receiver to identify the payment. Potentially this is always empty.

If the receiver wants to accept the payment it forwards it to the sender, via the sender-receiver communication channel. The sender can now decide (based on the amount in the packet sent by the receiver) if they accept this payment.

The sender passes either an ILP Fulfill or ILP Reject packet back to the receiver who then passes this back along the payment path (ultimately getting back to the sender).

Any management of either channel (the ILP channel from the sender to the receiver via the Interledger or the direct communication channel between the sender and receiver) can be done through an exchange of messages over the latter channel.

The sender has complete control in this protocol (they can reject any payment they choose) but the receiver can still apply back-pressure by rejecting payments at any time.

Questions:

1. Does the sender-receiver channel need to be BTP? (My answer is no)
2. What kind of control messages would need to be exchanged on that channel, if any? (Seems like the existing set of error messages is enough)
3. What attacks does this open up for s sender to try and lock up liquidity?

Small correction, sorry, the idea for Interledger Full Circle is more related to my previous work on LedgerLoops than to the DKP idea we discussed in the Luxembourg office shortly after I joined.

Sorry! I missed that nuance. Let's use the name Full Circle (FC) then to differentiate it from the DKP idea.

Let's use the name Full Circle (FC) then to differentiate it from the DKP idea.

@michielbdejong would you consider this to be different to circular payments as discussed previously for use cases like selling shares or non-fungible assets?

i.e. Is this different to buying a digital asset (like a share) by sending a payment to yourself using an address where you are the receiver the current holder of the asset is the last intermediary connector?

If not, I propose we just continue to call it circular payments?

Evan's question about push notifications is a good one, and Bob's is a good answer. When discussing this with Ben last night, we hit on the same question, and my conclusion was that yes, you probably would end up doing something like push notifications, if we think the need to send such messages is a reason not to have a direct comms channel between the two parties, then somewhere, something went wrong in our reasoning. :)
The way I see it:

• we started doing all-comms-over-ILP because we said all ledgers are different, and everybody talks a different protocol
• now, we do say everybody speaks the same protocol (namely, ILPv4), so our original reason for doing PSK no longer exists
• we could continue to do PSK, because now that we have it working anyway, it doesn't even seem all that complex. This goes against our "the simplest protocol will win" principle, but it does support our "let's try to stop changing things every few weeks" goal. :)
• reduced latency might be a new reason to keep doing PSK, because it gives a higher throughput
• if you turn the situation around, and think "if full-circle was what we had, would we want to invent PSK now?", then I think we would consider PSK to be a performance optimization

Ben's conclusion was more conservative, but at least I think the whole team agrees now that we should investigate Full Circle and PSK in parallel.

Another advantage of PSK as a transport layer is that it puts little restrictions on the application layers you could build on top of it. I think having many different application layers may be a bad thing rather than a good thing, but I'll think some more about that.

In response to @adrianhopebailie's questions, yes, you could do different bilateral communication protocols to transport ILPv4 + control messages. And yes, the 'Full' in 'Full Circle' alludes to the historical evolution of the Interledger protocol, and that part is not relevant to describing what it does, so for the actual RFC we can invent a name that has Circle but not Full.

Continued in #416 and #413