Skip to content

REST saga 3/3: REST API for subservers, websocket for streaming responses#4141

Merged
cfromknecht merged 4 commits into
lightningnetwork:masterfrom
guggero:rest-api-improvements
Jun 17, 2020
Merged

REST saga 3/3: REST API for subservers, websocket for streaming responses#4141
cfromknecht merged 4 commits into
lightningnetwork:masterfrom
guggero:rest-api-improvements

Conversation

@guggero

@guggero guggero commented Apr 2, 2020

Copy link
Copy Markdown
Collaborator

This is part 3 of 3 of the REST improvement saga which will ultimately replace #4141:

Depends on #4251, only the last 4 commits are new.

Updated API docs for unified gRPC/REST APIs can be previewed here.

Fixes #3622
Fixes #4222
Fixes #2344
Fixes #2730
Fixes #4100
Fixes #4229
Fixes #3430

@guggero guggero added this to the 0.11.0 milestone Apr 2, 2020
@guggero guggero force-pushed the rest-api-improvements branch 4 times, most recently from dbfeff0 to be034ad Compare April 7, 2020 07:07
@Roasbeef Roasbeef added enhancement Improvements to existing features / behaviour REST rpc Related to the RPC interface v0.11 labels Apr 14, 2020
This was referenced May 5, 2020
Comment thread rpcserver.go Outdated
@cfromknecht cfromknecht self-requested a review May 5, 2020 18:51
@guggero guggero changed the title [WIP]: REST API for subservers, websocket for streaming responses [WIP] REST saga 3/3: REST API for subservers, websocket for streaming responses May 6, 2020
@guggero guggero force-pushed the rest-api-improvements branch from be034ad to e17cbc5 Compare May 6, 2020 15:55
@guggero guggero force-pushed the rest-api-improvements branch from e17cbc5 to 7315f73 Compare May 23, 2020 12:46
@guggero guggero force-pushed the rest-api-improvements branch 2 times, most recently from 5fdd6ef to 56af0e1 Compare June 3, 2020 16:21
@guggero guggero changed the title [WIP] REST saga 3/3: REST API for subservers, websocket for streaming responses REST saga 3/3: REST API for subservers, websocket for streaming responses Jun 3, 2020
@guggero guggero marked this pull request as ready for review June 3, 2020 16:29
@guggero guggero requested a review from Roasbeef as a code owner June 3, 2020 16:29
@guggero guggero force-pushed the rest-api-improvements branch from 56af0e1 to 28581be Compare June 4, 2020 08:28
@kornpow

kornpow commented Jun 5, 2020

Copy link
Copy Markdown
Contributor

Hey Guggero, I started trying out some of the endpoints tonight. I may have found some quirks that can help with the process.

POST /v2/wallet/address/next
Cannot send empty string data in POST body
My code has an empty string in the POST body, and that has worked so far for v1 endpoints, though I'm not sure if its really a good or bad thing. Doing it for this endpoint gave Go unmarshalling errors.

This next endpoint is the big one everyone is probably eager to use
POST /v2/router/send
It seems like it is returning data for each attempt. It is returning multiple dictionaries with {"result":} keys, when I think it should only return one. At its current state, I cannot do json.loads({}).

I was splitting along the newline to watch as it tried the attempts:
for i in range(0,19): len(json.loads(r.text.split('\n')[i])['result']['htlcs'])

I also tried out: GET /v2/router/htlcevents
And it seemed to work, but you saidwebsockets weren't working yet, and think these streaming ones will work better with websockets instead of the python "for loop" syntax, so I might play around with that more later.

POST /v2/router/send return data:

'{"result":{"payment_hash":"5eb8b650f4708209bf39e14f09d05690991dc9ad949f60d4b8df1bad326819fe","value":"1000000","creation_date":"1591321010","fee":"0","payment_preimage":"0000000000000000000000000000000000000000000000000000000000000000","value_sat":"1000000","value_msat":"1000000000","payment_request":"lnbc10m1p0dngdfpp5t6utv585wzpqn0eeu98sn5zkjzv3mjddjj0kp49cmud66vngr8lqdq2w3jhxapqxgcqzpgsp52tnfp5dzv7w50pg42dd0lz6tgp5fpdn3c3ythpverj0hw2zkc4ks9qy9qsqv7d3qwkpz32dlhqrpf5nsv9f76xfd8yz6m5fw8lakc3z9f8l03usxcvzgyyh753elyds7ckdrw2rdc9ue46p4aevhsh9a9z5u78ufwcqe5myaz","status":"IN_FLIGHT","fee_sat":"0","fee_msat":"0","creation_time_ns":"1591321010843009472","htlcs":[],"payment_index":"3856","failure_reason":"FAILURE_REASON_NONE"}}\n{"result":{"payment_hash":"5eb8b650f4708209bf39e14f09d05690991dc9ad949f60d4b8df1bad326819fe","value":"1000000","creation_date":"1591321010","fee":"0","payment_preimage":"0000000000000000000000000000000000000000000000000000000000000000","value_sat":"1000000","value_msat":"1000000000","payment_request":"lnbc10m1p0dngdfpp5t6utv585wzpqn0eeu98sn5zkjzv3mjddjj0kp49cmud66vngr8lqdq2w3jhxapqxgcqzpgsp52tnfp5dzv7w50pg42dd0lz6tgp5fpdn3c3ythpverj0hw2zkc4ks9qy9qsqv7d3qwkpz32dlhqrpf5nsv9f76xfd8yz6m5fw8lakc3z9f8l03usxcvzgyyh753elyds7ckdrw2rdc9ue46p4aevhsh9a9z5u78ufwcqe5myaz","status":"IN_FLIGHT","fee_sat":"0","fee_msat":"0","creation_time_ns":"1591321010843009472","htlcs":[{"status":"IN_FLIGHT","route":{"total_time_lock":633190,"total_fees":"2","total_amt":"1000002","hops":[{"chan_id":"694210751119753216","chan_capacity":"4388592","amt_to_forward":"1000000","fee":"2","expiry":633150,"amt_to_forward_msat":"1000000000","fee_msat":"2000","pub_key":"0292052c3ab594f7b5e997099f66e8ed51b4342126dcb5c3caa76b38adb725dcdb","tlv_payload":true,"mpp_record":null,"custom_records":{}},{"chan_id":"694210751119753216","chan_capacity":"4388592","amt_to_forward":"1000000","fee":"0","expiry":633150,"amt_to_forward_msat":"1000000000","fee_msat":"0","pub_key":"0360a41eb8c3fe09782ef6c984acbb003b0e1ebc4fe10ae01bab0e80d76618c8f4","tlv_payload":true,"mpp_record":{"payment_addr":"UuaQ0aJnnUeFFVNa/4tLQGiQtnHESLuFmRyfdyhWxW0=","total_amt_msat":"1000000000"},"custom_records":{}}],"total_fees_msat":"2000","total_amt_msat":"1000002000"},"attempt_time_ns":"1591321010865159252","resolve_time_ns":"0","failure":null,"preimage":null}],"payment_index":"3856","failure_reason":"FAILURE_REASON_NONE"}}\n{"result":{"payment_hash":"5eb8b650f4708209bf39e14f09d05690991dc9ad949f60d4b8df1bad326819fe","value":"1000000","creation_date":"1591321010","fee":"2","payment_preimage":"7f2fc0cca0ecc486929e8b9cbf9659902748d483327c95d373956f3f556dcd05","value_sat":"1000000","value_msat":"1000000000","payment_request":"lnbc10m1p0dngdfpp5t6utv585wzpqn0eeu98sn5zkjzv3mjddjj0kp49cmud66vngr8lqdq2w3jhxapqxgcqzpgsp52tnfp5dzv7w50pg42dd0lz6tgp5fpdn3c3ythpverj0hw2zkc4ks9qy9qsqv7d3qwkpz32dlhqrpf5nsv9f76xfd8yz6m5fw8lakc3z9f8l03usxcvzgyyh753elyds7ckdrw2rdc9ue46p4aevhsh9a9z5u78ufwcqe5myaz","status":"SUCCEEDED","fee_sat":"2","fee_msat":"2000","creation_time_ns":"1591321010843009472","htlcs":[{"status":"SUCCEEDED","route":{"total_time_lock":633190,"total_fees":"2","total_amt":"1000002","hops":[{"chan_id":"694210751119753216","chan_capacity":"4388592","amt_to_forward":"1000000","fee":"2","expiry":633150,"amt_to_forward_msat":"1000000000","fee_msat":"2000","pub_key":"0292052c3ab594f7b5e997099f66e8ed51b4342126dcb5c3caa76b38adb725dcdb","tlv_payload":true,"mpp_record":null,"custom_records":{}},{"chan_id":"694210751119753216","chan_capacity":"4388592","amt_to_forward":"1000000","fee":"0","expiry":633150,"amt_to_forward_msat":"1000000000","fee_msat":"0","pub_key":"0360a41eb8c3fe09782ef6c984acbb003b0e1ebc4fe10ae01bab0e80d76618c8f4","tlv_payload":true,"mpp_record":{"payment_addr":"UuaQ0aJnnUeFFVNa/4tLQGiQtnHESLuFmRyfdyhWxW0=","total_amt_msat":"1000000000"},"custom_records":{}}],"total_fees_msat":"2000","total_amt_msat":"1000002000"},"attempt_time_ns":"1591321010865159252","resolve_time_ns":"1591321011235110235","failure":null,"preimage":"fy/AzKDsxIaSnoucv5ZZkCdI1IMyfJXTc5VvP1VtzQU="}],"payment_index":"3856","failure_reason":"FAILURE_REASON_NONE"}}\n'

Comment thread lnrpc/websocket_proxy.go Outdated
Comment thread lnrpc/websocket_proxy.go Outdated
Comment thread lntest/itest/rest_api.go Outdated
Comment thread lntest/itest/rest_api.go Outdated
@guggero

guggero commented Jun 5, 2020

Copy link
Copy Markdown
Collaborator Author

@sako0938 thank you very much for testing!

Cannot send empty string data in POST body

I don't think sending an empty POST worked before that PR. Or maybe it changed because of the library update. But in any case, a POST body always has to be set. If there's no content to send, just send a "{}".

It seems like it is returning data for each attempt. It is returning multiple dictionaries with {"result":} keys, when I think it should only return one. At its current state, I cannot do json.loads({}).

That's exactly why WebSockets are needed, because there are multiple messages being returned that are valid JSON each on its own. Adding WebSocket support is the whole point of this PR (while adding the REST endpoints was done in PR 2/3). So please test again with a WS library (code examples can be found in the preview of the API doc: https://www.guggero.org/lightning-api/#v2-router-send)

but you saidwebsockets weren't working yet

Was I referring to this PR specifically? Maybe there was a misunderstanding. They should work now :)

@guggero guggero force-pushed the rest-api-improvements branch from 28581be to 1fde724 Compare June 12, 2020 12:09
@guggero guggero requested a review from Roasbeef June 12, 2020 12:14

@Roasbeef Roasbeef left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🏦

@guggero guggero force-pushed the rest-api-improvements branch from 1fde724 to e86d73d Compare June 17, 2020 06:35
@guggero guggero force-pushed the rest-api-improvements branch from e86d73d to 1cda467 Compare June 17, 2020 06:35
@cfromknecht cfromknecht merged commit 87880c0 into lightningnetwork:master Jun 17, 2020
@guggero guggero deleted the rest-api-improvements branch June 18, 2020 06:23
bhandras pushed a commit to bhandras/lnd that referenced this pull request Jun 18, 2020
…ements

REST saga 3/3: REST API for subservers, websocket for streaming responses
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Improvements to existing features / behaviour REST rpc Related to the RPC interface

Projects

None yet

4 participants