Partial-Content - Transmitting file chunks over nostr#112
Partial-Content - Transmitting file chunks over nostr#112Egge21M wants to merge 6 commits intonostr-protocol:masterfrom Egge21M:master
Conversation
|
Please don't do this. |
|
@fiatjaf care to elaborate what problems you see if it is optional and Relays can decide wether to offer it or not? |
Semisol
left a comment
There was a problem hiding this comment.
Issue with this is that nostr is not designed for large files, this is just a hack to store files on Nostr. You get the overhead of:
- database queries for each chunk
- wasted bytes due to hex and event data
- relays were not meant for this
instead, file servers can exist, storing large files or chunks of files (at most ~5 chunks per file), which are suited for storing large files and not small JSON events that need indexing and have different uses
|
We could avoid query-hell by introducing a file-id tag that would be consistent over all chunks. Relays could index file-ids to allow a fast retrieval of all required events from database and would not need to look up all event-ids. Entry points would no longer need to specify all ids and it would allow transmitting live content, as chunks of data could be added while transmission has already begun. |
|
agreed with @fiatjaf this is a bad idea |
|
@fernandolguevara of course you can, but how would you share it with other clients without having a single point of failure? File servers and URLs in notes is what people use today, but it is not censorship resistant. Maybe we could start to mirror content on different servers and include URLs to all mirrors in a note... |
|
IPFS was created to solve this problem specifically. If it works, you can use that, if it doesn't work then we must make a new protocol to solve that problem. The problem is too big in itself to be solved with an unsustainable hack like this. |
|
@fiatjaf I am mostly aligned when it comes to long term storage of files, especially because of the inefficiencies with the approach proposed here. We will continue working on it. Now where Partial-Content shines IMO (especially if we replace the event-is-list proposal with a static file-id as proposed here: #112 (comment)) is live content that is ephemeral. We could have nostr spaces where audio is streamed to a relay and then to clients listening to the file-id. Of course not every Relay will be able to handle the load, so Relays can choose wether to support it or pay-gate it. |
|
But this is so inefficient. Considering that clients will already have to implement a very custom way of transmitting audio or files and the same for relays, wouldn't it be better to spec a more efficient protocol that used binary streams over WebSocket directly? Then relays that wanted to support it would have an easier time adding support, and client integration would be simpler too. |
|
NACK |
|
I have thought about implementing it as well, and I think the best way would be to send an epheremal DM with file metadata and hash (or magnet hash) and a websocket address that the other party can connect to. After establishing a direct connection, it's rasy to send the file. |
|
Just a bit of a followup: nostr already has deletion, so whenever the websocket address changes, the client could just delete the older dm. |
But that won't be using nostr to save the file on relays. So does it require a NIP? |
This draft introduces
So far there are no clients or relays implementing these. I have written an article about a proof-of-concept that I did: https://starbackr.com/sending-images-and-files-on-nostr/