In order to allow my time to be split between all the PRs/work on CrossPoint, I'll be reviewing #219 and getting it merged before coming back to review this one.

Would it be possible to specify a custom server instead of using a default one? For instance, I host a custom server on my NAS in a Docker and would like to use it: https://github.com/koreader/koreader-sync-server.

Yes but I felt this was easier to validate to start. Do custom servers use the exact same system as the official one?

Yes, the protocol is the same, only the URL would be different.

Ok sure I'll add it tonight

Would it be possible to specify a custom server instead of using a default one? For instance, I host a custom server on my NAS in a Docker and would like to use it: koreader/koreader-sync-server.

This is done and tested with https://sync.send2ereader.net. I also implemented file name matching because until the calibre wireless device stuff I did is merged I can't seem to get the hashes to match. I tested pushing and pulling and it works great.

I wanted to test this feature because it is something I also want in crosspoint, thank you for implementing. I am not sure I have the time and the technical know how to help out but I tried testing it:

I have built and flashed the pr branch and added https://sync.send2ereader.net on both my PocketBook and xteink4 onto the same account with only one .epub book with the same filename. However I push or pull progress it never matches up between the devices. The TOC has multiple books in it as it is the complete Witcher series.
Case 1.

1. On xteink device I go to Blood of Elves, Chapter 2. turn pages to 34 within the chapter.
2. Go to TOC, sync progress»upload local progess, progess up:dated appears.
   progress found screen said
   Local:
   Blood of Elves
   Page 34/116 23% overall
3. On Koreader go to progress sync, pull from other devices, it says sync successful
   Expected outcome:
   I am on the same page within Chapter 2
   Actual outcome:
   I am on the first page of Chapter 2 of the Book of book on Koreader. (page nr. 1425/5967)

Case 2.

1. On Koreader navigate to book 4 chapter 1, and turn 10 or so pages from chapter beginning (1350/5967)
2. Go to settings»progress sync»push progress from this device now
3. On xteink go to TOC, sync progress» apply remote progess, progess updated appears.
   progress found screen said:
   Remote:
   Blood of Elves
   Page 95, 23% overall
   From: PBTouchLux5
   Local:
   Blood of Elves
   Page 34/142
   Expected outcome:
   I am on the same page withing Chapter one as intended or equivalent
   Actual outcome:
   I am on page 95 of 142, but that page does not correspond to the Koreader page, which would be 1396/5967.

I tried to be concise, but might have failed and I can send you the epub to test with or I might have messed something up. I think this probably has to do with how the TOC and progress is being handled.
EDIT: Koreader is latest release version for pocketbooks, I installed it just for this.

Yes please send me the book. I am not sure if there's a unique setup with that book having multiple books inside it? Have you tried just using KoReader on two devices without crosspoint involved and seeing if it syncs fine? The way crosspoint handles progress is a bit different than koreader so I did % based mapping to the chapter.

You can upload the file here for me: https://files.sofriendly.com/i/2Xwdh4F/itsthisjustin

Uploaded. I will try installing Koreader on my phone as I only have the Pocketbook and Xteink.

Uploaded. I will try installing Koreader on my phone as I only have the Pocketbook and Xteink.

Ha well now that alt sync server appears to be down so I can't test anymore

With android and pocketbook koreader no matter what I do, it seems to think the latest progress was from the device Im trying to sync from. Hopefully this isn't an error message for the backup service being down. Sigh.

Nevermind figured out what I messed up, it works with Koreader on android and Koreader on pocketbook as expected.

Nevermind figured out what I messed up, it works with Koreader on android and Koreader on pocketbook as expected.

Ok I'll take a look.

Given #219 is now merged, I'm keen to dive into this one over the next day or so, I assume it's still ready for a look @itsthisjustin?

Given #219 is now merged, I'm keen to dive into this one over the next day or so, I assume it's still ready for a look @itsthisjustin?

Code quality wise yes. I still plan to see if there's some special case needed for compendium style books like the entire witcher series. This sync works for my simple 15 page epub I've been testing with though

Sorry for the delay in getting to this, I plan to look at it on Wednesday.

So @daveallie due to the way the epub parser works compared to how KOReader works, we'd have to have character level tracking in the epub to get EXACT sync. What I've done for now is basically chapter level sync unless you want to go down the rabbit hole of trying to do a deeper level conversion which could be insanely processor intensive. Chapter sync is at least super helpful for a book this large which doesn't even contain a TOC on a per book basis making it insanely hard to manually navigate this (i.e. the sync feature here is still super useful even if it's not granularly syncing down the sentence)

@itsthisjustin do you happen to know the purpose of the progress variable? From what I remember, resuming on another device with KO sync doesn't actually reference the progress, just jumps to the location itself. Perhaps that variable would be more useful to estimate where to resume going from other KO-device to the xteink, while the other way around would just have chapter-level syncing? Does that make sense?

I think it makes sense but it would require the same level of calculation I believe?

Yes, but might be able to save time by not calculating the progress when uploading from crosspoint if the device running KOReader is going to calculate it itself once it locates the xpath, so maybe the toKOReader() function of progressmapper isn’t necessarily needed and can be dummy’d in the upload packet.

Yes, but might be able to save time by not calculating the progress when uploading from crosspoint if the device running KOReader is going to calculate it itself once it locates the xpath, so maybe the toKOReader() function of progressmapper isn’t necessarily needed and can be dummy’d in the upload packet.

So I like it there because we can compare the progress to see which copy is further ahead. Helpful for determining oh yeah I was way further along on this device, I want to pull that progress. Hopefully I'm understanding what you're saying

Hey Justin - just trying to set this up. First time using KOreader on my Kobo.

Do you recommend using file name matching or hash matching? You mention MD5 hashing in the PR description.

I don't have either with me right now and I'm killing time so apologies for not trying myself.

I'm super excited to get this going though so thanks for your hard work on this.

I personally use file name as it was way easier to transfer the files over USB prior to calibre wireless device syncing working

Got this working now with the Calibre Library feature you added too!
I was having an issue where my Kobo was downloading with the file name as Author - Title, and the XTeink was the opposite. Edited some lua and the file name sync now works!!
I couldn't get binary sync working.

I'm a little bit confused. With this merge, it doesn't behave 1:1 like the official koreader sync right? Since crosspoint koreader sync only syncs the chapter that you're currently in and not where you're actually in the chapter. What happens if my epub doesnt have chapters?

I'm a little bit confused. With this merge, it doesn't behave 1:1 like the official koreader sync right? Since crosspoint koreader sync only syncs the chapter that you're currently in and not where you're actually in the chapter. What happens if my epub doesnt have chapters?

So it uses a combination of the % estimate and page number to try to find the closest spot. If an epub has no concept of chapters I think it will just estimate percentage and land somewhere close vs rounding down to start of chapter

@itsthisjustin / @bartiprime I have exactly the same question/issue. I'm only getting synced to beginning of the chapter which is not useful when the book I'm reading has big chapters. Any idea what is going wrong?

@Jelle-elaut would it be possible to test if this PR solves kosync issues? #783