Support deinflection entries in the Yomitan format#718
Conversation
daxida
changed the title
| def _readTermBank(self, termBankName: str) -> Generator[EntryType, None, None]: | ||
| def _readTermBanks(self) -> Generator[EntryType, None, None]: | ||
| termToAlts = self._readTermBanksAlts() | ||
| orphanedTerms: set[str] = {*termToAlts.keys()} |
There was a problem hiding this comment.
set(termToAlts) is much easier to read.
There was a problem hiding this comment.
Sure, in my mind that would have made a list of items, but apparently the python convention is to retain the keys. TIL
| if altInfo := termToAlts.get(item[0]): | ||
| orphanedTerms.discard(item[0]) | ||
| alts = [elt[0] for elt in altInfo] | ||
| term = [term, *alts] if isinstance(term, str) else [*term, *alts] |
There was a problem hiding this comment.
I don't like this syntax. Let's use term + alts
Oh and if term can be a list, would term = [term, reading] above still work as expected?
Maybe this:
l_term: list[str] = term if isinstance(term, list) else [term]
l_term.extend(alts)There was a problem hiding this comment.
I'm not sure I understand what you mean with this one. I still need to pass term to newEntry below.
So you just want?:
l_term: list[str] = term if isinstance(term, list) else [term]
l_term.extend(alts)
term = l_termThere was a problem hiding this comment.
Please annotate the type of term in this block. And use two different vars for two types (list and str). Seems kinda fragile.
There was a problem hiding this comment.
I'm not entirely sure I did what you wanted. Let me know if that suits you.
|
BTW, you can the tests with Can you upload a sample Yomichan glossary that contains about 10 to 20 entries that cover these cases? So I can compare the changes and add a test. |
Sure, it will need to be hand-made because all the dictionaries I have are quite large. I will try to come up with something simple. |
|
These are the contents: [
["term", "", "", "", 5, ["Prelude"], 26, ""],
["term0", "", "", "", 5, ["Prelude0"], 26, ""],
["term1", "", "", "", 5, ["term1 definition"], 26, ""],
["term2", "reading2", "", "", 5, ["term2 definition1", "term2 definition2"], 27, ""],
["infl1", "", "", "", 1, [["term1", ["plural"]]], 28, ""],
["infl2", "", "", "", 1, [["term2", ["first cause", "second cause"]]], 29, ""],
["infl3", "", "", "", 1, [["term3", ["first cause"]]], 29, ""],
["infl4", "", "", "", 1, [["term3", ["first cause", "second cause"]]], 29, ""]
]
|
|
I added more cases. In particular:
[
["term", "", "", "", 5, ["Prelude"], 26, ""],
["term0", "", "", "", 5, ["Prelude0"], 26, ""],
["term1", "", "", "", 5, ["term1 definition"], 26, ""],
["term2", "", "", "", 5, ["term2 no reading definition"], 27, ""],
["term2", "reading1", "", "", 5, ["term2 reading1 definition"], 27, ""],
["term2", "reading2", "", "", 5, ["term2 definition1", "term2 definition2"], 27, ""],
["infl1", "", "", "", 1, [["term1", ["plural"]]], 28, ""],
["infl2", "", "", "", 1, [["term1", ["genitive"]]], 28, ""],
["infl3", "", "", "", 1, [["term2", ["first cause", "second cause"]]], 29, ""],
["infl4", "", "", "", 1, [["term3", ["first cause"]]], 29, ""],
["infl5", "", "", "", 1, [["term3", ["first cause", "second cause"]]], 29, ""]
] |
2 participants
Closes #714
Notes:
wty-el-el(a big dict, 35MB zipped, 1GB unzipped), took 42secs. It crashed before so there is no reference measurement.Examples of 3.
Cons:
Because of 2. the code is going to be slower even if there are no alts, since we can't know that unless we read once the data.
Pros:
Supports every modern yomitan dictionary that includes redirection forms as described in the issue.