Also, the file org.omg.sysml.jupyter.kernel/src/main/resources/sysml/kernel.js still sets up a CodeMirror mode for classic Jupyter Notebook. Will this still work? Do we even need to support classic Jupyter Notebook anymore?

Also, the file org.omg.sysml.jupyter.kernel/src/main/resources/sysml/kernel.js still sets up a CodeMirror mode for classic Jupyter Notebook. Will this still work? Do we even need to support classic Jupyter Notebook anymore?

No, this shouldn't work. I always go for jupyter lab. I'm not sure if there is a high demand for classic Notebook.

I noticed that the new extension does not support folding of code between braces. That is, the old mode definition included:

modeProps: {
    fold: ['brace'],
}

but I don't see anything similar in the new sysmlparser definition. If there is no way to do this in the new approach, that is a serious degradation of functionality that I know is commonly used by users.

I've tried adding code folding as an extension in the frontend plugin but I couldn't get it working unfortunately. I couldn't really find any option either that would indicate some easy turn-on.

import {codeFolding, foldGutter} from '@codemirror/language' I tried using these. These seem to create the extensions for the folding. I tried to register these with IEditorExtensionRegistry but it didn't work...

I think we need to use foldService for CodeMirror V6.

Also, the file org.omg.sysml.jupyter.kernel/src/main/resources/sysml/kernel.js still sets up a CodeMirror mode for classic Jupyter Notebook. Will this still work? Do we even need to support classic Jupyter Notebook anymore?

No, this shouldn't work. I always go for jupyter lab. I'm not sure if there is a high demand for classic Notebook.

If it is not going to work anyway, we should take it out of the build.

I think we need to use foldService for CodeMirror V6.

The problem is that the v6 foldService doesn't work for legacy language support, because it requires real parsing of the language text. As far as I could determine, it is possible to get legacy code folding to work, but one needs to write a routine to determine the scope of the folding (e.g., based on { ... } delimiters). Surprisingly, I couldn't find anything in CodeMirror or otherwise publicly available to do this. I actually got what looked like a pretty good code suggestion from AI (even though using AI for code is against my usual principles!), so I might try seeing if that can work.

In any case, if we can't find a way to get code folding incorporated fairly quickly, I am going to defer this PR to the next sprint. I don't want to deploy it without code folding.

We can use foldService with @codemirror/legacy-modes/mode but I do not know how we enable CodeMirror 6 extensions in legacy JupyterLab extensions. (As I said, I already switched to new JupyterLab, Python Wheel-based extension) Anyway, it would not be a simple task.

I am going to remove this PR from the release. We figure out code folding on the next sprint.

@seidewitz, I made ST6RI-766-1 branch and could successfully make the fold service work. Can I change the source branch to it?

@seidewitz, I made ST6RI-766-1 branch and could successfully make the fold service work. Can I change the source branch to it?

I was able to get your fold service working. However, while it seems to handle braces in comments and strings correctly, its doesn't properly handle braces in unrestricted names. E.g.:

part p {
    attribute '}';
}

Opps. mode.ts define it as 'variable'. I will fix it.

I fixed to skip variableNames as well. Could you please check it?

@himi
Looks good! I merged branch ST6RI-766-1 into ST6RI-766 and changed this PR to a ready status again.

@TheKorpos
Would you be able to review Hisashi's update?

@himi Thank you for finishing this up!

The code looks good and seems to work as intended.

I also caught a mistake I made in the packaging that caused some infinite inclusion of the output directory. I managed to get a 50GB tgz as a result... I fixed the issue by re-adding the 'files' field, that I had removed, to the package.json.

One last kind of related thing. In Jupyter 2, when you typed an open brace {, it also included the closed brace }, which were then split when you hit return. Starting with the Jupyter 3 deployment, it no longer worked this way, so that you had to explicitly type a close brace } after entering the nested lines. It seems to be this way with the fold service in Jupyter 4.

Is it possible to restore the old functionality in which braces were automatically inserted in matched pairs?

It seems to be closeBrackets extension of autocomplete and it would be nothing to do with foldServce Does Auto Close Brackets in Settings menu work for you?

It seems to be closeBrackets extension of autocomplete and it would be nothing to do with foldServce Does Auto Close Brackets in Settings menu work for you?

Ah, that does it! I hadn't even noticed that setting. I guess it had been set by default in our old installation.

Thanks!

@himi
I just noticed a strange behavior.

If I create a new notebook and enter something into the first cell, code folding does not seem to be active. If I go on to the second cell, then code folding works in that and subsequent cells. If I save the notebook, close it, and reopen it, then the first cell now has code folding.

Can you reproduce this behavior?

It seems to be closeBrackets extension of autocomplete and it would be nothing to do with foldServce Does Auto Close Brackets in Settings menu work for you?

Ah, that does it! I hadn't even noticed that setting. I guess it had been set by default in our old installation.

Thanks!

The problem seems to be that JupyterHub does not preserve the setting of the Auto Close Bracket across stopping and starting of a user's virtual server. In the old JupyterHub deployment on jupyter.openmbee.org, that option was set by default and was always reset on when the virtual server as restarted. On the new JupyterHub deployment on systemsmodeling.com, the option is not set by default and is always reset off when the virtual server as restarted.

I don't know how to change that.

could reproduce the issue that the extension added by createConditionalExtension() is not enabled only in the first cell prepared in newly created notebooks. I'm suspecting it is because that factory is not properly evaluated for such cells because by reading the code, that factory is evaluated when new cells are created. I'm guessing it is an issue of JupyterLab.

I found one solution by using createImmutableExtensions() instead of createConditionalExtension(). But we will not be able to configure the editor extensions with settings. Is it ok for you?

It seems I'm missing something. I will investigate more.

I found the first cell in newly created notebooks is initialized as 'text/plain' and it won't enable foldService. So I enabled foldSerivce to 'text/plain' as well and check if the language is sysml also in the foldService. For this purpose, I named the language as sysml instead of clike in mode.ts

There is a template for mode.ts. That should be updated as well in case mode.ts requires changes.

@TheKorpos, thanks. I updated mode.ts and mode_template.ts.

OK, folding works in the first cell now. Thanks!

But that does seem harder than it should be...