feat(lua): add vim.iconv #18286

lewis6991 · 2022-04-27T13:14:57Z

Identical to vim.fn.iconv except:

There is no typval conversion.
Accepts any Lua string, whereas typval conversion may produce a Blob type which causes vim.fn.iconv to fail.
Does not rely on NUL byte for string length detection.
Can be run in fast events.
Can be run in Lua threads (via vim.loop.new_work).

Demo

Here is a screenshot of Gitsigns opening a PNG file that is managed under git.

(left vim.iconv, right vim.fn.iconv)

vim.fn.iconv:

   for i, l in ipairs(stdout) do
      -- Have to exclude Blob types, otherwise conversion will error.
      if vim.fn.type(l) == vim.v.t_string then
        stdout[i] = vim.fn.iconv(l, encoding, 'utf-8')
      end
    end

vim.iconv:

    for i, l in ipairs(stdout) do
      stdout[i] = vim.iconv(l, encoding, 'utf-8')
    end

justinmk

It is not sustainable for us to keep adding hand-crafted wrappers around Vimscript eval functions, just to avoid typval conversion, Blob, etc.

Accepts any Lua string, whereas typval conversion may produce a Blob type which causes vim.fn.iconv to fail.

Does not rely on NUL byte for string length detection.

Why can't these things be done by the Lua <=> Vimscript bridge? Can we annotate certain functions like f_iconv so that the bridge knows when we want a Lua caller to get this behavior?

vim.fn.iconv() should just do these things implicitly.

runtime/doc/lua.txt

lewis6991 · 2022-04-27T21:54:10Z

Why can't these things be done by the Lua <=> Vimscript bridge? Can we annotate certain functions like f_iconv so that the bridge knows when we want a Lua caller to get this behavior?

I've looked into this and we could annotate the function so it can run in a fast event, we should do this. I'll raise another PR at some point with all the relevant plumbing to do this for all functions that can. We'll need to somehow document this too. (EDIT: #18306)

The string typval conversion really doesn't work from Lua->vimscript for the main reason being typval strings don't store their length and need to rely on NUL termination.

It is not sustainable for us to keep adding hand-crafted wrappers around Vimscript eval functions, just to avoid typval conversion, Blob, etc.

I agree with this in general, but I don't think this needs to be a hard rule, especially for functions which are just basic bindings onto third-party libs (xdiff, iconv, etc).

I spent around two hours trying to fix f_iconv to support blob types, but didn't really get anywhere. Throwing together a basic Lua binding took about 30 minutes and only required standard Lua-C boilerplate.

justinmk · 2022-04-30T04:50:44Z

I spent around two hours trying to fix f_iconv to support blob types, but didn't really get anywhere. Throwing together a basic Lua binding took about 30 minutes and only required standard Lua-C boilerplate.

I wouldn't think we need f_iconv to support blob. If we're in Lua we can skip blob conversion and use only Lua strings? (Assuming the "bridge plumbing" comes to fruition)

lewis6991 · 2022-04-30T09:40:19Z

I think for this specific instance, I don't know how/if we can fix the bridge plumbing to make this work. The issue here is pretty integral to typval, and I would prefer to keep any changes to typval minimal/none.

Do you have any ideas?

Otherwise this PR is the best solution.

lewis6991 · 2022-04-30T10:29:22Z

If we're in Lua we can skip blob conversion and use only Lua strings?

Lua strings store their length, typval strings do not. f_iconv only has access to typval arguments.

justinmk · 2022-04-30T14:58:36Z

Lua strings store their length, typval strings do not. f_iconv only has access to typval arguments.

~~We could add typval_T.v_string_len. Then annotated "Lua bridge functions" such as f_iconv would need to set rettv->vval.v_string_len before they return.~~

Can't the Lua bridge generator code get the string length before passing it to Lua? It's not efficient, but if it works, we can think about optimizations later.

lewis6991 · 2022-05-01T10:35:03Z

I had a go at sizing typval strings before I saw your edit, #18339. As you can see, it's pretty hideous and still crashes as the new size field isn't being properly initialized somewhere despite my efforts.

Can't the Lua bridge generator code get the string length before passing it to Lua? It's not efficient, but if it works, we can think about optimizations later.

Not sure I fully understand this comment, but we need to pass the string length to typval and then pass it back to Lua, as f_iconv needs access to it. This is what #18339 does in addition to preventing conversion to Blob objects.

I believe messing with typval is the wrong call here. Not to mention any solution of this route will involve changes to f_iconv (and typval) which can't be applied back to Vim. In addition, anything in vim.fn.* can't (currently or ever) be run in Lua threads, which would be particularly useful for this function.

Let's leave legacy stuff as legacy and focus our efforts creating new better stuff using more robust and documented API's. It's nice the Lua<->Vimscript bridge exists, but this is really just a stop-gap rather than a first class solution. Ideas like #18306 are good because they are fairly low cost with high return.

With this PR we can effectively obsolete f_iconv which will only continue to exist for compat for older viml plugins, assuming any exist that use this function.

lewis6991 · 2022-05-03T16:16:40Z

It is not sustainable for us to keep adding hand-crafted wrappers around Vimscript eval functions, just to avoid typval conversion, Blob, etc.

Looping back on this. I opened #18393 to get an actual picture of where we are with Lua stdlib parity with vimscript. Turns out we've implemented ~50%, and much of the remaining 50% won't likely want/need a Lua equivalent. Additionally, #18306 will relieve the need to add a large fraction of remaining function. So, from that I don't agree with the statement.

The benefits gained and issues resolved from a Lua binding of iconv (which are entirely specific to the function at hand, with maybe the exception of the sha256 function), fully justify its inclusion and spending more effort on the Lua-vimscript bridge isn't going to be time well spent (IMO).

justinmk · 2022-05-04T15:08:17Z

I opened #18393 to get an actual picture of where we are with Lua stdlib parity with vimscript. Turns out we've implemented ~50%, and much of the remaining 50% won't likely want/need a Lua equivalent. Additionally, #18306 will relieve the need to add a large fraction of remaining function.

That's fantastic news. Thank you for digging into that. If we have a way forward that doesn't involve continuing to hand-craft Lua versions of Vimscript functions, that removes the major concern here.

we need to pass the string length to typval and then pass it back to Lua, as f_iconv needs access to it.

"pass it back to Lua" is done by the bridge. Can't the bridge do a strlen() there?

Not to mention any solution of this route will involve changes to f_iconv (and typval) which can't be applied back to Vim.

We must not make architecture decisions based on this. Vim-compat is "best effort" and has been for at least 2 years. It has zero weight on choices that have big maintainability and future impact on Nvim. Vim's typval is also essentially frozen since Vimscript9.

Let's leave legacy stuff as legacy and focus our efforts creating new better stuff using more robust and documented API's. It's nice the Lua<->Vimscript bridge exists, but this is really just a stop-gap rather than a first class solution

We definitely should create better interfaces where the old one is broken. But rewriting all of the eval functions is a big waste of time. Those functions are just editor functions, there's nothing yucky about them except typval conversion. If we can make the bridge work well, then that legacy is massively valuable and avoids a huge amount of redundant work, maintenance burden, documentation burden, and user cognitive burden.

justinmk · 2022-05-04T15:11:20Z

If vim.fn.iconv is broken in Lua (based on the screenshot), we should throw an error when it's called from Lua.

EXX: use vim.iconv() instead

clason · 2022-05-04T15:14:02Z

We already have a mechanism to error out when vim.fn.foo is called when vim.api.nvim_foo exists (#13875) -- maybe that check can be extended?

erw7 · 2022-05-05T05:21:45Z

src/nvim/lua/stdlib.c

+
+  // vim.iconv
+  lua_pushcfunction(lstate, &nlua_iconv);
+  lua_setfield(lstate, -2, "iconv");


iconv_string accesses p_ambw and p_emoji internally. Therefore, it is not thread-safe. I do not agree with exposing vim.iconv to threads as it is currently implemented, although the cases where it becomes a problem may be rare.

Thanks for catching this.

I'm not sure what the code in question is intended to do, but I don't think it makes sense to replace characters that cannot be converted with vim.iconv() with the number of cell widths "?".

lewis6991 · 2022-05-17T13:52:28Z

If vim.fn.iconv is broken in Lua (based on the screenshot), we should throw an error when it's called from Lua.

It's only broken for strings that contain NUL characters, and this is already documented (more or less):

		Note that Vim uses UTF-8 for all Unicode encodings, conversion
		from/to UCS-2 is automatically changed to use UTF-8.  You
		cannot use UCS-2 in a string anyway, because of the NUL bytes.

This new version doesn't have this limitation.

There might be applications that don't need to worry about handling NUL bytes, so I think it would be a mistake to now suddenly make those applications error.

lewis6991 · 2022-05-17T15:07:02Z

"pass it back to Lua" is done by the bridge. Can't the bridge do a strlen() there?

strlen simply counts characters until it reaches NUL. Lua strings are allowed to contain NUL, so strlen won't work properly. The size needs to be carried the whole way through, it's information that can't be reproduced once it is gone.

But rewriting all of the eval functions is a big waste of time.

I dunno. Rewriting typval seems like a bigger waste of time (and less enjoyable). We could probably just implement everything we don't have a Lua version of in vim.fn.*, and (IMO) it would probably take less time than re-architecting typval to work in Lua's full domain. I may be underestimating, but I don't think there's a huge amount when only considering the functions we would actually want to use. Not that I think we should do this, because we simply don't need to. It's just much easier to re-implement the edge-case functions that have issues.

Also given how the Lua-C API is a real documented API, these new versions will be much more approachable to new-comers. It's how I got involved. I looked at nvim/lua/treesitter.c, Googled Lua-C API, and I was away.

Those functions are just editor functions, there's nothing yucky about them except typval conversion.

We still don't have a solution to the fact that typval doesn't have boolean values, which makes using them in Lua awkward and error-prone if you forget to compare against 0 .... which I've done many times! Not really sure how/if we can approach this in the bridge.

If we can make the bridge work well, then that legacy is massively valuable and avoids a huge amount of redundant work, maintenance burden, documentation burden, and user cognitive burden.

I agree it is valuable, but for me it is more "we can learn from this" rather than "we can use and improve this". I would hope eventually vim.fn.* will become obsolete and thus deprecated (and only kept for legacy), which should remove those burdens.

One example, today I found out bufwinid() only works for the current tabpage, which I think is a bit weird. Why only the current tabpage? If we want to fix this, do we add a new option to the eval version of bufwinid, or can we just make something new that does and behaves exactly as we want with all the nice Lua ergonomics? I know which I'd prefer 😉 .

Anyway thanks for approving this.

lewis6991 · 2022-05-18T10:28:02Z

I still need to exclude (or fix) this from running in threads.

lewis6991 · 2022-05-18T11:07:48Z

Note I've needed to add #if defined(HAVE_ICONV) guards, since iconv is not a required build dependency.

erw7 · 2022-05-21T03:20:22Z

Crash(segmentation fault) with the following steps:

nvim --clean
:lua vim.iconv('a', 'foo', 'bar')

I think it is necessary to do the same as the following process of f_iconv, or call luaL_error() if ret is NULL. I think iconv() in vim script is insufficient for some purposes, such as not being able to know if there are characters that cannot be converted. It might be better to change it to vim.iconv({string}, {from}, {to} [, {opt}]) and allow fail and replacement character to be specified by opt.

neovim/src/nvim/eval/funcs.c

Lines 4620 to 4622 in b3453ea

    
           // If the encodings are equal, no conversion needed. 
        
           if (vimconv.vc_type == CONV_NONE) { 
        
             rettv->vval.v_string = xstrdup(str);

lewis6991 · 2022-06-14T10:50:53Z

@erw7:

Fixed the crash
Added an opts argument with a fail field
Didn't add replacement text as that is less-trivial and can be done in a future PR
Added a few tests
Also changed the return to be nil when conversion fails as that is more "Lua".

justinmk · 2022-07-05T13:46:01Z

asan failure:

[ RUN      ] lua stdlib vim.iconv can validate arguments: 29.55 ms OK
==================== File /home/runner/work/neovim/neovim/build/log/ubsan.36031 ====================
= 
= =================================================================
= ==36031==ERROR: LeakSanitizer: detected memory leaks
= 
= Direct leak of 11 byte(s) in 1 object(s) allocated from:
=     #0 0x4de7ad in malloc ??:?
=     #1 0x1267e92 in try_malloc /home/runner/work/neovim/neovim/src/nvim/memory.c:74:15
=     #2 0x126808c in xmalloc /home/runner/work/neovim/neovim/src/nvim/memory.c:108:15
=     #3 0x1208b6b in string_convert_ext /home/runner/work/neovim/neovim/src/nvim/mbyte.c:2547:14
=     #4 0x1200148 in string_convert /home/runner/work/neovim/neovim/src/nvim/mbyte.c:2519:10
=     #5 0x10a9c53 in nlua_iconv /home/runner/work/neovim/neovim/src/nvim/lua/stdlib.c:526:23
=     #6 0x1f2b8d7 in luaD_precall /home/runner/nvim-deps/build/src/lua/src/ldo.c:320:10
=     #7 0x1f3a1f3 in luaV_execute /home/runner/nvim-deps/build/src/lua/src/lvm.c:612:17
=     #8 0x1f2bb8a in luaD_call /home/runner/nvim-deps/build/src/lua/src/ldo.c:378:5
=     #9 0x1f28027 in f_call /home/runner/nvim-deps/build/src/lua/src/lapi.c:800:3
=     #10 0x1f2ab94 in luaD_rawrunprotected /home/runner/nvim-deps/build/src/lua/src/ldo.c:116:3
=     #11 0x1f2bf94 in luaD_pcall /home/runner/nvim-deps/build/src/lua/src/ldo.c:464:12
=     #12 0x1f280e9 in lua_pcall /home/runner/nvim-deps/build/src/lua/src/lapi.c:821:12
=     #13 0x1093729 in nlua_pcall /home/runner/work/neovim/neovim/src/nvim/lua/executor.c:130:16
=     #14 0x10959f3 in nlua_exec /home/runner/work/neovim/neovim/src/nvim/lua/executor.c:1322:7
=     #15 0x739f78 in nvim_exec_lua /home/runner/work/neovim/neovim/src/nvim/api/vim.c:459:10
=     #16 0x69bf71 in handle_nvim_exec_lua

runtime/doc/lua.txt

lewis6991 · 2022-08-24T11:44:52Z

Removed the opts field, instead just check for nil on the return value to know if conversion failed.

Identical to vim.fn.iconv except that there is no typval conversion and thus accepts **any** Lua string (UTF-16/32), whereas typval conversion may produce a Blob type which causes vim.fn.iconv to fail. Also does not rely on NUL byte for string length detection. Co-authored-by: Justin M. Keyes <[email protected]>

BREAKING CHANGES - Remove 'insertmode' 'remap' and 'terse' options - highlight: Rename attributes to match Vim (#19159) - highlight: Error on invalid names and allow '.' and '@' - terminal: Drop winpty, require Windows 10 #18253 - treesitter: Use @foo.bar style highlight groups - treesitter: Do not merge queries by default (#20105) FEATURES - runtime: Enable filetype.lua by default (#19216) - Add `undo!` - Add "prerelease" to version dict - Click support for 'statusline', 'winbar' #18650 - Add preview functionality to user commands - allow cmdheight=0 (EXPERIMENTAL) #16251 - Stdpath('run'), /tmp/nvim.user/ #18993 - Add 'mousescroll' option (#12355) - Allow :wincmd to accept a count (#19815) - Multibuffer preview support for inccommand - Man: Port to Lua (#19912) - api: Ui options relevant for remote TUI - api: Add `nvim_parse_cmd` And `nvim_cmd` - api: Support handling stdin stream in remote ui - api: Add `group_name` to `nvim_get_autocmds` - api: Enable nvim_exec_autocmds to pass arbitrary data (#18613) - api: Support pattern array for exec_autocmds - api: Add `unsilent` to command APIs - api: Add replace_keycodes to nvim_set_keymap (#19598) - api: Add support for :horizontal modifier - api: Add "move" to nvim_input_mouse - api/ui: "ui_watched" option for ui-side extmarks - build: Add_glob_target runs only on changed files #19070 - checkhealth: Check for slow shell #17829 - defaults: Session data in $XDG_STATE_HOME #15583 - defaults: Search selection by * and # in visual mode (#18538) - defaults: Nnoremap & :&&<CR> #19365 - defaults: enable mouse by default (set mouse=nvi) #19290 - diagnostic: Pass diagnostics as data to DiagnosticChanged autocmd (#20173) - docs: Gen_help_html.lua - edit: Insert an unsimplified key using CTRL-SHIFT-V - treesitter: Full support for spelling - filetype: Convert more patterns to filetype.lua - filetype: Remove side effects from vim.filetype.match (#18894) - filetype: Expand environment variables in filetype patterns (#20145) - fs: Add vim.fs functions: parents() dirname() basename() dir() find() normalize() - highlight: Implement CurSearch highlight - highlight: Support scoped @spam.eggs.baked_beans groups - input: Delay all simplifications - l10n: Turkish translations #19441 - l10n: Improve zh_CN translations (#19483) - lsp: Remove capabilities sanitization (#17814) - lsp: Show feedback on empty hover response (#18308) - lsp: Options to filter and auto-apply code actions (#18221) - lsp: Add vim.lsp.buf.format (#18193) - lsp: Add logging level "OFF" (#18379) - lsp: Add LspAttach and LspDetach autocommands - lsp: Add filter to vim.lsp.get_active_clients() - lsp: Option to reuse_win for jump actions (#18577) - lsp: Add a start function (#18631) - lsp: Send didChangeConfiguration after init (#18847) - lsp: Defaults: tagfunc, omnifunc, formatexpr (#19003, #19677) - lsp: Allow passing custom list handler to LSP functions that return lists (#19213) - lsp: Provide feedback if server can't compute rename result (#19546) - lsp: Add range option to code_action; deprecate range_code_action (#19551) - lsp: Disable exit_timeout by default (#19672) - lsp: Add tcp support - lua: Vim.deprecate() #18320 - lua: Vim.cmd() with kwargs acts like nvim_cmd() #18523 - lua: Allow some vim script functions to run in fast callbacks - lua: Measure require in --startuptime - lua: Allow vim.cmd to be indexed (#19238) - lua: Print source locations of lua callbacks (#19597) - lua: Add vim.iconv (#18286) - lua: Vim.ui_attach to get ui events from lua (EXPERIMENTAL) - man.vim: List command flags in "gO" outline #17558 - mappings: Do not replace existing mapping for simplified form - mappings: Allow special keys and modifiers in <Cmd> mapping - mapset: Support restoring "replace_keycodes" and "desc" - mapset: Support restoring Lua callback (#20024) - marks: Restore viewport on jump #15831 - nvim_create_user_command: Pass structured modifiers to commands - pum: Pretend 'mousemoveevent' is set when showing right-click menu - server: Set $NVIM, unset $NVIM_LISTEN_ADDRESS #11009 - server: Instance "name", store pipes in stdpath(state) - term: Add support for libvterm >= 0.2 - terminal: Implement <c-\><c-o> for terminal mode - terminal: Recognize underdouble and undercurl - terminfo: Bump built-in terminfo entries (#18570) - treesitter: Allow customizing language symbol name - treesitter: Add ability to retreive a tree/node given a range - treesitter: Upstream node methods from nvim-treesitter - treesitter: Include language in invalid query error (#14053) - treesitter: Bundle Lua parser and queries - treesitter: Add viml parser and queries - treesitter: Add injections - treesitter: Add vim.treesitter.start(), enable for Lua - treesitter: Bundle :help parser and queries - tui: Query terminal for CSI u support (#18181) - tui: Recognize keypad keys when using kitty keyboard protocol - tui: Try terminfo for [re]set_cursor_color OSC #19255 - tui: Allow grid and host to disagree on ambiguous-width chars (#19686) - tui: Recognize sidescroll events (#19992) - tui: Support 'mousemoveevent' - ui: Add `'winbar'` - ui: Clear message history explicitly with msg_history_clear event - ui: Make right-click menu work properly with ext_multigrid - ui: Allow to set the highlight namespace per window - ui: Use msg_grid based implementation for cmdheight=0 - ui-ext: Make 'mousemoveevent' a ui_option - eval: Make Lua Funcref work as method and in substitute() (#20217) - eval: Input() support any type for "cancelreturn" in a dict (#19357) BUG FIXES - Show autocmd output when F is in shortmess (#18251) - Has() should preserve v:shell_error #18280 - Suppress "is a directory" messages with shortmess 'F' (#18296) - Display global statusline correctly with ext_messages - Correct nlua_wait error message #18867 - Right-click in clickable statusline #19252 - Remote UI may get invalid 'pumblend' value #19379 - Assertion failure when requiring missing module in autocmd - api: Nvim_eval_statusline should validate input #18347 - api: Check error after getting win/buf handle (#19052) - api: Check for inclusive buffer line index out of bounds correctly (#19056) - api: Change default value of 'pattern' in nvim_exec_autocmds (#19115) - api: Do not switch win/buf if getting option in current win/buf (#19383) - api: Make nvim_set_hl(ns=0, ...) redraw screen properly - api: Nvim_set_hl bail out on invalid group name (#20021) - api: Notify dict watchers on nvim_set_var and vim.g setter - api/command: Fargs behavior when no arguments are passed (#19862) - autocmds: Separate command from desc (#18617) - buffer: Disable buffer-updates before removing from window #18933 - build: Missing definitions for sizeof macros #16393 - build: Only pass -municode if MINGW #19049 - build: Strip trailing newline from variable (#19084) - build: Don't disable byte precompilation on debug builds - build: Fails if git is missing #19366 - charclass: Make behavior with empty str match latest Vim (#19749) - checkhealth: Skip vim.health #18816 - ci: Remove 2000ms blocking wait in many plugin/lsp_spec.lua tests - cmd: Make :-tabmove work with modifiers (#18447) - cmdheight=0: Various issues part3 #19816 - cmdline: Fix passing -1 as char - cmdline: Trigger CmdlineEnter and ModeChanged earlier (#19474) - cmdline: Do not trigger completion at wrong time (#19920) - cmdline: Don't send invalid cursor with incsearch and cmdheight=0 - column: Move sign sentinel after inserting/deleting lines (#20400) - completion: Remove wrong FUNC_ATTR_NONNULL_ALL (#19627) - diagnostic: Use nvim_exec_autocmds to trigger DiagnosticChanged (#18188) - diagnostic: Check for negative column value (#18868) - diagnostic: Remove buf from cache on `BufWipeout` (#20099) - diagnostic: Populate data key in DiagnosticChanged autocmd in reset (#20207) - docs: Correct obsolete note about 'writedelay' in dev docs - docs: Remove internal function from docs - edit.c: Indentkeys double indent after "!" #12894 - eval: Check for v:lua when calling callback (#19855) - eval/f_getmatches: Return empty list for invalid win argument (#18893) - events: Make CursorHold behave as documented - ex_cmds: Correct flags for :const (#19387) - exceptions: Restore `did_throw` (#20000) - exmode: Do not throttle messages when pressing enter to print line - extmarks: Make virt_lines always start at 0 virtcol - filetype: Fix and improve filetype patterns - fillchars: Change fallback after setcellwidths() - float: Make `screen*()` functions respect floating windows - float: Fix glitch when making float window with border a split - float: Fix mouse drag position if float window turned to a split - folds: Fix fold regression with :move (#18685) - folds: Fix fold remains when :delete makes buffer empty (#19673) - ftdetect: Source plugins in autogroup (#18237) - gen_vimdoc.py: Handle missing luajit - getchar: Flush screen before doing a blocking wait - handlers: More specific error messages (#16772) - health: Correct shada file path #18603 - health: Handle non-existent log file #18610 - highlight: Use ctermbg/fg instead of bg/fg when use_rgb=false #18982 - highlight: Add missing 'nocombine' to nvim_get_hl apis (#19586) - highlight: Set the window namespace when redrawing statusline - hl: Set Normal hl group sg_attr value #18820 - hl: Return cterm fg/bg even if they match Normal #18981 - inccommand: Do not try to preview an ambiguous command (#18827) - inccommand: Avoid crash if callback changes inccommand option (#18830) - inccommand: Clear cmdpreview state if preview is not shown (#18923) - inccommand: Skip split window if not enough room #18937 - inccommand: Never preview if parsing command failed (#18944) - inccommand: Parse the command to check if it is previewable - inccommand: Deal with unsynced undo (#20041) - input: Allow Ctrl-C to interrupt a recursive mapping even if mapped (#18885) - input: Fix macro recording with ALT and special key (#18917) - input: Use correct grid when restoring cursor for <expr> mapping (#19047) - input: Do no reinterpret mouse keys with ALT modifiers - input: Use click number of last click for mouse drag (#20300) - inspect: Escape identifiers that are lua keywords (#19898) - keywordprg: Default to :help if set to empty string (#19983) - l10n: Improve zh_CN and zh_TW translations (#19969) - log: Back even again - logging: Skip recursion, fix crash #18764 - logging: Make logmsg() thread-safe again - logging: Try harder to resolve Nvim "name" #19016 - lsp: Unify progress message handling (#18040) - lsp: Fix unnecessary buffers being added on empty diagnostics (#18275) - lsp: Fix infinite loop in resolved_capabilities deprecation message (#18333) - lsp: Add missing bufnr argument (#18382) - lsp: Fix rename capability checks and multi client support (#18441) - lsp: Detach spawned LSP server processes (#18477) - lsp: Perform client side filtering of code actions (#18392) - lsp: Only send diagnostics from current buffer in code_action() (#18639) - lsp: Respect global syntax setting in open float preview (#15225) - lsp: Include cancellable in progress message table (#18809) - lsp: Adjust offset encoding in lsp.buf.rename() (#18829) - lsp: Set buflisted before switching to buffer (#18854) - lsp: Fix multi client handling in code action (#18869) - lsp: Small bugs in snippet-parser #18998 - lsp: Pcall nvim_del_augroup_by_name (#19302) - lsp: Abort pending changes after flush when debouncing (#19314) - lsp: Don't attach a client in lsp.start() if there is none (#19328) - lsp: Account for initializing servers in vim.lsp.start (#19329) - lsp: Move augroup define to if statement (#19406) - lsp: Set workspace.configuration capability (#19548) - lsp: Send didOpen if name changes on write (#19583) - lsp: Prevent unexpected position jumps (#19370) - lsp: Avoid ^M character in hover window on Windows (#19640) - lsp: Set end_col in formatexpr (#19676) - lsp: Handle multiple clients with incremental sync (#19658) - lsp: Fix some type annotations in lsp.rpc (#19714) - lsp: Avoid pipe leaks if lsp cmd isn't executable (#19717) - lsp: Handle nil client in onexit callback (#19722) - lsp: Fix nil value error in get_group (#19735) - lsp: Clean the diagnostic cache when buffer delete (#19449) - lsp: When buffer detach remove buffer from client attached buffers (#20081) - lsp: Schedule removal of client object (#20148) - lsp: Support `false` result in handlers (#20252) - lsp: Out of bounds error in lsp.util.apply_text_edits (#20137) - lsp: Use correct function name in deprecated message (#20308) - lsp: Create missing directory before creating file (#19835) - lua: Don't mutate opts parameter of vim.keymap.del (#18227) - lua: Stop pending highlight.on_yank timer, if any (#18824) - lua: Highlight.on_yank can close timer in twice #18976 - lua: Clear got_int when calling vim.on_key() callback (#18979) - lua: Don't leak memory on error - lua: Double entries in :lua completion #19410 - lua: Make it possible to cancel vim.wait() with Ctrl-C (#19217) - lua: Make ui_attach()/ui_detach() take effect immediately (#20037) - lua: Make vim.str_utfindex and vim.str_byteindex handle NUL bytes - lua: Free vim.ui_attach callback before lua close (#20205) - lua: Fix architecture-dependent behavior in usercmd "reg" (#20384) - mac: Use same $LANG fallback mechanism as Vim - mac: Add CoreServices to flake.nix #18358 - man.vim: Q in "$MANPAGER mode" does not quit #18443 - maparg: Remove double allocation (#20033) - mappings: Fix double-free when unmapping simplifiable Lua mapping - mapset: Remove existing abbreviation of same lhs (#20320) - mark: Set mark fnum from buffer (#19195) - mark: Mark without a view restores at topline #19224 - mark: Fix unexpected cursor movements (#19253) - mark: Give correct error message when mark is in another buffer (#19454) - menu: Make :menu still print header when there are no menus - messages: Add color when showing nvim_echo in :messages history - messages: Do not crash on cmdheight=0 and g< redisplay - messages: Validate msg_grid before silent! message with cmdheight=0 - mksession: Don't store floats in session #18635 - mouse: Click on global statusline with splits (#19390) - mouse: Fix using uninitialized memory with K_MOUSEMOVE (#19480) - mpack: Make sure a `bool` always is a `bool` - normal: Fix segfault with bracket command jumping to a mark - options: Properly free string options (#19510) - options: Mark `winhighlight` as list style (#19477) - packaging: Remove excess forward slash in Wix Patch (#18121) - paste: Ignore mappings in Cmdline mode (#18114) - path: Path_is_url returns false for "foo:/" #19797 - powershell: Filter ":!" commands with args #19268 - pum: Make right drag in anchor grid to select work in multigrid UI (#19382) - query: Fix unnatural order for inherits in treesitter queries (#20298) - redraw: Make sure :redraw! redraws command line - redraw: Handle switching to a tabpage with larger p_ch value - redraw: Avoid unnecessary redraws and glitches with floats+messages - redraw: Make redrawdebug=nodelta handle all the cases - rpc: Break nvim_error_event feedback loop between two nvim instances - runtime/genvimvim: Omit s[ubstitute] from vimCommand #18480 - screen: Restart win_update() if a decor provider changes signcols (#18768) - screen: Check for col instead of vcol when drawing fold (#19572) - session: Respect sessionoptions=terminal #19497 - shared: Avoid indexing unindexable values in vim.tbl_get() (#18337) - signs: Priority of extmark signs (#19718) - source: Make changing 'shellslash' change expand() result - source: Fix expand('<sfile>') no longer works for Lua - spell: Make setting 'encoding' clear word list - spell: Correct spell move behavior without "noplainbuffer" (#20386) - startup: Nvim with --clean should not load user rplugins - substitute: Subtract number of backslashes later - tabpage: Check if ROWS_AVAIL changed for resize (#19620) - terminal: Invalid pointer comparison #18453 - terminal: Do not trim whitespace that is actually in the terminal (#16423) - terminal: Scrollback delete lines immediately #18832 - terminal: Coverity USE_AFTER_FREE #18978 - terminal: Crash if TermClose deletes own buffer #19222 - terminal: Avoid reading over the end of cell.chars (#19580) - terminal: Skip aucmd_win when checking terminal size (#19668) - terminal: Adopt altscreen test for libvterm 0.2 changes - terminfo: Disable smglr for vtpcon and conemu (#18855) - termopen: Avoid ambiguity in URI when CWD is root dir (#16988) - tests: Fix some screen.lua warnings - tests: Fix some issues with ui/inccommand_spec.lua causing slowness - tests: Unreliable parser_spec #18911 - tests: Check for EOF on exit of nvim properly - tests: Missing clear() #18927 - tests: Remove misleading $TEST_PATH segment #19050 - tests: Remove irrelevant usage of display-=msgsep - tests: Use pending_c_parser when needed - tests: Indicate in test logs when nvim exit times out - tmpdir: Invalid tempname() if username has slashes #19323 - treesitter: Create new parser if language is not the same as cached parser (#18149) - treesitter: Bump match limit up - treesitter: Offset directive associates range with capture (#18276) - treesitter: Correct region for string parser (#18794) - treesitter: New iter if folded - treesitter: Don't error when node argument of predicate is nil (#19355) - treesitter: Free memory on removing parser (#19933) - treesitter: More efficient node:root() - treesitter: Make it get_captures_at_position - treesitter: Do not link @error by default - treesitter: Don't support legacy syntax in start() - treesitter: Use the right loading order for base queries (#20117) - treesitter: Prevent endless loop on self-inheritence - treesitter: Return full metadata for get_captures_at_position (#20203) - ts: Do not clobber spelloptions (#20095) - tui: Update modifyOtherKeys reporting (#18158) - tui: Disable extended keys before exiting alternate screen (#18318) - tui: Piping nodejs to nvim breaks input handling #18932 - tui: Resize at startup #17795 - tui: Add fixups for hterm family #19078 - tui: Handle padding requirements for visual bell (#20238) - ui: Require window-local value to show winbar on floating windows (#18773) - ui: Do not call showmode() when setting window height (#18969) - ui: Set redraw_cmdline when setting window height (#19630) - ui: Don't allow decor provider with ns_id==0 - ui: Ui compositor does not correctly free event callbacks - ui: Allow redrawing statusline when msgsep is used (#20337) - ui: Redraw end of buffer if last line is modified (#20354) - unittests: Coredump when running unit tests #18663 - usercmd: Also check for whitespace after escaped character (#19942) - version.c: Mark N/A vim patches #18587 - vim.ui.input: Accept nil or empty "opts" #19109 - window: Close floats first when closing buffer in other tab (#20284) - window: Fix equalization with cmdheight=0 (#20369) - windows: Stdpath("state") => "nvim-data" #18546 - windows: Exepath, stdpath return wrong slashes #19111 - winhl: Do not crash when unsetting winhl in just opened window - Make :undo! notify buffer update callbacks (#20344) - eval: Make Vim functions return inner window width and height (#19743) BUILD SYSTEM - Bump Doxyfile to minimum required version 1.9.0 #18118 - Bump msgpack to 4.0.0 - Enable EXITFREE on Debug builds (#17783) - Add formatting targets for c and lua files (#19488) - clang-format: Align with project style #18192 - clint: Remove all python2-isms with pyupgrade - clint: Remove "function size is too large" warning - clint: Remove rules for includes, whitespace, tabs #18611 - cmake: Simplify and speed up the uninstall target - cmake: Simplify def_cmd_target function - cmake: Use glob_wrapper instead of file(GLOB in main CMakeLists - cmake: Fix static `libintl` test on macOS - deps: Bump LuaJIT, Luv and libuv - deps: Support universal builds on macOS - deps: Bump tree-sitter to v0.20.7 (#20067) - deps: Bump tree-sitter parsers - deps: Bump required libvterm to v0.3 (#20222) - deps: Require libtermkey version 0.22 - deps: update neovim-qt, win32tools.zip PERFORMANCE - Only redraw for CurSearch when it is currently in use - highlight: Allocate permanent names in an arena for fun and cache locality - highlight: Use binary search to lookup RGB color names - map: Visit only one hash bucket instead of all, like an actual hash table - memory: Get rid of extraneous heap allocations - memory: implement arena memory allocation with a shared freelist - memory: Use an arena for RPC decoding and some API return values - messages: Don't call ui_flush() per message line in various places - treesitter: Use a reuse list for query cursors - ui: Reduce allocations when encoding and decoding "redraw" events - ui: Avoid ui_flush() work in headless mode REFACTOR - checkhealth: Rename to vim.health, move logic to Lua #18720 - Add pure attribute to pure functions - Replace char_u variables and functions with char - Enable -Wconversion warning for all Nvim source files - Add warnings for deprecated functions (#18662) - Change type of linenr_T from long to int32_t - Use nvim_get/set_option_value for vim.{b,w}o - Remove functions marked for deprecation in 0.8 (#19299) - Rename function prefix mb to the more accurate utf_cp (#19590) - Remove some unused includes - Change remaining sourcing_name/sourcing_lnum to exestack - Change FALSE/TRUE to false/true - api: Use a hashy hash for looking up api method and event names - api: Use a unpacker based on libmpack instead of msgpack-c - api: restructure api/vim.c and api/private/helpers.c code in separate files - api: Remove redundant fields of CmdParseInfo - aucmd: Call define_autocmd() directly for default autocmds - ci: Cleanup release.yml #19132 - cmd: Format do_one_cmd() - cmd: Hoist out some code into functions - cmd: Unify execute_cmd with do_one_cmd - decor: Use decor levels properly - drawline.c: Factor out utf8 multibyte check - eval: Use Hashy McHashFace instead of gperf - eval.c: Resolve all clint issues (#19774) - eval/funcs.c: Resolve all clint errors - events: Remove unnecessary fudging of updating_screen - ex_cd: Add an early return to fix clint warning - ex_docmd.c: Resolve most clint errors (#19775) - filetype: Allow vim.filetype.match to accept buf and filename (#19114) - highlight: Make hlattrs2dict always use pre-allocated dict - log: Simplify log_path_init #18898 - log: Use msg_schedule_semsg #18950 - lsp: Remove redundant client cleanup (#18744) - lsp: Make the use of local aliases more consistent - lsp: Use autocmd api (#19407) - lsp: Factor out read_loop function - lsp: Encapsulate rpc uv handle - lsp: Extract rpc client from rpc.start - lua: Replace hard-coded gsub with vim.pesc() (#18407) - lua: Reformat with stylua 0.14.0 (#19264) - lua: Git-blame-ignore stylua update PR (#19273) - lua: Replace vim.cmd use with API calls (#19283) - map: Simplify free_all_mem handling - map: Statically initialize maphash array - map: Simplify add_map params - normal: Convert function comments to doxygen format - object: Get rid of redundant FIXED_TEMP_ARRAY - ops: Doxygen docstrings #17743 - option: DRY get/set option value #19038 - plines: Use a struct for chartabsize state - provider: Use list comprehension #19027 - regexp_nfa.c: Match where Vim calls fopen() (#18778) - runtime: Convert dist#ft functions to lua (#18247) - runtime: Convert more dist#ft functions to lua (#18430) - runtime: Convert the remaining dist#ft functions to lua (#18623) - runtime: Port remaining patterns from filetype.vim to filetype.lua (#18814) - runtime: Refactor filetype.lua (#18813) - runtime: Port scripts.vim to lua (#18710) - setcellwidths: Use TV_LIST_ITEM_NEXT properly - signs: Handle non-sign attrs separately (#19784) - tests: Introduce testprg() - treesitter: Get_{nodes,captures}_at_{position,cursor} - typval: Change FC_CFUNC abstraction into FC_LUAREF - ui: Simplify stdin handling - uncrustify: Format all c code under /src/nvim/ - vim.opt: Remove del arg - vim.opt: Unify vim.bo/wo building - vim.opt: Optimize append/prepend/remove - Format runtime with stylua

BREAKING CHANGES - Remove 'insertmode' 'remap' and 'terse' options - highlight: Rename attributes to match Vim (neovim#19159) - highlight: Error on invalid names and allow '.' and '@' - terminal: Drop winpty, require Windows 10 neovim#18253 - treesitter: Use @foo.bar style highlight groups - treesitter: Do not merge queries by default (neovim#20105) FEATURES - runtime: Enable filetype.lua by default (neovim#19216) - Add `undo!` - Add "prerelease" to version dict - Click support for 'statusline', 'winbar' neovim#18650 - Add preview functionality to user commands - allow cmdheight=0 (EXPERIMENTAL) neovim#16251 - Stdpath('run'), /tmp/nvim.user/ neovim#18993 - Add 'mousescroll' option (neovim#12355) - Allow :wincmd to accept a count (neovim#19815) - Multibuffer preview support for inccommand - Man: Port to Lua (neovim#19912) - api: Ui options relevant for remote TUI - api: Add `nvim_parse_cmd` And `nvim_cmd` - api: Support handling stdin stream in remote ui - api: Add `group_name` to `nvim_get_autocmds` - api: Enable nvim_exec_autocmds to pass arbitrary data (neovim#18613) - api: Support pattern array for exec_autocmds - api: Add `unsilent` to command APIs - api: Add replace_keycodes to nvim_set_keymap (neovim#19598) - api: Add support for :horizontal modifier - api: Add "move" to nvim_input_mouse - api/ui: "ui_watched" option for ui-side extmarks - build: Add_glob_target runs only on changed files neovim#19070 - checkhealth: Check for slow shell neovim#17829 - defaults: Session data in $XDG_STATE_HOME neovim#15583 - defaults: Search selection by * and # in visual mode (neovim#18538) - defaults: Nnoremap & :&&<CR> neovim#19365 - defaults: enable mouse by default (set mouse=nvi) neovim#19290 - diagnostic: Pass diagnostics as data to DiagnosticChanged autocmd (neovim#20173) - docs: Gen_help_html.lua - edit: Insert an unsimplified key using CTRL-SHIFT-V - treesitter: Full support for spelling - filetype: Convert more patterns to filetype.lua - filetype: Remove side effects from vim.filetype.match (neovim#18894) - filetype: Expand environment variables in filetype patterns (neovim#20145) - fs: Add vim.fs functions: parents() dirname() basename() dir() find() normalize() - highlight: Implement CurSearch highlight - highlight: Support scoped @spam.eggs.baked_beans groups - input: Delay all simplifications - l10n: Turkish translations neovim#19441 - l10n: Improve zh_CN translations (neovim#19483) - lsp: Remove capabilities sanitization (neovim#17814) - lsp: Show feedback on empty hover response (neovim#18308) - lsp: Options to filter and auto-apply code actions (neovim#18221) - lsp: Add vim.lsp.buf.format (neovim#18193) - lsp: Add logging level "OFF" (neovim#18379) - lsp: Add LspAttach and LspDetach autocommands - lsp: Add filter to vim.lsp.get_active_clients() - lsp: Option to reuse_win for jump actions (neovim#18577) - lsp: Add a start function (neovim#18631) - lsp: Send didChangeConfiguration after init (neovim#18847) - lsp: Defaults: tagfunc, omnifunc, formatexpr (neovim#19003, neovim#19677) - lsp: Allow passing custom list handler to LSP functions that return lists (neovim#19213) - lsp: Provide feedback if server can't compute rename result (neovim#19546) - lsp: Add range option to code_action; deprecate range_code_action (neovim#19551) - lsp: Disable exit_timeout by default (neovim#19672) - lsp: Add tcp support - lua: Vim.deprecate() neovim#18320 - lua: Vim.cmd() with kwargs acts like nvim_cmd() neovim#18523 - lua: Allow some vim script functions to run in fast callbacks - lua: Measure require in --startuptime - lua: Allow vim.cmd to be indexed (neovim#19238) - lua: Print source locations of lua callbacks (neovim#19597) - lua: Add vim.iconv (neovim#18286) - lua: Vim.ui_attach to get ui events from lua (EXPERIMENTAL) - man.vim: List command flags in "gO" outline neovim#17558 - mappings: Do not replace existing mapping for simplified form - mappings: Allow special keys and modifiers in <Cmd> mapping - mapset: Support restoring "replace_keycodes" and "desc" - mapset: Support restoring Lua callback (neovim#20024) - marks: Restore viewport on jump neovim#15831 - nvim_create_user_command: Pass structured modifiers to commands - pum: Pretend 'mousemoveevent' is set when showing right-click menu - server: Set $NVIM, unset $NVIM_LISTEN_ADDRESS neovim#11009 - server: Instance "name", store pipes in stdpath(state) - term: Add support for libvterm >= 0.2 - terminal: Implement <c-\><c-o> for terminal mode - terminal: Recognize underdouble and undercurl - terminfo: Bump built-in terminfo entries (neovim#18570) - treesitter: Allow customizing language symbol name - treesitter: Add ability to retreive a tree/node given a range - treesitter: Upstream node methods from nvim-treesitter - treesitter: Include language in invalid query error (neovim#14053) - treesitter: Bundle Lua parser and queries - treesitter: Add viml parser and queries - treesitter: Add injections - treesitter: Add vim.treesitter.start(), enable for Lua - treesitter: Bundle :help parser and queries - tui: Query terminal for CSI u support (neovim#18181) - tui: Recognize keypad keys when using kitty keyboard protocol - tui: Try terminfo for [re]set_cursor_color OSC neovim#19255 - tui: Allow grid and host to disagree on ambiguous-width chars (neovim#19686) - tui: Recognize sidescroll events (neovim#19992) - tui: Support 'mousemoveevent' - ui: Add `'winbar'` - ui: Clear message history explicitly with msg_history_clear event - ui: Make right-click menu work properly with ext_multigrid - ui: Allow to set the highlight namespace per window - ui: Use msg_grid based implementation for cmdheight=0 - ui-ext: Make 'mousemoveevent' a ui_option - eval: Make Lua Funcref work as method and in substitute() (neovim#20217) - eval: Input() support any type for "cancelreturn" in a dict (neovim#19357) BUG FIXES - Show autocmd output when F is in shortmess (neovim#18251) - Has() should preserve v:shell_error neovim#18280 - Suppress "is a directory" messages with shortmess 'F' (neovim#18296) - Display global statusline correctly with ext_messages - Correct nlua_wait error message neovim#18867 - Right-click in clickable statusline neovim#19252 - Remote UI may get invalid 'pumblend' value neovim#19379 - Assertion failure when requiring missing module in autocmd - api: Nvim_eval_statusline should validate input neovim#18347 - api: Check error after getting win/buf handle (neovim#19052) - api: Check for inclusive buffer line index out of bounds correctly (neovim#19056) - api: Change default value of 'pattern' in nvim_exec_autocmds (neovim#19115) - api: Do not switch win/buf if getting option in current win/buf (neovim#19383) - api: Make nvim_set_hl(ns=0, ...) redraw screen properly - api: Nvim_set_hl bail out on invalid group name (neovim#20021) - api: Notify dict watchers on nvim_set_var and vim.g setter - api/command: Fargs behavior when no arguments are passed (neovim#19862) - autocmds: Separate command from desc (neovim#18617) - buffer: Disable buffer-updates before removing from window neovim#18933 - build: Missing definitions for sizeof macros neovim#16393 - build: Only pass -municode if MINGW neovim#19049 - build: Strip trailing newline from variable (neovim#19084) - build: Don't disable byte precompilation on debug builds - build: Fails if git is missing neovim#19366 - charclass: Make behavior with empty str match latest Vim (neovim#19749) - checkhealth: Skip vim.health neovim#18816 - ci: Remove 2000ms blocking wait in many plugin/lsp_spec.lua tests - cmd: Make :-tabmove work with modifiers (neovim#18447) - cmdheight=0: Various issues part3 neovim#19816 - cmdline: Fix passing -1 as char - cmdline: Trigger CmdlineEnter and ModeChanged earlier (neovim#19474) - cmdline: Do not trigger completion at wrong time (neovim#19920) - cmdline: Don't send invalid cursor with incsearch and cmdheight=0 - column: Move sign sentinel after inserting/deleting lines (neovim#20400) - completion: Remove wrong FUNC_ATTR_NONNULL_ALL (neovim#19627) - diagnostic: Use nvim_exec_autocmds to trigger DiagnosticChanged (neovim#18188) - diagnostic: Check for negative column value (neovim#18868) - diagnostic: Remove buf from cache on `BufWipeout` (neovim#20099) - diagnostic: Populate data key in DiagnosticChanged autocmd in reset (neovim#20207) - docs: Correct obsolete note about 'writedelay' in dev docs - docs: Remove internal function from docs - edit.c: Indentkeys double indent after "!" neovim#12894 - eval: Check for v:lua when calling callback (neovim#19855) - eval/f_getmatches: Return empty list for invalid win argument (neovim#18893) - events: Make CursorHold behave as documented - ex_cmds: Correct flags for :const (neovim#19387) - exceptions: Restore `did_throw` (neovim#20000) - exmode: Do not throttle messages when pressing enter to print line - extmarks: Make virt_lines always start at 0 virtcol - filetype: Fix and improve filetype patterns - fillchars: Change fallback after setcellwidths() - float: Make `screen*()` functions respect floating windows - float: Fix glitch when making float window with border a split - float: Fix mouse drag position if float window turned to a split - folds: Fix fold regression with :move (neovim#18685) - folds: Fix fold remains when :delete makes buffer empty (neovim#19673) - ftdetect: Source plugins in autogroup (neovim#18237) - gen_vimdoc.py: Handle missing luajit - getchar: Flush screen before doing a blocking wait - handlers: More specific error messages (neovim#16772) - health: Correct shada file path neovim#18603 - health: Handle non-existent log file neovim#18610 - highlight: Use ctermbg/fg instead of bg/fg when use_rgb=false neovim#18982 - highlight: Add missing 'nocombine' to nvim_get_hl apis (neovim#19586) - highlight: Set the window namespace when redrawing statusline - hl: Set Normal hl group sg_attr value neovim#18820 - hl: Return cterm fg/bg even if they match Normal neovim#18981 - inccommand: Do not try to preview an ambiguous command (neovim#18827) - inccommand: Avoid crash if callback changes inccommand option (neovim#18830) - inccommand: Clear cmdpreview state if preview is not shown (neovim#18923) - inccommand: Skip split window if not enough room neovim#18937 - inccommand: Never preview if parsing command failed (neovim#18944) - inccommand: Parse the command to check if it is previewable - inccommand: Deal with unsynced undo (neovim#20041) - input: Allow Ctrl-C to interrupt a recursive mapping even if mapped (neovim#18885) - input: Fix macro recording with ALT and special key (neovim#18917) - input: Use correct grid when restoring cursor for <expr> mapping (neovim#19047) - input: Do no reinterpret mouse keys with ALT modifiers - input: Use click number of last click for mouse drag (neovim#20300) - inspect: Escape identifiers that are lua keywords (neovim#19898) - keywordprg: Default to :help if set to empty string (neovim#19983) - l10n: Improve zh_CN and zh_TW translations (neovim#19969) - log: Back even again - logging: Skip recursion, fix crash neovim#18764 - logging: Make logmsg() thread-safe again - logging: Try harder to resolve Nvim "name" neovim#19016 - lsp: Unify progress message handling (neovim#18040) - lsp: Fix unnecessary buffers being added on empty diagnostics (neovim#18275) - lsp: Fix infinite loop in resolved_capabilities deprecation message (neovim#18333) - lsp: Add missing bufnr argument (neovim#18382) - lsp: Fix rename capability checks and multi client support (neovim#18441) - lsp: Detach spawned LSP server processes (neovim#18477) - lsp: Perform client side filtering of code actions (neovim#18392) - lsp: Only send diagnostics from current buffer in code_action() (neovim#18639) - lsp: Respect global syntax setting in open float preview (neovim#15225) - lsp: Include cancellable in progress message table (neovim#18809) - lsp: Adjust offset encoding in lsp.buf.rename() (neovim#18829) - lsp: Set buflisted before switching to buffer (neovim#18854) - lsp: Fix multi client handling in code action (neovim#18869) - lsp: Small bugs in snippet-parser neovim#18998 - lsp: Pcall nvim_del_augroup_by_name (neovim#19302) - lsp: Abort pending changes after flush when debouncing (neovim#19314) - lsp: Don't attach a client in lsp.start() if there is none (neovim#19328) - lsp: Account for initializing servers in vim.lsp.start (neovim#19329) - lsp: Move augroup define to if statement (neovim#19406) - lsp: Set workspace.configuration capability (neovim#19548) - lsp: Send didOpen if name changes on write (neovim#19583) - lsp: Prevent unexpected position jumps (neovim#19370) - lsp: Avoid ^M character in hover window on Windows (neovim#19640) - lsp: Set end_col in formatexpr (neovim#19676) - lsp: Handle multiple clients with incremental sync (neovim#19658) - lsp: Fix some type annotations in lsp.rpc (neovim#19714) - lsp: Avoid pipe leaks if lsp cmd isn't executable (neovim#19717) - lsp: Handle nil client in onexit callback (neovim#19722) - lsp: Fix nil value error in get_group (neovim#19735) - lsp: Clean the diagnostic cache when buffer delete (neovim#19449) - lsp: When buffer detach remove buffer from client attached buffers (neovim#20081) - lsp: Schedule removal of client object (neovim#20148) - lsp: Support `false` result in handlers (neovim#20252) - lsp: Out of bounds error in lsp.util.apply_text_edits (neovim#20137) - lsp: Use correct function name in deprecated message (neovim#20308) - lsp: Create missing directory before creating file (neovim#19835) - lua: Don't mutate opts parameter of vim.keymap.del (neovim#18227) - lua: Stop pending highlight.on_yank timer, if any (neovim#18824) - lua: Highlight.on_yank can close timer in twice neovim#18976 - lua: Clear got_int when calling vim.on_key() callback (neovim#18979) - lua: Don't leak memory on error - lua: Double entries in :lua completion neovim#19410 - lua: Make it possible to cancel vim.wait() with Ctrl-C (neovim#19217) - lua: Make ui_attach()/ui_detach() take effect immediately (neovim#20037) - lua: Make vim.str_utfindex and vim.str_byteindex handle NUL bytes - lua: Free vim.ui_attach callback before lua close (neovim#20205) - lua: Fix architecture-dependent behavior in usercmd "reg" (neovim#20384) - mac: Use same $LANG fallback mechanism as Vim - mac: Add CoreServices to flake.nix neovim#18358 - man.vim: Q in "$MANPAGER mode" does not quit neovim#18443 - maparg: Remove double allocation (neovim#20033) - mappings: Fix double-free when unmapping simplifiable Lua mapping - mapset: Remove existing abbreviation of same lhs (neovim#20320) - mark: Set mark fnum from buffer (neovim#19195) - mark: Mark without a view restores at topline neovim#19224 - mark: Fix unexpected cursor movements (neovim#19253) - mark: Give correct error message when mark is in another buffer (neovim#19454) - menu: Make :menu still print header when there are no menus - messages: Add color when showing nvim_echo in :messages history - messages: Do not crash on cmdheight=0 and g< redisplay - messages: Validate msg_grid before silent! message with cmdheight=0 - mksession: Don't store floats in session neovim#18635 - mouse: Click on global statusline with splits (neovim#19390) - mouse: Fix using uninitialized memory with K_MOUSEMOVE (neovim#19480) - mpack: Make sure a `bool` always is a `bool` - normal: Fix segfault with bracket command jumping to a mark - options: Properly free string options (neovim#19510) - options: Mark `winhighlight` as list style (neovim#19477) - packaging: Remove excess forward slash in Wix Patch (neovim#18121) - paste: Ignore mappings in Cmdline mode (neovim#18114) - path: Path_is_url returns false for "foo:/" neovim#19797 - powershell: Filter ":!" commands with args neovim#19268 - pum: Make right drag in anchor grid to select work in multigrid UI (neovim#19382) - query: Fix unnatural order for inherits in treesitter queries (neovim#20298) - redraw: Make sure :redraw! redraws command line - redraw: Handle switching to a tabpage with larger p_ch value - redraw: Avoid unnecessary redraws and glitches with floats+messages - redraw: Make redrawdebug=nodelta handle all the cases - rpc: Break nvim_error_event feedback loop between two nvim instances - runtime/genvimvim: Omit s[ubstitute] from vimCommand neovim#18480 - screen: Restart win_update() if a decor provider changes signcols (neovim#18768) - screen: Check for col instead of vcol when drawing fold (neovim#19572) - session: Respect sessionoptions=terminal neovim#19497 - shared: Avoid indexing unindexable values in vim.tbl_get() (neovim#18337) - signs: Priority of extmark signs (neovim#19718) - source: Make changing 'shellslash' change expand() result - source: Fix expand('<sfile>') no longer works for Lua - spell: Make setting 'encoding' clear word list - spell: Correct spell move behavior without "noplainbuffer" (neovim#20386) - startup: Nvim with --clean should not load user rplugins - substitute: Subtract number of backslashes later - tabpage: Check if ROWS_AVAIL changed for resize (neovim#19620) - terminal: Invalid pointer comparison neovim#18453 - terminal: Do not trim whitespace that is actually in the terminal (neovim#16423) - terminal: Scrollback delete lines immediately neovim#18832 - terminal: Coverity USE_AFTER_FREE neovim#18978 - terminal: Crash if TermClose deletes own buffer neovim#19222 - terminal: Avoid reading over the end of cell.chars (neovim#19580) - terminal: Skip aucmd_win when checking terminal size (neovim#19668) - terminal: Adopt altscreen test for libvterm 0.2 changes - terminfo: Disable smglr for vtpcon and conemu (neovim#18855) - termopen: Avoid ambiguity in URI when CWD is root dir (neovim#16988) - tests: Fix some screen.lua warnings - tests: Fix some issues with ui/inccommand_spec.lua causing slowness - tests: Unreliable parser_spec neovim#18911 - tests: Check for EOF on exit of nvim properly - tests: Missing clear() neovim#18927 - tests: Remove misleading $TEST_PATH segment neovim#19050 - tests: Remove irrelevant usage of display-=msgsep - tests: Use pending_c_parser when needed - tests: Indicate in test logs when nvim exit times out - tmpdir: Invalid tempname() if username has slashes neovim#19323 - treesitter: Create new parser if language is not the same as cached parser (neovim#18149) - treesitter: Bump match limit up - treesitter: Offset directive associates range with capture (neovim#18276) - treesitter: Correct region for string parser (neovim#18794) - treesitter: New iter if folded - treesitter: Don't error when node argument of predicate is nil (neovim#19355) - treesitter: Free memory on removing parser (neovim#19933) - treesitter: More efficient node:root() - treesitter: Make it get_captures_at_position - treesitter: Do not link @error by default - treesitter: Don't support legacy syntax in start() - treesitter: Use the right loading order for base queries (neovim#20117) - treesitter: Prevent endless loop on self-inheritence - treesitter: Return full metadata for get_captures_at_position (neovim#20203) - ts: Do not clobber spelloptions (neovim#20095) - tui: Update modifyOtherKeys reporting (neovim#18158) - tui: Disable extended keys before exiting alternate screen (neovim#18318) - tui: Piping nodejs to nvim breaks input handling neovim#18932 - tui: Resize at startup neovim#17795 - tui: Add fixups for hterm family neovim#19078 - tui: Handle padding requirements for visual bell (neovim#20238) - ui: Require window-local value to show winbar on floating windows (neovim#18773) - ui: Do not call showmode() when setting window height (neovim#18969) - ui: Set redraw_cmdline when setting window height (neovim#19630) - ui: Don't allow decor provider with ns_id==0 - ui: Ui compositor does not correctly free event callbacks - ui: Allow redrawing statusline when msgsep is used (neovim#20337) - ui: Redraw end of buffer if last line is modified (neovim#20354) - unittests: Coredump when running unit tests neovim#18663 - usercmd: Also check for whitespace after escaped character (neovim#19942) - version.c: Mark N/A vim patches neovim#18587 - vim.ui.input: Accept nil or empty "opts" neovim#19109 - window: Close floats first when closing buffer in other tab (neovim#20284) - window: Fix equalization with cmdheight=0 (neovim#20369) - windows: Stdpath("state") => "nvim-data" neovim#18546 - windows: Exepath, stdpath return wrong slashes neovim#19111 - winhl: Do not crash when unsetting winhl in just opened window - Make :undo! notify buffer update callbacks (neovim#20344) - eval: Make Vim functions return inner window width and height (neovim#19743) BUILD SYSTEM - Bump Doxyfile to minimum required version 1.9.0 neovim#18118 - Bump msgpack to 4.0.0 - Enable EXITFREE on Debug builds (neovim#17783) - Add formatting targets for c and lua files (neovim#19488) - clang-format: Align with project style neovim#18192 - clint: Remove all python2-isms with pyupgrade - clint: Remove "function size is too large" warning - clint: Remove rules for includes, whitespace, tabs neovim#18611 - cmake: Simplify and speed up the uninstall target - cmake: Simplify def_cmd_target function - cmake: Use glob_wrapper instead of file(GLOB in main CMakeLists - cmake: Fix static `libintl` test on macOS - deps: Bump LuaJIT, Luv and libuv - deps: Support universal builds on macOS - deps: Bump tree-sitter to v0.20.7 (neovim#20067) - deps: Bump tree-sitter parsers - deps: Bump required libvterm to v0.3 (neovim#20222) - deps: Require libtermkey version 0.22 - deps: update neovim-qt, win32tools.zip PERFORMANCE - Only redraw for CurSearch when it is currently in use - highlight: Allocate permanent names in an arena for fun and cache locality - highlight: Use binary search to lookup RGB color names - map: Visit only one hash bucket instead of all, like an actual hash table - memory: Get rid of extraneous heap allocations - memory: implement arena memory allocation with a shared freelist - memory: Use an arena for RPC decoding and some API return values - messages: Don't call ui_flush() per message line in various places - treesitter: Use a reuse list for query cursors - ui: Reduce allocations when encoding and decoding "redraw" events - ui: Avoid ui_flush() work in headless mode REFACTOR - checkhealth: Rename to vim.health, move logic to Lua neovim#18720 - Add pure attribute to pure functions - Replace char_u variables and functions with char - Enable -Wconversion warning for all Nvim source files - Add warnings for deprecated functions (neovim#18662) - Change type of linenr_T from long to int32_t - Use nvim_get/set_option_value for vim.{b,w}o - Remove functions marked for deprecation in 0.8 (neovim#19299) - Rename function prefix mb to the more accurate utf_cp (neovim#19590) - Remove some unused includes - Change remaining sourcing_name/sourcing_lnum to exestack - Change FALSE/TRUE to false/true - api: Use a hashy hash for looking up api method and event names - api: Use a unpacker based on libmpack instead of msgpack-c - api: restructure api/vim.c and api/private/helpers.c code in separate files - api: Remove redundant fields of CmdParseInfo - aucmd: Call define_autocmd() directly for default autocmds - ci: Cleanup release.yml neovim#19132 - cmd: Format do_one_cmd() - cmd: Hoist out some code into functions - cmd: Unify execute_cmd with do_one_cmd - decor: Use decor levels properly - drawline.c: Factor out utf8 multibyte check - eval: Use Hashy McHashFace instead of gperf - eval.c: Resolve all clint issues (neovim#19774) - eval/funcs.c: Resolve all clint errors - events: Remove unnecessary fudging of updating_screen - ex_cd: Add an early return to fix clint warning - ex_docmd.c: Resolve most clint errors (neovim#19775) - filetype: Allow vim.filetype.match to accept buf and filename (neovim#19114) - highlight: Make hlattrs2dict always use pre-allocated dict - log: Simplify log_path_init neovim#18898 - log: Use msg_schedule_semsg neovim#18950 - lsp: Remove redundant client cleanup (neovim#18744) - lsp: Make the use of local aliases more consistent - lsp: Use autocmd api (neovim#19407) - lsp: Factor out read_loop function - lsp: Encapsulate rpc uv handle - lsp: Extract rpc client from rpc.start - lua: Replace hard-coded gsub with vim.pesc() (neovim#18407) - lua: Reformat with stylua 0.14.0 (neovim#19264) - lua: Git-blame-ignore stylua update PR (neovim#19273) - lua: Replace vim.cmd use with API calls (neovim#19283) - map: Simplify free_all_mem handling - map: Statically initialize maphash array - map: Simplify add_map params - normal: Convert function comments to doxygen format - object: Get rid of redundant FIXED_TEMP_ARRAY - ops: Doxygen docstrings neovim#17743 - option: DRY get/set option value neovim#19038 - plines: Use a struct for chartabsize state - provider: Use list comprehension neovim#19027 - regexp_nfa.c: Match where Vim calls fopen() (neovim#18778) - runtime: Convert dist#ft functions to lua (neovim#18247) - runtime: Convert more dist#ft functions to lua (neovim#18430) - runtime: Convert the remaining dist#ft functions to lua (neovim#18623) - runtime: Port remaining patterns from filetype.vim to filetype.lua (neovim#18814) - runtime: Refactor filetype.lua (neovim#18813) - runtime: Port scripts.vim to lua (neovim#18710) - setcellwidths: Use TV_LIST_ITEM_NEXT properly - signs: Handle non-sign attrs separately (neovim#19784) - tests: Introduce testprg() - treesitter: Get_{nodes,captures}_at_{position,cursor} - typval: Change FC_CFUNC abstraction into FC_LUAREF - ui: Simplify stdin handling - uncrustify: Format all c code under /src/nvim/ - vim.opt: Remove del arg - vim.opt: Unify vim.bo/wo building - vim.opt: Optimize append/prepend/remove - Format runtime with stylua

Co-authored-by: Justin M. Keyes <[email protected]>

github-actions bot added the lua stdlib label Apr 27, 2022

lewis6991 force-pushed the iconv branch 3 times, most recently from b993fee to 96c4c85 Compare April 27, 2022 14:08

lewis6991 requested a review from bfredl April 27, 2022 14:10

lewis6991 mentioned this pull request Apr 27, 2022

vim.fn.iconv cannot handle Blob types lewis6991/gitsigns.nvim#549

Closed

lewis6991 force-pushed the iconv branch 2 times, most recently from a69520a to 3ceba25 Compare April 27, 2022 14:27

justinmk requested changes Apr 27, 2022

View reviewed changes

bfredl reviewed Apr 27, 2022

View reviewed changes

runtime/doc/lua.txt Outdated Show resolved Hide resolved

lewis6991 force-pushed the iconv branch from c3f0f43 to d65d3bf Compare April 27, 2022 22:15

This was referenced May 1, 2022

DNM(experiment): sized typval strings #18339

Closed

Lua stdlib feature parity against Vimscript #18393

Open

justinmk approved these changes May 4, 2022

View reviewed changes

erw7 reviewed May 5, 2022

View reviewed changes

This comment was marked as resolved.

Sign in to view

lewis6991 force-pushed the iconv branch from d65d3bf to b3dc680 Compare May 18, 2022 10:51

lewis6991 force-pushed the iconv branch 2 times, most recently from 558fc92 to 07e4d97 Compare May 18, 2022 11:06

lewis6991 force-pushed the iconv branch from 07e4d97 to 72a09cc Compare May 18, 2022 13:13

lewis6991 force-pushed the iconv branch from 72a09cc to 40f7267 Compare June 14, 2022 10:47

justinmk reviewed Jul 5, 2022

View reviewed changes

runtime/doc/lua.txt Outdated Show resolved Hide resolved

lewis6991 force-pushed the iconv branch 2 times, most recently from fc9d476 to 992d3bb Compare August 24, 2022 11:21

lewis6991 force-pushed the iconv branch from 992d3bb to a86c79b Compare August 24, 2022 11:48

lewis6991 merged commit b1eaa2b into neovim:master Aug 24, 2022

lewis6991 deleted the iconv branch August 25, 2022 17:11

smjonas pushed a commit to smjonas/neovim that referenced this pull request Dec 31, 2022

feat(lua): add vim.iconv (neovim#18286)

e7cd174

Co-authored-by: Justin M. Keyes <[email protected]>

Uh oh!

feat(lua): add vim.iconv #18286

feat(lua): add vim.iconv #18286

Uh oh!

Conversation

lewis6991 commented Apr 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Demo

Uh oh!

justinmk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lewis6991 commented Apr 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

justinmk commented Apr 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lewis6991 commented Apr 30, 2022

Uh oh!

lewis6991 commented Apr 30, 2022

Uh oh!

justinmk commented Apr 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lewis6991 commented May 1, 2022

Uh oh!

lewis6991 commented May 3, 2022

Uh oh!

justinmk commented May 4, 2022

Uh oh!

justinmk commented May 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clason commented May 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

erw7 May 5, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lewis6991 May 17, 2022

Choose a reason for hiding this comment

Uh oh!

erw7 May 21, 2022

Choose a reason for hiding this comment

Uh oh!

lewis6991 commented May 17, 2022

Uh oh!

lewis6991 commented May 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

lewis6991 commented May 18, 2022

Uh oh!

lewis6991 commented May 18, 2022

Uh oh!

erw7 commented May 21, 2022

Uh oh!

lewis6991 commented Jun 14, 2022

Uh oh!

justinmk commented Jul 5, 2022

Uh oh!

Uh oh!

lewis6991 commented Aug 24, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

lewis6991 commented Apr 27, 2022 •

edited

Loading

lewis6991 commented Apr 27, 2022 •

edited

Loading

justinmk commented Apr 30, 2022 •

edited

Loading

justinmk commented Apr 30, 2022 •

edited

Loading

justinmk commented May 4, 2022 •

edited

Loading

clason commented May 4, 2022 •

edited

Loading

erw7 May 5, 2022 •

edited

Loading

lewis6991 commented May 17, 2022 •

edited

Loading