|
|
Subscribe / Log in / New account

Rust in the 6.2 kernel

By Jonathan Corbet
November 17, 2022
The merge window for the 6.1 release brought in basic support for writing kernel code in Rust — with an emphasis on "basic". It is possible to create a "hello world" module for 6.1, but not much can be done beyond that. There is, however, a lot more Rust code for the kernel out there; it's just waiting for its turn to be reviewed and merged into the mainline. Miguel Ojeda has now posted the next round of Rust patches, adding to the support infrastructure in the kernel.

This 28-part patch series is focused on low-level support code, still without much in the way of abstractions for dealing with the rest of the kernel. There will be no shiny new drivers built on this base alone. But it does show another step toward the creation of a workable environment for the development of code in the Linux kernel.

As an example of how stripped-down the initial Rust support is, consider that the kernel has eight different logging levels, from "debug" through "emergency". There is a macro defined for each level to make printing simple; screaming about an imminent crash can be done with pr_emerg(), for example. The Rust code in 6.1 defines equivalent macros, but only two of them: pr_info!() and pr_emerg!(); the macros for the other log levels were left out. The first order of business for 6.2 appears to be to fill in the rest of the set, from pr_debug!() at one end through pr_alert!() at the other. There is also pr_cont!() for messages that are pieced together from multiple calls. This sample kernel module shows all of the print macros in action.

A rather more complex macro added in this series is #[vtable]. The kernel makes extensive use of structures full of pointers to functions; these structures are at the core of the kernel's object model. A classic example is struct file_operations, which is used to provide implementations of the many things that can be done with an open file. The functions found therein vary from relatively obvious operations like read() and write() through to more esoteric functionality like setlease() or remap_file_range(). Anything in the kernel that can be represented as an open file provides one of these structures to implement the operations on that file.

Operations structures like file_operations thus look a lot like Rust traits, and they can indeed be modeled as traits in Rust code. But the kernel allows any given implementation to leave out any functions that are not relevant; a remap_file_range() operation will make no sense in most device drivers, for example. In the kernel's C code, missing operations are represented by a null pointer; code that calls those operations will detect the null pointer and execute a default action instead. Null pointers, though, are the sort of thing that the Rust world goes out of its way to avoid, so representing an operations structure in Rust requires some extra work.

The #[vtable] macro is intended to perform the necessary impedance matching between C operations structures and Rust traits. Both the declaration of a trait and of any implementations will use this macro, so a trait definition will look like:

    #[vtable]
    pub trait Operations {
        /// Corresponds to the `open` function pointer in `struct file_operations`.
    	fn open(context: &Self::OpenData, file: &File) -> Result<Self::Data>;
    // ...
    }

While an implementation for a specific device looks like:

    #[vtable]
    impl kernel::file::Operations for some_driver {
        fn open(_data: &(), _file: &File) -> Result {
            Ok(())
        }
	// ...
    }

If this implementation is to be passed into the rest of the kernel, it must be turned into the proper C structure. Rust can create the structure, but it is hard-put to detect which operations have been implemented and which should, instead, be represented by a null pointer. The #[vtable] macro helps by generating a special constant member for each defined function; in the above example, the some_driver type would have a constant HAS_OPEN member set to true. The code that generates the C operations structure can query those constants (at compile time) and insert null pointers for missing operations; the details of how that works can be seen in this patch.

The submission for 6.2 adds #[vtable] but does not include any uses of it. The curious can see it in use by looking at this large patch posted in August; searching for #[vtable] and HAS_ will turn up the places where this infrastructure is used.

Yet another new macro is declare_err!(), which can be used to declare the various error-code constants like EPERM. The 6.2 kernel will likely include a full set of error codes declared with this macro rather than the minimal set included in 6.1. There is also a mechanism to translate many internal Rust error into Linux error codes.

The Rust Vec type implements an array that will grow as needed to hold whatever is put into it. Growing, of course, involves memory allocation, which can fail in the kernel. In 6.2, Vec as implemented in the kernel will likely have two methods called try_with_capacity() and try_with_capacity_in(). They act like the standard with_capacity() and with_capacity_in() Vec methods in that they preallocate memory for data to be stored later, but with the difference that they can return a failure code. The try_ variants will allow kernel code to attempt to allocate Vecs of the needed size and do the right thing if the allocation fails, rather than just calling panic() like the standard versions do.

One of the more confusing aspects of Rust for a neophyte like your editor is the existence of two string types: str and String; the former represents a borrowed reference to a string stored elsewhere, while the latter actually owns the string. The kernel's Rust support will define two variants of those, called CStr and CString, which serve the same function for C strings. Specifically, they deal with a string that is represented as an array of bytes and terminated with a NUL character. Rust code that passes strings into the rest of the kernel will need to use these types.

The series ends with a grab-bag of components that will be useful for future additions. The dbg!() macro makes certain types of debugging easier. There is code for compile-time assertions and to force build errors. The Either type can hold an object that can be either one of two distinct types. Finally, the Opaque type is for structures used by the kernel that are never accessed by Rust code. Using this type can improve performance by removing the need to zero-initialize the memory holding it before calling the initialization function.

As can be seen, these patches are slowly building the in-kernel Rust code up so that real functionality can be implemented in Rust, but this process has some ground to cover yet. It's not clear whether more Rust code will be proposed for 6.2, or whether this is the full set. The pace of change may seem slow to developers who would like to start doing real work in Rust, but it does have the advantage of moving in steps that can be understood — and reviewed — by the kernel community. The Rust-for-Linux work has been underway for a few years already; getting up to full functionality may well take a while longer yet.

Index entries for this article
KernelDevelopment tools/Rust


to post comments

Rust in the 6.2 kernel

Posted Nov 17, 2022 16:46 UTC (Thu) by atnot (subscriber, #124910) [Link] (16 responses)

> One of the more confusing aspects of Rust for a neophyte like your editor is the existence of two string types: str and String; the former represents a borrowed reference to a string stored elsewhere, while the latter actually owns the string.

On an interesting historical note, I recall there were a number of people who wanted to use Box<&str> as an owned string type instead, which would have been both easier to understand and more efficient given that the vast majority of strings are write-once anyway, making the capacity field unnecessary. The current growable string type would still have be kept under it's old StrBuf name. However as I understand this wasn't really possible in the language back then, so a new type was the direction they had to go, somewhat unfortunately I think.

Rust in the 6.2 kernel

Posted Nov 17, 2022 16:49 UTC (Thu) by atnot (subscriber, #124910) [Link]

Sorry, Box<str> of course, not Box<&str>

Rust in the 6.2 kernel

Posted Nov 17, 2022 17:03 UTC (Thu) by rillian (subscriber, #11344) [Link] (1 responses)

FWIW re our editor's difficulty, the C++17 standard library has a similar construction with std::string_view vs. std::string. Rust "simplifies" things a bit by making the string_view the same thing as a const std::string& and keeping track of the relative lifetimes of the two. In C it's all the same thing, of course.

Rust in the 6.2 kernel

Posted Nov 18, 2022 13:15 UTC (Fri) by walters (subscriber, #7396) [Link]

> In C it's all the same thing, of course.

Not quite; without a capacity value from a standard NUL terminated string, one can't do some useful patterns like having a single allocated "buffer" used in a loop without allocating/freeing. The equivalent of Rust's String is really things like https://developer-old.gnome.org/glib/stable/glib-Strings.html
(I'd assume there's some variant of this in the linux kernel? But not seeing it offhand)

Rust in the 6.2 kernel

Posted Nov 17, 2022 17:32 UTC (Thu) by khim (subscriber, #9252) [Link] (12 responses)

The difference between &str, String and Box<str> (which also) shows why Rust is a step forward while C++… not so much.

In C, of course, both “owned” and “borrowed” strings are represented as char *. C++ offers std::string and std::string_view, but… it's still responsibility of the developer to keep track of std::string_view's validity!

This makes C++ complication over C somewhat… unsatisfying: yes, we encoded difference in intents, but it's still our responsibility to keep track of everything… why do we need that complication?

But &str comes with additional assurances from the compiler: it's borrowed string, but it's compiler job to ensure that it's correctly borrowed! And String and Box<str> are owned, but it's compiler job to ensure they are correctly owned (initialized before use, etc).

That's why people say that Rust is attempting to raise the abstraction in the programming language: you genuinely can offload some of your knowledge into the machine and hope that it would verify that everything is done correctly.

It's similar to Sparse in some sense.

Rust in the 6.2 kernel

Posted Nov 18, 2022 5:25 UTC (Fri) by ma4ris5 (guest, #151140) [Link] (5 responses)

There is some information, that Rust language is near Calculus of Constructions,
which is a bit different than Lambda Calculus.

With Calculus of Constructions, it is possible to implement logical proofs.
This would imply, that Rust implementation needs to be within the logical proofs,
for being robust.

https://www.subarctic.org/is_rust_a_purely_functional_pro...
https://hbr.github.io/Lambda-Calculus/cc-tex/cc.pdf

Rust in the 6.2 kernel

Posted Nov 19, 2022 8:56 UTC (Sat) by gasche (subscriber, #74946) [Link] (4 responses)

I know about the Calculus of Construction, and I can tell you that this is mostly wrong. There is no obvious relation between Rust and the Calculus of Construction (which is about dependent types, no linear types), and the subarctic blog you cite is wrong. This also has no relevance to the Linux kernel or string types whatsoever.

Rust in the 6.2 kernel

Posted Nov 19, 2022 15:58 UTC (Sat) by hummassa (guest, #307) [Link] (3 responses)

Your #rude filter seem to be off.
The poster was making a good point (as does the linked page) that the dependent typing usage possible using Rust traits can be used to construct purely (or semi-purely) funcional Rust programs, that can be checked via automation. The correlation to the kernel is that the same tooling possible with Rust is not possible with C++ or C.

Rust in the 6.2 kernel

Posted Nov 19, 2022 21:46 UTC (Sat) by gasche (subscriber, #74946) [Link] (2 responses)

> Your #rude filter seem to be off.

Maybe? I find it rather perplexing to see important technical ideas of our field being cargo-culted around. Why would someone name-drop the Calculus of Constructions (a rather technical topic that is mostly of experts interest) if they clearly don't know what it is? (Otherwise they couldn't claim that Rust is closely related, which is grossly wrong.) It may be that some language communities or discussion spaces are used to this kind of pseudo-technical discourse, but I'm not,
and it hurts.

(I have tried to be short and factual in my post above, which I certainly did not intend to be insulting or deprecating.)

The message I was replying can be decomposed as follows:

> There is some information, that Rust language is near Calculus of Constructions,

This information is wrong, the Rust language is nowhere near the Calculus of Constructions. Rust has a strong type system with polymorphism, sure, but that's about it (so do many other languages). The characteristic feature of the Calculus of Constructions is its very powerful pi-types / dependent abstractions, which are completely absent from Rust -- or most programming languages.

> With Calculus of Constructions, it is possible to implement logical proofs.

This is true.

> This would imply, that Rust implementation needs to be within the logical proofs,
for being robust.

I don't know what this is supposed to mean, but my best guess is that there is a fundamental misunderstanding here. Even if the Rust system *could* in theory express logical proofs (Coq or Agda can, for example), it would be entirely possible to write code that contains bugs in the language (at less precise types).

The blog post that is being cited in the message ( https://www.subarctic.org/is_rust_a_purely_functional_pro... ) is similarly fundamentally wrong. Out of the five subsections, exactly 2 are correct ("What is the Calculus of Constructions" is essentially correct, and "So is Rust purely functional" is arguably correct), the 3 other contain gross mistakes. I mean, this blog post is titled "Is Rust a Purely Functional Programming Language?", and it starts with a definition of "purely functional programming language" that is wrong?! (ML is not a purely functional programming language.)

Rust in the 6.2 kernel

Posted Nov 19, 2022 22:45 UTC (Sat) by khim (subscriber, #9252) [Link] (1 responses)

I think you are talking past each other.

> Even if the Rust system *could* in theory express logical proofs (Coq or Agda can, for example), it would be entirely possible to write code that contains bugs in the language (at less precise types).

Rust type system can express logical proofs and it's not possible to circumvent it (except for bugs in the compiler, of course). That's precisely what the Ralf's Phd thesis is about.

Now, the weird part: Rust doesn't contain full-blown dependent types system which can be used pervasively, it's complicated system is centered on lifetimes and soundness.

Which basically means that it's enough to prove that there are no UBs in safe Rust, but not enough to prove much beyond that.

That's still significantly different property from what C/C++ have. And very practically useful.

As for how all that is related to functional programming… it's, basically, impossible to say.

> it starts with a definition of "purely functional programming language" that is wrong?! (ML is not a purely functional programming language.)

Who said it's “wrong?” Even Wikipedia's article on subject start with the exact difference between pure and impure functional programming is a matter of controversy sentence for crying out loud!

The big issue here is that reasonable people define “pureness” differently and then arrive at different conclusions.

Basically the best I can say about that link is… I couldn't say if he's even right or wrong because he talks about things which have not single “proper” definition.

Rust in the 6.2 kernel

Posted Nov 20, 2022 6:28 UTC (Sun) by gasche (subscriber, #74946) [Link]

> Rust type system *can* express logical proofs

You are changing what you mean by "express logical proofs" quite a bit from what the original poster said with "implement logical proofs", quote:

> With Calculus of Constructions, it is possible to implement logical proofs. This would imply, that Rust implementation needs to be within the logical proofs, for being robust.

With the Calculus of Constructions (or other similar logics), you can define types that correspond to interesting mathematical propositions, and then you can "implement" a proof of this proposition as a program fragment at this type. This is the "Curry-Howard" view of proving things using a typed lambda-calculus, it is what the Calculus of Propositions was designed for, and this is *not* something that is done in Rust.
(Of course, as any reasonably-powerful type system, it is possible through a lot of effort to encode something similar to this process for weaker notions of propositions, using for example singleton types and what not. This does not change the fact that claiming that Rust is related to the Calculus of Constructions is fundamentally nonsensical.)

Now you are talking about a much weaker (but still important/relevant) meaning of "logical proofs", which is: proof of safety guarantees guaranteed by the type system. The idea is not that you can define types to express mathematical properties of interest, but that each type come with behavioral guarantees that gives a property that each program fragment at this type must verify. (Working out precisely how to define these guarantees is the essence of the RustBelt project and Ralf Jung's thesis.) This has, again, nothing to do with the Calculus of Constructions -- well, this work was formulated in Coq, which is maybe how the crackpots above thought to claim a connection.

> and it's not possible to circumvent it (except for bugs in the compiler, of course)

and except for, you know, *unsafe*.

> Which basically means that it's enough to prove that there are no UBs in safe Rust, but not enough to prove much beyond that.

To be fair (I'm not trying to be critical here, and your point at least are informed and make sense), you can get more than the absence of UB when you look at programs at higher type. (For example I would expect that polymorphism gives you representation-independence properties that let you reason on whether some values remain "hidden" inside a module, or what API usage patterns are prevented by the types. Some of this stuff is standard in ML/Haskell grade type systems, and there are new Rust-specific tricks that we can play with lifetimes and the static discipline.)

> Who said it's “wrong?” Even Wikipedia's article on subject start with the exact difference between pure and impure functional programming is a matter of controversy sentence for crying out loud!

There is disagreement on the finer details (and sometimes the word is used in a completely different context, "pure lambda calculus" means something else), but there is no disagreement on the fact that ML-family languages are *impure* functional programming languages; they allow for unrestricted non-termination but also mutable state, exceptions... Anyone in the field agrees that ML-family languages are *not* purely functional.

Rust in the 6.2 kernel

Posted Nov 18, 2022 16:48 UTC (Fri) by ncm (guest, #165) [Link] (4 responses)

In fact, std::string_view validity does not need to be "kept track of". Passed down a call chain, it remains valid throughout, with no phony "complication". When you need to lie to make your case, it tells us all we need to know about your case. Hype reliant on spurious denigration of other languages adds no value here.

Rust in the 6.2 kernel

Posted Nov 18, 2022 17:05 UTC (Fri) by farnz (subscriber, #17727) [Link] (3 responses)

It does need to be kept track of - you need to ensure that the underlying string is not deallocated before the string view is deallocated.

For the very specific case of just passing a string view of a string you own down a call stack, there's no issues, but as soon as the string view relates to a string whose lifespan is not determined purely by the enclosing scope (e.g. because you put the string view in a heap-allocated data structure), you have a lifespan tracking issue to worry about.

Rust in the 6.2 kernel

Posted Nov 18, 2022 18:02 UTC (Fri) by khim (subscriber, #9252) [Link]

If someone claims that use of std::string_view doesn't lead to the problems then you can safely say that it's another crop of the “just don't do any mistakes and then C works fine… oh, and don't upgrade the compiler ever because these evil guys make it break it my programs”, just with C++ theme.

Here is discussion about dangers of std::string_view on the Core Guidelines site, here is article with more arguments and there are more, but they all are, obviously, wrong, because admitting that they are right means years of investment in C++ are in jeopardy.

Just leave these guys alone. It's the same thing as with systemd introduction: there would be lots of complains and there would be holdouts and yet it would happen in the classic planck's principle way:

> An important scientific innovation rarely makes its way by gradually winning over and converting its opponents: it rarely happens that Saul becomes Paul. What does happen is that its opponents gradually die out, and that the growing generation is familiarized with the ideas from the beginning.

Rust in the 6.2 kernel

Posted Dec 12, 2022 21:04 UTC (Mon) by oconnor663 (guest, #119484) [Link] (1 responses)

You (farnz) probably already know this but I think it's worth adding: Not only deallocated, but also reallocated. Calling .push_back() or .append() on the original string also potentially invalidates a string_view.

It could be totally fair to describe these as niche issues that don't affect most callers of string_view. But I don't agree with calling someone a "liar" because you (ncm) think it's niche. Or if these issues are new to you, great! None of us is ever done learning.

Rust in the 6.2 kernel

Posted Dec 13, 2022 11:14 UTC (Tue) by farnz (subscriber, #17727) [Link]

Yes - apologies for being unclear. I think of reallocations as an optimized form of allocate, copy and deallocate, so it's implicit to me that reallocation can invalidate the original string.

At heart, this is the same problem as iterator invalidation. You have a reference to some underlying data, and changes to that underlying data can result in your reference no longer being valid. C++ has no compiler checks for reference validity, and relies on the programmer not getting it wrong; this makes some sense, since any check for reference validity is going to be conservative and thus will need overriding from time to time, but the history of "rely on the programmer not getting it wrong" suggests it's not a great decision.

Illuminating background on Rust

Posted Dec 16, 2022 12:25 UTC (Fri) by sdalley (subscriber, #18550) [Link]

That's why people say that Rust is attempting to raise the abstraction in the programming language: you genuinely can offload some of your knowledge into the machine and hope that it would verify that everything is done correctly.

Thanks khim for that very illuminating link on the functional-programming origins of Rust!

Rust in the 6.2 kernel

Posted Nov 17, 2022 19:52 UTC (Thu) by tialaramex (subscriber, #21167) [Link] (8 responses)

Sadly interop with C means &CStr is way less useful than &str.

&str is a (aside from the UTF-8 promise which holds for all Rust strings including CStr I believe) just a slice, a "fat" pointer, an address (of the string) plus a length. Which means operations to get a substring don't mutate the string, they're just different addresses and lengths, the underlying string is unchanged. That includes strip_prefix, split_once, trim_end_matches, and a good many more.

Sadly &CStr can't do that, under the hood it too is an address plus a length (a slice), but it promises the last byte is always 0, ASCII NUL for C compatibility, and of course such substring operations wouldn't deliver that, so &CStr can't efficiently do them. [I guess it could do some of the strip_prefix type operations since those leave the far end alone]

Rust in the 6.2 kernel

Posted Nov 17, 2022 21:22 UTC (Thu) by djc (subscriber, #56880) [Link] (3 responses)

Also note that while the article seems to imply that CStr and CString are kernel-specific (and their implementation might well be), normal Rust code also has these types in the std library.

Rust in the 6.2 kernel

Posted Nov 17, 2022 22:57 UTC (Thu) by ssokolow (guest, #94568) [Link] (2 responses)

I suspect it has to do with accessing them through core::ffi::CStr and core::ffi::CString being an experimental/nightly-only feature (core_c_str) and them wanting to get to compatibility with stable compilers as quickly as possible.

It wouldn't surprise me if this contributes to the stabilization of the core_c_str feature.

Rust in the 6.2 kernel

Posted Nov 17, 2022 23:00 UTC (Thu) by ssokolow (guest, #94568) [Link]

Correction: I forgot to update my Dash/Zeal docset. core_c_str got stabilized in 1.64.

It's still possible that they're duplicating it to keep compatibility with earlier revisions of the Rust compiler that distros may be packaging though.

Rust in the 6.2 kernel

Posted Nov 17, 2022 23:02 UTC (Thu) by ssokolow (guest, #94568) [Link]

Correction: ...and "core::ffi::CString"? It's an allocating thing. I clearly need to go to sleep right now.

Rust in the 6.2 kernel

Posted Nov 17, 2022 21:45 UTC (Thu) by tux3 (subscriber, #101245) [Link]

>the UTF-8 promise which holds for all Rust strings including CStr I believe

CStr and CString only seem to promise to be NUL-terminated, I believe any valid char* should make a valid &CStr (which explains why I rememberd &CStr as _not_ cheap to convert to &str)

Rust in the 6.2 kernel

Posted Nov 18, 2022 3:18 UTC (Fri) by droundy (subscriber, #4559) [Link] (2 responses)

Actually &CStr differs from &str in that it is *not* a slice and doesn't hold a length asking with an address. So while I do prefer the standard &str, &CStr does have the advantage of taking up half as much space.

Rust in the 6.2 kernel

Posted Nov 18, 2022 3:59 UTC (Fri) by ABCD (subscriber, #53650) [Link] (1 responses)

CStr (both kernel::str::CStr and core::ffi::CStr) are implemented as dynamically-sized types (specifically, a newtype wrapper around [u8]), so a &CStr is a fat pointer containing both the pointer to the data and the length of that data, just like a &str or &[u8].

Rust in the 6.2 kernel

Posted Nov 18, 2022 14:21 UTC (Fri) by xav (guest, #18536) [Link]

It is yes, but AFAIK it's an oversight and could be replaced soon with a thin pointer - this is alluded to here for example: https://github.com/m-ou-se/rfcs/blob/c-str-literal/text/3...

Rust in the 6.2 kernel

Posted Nov 18, 2022 8:58 UTC (Fri) by rsidd (subscriber, #2582) [Link] (13 responses)

In the real world, Asahi Linya has been writing a kernel GPU driver for M1 Macs in Rust, and has made remarkable progress in a matter of weeks . Here's atwitter thread from early October. Clearly it uses much more than the minimal Rust framework code that is being upstreamed so far. Perhaps LWN can ask her for a guest post with some details on this?

Rust in the 6.2 kernel

Posted Nov 18, 2022 13:30 UTC (Fri) by Darkstar (guest, #28767) [Link] (10 responses)

I tried watching one of her streams once. Got an immediate headache because of the fake voice she's using :-D If anyone interviews her, please do it in text-only

Rust in the 6.2 kernel

Posted Nov 18, 2022 13:57 UTC (Fri) by [email protected] (guest, #121154) [Link]

This is a mean thing you're going entirely out of the way to say, particularly given how entirely unrelated it is to the original request for her to _write_ a guest post.

If you choose to post again here, I personally would appreciate it if you would choose to stay remotely on topic and to be a nicer person :-D.

Rust in the 6.2 kernel

Posted Nov 18, 2022 15:24 UTC (Fri) by Lumia (subscriber, #161010) [Link] (8 responses)

What an unpleasant thing to say.

Rust in the 6.2 kernel

Posted Nov 18, 2022 21:15 UTC (Fri) by beagnach (guest, #32987) [Link] (6 responses)

It's not actually... the complaint is about a very high-pitched machine-generated fake voice that the developer of the M1 GPU driver adds to their YouTube videos. So not a personal attach.

I also find it unbearable.

Example:
https://www.youtube.com/watch?v=XG9xUxIf73o

Rust in the 6.2 kernel

Posted Nov 18, 2022 22:00 UTC (Fri) by zdzichu (subscriber, #17118) [Link]

Which is, again, not relevant when discussing _writing_and article. It should ever be here on LWN, which we all consider a civilised place.

Rust in the 6.2 kernel

Posted Nov 18, 2022 22:00 UTC (Fri) by gspr (guest, #91542) [Link] (3 responses)

Has LWN done a lot of voice interviews? No. The comment clearly brought up something completely irrelevant in order to be mean.

Rust in the 6.2 kernel

Posted Nov 18, 2022 22:09 UTC (Fri) by ssokolow (guest, #94568) [Link] (2 responses)

Unless "you" was being used in the broader sense of "anyone who chooses to interview her", rather than LWN specifically.

Rust in the 6.2 kernel

Posted Nov 19, 2022 13:37 UTC (Sat) by Wol (subscriber, #4433) [Link] (1 responses)

Or they don't do email interviews and it has to be a video ... if they're a vlogger they might insist ...

Anyways, I consider it useful information to know that their videos are unwatchable ... there might be better ways to do it (I don't see how in this instance) but just telling the truth is not being nasty. Yes it can be told in a nasty way, but here it was just to inform ...

Cheers,
Wol

Rust in the 6.2 kernel

Posted Nov 19, 2022 15:07 UTC (Sat) by amacater (subscriber, #790) [Link]

It is, of course, possible that this content is effectively generated as anime-style content and a synthetic voice to preserve someone's
identity for some reason. There aren't many folk contributing to the kernel or other projects entirely pseudonymously but there are
a few, usually to prevent an obvious conflict of interest - maybe to preserve the ability to work on something as their own rather than as their employers' work - or to prevent themselves being marginalised for some other reason.

The synthetic-sounding voice *is* annoying but the content is useful.

Disclaimer: I only watched a few short segments of the video but I could certainly see that the voice would grate after a while.

Rust in the 6.2 kernel

Posted Jan 4, 2023 18:07 UTC (Wed) by sammythesnake (guest, #17693) [Link]

If it were me doing those videos, I would certainly have aimed for a clearer choice, as well as one that's less piercing. I doubt I could cope with that voice for 8 hours(!)

It's worth noting that the (auto-generated) subtitles are usable, even if not perfect, so you could watch with the audio turned down, though that wouldn't work well with having it on in the background of some other task and dipping in when something piques interest.

I wonder if there's an audio filter that approximates the inverse of whatever was used to mask the voice in the first place - that might provide a more natural sounding option that at least *my* ears would find kinder.

For now, I'll stick to reading her excellent blog posts as and when they get lunk to here on LWN :-P

Rust in the 6.2 kernel

Posted Nov 19, 2022 6:23 UTC (Sat) by eean (subscriber, #50420) [Link]

they're a vtuber so it actually isn't that bad, ha. it's cool that people are making content like this for different audiences.

Rust in the 6.2 kernel

Posted Nov 18, 2022 14:26 UTC (Fri) by corbet (editor, #1) [Link] (1 responses)

There's an awful lot of Rust infrastructure that has been posted in the past, including complete drivers; it's just not being pushed upstream yet. I've been fairly deliberately looking closely at the code as it heads toward the mainline just because it breaks the problem down into manageable pieces. The whole Rust-for-Linux patch set is a fair amount to absorb all at once.

Rust in the 6.2 kernel

Posted Dec 2, 2022 18:32 UTC (Fri) by ejr (subscriber, #51652) [Link]

And I very much thank you for the work. I'm trying to understand Rust's advantages and disadvantages, and this step-wise approach is hugely helpful to me.

Rust in the 6.2 kernel

Posted Nov 18, 2022 14:23 UTC (Fri) by xav (guest, #18536) [Link] (1 responses)

I wonder why they didn't use the standard log::info, log:debug etc. (and just added the missing levels) ?

Rust in the 6.2 kernel

Posted Nov 18, 2022 16:11 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

By "standard" do you mean the `log` crate? I believe some of its guarantees are around flushing and such, so `PR_CONT` might be tougher to implement using the APIs set down in the crate itself.

Rust in the 6.2 kernel

Posted Nov 19, 2022 9:02 UTC (Sat) by gasche (subscriber, #74946) [Link] (13 responses)

`Either` sounds like a generic type that would have its place in any "standard library for building stuff" for the Rust language, not particularly related to kernel programming. Do I correctly understand that the Rust-in-kernel people are not using any of the existing Rust stdlib, and are in fact reinventing their own stdlib? If so, then what are the mechanisms in place to ensure the ongoing quality and consistency of the design?

Rust in the 6.2 kernel

Posted Nov 19, 2022 9:42 UTC (Sat) by Wol (subscriber, #4433) [Link] (2 responses)

Bear in mind the kernel does not use the standard C stdlib either. Could they be re-writing the Rust stdlib for the exact same reason, namely (a) the stdlib breaks kernel invariants, and (b) the stdlib does not take into account that hardware breaks stdlib invariants?

Cheers,
Wol

Rust in the 6.2 kernel

Posted Nov 19, 2022 11:06 UTC (Sat) by gasche (subscriber, #74946) [Link] (1 responses)

I can see many reasons why kernel-side programming is a different enough environment to need its own stdlib, and there is nothing fundamentally wrong with that. But on the other hand, I would expect some principles about how to grow this kernel-stdlib. (Some people have experience growing standard libraries for languages/environments and some idea are probably worth porting over from the start.)

For example (this is just a random idea, not a suggestion or anything) Rust-in-kernel could have decided to minimize the diff with outside-kernel Rust code by sticking to subsets of standard APIs or well-known third party packages whenever possible, or taking other steps to avoid divergence and reuse existing design choices. I see no trace of such a process in the [Either patchset](https://lwn.net/ml/linux-kernel/20221110164152.26136-28-o...), which does not make any mention of pre-existing Either code in Rust outside the kernel.

Rust in the 6.2 kernel

Posted Nov 19, 2022 11:19 UTC (Sat) by khim (subscriber, #9252) [Link]

Rust doesn't have Either in it's stdlib. There are either crate with really featureful Either, and it's widely used, but from my understanding Linux kernel doesn't like to depend on external crates.

Rust in the 6.2 kernel

Posted Nov 20, 2022 14:57 UTC (Sun) by atnot (subscriber, #124910) [Link] (7 responses)

> `Either` sounds like a generic type that would have its place in any "standard library for building stuff"

This is somewhat unrelated to the rest of your question, but I personally really don't agree. Rust core already has types for Option and Result, which are the common case for things that can be one of two types. But to me the Either type ends up simultaneously too broad and specific compared to just defining your own enum. The generic "left" and "right" names are cryptic and confusing, there is little commonality betwen users and it has worse type safety and error reporting. It also lacks extensibility by only offering two variants, making for a painful refactoring if you notice you do need three options after all. Lastly a lot of the time when a function can take multiple things, those things usually share some property which can perhaps be better represented by a trait.

Rust in the 6.2 kernel

Posted Nov 20, 2022 20:42 UTC (Sun) by gasche (subscriber, #74946) [Link] (6 responses)

We've had this argument within the OCaml community, and actually I'm the person who ended up proposing 'either' for the standard library. I agree with your point that, very often, defining your own enum / variant type is the better approach, and also with your point that Option and Result already cover a lot of cases. But:

- The argument that "defining your own variant with domain-specific names etc." also applies as a criticism of both Option and Result, and the fact that actually we use Option and Result a lot shows that this criticism only goes so far. Sometimes there are common scenarios were having a enum of fixed shape with standard name is a good approach, and the benefits in terms of reusing other people's code (auxiliary support functions etc.) are higher than the cost of the less-specific names.

- There are generic functions for which Result could be used, but the more symmetric Either is more natural and thus a better API. (Result and Either are isomorphic so you can clearly always use one instead of the user, the question is how much of a conceptual mismatch this creates.) My main example in OCaml is

val partition_map : ('a -> ('b, 'c) either) -> 'a list -> 'b list * 'c list

It expects a function that, for any value of type 'a, will compute either a 'b or a 'c from it. Then it takes a list of elements of type 'a, and partitions it (using the function) into a list of 'b and a list of 'c. (Interestingly, this is an instance of a sort of generic operation that would split the 'a into an arbitrary sum of possible types, and return as many lists; but that function cannot be expressed easily in the OCaml type system, while 'partition_map' above can.)

Rust in the 6.2 kernel

Posted Nov 21, 2022 17:06 UTC (Mon) by steveklabnik (guest, #114343) [Link] (2 responses)

A small amount of history here: Rust *did* include Either in its standard library long, long ago. https://doc.rust-lang.org/0.7/std/either.html

My recollection of its removal was a survey of usage, and Result was used instead by 99.9% of the existing code instead.

That said, it is true that the crates.io has high usage; it is the dependent of two *extremely* popular packages, itertools and rayon. I'm not aware of any movement to move it back into the standard library, though.

Rust in the 6.2 kernel

Posted Nov 21, 2022 21:16 UTC (Mon) by micka (subscriber, #38720) [Link] (1 responses)

Looking at the Either crate api, it’s awful. At least Result gives you a meaningful name for each variant (ok, err).
Either gives you a... left and a right. Which is which? If both have the same content type, how do you differentiate?
By the way, some people (like me) can’t tell the right from the left. Or rather can’t give them a name (but never will for example drive on the wrong side of the road). I had to work on codd which used an Either type in another language and could never manage to understand any of it even after multiple years.
I guess it’self OK as write-only code...

Rust in the 6.2 kernel

Posted Nov 22, 2022 14:11 UTC (Tue) by TheGopher (subscriber, #59256) [Link]

Agree. Having worked with either in scala I can say that the overhead of a dedicated struct/union/variant is minimal - and the readability is much higher! This is the wrong place to be lazy.

Rust in the 6.2 kernel

Posted Nov 23, 2022 17:17 UTC (Wed) by khim (subscriber, #9252) [Link]

> - The argument that "defining your own variant with domain-specific names etc." also applies as a criticism of both Option and Result, and the fact that actually we use Option and Result a lot shows that this criticism only goes so far.

How? Option and Result are domain-specific.

Option is the fix for the billion-dollar mistake: it handles the case where object may or may not be present.

Result is for the case where function may return “normal” result or “error result”. Open POSIX specifications (or practically any API specification) and you'll find many such functions.

> - There are generic functions for which Result could be used, but the more symmetric Either is more natural and thus a better API.

It's not enough to have one such function. One function is always better server with ad-hock type. You need series of functions which may share a common type. It's easy to imagine such for Options: lot's of data structures have “leaf nodes”. It even easier to do that with Result: almost all functions which deal with files or network may suffer from the same errors (it doesn't matter whether you are creating file or removing it, if you don't have permission to do that error would be the exact same one).

Just what class of functions do you have in mind where you may have symmetrical two choices and these are the same across the whole range of functions?

Yes, partition_map example makes sense (and that's exactly where itertools are using Either), but Rust is imperative language with for. It's really not clear how often do you even need such thing in Rust.

Rust in the 6.2 kernel

Posted Nov 24, 2022 1:48 UTC (Thu) by atnot (subscriber, #124910) [Link] (1 responses)

> the benefits in terms of reusing other people's code (auxiliary support functions etc.) are higher than the cost of the less-specific names.

That is my problem with Either though. It's too vague to enable any reuse at all. I think a look at the associated functions of Option and Result in rust compared to the Either crate illustrate that quite clearly.

Option and Result have dozens of combinators each that compose usefully. You can turn Results into Options, Options into Results, Options into Iterators, Results of Options into Results, Iterators of Options into Options, Iterators of Results into Results...

Meanwhile on the Either side we get:
(https://docs.rs/either/1.8.0/either/enum.Either.html)
- A few map variations, duplicated for left and right of course
- Some forwarding of inner traits
- A bunch of methods that turn it back into the more useful Option and Result types
- Flip, that swaps the sides

This is not a swipe against the authors. It just shows you just can't really do very much with types that are completely unconstrained and semantically meaningless.

Rust in the 6.2 kernel

Posted Nov 26, 2022 14:21 UTC (Sat) by gasche (subscriber, #74946) [Link]

Another type that is "too vague" is the type of pair `(A, B)` -- and there are not that many standard-library functions defined on it. But we use it all the time! Honestly I think that the criticism against `Either` here comes from a lack of familiarity, rather than a deep reason. We programmers tend to be more familiar with product types than with sum types; for sums the most well-known is `Option<A>` that is the sum (A + 1), and we still need to time to collectively get used to using anonymous sum types when they make sense.

Rust in the 6.2 kernel

Posted Nov 30, 2022 12:07 UTC (Wed) by mlindner (guest, #162450) [Link] (1 responses)

I don't really understand the need of adding the `Either` type, it's just a very very thin wrapper around "enum" with functionality limited to only two types. I wonder why they added it.

Rust in the 6.2 kernel

Posted Dec 3, 2022 0:58 UTC (Sat) by nybble41 (subscriber, #55106) [Link]

`Either` is one of the two combinators needed to represent arbitrary arithmetic data types, together with the zero/void (`!`) and one/unit (`()`) atomic types. `Either` represents the capacity for sums (enums). Pairs / tuples are their dual, representing product types (structs). They're both very abstract, but sometimes that's exactly what you want.

It's true that you could just create your own custom enum type instead (which would be isomorphic to some tree of nested `Either` types) but your custom type won't benefit from any functions the standard library might provide for working generically with values which might be "either" one type or another. For example, partitioning a list of `Either<A,B>` into a list of `A` and a list of `B`. I'm not as familiar with what Rust provides for working with `Either` values but in Haskell the `Either` type is a key component of the Arrow abstraction, a more powerful (albeit less popular) alternative to monads, where they are used to represent conditional processing (`ArrowChoice`). It also benefits from ready-made instances of `Bifunctor`, `Bifoldable`, and `Bitraversable`.


Copyright © 2022, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds