I think a lot of these obfuscate the code rather than improve it.

I don't disagree. These were semi-manual fixes any way. Probably the most useful changes here are of this type, changing:

a = []
for x in sequence:
    if cond:
        a.append(x)

to:

a = [
    x
    for x in sequence
    if cond
]

List and dict comprehensions are more efficient, and for many programmers not more obfuscate, once you get used to them.

In any case, I'm OK with discarding this PR. Instead, perhaps we should focus on #26857 as it is a step towards #24994.

Thinking about it again:

• I can understand why you think "these obfuscate the code rather than improve it" and I don't disagree with you, but that's just an opinion based on your programming habits and exposure to Python code.
• List (and dict) comprehensions are a more efficient way to initialise a list (or dict). They were added to Python2 for a reason. They might appear less readable, but then that's the syntax of Python. This is a Python library after all.

• I can understand why you think "these obfuscate the code rather than improve it" and I don't disagree with you, but that's just an opinion based on your programming habits and exposure to Python code.

FWIW, list comprehensions are a commonly used example in the excellent The Programmer's Brain of a confusing construct. Generally speaking, it can sometimes aid readability in the simpler cases where you can transform a 4-line snippet with lots of repetition into a nice DRY one-liner. In Hermans' terms, this can reduce the diffuseness of the code.

But in the more complicated cases, like many of these in the PR, especially those with if clauses, we're not really saving on the linecount, but maybe only winning a bit in the DRYness. Comprehensions are kind of backwards things. Our eyes first see the expression of the item being added to the container before the variables are even defined. Then we get the for loop that defines the variables and where they come from. Then we get the if clause that recontextualizes the whole purpose of the thing. In the for loop version, we get each of those things in the order our brain needs them.

As a rule, I'm probably going to use a comprehension if it gets me to a one-liner, but probably opt for a for loop if I'm going to end up splitting up the comprehension over the same number of lines anyways. I am quite happy to keep PERF401 and PERF403 turned off in our configuration and let authors decide what to use.

we get each of those things in the order our brain needs them.

I am not aware of a neuroscience study showing that our brain needs that order. In German, the past participle is rejected at the end of the sentence and the language is not less clear than English.

In any case, the order would be a Python shortcoming. Numpy remains a Python library.

Point taken, and no I don't have any specific reference for you about that particular point. Most eye-tracking studies can't resolve movements that small (but I will note that natural language is not an ideal comparison; we have specialized wetware for our native languages, natural language sentences are generally shorter, and context often overdetermines which way a sentence is going to fall out in any case). There are a number of studies ranking the general readability/understandability of Python list comprehensions lower than the corresponding for loop, though. Google Scholar finds them readily.

for loops are also Python. No one is saying that comprehensions are forbidden. Generally, I'd be happy to accept either version of each of these in newly-authored code (but also resist changing from one to the other without specific reasons for each one). But I don't care to have these PERF rules requiring one or the other or forcing a switch to one for performance concerns that haven't been specifically demonstrated.

I understand, choice and readability are more important than (marginal gains in) performance. Perhaps most if not all PERF rules should remain disabled when switching to ruff.

BTW, it would be great if you could have a look at #26857, as a milestone towards #24994.