Make `ForEach-Object` faster for its commonly used scenarios #10454

daxian-dbw · 2019-08-27T17:59:13Z

PR Summary

NOTE: mark the PR as WIP because the rewriting causes some issues to stepping during debugging. Looking into that.

A follow-up PR for #10047
Make ForEach-Object faster for its commonly used scenarios by rewriting the command into a filter-like script block execution in the pipeline.
For the follow ForEach-Object usages, they will be rewritten to ... | . { process { ... } } | ... when constructing the pipeline.

... | % { ... } | ...
... | % -process { ... } | ...
... | % -process:{ ... } | ...

By doing this rewriting, we are able to eliminate all unnecessary overheads from:

parameter binding
invoking a script block with ScriptBlock.InvokeUsingCmdlet for every incoming object, which has to do the setting-up/tearing-down every time executing the script block.

This PR targets to improve the performance of ForEach-Object for the most common usages. It's not a goal to cover all possible scenarios, such as % -pro:{...} or % { ... } { ... } { ... }

Script	Before	After
`1..100kb \| % { }`	290ms	37ms
`1..100kb \| . { process {} }`	37ms	37ms
`foreach ($i in 1..100kb) { }`	28ms	28ms

This optimization will be turned off automatically in the following cases:

when debugger is enabled, namely when any breakpoint is set in the session. This is to make the debugger works as expected in case a breakpoint is set on ForEach-Object, e.g Set-PSBreakpoint -Command ForEach-Object.
when 'ConstrainedLanguageMode' has been used for the current Runspace. The language mode transition is tricky. It's better to stick with the same old code path for safety.

PR Context

#9731 (comment)
Draft PR: #10153

PR Checklist

…ommand in pipeline

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs

rjmholt

Naturally I'd be interested to know if there's some other body or context I could execute the optimised and unoptimised version in that isn't debugging where the behaviour would differ, but I can't think of one

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs

KirkMunro · 2019-08-29T15:40:50Z

I always assumed optimizations like this would be handled in an AstVisitor implementation, hoping such optimizations could be performed while retaining the extent in the original scripts so that neither the script author nor the debugger is aware of them while code just runs faster. This PR seems to indicate that my assumption is incorrect.

My naive questions about this: Is it simply not possible to perform optimizations such as this one transparently to the user and the debugger by doing the work in the AST itself? Or is it possible, but an approach like that is considered undesirable for other reasons?

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs

daxian-dbw · 2019-08-29T18:32:47Z

Is it simply not possible to perform optimizations such as this one transparently to the user and the debugger by doing the work in the AST itself?

The original AST should be preserved exactly as is because it should reflect exactly what the user typed on the console or wrote in the file.
If your thought is to generate another AST via a visitor before compiling, and use that one for compilation, then there is no guarantee the extent will be exactly the same or it will be transparent to the debugger.
Take this change as an example, once foreach-object { ... } is replaced by . { process { ... } } in the generated AST, you will have problem on the extent of the process { part, because that's new elements added the generated AST. Also, there won't be a CommandProcessor representing the ForEach-Object in the pipeline, and thus the debugger will break at the open curly instead of ForEach-Object for Set-PSBreakpoint -Command Foreach-Object.

For this specific change, I think there is something we can try if we really want to make it work with debugger: when rewriting the pipeline, instead of creating a command processor with the script block, we can construct a FunctionInfo with the name ForEach-Object and make its extent the same as the ForEach-Object extent. That might work, but of course might cause new issues that are unknown yet.

rjmholt · 2019-08-29T18:49:42Z

@KirkMunro

Is it simply not possible to perform optimizations such as this one transparently to the user and the debugger by doing the work in the AST itself? Or is it possible, but an approach like that is considered undesirable for other reasons

It's definitely possible. There's a good example in DynamicKeywordStatementAst:

PowerShell/src/System.Management.Automation/engine/parser/ast.cs

Lines 6817 to 7027 in 704ee90

    
           internal PipelineAst GenerateCommandCallPipelineAst() 
        
           { 
        
               if (_commandCallPipelineAst != null) 
        
                   return _commandCallPipelineAst; 
        
               ///////////////////////////////////////////////////////////////////////// 
        
               // 
        
               // Now construct the AST to call the function that defines implements the keywords logic. There are 
        
               // two different types of ASTs that may be generated, depending on the settings in the DynamicKeyword object. 
        
               // The first type uses a fixed set of 4 arguments and has the command with the signature 
        
               //  function moduleName\keyword 
        
               //  { 
        
               //      param ( 
        
               //          $KeywordData,       # the DynamicKeyword object processed by this rule. 
        
               //          $Name,              # the value of the name expression syntax element. This may be null for keywords that don't take a name 
        
               //          $Value,             # the body of the keyword - either a hashtable or a scriptblock 
        
               //          $SourceMetadata     # a string containing the original source line information so that errors 
        
               //      ) 
        
               //      # function logic... 
        
               //  } 
        
               // 
        
               // The second type, where the DirectCall flag is set to true, simply calls the command directly. 
        
               // In the original source, the keyword body will be a property collection where the allowed properties 
        
               // in the collection correspond to the parameters on the actual called function. 
        
               // 
        
               var cea = new Collection<CommandElementAst>(); 
        
               // 
        
               // First add the name of the command to call. If a module name has been provided 
        
               // then use the module-qualified form of the command name.. 
        
               // 
        
               if (string.IsNullOrEmpty(Keyword.ImplementingModule)) 
        
               { 
        
                   cea.Add( 
        
                       new StringConstantExpressionAst( 
        
                           FunctionName.Extent, 
        
                           FunctionName.Text, 
        
                           StringConstantType.BareWord)); 
        
               } 
        
               else 
        
               { 
        
                   cea.Add( 
        
                       new StringConstantExpressionAst( 
        
                           FunctionName.Extent, 
        
                           Keyword.ImplementingModule + '\\' + FunctionName.Text, 
        
                           StringConstantType.BareWord)); 
        
               } 
        
               ExpressionAst expr = BodyExpression; 
        
               HashtableAst hashtable = expr as HashtableAst; 
        
               if (Keyword.DirectCall) 
        
               { 
        
                   // If this keyword takes a name, then add it as the parameter -InstanceName 
        
                   if (Keyword.NameMode != DynamicKeywordNameMode.NoName) 
        
                   { 
        
                       cea.Add( 
        
                           new CommandParameterAst( 
        
                               FunctionName.Extent, 
        
                               "InstanceName", 
        
                               InstanceName, 
        
                               FunctionName.Extent)); 
        
                   } 
        
                   // 
        
                   // If it's a direct call keyword, then we just unravel the properties 
        
                   // in the hash literal expression and map them to parameters. 
        
                   // We've already checked to make sure that they're all valid names. 
        
                   // 
        
                   if (hashtable != null) 
        
                   { 
        
                       bool isHashtableValid = true; 
        
                       // 
        
                       // If it's a hash table, validate that only valid members have been specified. 
        
                       // 
        
                       foreach (var keyValueTuple in hashtable.KeyValuePairs) 
        
                       { 
        
                           var propName = keyValueTuple.Item1 as StringConstantExpressionAst; 
        
                           if (propName == null) 
        
                           { 
        
                               isHashtableValid = false; 
        
                               break; 
        
                           } 
        
                           else 
        
                           { 
        
                               if (!Keyword.Properties.ContainsKey(propName.Value)) 
        
                               { 
        
                                   isHashtableValid = false; 
        
                                   break; 
        
                               } 
        
                           } 
        
                           if (keyValueTuple.Item2 is ErrorStatementAst) 
        
                           { 
        
                               isHashtableValid = false; 
        
                               break; 
        
                           } 
        
                       } 
        
                       if (isHashtableValid) 
        
                       { 
        
                           // Construct the real parameters if Hashtable is valid 
        
                           foreach (var keyValueTuple in hashtable.KeyValuePairs) 
        
                           { 
        
                               var propName = (StringConstantExpressionAst)keyValueTuple.Item1; 
        
                               ExpressionAst propValue = new SubExpressionAst( 
        
                                   FunctionName.Extent, 
        
                                   new StatementBlockAst( 
        
                                       FunctionName.Extent, 
        
                                       new StatementAst[] { (StatementAst)keyValueTuple.Item2.Copy() }, null)); 
        
                               cea.Add( 
        
                                   new CommandParameterAst( 
        
                                       FunctionName.Extent, 
        
                                       propName.Value, 
        
                                       propValue, 
        
                                       LCurly.Extent)); 
        
                           } 
        
                       } 
        
                       else 
        
                       { 
        
                           // Construct a fake parameter with the HashtableAst to be its value, so that 
        
                           // tab completion on the property names would work. 
        
                           cea.Add( 
        
                               new CommandParameterAst( 
        
                                   FunctionName.Extent, 
        
                                   "InvalidPropertyHashtable", 
        
                                   hashtable, 
        
                                   LCurly.Extent)); 
        
                       } 
        
                   } 
        
               } 
        
               else 
        
               { 
        
                   // 
        
                   // Add the -KeywordData parameter using the expression 
        
                   //  ([type]("System.Management.Automation.Language.DynamicKeyword"))::GetKeyword(name) 
        
                   // To invoke the method to get the keyword data object for that keyword. 
        
                   // 
        
                   var indexExpr = new InvokeMemberExpressionAst( 
        
                       FunctionName.Extent, 
        
                       new TypeExpressionAst( 
        
                           FunctionName.Extent, 
        
                           new TypeName( 
        
                               FunctionName.Extent, 
        
                               typeof(System.Management.Automation.Language.DynamicKeyword).FullName)), 
        
                       new StringConstantExpressionAst( 
        
                           FunctionName.Extent, 
        
                           "GetKeyword", 
        
                           StringConstantType.BareWord), 
        
                       new List<ExpressionAst> 
        
                           { 
        
                               new StringConstantExpressionAst( 
        
                                   FunctionName.Extent, 
        
                                   FunctionName.Text, 
        
                                   StringConstantType.BareWord) 
        
                           }, 
        
                       true); 
        
                   cea.Add( 
        
                       new CommandParameterAst( 
        
                           FunctionName.Extent, 
        
                           "KeywordData", 
        
                           indexExpr, 
        
                           LCurly.Extent)); 
        
                   // 
        
                   // Add the -Name parameter 
        
                   // 
        
                   cea.Add( 
        
                       new CommandParameterAst( 
        
                           FunctionName.Extent, 
        
                           "Name", 
        
                           InstanceName, 
        
                           LCurly.Extent)); 
        
                   // 
        
                   // Add the -Value parameter 
        
                   // 
        
                   cea.Add( 
        
                       new CommandParameterAst( 
        
                           LCurly.Extent, 
        
                           "Value", 
        
                           expr, 
        
                           LCurly.Extent)); 
        
                   // 
        
                   // Add the -SourceMetadata parameter 
        
                   // 
        
                   string sourceMetadata = FunctionName.Extent.File 
        
                                           + "::" + FunctionName.Extent.StartLineNumber 
        
                                           + "::" + FunctionName.Extent.StartColumnNumber 
        
                                           + "::" + FunctionName.Extent.Text; 
        
                   cea.Add( 
        
                       new CommandParameterAst( 
        
                           LCurly.Extent, "SourceMetadata", 
        
                           new StringConstantExpressionAst( 
        
                               FunctionName.Extent, 
        
                               sourceMetadata, 
        
                               StringConstantType.BareWord), 
        
                           LCurly.Extent)); 
        
               } 
        
               // 
        
               // Build the final statement - a pipeline containing a single element with is a CommandAst 
        
               // containing the command we've built up. 
        
               // 
        
               var cmdAst = new CommandAst(FunctionName.Extent, cea, TokenKind.Unknown, null); 
        
               cmdAst.DefiningKeyword = Keyword; 
        
               _commandCallPipelineAst = new PipelineAst(FunctionName.Extent, cmdAst, background: false); 
        
               return _commandCallPipelineAst; 
        
           }

But to my mind, the questions are those of layering:

How faithfully should the AST represent the program as it was given to the parser?
How much should the AST know about what consumes it?
To what extent (heh) should I be able to reconstruct a program from its AST?
If we add syntactic sugar, does the parser own it, the AST or the compiler?

These problems (/restatements of the same problem) are inconsistently addressed in PowerShell:

The parser doesn't record where traps occur in a block at all, so the AST is created with some arguably important information missing
ScriptBlockAsts for scriptblocks without an explicit end block still have the End property populated because the AST type favours the consumer of the AST over structural fidelity
Dynamic keyword invocations get their own AST, but they do their own AST transformation to be compiled
The compiler's *Ops libraries have some syntactic sugar transformations in them, like for job control operators (Get-Command &) and here with the ForEach-Object now

If we were writing it all over today, we might have a syntactic sugar phase/layer between the parser and the compiler implemented as an AstVisitor that takes in one AST and spits out the transformed one, although the immutability of the AST (which I still think is a very good thing) makes that a bit more expensive. That's certainly the way other statically compiled. languages have gone.

But I do feel ambivalent about syntactic sugar (in the strict sense of a language-level AST-to-AST transformation, where one syntax is a macro for another) in an interpreted language. I'm not sure about Python, Ruby and friends, but both C# and Java seem to have a policy against it as far as I can tell; they go direct to bytecode and leave the optimisations to the runtime.

For PowerShell though, I think (1) we want to encourage good practice with syntax, (2) it's good to try and find performance wherever we can while still finding time for other work.

So basically, I don't think we have a good, consistent answer for how something like this should be implemented, but this implementation is one of the better ones in terms of both how it's done and where it sits.

KirkMunro · 2019-08-29T21:42:46Z

Those answers are very, very helpful. Thank you @daxian-dbw and @rjmholt for taking the time to provide that information.

iSazonov · 2019-08-30T05:35:49Z

I like the optimization because it works well for all scenarios.
Although it’s obvious that there could be optimizations that can work well in a particular situation. That's why the question of Ast optimizations arises.

For example, for a simple management script in a task scheduler, an optimization is not to do any optimization at all since the interpreter will be the fastest.
For data manning script (parse and analyze large log file(s)), we could say in advance pwsh -Optimization Full to apply all optimizations before run. In the scenario generating optimized Ast tree could bring significant acceleration. Currently single optimization we do for such scenarios - switch from interpreter to compiler for long running script blocks.

they go direct to bytecode and leave the optimizations to the runtime.

Since we have new modern runtime we could review if we can generate code so that it can optimize by the runtime better.

* Build(deps): Bump Microsoft.PowerShell.Archive from 1.2.2.0 to 1.2.3.0 in /src/Modules (PowerShell#9593) * Support line continuance with pipe at the start of a line (PowerShell#8938) Expands line continuance for pipelines to allow lines to continue automatically without backticks with the pipe symbol at the start of a line. This adds to the existing functionality where pipes can be used to continue lines by placing them at the end of a line. * Fix use of unicode ellipsis in xml (PowerShell#9589) * Updating committee membership (PowerShell#9577) * Build(deps): Bump System.Data.SqlClient from 4.6.0 to 4.6.1 (PowerShell#9601) * Build(deps): Bump PowerShellGet from 2.1.2 to 2.1.3 in /src/Modules (PowerShell#9600) * Build(deps): Bump Microsoft.Windows.Compatibility from 2.0.1 to 2.1.1 (PowerShell#9605) * Build(deps): Bump NJsonSchema from 9.13.37 to 9.14.1 (PowerShell#9616) Bumps [NJsonSchema](https://github.com/rsuter/NJsonSchema) from 9.13.37 to 9.14.1. - [Release notes](https://github.com/rsuter/NJsonSchema/releases) - [Commits](https://github.com/rsuter/NJsonSchema/commits) Signed-off-by: dependabot[bot] <[email protected]> * Convert windows CI to stages (PowerShell#9607) * Switch from BMP to PNG for graphical MSI installer assets (PowerShell#9606) This breaks rendering of images for Windows 7 and Server 2008R2, but it's required for accessibility purposes so that the High Contrast Black theme renders text correctly for accessibility. (Unfortunately, BMP does not support transparency.) * Add checkbox to PR checklist for experimental feature use (PowerShell#9619) * Make sure we always return an object in command searcher (PowerShell#9623) * Remove Workflow from PSSessionType (PowerShell#9618) Remove PSSessionType and IsWorkflowConfigurationType() * Port PowerShell to .NET Core 3.0 (PowerShell#9597) * Attributes.cs - Style / Formatting Fixes (PowerShell#9625) * Update README and metadata.json (PowerShell#9624) * Update version tests to use NextReleaseVersion from metadata.json (PowerShell#9646) * Build(deps): Bump PackageManagement from 1.3.2 to 1.4 in /src/Modules (PowerShell#9650) * Build(deps): Bump PackageManagement from 1.3.2 to 1.4 in /src/Modules Bumps PackageManagement from 1.3.2 to 1.4. Signed-off-by: dependabot[bot] <[email protected]> * Update vNext release branches in dependabot config (PowerShell#9658) * Build(deps): Bump Microsoft.CodeAnalysis.CSharp from 3.0.0 to 3.1.0 (PowerShell#9653) * Disable the debugger when in system lock-down mode (PowerShell#9645) Disable the debugger when in system lock-down mode Fixing master for CVE-2019-0733 * Build(deps): Bump PowerShellGet from 2.1.3 to 2.1.4 in /src/Modules (PowerShell#9691) * Run Start-PSBootStrap in Code Coverage build to install .NET SDK (PowerShell#9690) * Merged PR 8504: Fix syncing modules from powershell gallery by normalizing version numbers Fix syncing modules from powershell gallery by normalizing version numbers and added additional logging. * Fix daily `CodeCoverageAndTest` build by explicitly calling `Start-PSBootStrap` (PowerShell#9724) * Merged PR 8510: Update the target framework for reference assemblies to netcoreapp3.0 Update the target framework for reference assemblies to netcoreapp3.0 * Merged PR 8512: Update version for SDK tests and Microsoft.PowerShell.Native package Update version for SDK tests and Microsoft.PowerShell.Native package * Pin version of netDumbster to 2.0.0.4 (PowerShell#9748) * Update the target framework for reference assemblies to netcoreapp3.0 (PowerShell#9747) Update the target framework for reference assemblies to netcoreapp3.0 * Merged PR 8529: Add cleanup before building test package Add cleanup before building test package * Change log for release 6.2.1 (PowerShell#9760) Changelog for 6.2.1 and fix spellings * Change log 6.1.4 (PowerShell#9759) * Update change log for 6.1.4 Update change log for 6.1.4 * Resolve conflicts * Merged PR 8547: Fix the PowerShell version number in MSI packages Fix the PowerShell version number in MSI packages * Merged PR 8542: changelog draft changelog draft * Update README and metadata.json for 7.0.0-preview.1 release (PowerShell#9767) * Increase timeout of NuGet job to workaround build timeout (PowerShell#9772) * Code Cleanup: Tidy up scriptblock.cs (PowerShell#9732) * Fix `Get-ChildItem -Path` with wildcard char (PowerShell#9257) Unescape non-literal, non-glob path before existence checking. * Build(deps): Bump Microsoft.ApplicationInsights from 2.9.1 to 2.10.0 (PowerShell#9757) * Disable stale bot in favor of an internal bot that we can customize more (PowerShell#9785) * Build(deps): Bump System.Net.Http.WinHttpHandler from 4.5.3 to 4.5.4 (PowerShell#9786) * Use new string.ConCat() in Process.cs (PowerShell#9720) * Build(deps): Bump NJsonSchema from 9.14.1 to 10.0.13 (PowerShell#9805) * Support negative numbers in `-split` operator (PowerShell#8960) * Fix use of `Start-Process http://bing.com` (PowerShell#9793) * Improve whitespace for Parser tests (PowerShell#9806) * Build(deps): Bump NJsonSchema from 10.0.13 to 10.0.14 (PowerShell#9843) * Fix minor style issues come from last commits (PowerShell#9640) * Support DSC compilation on Linux. (PowerShell#9834) * Build(deps): Bump NJsonSchema from 10.0.14 to 10.0.15 (PowerShell#9854) * Have console host not enter command prompt mode when using `Read-Host -Prompt` (PowerShell#9743) `Read-Host` calls into `$Host.UI.Prompt()`. However, this method is also used when the host prompts for mandatory parameters that aren't provided. The method expects to be called when given a `FieldDescription` and if the input starts with `!` it enters `CommandPromptMode`. In this mode, you can type `!?` to request help, for example. However this mode is not something you can use via `Read-Host` (only if calling `$Host.UI.Prompt()` directly passing in a well constructed `FieldDescription`). When using `Read-Host -Prompt`, the cmdlet creates a `FieldDescription` where the name is the prompt and the rest of the properties are empty. The fix is that if `Label` is empty, we can assume it's being called from `Read-Host` rather than being called to prompt for a mandatory parameter and thus not enter `CommandPromptMode`. * Add ability to pass `InitialSessionState` to the `ConsoleShell.Start` (PowerShell#9802) * Build(deps): Bump NJsonSchema from 10.0.15 to 10.0.17 (PowerShell#9862) * Add launchSettings.json for better out of box experience when using Visual Studio (PowerShell#9818) * Build(deps): Bump NJsonSchema from 10.0.17 to 10.0.18 (PowerShell#9875) * Add module to support Pester tests for automating debugger commands (stepInto, stepOut, etc.), along with basic tests (PowerShell#9825) * Build(deps): Bump NJsonSchema from 10.0.18 to 10.0.19 (PowerShell#9885) * Make `UseAbbreviationExpansion` and `TempDrive` official features (PowerShell#9872) * Attempt to work around the zip download issue in Azure DevOps Windows CI (PowerShell#9911) * Fix unix project mappings (PowerShell#9892) * Update to use TSAv2 (PowerShell#9914) * Use yarn to install global tools (PowerShell#9904) * Build(deps): Bump PackageManagement from 1.4 to 1.4.1 in /src/Modules (PowerShell#9820) Bumps PackageManagement from 1.4 to 1.4.1. * Build(deps): Bump PackageManagement from 1.4.1 to 1.4.2 in /src/Modules (PowerShell#9918) * Import-DscResource should allow to overwrite DSC built-in resources. (PowerShell#9879) * Use the original precision (prior-dotnet-core-3) for double/fload-to-string conversion (PowerShell#9893) .NET Core changes to return "shortest roundtrippable string" by default for the ToString() method of double and float types. This results in ToString() for double/float values sometimes return a string in 17-digit/9-digit precision format. This PR updated the double/float-to-string conversion in PowerShell to continue using the old precision specifier before the change in .NET Core 3.0. * Build(deps): Bump PowerShellGet from 2.1.4 to 2.1.5 in /src/Modules (PowerShell#9933) Bumps [PowerShellGet](https://github.com/PowerShell/PowerShellGet) from 2.1.4 to 2.1.5. - [Release notes](https://github.com/PowerShell/PowerShellGet/releases) - [Changelog](https://github.com/PowerShell/PowerShellGet/blob/development/CHANGELOG.md) - [Commits](https://github.com/PowerShell/PowerShellGet/commits) Signed-off-by: dependabot-preview[bot] <[email protected]> * Clean up the use of 'SetProfileRoot' and 'StartProfile' in ConsoleHost (PowerShell#9931) * Clean up the use of 'SetProfileRoot' and 'StartProfile' in ConsoleHost * Remove 'pwsh.pdb' from our component file list * Update console startup and help url for PowerShell docs (PowerShell#9775) * Fix minor CodeFactor style issues in ModuleCmdletBase (PowerShell#9915) * Build(deps): Bump NJsonSchema from 10.0.19 to 10.0.20 (PowerShell#9954) Bumps [NJsonSchema](https://github.com/rsuter/NJsonSchema) from 10.0.19 to 10.0.20. - [Release notes](https://github.com/rsuter/NJsonSchema/releases) - [Commits](https://github.com/rsuter/NJsonSchema/commits) Signed-off-by: dependabot-preview[bot] <[email protected]> * Style fixes for CimAsyncOperations (PowerShell#9945) * Improve release testing Docker images (PowerShell#9942) * Remove some Travis-CI references (PowerShell#9919) * Fix minor typos in code comments (PowerShell#9917) * Indentation fixes in ci.psm1 (PowerShell#9947) * Update readme gitter badge (PowerShell#9920) * Display com method signature with argument names (PowerShell#9858) * Display Duration when displaying `HistoryInfo` (PowerShell#9751) * Remove dead code about 'IsTransparentProxy' (PowerShell#9966) * Fix style issues from last commits (PowerShell#9937) * Suppress sporadic exceptions from JumpList creation code (PowerShell#9928) * Remove EtwActivity empty constructor and make minor style fixes (PowerShell#9958) * Sync docs changes into the embedded help for pwsh (PowerShell#9952) * Fix amazonlinux install script (PowerShell#9967) * Fix wrong comparison in CertificateProvider (PowerShell#9987) * Cleanup Parser tests (PowerShell#9792) Co-Authored-By: Ilya <[email protected]> * Remove LCIDToLocaleName P/Invoke from GetComputerInfoCommand (PowerShell#9716) - Remove LCIDToLocaleName P/Invoke with StringBuilder - Reduce allocations by using GetCultureInfo to get cached CultureInfo - Use UInt32.TryParse to parse Hex - Minor cleanups after above changes * Fix gulp versions (PowerShell#9916) * Add quick steps for adding docs to cmdlets (PowerShell#9978) * Cleanup: Use EndsWith(char) and StartsWith(char) (PowerShell#9994) * Add test for `New-Item -Force` (PowerShell#9971) * Allow methods to be named after keywords (PowerShell#9812) * Move consts and methods to single CharExtensions class (PowerShell#9992) * Build(deps): Bump ThreadJob from 1.1.2 to 2.0.1 in /src/Modules (PowerShell#10003) * Code cleanup: use IndexOf(char) overload (PowerShell#9722) - Use IndexOf(char) overload instead of IndexOf(string) to benefit from Ordinal search - Replace IndexOf with Contains to get more understandable code - Use StringComparison.Ordinal for ASCII chars * Add automated RPM signing to release build (PowerShell#10013) * Download latest version (6.2.0) of PSDesiredStateConfiguration nuget package. (PowerShell#9932) * Update copyright symbol for NuGet packages (PowerShell#9936) * Avoid the `int[]` and `int[,]` allocation when tokenizing line comments and matching wildcard pattern (PowerShell#10009) - Reusing the same 2-dimensional integer array for processing line comments in the same Tokenizer. - Use ArrayPool<int> in PatternPositionVisitor to avoid creating transient int[]. * Bump NJsonSchema from 10.0.20 to 10.0.21 (PowerShell#10017) Bumps [NJsonSchema](https://github.com/rsuter/NJsonSchema) from 10.0.20 to 10.0.21. - [Release notes](https://github.com/rsuter/NJsonSchema/releases) - [Commits](https://github.com/rsuter/NJsonSchema/commits) Signed-off-by: dependabot-preview[bot] <[email protected]> * Consider `DBNull.Value` and `NullString.Value` the same as `$null` when comparing with `$null` and casting to bool (PowerShell#9794) - Adds `LanguagePrimitives.IsNullLike()` method to account for `DBNull.Value` and `NullString.Value` so that they can be considered the same as a null value where sensible in PowerShell. - Updates `-ne` and `-eq` binders to treat `DBNull.Value` and `NullString.Value` as equal to null/AutomationNull. - Update code paths for comparing objects in LanguagePrimitives to ensure consistency with how the `-eq` and `-ne` binders work when calling LanguagePrimitives methods to do the comparisons. - Make `LanguagePrimitives.IsNull()` and `LanguagePrimitives.IsNullLike()` public methods. - Added tests for null behaviours in `NullRepresentatives.Tests.ps1` * Start-PSBuild -Clean doeas not remove all untracked files (PowerShell#10022) * Avoid `Assembly.GetName()` in 'ClrFacade.GetAssemblies(string)' to reduce allocations of 'CultureInfo' objects (PowerShell#10024) * Use AddRange in GetModules() (PowerShell#9975) * Add fast path for wildcard patterns that contains no wildcard characters (PowerShell#10020) * Set request headers when request body is empty in Web Cmdlets (PowerShell#10034) * Suppress PossibleIncorrectUsageOfAssignmentOperator rule violation by adding extra parenthesis (PowerShell#9460) * Use `AddOrUpdate()` instead of `Remove` then `Add` to register runspace (PowerShell#10007) * Reduce allocations in NavigationCmdletProvider.NormalizePath() (PowerShell#10038) * Indent fix in markdown-link.tests.ps1 (PowerShell#10049) * Use a static cache for `PSVersionInfo.PSVersion` to avoid casting `SemanticVersion` to `Version` very time accessing that property (PowerShell#10028) * Bump Markdig.Signed from 0.17.0 to 0.17.1 (PowerShell#10062) * Cleanup workflow code (PowerShell#9638) * Remove WorkFlowInfo type * Remove condition for UseSharedProcess * Remove PSWorkflowJob condition * Remove workflow from ValidateSet * Remove workflow from CommandTypes enum * Remove workflow from EventManager * Remove workflow from Get-Help * Remove WorkflowFileExtension ".xaml" from ModuleIntrinsics * Remove WorkflowFileExtension ".xaml" from ModuleCmdletBase * Remove workflow from PSModuleInfo * Remove workflow from CustomShellCommands * Remove workflow from InitialSessionStateProvider * Remove WriteWorkflowDebugNotSupportedError() * Remove unneeded resource strings from Module.cs * Remove xaml from valid extension list * Add to /etc/shells on macOS (PowerShell#10066) Co-Authored-By: Robert Holt <[email protected]> * Check if a reparsepoint is a symlink before trying to get target (PowerShell#9895) * Make `Foreach-Object` 2 times faster by reducing unnecessary allocations and boxing (PowerShell#10047) * Update 'Start-PSBuild -Clean' logic of 'git clean' to ignore locked files from VS2019 (PowerShell#10071) * Create JumpList in STA thread as some COM APIs are strictly STA only to avoid sporadic CLR crashes (PowerShell#10057) * Upgrade .Net Core 3 SDK from preview5 to preview6 and related out of band Nuget packages from 2.1 to 3.0-preview6 (PowerShell#9888) * Bump PackageManagement from 1.4.2 to 1.4.3 in /src/Modules (PowerShell#10084) Bumps PackageManagement from 1.4.2 to 1.4.3. Signed-off-by: dependabot-preview[bot] <[email protected]> * add performance tag to change log command * Bump PowerShellGet from 2.1.5 to 2.2 in /src/Modules (PowerShell#10085) Bumps [PowerShellGet](https://github.com/PowerShell/PowerShellGet) from 2.1.5 to 2.2. - [Release notes](https://github.com/PowerShell/PowerShellGet/releases) - [Changelog](https://github.com/PowerShell/PowerShellGet/blob/development/CHANGELOG.md) - [Commits](https://github.com/PowerShell/PowerShellGet/commits) Signed-off-by: dependabot-preview[bot] <[email protected]> * Move some of the creations of `WildcardPattern` in outer loop to avoid unnecessary allocation (PowerShell#10053) * Add another fast path to `WildcardPattern.IsMatch` for patterns that only have an asterisk in the end (PowerShell#10054) * Update change log generation tool to deal with private commits (PowerShell#10096) * Bump System.Text.Encoding.CodePages (PowerShell#10112) Bumps System.Text.Encoding.CodePages from 4.6.0-preview6.19303.8 to 4.6.0-preview.19113.10. Signed-off-by: dependabot-preview[bot] <[email protected]> * Bump System.Security.AccessControl (PowerShell#10100) Bumps System.Security.AccessControl from 4.6.0-preview6.19303.8 to 4.6.0-preview.19113.10. Signed-off-by: dependabot-preview[bot] <[email protected]> * Bump System.Threading.AccessControl (PowerShell#10106) Bumps System.Threading.AccessControl from 4.6.0-preview6.19303.8 to 4.6.0-preview.19113.10. Signed-off-by: dependabot-preview[bot] <[email protected]> * Bump System.Data.SqlClient (PowerShell#10109) Bumps System.Data.SqlClient from 4.7.0-preview6.19303.8 to 4.7.0-preview.19113.10. Signed-off-by: dependabot-preview[bot] <[email protected]> * Bump System.Management (PowerShell#10110) Bumps System.Management from 4.6.0-preview6.19303.8 to 4.6.0-preview.19113.10. Signed-off-by: dependabot-preview[bot] <[email protected]> * Bump System.DirectoryServices (PowerShell#10105) Bumps System.DirectoryServices from 4.6.0-preview6.19303.8 to 4.6.0-preview.19113.10. Signed-off-by: dependabot-preview[bot] <[email protected]> * Bump System.Security.Cryptography.Pkcs (PowerShell#10107) Bumps System.Security.Cryptography.Pkcs from 4.6.0-preview6.19303.8 to 4.6.0-preview.19113.10. Signed-off-by: dependabot-preview[bot] <[email protected]> * Bump System.IO.Packaging (PowerShell#10108) Bumps System.IO.Packaging from 4.6.0-preview6.19303.8 to 4.6.0-preview.19113.10. Signed-off-by: dependabot-preview[bot] <[email protected]> * Bump System.Security.Principal.Windows (PowerShell#10101) Bumps System.Security.Principal.Windows from 4.6.0-preview6.19303.8 to 4.6.0-preview.19113.10. Signed-off-by: dependabot-preview[bot] <[email protected]> * Bump System.Security.AccessControl (PowerShell#10102) Bumps System.Security.AccessControl from 4.6.0-preview6.19303.8 to 4.6.0-preview.19113.10. Signed-off-by: dependabot-preview[bot] <[email protected]> * Bump System.Configuration.ConfigurationManager (PowerShell#10111) Bumps System.Configuration.ConfigurationManager from 4.6.0-preview6.19303.8 to 4.6.0-preview.19113.10. Signed-off-by: dependabot-preview[bot] <[email protected]> * Add -SecurityDescriptorSddl parameter to Set-Service (PowerShell#8626) * Bump System.Text.Encodings.Web (PowerShell#10123) Bumps System.Text.Encodings.Web from 4.6.0-preview6.19303.8 to 4.6.0-preview.19113.10. Signed-off-by: dependabot-preview[bot] <[email protected]> * Revert "Bump System.IO.Packaging (PowerShell#10108)" (PowerShell#10125) This reverts commit 45edbd7. * Revert "Bump System.Data.SqlClient (PowerShell#10109)" (PowerShell#10126) This reverts commit 83d8ec4. * Revert "Bump System.Threading.AccessControl (PowerShell#10106)" (PowerShell#10127) This reverts commit a7f0c6d. * Revert "Bump System.Text.Encodings.Web (PowerShell#10123)" (PowerShell#10124) This reverts commit b9501d9. * Revert "Bump System.Management (PowerShell#10110)" (PowerShell#10129) This reverts commit e5990ed. * Revert "Bump System.Security.Cryptography.Pkcs (PowerShell#10107)" (PowerShell#10130) This reverts commit d33c906. * Revert "Bump System.Text.Encoding.CodePages (PowerShell#10112)" (PowerShell#10131) This reverts commit 958cdda. * Revert "Bump System.Security.AccessControl (PowerShell#10102)" (PowerShell#10132) This reverts commit 729a29a. * Revert "Bump System.Configuration.ConfigurationManager (PowerShell#10111)" (PowerShell#10128) This reverts commit c8fe894. * Revert "Bump System.Security.Principal.Windows (PowerShell#10101)" (PowerShell#10133) This reverts commit f75c635. * Cleanup workflow - remove PSProxyJob (PowerShell#10083) * Revert "Bump System.DirectoryServices (PowerShell#10105)" (PowerShell#10135) This reverts commit 2852efc. # Conflicts: # src/System.Management.Automation/System.Management.Automation.csproj * Fix test password generation rule to meet Windows complexity requirements (PowerShell#10143) * Cleanup CompiledScriptBlock.cs (PowerShell#9735) * Add alias for Service 'StartType' (PowerShell#9940) * Mark Set-Service tests with password as `Pending` (PowerShell#10146) * Split the fxdependent package on Windows into two packages (PowerShell#10134) * Remove 'kill' alias for Stop-Process cmdlet on Unix (PowerShell#10098) * Identify renaming a parameter set as non-breaking in the breaking change contract document (PowerShell#10113) * Identify renaming a parameter set as non-breaking Related discussion: PowerShell#10058. * Add note about S.M.A.Internal * Update DotNet Support links (PowerShell#10145) * Remove `markdownlint` tests due to security issues (PowerShell#10163) * Remove `markdownlint` tests due to security issues (PowerShell#10163) * Merged PR 5767: Fix RegEx DoS issues Fix RegEx DoS issues * Skip `JumpList` on `NanoServer` and `IoT` (PowerShell#10164) * Skip `JumpList` on `NanoServer` and `IoT` (PowerShell#10164) * Merged PR 9185: Update PowerShell SDK version for hosting tests Update PowerShell SDK version for hosting tests * Merged PR 9168: Disable Enter-PSHostProcess cmdlet when system in lock down mode This is based on an issue, where Enter-PSHostProcess on a locked down (WDAC enforced) machine allows any admin to connect to any another local hosted PowerShell process and execute commands as that user. This amounts to privilege escalation on the policy locked down machine and something we want to prevent. Fix is to check for system lock down and disable Enter-PSHostProcess cmdlet with an error message. * Update README.md and metadata.json for next releases (PowerShell#10087) * Add tooling section to PR template (PowerShell#10144) * Add .devcontainer configuration (PowerShell#10114) * Add the license header to `nanoserver.tests.ps1` (PowerShell#10171) * Refactor security policy documentation so that they appear in the Security policy tab of GitHub (PowerShell#9905) Co-Authored-By: Travis Plunk <[email protected]> * PSSA also includes formatting (PowerShell#10172) * Merged PR 9166: Update changelog for v7.0.0-preview.2 Update changelog for v7.0.0-preview.2 Updates are expected when STA / MTA change comes in. * Fix hungarian prefix 'my' (PowerShell#9976) * Update docs for `7.0.0-preview.2` release (PowerShell#10160) * Update release date for `v7.0.0-preview.2` (PowerShell#10176) * Reduce allocations in Escape() and Unescape() (PowerShell#10041) * Add -Raw switch to Select-String for convenience (PowerShell#9901) * Avoid boxing when passing value type arguments to `PSTraceSource.WriteLine` (PowerShell#10052) * Fix macOS build break (PowerShell#10207) * Use nameof() in LocationGlobber and PathInfo (PowerShell#10200) * Use nameof() in LocationGlobber * Use nameof() in PathInfo * Fix: Removed dependency file with Dependabot (PowerShell#10212) * Enable `-sta` and `-mta` switches for pwsh (PowerShell#10061) * Use 'Platform.IsWindowsDesktop' instead of checking both NanoServer and IoT (PowerShell#10205) * Special case the posix locale in WildcardPattern (PowerShell#10186) * Quote arguments in `.vscode/tasks.json` in case of spaces (PowerShell#10204) * Create`Distribution_Request` issue template (PowerShell#10253) * Fix spelling error in issue template (PowerShell#10256) * Revert the temporary change (PowerShell#10260) * Bump Microsoft.CodeAnalysis.CSharp from 3.1.0 to 3.2.0 (PowerShell#10273) Bumps [Microsoft.CodeAnalysis.CSharp](https://github.com/dotnet/roslyn) from 3.1.0 to 3.2.0. - [Release notes](https://github.com/dotnet/roslyn/releases) - [Changelog](https://github.com/dotnet/roslyn/blob/master/docs/Breaking%20API%20Changes.md) - [Commits](https://github.com/dotnet/roslyn/commits) Signed-off-by: dependabot-preview[bot] <[email protected]> * Cleanup: ThreadAbortException is now in .Net Core 3.0 (PowerShell#10230) * Cleanup: ThreadAbortException is now in .Net Core 3.0 * Fix CodeFactor issue * Remove ThreadAbortException * Remove ThreadAbortException relared code * Remove stable and servicing branches from dependabot (PowerShell#10289) * Update dead links from powershell.com (PowerShell#10297) * Remove extra check that the system dll exists (PowerShell#10244) * Update PowerShell to build against .NET Core 3.0-preview7 (PowerShell#10227) * Fix debugger disable performance regression (PowerShell#10269) * Don't collect process starttime as it's not being used on consolehost startup (PowerShell#10294) * mark _readyForInputTimeInMS member as part of LEGACYTELEMETRY * move _interactiveCommandCount to LEGACYTELEMETRY * Cleanup Docker release testing (PowerShell#10310) * Update our language on our policy applying to security issues (PowerShell#10304) * Add tests for WildcardPattern.Escape() and Unescape() (PowerShell#10090) * Update 'Microsoft.PowerShell.CoreCLR.Eventing' to resolve conflict with 'System.Diagnostics.EventLog' (PowerShell#10305) * Update 'Microsoft.PowerShell.CoreCLR.Eventing' to resolve conflicts * Add reference to 'System.Diagnostics.EventLog' to build Microsoft.PowerShell.Commands.Diagnostics on Unix * Stop compiling Get/New-WinEvent on Unix * Enable Experimental Features by default on Preview builds (PowerShell#10228) * Bump Microsoft.CodeAnalysis.CSharp from 3.2.0 to 3.2.1 (PowerShell#10330) Bumps [Microsoft.CodeAnalysis.CSharp](https://github.com/dotnet/roslyn) from 3.2.0 to 3.2.1. - [Release notes](https://github.com/dotnet/roslyn/releases) - [Changelog](https://github.com/dotnet/roslyn/blob/master/docs/Breaking%20API%20Changes.md) - [Commits](https://github.com/dotnet/roslyn/commits) Signed-off-by: dependabot-preview[bot] <[email protected]> * Expose PreRelease label in PSModuleInfo formatter (PowerShell#10316) * Reduce allocations in MakePath() method (PowerShell#10027) * Add `Microsoft.PowerShell.CoreCLR.Eventing.dll` to exception list for build fix (PowerShell#10337) * Add dispose of _runspaceDebugCompleteEvent event object. (PowerShell#10323) * Make module name matching for `get-module -FullyQualifiedName`… (PowerShell#10329) * Fix `#requires -version` for pwsh 7 to include 6.1 and 6.2 in `PSCompatibleVersions` (PowerShell#9943) * Implement ForEach-Object -Parallel feature (PowerShell#10229) * Add support for AppX reparse points (PowerShell#10331) * Make Get-DscResource work with class based resources (PowerShell#10350) * pwsh -Login support (PowerShell#10050) * Deprecate internal HelpCategory.Workflow enumeration (PowerShell#10319) * Fix style issues in InternalCommands.cs (PowerShell#10352) * Deprecate workflow debugging code (PowerShell#10321) * Bump NJsonSchema from 10.0.21 to 10.0.22 (PowerShell#10364) Bumps [NJsonSchema](https://github.com/RicoSuter/NJsonSchema) from 10.0.21 to 10.0.22. - [Release notes](https://github.com/RicoSuter/NJsonSchema/releases) - [Commits](https://github.com/RicoSuter/NJsonSchema/commits) Signed-off-by: dependabot-preview[bot] <[email protected]> * Move to .NET Core 3.0 preview.8 (PowerShell#10351) * Update Microsoft.Management.Infrastructure version to 2.0.0-preview.2 (PowerShell#10366) * Mark `-parallel` and `-throttlelimit` reserved for `foreach` and `switch` statements (PowerShell#10328) * Fix minor breakpoint re-hydration bug (PowerShell#10339) * Additional Telemetry - Implementation of `RFC0036` (PowerShell#10336) * Formatting: Handle `XTPUSHSGR` and `XTPOPSGR` control sequences (PowerShell#10208) * .CPL should be added to PATHEXT (PowerShell#9828) * Add experimental check for `ForEach-Object -Parallel` tests (PowerShell#10354) * Bump `PackageManagement` from `1.4.3` to `1.4.4` (PowerShell#10383) Bumps PackageManagement from 1.4.3 to 1.4.4. Signed-off-by: dependabot-preview[bot] <[email protected]> * Bump PowerShellGet from 2.2 to 2.2.1 in /src/Modules (PowerShell#10382) Bumps [PowerShellGet](https://github.com/PowerShell/PowerShellGet) from 2.2 to 2.2.1. - [Release notes](https://github.com/PowerShell/PowerShellGet/releases) - [Changelog](https://github.com/PowerShell/PowerShellGet/blob/development/CHANGELOG.md) - [Commits](https://github.com/PowerShell/PowerShellGet/commits) Signed-off-by: dependabot-preview[bot] <[email protected]> * Merged PR 9744: Update changelog for preview.3 release Update changelog for preview.3 release * Update `README.md` and `metadata.json` for `7.0.0-preview.3` (PowerShell#10393) * Add `PSKoans` to Learning Resources documentation (PowerShell#10369) * Add some regular contributors to allow access to retry CI (PowerShell#10397) * Some dependencies for Alpine need `linux-x64` folder (PowerShell#10407) * Block type update in Add-Type cmdlet (PowerShell#9609) * Increase built-with-PowerShell module versions to 7.0.0.0 (PowerShell#10356) * Rename default ParameterSetName back to `Delimiter` for `ConvertTo-Csv` & `ConvertFrom-Csv` (PowerShell#10425) * rename default parameterset name back to `Delimiter` * also fix convertfrom-csv * rename parametersetname for UseCulture back to UseCulture since there is no path for ConvertFrom-Csv * Skip auto-loading PSReadLine on Windows if the NVDA screen reader is active (PowerShell#10385) * Bump NJsonSchema from 10.0.22 to 10.0.23 (PowerShell#10421) Bumps [NJsonSchema](https://github.com/RicoSuter/NJsonSchema) from 10.0.22 to 10.0.23. - [Release notes](https://github.com/RicoSuter/NJsonSchema/releases) - [Commits](https://github.com/RicoSuter/NJsonSchema/commits) Signed-off-by: dependabot-preview[bot] <[email protected]> * Add support to `ActionPreference.Break` to break into debugger (PowerShell#8205) * Alpine validation changes (PowerShell#10428) * Remove yet another appveyor reference (PowerShell#10445) * Update the combined package build to release the daily builds (PowerShell#10449) * Cleanup `AutomationEngine` and remove extra `SetSessionStateDrive` method call (PowerShell#10416) * Make sure the daily coordinated build, knows it is a daily bui… (PowerShell#10464) * Minor fix for recursion into OneDrive - Change FindFirstFileEx() to use SafeFindHandle type (PowerShell#10405) * Update README.md (PowerShell#10465) Fix typo. * Make `ForEach-Object` faster for its commonly used scenarios (PowerShell#10454) * Fix global tool issues around exit code, command line parameters and path with spaces (PowerShell#10461)

KirkMunro · 2019-09-03T16:06:36Z

@daxian-dbw This PR introduces a breaking change. 💥

Step 1: Create a file with the following contents:

1..2 | ForEach-Object{

    Write-Host('MyInvocation in ForEach-Object: ' + $MyInvocation.MyCommand.Name)

}

Step 2: Dot-source or invoke that file with the call operator.

Before this change (using preview 3):
$MyInvocation.MyCommand.Name results in the name of the file.

After this change (using a custom build with the latest bits, pulled just now):
$MyInvocation.MyCommand.Name is $null.

Related: #10477.

daxian-dbw · 2019-09-03T21:52:20Z

Interesting, I knew there might be cases that I didn't catch :) I will be working on a fix.

I guess we also need an answer to this question:

why does the behaviour differ between Where-Object and ForEach-Object? They are both cmdlets, and I don't think it's unreasonable for users to expect a consistent experience between these two commands in terms of how their scriptblock implementations behave.

…10454)" This reverts commit ff29282.

daxian-dbw · 2019-09-05T00:35:42Z

#10485 was submitted to revert this PR given that it introduced a breaking change to the value of $MyInvocation within the script block argument.

…ell#10454)

ghost · 2019-09-19T17:52:15Z

🎉v7.0.0-preview.4 has been released which incorporates this pull request.:tada:

Handy links:

Release Notes

Make ForEach-Object 6x faster for its common usage by rewriting the c…

0dbf683

…ommand in pipeline

daxian-dbw changed the title ~~Make ForEach-Object faster for its commonly used scenarios~~ WIP: Make ForEach-Object faster for its commonly used scenarios Aug 27, 2019

daxian-dbw assigned adityapatwardhan Aug 27, 2019

PoshChan reviewed Aug 27, 2019

View reviewed changes

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Show resolved Hide resolved

PoshChan reviewed Aug 27, 2019

View reviewed changes

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Show resolved Hide resolved

PoshChan reviewed Aug 27, 2019

View reviewed changes

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Show resolved Hide resolved

Set the parent of the rewritten script block

efaa319

PoshChan reviewed Aug 27, 2019

View reviewed changes

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Show resolved Hide resolved

PoshChan reviewed Aug 27, 2019

View reviewed changes

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Show resolved Hide resolved

PoshChan reviewed Aug 27, 2019

View reviewed changes

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Show resolved Hide resolved

iSazonov reviewed Aug 28, 2019

View reviewed changes

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Show resolved Hide resolved

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Outdated Show resolved Hide resolved

daxian-dbw added 2 commits August 28, 2019 11:23

Skip the optimization for foreach-object when debugger is enabled

fb0e9df

Add comments about the target syntax for optimization

3d2cf3f

KirkMunro reviewed Aug 28, 2019

View reviewed changes

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Show resolved Hide resolved

PoshChan reviewed Aug 28, 2019

View reviewed changes

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Show resolved Hide resolved

Skip optimization when constrained language mode has been used

2e4e1b1

daxian-dbw changed the title ~~WIP: Make ForEach-Object faster for its commonly used scenarios~~ Make ForEach-Object faster for its commonly used scenarios Aug 28, 2019

daxian-dbw requested review from JamesWTruher, PaulHigin and rjmholt August 28, 2019 21:26

rjmholt approved these changes Aug 29, 2019

View reviewed changes

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Outdated Show resolved Hide resolved

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Outdated Show resolved Hide resolved

iSazonov reviewed Aug 29, 2019

View reviewed changes

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Show resolved Hide resolved

PaulHigin approved these changes Aug 29, 2019

View reviewed changes

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Outdated Show resolved Hide resolved

src/System.Management.Automation/engine/runtime/Operations/MiscOps.cs Show resolved Hide resolved

Address Rob and Paul's comment about early return

c34fdd2

daxian-dbw added the CL-Performance Indicates that a PR should be marked as a performance improvement in the Change Log label Aug 29, 2019

adityapatwardhan approved these changes Aug 30, 2019

View reviewed changes

adityapatwardhan merged commit ff29282 into PowerShell:master Aug 30, 2019

daxian-dbw deleted the foreach branch August 30, 2019 21:14

daxian-dbw mentioned this pull request Sep 3, 2019

$MyInvocation.MyCommand.Name becomes null in Where-Object block #10477

Closed

daxian-dbw added a commit that referenced this pull request Sep 4, 2019

Revert "Make ForEach-Object faster for its commonly used scenarios (#…

76d446a

…10454)" This reverts commit ff29282.

daxian-dbw mentioned this pull request Sep 4, 2019

Revert the PR "Make ForEach-Object faster for its commonly used scenarios" #10485

Merged

TravisEz13 added this to the 7.0.0-preview.4 milestone Sep 5, 2019

TravisEz13 pushed a commit to TravisEz13/PowerShell that referenced this pull request Sep 14, 2019

Make ForEach-Object faster for its commonly used scenarios (PowerSh…

5a0c4df

…ell#10454)

daxian-dbw mentioned this pull request Nov 5, 2019

Speed up Foreach-Object and Where-Object by using Steppable Pipeline #10982

Closed

iSazonov mentioned this pull request May 18, 2020

v5: Discovery Performance pester/Pester#1207

Closed

Make ForEach-Object faster for its commonly used scenarios #10454

Make ForEach-Object faster for its commonly used scenarios #10454

Uh oh!

Conversation

daxian-dbw commented Aug 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Context

PR Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rjmholt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

KirkMunro commented Aug 29, 2019

Uh oh!

Uh oh!

Uh oh!

daxian-dbw commented Aug 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rjmholt commented Aug 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KirkMunro commented Aug 29, 2019

Uh oh!

iSazonov commented Aug 30, 2019

Uh oh!

KirkMunro commented Sep 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daxian-dbw commented Sep 3, 2019

Uh oh!

daxian-dbw commented Sep 5, 2019

Uh oh!

ghost commented Sep 19, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

Make `ForEach-Object` faster for its commonly used scenarios #10454

Make `ForEach-Object` faster for its commonly used scenarios #10454

daxian-dbw commented Aug 27, 2019 •

edited

Loading

daxian-dbw commented Aug 29, 2019 •

edited

Loading

rjmholt commented Aug 29, 2019 •

edited

Loading

KirkMunro commented Sep 3, 2019 •

edited

Loading