Unhandled exception in Interop.Sys.AllocNullTerminatedArray while starting a process

Description

Here's my code that I'm running with .NET Core 3.1 on Ubuntu Linux 20.04:

var psi = new ProcessStartInfo
{
	FileName = fileName,
	WorkingDirectory = "/",
	UseShellExecute = false,
	CreateNoWindow = true
};
startInfo.RedirectStandardOutput = true;
startInfo.RedirectStandardError = true;
startInfo.RedirectStandardInput = false;
using var process = Process.Start(startInfo);

I get this exception:

- System.ArgumentNullException: String reference not set to an instance of a String. (Parameter 's')
  at System.Text.Encoding.GetBytes(String s)
  at Interop.Sys.AllocNullTerminatedArray(String[] arr, Byte**& arrPtr)
  at Interop.Sys.ForkAndExecProcess(String filename, String[] argv, String[] envp, String cwd, Boolean redirectStdin, Boolean redirectStdout, Boolean redirectStderr, Boolean setUser, UInt32 userId, UInt32 groupId, UInt32[] groups, In>
  at System.Diagnostics.Process.ForkAndExecProcess(String filename, String[] argv, String[] envp, String cwd, Boolean redirectStdin, Boolean redirectStdout, Boolean redirectStderr, Boolean setCredentials, UInt32 userId, UInt32 groupI>
  at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
  at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)
  at MyApp.Shared.Util.ProcessHelper.Execute(ProcessStartInfo startInfo, TimeSpan timeout, String stdin, CancellationToken cancellationToken) in (...)
  ...

I have no idea what Interop.Sys.AllocNullTerminatedArray is and where the source code of it can be inspected. .NET Core code finding is a challenge every time, especially since the search capabilities of GitHub are not worth mentioning and the full source code never even loads in Visual Studio on my computer.

So that method has some trouble when it gets null from somewhere and tries to pass it into Encoding.GetBytes. A null check must be added somewhere. You know where. I have no means to do that in my application. Until then, my task of running that external program cannot be accomplished with .NET.

Configuration

.NET Core SDK (reflecting any global.json):
 Version:   3.1.402
 Commit:    9b5de826fd

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  20.04
 OS Platform: Linux
 RID:         ubuntu.20.04-x64
 Base Path:   /usr/share/dotnet/sdk/3.1.402/

Host (useful for support):
  Version: 3.1.8
  Commit:  9c1330dedd

.NET Core SDKs installed:
  3.1.402 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.App 3.1.8 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.8 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

Additional info: It seems the started process only generates a single line of output on stderr but nothing on stdout. Can't .NET handle that?

I have no idea what Interop.Sys.AllocNullTerminatedArray is and where the source code of it can be inspected.

You can search here:
https://source.dot.net/#System.Diagnostics.Process/Interop.ForkAndExecProcess.cs,d5a7621387c9aab4

Here's my code

Can you share a standalone repro we can try running?

This should do, but "Try .NET" has no ls command on its Unix system. Strange. Maybe a bad chroot environment.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;

namespace myApp
{
    class Program
    {
        static void Main()
        {
            var startInfo = new ProcessStartInfo
            {
                FileName = "/usr/bin/ls",
                Arguments = "--xyz",
                WorkingDirectory = "/",
                UseShellExecute = false,
                CreateNoWindow = true,
                RedirectStandardOutput = true,
                RedirectStandardError = true
            };
            var process = Process.Start(startInfo);
        }
    }
}

Update: This may not be enough because it doesn't read the output streams. (That's not in the originally reported stack trace, but there's async code around and you never know what that does.) But doing that in a way that won't deadlock isn't exactly trivial with .NET. I'd need to prepare a more extensive project that handles all the other shortcomings in the API. If you need that, please let me know. If you know how to read from a process, go ahead.

But anyway, judging from the stack trace, I'd just add a null check at the Encoding.GetBytes call in the method you linked to. It's missing. I don't know what should happen then, though. Maybe treat it like an empty string.

Update: This may not be enough because it doesn't read the output streams.

Like this you mean?

Console.WriteLine($"Stdout: {await process.StandardOutput.ReadToEndAsync()}");
Console.WriteLine($"Stderr: {await process.StandardError.ReadToEndAsync()}");

I couldn't reproduce any unexpected behaviour using the 3.1.403 sdk.

No, that won't work reliably. If the process writes a bit to stdout, then much to stderr before closing both streams, this will block forever because the pipe buffer is limited and stderr won't be read until stdout is closed. You need to await both in parallel. I'll prepare some code for it, but it'll be longer than a short example. Process handling in .NET isn't trivial.

@ygoe

I'll prepare some code for it, but it'll be longer than a short example.

Hi @ygoe , any update on this?

@ygoe

@ygoe ping

I'm sorry I don't know anymore where the original issue appeared. So I probably cannot help much to fix this bug in .NET now. But it's certainly still there and will affect others.

Here's my process helper class that should actually be included in .NET because it took me weeks to find out how process execution really works. The existing BCL API is insufficient to accomplish that. It just deadlocks very easily.

https://gist.github.com/ygoe/94ec60d80a760b189e8030bdd20691a4

You could try to use it with a script like that:

#!/bin/bash
echo test >&2

Or use the sample code above.

@stephentoub

@stephentoub Nice change, but I don't see how it's related to the issue reported here.

@tmds

See #59562 (comment). Basically @tmds cited that the repro here doesn't actually reproduce the cited issue, but called out that the error being reported happens when a null string ends up in the ArgumentList. A null there isn't allowed (it's a developer error) but the exception stack/message are confusing, so that change fixed the exception stack/message to be more clear about what the problem is. If you have a repro on .NET 6 that demonstrates the issue without putting a null into ArgumentList, please re-open this and share the repro. Thanks.

I never showed code that puts a null in that list. Looking at my old code samples here, that list isn't even touched anywhere. But since I don't know where I had this issue a year ago, I can't provide more detail here. Let's just call this issue closed/wontfix and something else improved. Process execution with stream piping is a minefield in .NET.

@ygoe

Process execution with stream piping is a minefield in .NET.

@ygoe I admit that it's far from perfect. Recently we started doing some research about areas where .NET is lagging and it was recognized that System.Diagnostics.Process is one of them.

For now I encourage you to give the amazing CliWrap a try.

Navigation Menu

Repository navigation

Unhandled exception in Interop.Sys.AllocNullTerminatedArray while starting a process #44034

Description

Configuration

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Unhandled exception in Interop.Sys.AllocNullTerminatedArray while starting a process #44034

Description

Description

Configuration

Activity

ygoe commented on Oct 29, 2020

stephentoub commented on Oct 29, 2020

ygoe commented on Oct 29, 2020

eiriktsarpalis commented on Oct 30, 2020

ygoe commented on Oct 30, 2020

adamsitnik commented on Nov 13, 2020

adamsitnik commented on Dec 16, 2020

ygoe commented on Dec 17, 2020

ygoe commented on Oct 26, 2021

stephentoub commented on Oct 26, 2021

ygoe commented on Oct 26, 2021

adamsitnik commented on Nov 17, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions