Should they just SET it? or should they be correctly determining if they are being run in a CI context using some mechanism (presumably some mechanism that can be shared).

The point that I'm trying to get across to users is that, in order for logging to prepend the vso logging headers (or whatever we want to call them), $ci must be $true. How they set that/determine that they are in a $ci environment is up to the user.

Historically, it's a flag that gets passed to build.cmd (or build.ps1). ie, in CI you'd say "build.cmd -ci" and on your local dev box you'd just say "build.cmd". Of course, locally you could still use "build.cmd -ci". The thinking (I believe) being that adding "ci" changes behavior and we want to be very intentional about when behavior changes vs just having a thing that magically gets set and a dev not understanding why running the exact same command locally vs in ci causes different behavior. I think that there's competing desires at play, 1) to have a thing that just does the right thing always (regardless of context, and 2) to have a thing that is understandable (non-magical). This also means that devs working on Arcade need to have the same mindset and be aware of the "ci" flag.

This documention feels misleading. To me, as someone who doesn't know better, this documentation just says to ALWAYS set it to true. It doesn't say "set it to true in the correct circumstances", it just says set it to true if I want correct behavior.

Set it to false if I don't want proper reporting? Why would I not want proper behavior? The contrapositive doesn't make any sense here, so the documentation is confusing.

It feels like this documention only helps if you already know the answer. To me, as an ignorant person, this is very confusing, and would probably cause me to just add $ci = $True #do proper reporting to the top of every powershell file I wrote.

Nobody should have to set $ci = true. If they're following Arcade guidance (defined elsewhere), then this is transparent and just works. They use the proscribed entry point of build.cmd, and things are fine. Some times people (mostly us), write scripts that use Arcade stuff, but separate from the traditional Arcade system. ie, Arcade was originally intended for our customers (corefx, runtime, aspnet, roslyn, sdk, fsharp, etc...), we find some of the pieces useful (like telemetry), and tend to grab just those pieces and it takes extra care to make sure that behavior is still preserved. This doc was intended for repos adding telemetry to their existing Arcade builds and to help them transition to adding telemetry to their builds (for CI council). For that audience, a valid Arcade build is the pre-req and things should work.

So the point of this comment was that, when writing new scripts that will run in ci, but not through build.ps1 or whatever, something, somewhere needs to set $ci to true, whether that be a switch to the script being written, or baked in the script or what have you. See: https://github.com/dotnet/arcade/blob/main/eng/common/sdk-task.ps1#L13, as an example (there are more). sdk-task.ps1 isn't run in the build, but it uses the logging functions. So it sets $ci to true in the script. This also came up in arcade-validation where a script was using Write-PipelineTelemetryError, but no error was being reported, because nothing was setting $ci to true. The point of this guidance is about enabling logging in ci for scripts that want to use the logging functions, which is our guidance to folks who are writing scripts (use these functions we provide to you). Not everyone can be assumed to know that these logging functions only exhibit the expected behavior if $ci is true (not even people on our team), and therefore they need to make sure that they get that set to true. Hence this doc update.

Not everyone can be assumed to know that these logging functions only exhibit the expected behavior if $ci is true (not even people on our team), and therefore they need to make sure that they get that set to true. Hence this doc update.

Agreed. Let's document our tribal knowledge because folks who are here aren't going to remember everything, or even have interacted with Arcade enough to know this. If we get new folks on the team (or new customers who want to use this functionality), we'll need to share this information with them, and easier to have it written down than try to remember and impart tribal knowledge to them.

Just to recap... I have no problem with providing more information and I've already approved this PR. But... I do think there's some remaining confusion

This doc was intended for Arcade customer's, not for the engineering team or others trying to just use the Telemetry piece for some other purpose.

The scenarios you described are very clearly not the mainstream scenarios for Arcade and are instances where Telemetry was forced on to another thing.

I feel like this PR, and most of the motivation, are because of some perceived tribal knowledge that is missing and needs to be added to this doc. To Chad's point, adding the comment here, kind of muddies the waters because an audience (not the intended audience) is providing additional information but without additional context, and that leads one to believe that you should always add ci = true. Perhaps a link to the note about build.cmd vs cibuild.cmd would help? That would at least highlight that a pre-req to updating the Arcade scripts, is understanding Arcade's principles (I don't intend that as a judgement or any negative sentiment towards anybody). Either that, or a different doc entirely, or maybe even just a big note at the top of the telemetry scripts.

Regardless, I'm ok with the comment being added to the doc, and we can delete it if it causes confusion.

blah blah blah anyways. Thanks for the effort to keep the docs updated!