apiStatus annotation, user-land compilation warning/errors #8820

eed3si9n · 2020-03-22T10:16:39Z

This implements apiStatus annotation as a generic form of deprecated annotation. apiStatus takes category and defaultAction parameters, corresponding to configurable warning's category and action.

One of the usage is to trigger compiler error from the library when a method is invoked to display migration message. Another usage would be to denote bincompat status of the API as warning.

This is a resend of #7790 based on the configurable warnings.
Ref #8373 / https://twitter.com/not_xuwei_k/status/1240354073297268737
Ref #7528

From Scaladoc

An annotation to denote the API status, like scala.deprecated but more powerful.
While @deprecated is only able to discourage someone from using the some API,
@apiStatus can be more nuanced about the state (for instance Category.ApiMayChange),
and choose the default compile-time actions (Action.Error, Action.Warning, etc).

In other words, this gives library authors the lever to trigger compilation warning or
compilation errors! Here's an example of displaying a migration message:

import scala.annotation.apiStatus, apiStatus._
@apiStatus(
  "method <<= is removed; use := syntax instead",
  category = Category.ForRemoval,
  since = "foo-lib 1.0",
  defaultAction = Action.Error,
)
def <<=() = ???

The compilation will fail and display the migration message if the method is called:

example.scala:26: error: method <<= is removed; use := syntax instead (foo-lib 1.0)
  <<=()
  ^

Here's another example of displaying a warning:

import scala.annotation.apiStatus, apiStatus._
@apiStatus(
  "should DSL is incubating, and future compatibility is not guaranteed",
  category = Category.ApiMayChange,
  since = "foo-lib 1.0",
)
implicit class ShouldDSL(s: String) {
  def should(o: String): Unit = ()
}

The compiler will emit warnings:

example.scala:28: warning: should DSL is incubating, and future compatibility is not guaranteed (foo-lib 1.0)
  "bar" should "something"
  ^

Using -Wconf:cat=api-may-change&origin=foo\..*:silent option, the user of the library can opt out of the api-may-change warnings afterwards.

Defining a custom status annotation

Instead of directly using @apiStatus we can create a specific status annotation by extending apiStatus. However, due to the information available to the compiler, the default values for message, category, or defaultAction can be specified through the corresponding meta-annotations.

import scala.annotation.{ apiStatus, apiStatusCategory, apiStatusDefaultAction }
import scala.annotation.meta._

@apiStatusCategory(apiStatus.Category.ApiMayChange)
@apiStatusDefaultAction(apiStatus.Action.Warning)
@companionClass @companionMethod
final class apiMayChange(
  message: String,
  since: String = "",
) extends apiStatus(message, since = since)

This can be used as follows:

@apiMayChange("should DSL is incubating, and future compatibility is not guaranteed")
implicit class ShouldDSL(s: String) {
  def should(o: String): Unit = ()
}

sjrd · 2020-03-22T11:58:36Z

What's wrong with @compileTimeOnly?

src/library/scala/deprecated.scala

som-snytt · 2020-03-22T16:36:51Z

In line with "deprecation" as a curse, @anathema.

eed3si9n · 2020-03-22T21:13:33Z

@sjrd

What's wrong with @compileTimeOnly?

I guess the name is wrong. If I'm removing def <<= I want to say that it's an error, typer phase or otherwise.

som-snytt · 2020-03-22T21:48:52Z

Someone claimed previously that @deprecated in Scala always means "forRemoval" (which I think is patently false, witness the gazillion deprecations since 2.10 messages, except now it says 100000000 instead of writing out gazillion). I may have PRd "forRemoval"?

Maybe it's coronavirus, sheltering in place, the general need to declutter my life, but now I want no warnings from the compiler, but better integration with tooling for diagnostics. I don't even want deprecations from the compiler, but from a linter, which can decide what is an error and what can be ignored.

It's also true that compileTimeOnly is still a compile-time dependency. If the intention is to error because forRemoval, then it doesn't suffice; the element could be used but eliminated by a macro or elidable. If the intention is just don't call me, because the implementation is ???, then an alias of compileTimeOnly would work.

lrytz

To me, #7790 always felt like addressing a mixed bag of concerns and I didn't manage to grasp "what's the goal", that's why I was concerned.

Now this PR is different, I can understand what it does and I can see how some projects would want to use it, so I'm fine with adding it.

🎨 how about adding a parameter to deprecated instead? @deprecated(message = "...", since = "...", error = true)

src/compiler/scala/tools/nsc/settings/Warnings.scala

dwijnand · 2020-03-23T13:29:04Z

@compileTimeOnly("use := syntax instead") looks good enough, without adding more copies. Perhaps all this PR needs to do is add this example to its Scaladoc?

test/files/run/settings-parse.check

eed3si9n · 2020-03-23T19:22:12Z

@lrytz

🎨 how about adding a parameter to deprecated instead? @deprecated(message = "...", since = "...", error = true)

I thought DeprecationError should be a category so if someone wants to ignore it (and deal with the consequences) they can using -Wconf.

lrytz · 2020-03-23T19:45:30Z

DeprecationError should be a category

I agree, but that could be done with the additional parameter too

eed3si9n · 2020-03-23T20:13:21Z

I agree, but that could be done with the additional parameter too

That's true. I can make Reporting#deprecationWarning do the deprecation error depending on the parameter.

src/library/scala/annotation/apiStatus.scala

lrytz · 2020-03-25T07:51:53Z

src/library/scala/annotation/apiStatus.scala

+    final val Error = "error"
+    final val Warning = "warning"
+    final val WarningSummary = "warning-summary"
+    final val WarningVerbose = "warning-verbose"


Probably we should not expose the "verbose" options here?

Fixed in 9f6413c. Also added Scaladoc in there.

He-Pin · 2020-03-26T05:40:57Z

I think it would be better annotate the apiMayChange directly not the ApiStatus

eed3si9n · 2020-03-26T18:11:23Z

@hepin1989

I think it would be better annotate the apiMayChange directly not the ApiStatus

Different library authors want different ways of emitting compiler errors and warnings. Rather than adding each piecemeal, @apiStatus provides a toolbox so they can add different categories themselves.

For example, some may opt to display ApiMayChange as a warning, while others may want to keep it silent by default, and let user opt into it as a warning. Same could be said of other more nuanced notifications such as semantics change, performance regression, or non-deterministic output.

This implements `apiStatus` annotation as a generic form of `deprecated` annotation. While deprecated is only able to discourage someone from using the some API, `apiStatus` can be more nuanced about the state (for instance Category.ApiMayChange), and choose the default compile-time actions (Action.Error, Action.Warning, etc). In other words, this gives library authors the lever to trigger compilation warning or compilation errors! One of the usage is to trigger compiler error from the library when a method is invoked to display migration message. Another usage would be to denote bincompat status of the API as warning. This is a resend of scala#7790 based on the configurable warnings. Ref scala#8373 / https://twitter.com/not_xuwei_k/status/1240354073297268737

nafg · 2020-03-29T06:24:19Z

I feel like the name apiStatus refers to particular use case rather than describing what it's the name of. Can I suggest something like compilationProblem instead? ("Problem" seems to be the term IDEs use to lump errors and warnings together, so something that can be either an error or warning is called a problem.)

eed3si9n · 2020-03-29T07:16:16Z

@nafg @compileTimeAction might be a better procedural name, since compilation problem sounds like you're talking about a bug in compiler.

@apiStatus is meant to be a metadata of the API, like it's intended for internal-use. The fact that it's internal is not a problem per se.

Library authors can extend apiStatus attribute to provide a custom status annotation. Due to the information available during compilation, the default values for `message`, `category`, or `defaultAction` are specified through the corresponding meta-annotations.

eed3si9n · 2020-04-05T06:31:47Z

Added a few meta-annotations so we can define custom annotations:

import scala.annotation.{ apiStatus, apiStatusCategory, apiStatusDefaultAction }
import scala.annotation.meta._
 
@apiStatusCategory("api-may-change")
@apiStatusDefaultAction(apiStatus.Action.Warning)
@companionClass @companionMethod
final class apiMayChange(
  message: String,
  since: String = "",
) extends apiStatus(message, since = since)

This would simplify the tagging, comparable to deprecated etc:

@apiMayChange("can DSL is incubating, and future compatibility is not guaranteed")
implicit class CanDSL(s: String) {
  def can(o: String): Unit = ()
}

dwijnand · 2020-10-14T15:10:21Z

I don't think we have the bandwidth with the time remaining, but let's try to discuss this with you for the next release.

lrytz · 2020-10-15T14:19:51Z

Before we repeat another go-around, could we decide if Scala 2.x is at all interested in having a user-land compilation errors or user-land warnings? And come up with criteria of what would make the solution merge-able?

I very much agree. For the first question, I don't know myself.

If the answer is yes, I find it difficult to know what exactly is needed / will be used in reality by the various library authors. For this reason, a very general and configurable solution like it's implemented here makes sense to me (the details should be discussed of course). The simpler proposals in the past always caused a lot of questions in both directions: why not just use existing feature X? Why doesn't it support use case Y?

lrytz · 2020-12-18T14:21:08Z

@eed3si9n I played a bit around with new meta-annotations for defaults which would allow changing

@apiStatusCategory(apiStatus.Category.ApiMayChange)
@apiStatusDefaultAction(apiStatus.Action.Warning)
@companionClass @companionMethod
final class apiMayChange(
  message: String,
  since: String = "",
) extends apiStatus(message, since = since)

to

@companionClass @companionMethod
final class apiMayChange(
  message: String,
  since: String = "",
) extends apiStatus(
  message,
  category = apiStatus.Category.ApiMayChange,
  since = since,
  defaultAction = apiStatus.Action.Warning)

See the last commit message in 2.13.x...lrytz:constAnnDefaults

eed3si9n · 2020-12-18T14:28:17Z

@companionClass @companionMethod

final class apiMayChange(

  message: String,

  since: String = "",

) extends apiStatus(

  message,

  category = apiStatus.Category.ApiMayChange,

  since = since,

  defaultAction = apiStatus.Action.Warning)

See the last commit message in 2.13.x...lrytz:constAnnDefaults

That's great. Much more straightforward.

… adding annotations to classes on the classpath. External annotations can be used for low-overhead API-level linting by adding deprecations and potentially making them fatal using `-Wconf`. The propsed [`@apiStatus` annotation](scala#8820) which generalizes deprecations would also be a good match for external annotations. There is overlap between external annotations and existing Scala linters like [https://github.com/scalacenter/scalafix](scalafix) and [wartremover](https://github.com/wartremover/wartremover). The advantage of external annotations is the low overhead (low compile-time overhead, no new tool in the toolchain). External annotations could potentially become useful for [explicit null checking in Scala 3](http://dotty.epfl.ch/docs/internals/explicit-nulls.html). There are two new compiler flags: - `-Yexternal-annotation-files annots.txt:more-annots.txt` enables reading external annotations from specific files - `-Yexternal-annotations` enables reading external annotations from jars / directories on the classpath, it scans each classpath entry for a file `/external-annotations.txt`. This means that external annotations can be shared across projects by publishing them to a repository and adding them as `libraryDependencies`. External annotations are added to symbols at the moment their lazy type is completed. This makes external annotations available already in the type checking phase, but does not cause symbol infos to be forced eagerly. An alternative implementation strategy is to add external annotations in a separate phase after type checking. This would make external annotations only visible after type checking, which would be OK for deprecations. This strategy could be implemented in a compiler plugin. Syntax for external annotations: ``` @path.to.annotation @another.annot path.to.Class.method some.Class @scala.deprecated("don't use ???", "2.13.0") scala.Predef$.??? @scala.deprecated("no mutable collections") scala.collection.mutable._ ``` Details: - All paths need to be fully qualified - Annotation arguments need to be constants - `lwoercase` and symbolic names (`???`) refer to term symbols - `Uppercase` names refer to type symbols - A trailing `$` changes the name into a term (`Predef$`) - A trailing `#` makes a type name (``scala.collection.immutable.`::`#``) - Backticks can be used (``scala.sys.process.ProcessBuilder.`###` ``) - A wildcard `_` applies to all symbols that share the parent - Packages can't be annotated - For overloaded methods, annotations are applied to all alternatives (for now). Issues with external annotation files (e.g. syntax errors, unknown annotations, non-constant annotation arguments) are reported as warnings with category `other-external-annotations` (not as errors).

This change adds support for defining external annotations, i.e., for adding annotations to classes on the classpath. External annotations can be used for low-overhead API-level linting by adding deprecations and potentially making them fatal using `-Wconf`. The propsed [`@apiStatus` annotation](scala#8820) which generalizes deprecations would also be a good match for external annotations. There is overlap between external annotations and existing Scala linters like [https://github.com/scalacenter/scalafix](scalafix) and [wartremover](https://github.com/wartremover/wartremover). The advantage of external annotations is the low overhead (low compile-time overhead, no new tool in the toolchain). External annotations could potentially become useful for [explicit null checking in Scala 3](http://dotty.epfl.ch/docs/internals/explicit-nulls.html). There are two new compiler flags: - `-Yexternal-annotation-files annots.txt:more-annots.txt` enables reading external annotations from specific files - `-Yexternal-annotations` enables reading external annotations from jars / directories on the classpath, it scans each classpath entry for a file `/external-annotations.txt`. This means that external annotations can be shared across projects by publishing them to a repository and adding them as `libraryDependencies`. External annotations are added to symbols at the moment their lazy type is completed. This makes external annotations available already in the type checking phase, but does not cause symbol infos to be forced eagerly. An alternative implementation strategy is to add external annotations in a separate phase after type checking. This would make external annotations only visible after type checking, which would be OK for deprecations. This strategy could be implemented in a compiler plugin. Syntax for external annotations: ``` @path.to.annotation @another.annot path.to.Class.method some.Class @scala.deprecated("don't use ???", "2.13.0") scala.Predef$.??? @scala.deprecated("no mutable collections") scala.collection.mutable._ ``` Details: - All paths need to be fully qualified - Members need to be selected from the class in which they are defined, inheritance is not supported - Annotation arguments need to be constants - `lwoercase` and symbolic names (`???`) refer to term symbols - `Uppercase` names refer to type symbols - A trailing `$` changes the name into a term (`Predef$`) - A trailing `#` makes a type name (``scala.collection.immutable.`::`#``) - Backticks can be used (``scala.sys.process.ProcessBuilder.`###` ``) - A wildcard `_` applies to all symbols that share the parent - Packages can't be annotated - For overloaded methods, annotations are applied to all alternatives (for now). Issues with external annotation files (e.g. syntax errors, unknown annotations, non-constant annotation arguments) are reported as warnings with category `other-external-annotations` (not as errors).

This change adds support for defining external annotations, i.e., for adding annotations to classes on the classpath. ### Use case External annotations can be used for low-overhead API-level linting by adding deprecations and potentially making them fatal using `-Wconf`. The propsed [`@apiStatus` annotation](scala#8820) which generalizes deprecations would also be a good match for external annotations. There is overlap between external annotations and existing Scala linters like [https://github.com/scalacenter/scalafix](scalafix) and [wartremover](https://github.com/wartremover/wartremover). The advantage of external annotations is the low overhead (low compile-time overhead, no new tool in the toolchain). External annotations could potentially become useful for [explicit null checking in Scala 3](http://dotty.epfl.ch/docs/internals/explicit-nulls.html). ### Details There are two new compiler flags: - `-Yexternal-annotation-files annots.txt:more-annots.txt` enables reading external annotations from specific files - `-Yexternal-annotations` enables reading external annotations from jars / directories on the classpath, it scans each classpath entry for a file `/external-annotations.txt`. This means that external annotations can be shared across projects by publishing them to a repository and adding them as `libraryDependencies`. External annotations are added to symbols at the moment their lazy type is completed. This makes external annotations available already in the type checking phase, but does not cause symbol infos to be forced eagerly. An alternative implementation strategy is to add external annotations in a separate phase after type checking. This would make external annotations only visible after type checking, which would be OK for deprecations. This strategy could be implemented in a compiler plugin. Syntax for external annotations: ``` @path.to.annotation @another.annot path.to.Class.method some.Class @scala.deprecated("don't use ???", "2.13.0") scala.Predef$.??? @scala.deprecated("no mutable collections") scala.collection.mutable._ ``` Details: - All paths need to be fully qualified - Members need to be selected from the class in which they are defined, inheritance is not supported - Annotation arguments need to be constants - `lwoercase` and symbolic names (`???`) refer to term symbols - `Uppercase` names refer to type symbols - A trailing `$` changes the name into a term (`Predef$`) - A trailing `#` makes a type name (``scala.collection.immutable.`::`#``) - Backticks can be used (``scala.sys.process.ProcessBuilder.`###` ``) - A wildcard `_` applies to all symbols that share the parent - Packages can't be annotated - For overloaded methods, annotations are applied to all alternatives (for now). Issues with external annotation files (e.g. syntax errors, unknown annotations, non-constant annotation arguments) are reported as warnings with category `other-external-annotations` (not as errors).

… adding annotations to classes on the classpath. ### Use case External annotations can be used for low-overhead API-level linting by adding deprecations and potentially making them fatal using `-Wconf`. The propsed [`@apiStatus` annotation](scala#8820) which generalizes deprecations would also be a good match for external annotations. There is overlap between external annotations and existing Scala linters like [scalafix](https://github.com/scalacenter/scalafix) and [wartremover](https://github.com/wartremover/wartremover). The advantage of external annotations is the low overhead (low compile-time overhead, no new tool in the toolchain) and simplicity (writing wartremover or scalafix uses non-trivial APIs). External annotations could potentially become useful for [explicit null checking in Scala 3](http://dotty.epfl.ch/docs/internals/explicit-nulls.html). ### Details There are two new compiler flags: - `-Yexternal-annotation-files annots.txt:more-annots.txt` enables reading external annotations from specific files - `-Yexternal-annotations` enables reading external annotations from jars / directories on the classpath. Each each classpath entry is scanned for the file `/external-annotations.txt`. External annotations can be shared across projects by publishing them to a repository and adding them as `libraryDependencies`. External annotations are added to symbols at the moment their lazy type is completed. This makes external annotations available already in the type checking phase, but does not cause symbol infos to be forced eagerly. An alternative implementation strategy is to add external annotations in a separate phase after type checking. This would make external annotations only visible after type checking, which would be OK for deprecations (they are checked in the refchecks phase). This strategy could be implemented as a compiler plugin. Syntax for external annotations: ``` @path.to.annotation @another.annot path.to.Class.method some.Class @scala.deprecated("don't use ???", "2.13.0") scala.Predef$.??? @scala.deprecated("no mutable collections") scala.collection.mutable._ ``` Details: - All paths need to be fully qualified - Members need to be selected from the class in which they are defined (don't select inherited members) - Annotation arguments need to be constants - `lwoercase` and symbolic names (`???`) refer to term symbols - `Uppercase` names refer to type symbols - A trailing `$` changes the name into a term (`Predef$`) - A trailing `#` makes a type name (``scala.collection.immutable.`::`#``) - Backticks can be used for symbolic names (``scala.sys.process.ProcessBuilder.`###` ``) - A wildcard `_` applies to all symbols that share a parent - Packages can't be annotated - For overloaded methods, annotations are applied to all alternatives (for now) Issues with external annotation files (e.g. syntax errors, unknown annotations, non-constant annotation arguments) are reported as warnings with category `other-external-annotations` (not as errors).

This change adds support for defining external annotations, i.e., for adding annotations to classes on the classpath. ### Use case External annotations can be used for low-overhead API-level linting by adding deprecations and potentially making them fatal using `-Wconf`. The propsed [`@apiStatus` annotation](scala#8820) which generalizes deprecations would also be a good match for external annotations. There is overlap between external annotations and existing Scala linters like [scalafix](https://github.com/scalacenter/scalafix) and [wartremover](https://github.com/wartremover/wartremover). The advantage of external annotations is the low overhead (low compile-time overhead, no new tool in the toolchain) and simplicity (writing wartremover or scalafix uses non-trivial APIs). External annotations could potentially become useful for [explicit null checking in Scala 3](http://dotty.epfl.ch/docs/internals/explicit-nulls.html). ### Details There are two new compiler flags: - `-Yexternal-annotation-files annots.txt:more-annots.txt` enables reading external annotations from specific files - `-Yexternal-annotations` enables reading external annotations from jars / directories on the classpath. Each each classpath entry is scanned for the file `/external-annotations.txt`. External annotations can be shared across projects by publishing them to a repository and adding them as `libraryDependencies`. External annotations are added to symbols at the moment their lazy type is completed. This makes external annotations available already in the type checking phase, but does not cause symbol infos to be forced eagerly. An alternative implementation strategy is to add external annotations in a separate phase after type checking. This would make external annotations only visible after type checking, which would be OK for deprecations (they are checked in the refchecks phase). This strategy could be implemented as a compiler plugin. Syntax for external annotations: ``` @path.to.annotation @another.annot path.to.Class.method some.Class @scala.deprecated("don't use ???", "2.13.0") scala.Predef$.??? @scala.deprecated("no mutable collections") scala.collection.mutable._ ``` Details: - All paths need to be fully qualified - Members need to be selected from the class in which they are defined (don't select inherited members) - Annotation arguments need to be constants - `lwoercase` and symbolic names (`???`) refer to term symbols - `Uppercase` names refer to type symbols - A trailing `$` changes the name into a term (`Predef$`) - A trailing `#` makes a type name (``scala.collection.immutable.`::`#``) - Backticks can be used for symbolic names (``scala.sys.process.ProcessBuilder.`###` ``) - A wildcard `_` applies to all symbols that share a parent - Packages can't be annotated - For overloaded methods, annotations are applied to all alternatives (for now) Issues with external annotation files (e.g. syntax errors, unknown annotations, non-constant annotation arguments) are reported as warnings with category `other-external-annotations` (not as errors). # Please enter the commit message for your changes. Lines starting # with '#' will be kept; you may remove them yourself if you want to. # An empty message aborts the commit. # # Date: Wed Jan 6 16:30:34 2021 +0100 # # On branch extAnn # Your branch is up to date with 'origin/extAnn'. # # Changes to be committed: # new file: ../src/compiler/scala/tools/nsc/ExternalAnnotations.scala # modified: ../src/compiler/scala/tools/nsc/Global.scala # modified: ../src/compiler/scala/tools/nsc/Reporting.scala # modified: ../src/compiler/scala/tools/nsc/settings/ScalaSettings.scala # modified: ../src/compiler/scala/tools/nsc/symtab/SymbolLoaders.scala # modified: ../src/reflect/scala/reflect/internal/Symbols.scala # modified: ../src/reflect/scala/reflect/internal/pickling/UnPickler.scala # modified: ../src/reflect/scala/reflect/runtime/SymbolLoaders.scala # new file: ../test/files/run/external-annotations.check # new file: ../test/files/run/external-annotations/Test.scala # new file: ../test/files/run/external-annotations/annots.txt # new file: ../test/junit/scala/tools/nsc/ExternalAnnotationsTest.scala #

This change adds support for defining external annotations, i.e., for adding annotations to classes on the classpath. ### Use case External annotations can be used for low-overhead API-level linting by adding deprecations and potentially making them fatal using `-Wconf`. The propsed [`@apiStatus` annotation](scala#8820) which generalizes deprecations would also be a good match for external annotations. There is overlap between external annotations and existing Scala linters like [scalafix](https://github.com/scalacenter/scalafix) and [wartremover](https://github.com/wartremover/wartremover). The advantage of external annotations is the low overhead (low compile-time overhead, no new tool in the toolchain) and simplicity (writing wartremover or scalafix uses non-trivial APIs). External annotations could potentially become useful for [explicit null checking in Scala 3](http://dotty.epfl.ch/docs/internals/explicit-nulls.html). ### Details There are two new compiler flags: - `-Yexternal-annotation-files annots.txt:more-annots.txt` enables reading external annotations from specific files - `-Yexternal-annotations` enables reading external annotations from jars / directories on the classpath. Each each classpath entry is scanned for the file `/external-annotations.txt`. External annotations can be shared across projects by publishing them to a repository and adding them as `libraryDependencies`. External annotations are added to symbols at the moment their lazy type is completed. This makes external annotations available already in the type checking phase, but does not cause symbol infos to be forced eagerly. An alternative implementation strategy is to add external annotations in a separate phase after type checking. This would make external annotations only visible after type checking, which would be OK for deprecations (they are checked in the refchecks phase). This strategy could be implemented as a compiler plugin. Syntax for external annotations: ``` @path.to.annotation @another.annot path.to.Class.method some.Class @scala.deprecated("don't use ???", "2.13.0") scala.Predef$.??? @scala.deprecated("no mutable collections") scala.collection.mutable._ ``` Details: - All paths need to be fully qualified - Members need to be selected from the class in which they are defined (don't select inherited members) - Annotation arguments need to be constants - `lwoercase` and symbolic names (`???`) refer to term symbols - `Uppercase` names refer to type symbols - A trailing `$` changes the name into a term (`Predef$`) - A trailing `#` makes a type name (``scala.collection.immutable.`::`#``) - Backticks can be used for symbolic names (``scala.sys.process.ProcessBuilder.`###` ``) - A wildcard `_` applies to all symbols that share a parent - Packages can't be annotated - For overloaded methods, annotations are applied to all alternatives (for now) Issues with external annotation files (e.g. syntax errors, unknown annotations, non-constant annotation arguments) are reported as warnings with category `other-external-annotations` (not as errors).

SethTisue · 2021-04-17T15:21:41Z

relevant: Scala 3 appears set to add @experimental: scala/scala3#12102

update: it's been merged

This change adds support for defining external annotations, i.e., for adding annotations to classes on the classpath. ### Use case External annotations can be used for low-overhead API-level linting by adding deprecations and potentially making them fatal using `-Wconf`. The propsed [`@apiStatus` annotation](scala#8820) which generalizes deprecations would also be a good match for external annotations. There is overlap between external annotations and existing Scala linters like [scalafix](https://github.com/scalacenter/scalafix) and [wartremover](https://github.com/wartremover/wartremover). The advantage of external annotations is the low overhead (low compile-time overhead, no new tool in the toolchain) and simplicity (writing wartremover or scalafix uses non-trivial APIs). External annotations could potentially become useful for [explicit null checking in Scala 3](http://dotty.epfl.ch/docs/internals/explicit-nulls.html). ### Details There are two new compiler flags: - `-Yexternal-annotation-files annots.txt:more-annots.txt` enables reading external annotations from specific files - `-Yexternal-annotations` enables reading external annotations from jars / directories on the classpath. Each each classpath entry is scanned for the file `/external-annotations.txt`. External annotations can be shared across projects by publishing them to a repository and adding them as `libraryDependencies`. External annotations are added to symbols at the moment their lazy type is completed. This makes external annotations available already in the type checking phase, but does not cause symbol infos to be forced eagerly. An alternative implementation strategy is to add external annotations in a separate phase after type checking. This would make external annotations only visible after type checking, which would be OK for deprecations (they are checked in the refchecks phase). This strategy could be implemented as a compiler plugin. Syntax for external annotations: ``` @path.to.annotation @another.annot path.to.Class.method some.Class @scala.deprecated("don't use ???", "2.13.0") scala.Predef$.??? @scala.deprecated("no mutable collections") scala.collection.mutable._ ``` Details: - All paths need to be fully qualified - Members need to be selected from the class in which they are defined (don't select inherited members) - Annotation arguments need to be constants - `lwoercase` and symbolic names (`???`) refer to term symbols - `Uppercase` names refer to type symbols - A trailing `$` changes the name into a term (`Predef$`) - A trailing `#` makes a type name (``scala.collection.immutable.`::`#``) - Backticks can be used for symbolic names (``scala.sys.process.ProcessBuilder.`###` ``) - A wildcard `_` applies to all symbols that share a parent - Packages can't be annotated - For overloaded methods, annotations are applied to all alternatives (for now) Issues with external annotation files (e.g. syntax errors, unknown annotations, non-constant annotation arguments) are reported as warnings with category `other-external-annotations` (not as errors).

SethTisue · 2022-04-20T16:13:09Z

It remains unclear whether this is ever going to progress. Closing to get it out of the PR queue, at least for the time being.

eed3si9n requested a review from lrytz March 22, 2020 10:16

scala-jenkins added this to the 2.13.3 milestone Mar 22, 2020

som-snytt reviewed Mar 22, 2020

View reviewed changes

src/library/scala/deprecated.scala Outdated Show resolved Hide resolved

lrytz reviewed Mar 23, 2020

View reviewed changes

src/compiler/scala/tools/nsc/settings/Warnings.scala Outdated Show resolved Hide resolved

dwijnand reviewed Mar 23, 2020

View reviewed changes

test/files/run/settings-parse.check Outdated Show resolved Hide resolved

lrytz reviewed Mar 25, 2020

View reviewed changes

src/library/scala/annotation/apiStatus.scala Outdated Show resolved Hide resolved

lrytz reviewed Mar 25, 2020

View reviewed changes

eed3si9n force-pushed the wip/restligeist-returns branch 2 times, most recently from 9f6413c to c9bef7b Compare March 26, 2020 01:29

eed3si9n force-pushed the wip/restligeist-returns branch from c9bef7b to f9de842 Compare March 26, 2020 16:19

eed3si9n changed the title ~~deprecatedError annotation, take 2~~ apiStatus annotation Mar 29, 2020

eed3si9n force-pushed the wip/restligeist-returns branch from f9de842 to a0f20e8 Compare March 29, 2020 02:56

eed3si9n changed the title ~~apiStatus annotation~~ apiStatus annotation, user-land compilation warning/errors Mar 29, 2020

eed3si9n force-pushed the wip/restligeist-returns branch from e885c91 to 5167e79 Compare April 5, 2020 04:13

eed3si9n force-pushed the wip/restligeist-returns branch from 5167e79 to 9bf3a32 Compare April 5, 2020 05:29

dwijnand added this to the 2.13.5 milestone Oct 14, 2020

lrytz mentioned this pull request Oct 28, 2020

Revert ExecutionContext.global to not be a BatchingExecutor #9270

Merged

SethTisue marked this pull request as draft November 22, 2020 18:54

lrytz mentioned this pull request Jan 21, 2021

Support for external annotations #9447

Closed

5 tasks

SethTisue modified the milestones: 2.13.5, 2.13.6 Feb 9, 2021

SethTisue added the release-notes worth highlighting in next release notes label Apr 8, 2021

dwijnand modified the milestones: 2.13.6, 2.13.7 May 10, 2021

SethTisue modified the milestones: 2.13.7, 2.13.8 Oct 26, 2021

lrytz modified the milestones: 2.13.8, 2.13.9 Nov 18, 2021

SethTisue closed this Apr 20, 2022

SethTisue removed this from the 2.13.9 milestone Apr 20, 2022

lrytz mentioned this pull request Nov 22, 2024

Fix AnnotationInfo when using defaults scala/scala-dev#884

Closed

apiStatus annotation, user-land compilation warning/errors #8820

apiStatus annotation, user-land compilation warning/errors #8820

Uh oh!

Conversation

eed3si9n commented Mar 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

From Scaladoc

Defining a custom status annotation

Uh oh!

sjrd commented Mar 22, 2020

Uh oh!

Uh oh!

som-snytt commented Mar 22, 2020

Uh oh!

eed3si9n commented Mar 22, 2020

Uh oh!

som-snytt commented Mar 22, 2020

Uh oh!

lrytz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dwijnand commented Mar 23, 2020

Uh oh!

Uh oh!

eed3si9n commented Mar 23, 2020

Uh oh!

lrytz commented Mar 23, 2020

Uh oh!

eed3si9n commented Mar 23, 2020

Uh oh!

Uh oh!

lrytz Mar 25, 2020

Choose a reason for hiding this comment

Uh oh!

eed3si9n Mar 25, 2020

Choose a reason for hiding this comment

Uh oh!

eed3si9n Mar 26, 2020

Choose a reason for hiding this comment

Uh oh!

He-Pin commented Mar 26, 2020

Uh oh!

eed3si9n commented Mar 26, 2020

Uh oh!

nafg commented Mar 29, 2020

Uh oh!

eed3si9n commented Mar 29, 2020

Uh oh!

eed3si9n commented Apr 5, 2020

Uh oh!

dwijnand commented Oct 14, 2020

Uh oh!

lrytz commented Oct 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lrytz commented Dec 18, 2020

Uh oh!

eed3si9n commented Dec 18, 2020

Uh oh!

SethTisue commented Apr 17, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SethTisue commented Apr 20, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

eed3si9n commented Mar 22, 2020 •

edited

Loading

lrytz commented Oct 15, 2020 •

edited

Loading

SethTisue commented Apr 17, 2021 •

edited

Loading