Skip to content

Conversation

@ScalaWilliam
Copy link

@ScalaWilliam ScalaWilliam commented Oct 9, 2018

This is in follow up to the excellent PR by @eed3si9n at #7007.


The operator is common throughout many programming languages and is a familiar syntax to many, especially those coming from Bash and F#.

I modified the test examples, so that the order of function application is completely unambiguous. *6 and math.abs in this particular case make it which way round the functions are applied, so I replaced it with a simplified test case that makes it clearer.

This is the one symbol that is good to have as a symbol. It is very special.

Inspired by @eed3si9n's comment here: https://github.com/scala/scala/pull/7007/files#r222332538

Multi-line:
What I especially enjoyed in F# days, and also love with Bash, is being able to multi-line it:

input
  |> foo
  |> bar
  |> baz

This would sadly yield a compile error but I'm sure such syntax could be added eventually.

@Ichoran
Copy link
Contributor

Ichoran commented Oct 9, 2018

Why do we want to have |> but not pipe rather than having |> be the symbolic alias for pipe?

@dwijnand
Copy link
Member

dwijnand commented Oct 9, 2018

I'm 👍, but I wouldn't be against keeping pipe, especially if that's the requirement for having this.

* `f` to this value.
*/
def pipe[B](f: A => B): B = f(self)
def |>[B](f: A => B): B = f(self)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO if |> will be introduced it has to be an alias.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elixir has the |>:) too

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, just add an alias.

.pipe(times6)
.pipe(scala.math.abs)
val plus3 = (_: Int) + 3
val result = (1 - 2) |> plus3 |> scala.math.abs
Copy link

@martin-g martin-g Oct 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the user doesn't like postfix syntax ?
You may want to add an import: import scala.languageFeature.postfixOps

I think pipe() should be there for us who do not like ascii art in our code.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will it look on multiple lines?

    val result = (1 - 2 - 3)
      |> times6
      |> scala.math.abs

Does that parse ok? Or would it need surrounding parentheses?

If not, I'm not sure these:

    val result = ((1 - 2 - 3)
      |> times6
      |> scala.math.abs)

    val result = (1 - 2 - 3)
      .|>(times6)
      .|>(scala.math.abs)

are nicer than with .pipe

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the user doesn't like postfix syntax ?

@martin-g eh? there isn't any postfix syntax here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing you meant infix, rather than postfix.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately it does not parse, but like in the other languages, I think it should. But that shall be handled as a separate matter.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd just write

val result = (
  (1 - 2 - 3)
    |> times6
    |> scala.math.abs
)

but I know there are people who think that looks ugly.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would too, if it compiled :-)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome to Scala 2.13.0-20181012-195043-19fa66d (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_161).
Type in expressions for evaluation. Or try :help.

scala> import scala.util.chaining._
import scala.util.chaining._

scala> def times6(i: Int) = i*6
times6: (i: Int)Int

scala> val result = (
     |   (1 - 2 - 3)
     |     |> times6
     |     |> scala.math.abs
     | )
result: Int = 24

it does! :-)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, did the brackets do the trick? Excellent. Will change.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, can't infer a semi-colon within parentheses.

@japgolly
Copy link
Contributor

japgolly commented Oct 9, 2018

I suggest marking this (and pipe) as @inline

@eed3si9n
Copy link
Member

eed3si9n commented Oct 9, 2018

OCaml

Composition operators

val (|>) : 'a -> ('a -> 'b) -> 'b

Reverse-application operator: x |> f |> g is exactly equivalent to g (f (x)). Left-associative operator at precedence level 4/11.

F#

Function Composition and Pipelining:

Pipelining enables function calls to be chained together as successive operations. Pipelining works as follows:

let result = 100 |> function1 |> function2

Elixir

The pipe operator

iex> total_sum = 1..100_000 |> Enum.map(&(&1 * 3)) |> Enum.filter(odd?) |> Enum.sum
7500000000

The |> symbol used in the snippet above is the pipe operator: it takes the output from the expression on its left side and passes it as the first argument to the function call on its right side. It’s similar to the Unix | operator. Its purpose is to highlight the data being transformed by a series of functions. To see how it can make the code cleaner, have a look at the example above rewritten without using the |> operator:

iex> Enum.sum(Enum.filter(Enum.map(1..100_000, &(&1 * 3)), odd?))
7500000000

ECMAScript proposal

ESNext Proposal: The Pipeline Operator

Swift library

Swift Data Pipelines

R library

R for Data Science: 18 Pipes

The pipe, %>%, comes from the magrittr package by Stefan Milton Bache. Packages in the tidyverse load %>% for you automatically, so you don’t usually load magrittr explicitly.

@eed3si9n
Copy link
Member

eed3si9n commented Oct 9, 2018

Here's my original reason for calling it pipe instead of |>: #6767 (comment)

@SethTisue
Copy link
Member

I'm on the fence about this.

The thing is, any one symbolic operator considered in isolation looks appealing. The |> examples look nice, it's no problem... or so it's tempting to think.

But the problem is symbol overload in the language, standard library, and ecosystem as a whole.

@RawToast
Copy link

RawToast commented Oct 10, 2018

I'd be happy for both pipe and |> to exist; even though I can't see myself ever using pipe 😸

I agree that any new operators should be challenged and this is good to see. I really dislike :\ and :/, Those unique or obscure are the sort of operators that should generally be avoided.

However, |> is different to those two as it is common operator for a simple task used by a number of popular languages.

I'll mention ReasonML again, the multi-lining comment in the OP is exactly how I use pipe.

Functions ReasonML

The reverse-application operator (|>)
The operator |> is called reverse-application operator or pipe operator. It lets you chain function calls: x |> f is the same as f(x). That may not look like much, but it is quite useful when combining function calls.

Example: piping ints and strings
Let’s start with a simple example. Given the following two functions.

let times2 = (x: int) => x * 2;
let twice = (s: string) => s ++
If we use them with traditional function calls, we get:

// twice(string_of_int(times2(4)));

  • : string = "88"
    First we apply times2 to 4, then string_of_int (a function in the standard library) to the result, etc. The pipe operator lets us write code that is closer to the description that I have just given:

let result = 4 |> times2 |> string_of_int |> twice;

@joroKr21
Copy link
Member

Sometimes symbolic operators like that look nice on paper but quickly fall flat on their face.

Consider a case where we reach a type inference limitation and have to supply the type parameter explicitly:

error |> [Either[String, Int](Left)

yuck

What about formatting a long chain of pipe calls across many lines:

foo
  .|>(bar)
  .|>(baz)
  ...
  .getTheGoodStuff

not cool

What if we want to use case function syntax (and we also have to supply the type parameter and we also want to format):

foo
  .|>[Either[String, Int]] {
    case Bar => Left("error")
    case Baz => Right(42)
  }.|>(...)

This reads like a soup of symbols to me.

@RawToast
Copy link

foo
  .|>(bar)
  .|>(baz)
  ...
  .getTheGoodStuff

If you want to use dots/braces use pipe (hence it would be nice to offer both). If not use the operator and it looks fine:

foo
  |> bar
  |> baz
  ...
  |> getTheThing
foo
  .|>[Either[String, Int]] {
    case Bar => Left("error")
    case Baz => Right(42)
  }.|>(...)
This reads like a soup of symbols to me.

This would be written as something like:

val either42Function: Foo => [Either[String, Int]] =  {
    case Bar => Left("error")
    case Baz => Right(42)
  }

foo |> either42Function

@dwijnand
Copy link
Member

Btw, whatever the end result, thank you @ScalaWilliam for leading the effort here.

@martijnhoekstra
Copy link
Contributor

+1 for having |> as an alias to pipe

@He-Pin
Copy link
Contributor

He-Pin commented Oct 10, 2018

I think we should keep pipe and add |> as an alias.

@cunchen
Copy link

cunchen commented Oct 10, 2018

+1 for having |> as an alias to pipe

@ScalaWilliam
Copy link
Author

I'll make it an alias.

@ScalaWilliam
Copy link
Author

@japgolly if I use @inline, does it mean I could remove the overhead comment in the scaladoc?
cc @eed3si9n

@ScalaWilliam ScalaWilliam changed the title Replace 'pipe' with the pipe operator |> in ChainingOps Include pipe operator '|>' in ChainingOps Oct 10, 2018
@ScalaWilliam
Copy link
Author

Applied the change.

@eed3si9n
Copy link
Member

@ScalaWilliam I don't think you can remove the Scaladoc comment with just @inline.
My original PR used macro to inline the function call - b7f6955

@ScalaWilliam
Copy link
Author

Right. Thanks for clarifying.

@ScalaWilliam
Copy link
Author

I'm trying these out on a real codebase... and noticed just how similar pipe is to match, except for the case statement.

@Ichoran
Copy link
Contributor

Ichoran commented Oct 10, 2018

Yes, hence my comment somewhere or other that we should just allow .match, and for it not to require case, and then we already have pipe built in and it's fast.

@ScalaWilliam
Copy link
Author

ScalaWilliam commented Oct 10, 2018

@Ichoran that sounds interesting.

@ScalaWilliam
Copy link
Author

For multiline:

I wish we could do this (& imagine the beauty when the pipes are all aligned!):

  1 |> plusOne
    |> plusOne
    |> plusTwo
    |> plusThree

but sadly have to do this instead:

  1 |>
    plusOne |>
    plusOne |>
    plusTwo |>
    plusThree

@ScalaWilliam
Copy link
Author

For single line:

val two1 = one |> plusOne |> plusTwo |> plusThree
val two2 = one.pipe(plusOne).pipe(plusTwo).pipe(plusThree)
val two3 = one pipe plusOne pipe plusTwo pipe plusThree

The first is definitely the most obvious and easiest for scanning, the third is the worst.
The second could also be auto-formatted into a multi-liner:

val two2 = one
  .pipe(plusOne)
  .pipe(plusTwo)
  .pipe(plusThree)

@eed3si9n
Copy link
Member

Ref Pre SIP: Demote match keyword to a method.

Another what-if possibility, that might be considered Scala idiomatic is treating Id as datatype like List(...):

val two2 = Id(one)
  .map(plusOne)
  .map(plusTwo)
  .map(plusThree)

@ScalaWilliam
Copy link
Author

@eed3si9n that is very interesting. Pity that I missed all these discussions.

@viktorklang
Copy link
Contributor

@Ichoran

Yes, hence my comment somewhere or other that we should just allow .match, and for it not to require case, and then we already have pipe built in and it's fast.

Many years ago I proposed that all Functions had isDefinedAt and function literals would automatically have it return true (default method impl) which would make { case _ } a DSL which would generate both the apply and the isDefinedAt, which would mean that Any could have a method named match as in def match[T](f: MyType => T): T.

A consequence of this would be that there'd be no more PartialFunction type, and there would have to be no language support for match. It seemed nice (since any Function/method on the JVM can be partial anyway)

@RawToast
Copy link

@ScalaWilliam That's a shame about the multiline... but the example shows a solid use case for having pipe alongside |>

@nafg
Copy link
Contributor

nafg commented Oct 12, 2018

I'm trying these out on a real codebase... and noticed just how similar pipe is to match, except for the case statement.

@ScalaWilliam actually you can use pipe, |>, or anything that expects a function, with case statements. For example,

List(Some(1), None) map {
  case Some(_) => true
  case None => false
}

The resulting function will throw a MatchError if a value isn't handled, just like match.

The operator is common throughout many programming languages and is a
familiar syntax to many, especially those coming from Bash and F#.

Work done:
- Added the `|>` method to `ChainingOps`.
- Simplified the examples for `|>` so they are less ambiguous. In
particular, as `* 6` and `math.abs` are commutative, we must demonstrate
a clear order of chaining application with a non-commutative example.
- Added `@inline` to minimise performance impact.
@ScalaWilliam
Copy link
Author

Please see the updated commit. I returned the comment regarding overhead, but put it at the comment of the ChainingOps class.

/** Adds chaining methods `tap` and `pipe` to every type.
/** Adds chaining methods `tap`, `pipe` (aliased as `|>`) to every type.
*
* Note: these methods may have a small amount of overhead compared their expanded forms.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add "or the equivalent match statement".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's also a missing preposition after "compared"

@Krever
Copy link

Krever commented Oct 18, 2018

If I understand correctly, naming it map would allow us to use it in for comprehension. This may not sound like much of a benefit, but allows to skip val which could make a difference for long assignment chains...

Also it promotes the Id monad, which may make more people learn what it is and dive into FP

@ScalaWilliam
Copy link
Author

Seems it's converged now.

@hrhino
Copy link
Contributor

hrhino commented Oct 20, 2018

If I understand correctly, naming it map would allow us to use it in for comprehension.

True. But it also would mean that any value could be used in a for-comprehension, and that seems undesirable and likely to hide mistakes and typos.

Also it promotes the Id monad...

True, but the "ambient" Id monad (type Id[A] = A) is not a very good representation of the identity monad, largely because it doesn't really exist, so there's no distinguishing Id[List[A]]'s map from List[A]'s map (and in fact, you can't access the former no matter what you want to do. class Id[A](val runId: A) extends AnyVal is a far better identity monad.

@SethTisue
Copy link
Member

SethTisue commented Nov 6, 2018

@lrytz should these @inline annotations be included? not, right? ("In general, our recommendation is to avoid using these annotations", as your about-to-be-published blog post says)

@japgolly Lukas's blog post explains its recommendations very thoroughly, keep an eye out for it

@SethTisue
Copy link
Member

SethTisue commented Nov 6, 2018

to summarize the above discussion, to help final review:

  • there was discussion of whether we should still keep pipe if we add |>, the consensus was that pipe definitely stays
  • there were some tangents about match and map and Id which were interesting, but not ultimately relevant, in the end, afaict, to a thumbs up/down call here

personally, I half want to support this but I'm still wringing my mental hands about "symbol overload in the language, standard library, and ecosystem as a whole", as I said above

I'll try to find out what the rest of the team here at Lightbend thinks

@lrytz
Copy link
Member

lrytz commented Nov 7, 2018

The @inline annotations don't hurt either, and we have many of them already. So I personally don't mind. Maybe one day I'll send a PR that removes them all, last time I checked they had no impact on performance :-)

@ScalaWilliam
Copy link
Author

I don't mind removing them.

@eed3si9n
Copy link
Member

eed3si9n commented Nov 7, 2018

  • there were some tangents about match and map and Id which were interesting, but not ultimately relevant, in the end, afaict, to a thumbs up/down call here

personally, I half want to support this but I'm still wringing my mental hands about "symbol overload in the language, standard library, and ecosystem as a whole", as I said above

English method name is our default

Ultimately, it comes down to subjective taste of how code written in idiomatic Scala should feel like.

The reason why I am drawing comparison with isometric match and Id#map is because they can act as examples of how Scala code conveying the same meaning feels like.

val something = List("foo")
  .map({ case "foo" => 2; case _ => 3 })
  .map(plusTwo)
  .map(plusThree)

The above hopefully feels idiomatic to many readers, and it does not use symbolic operator to process passed in functions. We say flatMap as opposed to >>=.

when symbolic DSLs are used today

There are certain exceptions to the general English rule, and I can think of the following motivations:

  • When the language/library authors want to promote some datatype to give built-in feel.
  • When brevity aids readability.

Some examples of the first might be 'foo (Symbol), 1 :: Nil (List), (1, "foo") (tuple), 1 -> "foo" (tuple), "org" %% "name" % "1.0" (ModuleID).

The latter example might be 1.0 + 2.0 (arithmetic operations), 1 + "foo" (any2stringadd), /: (foldLeft).

Note that some of these are actively being deprecated in Scala 2.13. (could someone link to why /: is getting axed?)

what this means for |>?

For the symbolic DSL like |> to be successful, I think it's crucial that it is taught as part of the language early and often, like 1 :: Nil. Otherwise, some of us would use it while the others would be left confused.

To me the benefit of pipe or |> is when you have a big expression in the beginning, and then immediately afterwards you want to pass the result to a function, not the chaining itself.

// using pipe
(Right("foo") match {
  case Right("x") => Some("y")
  ....
  case Right(x)   => None
  case _          => Some("bar")
}) pipe (_.isDefined)

// using |>
(Right("foo") match {
  case Right("x") => Some("y")
  ....
  case Right(x)   => None
  case _          => Some("bar")
}) |> (_.isDefined)

The |> example looks cooler, but doesn't add much in terms of readability.

@SethTisue
Copy link
Member

I'm going to close this, on overall-language-surface-area-especially-when-symbolic grounds, unless someone else from the Scala team speaks up clearly in favor of merging it. going once, going twice, ...

@dwijnand
Copy link
Member

To further explain my 👎: I don't feel that strongly that we need this, but:

  • there doesn't seem to be a strong opposition to it (currently 5 👎)
  • there seems to be some support for it (currently 25 👍)

So I think on balance it should be merged.

Ideally, we would be able to merge this in some kind of non-committal way, but I don't think we really have one... We're moving away from compiler flags, experimenting in Dotty puts it too far out of reach, having it in an external library also puts it out of reach.

The other options I see are annotating it with @ApiMayChange, like Akka does and would require introducing that annotation, or putting it in some kind of experimental module that ships in the Scala distribution (something like scala-library-experimental).

Or we can just accept this one-liner, and go through the regular deprecation cycle if we change our minds later. 🙂

@dwijnand
Copy link
Member

Proposing @ApiMayChange in #7528, so if that lands we could use it here.

@Ichoran Ichoran self-assigned this Jan 10, 2019
@Ichoran
Copy link
Contributor

Ichoran commented Jan 14, 2019

@SethTisue - I'm throwing this one back to you. I don't know how to interpret the discussion, and I object to having pipe at all as something that looks like an alternative to match but in fact is a performance trap for the unwary. So I'm especially not in the frame of mind to decide on pipe alone vs. pipe with |> as an alias.

@Ichoran Ichoran assigned SethTisue and unassigned Ichoran Jan 14, 2019
@SethTisue SethTisue closed this Jan 14, 2019
@SethTisue SethTisue removed this from the 2.13.0-RC1 milestone Jan 14, 2019
@palanga
Copy link

palanga commented Jan 14, 2019

So sad

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.