Skip to content

AirSpec: A new testing framework for Scala#580

Merged
xerial merged 62 commits intowvlet:masterfrom
xerial:test-framework
Aug 9, 2019
Merged

AirSpec: A new testing framework for Scala#580
xerial merged 62 commits intowvlet:masterfrom
xerial:test-framework

Conversation

@xerial
Copy link
Copy Markdown
Member

@xerial xerial commented Aug 7, 2019 •
edited
Loading

image

Basic ideas:

  • Extends AirSpec trait to define test cases
  • Public methods in AirSpec will be test cases
  • Function arguments will be injected by Airframe DI. Session will be created/discarded per AirSpec class => configure(Design):Design, configureLocal(Design):Design

Considerations:

  • Should we run all public methods? => Yes
  • How to add a test description to each test function? => Use symbol... method names
  • Scala.js support (How to embed Surface in Scala.js, which has no runtime reflection?)
    • => Resolved: Users need to implement AirSpec.methodSurface = Surface.methodSurfaceOf[X]. We may be able to simplify this by using Scala annotation macro in future. => Added scalaJsSupport method

Tasks:

@codecov
Copy link
Copy Markdown

codecov Bot commented Aug 7, 2019 •
edited
Loading

Codecov Report

Merging #580 into master will decrease coverage by 0.16%.
The diff coverage is 63.97%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #580      +/-   ##
==========================================
- Coverage   82.95%   82.78%   -0.17%     
==========================================
  Files         226      236      +10     
  Lines        8284     8482     +198     
  Branches      608      621      +13     
==========================================
+ Hits         6872     7022     +150     
- Misses       1412     1460      +48
Impacted Files Coverage Δ
...rframe/src/main/scala/wvlet/airframe/Session.scala 92% <ø> (ø) ⬆️
...irframe/src/main/scala/wvlet/airframe/Design.scala 98.07% <ø> (ø) ⬆️
...d/src/main/scala/wvlet/airframe/AirframeSpec.scala 100% <ø> (ø)
...rame-log/jvm/src/main/scala/wvlet/log/LogEnv.scala 0% <0%> (ø) ⬆️
...ared/src/main/scala/wvlet/airframe/log/Color.scala 0% <0%> (ø)
...irframe/surface/reflect/ReflectMethodSurface.scala 83.33% <0%> (ø) ⬆️
...c/main/scala/wvlet/airframe/surface/Surfaces.scala 67.02% <0%> (-0.73%) ⬇️
...vm/src/main/scala/wvlet/airframe/spec/Compat.scala 0% <0%> (ø)
...jvm/src/main/scala/wvlet/log/LogLevelScanner.scala 96.49% <100%> (ø) ⬆️
...c/src/main/scala/wvlet/airframe/spec/package.scala 100% <100%> (ø)
... and 24 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1c6a096...c8d2c9a. Read the comment docs.

@xerial xerial mentioned this pull request Aug 7, 2019
Closed
66 tasks
@xerial xerial mentioned this pull request Aug 8, 2019
Closed
xerial added 26 commits August 8, 2019 18:12
@xerial
Hide internal classes
dd488c5
@xerial
Createa a child session for each test
d4fc66e
@xerial
Wrap arg builder with Try
d19bbc8
@xerial
Add AirSpecCore
995cd05
@xerial
Add an accessor to protected methods
0166569
@xerial
Use compact error reporting
877a88d
@xerial
Fix Scala.js example
9b78be2
@xerial
use protected for methodSurfaces
891bacf
@xerial
Add platform-compatible scalaJsSupport
4089bd8
@xerial
Add PlatformAirSpec for adding platform-specific impl
dd453bd
@xerial
Add notes for Scala.js
57af75d
@xerial
[ci skip] Fix typo
5f1749d
@xerial
Use the same set of test specs for Scala and Scala.js
e881531
@xerial
Support object spec in Scala.js
1e29608
@xerial
Rename methods
1c8141e
@xerial
Fix child session design
20c45af
@xerial
Add configure(Design), configureLocal(Design)
7433514
@xerial
Add tests for server initialization counts
64967a9
@xerial
Customize colors
6153493
@xerial
Refactoring spec logger
6982e4a
@xerial
Allow accessing CompatApi
1498642
@xerial
Ignore failure tests
b81d7eb
@xerial
Fix for Scala.js
6adef64
@xerial
Add comments
6040515
@xerial
Rename projet folders
0ef0882
@xerial
Support wildcard and regex filtering
a137e18
@xerial xerial changed the title [WIP] AirSpec: A new testing framework for Scala AirSpec: A new testing framework for Scala Aug 9, 2019
@xerial xerial merged commit afaf515 into wvlet:master Aug 9, 2019
@xerial
Copy link
Copy Markdown
Member Author

xerial commented Aug 9, 2019

The first implementation that supports minimum requirements is done.

  • Rename airframe-spec -> airframe-scalatest for avoiding naming confusion
  • DI-based testing is working, but it seems we need more experiments by actually writing tests with AirSpec.
  • Other enhancement (e.g., power assertion, integration with ScalaCheck) can be done later.

cc: @takezoe @Lewuathe

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

sbt: Clean-up java.util.logging LogHandlers to avoid ClassNotFound issue

1 participant

@xerial