Skip to content

feat: updates in SBT and new site engine #673

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
SemyonSinchenko merged 19 commits into from
Sep 9, 2025
Merged

feat: updates in SBT and new site engine #673

SemyonSinchenko merged 19 commits into from
Sep 9, 2025

Conversation

@SemyonSinchenko
Copy link
Collaborator

@SemyonSinchenko SemyonSinchenko commented Aug 26, 2025 •
edited
Loading

What changes were proposed in this pull request?

  • updates to SBT
  • stricter code-style rules
  • corresponding updates
  • new site engine (Laika)
  • docs redesign

Why are the changes needed?

Close #672

Features:

  • A new blog for graphframes with auto-generated Atom feed
  • Redesign of the documentation to be more structured
  • Automatic benchmarks run and text generation
  • New Pregel API docs
  • Property Graphs API docs
  • A lot of updates
  • Updates to sbt to make the build simpler and linter more strict
  • Auto populating of the version in all the bash commands
  • Full integration of docs build to sbt build

@SemyonSinchenko SemyonSinchenko self-assigned this Aug 26, 2025
@codecov-commenter
Copy link

codecov-commenter commented Aug 30, 2025 •
edited
Loading

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 75.00000% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.62%. Comparing base (755c7ef) to head (3c95fb2).

Files with missing lines Patch % Lines
...n/scala/org/graphframes/lib/LabelPropagation.scala 0.00% 6 Missing ⚠️
.../scala/org/graphframes/lib/GraphXConversions.scala 0.00% 1 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files 
@@            Coverage Diff             @@
##           master     #673      +/-   ##
==========================================
+ Coverage   78.21%   81.62%   +3.40%     
==========================================
  Files          29       28       -1     
  Lines        1368     1306      -62     
  Branches      167      148      -19     
==========================================
- Hits         1070     1066       -4     
+ Misses        298      240      -58

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@SemyonSinchenko SemyonSinchenko marked this pull request as ready for review September 1, 2025 16:36
@SemyonSinchenko
Copy link
Collaborator Author

SemyonSinchenko commented Sep 1, 2025 •
edited
Loading

How to test it?

  • Run benchmarks sbt "benchmarks/Jmh/run -rf json"
  • Generate python docs cd python && poetry run bash -c "cd docs && make clean && make html"
  • Run Laika preview server sbt docs/laikaPreview
  • Go to localhost:4242

james-willis
james-willis reviewed Sep 2, 2025
View reviewed changes
james-willis
james-willis reviewed Sep 2, 2025
View reviewed changes
james-willis
james-willis requested changes Sep 2, 2025
View reviewed changes
rjurney
rjurney approved these changes Sep 4, 2025
View reviewed changes
Copy link
Collaborator

@rjurney rjurney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

docs/src/04-user-guide/11-property-graphs.md

val moviesGroup = VertexPropertyGroup("movies", moviesData, "id", applyMaskOnId = false)
```

Copy link
Collaborator

@rjurney rjurney Sep 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool!

Copy link
Collaborator

@rjurney rjurney Sep 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is called graph construction via deconvolution. Maybe worth using the term "deconvolution." Search for "deconvolution" in this PDF from Stanford CS224W in 2018: https://snap.stanford.edu/class/cs224w-2018/handouts/04-netconstruct.pdf for more info.

@SemyonSinchenko SemyonSinchenko mentioned this pull request Sep 7, 2025
Closed
6 tasks
@SemyonSinchenko
Copy link
Collaborator Author

SemyonSinchenko commented Sep 7, 2025

I fixed all the tests, benchmarks, etc.

  • I put Python API first and resolves all the comments.

@SemyonSinchenko
Copy link
Collaborator Author

SemyonSinchenko commented Sep 7, 2025

@rjurney @james-willis take another look please

@SemyonSinchenko SemyonSinchenko merged commit f2dc5b4 into graphframes:master Sep 9, 2025
5 checks passed
@SemyonSinchenko SemyonSinchenko deleted the 672-laika branch September 9, 2025 04:53
@SemyonSinchenko SemyonSinchenko mentioned this pull request Sep 9, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@james-willis james-willis james-willis approved these changes

@rjurney rjurney Awaiting requested review from rjurney

Assignees

@SemyonSinchenko SemyonSinchenko

Labels

documentation scala

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

feat: switch website backend from Jekyll to Laika

4 participants

@SemyonSinchenko @codecov-commenter @rjurney @james-willis