Skip to content

Remove external ply dependency#3840

Merged
blnicho merged 29 commits intoPyomo:mainfrom
jsiirola:embed-ply
Feb 11, 2026
Merged

Remove external ply dependency#3840
blnicho merged 29 commits intoPyomo:mainfrom
jsiirola:embed-ply

Conversation

@jsiirola
Copy link
Copy Markdown
Member

Fixes #3823

Summary/Motivation:

This removes the external dependency on ply by bringing it into the codebase as a "third-party library". This resolves #3823 by modifying ply to disallow loading parse tables from pickle files (additionally, it disallows loading them from arbitrary files). Instead, vacc() only accepts an already-imported parsetable module.

This PR moves to a new workflow where we distribute the parsetable as part of the Pyomo codebase, and the DAT parser imports it like any other module. The parse table module is created by running pyomo/dataportal/parse_datacmds.py as a script. As a second check to ensure the two files are kept in sync, we srote a sha256 hash of parse_datacmds.py in the parse table module, and a DeveloperError is raised if they differ.

The new pyomo/tpl module space is excluded from Black, Coverage, and Typos checks, to support the minimal changes to any TPL code.

Changes proposed in this PR:

  • Add pyomo/tpl/ply
  • Update the DAT parser to simplify logic around creating, loading, and validating the yacc parse table
  • Remove references to ply as an external dependency
  • Update LICENSE.md with all copyright / license statements for any derived code in Pyomo.

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the BSD license.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

Copy link
Copy Markdown
Member

@blnicho blnicho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! I have a few minor comments. I only skimmed yacc.py and lex.py. If there are specific portions of those files I should review in detail let me know.

@blnicho blnicho moved this from Todo to Review In Progress in Pyomo 6.10 Feb 11, 2026
Copy link
Copy Markdown
Member

@blnicho blnicho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good pending tests

@github-project-automation github-project-automation bot moved this from Review In Progress to Reviewer Approved in Pyomo 6.10 Feb 11, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Feb 11, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.48%. Comparing base (31fdb1a) to head (9fb75e3).
⚠️ Report is 201 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3840      +/-   ##
==========================================
- Coverage   89.66%   89.48%   -0.19%     
==========================================
  Files         902      905       +3     
  Lines      105609   106462     +853     
==========================================
+ Hits        94693    95264     +571     
- Misses      10916    11198     +282     
Flag Coverage Δ
builders ?
default 83.78% <100.00%> (?)
expensive 35.53% <92.72%> (?)
linux 86.84% <100.00%> (-2.57%) ⬇️
linux_other 86.84% <100.00%> (-0.14%) ⬇️
oldsolvers 29.68% <27.27%> (-0.01%) ⬇️
osx 83.01% <100.00%> (-0.13%) ⬇️
win 85.07% <100.00%> (-0.14%) ⬇️
win_other 85.07% <100.00%> (-0.14%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Member

@blnicho blnicho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found a few typos in docstrings. Since these are NFCs we can fix them and merge without waiting for tests to run again.

@blnicho blnicho merged commit 2516cd0 into Pyomo:main Feb 11, 2026
33 of 35 checks passed
@github-project-automation github-project-automation bot moved this from Reviewer Approved to Done in Pyomo 6.10 Feb 11, 2026
@jsiirola jsiirola deleted the embed-ply branch February 11, 2026 22:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

Critical security vulnerability in ply dependency (CVE-2025-56005)

2 participants