Conversation
There was a problem hiding this comment.
Pull request overview
This pull request adds support for AtCoder judge version 202510 (October 2025) to the ALE-Bench repository. The PR updates the judge environment to match the AtCoder language update from October 2025, introducing Docker images for 14 programming languages with their latest compilers, interpreters, and libraries.
Changes:
- Added Docker build and pull scripts for the 202510 judge version across 14 languages
- Created Dockerfile configurations for all supported languages (bash, cpp23, csharp, fish, fortran, go, javascript, lean, ocaml, perl, pypy, python, rust, typescript) plus an "all" image containing all languages
- Added language-specific dependency files including Rust Cargo configuration, Python package requirements, C# project files, Node.js scripts, and Fortran test files
Reviewed changes
Copilot reviewed 41 out of 42 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/docker_pull_202510.sh | Script to pull pre-built Docker base images for all 14 languages from Docker Hub |
| scripts/docker_build_202510_base.sh | Script to build base Docker images locally with all dependencies |
| scripts/docker_build_202510.sh | Script to build local runnable images from pre-built base images with user permissions |
| dockerfiles/rust_202510_* | Rust 1.89.0 configuration with Cargo.toml specifying edition 2024 and extensive crate dependencies |
| dockerfiles/python_202510_* | Python 3.13.7 configuration with frozen package versions including numpy, pandas, torch, and scientific libraries |
| dockerfiles/csharp_202510_* | C# .NET 9.0 configuration with NuGet packages for ML.NET and MathNet |
| dockerfiles/node_202510_node.sh | Node.js execution wrapper script with stack size configuration |
| dockerfiles/fortran_202510_Main.f90 | Fortran test program using stdlib and ac-library-fortran |
| dockerfiles/Dockerfile_*_202510_base | Base Dockerfiles for 14 languages with compiler/interpreter installation and library setup |
| dockerfiles/Dockerfile_*_202510 | User-space Dockerfiles that set workdir permissions for each language |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 45 out of 46 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 45 out of 46 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 60 out of 61 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 64 out of 65 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 73 out of 74 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 145 out of 146 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| execution_time = max(profiles.elapsed_time_seconds, profiles.user_cpu_seconds + profiles.system_cpu_seconds) | ||
| memory_usage = profiles.max_resident_set_size_kbytes * 1024 | ||
| is_timeout_via_wrapper = exited_with_status_137 and execution_time > time_limit | ||
| # Check the resource usage | ||
| if exit_status != 0: | ||
| if exit_status != 0 and not is_timeout_via_wrapper: |
There was a problem hiding this comment.
is_timeout_via_wrapper is currently derived from execution_time parsed from GNU time output. That value is typically low-precision (often rounded to 0.01s), so a wrapper-killed run can end up with execution_time == time_limit and be misclassified as a runtime error instead of TLE. Consider basing this check on the already-available execution_time_host (wall time on the host) and/or using an epsilon/>= comparison consistent with the timeout margins used elsewhere.
| execution_time = max(profiles.elapsed_time_seconds, profiles.user_cpu_seconds + profiles.system_cpu_seconds) | ||
| memory_usage = profiles.max_resident_set_size_kbytes * 1024 | ||
| is_timeout_via_wrapper = exited_with_status_137 and execution_time > time_limit | ||
| # Check the resource usage | ||
| if exit_status != 0: | ||
| if exit_status != 0 and not is_timeout_via_wrapper: |
There was a problem hiding this comment.
Same issue as in code_runner.parse_profiles: is_timeout_via_wrapper uses execution_time from the GNU time profile, which may be rounded such that a timeout killed via wrapper still reports exactly time_limit. Using execution_time_host here (or an epsilon/>= check) would avoid misclassifying a wrapper timeout as RUNTIME_ERROR.
| # Error codes | ||
| ERROR_CODE_137 = 137 # SIGKILL | ||
|
|
There was a problem hiding this comment.
ERROR_CODE_137 is documented as SIGKILL, but 137 is typically the exit status for a process terminated by SIGKILL (128 + 9), not the signal number itself. Updating the comment (e.g., “exit status for SIGKILL”) would avoid confusion when interpreting profile data.
AtCoder judge version update