Skip to content

Fix ScriptRunner handling crashed state#2563

Merged
jmthomas merged 2 commits intomainfrom
script_states
Nov 25, 2025
Merged

Fix ScriptRunner handling crashed state#2563
jmthomas merged 2 commits intomainfrom
script_states

Conversation

@jmthomas
Copy link
Copy Markdown
Member

@jmthomas jmthomas commented Nov 24, 2025

closes #2534

This would replace #2552 which had good ideas but I wanted to disallow retry for a crashed script ... it doesn't make sense to retry because unless you modify the script it will continue to crash. Also I wanted to explicitly handle all the script states in the case statement (spawning and init were not handled).

@jmthomas jmthomas requested review from clayandgen, ryan-pratt and ryanmelt and removed request for clayandgen, ryan-pratt and ryanmelt November 24, 2025 16:07
RunningScript.instance.script_binding[0]["__file__"] = procedure_name
exec(instrumented_script, RunningScript.instance.script_binding[0], RunningScript.instance.script_binding[1])
exec(
instrumented_script, RunningScript.instance.script_binding[0], RunningScript.instance.script_binding[1]
Copy link
Copy Markdown
Contributor

@aikido-pr-checks aikido-pr-checks bot Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsafe exec usage can lead to remote code execution - high severity
Using exec with expressions based on user input can execute arbitrary code.

Remediation: If you must use exec, create a allowlist for the input that goes into it to filter the strings.
View details in Aikido Security

@codecov
Copy link
Copy Markdown

codecov bot commented Nov 24, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.18%. Comparing base (976eb3b) to head (ad2970f).
⚠️ Report is 15 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2563      +/-   ##
==========================================
- Coverage   79.21%   79.18%   -0.04%     
==========================================
  Files         662      662              
  Lines       51218    51271      +53     
  Branches      736      736              
==========================================
+ Hits        40573    40599      +26     
- Misses      10565    10592      +27     
  Partials       80       80              
Flag Coverage Δ
python 81.29% <ø> (ø)
ruby-api 84.70% <ø> (+0.13%) ⬆️
ruby-backend 82.12% <100.00%> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

})
},
async scriptComplete() {
this.fatal = false
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're no longer marking fatal lines ... the output shows which line is fatal and the script simply stops with the status set to 'Crashed'

if (this.state === 'completed_errors') {
// Displaying 'Completed_errors' is not very user friendly
this.state = 'completed'
}
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than always setting to 'Stopped', we preserve the state unless 'completed_errors' which I just translate. The error will be in the script output.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't this cause users to just see "completed" at the top and exit out without noticing the errors? I feel like there should be some sort of indication of errors where we display state. Maybe an alert icon, or shade it orange/yellow?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Execution Status remains with completed_errors, which is good. I do tend to agree that the UI should make it clear that the script did encounter errors.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script still completed but it potentially had errors like Check Errors or even logical errors (divide by zero). The user clicked Retry or Go and the script completed. So I'm not sure what you mean by exit out. The status is in the script execution status. Perhaps we do textcolor changes of the status where completed_errors would display Completed but it would be orange. Crashed / Killed is red and all others green. Thoughts?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make completed_errors state display in the UI as Completed with Errors? I feel like that's the most user-friendly.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes the State box much wider to be readable

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add an alert icon or shade the box a color? Maybe also a tooltip, but that might not be necessary.

By "exit out," I mean leave the page (close the browser tab or navigate away)

case 'completed_errors':
case 'stopped':
case 'crashed':
case 'killed':
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are all the possible 'completed' states. The script is done in the backend.

end
end

require 'openc3/utilities/logger'
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving this fixes the circular dependency in Enterprise tests where OpenC3.disable_warnings is not defined.

from openc3.tools.test_runner.test import SkipTestCase
from openc3.script.suite import Group
from openc3.utilities.script_instrumentor import ScriptInstrumentor
import openc3.utilities.target_file_importer
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not used

# All ask and prompt dialogs should include a 'Cancel' button
# If they cancel we wait so they can potentially stop
if input == "Cancel":
if user_input == "Cancel":
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

input shadows a built-in so renamed

kill_thread(self, RunningScript.output_thread)
RunningScript.output_thread = None
self.mark_completed()
self.current_filename = None
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually a bug because we're setting the current_filename method to None which basically removes it. I think any call to RunningScript.instance.current_filename() would crash after this. Maybe didn't matter because this is in Exception handling and we're already complete.

formatted_lines = traceback.format_exception(exc_type, exc_value, exc_traceback)
# Print the last 4 lines to show the exception, the ^^^^ line,
# the line itself, and the filename / line number
Logger.error("".join(formatted_lines[-4:]))
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to show the actual line number of the exception. I noticed that previous we just print only the exception which doesn't show where it is. Makes it a lot harder to debug.

Copy link
Copy Markdown
Contributor

@clayandgen clayandgen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious, do you have a new formatter or were these formatting changes manual? (or perhaps this file just hadn't been modified in some time)

@jmthomas
Copy link
Copy Markdown
Member Author

Not sure if the reformatted was due to pylance or black. I think the former.

@jmthomas jmthomas requested a review from clayandgen November 24, 2025 23:13
Copy link
Copy Markdown
Contributor

@clayandgen clayandgen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Big fan of this update!
Screenshot 2025-11-25 at 9 45 21 AM

@jmthomas jmthomas merged commit 0706d53 into main Nov 25, 2025
48 of 50 checks passed
@jmthomas jmthomas deleted the script_states branch November 25, 2025 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python ScriptRunner Execute Selection and Run From Line Not Working when stopped

3 participants