Repository navigation
Commit 40d0f06
committed
Triple-check hardening: perceptron TOCTOU squat, pathsec fd-leak, bcp47 entity-safe XML (CWE-59/377/400/776)
Follow-up from an independent adversarial re-check of the previous commit (two
fresh agents attacking the fixes + re-sweeping the whole repo). Three real items
surfaced; each was reproduced out-of-process, fixed, and added to the harness.
- tag/perceptron.py -- the previous `os.path.islink(loc)` pre-check on the
trained-model save dir was non-atomic (TOCTOU) and inspected only the final
component, so the guessable shared-temp name could still be raced or reached
via a symlinked parent. Replace it with `_open_private_model_dir`: create the
leaf with `os.mkdir` (atomic) and re-open with `O_NOFOLLOW|O_DIRECTORY`
(refusing a pre-planted/raced symlink), verify it is a real directory owned
by the current user and not group-/world-writable, and write each model file
relative to that pinned directory fd with O_NOFOLLOW (CWE-59/377/378).
- pathsec.py -- `_hardened_open` called `os.fdopen` outside the fd-closing
guard. A mode string that `os.open` accepts but `os.fdopen` rejects (e.g. an
empty mode) leaked the descriptor; repeated calls could exhaust the fd table
(DoS). Wrap the `os.fdopen` so the fd is closed on any failure.
- corpus/reader/bcp47.py -- the only XML corpus reader still using raw
`ElementTree.parse`; a malicious `bcp47` data package could ship a
billion-laughs `common-subdivisions-en.xml` (reachable via
`nltk.langnames.langname()`), expanding internal entities to exhaust memory
(CWE-776/400). Route it through `nltk.xmlsec.parse`, which refuses entity
declarations, like every other XML consumer in the tree.
Extend test_path_traversal_security.py: fd-leak-on-bad-mode, world-writable
save-dir refusal, and bcp47 entity-expansion rejection + benign-parse. The
independent re-check also confirmed the write-side symlink escape is genuinely
closed (fd-realpath re-validation) and cleared the rest of the tree
(framenet, wordnet_app whitelist, downloader zip, external-tool wrappers) with
reasons. Full unit suite: 1521 passed, 0 failed; touched-module doctests green;
pre-commit clean.1 parent 483c5fe commit 40d0f06
4 files changed
Lines changed: 145 additions & 28 deletions
File tree
- nltk
- corpus/reader
- tag
- test/unit
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
11 | 10 | | |
12 | 11 | | |
| 12 | + | |
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
40 | | - | |
| 40 | + | |
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
734 | 734 | | |
735 | 735 | | |
736 | 736 | | |
737 | | - | |
| 737 | + | |
| 738 | + | |
| 739 | + | |
| 740 | + | |
| 741 | + | |
| 742 | + | |
| 743 | + | |
| 744 | + | |
738 | 745 | | |
739 | 746 | | |
740 | 747 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
51 | 94 | | |
52 | 95 | | |
53 | 96 | | |
| |||
289 | 332 | | |
290 | 333 | | |
291 | 334 | | |
292 | | - | |
293 | | - | |
294 | | - | |
295 | | - | |
296 | | - | |
297 | | - | |
298 | | - | |
299 | | - | |
300 | | - | |
301 | | - | |
302 | | - | |
303 | | - | |
304 | | - | |
305 | | - | |
306 | | - | |
307 | | - | |
308 | | - | |
309 | | - | |
310 | | - | |
311 | | - | |
312 | | - | |
313 | | - | |
314 | | - | |
315 | | - | |
316 | | - | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
317 | 364 | | |
318 | 365 | | |
319 | 366 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
256 | 256 | | |
257 | 257 | | |
258 | 258 | | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
259 | 274 | | |
260 | 275 | | |
261 | 276 | | |
| |||
296 | 311 | | |
297 | 312 | | |
298 | 313 | | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
0 commit comments