Python signal delivery under BSD 4.4

Re: Python signal delivery under BSD 4.4

In article <m3bruuk080.fsf @drizzle.com>,
"Elf M. Sternberg" <[email protected] m> wrote:[color=blue]
> I'm having a devil of a time with signal handling under BSD 4.4 with
> Python and I was hoping that this might ring a bell with someone. Using
> the Webware Application Server (a multi-threaded Python application), I
> fork off a process to run independently and do some heavy-duty multi-
> threaded I/O churning. The process is launched with os.spawnvp().
>
> The user may, at any time, elect to terminate this process
> before it reaches its end. I do that by having the Application Server
> send a SIGUSR1 signal to the process ID returned by os.spawnvp(), which
> so far has been reliably returned and stored in the appserver's session
> mananger. So far, so good.
>
> This worked well under Python 2.1, where I used os.fork() instead
> of os.spawnvp().
>
> We have upgraded to Python 2.2.3 and it has mysteriously stopped
> working. Processes launched off the appserver, whether using os.fork()
> or os.spawnvp(), are now completely deaf to signals. Nothing gets its
> attention except, of course, SIGKILL. Not when sent from the appserver,
> and not when sent from the shell, not even by root.
>
> If the process is launched from the command line, however, it
> is well-behaved and responds to signal reliably.
>
> What gives?[/color]

Don't know. Can you duplicate it in a simpler environment?
I'd like to run the following C program from spawnv, from a
Python program like

import os
import time
import signal

p = os.spawnv(os.P_ NOWAIT, 'prsig', ['prsig'])
time.sleep(1.0)
os.kill(p, signal.SIGTERM)
print 'process exit', os.waitpid(p, 0)

if you think that represents what you're doing. This seems
to work as expected with Python 2.2 on a BSD platform (if
your "BSD 4.4" is actually NetBSD 1.6 or FreeBSD 5.1, I could
try that.)

The reasons I can think of in principle are
1. handler set to SIG_IGN (if the child process is supposed
to exit due to the default signal handler.)
2. blocked via signal mask
3. wrong process

Donn Cave, [email protected] on.edu/[email protected] m

include <signal.h>
#include <stdio.h>
#include <string.h>

static void
handler(int sig)
{
switch (sig) {
case SIGALRM:
fprintf(stderr, "caught SIGALRM\n");
break;
case SIGUSR1:
fprintf(stderr, "caught SIGUSR1\n");
break;
default:
fprintf(stderr, "caught %d\n", sig);
}
exit(0);
}

static void
prsig(int sig)
{
struct sigaction sa;
int status;
memset(&sa, 0, sizeof(sa));
status = sigaction(sig, 0, &sa);
if (status == 0) {
printf("sigacti on {\n");
printf(" sa_handler 0x%08x,\n", sa.sa_handler);
printf(" sa_flags 0x%08x\n", sa.sa_flags);
/* printf(" sa_mask 0x%08x\n", sa.sa_mask); */
printf("}\n");
} else {
perror("sigacti on query");
exit(2);
}
}

static void
prmask()
{
sigset_t mask;
int status;
status = sigprocmask(SIG _SETMASK, 0, &mask);
if (status == 0)
fprintf(stdout, "mask 0x%lx\n", mask);
else {
perror("sigproc mask");
exit(2);
}
}

static void
sethandler(int sig, void (*hf)(int))
{
struct sigaction sa;
int status;
memset(&sa, 0, sizeof(sa));
sa.sa_handler = hf;
status = sigaction(sig, &sa, 0);
if (status == 0) {
} else {
perror("sigacti on");
exit(2);
}
}

static void
pause()
{
char buffer[4];
read(0, buffer, 4);
}

int
main(int argc, char **argv)
{
prsig(SIGUSR1);
prmask();
sethandler(SIGU SR1, handler);
sethandler(SIGA LRM, handler);
alarm(3);
pause();
return 0;
}

Re: Python signal delivery under BSD 4.4

Donn Cave <[email protected] ton.edu> writes:
[color=blue]
> In article <m3bruuk080.fsf @drizzle.com>,
> "Elf M. Sternberg" <[email protected] m> wrote:[/color]
[color=blue][color=green]
>> This worked well under Python 2.1, where I used os.fork() instead
>> of os.spawnvp().[/color][/color]
[color=blue][color=green]
>> We have upgraded to Python 2.2.3 and it has mysteriously stopped
>> working. Processes launched off the appserver, whether using os.fork()
>> or os.spawnvp(), are now completely deaf to signals. Nothing gets its
>> attention except, of course, SIGKILL. Not when sent from the appserver,
>> and not when sent from the shell, not even by root.[/color][/color]

I'm sorry about the confusion. FreeBSD 4.4 is what we're
running where I work.
[color=blue]
> Can you duplicate it in a simpler environment?
> I'd like to run the following C program from spawnv, from a
> Python program like[/color]
[color=blue]
> import os
> import time
> import signal[/color]
[color=blue]
> p = os.spawnv(os.P_ NOWAIT, 'prsig', ['prsig'])
> time.sleep(1.0)
> os.kill(p, signal.SIGTERM)
> print 'process exit', os.waitpid(p, 0)[/color]

I'm going to assume you meant 'SIGUSR1' here since that's what
you're catching in the C code you included.

In any event, your code worked as expected.

However, this variant worked as I've reported:

import os
import sys
import time
import signal
import thread


def do_it():
p = os.spawnv(os.P_ NOWAIT, 'prsig', ['prsig'])
print >>sys.stderr, "Sleeping"
time.sleep(1.0)
os.kill(p, signal.SIGUSR1)
print 'process exit', os.waitpid(p, 0)
sys.exit()

thread.start_ne w_thread(do_it, ())
time.sleep(4.0)

Not only did it behave as I reported, but it slept the full four
seconds even though the sys.exit() call was made. This leads me to the
conclusion that something rather unusual is going on within Python's
threading module and the libc_r.a within FreeBSD4.4.

Right now, I just want a way to make signal delivery work.

Thanks for your help so far, though.

Elf

Re: Python signal delivery under BSD 4.4

In article <m3n0eds0bd.fsf @drizzle.com>,
"Elf M. Sternberg" <[email protected] m> wrote:
....[color=blue]
> I'm going to assume you meant 'SIGUSR1' here since that's what
> you're catching in the C code you included.[/color]

Right, sorry.
[color=blue]
> However, this variant worked as I've reported:[/color]
[... forks from thread ]

I tried it from FreeBSD 5.1, using 2.2.2 built from ports,
and didn't see this problem. I believe 5.1 has a significantly
different thread implementation from 4.x, though.

Note that there is pthread_sigmask (3) with the same arguments as
sigprocmask(2), and Python's thread_pthread. h does appear to block
all signals that way when starting a new thread. That's a clue,
perhaps. If sigprocmask() didn't show any signals blocked, maybe
pthread_sigmask () would, or maybe at any rate you can set the
mask to 0 using one or the other of those functions.

Donn Cave, [email protected] on.edu

Re: Python signal delivery under BSD 4.4

On Wed, 13 Aug 2003, Donn Cave wrote:
[color=blue]
> In article <m3n0eds0bd.fsf @drizzle.com>,
> "Elf M. Sternberg" <[email protected] m> wrote:
>[color=green]
> > However, this variant worked as I've reported:[/color]
> [... forks from thread ]
>
> I tried it from FreeBSD 5.1, using 2.2.2 built from ports,
> and didn't see this problem. I believe 5.1 has a significantly
> different thread implementation from 4.x, though.[/color]

And later releases of FreeBSD 4.x have a number of bugfixes to the pthread
support, including some signal related changes. The 4.x pthread support
still has some issues though.

Elf refers to this working with Python 2.1 - I vaguely recall some changes
to Python's pthreads support to try and "harmonise" the behaviour across a
number of platforms, and signal delivery may have been affected such that
signals are only delivered to the primary thread. As this is only a vague
recollection, treat with caution...

Does building Python with the linuxthreads port, instead of FreeBSD's
native pthreads, behave as expected?
[color=blue]
> Note that there is pthread_sigmask (3) with the same arguments as
> sigprocmask(2), and Python's thread_pthread. h does appear to block
> all signals that way when starting a new thread. That's a clue,
> perhaps. If sigprocmask() didn't show any signals blocked, maybe
> pthread_sigmask () would, or maybe at any rate you can set the
> mask to 0 using one or the other of those functions.[/color]

I also recall Michael Hudson trying to get signal mask support working in
the presence of threads, but he could only get it to work on Linux. We
did eventually get a required fix to FreeBSD's libc_r committed (in the
4.7 timeframe I think), but Michael had given up by then and canned his
code.

--
Andrew I MacIntyre "These thoughts are mine alone..."
E-mail: andymac@bullsey e.apana.org.au (pref) | Snail: PO Box 370
[email protected] g.au (alt) | Belconnen ACT 2616
Web: http://www.andymac.org/ | Australia

Re: Python signal delivery under BSD 4.4

Andrew MacIntyre <andymac@bullse ye.apana.org.au > writes:
[color=blue][color=green]
>> I tried it from FreeBSD 5.1, using 2.2.2 built from ports,
>> and didn't see this problem. I believe 5.1 has a significantly
>> different thread implementation from 4.x, though.[/color][/color]
[color=blue]
> And later releases of FreeBSD 4.x have a number of bugfixes to the pthread
> support, including some signal related changes. The 4.x pthread support
> still has some issues though.[/color]

I went and looked again at the evidence.

When calling the function with apply(), the child process spawned off
reports:

mask 0x0

When calling the function with thread.start_ne w_thread(), the child
process reports:

mask 0xfffefeff

So I think the two of you have more or less hit it squarely on the
head. Now, I just have to find a way around it... *Sigh*

Elf

Re: Python signal delivery under BSD 4.4

Michael Hudson wrote:
[color=blue]
> "Elf M. Sternberg" <[email protected] m> writes:
>[color=green]
>> Right now, I just want a way to make signal delivery work.[/color]
>
> At some point, the threads implementation got changed so that new
> threads get their signal mask set to "block everything and it's cat"
> (there was a reason for this, but I'm not really sure what it was).
> As I see it, you have two options: 1) write a little C to unset the
> procmask after fork() but before execve() 2) only launch apps from the
> main thread.[/color]

Incidentally, these aren't far from the advice in "Pthreads Programming"
(by Nichols, Buttlar and Proulx Farrell -- O'Reilly) in Chapter 5, under
"Threads and Process Management". The authors are dealing with C issues,
not Python by any means, but still they conclude that "forking from a
multithreaded program is no picnic" and suggest 1) fork before you've
created any threads, and 2) consider the surrogate parent model (fork a
child process at init time, IPC with the child -- which remains single
threaded -- to delegate forking and execing on our behalf).


Alex

Re: Python signal delivery under BSD 4.4

Alex Martelli <aleaxit@yahoo. com> writes:
[color=blue]
> Incidentally, these aren't far from the advice in "Pthreads Programming"
> (by Nichols, Buttlar and Proulx Farrell -- O'Reilly) in Chapter 5, under
> "Threads and Process Management". The authors are dealing with C issues,
> not Python by any means, but still they conclude that "forking from a
> multithreaded program is no picnic" and suggest 1) fork before you've
> created any threads, and 2) consider the surrogate parent model (fork a
> child process at init time, IPC with the child -- which remains single
> threaded -- to delegate forking and execing on our behalf).[/color]

(1) wasn't really viable since the parent process is ruled by an
application server (Webware, in this case) and I have no desire to go
trolling through the guts of that excellent program to make it "work"
the way I want. I almost went with (2). But, fortunately, we're
developing a single-OS appliance and don't really care about
cross-platform compatibility, so I worked with the FreeBSD 4.4 "model"
of forking from multithreaded applications. It's fortunate that the
next call is an exec(); I'd hate to have to manage a forked copy of the
process with that one thread...

Elf

Re: Python signal delivery under BSD 4.4

Quoth "Elf M. Sternberg" <[email protected] m>:
| ... But, fortunately, we're
| developing a single-OS appliance and don't really care about
| cross-platform compatibility, so I worked with the FreeBSD 4.4 "model"
| of forking from multithreaded applications. It's fortunate that the
| next call is an exec(); I'd hate to have to manage a forked copy of the
| process with that one thread...

So, you resolved the problem and the spawnv child process can now
receive signals? Did you use sigprocmask() to clear the signal
mask? What is the FreeBSD 4.4 model of forking from multithreaded
applications? (And is it the same as the FreeBSD 4.7 model, 5.1, etc.?)

Donn Cave, [email protected] m