How To Re-invoke Python Script Within Itself

I am trying to find the best way of re-invoking a Python script within itself. Currently it is working like http://github.com/benoitc/gunicorn/blob/master/gunicorn/arbiter.py#L285.

Solution 1:

I think the real issue here is that the gunicorn/arbiter.py code wants to execute the Python script with the exact same environment every time. This is important because the Python script being invoked is an unknown and it is important for it be called exactly the same way every time.

My feeling is that the problem you are experiencing has to do with the environment having changed between invokations of the Python script by the arbiter.

1. In http://github.com/benoitc/gunicorn/blob/master/gunicorn/arbiter.py#L85-89, we see that the python executable and the args are being stored by the arbiter into self.START_CTX.

2. Then in http://github.com/benoitc/gunicorn/blob/master/gunicorn/arbiter.py#L303-305, we see that the execvpe is called with the sys.executable, the modified args and then os.environ.

If os.environ had changed somewhere else (i.e. the PWD variable), then your executable will fail to be called properly (because you're no longer in the correct folder). The arbiter seems to take care of that possibility by storing the cwd in START_CTX. So the question remains, why is the invokation failing for you?

I tried some test code out which I wrote as follows:

#!/usr/bin/env pythonimport sys
import os

defmain():
    """Execute twice"""

    cwd = os.getcwd()

    print cwd
    print sys.argv

    if os.path.exists("/tmp/started.txt"):
        os.unlink("/tmp/started.txt")
        print"Deleted /tmp/started.txt"printreturn

    args = [sys.executable] + sys.argv[:]
    os.system("touch /tmp/started.txt")
    print"Created /tmp/started.txt"print
    os.execvpe(sys.executable, args, os.environ)

if __name__ == '__main__':
    main()

When I execute this code from the command line, it works just fine:

guest@desktop:~/Python/Test$ python selfreferential.py 
/Users/guest/Python/Test
['selfreferential.py']
Created /tmp/started.txt

/Users/guest/Python/Test
['selfreferential.py']
Deleted /tmp/started.txt

guest@desktop:~/Python/Test$ python ./selfreferential.py 
/Users/guest/Python/Test
['./selfreferential.py']
Created /tmp/started.txt

/Users/guest/Python/Test
['./selfreferential.py']
Deleted /tmp/started.txt

guest@desktop:~/Python/Test$ cd
guest@desktop:~$ python Python/Test/selfreferential.py 
/Users/guest
['Python/Test/selfreferential.py']
Created /tmp/started.txt

/Users/guest
['Python/Test/selfreferential.py']
Deleted /tmp/started.txt

guest@desktop:~$ python /Users/guest/Python/Test/selfreferential.py 
/Users/guest
['/Users/guest/Python/Test/selfreferential.py']
Created /tmp/started.txt

/Users/guest
['/Users/guest/Python/Test/selfreferential.py']
Deleted /tmp/started.txt

guest@desktop:~$ 

As you can see, there was no problem doing what gunicorn was doing. So, maybe your problem has something to do with the environment variable. Or maybe is has something to do with the way your operating system executes things.

Solution 2:

I would suggest a different approach. Wrapp all of the script functionality into a single function, which gets called when the script is executed, which can recursively call itself, instead of executing a new process.