def foo():
    global database_index
    bar()
    import code; code.interact(local=locals())

Have you ever called code.interact() and forgot to pass local=locals() or local={**globals(), **locals()}. Most of the time you may just exit interactive console, add missing parameters and run program again. But what if the program was executing for couple of hours before interactive console was started? You might want to access e.g. global variables without running it again. Fortunately Python is a language for adults, so it’s totally doable.

The first option to access all variables etc is to use sys._getframe:

>>> import sys
>>> frame = sys._getframe(6)  # in my setup it happens to be 6th frame
>>> database_index = frame.f_globals['dataframe_index']

But in the help of _getframe we can read:

This function should be used for internal and specialized purposes only.

Fortunately there’s another module that also does what we want: inspect. It’s a little bit more verbose, but is not private.

>>> import inspect
>>> database_index = inspect.stack()[6].frame.f_globals['database_index']

But frankly speaking if you look at inspect’s code you discover that it uses sys under the bonnet. So my suggestion is to:

  • use sys._getframe in emergency situations (like in interactive console)
  • use inspect module if you are doing this in a script
def stack(context=1):
    """Return a list of records for the stack above the caller's frame."""
    return getouterframes(sys._getframe(1), context)

Obviously instead of code.interact an alternative can be used: pdb.set_trace. It doesn’t suffer such problems at all.

Bonus: python frames and surprising setter of f_lineno

Out of curiosity I looked into CPython sources. It looks like _getframe function does simple O(n) stack traversal. It retrieves frames from current thread state.

static PyObject * sys__getframe_impl(PyObject *module, int depth)
{
    PyFrameObject *f = _PyThreadState_GET()->frame;

    while (depth > 0 && f != NULL) {
        f = f->f_back;
        --depth;
    }
	...
    Py_INCREF(f);
    return (PyObject*)f;
}

_PyThreadState_GET as name suggests returns thread state object, where frame is one of the most important fields. Quick look at the definition of frame struct reveals what potentially can be done with it: f_back, f_code, f_globals, f_locals, f_lineno etc. My inner hacker woke up and I tried to change f_lineno of a frame to see what happens:

>>> sys._getframe().f_lineno = 1
ValueError: f_lineno can only be set by a trace function

This error is baked right into CPython! Apparently frame_setlineno function bails out when the caller is not a trace function. From the docs of the function we can also learn that f_lineno is used by tracking mechanism. It also describes some exceptions where you cannot jump:

  • Lines with an ‘except’ statement on them can’t be jumped to, because they expect an exception to be on the top of the stack.
  • Lines that live in a ‘finally’ block can’t be jumped from or to, since the END_FINALLY expects to clean up the stack after the ‘try’ block.
  • ‘try’, ‘with’ and ‘async with’ blocks can’t be jumped into because the blockstack needs to be set up before their code runs.
  • ‘for’ and ‘async for’ loops can’t be jumped into because the iterator needs to be on the stack.
  • Jumps cannot be made from within a trace function invoked with a ‘return’ or ‘exception’ event since the eval loop has been exited at that time.

I can only say that whatever detail you pick it becomes a rabbit hole. This is so beautiful.