INNER CODE UNIT · Python
NicerTrace
huggingface/large_language_model_training_playbook · debug/NicerTrace.py:18
class NicerTrace(trace.Trace):
# as the 2 paths overlap the longer with site-packages needs to be first
py_dirs = [sysconfig.get_paths().get(k) for k in ["purelib", "stdlib"]]
site_packages_dir = sysconfig.get_paths()["purelib"]
stdlib_dir = sysconfig.get_paths()["stdlib"]
def __init__(self, *args, packages_to_include=None, log_pids=False, **kwargs):
"""normal init plus added package/dir exclusion overrides:
While preserving the original behavior a new optional arg is added `packages_to_include`
with the following behavior:
1. if ignoredirs is a list the original trace behavior is used - only those dirs and subdirs will be excluded
2. if ignoredirs is None and packages_to_include is None - everything is included
3. if packages_to_include="uninstalled" all packages found under /.../site-packages will be excluded. I couldn't find a way to exclude core python packages under /.../lib/python3.8 since it'd then exclude site-packages as well
3. if packages_to_include=["PIL", "numpy", "pytorch"] all packages found under /.../site-packages, and /.../lib/python3.8 will be excluded except the packages that were listed to be included - use top-level package name here
4. if packages_to_include=None, everything under /.../site-packages, and /.../lib/python3.8 will be excluded and any packages that are installed via `pip install -e .` will be included