HiveBrain v1.2.0
Get Started
← Back to all entries
debugpythonMinor

Dynamic module import in Python with exception handling

Submitted by: @import:stackexchange-codereview··
0
Viewed 0 times
handlingexceptionwithmodulepythondynamicimport

Problem

import nltk

req_modules = {'from nltk import punkt': 'punkt',
               'from nltk.corpus import stopwords': 'stopwords',
               'from nltk import pos_tag': 'averaged_perceptron_tagger',
               'from nltk import ne_chunk': 'maxent_ne_chunker'}

for m in req_modules:
    try:
        print("Trying: '%s'" % m)
        exec(m)    # try to import the package
        print("Success.")
    except (LookupError, ImportError):
        # if data not found (not already installed), download it
        print("Tried: %s. Resource '%s' was not available \
              and is being downloaded.\n" % (m, req_modules[m]))
        nltk.download(req_modules[m])
        exec(m)


I have some code I'm distributing to folks at work, and the lines above are at the top of the file. In order to run the code underneath (which harnesses several subpackages in nltk), the user needs to have downloaded these subpackages. Is this is safe/reliable way of doing dynamic module import? It has worked for me thus far.

Solution

I mean I'm not a fan of it since it's basically just a list of
statements to execute, but then again, the solution via globals and
importlib
doesn't look that much better:

import importlib
import nltk

req_modules = {'nltk.punkt': 'punkt',
               'nltk.corpus.stopwords': 'stopwords',
               'nltk.pos_tag': 'averaged_perceptron_tagger',
               'nltk.ne_chunk': 'maxent_ne_chunker'}

def try_load(module, name):
    print("Trying to load: '%s'" % module)
    globals()[name] = importlib.import_module(module)
    print("Success.")

for module, name in req_modules.items():
    try:
        try_load(module, name)
    except (LookupError, ImportError):
        # if data not found (not already installed), download it
        print("Tried to load: '%s'. Resource '%s' was not available \
               and is being downloaded.\n" % (module, name))
        nltk.download(name)
        try_load(module, name)


N.b. I'd say use a list of module/name pairs instead to enforce the
order to make diagnostics a bit easier (or load dependencies for one of
the later modules first).

Now that I read it again, this seems alright if it's in a module and reusable, otherwise doesn't look like it's less safe than other options that involve nltk.download and it provides informative output while it gets things, so that's nice.

Code Snippets

import importlib
import nltk

req_modules = {'nltk.punkt': 'punkt',
               'nltk.corpus.stopwords': 'stopwords',
               'nltk.pos_tag': 'averaged_perceptron_tagger',
               'nltk.ne_chunk': 'maxent_ne_chunker'}

def try_load(module, name):
    print("Trying to load: '%s'" % module)
    globals()[name] = importlib.import_module(module)
    print("Success.")

for module, name in req_modules.items():
    try:
        try_load(module, name)
    except (LookupError, ImportError):
        # if data not found (not already installed), download it
        print("Tried to load: '%s'. Resource '%s' was not available \
               and is being downloaded.\n" % (module, name))
        nltk.download(name)
        try_load(module, name)

Context

StackExchange Code Review Q#147217, answer score: 3

Revisions (0)

No revisions yet.