I have found myself wanting to import module and provide arguments to them. There's two main reason I could think of for this. First is to prevent a circular import, though most of circular imports can be prevented by changing the design. The second reason is to invert dependencies between two modules. It occurred to me when using libraries like FastAPI or Flask that it would be nicer to use and leaner if instead of:
1. Import FastAPI Router or FastAPI object directly
2. Create a Router
3. Use the route decorators

We could simply import a module with importlib and provide the router itself. You can then keep all the FastAPI related code inside its module and only have clean list of endpoints. Now this is actually doable right now, here's the snippet: https://gist.github.com/BinarSkugga/c281cbbe36e7f11bc0fd143ea1bb4dd4

Disclaimer: I have no idea what are the implication of this but I'd like to have it as a feature instead of this hack.

On Sat, Jul 31, 2021 at 3:48 AM Charles Smith <charles@sollum.tech> wrote:
>
> I have found myself wanting to import module and provide arguments to them. There's two main reason I could think of for this. First is to prevent a circular import, though most of circular imports can be prevented by changing the design. The second reason is to invert dependencies between two modules. It occurred to me when using libraries like FastAPI or Flask that it would be nicer to use and leaner if instead of:
> 1. Import FastAPI Router or FastAPI object directly
> 2. Create a Router
> 3. Use the route decorators
>
> We could simply import a module with importlib and provide the router itself. You can then keep all the FastAPI related code inside its module and only have clean list of endpoints. Now this is actually doable right now, here's the snippet: https://gist.github.com/BinarSkugga/c281cbbe36e7f11bc0fd143ea1bb4dd4
>
> Disclaimer: I have no idea what are the implication of this but I'd like to have it as a feature instead of this hack.
>

One problem here is that imports are cached. If module #1 says "import
flask", Python runs the module and gives back the result; if module #2
subsequently says "import flask", Python returns the exact same module
from the cache. That won't work well with parameterization.

I would recommend either:

1) Import a class from the module, then instantiate; or
2) Import the module, then make a change to its globals.

The first one allows different modules to have different parameters;
the second would have different parameters overwrite each other. (And
it would be very clear that it's going to do so.)

The idiom "from flask import Flask; flask = Flask()" is a bit clunky,
but it's probably the best you have for adding parameters. (In this
specific case, "app = Flask()", which is more common, but same
difference.) I can't really imagine shortening it very much; a syntax
like "import flask()" would be confusing, and every other syntax I can
think of will be only marginally better than the standard idiom.

There is one important situation where you can't just import the whole
module and then do stuff, and that's choosing to import *less* than
the full module. For that use-case, it kinda has to be a package, but
then you should be able to do something like:

package/__init__.py
# all the basic stuff in this file

package/expensive1.py
package/expensive2.py
package/expensive3.py
# different parts

package/all.py
from . import expensive1, expensive2, expensive3

Then you can say something like:

import package.expensive2

and it'll only get some part of the package. It's a bit unideal in
that the namespace has to reflect this, but with careful use of
__getattr__, you could conceal those details (at the cost of even more
complexity). On the upside, though, the parameterization doesn't have
to break anything - if module 1 imports one part of the package, and
module 2 imports another part, both parts will coexist nicely.

ChrisA

First off, thanks for the answer. I don't see the cached module as a problem here. If you provide arguments to a module, the goal is "most likely" to alter/parameterize the behavior of the first import. Now, I agree that behavior becomes unpredictable because passing different parameters on subsequent imports won't change the behavior of the imported module . One thing that could be done is to memoize those imports based on the parameters (this is a huge change to how imports works so its a bit yikes). Another solution would be to simply not cache parameterized imports. You sacrifice performance for added flexibility which I think is fair. I do however agree about partial imports. It would be rather complex to properly pass the context but its certainly doable.

On Sat, Jul 31, 2021 at 5:11 AM Charles Smith <charles@sollum.tech> wrote:
>
> First off, thanks for the answer. I don't see the cached module as a problem here. If you provide arguments to a module, the goal is "most likely" to alter/parameterize the behavior of the first import. Now, I agree that behavior becomes unpredictable because passing different parameters on subsequent imports won't change the behavior of the imported module .
>

The trouble is that if any other module has already imported the one
you're trying to parameterize, then your code won't know that the
parameters haven't been applied. It's action at a distance.

> One thing that could be done is to memoize those imports based on the parameters (this is a huge change to how imports works so its a bit yikes).

That's functionally equivalent to importing a class and then
instantiating, but with an automated cache. Could be interesting, but
belongs inside the module, not as a language mechanic, IMO.

> Another solution would be to simply not cache parameterized imports. You sacrifice performance for added flexibility which I think is fair.

And that's functionally equivalent to importing a class and then
instantiating, *without* the automated cache. :)

> I do however agree about partial imports. It would be rather complex to properly pass the context but its certainly doable.
>

Yeah.

So here's a possibility. Craft a module-level __getattr__ function
that does the caching behaviour you want. Then you can "from
modulename import Thing", where the precise name you want is the
selection of parameter. The __getattr__ function can also set it as an
intrinsic attribute (thus offering high performance caching - it'll
bypass __getattr_ the second time). Other than that, though, there's
not really a lot of parameterization that would make sense as part of
the import (if you need to say something like "import random;
random.seed(12345)", that's not really going to fit into the import
statement).

ChrisA