245 lines
8.5 KiB
Plaintext
245 lines
8.5 KiB
Plaintext
![]() |
Metadata-Version: 2.1
|
|||
|
Name: publication
|
|||
|
Version: 0.0.3
|
|||
|
Summary: Publication helps you maintain public-api-friendly modules by preventing unintentional access to private implementation details via introspection.
|
|||
|
Home-page: https://github.com/glyph/publication
|
|||
|
License: UNKNOWN
|
|||
|
Author: Glyph
|
|||
|
Author-email: glyph@twistedmatrix.com
|
|||
|
Description-Content-Type: text/x-rst
|
|||
|
Classifier: License :: OSI Approved :: MIT License
|
|||
|
Classifier: Programming Language :: Python :: 3
|
|||
|
Classifier: Programming Language :: Python :: 2
|
|||
|
|
|||
|
What is this?
|
|||
|
=============
|
|||
|
|
|||
|
Setting expectations around what APIs you can rely on in a Python
|
|||
|
library is very difficult.
|
|||
|
|
|||
|
Publication makes it easy.
|
|||
|
|
|||
|
The Problem
|
|||
|
-----------
|
|||
|
|
|||
|
As `Hyrum's Law <http://www.hyrumslaw.com>`_ somewhat grimly states,
|
|||
|
|
|||
|
| With a sufficient number of users of an API,
|
|||
|
| it does not matter what you promise in the contract:
|
|||
|
| all observable behaviors of your system
|
|||
|
| will be depended on by somebody.
|
|||
|
|
|||
|
In general, Python famously has a somewhat different philosophical view of this
|
|||
|
reality. We assume each other to be `responsible users
|
|||
|
<https://github.com/realpython/python-guide/pull/524/files>`_ of the libraries
|
|||
|
we consume. Mucking with implementation details might break every time you
|
|||
|
upgrade, but it's sufficiently useful for testing, debugging, and
|
|||
|
experimentation that retaining that ability is worth paying the cost.
|
|||
|
|
|||
|
But, critical to this assumption is that everybody *knows* when they're
|
|||
|
breaking into the "private" area of the library's interface. Here, there's a
|
|||
|
mismatch of expectations:
|
|||
|
|
|||
|
- *library authors* write documentation, and then think that users sit down and
|
|||
|
read the documentation, front to back, and learn about what the "public"
|
|||
|
interface is by doing so. they then assume that users will know that they've
|
|||
|
used private implementation details if they ever deviate from these
|
|||
|
documented features.
|
|||
|
- *library users* ``pip install`` a thing, open up a REPL, import the module,
|
|||
|
and discover the library by doing ``dir()`` on the module and its contents,
|
|||
|
assuming that their program is not using any private implementation details
|
|||
|
as long as they never had to type ``library._something_private()`` while
|
|||
|
doing so. If they ever encounter a traceback they may consult the
|
|||
|
documentation, briefly, until it is resolved.
|
|||
|
|
|||
|
Publication makes it possible to align the wildly divergent expectations of
|
|||
|
these groups, so that users can still get the benefits of being able to use
|
|||
|
internal details if they want, but they'll know that they're doing so. It
|
|||
|
makes the runtime namespace of your module look like the public documentation
|
|||
|
of your library.
|
|||
|
|
|||
|
How does this look in practice?
|
|||
|
-------------------------------
|
|||
|
|
|||
|
You, a prospective library author, want to write a library that makes it easy to zorf a sprocket.
|
|||
|
Great! You do, and it looks like this:
|
|||
|
|
|||
|
.. code:: python
|
|||
|
|
|||
|
# sprocket_zorfer.py
|
|||
|
from sprocket import sprocket_with_name
|
|||
|
from zorf import zorfable_thing
|
|||
|
|
|||
|
def zorf_sprocket_internal(sprocket, zorfulations):
|
|||
|
...
|
|||
|
def compute_zorfulations():
|
|||
|
...
|
|||
|
|
|||
|
def zorf_sprocket_named(sprocket_name, how_much):
|
|||
|
sprocket = sprocket_with_name(sprocket_name)
|
|||
|
zorfulations = compute_zorfulations(how_much)
|
|||
|
return zorf_sprocket_internal(sprocket, zorfulations)
|
|||
|
|
|||
|
|
|||
|
__all__ = [
|
|||
|
'zorf_sprocket_named'
|
|||
|
]
|
|||
|
|
|||
|
Your intent here, of course, is that you have exposed a module with a
|
|||
|
single function: ``zorf_sprocket_named``, and everything else is an
|
|||
|
implementation detail. You even said so, explicitly, with ``__all__``.
|
|||
|
Your API documentation says the same.
|
|||
|
|
|||
|
However, reading reference documentation and cleanly respecting
|
|||
|
conventions is not how working programmers really figure out how to use
|
|||
|
stuff. Your users all do stuff like:
|
|||
|
|
|||
|
- Load up an interactive ``python`` interpreter and call ``dir()`` on
|
|||
|
your module
|
|||
|
- Install Jupyter and tab-complete their way around your module to find
|
|||
|
what they want
|
|||
|
- use the auto-import function in PyCharm to grab some private
|
|||
|
implementation detail
|
|||
|
|
|||
|
and, before you know it, you have thousands of users of your library
|
|||
|
with code like
|
|||
|
|
|||
|
.. code:: python
|
|||
|
|
|||
|
from sprocket_zorfer import compute_zorfulations, zorf_sprocket_internal, sprocket_with_name
|
|||
|
|
|||
|
sprocket = sprocket_with_name(name)
|
|||
|
zorf_sprocket_internal(sprocket, compute_zorfulations(7) * 2)
|
|||
|
|
|||
|
Now you can never change *any* of your implementation details! Worse
|
|||
|
yet, ``sprocket_with_name`` isn’t even your own code; that’s something
|
|||
|
you got from a library! But when someone does
|
|||
|
``import sprocket_zorfer; sprocket_zorfer.<tab>`` in an interactive
|
|||
|
shell, none of that information comes through.
|
|||
|
|
|||
|
Underscore Paranoia
|
|||
|
-------------------
|
|||
|
|
|||
|
The convention in Python is that we use ``_`` to indicate private names.
|
|||
|
So when we library authors notice this problem starting to happen, a
|
|||
|
common reaction is to start putting ``_`` in front of *everything* –
|
|||
|
class names, function names, module names – and only explicitly export
|
|||
|
those things that should be public by “moving” them via an import and an
|
|||
|
entry in a public module’s ``__all__``.
|
|||
|
|
|||
|
However, this has a bunch of disadvantages:
|
|||
|
|
|||
|
- Most code inspection tooling and IDEs won’t see that the public name
|
|||
|
is “moved”, so code exploration just makes it seem like *everything*
|
|||
|
is an implementation detail now, rather than making it seem like
|
|||
|
nothing is.
|
|||
|
|
|||
|
- All your ``__repr__``\ s now have ugly and inaccurate function and
|
|||
|
class names in them, at least from the perspective of your users; how
|
|||
|
are they supposed to know ``zorf_sprocket_named`` is actually defined
|
|||
|
in ``zorf_sprocket._impl_details.funcs._zorf_sprocket_public`` now?
|
|||
|
How are they supposed to find the good, public name once they’re
|
|||
|
looking at the goofy internal one?
|
|||
|
|
|||
|
- You constantly need to remember to put *all* of your code in these
|
|||
|
ugly ``_``-prefixed modules, and educate new contributors as to the
|
|||
|
risks of creating new modules in your package that are not carefully
|
|||
|
hidden away from public users.
|
|||
|
|
|||
|
A Better World
|
|||
|
--------------
|
|||
|
|
|||
|
What if you could write all your code *as if* it were just regular
|
|||
|
public code, and have all your implementation details and imports
|
|||
|
automatically squirreled away in an underscore namespace so that curious
|
|||
|
coders won’t accidentally find every module you ever imported and every
|
|||
|
temporary helper function you ever defined and think they’re part of the
|
|||
|
permanent public face of your library?
|
|||
|
|
|||
|
Enter ``publication``.
|
|||
|
|
|||
|
``publication`` uses the existing convention of ``__all__`` and a little
|
|||
|
runtime hackery to hide everything that you have not marked as
|
|||
|
explicitly public, like so:
|
|||
|
|
|||
|
.. code:: python
|
|||
|
|
|||
|
# sprocket_zorfer.py
|
|||
|
|
|||
|
from publication import publish
|
|||
|
|
|||
|
from sprocket import sprocket_with_name
|
|||
|
from zorf import zorfable_thing
|
|||
|
|
|||
|
def zorf_sprocket_internal(sprocket, zorfulations):
|
|||
|
...
|
|||
|
def compute_zorfulations():
|
|||
|
...
|
|||
|
|
|||
|
def zorf_sprocket_named(sprocket_name, how_much):
|
|||
|
sprocket = sprocket_with_name(sprocket_name)
|
|||
|
zorfulations = compute_zorfulations(how_much)
|
|||
|
return zorf_sprocket_internal(sprocket, zorfulations)
|
|||
|
|
|||
|
|
|||
|
__all__ = [
|
|||
|
'zorf_sprocket_named'
|
|||
|
]
|
|||
|
|
|||
|
publish()
|
|||
|
|
|||
|
That’s it! Now, ``from sprocket_zorfer import zorf_sprocket_named``
|
|||
|
works as intended, but
|
|||
|
``from sprocket_zorfer import compute_zorfulations`` is an
|
|||
|
``ImportError``.
|
|||
|
|
|||
|
But what about…
|
|||
|
---------------
|
|||
|
|
|||
|
Other modules in my package, like tests, that need to peek at implementation details?
|
|||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|||
|
|
|||
|
Don’t worry, your code didn’t go anywhere. The original module is still
|
|||
|
available as a special pseudo-module called ``<your_module>._private``.
|
|||
|
In the example above, ``sprocket_zorfer.py``\ ’s tests can still do:
|
|||
|
|
|||
|
.. code:: python
|
|||
|
|
|||
|
from sprocket_zorfer._private import compute_zorfulations
|
|||
|
|
|||
|
def test_compute_zorfulations():
|
|||
|
assert compute_zorfulations(0) > 7
|
|||
|
|
|||
|
Mypy?
|
|||
|
~~~~~
|
|||
|
|
|||
|
Your types should *probably* just be part of your published API, if
|
|||
|
you’re expecting that users will need to know about them. But, if there
|
|||
|
are cases which need to be type-checked internally in your library, as
|
|||
|
far as Mypy is concerned, all your private classes are still there. So,
|
|||
|
in the simple case you can just do this:
|
|||
|
|
|||
|
.. code:: python
|
|||
|
|
|||
|
from typing import TYPE_CHECKING
|
|||
|
if TYPE_CHECKING:
|
|||
|
from something import T
|
|||
|
|
|||
|
def returns_a() -> "T":
|
|||
|
...
|
|||
|
|
|||
|
and in the hopefully very unusual case you need to mix runtime and
|
|||
|
type-checking access to a different module’s private details,
|
|||
|
|
|||
|
.. code:: python
|
|||
|
|
|||
|
from typing import TYPE_CHECKING
|
|||
|
if TYPE_CHECKING:
|
|||
|
from something import T
|
|||
|
else:
|
|||
|
from something._private import T
|
|||
|
|
|||
|
def returns_a() -> A:
|
|||
|
...
|
|||
|
|