Packages, Dependencies, And Architecture

As soon as your application gets a bit bigger, you'll probably divide it into subsystems. In Python, this usually means packages. How well the boundaries between those packages are defined and how the communication flows between them greatly influences the maintainability of your project.

In this blog post, we're going to:

1. Visualize the optimal flow of communication between packages.
2. Phrase some rules (in English) about the dependencies between those packages based on the visual.
3. Translate the rules into code, so that we can check how much our application follows the optimal architecture.

Following this example, you can create architecture rules for your own projects.

The Example API

We're using a FastAPI application as an example. It stores its data in a PostgreSQL database via SQLAlchemy. It's divided into 4 packages:

• app.api
• app.core
• app.db
• app.schemas

Visualizing the Communication Between Packages

How do these packages communicate with each other? And with the external libraries powering them?

This might take many different forms. One possible architecture is this:

But it can also look something like this:

You can make things even more complicated by introducing circular dependencies:

While Python doesn't allow circular dependencies between modules, it won't stop you from introducing circular dependencies between packages.

The first architecture looks more simple. And this has some advantages:

• It's easier to understand and reason about.
• It's easier to see the consequences of a change.
• It's easier to replace a single element of the application.

Defining the Dependency Rules

Now, we have a picture of how these packages should communicate. Let's define some rules to achieve this architecture. First, we'll establish some rules on how the packages of our application communicate with each other:

• No other package depends on api.
• Only api depends core.
• Only core depends on db.

Let's turn to the external dependencies. We want to ensure that each of them is accessed only by the relevant package of our system:

• FastAPI from api
• SQLAlchemy from db
• Pydantic from models

Note that you probably don't want to define such a rule for every single dependency. But for libraries that are a crucial part of the architecture, it's a good idea to keep the interface clean.

Express the Dependency Rules as Code

So far, we've defined the dependency rules of our system: in a picture and in some sentences. Let's see how it matches up with the reality of our system. 😏 For that, we'll take the rules from the previous section and translate them into code.

For this translation, we'll use the tool Sourcery Rules Generator It creates Sourcery's custom rules in YAML format based on your input. You can install it from PyPI:

pip install sourcery-rules-generator

Setting Up Our Boundaries

Let's start with the first rule:

No other package depends on api.

To create such a rule about dependencies, run the command:

sourcery-rules dependencies create

You'll see:

• a brief introduction to the Dependencies rule template
• a prompt where you can enter the package's name
• after that, a second prompt where you can enter which packages are allowed to import the first package

Provide the parameters, so that the dependency rules can be generated:

• In the first prompt, enter the fully qualified name of the package: app.api
• Leave the second prompt empty. (Because no package is allowed to import app.api.)

Now, you'll get an output with 2 rules in YAML format:

• one detecting import statements
• one detecting from ... import statements

These rules detect if app.api or any of its subpackages gets imported within your project.

rules:
  - id: dependency-rules-app-api-import
    description: Do not import `app.api` in other packages
    pattern: import ..., ${module}, ...
    condition: module.matches_regex(r"^app\.api\b")
    paths:
      exclude:
        - app/api/
        - tests/
    tags:
      - architecture
      - dependencies
  - id: dependency-rules-app-api-from
    description: Do not import `app.api` in other packages
    pattern: from ${module} import ...
    condition: module.matches_regex(r"^app\.api\b")
    paths:
      exclude:
        - app/api/
        - tests/
    tags:
      - architecture
      - dependencies

What do these rules contain?

• A pattern with the syntax that we want to detect.
• A condition ensuring that the rule detects only imports referencing the app.api package.
• Some excluded paths: It's OK to reference the app.api package in itself and in tests.
• The architecture and dependencies tags. This way, we can check for all the architecture rules together.

You can copy these rules into your projects .sourcery.yaml config file. For that it's recommended to use the --plain option:

sourcery-rules dependencies create --plain

Now, you can run the sourcery review command of the Sourcery CLI to check for these rules:

sourcery review --enable architecture .

Further Rules for Internal Dependencies

Now, we can create similar Sourcery custom rules for our other internal dependency rules.

Only api depends core.

Again, we run the same command:

sourcery-rules dependencies create

This time, we fill both parameters:

• app.core is the package
• app.api is the only package importing it

Again, 2 rules are created:

rules:
  - id: dependency-rules-app-core-import
    description: Only `app.api` should import `app.core`
    pattern: import ..., ${module}, ...
    condition: module.matches_regex(r"^app\.core\b")
    paths:
      exclude:
        - app/core/
        - tests/
        - app/api/
    tags:
      - architecture
      - dependencies
  - id: dependency-rules-app-core-from
    description: Only `app.api` should import `app.core`
    pattern: from ${module} import ...
    condition: module.matches_regex(r"^app\.core\b")
    paths:
      exclude:
        - app/core/
        - tests/
        - app/api/
    tags:
      - architecture
      - dependencies

You can also create similar rules ensuring that only app.core should import app.db.

Rules for External Dependencies

Let's turn to the rules regarding the external dependencies:

• Only api depends on FastAPI
• Only db depends on SQLAlchemy
• Only models depends on Pydantic

We can create these rules the same way we did for the internal dependencies. This time, the package won't be a package in our project, but an external dependency.

Only app.api imports FastAPI.

sourcery-rules dependencies create

This time, we fill both parameters:

• fastapi is the package
• app.api is the only package importing it

Again, the output shows 2 rules:

rules:
  - id: dependency-rules-fastapi-import
    description: Only `app.api` should import `fastapi`
    pattern: import ..., ${module}, ...
    condition: module.matches_regex(r"^fastapi\b")
    paths:
      exclude:
        - fastapi/
        - tests/
        - app/api/
    tags:
      - architecture
      - dependencies
  - id: dependency-rules-fastapi-from
    description: Only `app.api` should import `fastapi`
    pattern: from ${module} import ...
    condition: module.matches_regex(r"^fastapi\b")
    paths:
      exclude:
        - fastapi/
        - tests/
        - app/api/
    tags:
      - architecture
      - dependencies

You can create rules in the same way for the 2 other external dependencies as well:

• only app.db imports SQLAlchemy
• only app.models imports Pydantic

Now, we have 12 Sourcery custom rules to express our 6 architecture rules. We can run all of them with the sourcery review command in the Sourcery CLI:

sourcery review --enable architecture .

Generalization: Create Architecture Rules in Your Project

Following the examples above, you can create rules for dependencies in your own projects. The recommended steps:

1. Draw a diagram showing the optimal dependencies between your packages.
2. Phrase some rules in a human language based on the diagram: Which package should depend on which?
3. Translate the rules into code with Sourcery custom rules with Sourcery Rules Generator.

If you're working on a huge application, don't try to tackle the whole architecture at once. Pick a subset of packages and go through the 3 steps with this smaller scope.

You can also check out the how-to guide Establish Rules for Dependencies Between Your Packages in the Sourcery docs.

Have you created architecture rules for your system? Do you have any questions, thoughts on this? Let us know.

Reach out at hello@sourcery.ai or on Twitter @SourceryAI. Join the Sourcery Discord Community.

Related Blog Posts

• Law of Demeter, Gateway Pattern, and Package Maturity
• Consistent Naming with Voldemort Rules