The registries pattern

published on Jan. 10, 2016, 5:54 a.m. by eliotberriot | 2

I've been using this pattern for many years in my applications, and I always found strange nobody ever mentioned it. Since it has prove useful to me in many different projects, I think it's a perfect occasion to put some life again in my blog.

Use case

The problem solved by registries could be expressed this way:

Assuming I have a project splitted in many, specific apps, how can these apps communicate together when they need to share transversal data or logic?

I'm working with Python on a daily basis, and more specifically on web applications using the Django framework, so I'll target this ecosystem in my examples.

Existing implementations

If you've used Django, it's almost certain you already manipulated some registries.

Let's take one of the most trivial task in Django, declaring an admin module.

First of all, let's remember what the admin app does: it provides an easy way to generate a web interface to do CRUD (Create/Read/Update/Delete) actions on our project models. Now, how can we tell Django that we want an admin interface for a specific model, without messing with the admin app code?

We're writing a simple blog engine, so our models and admin files would look like this:

# app1/models.py

from django.db import models

class BlogPost(models.Model):
    title = models.CharField(max_length=50)
    content = models.TextField()

# app1/admin.py

from django.contrib import admin
from .models import BlogPost

class BlogPostAdmin(admin.ModelAdmin):
    pass

admin.site.register(BlogPost, BlogPostAdmin)

Let's explain what the previous lines do:

  1. First, we create a ModelAdmin class. This class will be used to generate an admin interface for our model
  2. Then, via the admin.site.register function, we tell Django: "Hey! I've written an admin module for my model, would you be nice and bind it to my BlogPost model?"

The admin-related code in a Django project goes into an admin.py file inside our app. A project layout could look like:

project/
??? app1
?   ??? __init__.py
?   ??? admin.py
?   ??? models.py
??? app2
    ??? __init__.py
    ??? admin.py
    ??? models.py

Under the hood, when we start our Django server, all admin.py files are imported by the framework, and the admin.site registry is populated with the Model/ModelAdmin bindings we declared. Then, if we visited the admin url in a web browser, we would see all these ModelAdmin.

As you can see, this is a really elegant way to solve the problem: we write our app-specific ModelAdmin in dedicated files, and we inject them inside the admin app in a single line. All admin.py belonging to installed apps are, by convention, discovered and imported automatically by the framework. When the admin interface is displayed, the admin app loops on the registry and uses the bindings to build everything.

A similar pattern is used in various other apps, such as DjangoCMS or django-haystack, even if he registration step is not always explicit. In haystack, for example, there is no register() function, you just need to put your logic inside search_indexes.py files to trigger the registration. Besides that, the whole process is similar.

Leveraging the power of registries

Now, how can we use registries in our projects and, more important, why would we do that?

Imagine you're building a social website with Django. You're website will have a blog, a forum where people would be able to signup and login, and even a basic store to buy goodies and other products.

When building a website, one of the most common brick is the navigation menu. If we write the template for our website menu, we should end up with something like:

<nav class="main-menu">
    <ul>
        <li><a href="{% url 'index' %}">Home</a></li>
        <li><a href="{% url 'blog:index' %}">Blog</a></li>
        <li><a href="{% url 'forum:index' %}">Forum</a></li>
        <li><a href="{% url 'store:index' %}">Store</a></li>
    </ul>
</nav>

We would write this inside a menu.html file in your templates directory, and include this in our base template. It's okay, our menu is quite simple, no big deal.

But then, as time goes, we will add some link to this menu. A login link maybe, and a signup link. And we will want to hide the login and signup links if the user is logged in. And we'll add other apps to our project, that also need their menu links. And we'll want different menus, depending on which section of our website is displayed.

Rapidly, our template will become a horrible mess, full of if clauses, mixing presentation and logic, in short, an unmaintainable, ugly beast.

Let's rephrase our issue: our apps are domain-specific (user, forum, blog, store...), but, somehow, they are all involved in the rendering process of any page of our website, because of the navigation menu. We must find a way to let them plug some logic on the navigation menu template.

Maybe registries can help? ;)

(I wrote a more complete solution for this specific issue, using registries, in a dedicated Django app, named django-navutils, check it out if you're interested!)

Implementing a registry

Here is our project layout:

project/
??? blog
?   ??? __init__.py
?   ??? menu.py
?   ??? models.py
?   ??? views.py
??? forum
?   ??? __init__.py
?   ??? menu.py
?   ??? models.py
?   ??? views.py
??? manage.py
??? project
?   ??? apps.py
?   ??? context_processors.py
?   ??? __init__.py
?   ??? settings.py
?   ??? urls.py
?   ??? utils.py
??? store
?   ??? __init__.py
?   ??? menu.py
?   ??? models.py
?   ??? views.py
??? templates
?   ??? base.html
?   ??? menu.html
??? users
    ??? __init__.py
    ??? menu.py
    ??? models.py
    ??? views.py

The implementation of a registry is quite simple. First you need a method to import arbitrary modules by name. Then you need a method to register data in the registry.

I've already released a real, generic, implementation of a registry, named persisting-theory. The whole thing is available on PyPi and you are free to use it. The following code is heavily inspired from the work I've already done on this package.

So, let's write our registry base class:

from collections import OrderedDict

class Registry(OrderedDict):
    """Our registry use OrderedDict insted of dict as a base class, because I think maintaining the order
    in which data is registered may be useful. Feel free to use dict instead"""

    def autodiscover(self, apps):
        for app in apps:
            try:
                app_package = __import__(app)
                package = '{0}.{1}'.format(app, self.look_into)
                module = __import__(package)
            except ImportError:
                pass

    def register(self, name, data):
        self[name] = data

Done! Of course, it's too naive to be used in production, but this basic prototype will work for our example.

Now, let's create our menu registry:

class MenuRegistry(Registry):
    look_into = 'menu' # the path that our registry will try to import inside each app

menu = MenuRegistry()

You can place this code anywhere, but putting it inside a project/registries.py file seems logic to me. You could also create a menu app for handling all menu-related code.

After that, we write the menu logic:

# project/utils.py

class MenuNode(object):
    def __init__(self, label, url, require_authentication=False):
        self.label = label
        self.url = url
        self.require_authentication = require_authentication

And the template:

<!-- templates/menu.html -->
<nav class="main-menu">
    <ul>
        {% for node in menu_nodes %}
            {% if not node.require_authentication %}
                <li><a href="{% url node.url %}">{{ node.label }}</a></li>
            {% elif node.require_authentication and request.user.is_authenticated %}
                <li><a href="{% url node.url %}">{{ node.label }}</a></li>
            {% endif %}
        {% endfor %}
    </ul>
</nav>

We also add a context processor to pass our menu nodes to the template context:

# project/context_processors.py

from .registries import menu

def menu_nodes(request):
    # remember to add this in your settings.TEMPALTES['context_processors']
    return {'menu_nodes': menu.values()}

And we trigger the autodiscovering process when all apps are ready:

# project/apps.py

from django.apps import AppConfig
from django.conf import settings
from .registries import menu

class ProjectConfig(AppConfig):
    """
    We use Django's built-in app system to trigger autodiscovering when all apps are loaded.
    Remember to add this to `project/__init__.py`:

        default_app_config = 'project.apps.ProjectConfig'
    """
    name = 'project'

    def ready(self):
        # what do we do here? We tell our registry to loop on all the installed apps
        # and to import the menu package, if any"
        menu.autodiscover(settings.INSTALED_APPS)

Here we go. The last thing to do is to register our actual menu nodes:

from project.utils import MenuNode
from project.registries import menu

# project/menu.py
menu.register('index', MenuNode('Home', 'index'))

# blog/menu.py
menu.register('blog', MenuNode('Blog', 'blog:index'))

# forum/menu.py
menu.register('forum_index', MenuNode('Forum', 'forum:index'))

# store/menu.py
menu.register('store_index', MenuNode('Store', 'store:index'))
menu.register('store_basket', MenuNode('Basket', 'store:basket', require_authentication=True))

# users/menu.py
menu.register('login', MenuNode('Login', 'account_login'))
menu.register('signup', MenuNode('Signup', 'account_signup'))
menu.register('settings', MenuNode('Account settings', 'account_settings'))

Voilà! Our registry-based menu is working, assuming corresponding URLs exist in your project. We'll never have to edit our template again when we want to add, edit or remove a menu label. As a bonus, all our menu nodes are stored in the corresponding app directory, as plain Python objects, instead of being all mixed up inside one template file.

We could improve these objects to handle better permission management (e.g. display nodes to anonymous users or staff users only) or display (by adding a css_class argument, or even use custom templates to render nodes), but that's not in the scope of the current post.

Conclusion

It's often overkill to create your own registries if you're working on a small project with only a few apps. But if it grows, I bet you'll miss them.

Also remember that registries really shine when you're writing reusable apps. You're developping a generic, WordPress-like, shortcode app? At some point, you'll need some kind of registry structure to let apps declare and register their own shortcodes.

Because they encourage loose-coupling between apps, and provide an easy to use API, registries are usually a good choice to pick when you face a transversal problematic that involves many different apps, or want to write something that can be easily extended. And because registries are regular data structures subclasses, you can manipulate them as any usual collections and still be able to implement custom behaviour, such as filtering, ordering or registered data validation.

I hope you found this post useful, happy registries!

2 comments

  • Nikkos - 7 months, 1 week ago

    Useful article, thanks. :-)

  • will - 6 months, 2 weeks ago

    Thanks for a nice article!

publish