Components system?

Hello!

Thank you for creating Starlette! I really like what I’m seeing so far and have high hopes for the future. :heart:

I would like to kick off the discussion about the possible implementation of reusable components system for Starlette. I feel that the framework should offer some sort of contract for developers to create components that may then be shared and used by others to add features to their applications.

The advantages of such a system are obvious, and it’s easy to come up with examples from other established frameworks that implement it in one form or another:

  • Django Apps
  • Rails Engines
  • Symfony Bundles
  • Laravel Packages

I’ve been musing about this for the last few days and I’ve came up with some ideas about how such system should’ve been implemented or behave:

Opt-in

From my understanding Starlette aims to be a small framework that is a piece of larger, “opinionated enough” collection of optional components that will enable developers to get up to speed quickly. Likewise, if they don’t like one of the components, they can roll in custom solution instead.

Following this approach, the components system itself could be an optional component. It would either live as a separate Python package installable from PyPi, or module in Starlette (eg. starlette.components).

from components import Components

components = Components(["component_a", "component_b"])

Unopinionated

Component system should make no assumptions about the structure of components, and instead focus on providing utilities that enable developers to implement custom contracts for components. It would be layer of abstraction above the importlib and os.path:

# Some feature imports entire modules
for component, signals in components.import("signals"):
    if signals:
        logger.info(`Registered signals for ${component}`)
    else:
        logger.info(`${component} has no signals`)

# Other feature imports names from modules
for component, migration in components.import("migrations.Migration"):
    if migration:
        db.run_migration(component, migration)
    else:
        ...  # somehow handle missing migration

# Different one just scans those for directories
for component, manifest in components.files("manifest.json"):
    if manifest:
        app_manifest.extend(manifest)
    else:
        ...  # component has no manifest.json

# Different one just scans those for directories
for component, static_dir in components.dirs("static"):
    if static_dir:
        static_dirs.append(static_dir)
    else:
        ...  # component has no static dirs

Foundation for future contracts

Starlette (and friends) could use the component system to provide modularization features:

from databases import migrate
from .components import components

migrate(components)

Or:

from starlette.routing import Router, Mount
from starlette.staticfiles import StaticFiles
from .components import components

app = Router(routes=[
    Mount('/static', app=StaticFiles(components.dirs("static")), name="static"),
])

Just food for tough here. There still some things that need working out:

Settle on semantics and location. Should it be components, extensions? Should it be a standalone package or live under Starlette (eg. starlette.extensions)? The former will be safer, but the latter will encourage adoption by developers.

Return lists, tuples or generators?

Import __init__'s on Components() initialization, lazily on first call to one of getters, or by explicit method? Lazy loading would enable us to provider Components.add("other_component"), but may be overcomplicating things

Should component part of tuple returned by getter be plain string, or Component instance with some utils on its own? Perhaps we could skip utils on Components altogether and move those to Component class? That would make our API more open:

for component in components:
    static_dir = os.path.join(component.dirname, "statics")
    ...  # do something

for component in components:
    try:
        migration = getattr(component.import("migrations"), "Migration")
    except ImportError:
        pass

Thats it for now. Let me know what you guys think about it. I’ll be happy to provide implementation, tests and docs if there’s an interest for it.

Hey!

When APIStar changed after v0.5.0 from a framework to an API tool I started developing a framework on top of Starlette aiming to include all the functionality provided by APIStar (components, dependency injection, api schema and auto-generated docs) as well as some new features like generic views for creating REST resources.

This framework is named Flama and here you can find the project source code and the docs.

Our tooklit around starlette is based on the zope component system + some decorators.
Most of the functionality comes from zope.interface, but we just ported the component registry from guillotina.
The main advantage of this kind of architecthure, if that you can override anything from anywhere. Plone, it’s based on this component architecture, and the amount of addons, plugins, an extenders available are impressive.

Anyway, as a django developer also, I know, the app model from django (it’s so tied and not so usable… so many times you end monkey patching things on thirth party apps)

Thanks for raising this! From my POV I don’t feel like I know enough yet about exactly what we’d need a components API to provide in order to comment properly on this.

I guess I’d probably like to see some ideas pushed out as third-party packages to start with.