Skip to main content

Creating a code location with components

info

This feature is still in development and might change in patch releases. It’s not production ready, and the documentation may also evolve. Stay tuned for updates.

Prerequisites

Before creating a project with components, you must follow the steps to install uv and dg.

After installing dependencies, you can scaffold a components-ready code location for your project. In the example below, we scaffold a code location called jaffle-platform:

dg scaffold code-location jaffle-platform
Creating a Dagster code location at /.../jaffle-platform.
Scaffolded files for Dagster project in /.../jaffle-platform.
...

This command builds a code location and initializes a new Python virtual environment inside of it. When using dg's default environment management behavior, you won't need to worry about activating this virtual environment yourself.

Overview of files and directories​

Let's have a look at the scaffolded files:

cd jaffle-platform && tree
.
├── jaffle_platform
│   ├── __init__.py
│   ├── components
│   ├── definitions.py
│   └── lib
│       └── __init__.py
├── jaffle_platform.egg-info
│   ├── PKG-INFO
│   ├── SOURCES.txt
│   ├── dependency_links.txt
│   ├── entry_points.txt
│   ├── requires.txt
│   └── top_level.txt
├── jaffle_platform_tests
│   └── __init__.py
├── pyproject.toml
└── uv.lock

6 directories, 12 files

You can see that we have a fairly standard Python project structure. The following files and directories are included:

  • A Python package jaffle_platform-- the name is an underscored inflection of the project root directory (jaffle_platform).
  • An (empty) jaffle_platform_tests test package
  • A uv.lock file
  • A pyproject.toml file

pyproject.toml​

The pyproject.toml contains a tool.dagster and tool.dg section that look like this:

jaffle-platform/pyproject.toml
...
[tool.dagster]
module_name = "jaffle_platform.definitions"
code_location_name = "jaffle-platform"

[tool.dg]
is_code_location = true
is_component_lib = true
...

tool.dagster section​

The tool.dagster section of pyproject.toml is not dg-specific. This section specifies that a set of definitions can be loaded from the jaffle_platform.definitions module.

tool.dg section​

The tool.dg section contains two settings requiring more explanation: is_code_location and is_component_lib.

is_code_location setting​

is_code_location = true specifies that this project is a dg-managed Dagster code location. Code locations created with components are regular Dagster code locations with a particular structure.

To understand the structure, let's look at the content of jaffle_platform/definitions.py:

jaffle-platform/jaffle_platform/definitions.py
from pathlib import Path

from dagster_components import build_component_defs

defs = build_component_defs(components_root=Path(__file__).parent / "components")

This call to build_component_defs will:

  • discover the set of components defined in the project
  • compute a set of Definitions from each component
  • merge the component-specific definitions into a single Definitions object

is_code_location is telling dg that the project is structured in this way and therefore contains component instances. In the current project, component instances will be placed in the default location at jaffle_platform/components.

is_component_lib setting​

is_component_lib = true specifies that the project is a component library. This means that the project may contain component types that can be referenced when generating component instances.

In a typical code location, most components are likely to be instances of types defined in external libraries (e.g. dagster-components), but you can also define custom component types scoped to your project. That is why is_component_lib is set to true by default. Any scaffolded component types in jaffle_platform will be placed in the default location at jaffle_platform/lib.

You can also see that this module is registered under the dagster.components entry point in pyproject.toml. This is what makes the components discoverable to dg:

jaffle-platform/pyproject.toml
...
[project.entry-points]
"dagster.components" = { jaffle_platform = "jaffle_platform.lib"}
...

Next steps​

After scaffolding your code location with components, you can add more components to complete your pipeline.