Feast

Important Capabilities

Capability	Status	Notes
Descriptions	✅	Enabled by default
Schema Metadata	✅	Enabled by default
Table-Level Lineage	✅	Enabled by default

This plugin extracts:

• Entities as MLPrimaryKey
• Fields as MLFeature
• Feature views and on-demand feature views as MLFeatureTable
• Batch and stream source details as Dataset
• Column types associated with each entity and feature

CLI based Ingestion

Install the Plugin

The feast source works out of the box with acryl-datahub.

Starter Recipe

Check out the following recipe to get started with ingestion! See below for full configuration options.

For general pointers on writing and running a recipe, see our main recipe guide.

source:
  type: feast
  config:
    # Coordinates
    path: "/path/to/repository/"
    # Options
    environment: "PROD"

sink:
  # sink configs

Config Details

Options

Note that a . is used to denote nested fields in the YAML recipe.

Field	Description
path ✅ string	Path to Feast repository
enable_owner_extraction boolean	If this is disabled, then we NEVER try to map owners. If this is enabled, then owner_mappings is REQUIRED to extract ownership. Default: False
enable_tag_extraction boolean	If this is disabled, then we NEVER try to extract tags. Default: False
environment string	Environment to use when constructing URNs Default: PROD
fs_yaml_file string	Path to the feature_store.yaml file used to configure the feature store
owner_mappings array	Mapping of owner names to owner types
owner_mappings.map map(str,string)

Schema

The JSONSchema for this configuration is inlined below.

{
  "title": "FeastRepositorySourceConfig",
  "type": "object",
  "properties": {
    "path": {
      "title": "Path",
      "description": "Path to Feast repository",
      "type": "string"
    },
    "fs_yaml_file": {
      "title": "Fs Yaml File",
      "description": "Path to the `feature_store.yaml` file used to configure the feature store",
      "type": "string"
    },
    "environment": {
      "title": "Environment",
      "description": "Environment to use when constructing URNs",
      "default": "PROD",
      "type": "string"
    },
    "owner_mappings": {
      "title": "Owner Mappings",
      "description": "Mapping of owner names to owner types",
      "type": "array",
      "items": {
        "type": "object",
        "additionalProperties": {
          "type": "string"
        }
      }
    },
    "enable_owner_extraction": {
      "title": "Enable Owner Extraction",
      "description": "If this is disabled, then we NEVER try to map owners. If this is enabled, then owner_mappings is REQUIRED to extract ownership.",
      "default": false,
      "type": "boolean"
    },
    "enable_tag_extraction": {
      "title": "Enable Tag Extraction",
      "description": "If this is disabled, then we NEVER try to extract tags.",
      "default": false,
      "type": "boolean"
    }
  },
  "required": [
    "path"
  ],
  "additionalProperties": false
}

Code Coordinates

• Class Name: datahub.ingestion.source.feast.FeastRepositorySource
• Browse on GitHub

Questions

If you've got any questions on configuring ingestion for Feast, feel free to ping us on our Slack.