Announcing Qwak Feature Store support for Snowflake data sources

Exciting News: Qwak Feature Store Now Supports Snowflake Data Sources!
Pavel Klushin
Pavel Klushin
Head of Solution Architecture at Qwak
September 27, 2021
Table of contents
Announcing Qwak Feature Store support for Snowflake data sources

Qwak Feature Store provides a unified feature store for training and real-time operations without the need to write additional code or create manual processes to keep features consistent.  There are many ways to create features in the Qwak Feature Store, including Batch-based features, streaming or non-materialized features.

Starting today, Qwak supports Snowflake as a data source for Batch features

The Data Source connectors provide you with a consistent data source interface for any database and create a standard way to combine stream and batch data sources for Feature Transformations.

Integrating Qwak Feature Store and Snowflake

Defining a Feature Set enables you to create features from your analytical data: when calculating feature values, Qwak will simply read from the underlying data source.

For example, in a fraud detection model use case, we might have two values to pull from a Snowflake data source:

  • Average transaction per customer - avg_amount
  • Standard deviation of a transaction per customer - sttdev_amount

And two from Streaming events from Kafka:

  • Last transaction amount
  • Last transaction time

Architecture

Qwak feature store integrates with Snowflake

How to configure Snowflake as Data Source

Snowflake data source connector definition:


from qwak.feature_store.sources.data_sources import SnowflakeSource

some_snowflake_source = SnowflakeSource(name='users_demo',
                                        description='a snowflake source description',
                                        date_created_column='insert_date_column',
                                        host='Snowflake DNS address',
                                        username_secret_name='snowflake-username',
                                        password_secret_name='snowflake-password',
                                        database='db_name',
                                        schema='schema_name',
                                        warehouse='data_warehouse_name',
                                        sql='SQL query for feature extraction')

Register batch feature using the Snowflake connector:


BatchFeatureSet(
  name=”batch_transaction_features”,
  data_sources=[“snowflake_transactions_history”],
  scheduling_policy=”daily”,
  validations=[expect_column_values_to_be_between(
    column=”amount”, min_value=0, max_value=None)]
  function=SqlFunction(
    “””
    SELECT User_ID,
      avg(Amount) AS avg_amount,
      sttdev(Amount) AS stddev_amount
    FROM snowflake_trasactions_history
    GROUP BY User_ID
    “””)
)

Qwak Feature Store helps ensure models make accurate predictions by making the same features available for both training and inference. 

Chat with us to see the platform live and discover how we can help simplify your ML journey.

say goodbe to complex mlops with Qwak