Patch logo

Patch Changelog

Jul

12

Python Data Packages

Announcement

Data Packages are code libraries with a live connection to an underlying data source. They provide a powerful interface for querying, access control, versioning, performance optimization and more; over all your data, no matter where it lives. This could be any database or file system.

The Data Package can be installed using a package manager like pip . Then, it's imported like a library dependency into your code, whether that’s a backend service performing machine learning or enrichment tasks, a customer-facing application, or even an external consumer buying access from you directly to build using the package.

A dpm-agent intelligently routes queries submitted by consumers of a Data Package to the appropriate backend source, enforces access policies and applies performance optimizations. 

Today, we're excited to announce support for generated Python packages!

Sign up for early access at www.dpm.sh!

Jul

05

TypeScript / Node.js Data Packages

Announcement

Data Packages are code libraries with a live connection to an underlying data source. They provide a powerful interface for querying, access control, versioning, performance optimization and more; over all your data, no matter where it lives. This could be any database or file system.

The Data Package is imported like a library dependency into your code, whether that’s a backend service performing machine learning or enrichment tasks, a customer-facing application, or even an external consumer buying access from you directly to build using the package.

A dpm-agent intelligently routes queries submitted by consumers of a Data Package to the appropriate backend source, enforces access policies and applies performance optimizations. 

Today, we're excited to announce support for generated Node.js & TypeScript packages!

Feb

28

Advanced query filters: support for AND and OR operators

Data API

We're excited to announce a major update to our platform! We've added support for GraphQL queries with Boolean filters, which will give users the ability to apply AND and OR operators across filters, including nesting. Check out our docs for more info and examples!

Ex: "items that have 'shirt' and 'cotton' in the name, OR items that are of type 'top' and season 'summer'"

itemsQuery(filter: {
    or: [
        { and: [{ name: { like: "%shirt%" } }, { name: { like: "%cotton%" } }] },
        { and: [{ item_type: { eq: "top" } }, { season: { eq: "summer" } }] }
    ]
} ) { ... }

Feb

16

Feb

02

Dec '22

15

Row level access policies with custom signed JWTs

Data API

Many teams have multiple customers' data in the same table. In a customer-facing feature, you only want to display that particular customer's data.  To ensure you can safely limit customer access to the right data, you can now rely on row-level access controls encoded in JWTs.  Patch can issue JWTs or respect ones that you supply with the filters encoded.

$ pat access signing-template create <name> <JWK file> --source snowflake_prod --dataset exec_dashboard --filters user_events:account_id:384752

Nov '22

07

Aggregating by a column's median

Data API

When querying datasets, you've always been able to use common aggregations like avg, sum, max, and min. Now, you can also aggregate by any numeric column's median. Check out our docs for more info.

query {
  engagementTableQuery(
    filter: {
      videoViewStartTime: {
        after: "2022-05-31T00:00:00"
        before: "2022-11-01T00:00:00"
      }
    }
    groupBy: [videoViewStartTimeYear, videoViewStartTimeMonth]
    orderBy: [
      { videoViewStartTimeYear: asc }
      { videoViewStartTimeMonth: asc }
    ]
    limit: 1000
    offset: 0
  ) {
    videoViewStartTimeYear
    videoViewStartTimeMonth
    videoViewDurationMedian
  }
}

Oct '22

05

Support for multi-select filtering

Data API

Today, we introduced support for ARRAY columns. This means you can run queries that return ARRAY column values, but also write filters that check if a column hasAny or hasAll values in an array you specify. This is helpful when supporting features like multi-select filters, including the case when you are aggregating the result set.

query {
  eventsTableQuery(
    filter: {tags:{hasAny: ["prod", "bot_user"]}}
    groupBy: [eventType, eventCreatedTimeDate]
  ) {
    eventType
    eventCount
    eventCreatedTimeDate
  }
}

Sep '22

30

Support for Rolling Time Windows

Data API

When querying datasets, you can now look up data for the "last 30 days" or "between 7 and 14 days ago". Simply specify a filter on a timestamp or datetime column with the newerThan and/or olderThan operators. Check out our docs here!

query {
  eventsTableQuery(
    filter: {
      eventCreatedTime: {
        newerThan: {days: 7},
          olderThan: {days: 3}
      }
    }
    orderBy: {eventCreatedTimeDate: desc}
    groupBy: [eventType, eventCreatedTimeDate]
  ) {
    eventType
    eventTypeCount
    eventCreatedTimeDate
  }
}

Sep '22

26

GraphQL Data APIs: Support for rollups by Week

Data API

When querying time series datasets, you can now easily group data by week. A new derived field <columnName>WeekDate will return the first day of the week the time value falls within. With an ISO-standard default of MONDAY, the firstDayOfWeek is configurable with an options configuration parameter.

query {
  <tableName>Query(
    groupBy: [eventCreatedTimeWeekDate]
    orderBy: {eventCreatedTimeWeekDate:desc}
    options: {firstDayOfWeek: SUNDAY}
  ) {
    eventCount
    eventCreatedTimeWeekDate
  }
}

Sep '22

08

Support for groupBy and time series aggregations in GraphQL Data APIs

Feature

When querying datasets with the GraphQL Data APIs, you can now bucket time series data by common grains (day, week, etc) and aggregate data within time buckets. You can also groupBy other dimensions to segment the data. Check out our docs for more info!

query {
    purchasesQuery(
      filter: {
        item_type: {
          in: ["pants", "skirts", "shorts"]
        }
        payment_status: {
          eq: "successful"
        }
      }
      groupBy: [purchase_timeDay,item_type]
    ) {
      purchase_amountSum 
      purchase_timeDay 
      item_type 
    }
}

Aug '22

31

Aug '22

22

Debugging tool: Compare data between Source and Patch

Dataset Management

When creating datasets, it's helpful to verify that individual rows of data match between the source table in your data warehouse and the data in Patch. You can now easily look up records by primary key.

$ pat dataset inspect test_columns uk_real_estate_records_fl pk_6H2rAn0vuIGeNyeBSJsB
Active source: snowflake-source
                     Table uk_real_estate_records_fl                     
                    ╷                         ╷                          
  Column            │ Cache                   │ Source (count: 1)        
╶───────────────────┼─────────────────────────┼─────────────────────────╴
  city              │ WASHINGTON              │ WASHINGTON               
  county            │ TYNE AND WEAR           │ TYNE AND WEAR            
  date_of_transfer  │ 2004-05-14              │ 2004-05-14               
  district          │ SUNDERLAND              │ SUNDERLAND               
  duration          │ F                       │ F                        
  old_new           │ N                       │ N                        
  ppd_category_type │ A                       │ A                        
  price             │ 167000                  │ 167000                   
  property_type     │ D                       │ D                        
  record_status     │ A                       │ A                        
  transaction_id    │ pk_6H2rAn0vuIGeNyeBSJsB │ pk_6H2rAn0vuIGeNyeBSJsB  
Made with Makelog