Debugging tool: Compare data between Source and Patch

Dataset Management

When creating datasets, it's helpful to verify that individual rows of data match between the source table in your data warehouse and the data in Patch. You can now easily look up records by primary key.

$ pat dataset inspect test_columns uk_real_estate_records_fl pk_6H2rAn0vuIGeNyeBSJsB
Active source: snowflake-source
                     Table uk_real_estate_records_fl                     
                    ╷                         ╷                          
  Column            │ Cache                   │ Source (count: 1)        
╶───────────────────┼─────────────────────────┼─────────────────────────╴
  city              │ WASHINGTON              │ WASHINGTON               
  county            │ TYNE AND WEAR           │ TYNE AND WEAR            
  date_of_transfer  │ 2004-05-14              │ 2004-05-14               
  district          │ SUNDERLAND              │ SUNDERLAND               
  duration          │ F                       │ F                        
  old_new           │ N                       │ N                        
  ppd_category_type │ A                       │ A                        
  price             │ 167000                  │ 167000                   
  property_type     │ D                       │ D                        
  record_status     │ A                       │ A                        
  transaction_id    │ pk_6H2rAn0vuIGeNyeBSJsB │ pk_6H2rAn0vuIGeNyeBSJsB

Patch Changelog

patch.tech

Jul

Python Data Packages

Announcement

Data Packages are code libraries with a live connection to an underlying data source. They provide a powerful interface for querying, access control, versioning, performance optimization and more; over all your data, no matter where it lives. This could be any database or file system.

The Data Package can be installed using a package manager like pip . Then, it's imported like a library dependency into your code, whether that’s a backend service performing machine learning or enrichment tasks, a customer-facing application, or even an external consumer buying access from you directly to build using the package.

A dpm-agent intelligently routes queries submitted by consumers of a Data Package to the appropriate backend source, enforces access policies and applies performance optimizations.

Today, we're excited to announce support for generated Python packages!

Jul

TypeScript / Node.js Data Packages

Announcement

The Data Package is imported like a library dependency into your code, whether that’s a backend service performing machine learning or enrichment tasks, a customer-facing application, or even an external consumer buying access from you directly to build using the package.

A dpm-agent intelligently routes queries submitted by consumers of a Data Package to the appropriate backend source, enforces access policies and applies performance optimizations.

Today, we're excited to announce support for generated Node.js & TypeScript packages!

Jun

Alpha release: Data Packages

Announcement

Today, upstream schema changes break data pipelines and subsequently break application code. Data Packages prevent this by decoupling the client from the infrastructure to enable stable versions of a dataset to continue being served even if upstream systems have broken.

Next, Data Packages decouple application code from specific data infrastructure so that a dataset can be moved to different storage systems by data teams without impacting application teams.

Finally, Data Package versioning enables rollbacks & proper testing against newer versions - achieving a true CI/CD flow.

Data packages make engineers immediately productive with data by decoupling applications from data infrastructure.

You can read more on our blog, our docs, or get in touch to request access. The alpha release supports Typescript and Python, and the project will be open sourced soon.