国内市場で進むデータとAIの融合と課題

“データとAIの民主化”を掲げる米Databricks。ここ数年で日本における認知も広がり、導入企業も拡大している。今回、データブリックス・ジャパン 代表取締役社長の笹俊文氏に2025年の振り返りに加え、将来の展望について話を聞いた。
GitHub

Databricks Python Data Sources

Introduced in Spark 4.x, Python Data Source API allows you to create PySpark Data Sources leveraging long standing python libraries for handling unique file types or specialized interfaces with spark ...
GitHub

Databricks SDK for Python (Beta)

Beta: This SDK is supported for production use cases, but we do expect future releases to have some interface changes; see Interface stability. We are keen to hear feedback from you on these SDKs.
VentureBeat

Databricks open-sources declarative ETL framework powering 90% faster pipeline builds

Today, at its annual Data + AI Summit, Databricks announced that it is open-sourcing its core declarative ETL framework as Apache Spark Declarative Pipelines, making it available to the entire Apache ...
note

Databricks Declarative PipelineのExpectationsはPythonとSQLどちらで書くべきか？

DatabricksのDeclarative Pipelineでは、データ品質を担保するためのルール（Expectations）を宣言的に定義できます。ただ、実装方法として「SQL（YAMLに近い形式）」と「Python（DLTなどを活用）」の2つの選択肢があり、どちらを選ぶべきか迷う方も多いのではない ...
Security Boulevard

Securing Azure Databricks Secrets with Akeyless

In modern data engineering and machine learning (MLOps) workflows, secure and scalable secrets management is crucial. Azure Databricks, a powerful cloud-based analytics platform, offers native options ...