ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial3 min readDisconfirmAI Editorial

Databricks Connects AWS HealthLake Clinical Data via OpenSharing: Zero-Copy Access and Genie One Natural Language Query

The Databricks team describes a new architecture based on OpenSharing that allows FHIR data managed by AWS HealthLake to be read directly by Databricks and governed within Unity Catalog without export pipelines, while supporting natural language clinical data exploration through Genie One.

2026-11-17 16:30 GMT+8

Many healthcare organizations have adopted AWS HealthLake as their clinical data management platform, using the FHIR standard to store patient records, observations, and encounter information. However, a long-standing architectural limitation is that HealthLake's analytics tables are stored in Apache Iceberg format within S3 buckets owned by the HealthLake service itself, meaning customer IAM roles cannot read these buckets directly. This implies that external governance tools like Unity Catalog lack a path to the underlying data; if an organization wishes to perform analysis and AI applications outside of HealthLake, it must build export pipelines to copy the data out and re-establish governance on the target platform.

Michael McCarthy, Lily Goldstein, and Douglas Moore described a new solution to fill this gap in a November 17, 2026 Databricks blog post. The core idea is to deploy a custom OpenSharing server running as a Databricks App in a fully managed serverless environment. When Databricks requests a specific HealthLake table, the server uses temporary credentials generated by Lake Formation to resolve the table's current Iceberg snapshot, pre-signs the data files, and returns file references in a standard OpenSharing response format. The Databricks compute engine then reads these pre-signed files directly from S3, with no data passing through the OpenSharing server itself. AWS trusts the identity of the Databricks App through OIDC federation, achieving zero-copy access while protecting sensitive medical data.

At the governance level, Databricks mounts HealthLake data as a shared directory within Unity Catalog, treating it like any other OpenSharing share. Organizations can apply grants, fine-grained access control (FGAC), attribute-based access control (ABAC), and role-based access control (RBAC) to this clinical data within Unity Catalog, and audit access and usage records just as they would for other data assets.

Sign in / Sign up
ReadingResearchRadarInvestment framework

Another layer of significance in this integration is natural language exploration. Once the shared directory is established, organizations can configure Genie One agents on top of it, defining core clinical tables, adding plain-language descriptions, and teaching the agent how to interpret clinical concepts such as A1C. A clinical analyst can ask a question like 'What is the monthly trend for A1C?', and Genie One will understand the context, locate the corresponding Genie agent, generate SQL, and return a summarized answer with tables and charts, all while adhering to Unity Catalog permissions. Questions that previously required SQL skills or custom dashboards can now be answered in seconds.

The Databricks team notes that this solution frees HealthLake customers from being limited to Athena or Redshift as their only analytical paths and eliminates the need to rely on complex export workflows. A complete reference implementation has been published in the Databricks Industry Solutions repository, deploying the OpenSharing server as a Declarative Automation Bundle.

It should be noted that the above description comes entirely from vendor self-reports within the Databricks and AWS ecosystems. The article does not provide independent benchmark tests, production environment latency data, or third-party verification results. The actual performance of this architecture, its behavior on large-scale FHIR datasets, and the governance maturity of cross-organizational deployments currently remain in a state awaiting verification.

Sources

  • databricks.comDatabricks↗
Continue researching →