Databricks GetSchemas API Error When Running Liquibase Commands

Applies to

• Liquibase Secure (Pro)

Conditions

Database: Databricks with Unity Catalog

Affected Versions: Liquibase versions prior to 5.0

Environment Details:

• Databricks JDBC Driver (any version)
• Workspaces with multiple catalogs, especially those with broken or failed connections

Issue Summary

When running Liquibase commands (such as update, update-sql, or snapshot) against a Databricks database with Unity Catalog enabled, users may encounter connection errors even when credentials are valid and the target catalog/schema is accessible. The error occurs during schema discovery when Liquibase attempts to scan metadata, and the connection appears to fail with a "FAILED state" error message.

This issue is particularly common in Databricks workspaces that have multiple catalogs assigned, especially if any of those catalogs have broken or invalid connections—even if those catalogs are unrelated to the catalog specified in your Liquibase connection string.

Error Message

SEVERE [liquibase.integration] [Databricks][JDBCDriver](500004) Error getting schema information: Metadata Initialization Error.
  - Caused by: HIVE_METADATA_SCHEMA_ERR
  - Caused by: [Databricks][JDBCDriver](500594) Error calling GetSchemas API call. Error code from server: 0.

Root Cause

Prior to Liquibase version 5.0, when performing snapshot operations or schema discovery, Liquibase would scan all catalogs available in the Databricks workspace rather than limiting the scope to only the catalog and schema specified in the connection string.

If any catalog in the workspace had a broken connection, invalid credentials, or was in a FAILED state (even catalogs completely unrelated to your Liquibase operations), the GetSchemas API call would fail. This caused Liquibase commands to fail, even though:

• Your authentication token was valid
• You had appropriate permissions on your target catalog and schema
• Your target catalog/schema was functioning correctly

This created an unnecessarily broad "blast radius" where unrelated catalog issues could break Liquibase deployments.

Resolution

Permanent Fix - Upgrade to Liquibase 5.0 or Later

This issue has been resolved in Liquibase 5.0 (now called Liquibase Secure 5.0).

The fix is documented in the Liquibase 5.0 release notes under ticket INT-1398.

In Liquibase 5.0 and later:

• Snapshot operations are now properly scoped to only the catalog and schema specified in your connection string
• Liquibase no longer scans across all catalogs in your workspace
• Broken connections in unrelated catalogs will not cause your Liquibase operations to fail

Upgrading to Liquibase Secure 5.0

Liquibase Pro has been rebranded as Liquibase Secure starting with version 5.0. Important changes are documented here: New Distribution Channels for Liquibase Secure 5.0

Temporary Workaround (If Unable to Upgrade)

If you cannot immediately upgrade to Liquibase 5.0, you can work around this issue by:

1. Remove broken catalog connections from your workspace:
   • Work with your Databricks administrator to identify catalogs with failed connections
   • Unbind or remove catalogs that are not actively being used
   • Fix connection credentials for catalogs that need to remain in the workspace
2. Create a dedicated workspace:
   • Use a Databricks workspace that only contains the catalogs you need for Liquibase operations
   • This limits exposure to unrelated catalog connection issues

Related Article(s)

• Connect Liquibase with Databricks SQL