Applies to:
- Liquibase Pro
- Liquibase Open Source (Community)
Summary:
In a standard CI/CD setup using Liquibase, pipelines typically target a specific environment's database, such as DEV, QA, PROD, etc. Each database will have its own DATABASECHANGELOG and DATABASECHANGELOGLOCK table, which gives Liquibase exclusive access to that database. This ensures that the DATABASECHANGELOG table gets updated correctly and there are no simultaneous, conflicting changes. However, scenarios, where multiple pipelines are connected to a single database, make this type of behavior not ideal, especially when deploying long-running scripts.
For example, if a deployment is started for changesets with label A and then another deployment is started for changesets with label B, the first deployment will lock the DATABASECHANGELOGLOCK tables, which will cause the second deployment to wait until the table is no longer locked.
In this scenario, the second deployment will start after the first deployment completes. So if there is only one changeset with label A, but it is a long-running query that will take 5 hours to deploy, the second deployment will wait (or potentially time out), which could have 25 changesets but would only take 5 minutes to deploy.
Environment Conditions:
- Any version of Liquibase Pro or Open Source (Community)
- Any supported database
- Multiple pipelines targeting a single database
How to set up a project/pipeline for a long-running query
The goal is to have (at minimum) two separate pipelines/jobs. One for short runtime scripts, and the other for long-running scripts.
- If setting this up for the first time, please follow Get Started with Liquibase to set up the first Liquibase project and create the first pipeline/job for a Liquibase deployment.
Once the main Liquibase project/pipeline is set up, create an additional pipeline/job that will be used for the long-running scripts.
This additional pipeline/job will utilize two attributes to change the table names Liquibase will use for tracking/locking to create a second set of Liquibase tracking tables in place of the default DATABASECHANGELOG and DATABASECHANGELOGLOCK tables.
By default, Liquibase will create and use the default tracking table names, DATABASECHANGELOG and DATABASECHANGELOGLOCK. However, the following two attributes can be used to create custom names for these two Liquibase tracking tables.
- Set the custom table name to use in place of DATABASECHANGELOGLOCK
- Liquibase CLI argument:
--database-changelog-lock-table-name=<lock_table_name>
- Liquibase Flow file:
globalArgs: {database-changelog-lock-table-name: "<lock_table_name>"}
- Liquibase properties file:
liquibase.databaseChangelogLockTableName=<lock_table_name>
- Liquibase environment variable:
LIQUIBASE_DATABASE_CHANGELOG_LOCK_TABLE_NAME=<lock_table_name>
- Liquibase CLI argument:
- Set the custom table name to use in place of DATABASECHANGELOG
- Liquibase CLI argument:
--database-changelog-table-name=<table_name>
- Liquibase Flow file:
globalArgs: {database-changelog-table-name: "<table_name>"}
- Liquibase properties file:
liquibase.databaseChangelogTableName=<table_name>
- Liquibase environment variable:
LIQUIBASE_DATABASE_CHANGELOG_TABLE_NAME=<table_name>
- Liquibase CLI argument:
- During the first deployment, Liquibase will create the two tables and subsequently lock the <lock_table_name> and update the <table_name> with the changesets deployed from this pipeline.
After performing the above steps, this pipeline/job will use the two specified custom table names for Liquibase tracking tables. This allows both pipelines/jobs to be able to deploy to a single database without the long-running scripts blocking the other pipeline/job.
Related Article(s):
N/A
Comments
0 comments
Article is closed for comments.