Xperience by Kentico CI/CD developer scenarios

Sharing data with Continuous Integration

For the past decade, Git has been the standard tool used to share changes in source code. Its distributed source control model enables software developers to work both independently and collaboratively.

Unlike source code, data is a common feature of applications that has not been well managed by version control systems like Git.

In a typical application, data schema changes might be versioned as a definition file plus any migration processes that adapt the previous state of data to the current. This can easily be tracked in source control, but the data itself, especially changing over time and managed in large volumes, becomes difficult to share between team members. Data seeding processes might guarantee all team members start in the same place, but the siloed data sources on each developer’s machines eventually diverge 😔.

Xperience by Kentico’s Continuous Integration (CI) tooling attempts to solve this problem by providing a set of tools and processes for sharing schema and data between development team members through source control.

When used correctly, these tools effectively enable development teams to “branch” their databases and make sure they always stay in sync with source code and application features.

Summary: CI is used to share database changes between developers in local environments.

Deploying data with Continuous Deployment

Sharing schema and data changes between developers working in their local environments is great, but what about the inevitable scenario when someone wants to see those changes in a non-local environment, like production 😅?

There are many ways to do bulk data import into production scenarios - flat files, queues and event buses, cross database queries (if your environment allows it), API calls, ect... but all of these require additional development work, to implement and then connect to the deployment of specific features.

Xperience's Continuous Deployment (CD) functionality enables developers to take some of the same techniques they use for CI and adapt them for deployment.

It should be noted that this CD feature is not a replacement for transferring data between environments (also known as Content Staging in previous versions of Kentico), but it might be a more appropriate tool when deploying data or schema changes that are tightly coupled with code level changes.

Summary: CD is used to propagate database changes from a developer's local environment to other non-local environments.

Developer Scenarios

To better understand how individuals and teams should work with Xperience's CI and CD features, let's look at some real-world scenarios that a software developer might find themselves in 😊.

Scenario 1: Setting up CI/CD for the first time on a new Xperience by Kentico solution

Let's break this down into smaller parts and introduce topics we'll rely on later 👍.

Tracking Database Backups

A SQL .bacpac or .bak file of a newly created Xperience database should be created as a project starting point that developers can restore into their local environment.

This can be stored in the repository, assuming it is small enough in size, and updated after releases or upgrades of Xperience by Kentico when CI repository data needs to be restored into the database.

A .zip of the backup, especially .bak files, can help reduce the impact on repository size.

A dedicated file or entry in the project README.md should indicate which backup (if there are multiple) is the correct one for the project at the given commit/branch. This enables developers to check out a branch and quickly restore the project to its running state at that point in time by combining the database backup with a CI restore.

CI and CD Responsibilities

Although they share a common configuration file syntax, the CI and CD features in Xperience serve very different purposes (as explained above). Because of this, their configuration needs to be handled separately (separate repository.config files for CI and CD).

CI is used to synchronize code and database changes between developers on a team. This will include test data or incomplete features that should not be deployed to other environments but need to be shared with code changes to keep local environments functional.

CD is used to deploy code and database changes from the local environment to other non-local environments. This will include new data and content types and any individual content items required for a solution to function.

There are different goals and guidance when managing inclusions and exclusions for each repository.

• CI repositories should almost always include all objects

  • Example Exclusions: bulk data or content generated by a developer's environment that does not have relations to other records and does not impact functionality

• CD repositories should exclude any data that might be created by developers locally for testing or content that is managed by marketers (like website home pages, emails, or headless items)

  • Example Exclusions: cms.webpageitem or cms.contentitem with a filter based on code name, cms.userrole, cms.user, cms.settingskey
  • Example Includes: cms.contenttype, cms.class

CI/CD Configuration and Source Control

Both the CI and CD repositories should be in the App_Data folder of the Xperience by Kentico application:

• App_Data\CIRepository\repository.config
• App_Data\CDRepository\repository.config

Xperience will generate the CI repository.config when enabling CI in the application Settings via the Administration UI, however it's up to the developer to create the CD repository.config, using the following commands:

cd .\MyXperienceProjectDirectory
dotnet run --no-build -- --kxp-cd-config --path ".\App_Data\CDRepository\repository.config"

While the CD repository configuration (everything in App_Data\CDRepository\**) is always committed to source control, the generated CD repository output should not be committed to source control and instead should be generated from the artifacts in source control, creating a deployment artifact, using scripts and tooling 🧐.

The entire CD repository configuration App_Data\CDRepository\** should also be copied to generated output folder during the build pipeline.

• This includes the repository.config, @migrations folder and its migration scripts, Before.txt and After.txt files for coordinating SQL migrations.
• These files should always be committed to source control because they are a history of the deployments and database changes of the project. They need to match the features being worked on in the Git repo so that deployments can be generated automatically from a build automation pipeline.

The CD repository generated output folder should always be cleaned before a new deployment is created to prevent issues with a previous deployment affecting a later one 👍.

After running the CD store command, the CD repository folder should include everything in the CI repository that isn’t excluded by the CD repository.config, because the CD store command exports all deployable changes from the database in the same way that the CI store command exports the state of the database to the CI repository folder.

PowerShell Scripts

Build scripts (typically using PowerShell) should be created to make the CI and CD processes easier to execute as part of a developer’s inner loop.

Teams should outline how to execute these scripts, what preconditions need to be met if the script does not handle them, and how to get back to a “clean” state in case any commands fail or a developer doesn’t understand what effect the commands have had on the Git repository. Ideally, these instructions would be part of the project’s README.md file or related project documentation.

If your editor supports it, you can create shortcuts to the most often used scripts to make easy to execute 👏.

These scripts should support both appsettings.json and User Secrets if they need to access settings, like connection strings.

Scenario 2: I’ve just joined an active project and I’m setting up my Xperience by Kentico solution

A developer setting up their environment to begin working on a project should have a clear understanding of the "steps to productivity" 🙏, ideally from a README.md or Xperience’s documentation.

One way of simplifying this for a new developer is to only have one database backup available - the “latest” created after each release or Xperience update. The restore process for the backup should be included in the project documentation.

Once the backup is restored, the developer can run the CI restore command to update the restored database backup with the CI repository updates made since the backup was created. This should also execute all CI Migrations.

Finally, the developer can start the site and begin working 🙌.

Scenario 3: I’m coming back to a project I worked on and I want to bring my environment up to current

First, the developer will need to decide if they have any work-in-progress they want to keep.

Start fresh

If the developer does not need to retain any work-in-progress, or their previous work has already been merged into main, they can take the following steps:

1. Pull down changes on the main branch
2. Create a new Git branch
3. Perform a CI restore without first performing a CI store. This will effectively reset the local database to match the latest approved and merged (via PR) changes.

Alternatively:

1. Delete the database and follow the steps in Scenario 2. This produces the same effect as the option above with a slightly stronger guarantee that there's no old data in their environment.

Continue a work in progress

If a developer has work in progress then they will want to pull down the latest changes of the main branch and merge those into their branch, resolving any conflicts (see Scenario 5).

If the merged changes include an update of Xperience by Kentico, then the steps to update a local Xperience database outlined in the docs should be followed next.

Scenario 4: I’m starting work on a new feature and I plan to make database changes

As with any new work in a repository, the developer will need to consider what they are working on and communicate with their team about the area of their work, and create a new branch in Git.

If new custom object types (module classes) are being added, those object types will need configured properly to work with CI. This is outlined in Scenario 6 🧐.

Does the work require changes to the CI repository configuration? Are there going to be new included or excluded object types or changes to the filters? Probably not, since most items should be included in the CI repository.

Will there be CI migrations that need to be committed? Do these migrations handle any work-in-progress of their teammates?

If a developer is authoring SQL migrations, they need to decide where the migrations need to apply

• CI only - the migrations only affect local data that has been shared across a team, but hasn’t been deployed.
• CD only - migrations to modify the database of some or all environments outside the developers' local environments because the migrations only affect non-local data that has been authored in those environments.
• CI and CD - migrations to modify databases of some or all environments because the data change needs to be consistently propagated everywhere. This is usually the result of some large schema change that wasn’t captured with CI. Alternatively, developers could apply similar migrations for CI and CD but customize them for slightly different use-cases 🤔.

Migration .sql files should follow a naming convention that uses the date with a format that is easily alphabetized - ex: 20231018_AddMemberMetadata.sql.

Projects could contain a large and growing number of migrations over time, and having them sorted by date in the file system will make them easier to understand and explore. Teams should establish conventions like these as early as possible, with the understanding that some exploration into a feature will need to be done before the scope of changes is completely known.

Once a developer understands the general scope of their work and has coordinated with their teammates, they should make sure their environment is up to date, as outlined in Scenario 3.

Scenario 5: I’m working on a feature and merging in changes from our main branch

This is similar to Scenario 3. The database changes made by the developer’s teammates are going to be tracked in the CI repository folder and these could conflict with the developer’s own database changes.

Resolving conflicts

The whole idea of distributed source control and separate branches is being able to work on different areas of a project without interfering with team members. But, sometimes developers still end up working in the same area of a feature or codebase.

Developers can run into conflicts when working with Xperience's CI feature. The documentation separates these types of conflicts into planned and unplanned.

Planned conflicts

Planned conflicts clearly appear in source control. An example is when two developers make a change to the same content item in different ways, store that item in their CI repository, and then commit the change. This is no different than a merge conflict in a C# or JavaScript file.

An example of a planned conflict would be if two developers both changed the ArticleShortDescription field of the same ArticlePage web page item. The .xml file that represents that item in the CI repository would show a modification in both developers branches and Git would likely be unable to resolve the conflict when merging branches. The developer merging in the main branch would need to change the field value in the .xml file and then commit it before they could restore the merged CI repository changes to their database.

Work with your teammates to figure out how best to combine the changes in the CI repository .xml files 😉.

Unplanned conflicts

Unplanned conflicts occur when a developer believes they are working on different data or content than their team, but ends up generating CI files of the same type that have conflicting identifiers or slightly different relationships. Let's take a look at a more concrete example!

---

In this scenario, I've created a web page named "Test Page" which has a code name of "TestPage-h2j0p6x6".

When creating a content item (web page, headless, or reusable), there are several related CI repository .xml files generated to represent the normalized SQL data for that item.

We can see that CI generates a file for the content item and the web page item.

CI file for a web page item.

CI file for a content item.

The testpage-h2j0p6x6.xml file represents the content item and its file name is the code name of the content item, while the [email protected] file represents the web page item and its file is a combination of the normalized WebPageItemTreePath and a hash of that name lowecase'd with length limits on both parts.

Hopefully, with this setup it starts to become clear how we could end up in a tricky situation if multiple developers create content items with the same code name (which is an editable field).

Below, we see a Git merge conflict caused by this exact kind of CI conflict.

The complicated part of this conflict is not the merging within the CI files where there are different GUIDs from different databases - we just pick the GUID for the item we want to keep 🤷.

But, what about all the other related files of either the original content item or the new one? If the CI repository isn't in a consistent state after the merge, it cannot be restored to the local Xperience database 😬.

The core of the problem here is Git identifies a file as being the same by its filename, but Xperience's CI identifies a repository item as the same by its code name and GUID and needs to be internally consistent.

The most common resolution will be the following:

1. After trying to merge, identify the local database items causing the conflicts
2. Abort the merge in process and ensure no un-tracked files remain from the merge
3. Run the Xperience application
4. Delete all the items causing conflicts (saving their content if it needs merged)
5. Stop the Xperience application
6. Commit CI repository changes (file deletions from the deleted items)
7. Merge the incoming Git changes
8. Run a CI restore
9. Run the Xperience application and validate the previously conflicting items are accessible
10. Manually add back any data from the deleted items

This merging can be handled by editing the CI repository .xml files if you know what you are doing, and in some cases this might be the best approach. But it will require more validation and testing 🫡 after running a CI restore.

Scenario 6: I'm creating a new custom module class and want to share its data

Custom object types (aka custom module classes) are configured through the Admin UI, just like custom content types.

For these types to work with Xperience's CI/CD features, the types need to be correctly configured through the generated C# class static ObjectTypeInfo.ContinuousIntegrationSettings property (Enabled needs to be true).

It's worth noting that in order for Xperience to serialize custom object types to the CI repository, a unique file path pattern has to be available for the files which represent the data records in the database. By default, these file paths use a record's GUID or code name, which must be specified in the ObjectTypeInfo constructor.

Setting the object type dependencies is also important for a consistent CI repository state. For example, if a custom object type references a web page item we want this relationship to be part of the CI repository .xml file for that custom object type record.

Scenario 7: I have database changes in my feature branch and I’m merging in changes from a team member’s branch

This is very similar to Scenario 5 with the additional complexity that a teammates “feature” branch could be in an unfinished state, unlike a branch merged into the main branch that has passed through the dev ops CI pipeline and validation.

A developer would need to understand that if they make a PR for their branch before their teammate, the developer making the PR becomes responsible for their teammate’s changes in addition to their own 🫠.

Therefore, the developer would likely want to perform the “clean slate” simulation outlined in Scenario 9 to make sure the merge of their teammate’s changes was completed correctly.

Scenario 8: I want to selectively share only some of my recent database changes with a team member

In general, all database changes should be tracked in the CI repository, or at least as many as possible.

We want to avoid the situation where a developer runs the CI store script or command at a later date and ends up with database changes from their local environment that were related to previous work - this could lead to confusion for both the developer and their team.

If they want to make temporary changes, they should track them with CI and then delete those changes in the database via the Admin. They could also, ⚠️ very carefully ⚠️, delete specific CI repository files when they are done with the temporary changes and run a CI Restore on their own database before testing to make sure things are in a consistent state.

However, the best approach will always be to delete the changes through the Admin or Xperience API and validate the CI repository is in a clean 🧼 state with no "temporary" changes before they make their PR.

Scenario 9: My branch has database changes and I want to prepare it for a PR

To ensure that their branch has all the features that it needs and doesn’t include temporary changes or rely on database changes that were not serialized to the filesystem, the developer can take steps to simulate what their teammates would experience after pulling down their changes:

1. Restore a new database from the latest database backup
2. Update their User Secrets to point to the newly restored database
3. Run the CI restore command, which copies the local CI repository changes into the newly restored database
4. Test their feature to make sure it still works correctly and all the data is in place
5. (optional) Record the new feature with screenshots or short .mp4/.webp videos to share with their team in the PR description 🧠
6. The developer now knows there is no state in their local database that isn’t included in the project CI repository 👏

Next, the developer needs to consider how they want to deploy the database changes with their code to a separate environment (likely “Test” or “Development”).

Assuming they are using a dev ops CI/CD pipeline that auto deploys their code after it is merged into the main branch, the CD repository.config should be updated to include or exclude any new data or schema changes from their feature work so that their local data, work-in-progress, or test content does not get deployed.

Scenario 10: I'm responsible for applying a Refresh to Xperience by Kentico

The steps for applying Refreshes to an Xperience by Kentico project are outlined in our documentation on the Update Xperience by Kentico projects page.

After applying a Refresh, follow the guidance in Scenario 1 that details how to manage SQL database backups for a project.

Run the CI store command after applying updates to serialize all database migrations to the CI repository, commit all code and CI repository changes, and finally create a PR.

Scenario 11: I'm going to pull down updates that include applying a new Refresh

This process is very similar to the steps taken by the developer creating the PR that includes a Refresh or hotfix update (outlined in Scenario 10).

The main difference here is the developer pulling down the changes might encounter a conflict, as outlined in Scenario 5. They will need to resolve all conflicts - in code and the CI repository - before they can run the upgrade process on their environment, because the upgrade command requires a project that can compile.

Developers do not need to be concerned about their branch’s changes in the CI repository being upgraded because the CI repository is a reflection of the database, assuming the CI feature is enabled in their Xperience database - if a change is in the CI repository, it's also in the database 😎. Also, any incoming CI repository changes have already been updated in the branch where the upgrade was originally run.

The update steps are outlined in the Xperience documentation, but are summarized here as well:

1. Ensure the application is stopped
2. Ensure all Xperience package references (NuGet and npm) are consistent and match the version of the incoming branch’s Xperience version.
3. Disable CI in the database
4. Update the database - dotnet run --no-build --kxp-update
5. Enable CI in the database
6. Run CI store to serialize the database (which has been upgraded) to the CI repository
   1. At this point both the incoming CI repository content and the developer’s database changes have been updated to a newer version of Xperience.
7. Commit all changes.
8. Continue developing or create a PR 👏.

There are quite a few steps here that might be repeated often since Xperience ships about 4 hotfixes and 1 Refresh every month! This process can be scripted to improve the developer's workflow 👏.

Scenario 12: I want to share some SQL-based database migrations with my code changes

The CI repository includes an @migrations folder which contains the migration .sql scripts to execute and Before.txt and After.txt files which list all the migrations to run (and the order to run them in).

A developer can add a new .sql file to the @migrations folder and an entry in one of the .txt files based on whether they want the migrations to run before or after the CI restore.

The next time that the CI restore command is run, the migration SQL will be executed. If you use the PowerShell script (linked in the Xperience documentation) a new entry will be made in the CI_Migration database table to ensure migrations only execute once per environment (e.g. per developer).

If a developer needs to re-execute the same migration multiple times (maybe with adjustments or to handle data added at a later time), they’ll want to create multiple versions of the .sql file in the @migrations folder and entries in the .txt file. They should not get in the habit of manually modify the CI_Migration table 😰.

These migration related files in the CI (or CD) repositories should be committed to source control and shared across a team.

Scenario 12: I want to share some C#-based database migrations with my code changes

We can hook into the ApplicationEvents.Initialized.Execute event but this only lets us modify data after the application has already started and the CI restore process has completed, so if we need to prep some data before running CI restore, this won’t work 😞.

What we probably need are Before/After .NET events we can hook into or something similar to ApplicationEvents.ExecuteMigrations.Execute, which is only used for CD migrations.

This is being investigated by the product and development teams. Let us know ✍️ on the Kentico Roadmap if this is an important scenario for you.

Scenario 13: I want to start or stop tracking changes to some data in our solution

In general, development teams should avoid specific inclusions or exclusions of data with the CI repository because this increases the likelihood that each developer’s environment will begin to drift from others over time. The differences in environments can lead to bugs that are difficult to reproduce, surprising failures in automated testing in a CI pipeline, or features that are started by one developer but can’t be finished by another 😒.

However, some data might require exclusions, either for testing while a feature is being developed, or to reduce noise in the CI repository in source control. To include or exclude specific data, the repository.config file can be used to limit what Xperience serializes to the file system.

Wrap up

Wow 😅! That was a lot of information about Xperience's CI/CD and we didn't even dive deep into the CI/CD repository structures or application deployments using Xperience's CD.

But, hopefully, the scenarios above are helpful and answer some questions about the development workflow you have or plan to have with Xperience by Kentico solutions.

Leave a comment if you have additional questions or scenarios you'd like to see covered. There's no reason we can't create a "part 2" 😅.