Fix entity and group duplicates

Fix duplicate identities for entities and groups targeted for renaming.

Why duplicates happen

Historical bugs in Vault allowed authN mounts to create duplicate entries of the same entity or group name if the names used difference cases. By default, Vault uses case-insensitive matching for names and considers bob and BOB to be the same name. But, past bugs have allowed for identical or different-case duplicates in entity and group names.

Duplicate entity and group names can cause unexpected behavior and make working with Vault more difficult:

• Fetching entities/groups by name returns the first identity found. Depending on your Vault version, the identity returned can vary depending on the server handling the request and when the last seal/unseal event occurred.
• Deleting entities/groups by name is not guaranteed to delete all duplicates.
• All lookup requests must use ID to find the right identity. individual identity.

Example server log

The Vault system log provides warnings about duplicate entities and groups, including:

• the entity/group name
• the associated namespace (namespace ID)
• the entity/group ID (id)
• the expected deduplication action (force_deduplication)

2025-01-28T13:15:13.641-0800 [WARN]  identity: 2 entity duplicates found
2025-01-28T13:15:13.641-0800 [WARN]  identity: entity "entity-cAsE" with namespace ID "root" duplicates 2 others: id=2562b42d-f603-ac6b-2591-8a20dd050897 force_deduplication="would not rename"
2025-01-28T13:15:13.641-0800 [WARN]  identity: entity "entity-case" with namespace ID "root" duplicates 2 others: id=290a643d-6043-da5e-943f-3a3d09e4ecbd force_deduplication="would rename to entity-case-290a643d-6043-da5e-943f-3a3d09e4ecbd"
2025-01-28T13:15:13.641-0800 [WARN]  identity: entity "entity-case" with namespace ID "root" duplicates 2 others: id=b0141be5-3f03-a1c7-a57b-02f045d04426 force_deduplication="would rename to entity-case-b0141be5-3f03-a1c7-a57b-02f045d04426"
2025-01-28T13:15:13.641-0800 [WARN]  identity: entity "entity-cAsE" with namespace ID "sYMXY" duplicates 2 others: id=95f0743b-a1d5-26da-b4ef-a50490da0787 force_deduplication="would not rename"
2025-01-28T13:15:13.641-0800 [WARN]  identity: entity "entity-case" with namespace ID "sYMXY" duplicates 2 others: id=9d3be96f-490a-9625-3118-896bd2b3a5f3 force_deduplication="would rename to entity-case-9d3be96f-490a-9625-3118-896bd2b3a5f3"
2025-01-28T13:15:13.641-0800 [WARN]  identity: entity "entity-case" with namespace ID "sYMXY" duplicates 2 others: id=d82231d2-3716-6b9c-e80d-7d09c0409739 force_deduplication="would rename to entity-case-d82231d2-3716-6b9c-e80d-7d09c0409739"
2025-01-28T13:15:13.641-0800 [WARN]  identity: end of entity duplicates
2025-01-28T13:15:13.641-0800 [WARN]  identity: 2 group duplicates found
2025-01-28T13:15:13.641-0800 [WARN]  identity: group "group-case" with namespace ID "root" duplicates 2 others: id=8ad26e0c-8cf6-5b67-7c77-6571fa374f34 force_deduplication="would not rename"
2025-01-28T13:15:13.641-0800 [WARN]  identity: group "group-cAsE" with namespace ID "root" duplicates 2 others: id=9fe86ea0-f80c-1131-5be1-1d6e3b70237f force_deduplication="would rename to group-cAsE-9fe86ea0-f80c-1131-5be1-1d6e3b70237f"
2025-01-28T13:15:13.641-0800 [WARN]  identity: group "group-case" with namespace ID "root" duplicates 2 others: id=32dd070c-c1f8-c796-9a71-15887014b813 force_deduplication="would rename to group-case-32dd070c-c1f8-c796-9a71-15887014b813"
2025-01-28T13:15:13.641-0800 [WARN]  identity: group "group-case" with namespace ID "sYMXY" duplicates 2 others: id=8aaeff7e-7343-c883-1e0c-c5c9968f75a5 force_deduplication="would not rename"
2025-01-28T13:15:13.641-0800 [WARN]  identity: group "group-case" with namespace ID "sYMXY" duplicates 2 others: id=f11277b3-d985-4d72-d2e9-9c8c6c0db02c force_deduplication="would rename to group-case-f11277b3-d985-4d72-d2e9-9c8c6c0db02c"
2025-01-28T13:15:13.641-0800 [WARN]  identity: group "group-cAsE" with namespace ID "sYMXY" duplicates 2 others: id=7c753d07-b0d9-e13b-6184-48247b8f7504 force_deduplication="would rename to group-cAsE-07c753d07-b0d9-e13b-6184-48247b8f7504"
2025-01-28T13:15:13.641-0800 [WARN]  identity: end of group duplicates

Duplicate entities and groups might be exact matches or differ in case (for example, Admin and admin). But when reviewing system logs for duplicate resolution, always consider the associated namespace when grouping potential duplicates. For example, despite having identical, case-insensitive names in the example logs, Vault only targets 8 for renaming because the names belong to different namespaces:

The automatic renaming process is security safe and preserves existing data. Renaming does not grant new permissions to existing tokens and does not delete data, so you can manually roll back the change if necessary.

Resolving entity and group duplicates

In most cases, renaming preserves resource access such that logins and permissions remain unaffected. But, there are two edge cases you may need to address before deduplication: templated policies and external references.

• If you use templated ACL policies that reference an entity or group name as part of the resource path, deduplication may affect access to those resources.

• If you use Terraform to manage Vault resources or have other systems outside Vault that reference entity or group names, those references will break when the deduplication process renames those entities/groups.

For each duplicate identified by the logs, you have 3 options:

1. If you know you do not need or use the renaming target, you can use the /identity/entity/id/{id} and /identity/group/id/{id} endpoints to delete the identities by ID.

2. If you know the risk to templated policies or external references is low or nonexistent, you can opt to ignore the duplicates and address issues if/when they occur.

1. If you cannot confirm the risk to templated policies or external references, review the guidance in the following troubleshooting guide:
   • Resolve deduplication impact on ACL policy templates
   • Resolve deduplication impact on Terraform resource references

Next steps

Once you are comfortable that all the entity and group duplicates are properly addressed, you can enable forced identity deduplication.