Adoption of a token-based authentication model for the CMS Submission Infrastructure

TL;DR

The paper addresses the CMS Submission Infrastructure's reliance on GSI/X.509 for authentication and documents a transition to token-based authentication using IDTokens and Scitokens within HTCondor-enabled resources. It details the deployment across internal components, HTCondor-CEs, and ARC-CEs, including REST interface adoption and CMS IAM-driven token provisioning, alongside enhancements in secret management with Teigi. The results show successful migration of internal components by Spring 2022 and broader adoption of Scitokens for HTCondor-CEs by January 2023, with LDAP-ARC-CE submission deprecated and a plan to complete the GSI phase-out, improve resilience, and perform security drills. The work significantly modernizes authentication for distributed CMS workloads, improving security, scalability, and operational integrity across WLCG resources.

Abstract

The CMS Submission Infrastructure (SI) is the main computing resource provisioning system for CMS workloads. A number of HTCondor pools are employed to manage this infrastructure, which aggregates geographically distributed resources from the WLCG and other providers. Historically, the model of authentication among the diverse components of this infrastructure has relied on the Grid Security Infrastructure (GSI), based on identities and X509 certificates. In contrast, commonly used modern authentication standards are based on capabilities and tokens. The WLCG has identified this trend and aims at a transparent replacement of GSI for all its workload management, data transfer and storage access operations, to be completed during the current LHC Run 3. As part of this effort, and within the context of CMS computing, the Submission Infrastructure group is in the process of phasing out the GSI part of its authentication layers, in favor of IDTokens and Scitokens. The use of tokens is already well integrated into the HTCondor Software Suite, which has allowed us to fully migrate the authentication between internal components of SI. Additionally, recent versions of the HTCondor-CE support tokens as well, enabling CMS resource requests to Grid sites employing this CE technology to be granted by means of token exchange. After a rollout campaign to sites, successfully completed by the third quarter of 2022, the totality of HTCondor CEs in use by CMS are already receiving Scitoken-based pilot jobs. On the ARC CE side, a parallel campaign was launched to foster the adoption of the REST interface at CMS sites (required to enable token-based job submission via HTCondor-G), which is nearing completion as well. In this contribution, the newly adopted authentication model will be described. We will then report on the migration status and final steps towards complete GSI phase out in the CMS SI.

Figures (3)

• Figure 1: GlideinWMS and HTCondor components building a dynamically sized pool of compute resources.
• Figure 2: The CMS Global Pool components, and where tokens are used for authentication as described in section \ref{['sec-Internal']}.
• Figure 3: The use of Scitokens in GlideinWMS to access Grid sites. Tokens are fetched from the CMS IAM instance by the Frontend and then propagated to the Factory to authorize pilot submission to CEs.