Kitaev’s finding of an exactly solvable magnetic model on a twodimensional honeycomb lattice [1], which promises a fault-tolerant quantum computation via realization of Majorana fermions with non- Abelian statistics, ushered in a plethora of theoretical and experimental studies that aimed at realizing the Kitaev physics within condensed matter systems. A general guiding principle was reported by Jackeli and Khaliullin [2], followed by a number of theoretical and experimental studies that attempted to determine and evaluate the viability of the Jackeli-Khaliullin mechanism in realistic systems [3, 4]. Currently, α-RuCl₃ is considered the most promising candidate [5–15], wherein half-quantized thermal Hall conductivity, considered as the essential evidence for the presence of the Kitaev spin liquid (KSL) state [1, 13, 15], is reported.

However, attaining the KSL state in α-RuCl₃ is hindered by the presence of magnetic exchange interactions, in addition to Kitaev interactions, which induce static magnetism and disrupt the KSL phase. Specifically, the presence of the first- and third-nearest-neighbor Heisenberg interaction is detrimental for the Kitaev magnetism [16], and completely removing them is challenging owing to the multi-orbital nature of candidate systems and the resulting various hopping processes [17]. Therefore, selective enhancement of relevant electron hopping channels that contribute to the Kitaev’s exchange interaction are essential for the realization of the KSL phase in realistic situations.

Recently, certain theoretical and experimental reports on α-RuBr₃ [18–20] and α-Rul₃ [19, 21, 22] have been presented, focusing on the possibility of promoting and realizing the KSL phase. As discussed in further detail later, the heavier ligand ion results in stronger hybridization between Ru 4d- and ligand (Cl, Br, I) p-orbitals, which may lead to stronger Kitaev interactions. However, it may also induce unwanted enhancement of further-neighbor and even inter-layer Heisenberg interactions, which are not favorable to realizing the KSL phases. Moreover, enhanced bandwidth due to larger d-p hybridization can also disturb the formation of the local spin-orbital-entangled j_eff = 1/2 moments, a critical element of the Kitaev’s exchange interactions. Therefore, evaluation of such quantities via first-principles electronic structure calculations is crucial during the early stage of studies regarding α-RuBr₃ and α-Rul₃.

Thus, this study focused on the electronic structure of α-RuBr₃ and α-Rul₃, comparisons with α-RuCl₃, and the formation of the spinorbital- entangled j_eff = 1/2 moments in the compounds. Furthermore, the magnitudes of magnetic exchange interactions were estimated via employing first-principles Wannierization of Ru t_2g bands, showing α-RuBr₃ to be a promising platform to further investigate the Kitaev magnetism in condensed matter systems.

For the optimizations of cell parameters and internal coordinates of all three compounds, the Vienna ab-initio Simulation Package (VASP), which uses the projector-augmented wave (PAW) basis set [23,24] was employed. 500 eV of plane-wave energy curoff and 7×7×3 Γ-centered k-grid sampling were adopted. Further, effects of electron correlations in structural optimizations were considered via the Strongly Constrained and Appropriately Normed (SCAN) semi-local functional [25], which is parameter-free, combined with a revised Perdew-Burke-Ernzerhof (PBEsol) [26] exchange-correlation functional. Following the structural optimizations, effects of atomic spin-orbit coupling (SOC) and on-site Coulomb repulsions were studied employing a linear-combination- of-pseudo-atomic-orbital basis code OPENMX [27, 28], where projections onto the spin-orbit-entangled j_eff = 1/2 and 3/2 states and the Wannierization of Ru t_2g bands were performed [29–31]. Double zeta plus polarization (DZP) bases and 300 Ry of energy cut-off for real space integrations were employed in the OPENMX calculations, and the effects of on-site Coulomb repulsions were incorporated via a simplified variant of the rotationally-invariant DFT+U formalism [32] combined with the local density approximation [33, 34]. The value of U_eff ≡ U - J was chosen as 2 eV, which has been considered to be a reasonable value in previous studies [6, 10]. Electron hopping integrals were computed using Wannier orbital method in the nonmagnetic phase without including SOC and on-site Coulomb interactions, thereby eliminating the double-counting of such effects.

3.1. Crystal structures

Figure 1 shows the model crystal structure we chose for α-RuX₃ (X = Cl, Br, I) in this study (Space group: P3₁12). There have been reports of different space group symmetries and stacking patterns for these compounds [20,22], but our choice of the P3₁12 stacking is not a serious concern because interlayer interactions are fairly small in all three compounds [10]. Moreover, while the zigzag-type antiferromagnetic order (which requires larger in-plane periodicity) has been reported as the true ground state for α-RuCl₃, the effect of different antiferromagnetism within the RuCl3 layer on structural and electronics properties is insignificant. Therefore, a Néel-type antiferromagnetism was employed in our structural optimizations.

Figure 1. (a) Crystal structure of α-RuX₃ (X = Cl, Br, I) with P3₁12 space group. Solid lines depict the trigonal unit cell. (b) A top-down view of the single RuX₃ layer, where the stacking of neighboring Ru honeycomb layers are shown in (c).

Table I presents PBEsol+SCAN-optimized lattice parameters of all three compounds in the presence of SOC. Our result predicted lattice parameters of α-RuCl₃ and α-Rul₃ to be approximately 1% larger than experimentally reported values [22,35]; however, the tendency toward cell expansion with the ligand ion becoming heavier was well-captured within our results.

Table I. Lattice parameters of α-RuCl₃, α-RuBr₃, and α-RuI₃ from PBEsol+SCAN optimizations..

(in Å)	|a|	|c|
α-RuCl3	6.043	17.601
α-RuBr3	6.404	18.634
α-RuI3	6.888	20.236

3.2. Electronic structure and spin-orbital-entangled states

Figure 2 summarizes our calculation results for all three compounds, showing the band structures and projected densities of states (PDOS) onto the spin-orbit-entangled j_eff =1/2 and 3/2 states. The Ru j_eff states, eigenstates of the SOC Hamiltonian within the Ru t_2g orbital, are defined as

Figure 2. Band structures and projected densities of states (PDOS) onto the Ru jeff =1/2 and 3/2 states. Top (a-c), middle (d-f), and bottom (g-i) rows show results obtained for α-RuCl₃, α-RuBr₃, and α-RuI₃, respectively. Left (a, d, and g), middle (b, e, and h), and right (c, f, and i) columns show results with SOC, SOC+U_eff (2 eV), and SOC+U_eff+antiferromagnetism (Neel type), respectively. Note that nonmagnetic constraint was enforced when obtaining results in the left and middle columns.

On comparing the left and right columns in Fig. 2, it is evident that the inclusion of the on-site Coulomb interaction enhanced the splitting between the j_eff = 1/2 and 3/2 states. The SOC being effectively enhanced by U_eff, has been discussed in previous studies regarding α-RuCl₃ [6, 22]. Further, the inclusion of magnetism and the opening of the band gap further enhances the splitting (comparing the middle and right columns in Fig. 2), consequently driving α-RuCl₃ and α-Rul₃ good Kitaev magnet candidates. However, in contrast to the results obtained in our study [Fig. 2(a)] α-Rul₃ has been reported to be metallic [21]. The discrepancy may originate from technical details (e.g., the choice of local atomic projectors employed in the DFT+U_eff methodology). We comment that, the results of this study indicate that the band gap size of α-Rul₃ is much smaller than that of α-RuCl₃ and α-RuBr₃ [comparing Figs. 2(c), 2(f), and 2(i)], which is consistent with the fact that larger d-p hybridization in α-Rul₃ induces larger bandwidth and weaker electron correlation effects. In addition, the enhancement of the j_eff = 1/2 − 3/2 splitting introduced by U_eff is not significant [Figs. 2(g) and 2(h)] compared to other compounds, possibly because of the larger d-p hybridization in α-Rul₃.

3.3. Electron hopping channels and magnetic exchange interactions

Having established the spin-orbit-coupled nature of our systems, we next switch to magnetic exchange interactions. Figure 3 represents major nearest-neighbor hopping channels between Ru t_2g orbitals. Among these the first t-term, which is mediated via an intermediate anion p-orbital, is critical to inducing the Kitaev interaction, whereas the remaining terms result in conventional Heisenberg or symmetric anisotropy interactions. Specifically, the general magnetic Hamiltonian in α-RuX₃ is approximately expressed as [17],

Figure 3. Illustration of major hopping channels between nearest-neighboring Ru t_2g orbitals.

where α, β, γ denote bond directions and the relevant spin components (x, y, z) involved in the Kitaev interactions, respectively, and ⟨ij⟩ and ⟨⟨⟨ij⟩⟩⟩ indicate the nearest-neighbor and third- nearest-neighbor i-j sites, respectively. The J, K, and Γ are explicitly obtained via perturbation theory as follows [17],

where U and J_H are the strength of the on-site Coulomb repulsion (not necessarily the same as U_eff) and Hund’s coupling within the Ru t_2g orbital, respectively.

The hopping terms t, t', and t'' can be obtained using the Wannier function method [29, 30], which are tabulated in Table II The magnitude of the d-p-d hopping term t decreased slightly when changing from α-RuCl₃ to α-Rul₃. In addition, the magnitudes of t' and t'', which introduce Heisenberg J, decreased when Cl was replaced with Br or I. The origin of this behavior is unclear at this point and deserves further study.

Table II. Activation energies and errors for various CdTe film thicknesses..

(in eV)	t	t'	t''	t3rd	$t_{\mathrm{inter}}^{\max }$
α-RuCl3	+0.184	−0.054	+0.035	−0.041	−0.028
α-RuBr3	+0.169	−0.030	+0.024	−0.040	−0.040
α-RuI3	+0.170	+0.007	+0.009	−0.050	−0.049

Because we obtained the hopping parameters from ab-initio calculations, the magnitudes of J, K, and Γ could be estimated as a function of U and JH . We set U = U_eff=2 eV, and computed exchange interactions as a function of JH /U. Figure 4 shows the results, and owing to the diminishing t' and t'' in α-RuBr₃ and α-Rul₃, the size of J and Γ reduced when changing from α-RuCl₃ to α-Rul₃. As α-Rul₃ has been reported to be metal, α-RuBr₃ appears to be the best candidate to host Kitaev magnetism in this series of compounds.

Figure 4. Nearest-neighbor magnetic exchange interactions for α-RuCl₃, α-RuBr₃, and α-RuI₃ as a function of J_H/U.

We conclude this section by mentioning third-nearest-neighbor and inter-layer hopping elements. Table II indicates that both terms increased as the ligand anion became heavier, as expected in the beginning. However, the enhancement is not that significant compared to the changes in t' and t''. Although many different channels can constructively contribute to further neighbor and interlayer hopping channels, as discussed in Winter et al. [16], and Catuneanu et al. [36], it can be speculated that the enhancement in further neighbor and interlayer hopping channels may not cause substantial changes in α-RuBr₃ and α-Rul₃, thereby rendering α-RuBr₃ a promising candidate for realizing Kitaev magnetism in this family.

In this study preliminary first-principles density functional theory calculations were performed for α-RuCl₃, α-RuBr₃, and α-Rul₃, to assess their viability for realizing Kitaev magnetism. The results indicated that α-RuCl₃ and α-RuBr₃ may host the spin-orbit-entangled j_eff = 1/2 moment, and that α-RuBr₃ shows even stronger Kitaev magnetism compared to α-RuCl₃.

Note that some possible effects of minor lattice distortions, for example the effect of trigonal crystal fields, were not discussed in this study [37]. Estimation of next- and third-nearest-neighbor Heisenberg interactions that induce long-range magnetic orders needs to be done as well for further studies on magnetic properties of these systems. Hence continuing theoretical and experimental studies are needed at this moment, but we believe that α-RuBr₃ can be another interesting system to study Kitaev physics in addition to α-RuCl₃. Lastly, since α-Rul₃ has been reported to be nonmagnetic and metallic, it has potentials to host topological band insulating phases with weakto- intermediate electron correlations as previously suggested [36].

We thank K.-Y. Choi for useful discussions, and acknowledge the support from the Korea Research Fellow (KRF) Program and the Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Education (NRF-2019H1D3A1A01- 102984 and NRF- 2020R1C1C1005900). We also thank the support of computational resources including technical assistance from the National Supercomputing Center of Korea (Grant No. KSC- 2020-CRE- 0156)

The authors declare no conflicts of interest.