A low-complexity 3D massive MIMO scheme jointly using statistical and instantaneous CSIT

Abstract

In this paper, we propose a three-dimensional (3D) beamforming scheme for the massive multiple-input multiple-output (MIMO) system where the base station (BS) employs a uniform rectangular array (URA). In order to avoid the high computational complexity involving large-dimensional channel matrices, a two-stage beamforming method is applied where the second-stage beamforming is a Kronecker product of azimuth and elevation discrete Fourier transform (DFT) beamforming. These DFT prebeamformers are used for cell splitting and form effective channels with lower dimension for first-stage precoding. We develop a low-complexity user grouping algorithm based on the statistical channel state information at the transmitter (CSIT) to partition users. Each group of users is served by the signal-to-leakage-and-noise ratio (SLNR) precoding aiming at suppressing the intra-group and adjacent-group interferences, which is a good balance between performance and complexity. We derive the approximate signal-to-interference-plus-noise ratio (SINR) of our proposed scheme. Numerical results validate that the SINR approximations are tight and indicate the significance of the proposed 3D beamforming scheme.

1 Introduction

In order to meet the demand of explosively increasing data services, the massive multiple-input multiple-output (MIMO) system has emerged as a promising technology for the next generation of cellular systems [1–3]. The basic premise behind massive MIMO is to reap all the benefits of conventional MIMO on a much greater scale, by deploying a few hundred antennas at the base station (BS) to serve a multiplicity of users simultaneously in the same time-frequency resource [4–7]. However, in practice, it is infeasible to place a large number of antennas only in the azimuth direction at the BS. To cope with this limitation, three-dimensional (3D) MIMO has been introduced, where antennas are deployed in a two-dimensional (2D) grid at the BS to enable the multiplexing of many users in a multi-user MIMO (MU-MIMO) fashion [8–12].

In 3D MIMO, elevation antennas are exploited to design 3D beamforming. More users can thus be served by the 3D beamformer with the same azimuth but different elevation angles [13]. A practical method for performing per-user adaptation of the elevation direction is presented in [14], which is transparent to the Long-Term Evolution (LTE) standard and requires no changes to the existing mobiles. But no performance analysis is given. The achievable sum rate is analyzed for uplink 3D MIMO systems with zero-forcing (ZF) receivers in [15, 16]. In [17], 3D beamforming is developed which consists of azimuth two-stage beamforming and one elevation prebeamformer. This scheme takes advantage of cell splitting by prebeamformers and functions efficiently when users in the same group have identical angle of arrival (AoA) intervals but have nonoverlapping AoA intervals in the different groups. However, users are usually randomly distributed and the angle requirements cannot be guaranteed. Besides, elevation groups are designated by orthogonal time-frequency slots to assure near orthogonality, which does not exploit the full use of resources. And the specific user grouping algorithm for 3D massive MIMO is not developed.

In this paper, a low-complexity 3D beamforming scheme is proposed for the massive MIMO system where the BS deploys a uniform rectangular array (URA). We apply two-stage beamforming to avoid the high complexity involving the large-dimensional channel matrices. The second-stage beamforming is a Kronecker product of azimuth and elevation discrete Fourier transform (DFT) prebeamformers, since the 3D channel covariance can be approximated by a Kronecker product of azimuth and elevation correlations and it is possible to separate the 3D channel into azimuth and elevation directions which are respectively served by uniform linear arrays (ULAs) in associated directions at the BS [18]. Considering the one-ring scattering model, the azimuth and elevation correlations are characterized by Toeplitz matrices, and the eigenvector matrices of these Toeplitz matrices are approximated by submatrices of DFT matrix when the number of antennas is large [19, 20]. So, we apply the DFT beamforming as the azimuth and elevation prebeamformers, and their Kronecker product constructs the 3D prebeamformer. These DFT prebeamformers are used for cell splitting, and all groups are all working in the same time-frequency resource. We develop a low-complexity user grouping algorithm to partition users into groups using statistical channel state information at the transmitter (CSIT).

The first-stage precoding is designed based on the effective channels formed by large-dimensional instantaneous channels and DFT prebeamformers, which has low complexity. We employ the signal-to-leakage-and-noise ratio (SLNR) precoding considering the intra-group and adjacent-group interferences which dominate the inter-group interferences. The SLNR precoding is designed based on the signal-to-leakage-and-noise ratio as the optimization metric, where leakage is a measure which quantifies the interference power caused by the desired user on the signal received by others [21]. It is a good balance between eliminating co-channel interference (CCI) and noise, while zero forcing (ZF) design considers the CCI only and suffers from noise enhancement. Moreover, the ZF precoding imposes a restriction on the number of antennas, while for SLNR precoding there is no requirement on the relation between the number of transmit and receive antennas. Compared to the signal-to-interference-plus-noise ratio (SINR) precoding which is obtained iteratively due to the coupled optimization problem [22], the SLNR precoding admits a closed-form solution, since the SLNR metric results in a decoupled optimization problem. However, the SLNR precoding aims at maximizing SLNR instead of SINR, so it leads to suboptimal performance. In terms of complexity, the SLNR precoding involves matrix inversion which has high complexity with large-dimensional channel matrix in massive MIMO systems. In this paper, we partition the cell and group users, so the effective number of BS antennas for each group becomes small, and the requirement on the antenna number of ZF precoding may not be satisfied. Thus, we apply the SLNR precoding. Furthermore, the matrix inversion does not cause high complexity because of the lower dimensional effective channels.

We derive the approximate SINR for our proposal using deterministic equivalents [3, 23–26]. This method based on the random matrix theory enables us to approximate the stochastic SINR with deterministic expression which is dependent only on statistical channel state information (CSI). Numerical results show that the SINR approximations are tight when the number of BS antennas is large and validate the effectiveness of the proposed low-complexity user grouping algorithm. The proposed SLNR precoding is illustrated to be a good balance between performance and complexity.

The rest of this paper is organized as follows. We present the system model in Section 2. In Section 3, we introduce the proposed 3D beamforming scheme. We derive the approximate SINR of our proposal in Section 4. The numerical results are shown in Section 5, and the paper is concluded in Section 6.

Notation: We use boldface capital letter X for matrix, boldface small letter x for vector, and small letter x for scalar. X ^H denotes the Hermitian transpose of X;tr(X) denotes the trace of X. X⊗Y denotes the Kronecker product of X and Y. X[:,(x ₁:x ₂)] denotes a matrix formed by the x ₁ to x ₂ columns of X. The identity matrix with rank b is denoted by I _b .

2 System model

Consider a single-cell downlink massive MIMO system in Fig. 1. The BS is equipped with an N×M URA where M,N≫1. The large URA can become an integral part of a large infrastructure, deployed along walls of a shopping mall or facade of a building [27]. In our paper, we assume that the BS antennas are mounted on the top of a building [28] and seldom obstructed by local scatter. K single-antenna users are located on the street level and surrounded by scatters. They are all served simultaneously in the same time-frequency resource.

Fig. 1

A single-cell 3D massive MIMO system where the BS employs a URA

2.1 3D channel model

For exploiting the potential of elevation direction and designing 3D beamforming, the investigation of 3D channel modelling is necessary. In a massive MIMO system, 3D channels are always correlated, since a large number of BS antennas is packed in a limited space [29]. There are two popular approaches to model the correlated channels. One is the parametric approach wherein channel realizations are generated by summing all paths with physical channel parameters [30, 31]. But this method is not suitable for the performance analysis. The other one is a non-parametric approach where the spatial correlation is reproduced using theoretical spatial correlations [11, 12, 17].

In this paper, we consider the correlated channel model [3, 32] which is given by

$$\begin{array}{@{}rcl@{}} {\mathbf{h}}_{k}=\beta_{k}^{\frac{1}{2}}{\mathbf{R}}_{k}^{\frac{1}{2}}{\mathbf{w}}_{k}, \end{array} $$

(1)

for the kth user, where w _k is a complex circularly symmetric Gaussian vector with mean 0 and covariance matrix I _MN , i.e., ${\mathbf {w}}_{k}\in \mathbb {C}^{MN}\sim \mathcal {CN}\left ({{\mathbf {0}},{\mathbf {I}}_{MN}} \right)$, and β _k denotes the path loss, which is

$$\begin{array}{@{}rcl@{}} \beta_{k}=\frac{1}{r_{k}^{\gamma}}, \end{array} $$

(2)

where r _k is the distance between the kth user and the foot of BS, γ is the decay exponent, and ${\mathbf {R}}_{k} \in {\mathbb {C}}^{MN \times MN}$ is the 3D channel correlation. Since the eigenvalue distributions and the eigenvectors of 3D channel correlation and the Kronecker product of azimuth and elevation correlations are close, we can model the channel correlation for the kth user as [17, 18]

$$\begin{array}{@{}rcl@{}} {\mathbf{R}}_{k} \approx {\mathbf{R}}_{k,\text{az}} \otimes {\mathbf{R}}_{k,\text{el}}, \end{array} $$

(3)

where ${\mathbf {R}}_{k,\text {az}} \in {\mathbb {C}}^{M \times M}$ and ${\mathbf {R}}_{k,\text {el}} \in {\mathbb {C}}^{N \times N}$ are the azimuth and elevation correlations, respectively. Hence, the 3D channel can be separated into azimuth and elevation directions and can be treated as independent 2D channels with an M-antenna ULA in the azimuth direction and an N-antenna ULA in the elevation direction, respectively. We assume that the BS is elevated and seldom obstructed by local scatters, while users are surrounded by local scatters, resulting in the one-ring model [33]. Assuming that the user receives signal from all scatters surrounded with the same energy, the (m,p)-th entry of R _k,az is

$$\begin{array}{@{}rcl@{}} \left[{\mathbf{R}}_{k,\text{az}}\right]_{m,p}={1 \over 2\Delta}\int_{-\Delta +\theta}^{\Delta +\theta}{{{e}^{-j2\pi D\left(m-p\right)\sin \alpha }}d\alpha}, \end{array} $$

(4)

where θ is the azimuth AoA, Δ is the associated angular spread (AS), and D is the antenna spacing divided by the carrier wavelength. The entries of R _k,el are derived similarly as R _k,az.

Combining (1) and (3), the channel for the kth user is

$$\begin{array}{@{}rcl@{}} {\mathbf{h}}_{k}&=& \beta_{k}^{\frac{1}{2}} \left({\mathbf{R}}_{k,\text{az}} \otimes {\mathbf{R}}_{k,\text{el}} \right)^{\frac{1}{2}}{\mathbf{w}}_{k} \\ &=& \beta_{k}^{\frac{1}{2}}\left({\mathbf{U}}_{k,\text{az}} \mathbf{\Lambda}_{k,\text{az}}{\mathbf{U}}_{k,\text{az}}^{\textsf{H}} \otimes {\mathbf{U}}_{k,\text{el}} \mathbf{\Lambda}_{k,\text{el}}{\mathbf{U}}_{k,\text{el}}^{\textsf{H}} \right)^{\frac{1}{2}}{\mathbf{w}}_{k} \end{array} $$

(5)

$$\begin{array}{@{}rcl@{}} &=& \beta_{k}^{\frac{1}{2}} \left({{\mathbf{U}}_{k,\text{az}} \otimes {\mathbf{U}}_{k,\text{el}}} \right)\left({\mathbf{\Lambda}_{k,\text{az}}^{1 \over 2} \otimes \mathbf{\Lambda}_{k,\text{el}}^{1 \over 2}}\right){\mathbf{v}}_{k}, \end{array} $$

(6)

where (5) results from the eigenvalue decompositions (EVDs) of R _k,az and R _k,el, respectively, $\mathbf {\Lambda }_{k,\text {az}}\in \mathbb {C}^{r_{k,\text {az}} \times r_{k,\text {az}}}$ is a diagonal matrix with eigenvalues of R _k,az arranged in the descending order, ${\mathbf {U}}_{k,\text {az}} \in \mathbb {C}^{M \times r_{k,\text {az}}}$ is the unitary eigenvector matrix of R _k,az,r _k,az is the rank of ${\mathbf {R}}_{k,\text {az}}; {\mathbf {\Lambda }}_{k,\text {el}}\in \mathbb {C}^{r_{k,\text {el}} \times r_{k,\text {el}}}$ is a diagonal matrix with eigenvalues of R _k,el arranged in the descending order, ${\mathbf {U}}_{k,\text {el}} \in \mathbb {C}^{N \times {r_{k,\text {el}}}}$ is the unitary eigenvector matrix of R _k,el,r _k,el is the rank of R _k,el. (6) is obtained using the Kronecker product rule. ${\mathbf {v}}_{k} = \left ({\mathbf {U}}_{k,\text {az}}^{\textsf {H}} \otimes {\mathbf {U}}_{k,\text {el}}^{\textsf {H}}\right){\mathbf {w}}_{k}$ is distributed as $\mathcal {CN}\left ({{\mathbf {0}},{\mathbf {I}}_{{r_{k,\text {az}}}{r_{k,\text {el}}}}} \right)$. Obviously, U _k =U _k,az⊗U _k,el is the unitary eigenvector matrix of R _k and $\beta _{k}^{\frac {1}{2}}{\mathbf {\Lambda }_{k,\text {az}}^{1 \over 2} \otimes \mathbf {\Lambda }_{k,\text {el}}^{1 \over 2}}$ is the diagonal matrix with eigenvalues of R _k .

2.2 Downlink transmission model

The received signal of the kth user is

$$\begin{array}{@{}rcl@{}} y_{k}={\sqrt\rho}\mathbf{h}^{\textsf{H}}_{k} \mathbf{s}+n_{k}, \end{array} $$

(7)

where $n_{k} \sim \mathcal {CN}(0,1)$ is receiver noise, ρ denotes the transmit power, and $\mathbf {s} \in \mathbb {C}^{MN}$ is the transmit vector from BS, which is

$$\begin{array}{@{}rcl@{}} {\mathbf s}=\sum_{k=1}^{K} \mathbf{a}_{k} d_{k}, \end{array} $$

(8)

where $\mathbf {a}_{k} \in \mathbb {C}^{MN}$ is the precoding vector satisfying ∥a _k ∥=1, and $d_{k} \sim \mathcal {CN}(0,1)$ is the data symbol for the kth user chosen from a Gaussian codebook [3].

3 Low-complexity 3D beamforming

We adopt two-stage beamforming for the 3D massive MIMO system, which is a _k =B _k p _k for the kth user, where $\mathbf {B}_{k} \in \mathbb {C}^{MN \times b}$ is a prebeamformer and $\mathbf {p}_{k} \in \mathbb {C}^{b}$ is multi-user precoding. Prebeamforming B _k and the large-dimensional instantaneous CSI h _k form the effective channel $\mathbf {B}_{k}^{\textsf {H}}\mathbf {h}_{k} \in \mathbb {C}^{b}$ which has lower dimension. p _k can thus be designed based on $\mathbf {B}_{k}^{\textsf {H}}\mathbf {h}_{k}$ with lower complexity.

3.1 Prebeamforming

The prebeamformers are determined by the statistical CSI, exploiting the spatial characteristics of channels. The 3D channel correlation is approximated by the Kronecker product of the azimuth and elevation correlations, as given in (3). And these two separable correlations are formed with an M-antenna ULA in the azimuth direction and an N-antenna ULA in the elevation direction, respectively [18]. Considering the one-ring scattering model, the azimuth and elevation correlations are characterized by (4), which are Toeplitz matrices. The eigenvector matrices of these Toeplitz matrices are approximated by submatrices of DFT matrix when the number of antennas is large [19, 20]. So, we apply the DFT beamforming as the azimuth and elevation prebeamformers, and their Kronecker product constructs the 3D prebeamformer.

The impact of DFT prebeamformers can be regarded as cell splitting. The cell is cut into subsectors in the azimuth direction and annular regions in the elevation directions, as depicted in Fig. 2. We denote by G the number of subsectors; L the number of annular regions. These subsectors and annular regions form groups, and the user set in the (ℓ,g)-th group is denoted by $\mathcal {S}_{\ell,g}$. The received signal of the kth user is

$$ \begin{aligned} {}y_{k}= \sqrt\rho {\mathbf{h}}^{\textsf{H}}_{k} \sum_{\ell', g'}\sum_{u \in {\mathcal{S}_{\ell',g'}}} \left({\mathbf{B}}_{{g'},\text{az}} \otimes {\mathbf{B}}_{{\ell'},\text{el}}\right){\mathbf{p}}_{u} d_{u} +\! n_{k}, \;k \in \mathcal{S}_{\ell,g}, \end{aligned} $$

(9)

Fig. 2

Cell splitting by DFT prebeamforming in the azimuth and elevation directions, respectively

where $\phantom {\dot {i}\!}{\mathbf {B}}_{g^{\prime },\text {az}}$ and $\phantom {\dot {i}\!}{\mathbf {B}}_{\ell ^{\prime },\text {el}}$ are the DFT prebeamforming for the g ^′th subsector and the ℓ ^′th annular region, respectively. Assume that the cell is split uniformly and DFT prebeamformers have an equal number of columns of DFT matrix, i.e., $\phantom {\dot {i}\!}{\mathbf {B}}_{g^{\prime },\text {az}}={\mathbf {F}}_{\text {az}} \left [{:,\left ({g^{\prime }-1}\right)b_{\text {az}}+\left ({1:b_{\text {az}}}\right)}\right ]$, where the (m,p)-th entry of the DFT matrix ${\mathbf {F}}_{\text {az}}\in \mathbb {C}^{M \times M}$ is $[{\mathbf {F}}_{\text {az}}]_{m,p}=\frac {e^{-j2\pi (m-1)(p-1)/M}}{\sqrt {M}}$, and $b_{\text {az}}=\left \lfloor {M \over G} \right \rfloor$. Similarly, $\phantom {\dot {i}\!}{\mathbf {B}}_{\ell ^{\prime },\text {el}}={\mathbf {F}}_{\text {el}} [{:,\left ({\ell ^{\prime }-1}\right)b_{\text {el}}} {+\left ({1:b_{\text {el}}}\right)}]$, where the (m,p)-th entry of the DFT matrix ${\mathbf {F}}_{\text {el}}\in \mathbb {C}^{N \times N}$ is $[{\mathbf {F}}_{\text {el}}]_{m,p}=\frac {e^{-j2\pi (m-1)(p-1)/N}}{\sqrt {N}}$, and $b_{\text {el}}=\left \lfloor {N \over L} \right \rfloor$. The 3D prebeamformer for the kth user is $\phantom {\dot {i}\!}{\mathbf {B}}_{\ell,g}={\mathbf {B}}_{g,\text {az}} \otimes {\mathbf {B}}_{\ell,\text {el}} \in C^{MN \times b_{\text {az}}b_{\text {el}}}$ and the effective channel is ${\mathbf {B}}^{\textsf {H}}_{\ell,g}\mathbf {h}_{k} \in \mathbb {C}^{b}$, where b=b _az b _el. The DFT prebeamforming provides the lower dimensional effective channels, compared to the large-dimensional instantaneous CSI. Based on these effective channels, the complexity of the multi-user precoding becomes lower. A block diagram of the 3D beamforming scheme is shown in Fig. 3.

Fig. 3

A block diagram of 3D beamforming. The second-stage beamforming is the Kronecker product of DFT prebeamforming in the azimuth and elevation directions and the first-stage precoding is designed based on effective channels with lower dimension

3.2 Low-complexity user grouping algorithm

DFT prebeamformers are used for cell splitting and form groups and then users are partitioned into these groups based on the statistical CSI. We develop a low-complexity 2D fixed quantization user grouping (2D-FQUG) algorithm in Algorithm 1. Due to the separability of the 3D channel, We first find the annular region index for the user and then the subsector index.

Note that the assumption of (4) is that the user receives signal from all scatters surrounded with the same energy. But in reality, the received power at the user may not be uniformly distributed due to some strong scatters. Hence, the eigenvalues of the correlation of each user change [34], so does the eigenvector matrix corresponding to the ordered eigenvalues; our proposed algorithm still works but is achieving a different user grouping result.

The 2D-FQUG is a separable user grouping algorithm with lower complexity, comparing to the overall algorithm which is based on the criterion,

$$\begin{array}{@{}rcl@{}} {}(\ell,g) = \arg \min \limits_{\left({\ell}',g'\right)} {\left\|{\mathbf{U}}_{k}\mathbf{U}^{\textsf{H}}_{k}-{\mathbf{B}}_{\ell',g'}{\mathbf{B}}_{\ell',g'}^{\textsf{H}} \right\|}^{2}, \, \forall \ell',g'. \end{array} $$

(12)

The complexities of these two algorithms are analyzed in terms of floating-point operation (FLOP) and we count each complex operation as one FLOP [35]. The complexity mainly comes from matrix products and EVDs. In (10), ${\mathbf {U}}_{k,\text {el}} {\mathbf {U}}_{k,\text {el}}^{\textsf {H}}$ needs (2r _k,el−1)N ² FLOPs, ${\mathbf {B}}_{\ell ',\text {el}}{\mathbf {B}}_{\ell ',\text {el}}^{\textsf {H}}$ requires (2b _el−1)N ² FLOPs, ${\mathbf {U}}_{k,\text {el}} {\mathbf {U}}_{k,\text {el}}^{\textsf {H}} -{\mathbf {B}}_{\ell ',\text {el}}{\mathbf {B}}_{\ell ',\text {el}}^{\textsf {H}}$ costs N ² FLOPs, and ${\left \|{\mathbf {U}}_{k,\text {el}} {\mathbf {U}}_{k,\text {el}}^{\textsf {H}} -{\mathbf {B}}_{\ell ',\text {el}}{\mathbf {B}}_{\ell ',\text {el}}^{\textsf {H }} \right \|}^{2}$ needs 2N ²−1 FLOPs, yielding a total number of (2r _k,el+2b _el+1)N ²−1 FLOPs. Similarly, (11) requires (2r _k,az+2b _az+1)M ²−1 FLOPs. In the overall algorithm, (12) needs (2r _k,az r _k,el+2b _az b _el+1)(M N)²−1 FLOPs. For matrix EVD computation of each user, we consider a maximum number of N ³+M ³ and M ³ N ³ FLOPS in 2D-FQUG and overall algorithms, respectively [36].

Thus, the total complexity of 2D-FQUG algorithm is

$$ \begin{aligned} &{}C_{\mathrm{f}}= \left(2b_{\text{el}}N^{2}+N^{2}+2b_{\text{az}}M^{2}+M^{2}-2+N^{3}+M^{3}\right)\\ &\quad K+ \sum_{k} 2r_{k,\text{el}}N^{2}+2r_{k,\text{az}}M^{2} \\ &{}\approx \left(\left(\frac{2}{L}+1\right)N^{3}+N^{2}+\left(\frac{2}{G}+1\right)M^{3}+M^{2}-2\right)K \\ &{}+ \sum_{k} 2r_{k,\text{el}}N^{2}+2r_{k,\text{az}}M^{2}\;\textrm{FLOPs}, \end{aligned} $$

(13)

which is O(N ³+M ³). And the total complexity of the overall algorithm is

$$\begin{array}{@{}rcl@{}} C_{\mathrm{o}}&=&\left(2b_{\text{az}}b_{\text{el}}M^{2}N^{2}+M^{2}N^{2}-1+M^{3}N^{3}\right)\\ &&\quad K+2M^{2}N^{2}\sum_{k} r_{k,\text{az}}r_{k,\text{el}} \\ &\approx&\left(\left(\frac{2}{GL}+1\right)M^{3}N^{3}+M^{2}N^{2}-1\right)K \\ &+&2M^{2}N^{2}\sum_{k} r_{k,\text{az}}r_{k,\text{el}} \;\textrm{FLOPs}, \end{array} $$

(14)

which is O(M ³ N ³). So, the proposed 2D-FQUG algorithm has lower complexity than the overall one, since the matrix dimension becomes smaller due to the Kronecker product factorization of channel correlations. Moreover, from (13)–(14), we know that when the number of groups is increasing, the complexity is reduced, and when the numbers of BS antennas and users become larger, the complexity increases.

3.3 Multi-user precoding

The first-stage precoding can be designed per group individually, after user partition. However, inter-group interferences exist, because the eigenvector matrix of channel correlation is not exactly equal to the Kronecker product of the azimuth and elevation DFT prebeamformers. This is because in practice, the angular supports of users may be overlapping and users cannot be separated strictly. Moreover, the inter-group interferences are severe between the adjacent groups, and hence the multi-user precoders need to consider not only the intra-group interferences but also the adjacent-group interferences.

Conventional ZF precoding can achieve this goal, but it requires that the number of transmit antennas must be equal to or larger than that of receive antennas combined. Nevertheless, due to cell splitting, the effective number of BS antennas serving each group is equal to b which is smaller than MN. When users are located in one area, the requirement of ZF precoding may not be satisfied. Thus, we adopt SLNR precoding as the multi-user precoders, which has no restriction on the number of antennas [21]. For mitigating both the intra-group and adjacent-group interferences, the SLNR precoding for the kth user is

$$\begin{array}{@{}rcl@{}} {}{\mathbf{p}}_{k}\,=\, \sqrt{\lambda_{k}}\left({{1\over\rho}{\mathbf{I}}_{b}}\,+\, {\mathbf{B}}^{\textsf{H}}_{\ell,g} {\tilde{\mathbf{H}}}_{k}{\tilde{\mathbf{H}}}_{k}^{\textsf{H}} {\mathbf{B}}_{\ell,g} \right)^{-1} \!{\mathbf{B}}^{\textsf{H}}_{\ell,g} {\mathbf{h}}_{k},\; k \!\in\! \mathcal{S}_{\ell,g}, \end{array} $$

(15)

where λ _k is a regularization parameter to guarantee $\left \|{{\mathbf {p}}_{k}}\right \|=1, {\tilde {\mathbf {H}}}_{k}=[{\mathbf {H}}_{(\ell,g-1)},\! {\mathbf {H}}_{(\ell,g+1)}, {\mathbf {H}}_{(\ell -1,g)}, {\mathbf {H}}_{(\ell +1,g)}, {\underline {{\mathbf {H}}}_{k}}] \in \mathbb {C}^{MN \times (U_{k}-1)}$, where [H _(ℓ,g−1),H _(ℓ,g+1),H _(ℓ−1,g),H _(ℓ+1,g)] is an extended channel matrix formed by user channels in the adjacent groups of the kth user, ${\underline {{\mathbf {H}}}_{k}} \in \mathbb {C}^{V_{k}-1}$ is the extended channel matrix of the (ℓ,g)-th group excluding the channel of the kth user, $V_{k}=|\mathcal {S}_{\ell,g}|$ is the number of users in the (ℓ,g)-th group, and $U_{k}=\sum _{{\ell '}=\ell -1}^{\ell +1}\sum _{{g'}=g-1}^{g+1}|\mathcal {S}_{\ell ',g'}|$.

This SLNR precoding surely is not as good as that considering all-group interferences, but it has lower complexity. For comparison, we consider the SLNR precoding for suppressing all-group and intra-group interferences, respectively, which are

$$ \begin{aligned} {}{\mathbf{p}}_{k, \text{all}}= \sqrt{\lambda_{k, \text{all}}}\left({{1\over\rho}{\mathbf{I}}_{b}}+ {\mathbf{B}}^{\textsf{H}}_{\ell,g} {\dot{\mathbf{H}}}_{k}{\dot{\mathbf{H}}}_{k}^{\textsf{H}} {\mathbf{B}}_{\ell,g} \right)^{-1} {\mathbf{B}}^{\textsf{H}}_{\ell,g} {\mathbf{h}}_{k}, k \in \mathcal{S}_{\ell,g}, \end{aligned} $$

(16)

$$ \begin{aligned} {}{\mathbf{p}}_{k, \text{intra}}\,=\, \sqrt{\lambda_{k, \text{intra}}}\!\left(\!{{1\over\rho}{\mathbf{I}}_{b}}\,+\, {\mathbf{B}}^{\textsf{H}}_{\ell,g} {\underline{{\mathbf{H}}}_{k}}{\underline{{\mathbf{H}}}^{\textsf{H}}_{k}} {\mathbf{B}}_{\ell,g} \!\right)^{-1} \!{\mathbf{B}}^{\textsf{H}}_{\ell,g} {\mathbf{h}}_{k}, k\! \in\! \mathcal{S}_{\ell,g}, \end{aligned} $$

(17)

where $\dot {\mathbf {H}}_{k}$ is an extended channel matrix that excludes h _k only.

The source of the complexity difference is ${\mathbf {B}}^{\textsf {H}}_{\ell,g} {\tilde {\mathbf {H}}}_{k}{\tilde {\mathbf {H}}}_{k}^{\textsf {H}} {\mathbf {B}}_{\ell,g}, {\mathbf {B}}^{\textsf {H}}_{\ell,g} {\dot {\mathbf {H}}}_{k}{\dot {\mathbf {H}}}_{k}^{\textsf {H}} {\mathbf {B}}_{\ell,g }$, and ${\mathbf {B}}^{\textsf {H}}_{\ell,g} {\underline {{\mathbf {H}}}_{k}}{\underline {{\mathbf {H}}}^{\textsf {H}}_{k}} {\mathbf {B}}_{\ell,g}$. The general complexity of these terms can be written as

$$\begin{array}{@{}rcl@{}} C_{\mathrm{t}}=(2bMN-b+2b^{2})x-b^{2} \;\textrm{FLOPs}, \end{array} $$

(18)

where x=U _k −1, K−1, and V _k −1 for (15), (16), and (17), respectively. We denote this term by Ω, so the general SLNR precoding is expressed as ${\mathbf{p}}_{k,\mathrm{g}}=\sqrt{\lambda_{k,\mathrm{g}}}{\left(\frac{1}{\rho }{\mathbf{I}}_b+\varOmega \right)}^{-1}{\mathbf{B}}_{\ell, g}^{\mathsf{H}}{\mathbf{h}}_k$. Assuming that ${{1\over \rho }{\mathbf {I}}_{b}}$ is known a prior, $\frac{1}{\rho }{\mathbf{I}}_b+\varOmega$ needs b real summations, which is equivalently transferred to $\frac {b}{2}$ FLOPs here. ${\left(\frac{1}{\rho }{\mathbf{I}}_b+\varOmega \right)}^{-1}$ requires b ³+b ²+b FLOPs. ${\mathbf {B}}^{\textsf {H}}_{\ell,g} {\mathbf {h}}_{k}$ costs 2b M N−b FLOPs wherein Hermitian transposition requires no FLOP. ${\left(\frac{1}{\rho }{\mathbf{I}}_b+\varOmega \right)}^{-1}{\mathbf{B}}_{\ell, g}^{\mathsf{H}}{\mathbf{h}}_k$ consumes 2b ²−b FLOPs. The regularization parameter is

$$ \sqrt{\lambda_{k,\mathrm{g}}}=\frac{1}{\parallel {\left(1\rho {\mathbf{I}}_b+\varOmega \right)}^{-1}{\mathbf{B}}_{\ell, g}^{\mathsf{H}}{\mathbf{h}}_k\parallel }, $$

(19)

which requires 2b+1 FLOPs. A simple multiplication $\sqrt{\lambda_{k,\mathrm{g}}}{\left(\frac{1}{\rho }{\mathbf{I}}_b+\varOmega \right)}^{-1}{\mathbf{B}}_{\ell, g}^{\mathsf{H}}{\mathbf{h}}_k$ of a vector ${\left(\frac{1}{\rho }{\mathbf{I}}_b+\varOmega \right)}^{-1}{\mathbf{B}}_{\ell, g}^{\mathsf{H}}{\mathbf{h}}_k$ with a scalar $\sqrt {\lambda _{k,\mathrm {g}}}$ requires b multiplications.

Thus, for our proposal and SLNR precoding considering all-group and intra-group interferences, the respective total complexities for all users are

$$ \begin{aligned} {}C_{\mathrm{p}}\,=\,\left(b^{3}\,+\,\frac{7}{2}b+1)K\,+\,(2bMN\,-\,b\,+\,2b^{2}\right)\sum_{k} U_{k} \;\textrm{FLOPs}, \end{aligned} $$

(20)

$$ \begin{aligned} {}C_{\mathrm{a}}=\left(b^{3}+\frac{7}{2}b+1)K+(2bMN-b+2b^{2}\right)K^{2} \;\textrm{FLOPs}, \end{aligned} $$

(21)

$$ \begin{aligned} {}C_{\mathrm{i}}=\left(b^{3}\,+\,\frac{7}{2}b+1)K\,+\,(2bMN\,-\,b+2b^{2}\right)\sum_{k} V_{k} \;\textrm{FLOPs}. \end{aligned} $$

(22)

We unify (20)–(22) into a general form as

$$ {}C_{\mathrm{g}}=\left(b^{3}+\frac{7}{2}b+1)K+(2bMN-b+2b^{2}\right)z \;\textrm{FLOPs}, $$

(23)

where $z=\sum _{k} U_{k},K^{2},\sum _{k} V_{k}$ for (20), (21), and (22), respectively. The first derivative of C _g with respect to b is

$$\begin{array}{@{}rcl@{}} \frac{\partial {C_{\mathrm{g}}}}{\partial b}=\left(3b^{2}+\frac{7}{2}\right)K+(2MN-1+4b)z. \end{array} $$

(24)

Since $M,N\gg 1, \frac {\partial {C_{\mathrm {g}}}}{\partial b}>0$. So, C _g is a monotonically increasing function with respect to b. When the number of groups increases, the dimension of effective channel b decreases and the complexity is reduced. The first derivative of C _g with respect to M is

$$ \begin{aligned} {}\frac{\partial {C_{\mathrm{g}}}}{\partial M}=\frac{3M^{2}N^{3}K}{(GL)^{3}}+\frac{7NK}{2GL}+\frac{4MN^{2}z}{GL}-\frac{Nz}{GL}+\frac{4MN^{2}z}{(GL)^{2}}. \end{aligned} $$

(25)

Since $M,N\gg 1, M N\gg K, \frac {\partial {C_{\mathrm {g}}}}{\partial M}>0$. So, C _g is a monotonically increasing function with respect to M. Similarly, C _g is a monotonically increasing function with respect to N. Thus, when the number of BS antennas increases, the complexity becomes high. And larger number of users brings higher complexity. Evidently, there is a trade-off between performance and complexity, and in the simulation section, we will show that our proposal is a good balance.

4 Performance analysis

In this section, we analyze the performance of our proposal. The performance metric is sum rate, which is

$$\begin{array}{@{}rcl@{}} r= \sum_{k=1}^{K} \log_{2}(1+{\textsf{SINR}}_{k}), \end{array} $$

(26)

where the SINR for the kth user is

$$ \begin{aligned} {}\textsf{SINR}_{k} \,=\, \frac{\rho |{\mathbf h}_{k}^{\textsf{H}} {\mathbf{B}}_{\ell,g} {\mathbf{p}}_{k} |^{2}}{\rho \sum_{u \neq k} |\mathbf{h}_{k}^{\textsf{H}} {\mathbf{B}}_{\ell',g'}{\mathbf{p}}_{u} |^{2} \,+\,1 }, k\!\in\! \mathcal{S}_{\ell,g},\,u \in \mathcal{S}_{\ell',g'}, \! \forall \ell'\!,g'. \end{aligned} $$

(27)

The numerator is the signal power, the first term in the denominator is the interference power, and the second term in the denominator is the noise power which is equal to 1. We compute the SINR via the deterministic equivalent method [3, 23–26] and then obtain the sum rate by (26). The derived SINR is a deterministic quantity in the almost sure (a.s.) asymptotics. Since we apply the DFT prebeamformer to reduce the system dimension and the SLNR precoding is dependent on the effective channel, the studied system dimension in deterministic equivalent approximation is b instead of MN, and the a.s. asymptotics can be written as

$$\begin{array}{@{}rcl@{}} \textsf{SINR}-\textsf{SINR}^{\circ} {\underset{b,K_{u}\to+\infty}{\xrightarrow{{\phantom{0000}}\text{a.s.{\phantom{0001}}}}}} 0, \end{array} $$

(28)

where K _u is the number of users in the studied system, SINR is regarded as a sequence with increasing system dimension b and K _u , and SINR° is its deterministic equivalent. b,K _u →+∞ denotes b and K _u go to infinity at the same speed, i.e.,

$$\begin{array}{@{}rcl@{}} 0 < \lim \inf {K_{u}\over b} \le \lim \sup {K_{u} \over b} < +\infty. \end{array} $$

(29)

For brevity of expression, we rewrite (28) as SINR≍SINR°. SINR° is obtained by random matrix theory [23]. Referring to [37], when the number of antennas increases, the variance of channel capacity is reduced such that the instantaneous mutual information is deterministic approximately and only depends on the statistical CSI. So, we can analyze the performance based on deterministic equivalents without the need of heavy Monte Carlo simulations. Using deterministic equivalents, we derive the approximate SINR in the following theorem.

Theorem 1

In a 3D massive MIMO system depicted in (9) where the SLNR precoding is given in (15), the approximate SINR of the kth user, in the sense of (28), is

$$ {{\begin{aligned} {}\text{SINR}^{\circ}_{k} &=\left| \text{tr} \left(\mathbf{B}^{\textsf{H}}_{\ell,g} \mathbf{R}_{k} \mathbf{B}_{\ell,g} \mathbf{T}_{k}\right) \right|^{2} \left/ \left(\vphantom{\sum_{u \in\mathcal{I}_{\mathrm{n},k}}} \text{tr} \left(\mathbf{B}^{\textsf{H}}_{\ell,g}\mathbf{R}_{k}\mathbf{B}_{\ell,g} \mathbf{T}'_{\lambda_{k}} \right) \right. \right. \\ &\times\left(\sum_{u \in\mathcal{I}_{\mathrm{n},k}} {{\text{tr} \left(\mathbf{B}^{\textsf{H}}_{\ell',g'} \mathbf{R}_{u} \mathbf{B}_{\ell',g'} \mathbf{T}'_{u,k} \right)} \over { \text{tr} \left(\mathbf{B}^{\textsf{H}}_{\ell',g'}\mathbf{R}_{u}\mathbf{B}_{\ell',g'} \mathbf{T}'_{\lambda_{u}} \right)}}\right. \\ &+\sum_{u' \in\mathcal{I}_{\mathrm{y},k}} {{\frac{\text{tr} \left(\mathbf{B}_{\ell',g'}^{\textsf{H}} \mathbf{R}_{k}\mathbf{B}_{\ell',g'} \mathbf{S}'_{u',k} \right)}{\left(\! 1\,+\, \frac{\beta_{k}}{b} \text{tr}\left(\!\mathbf{B}_{\ell',g'}^{\textsf{H}} \mathbf{R}_{k} \mathbf{B}_{\ell',g'} \mathbf{S}'_{u',k} \!\right)\!\right)^{2} \!{\text{tr} \left(\! \mathbf{B}^{\textsf{H}}_{\ell',g'}\mathbf{R}_{k}\mathbf{B}_{\ell',g'} \mathbf{T}'_{\lambda_{u'}}\! \right)}} }} \\ &+\left. \left.{\frac{1}{\beta_{k} \rho}} \vphantom{\sum_{u \in\mathcal{I}_{\mathrm{n},k}}}\right)\vphantom{\sum_{u \in\mathcal{I}_{\mathrm{n},k}}}\right),\; k \in \mathcal{S}_{\ell,g},\,u,u' \in \mathcal{S}_{\ell',g'}, \, u,u' \neq k,\, \forall \ell',g', \end{aligned}}} $$

(30)

where

$$\begin{array}{@{}rcl@{}} \mathbf{T}_{k} =\left({1\over {b}} \sum_{s \in \mathcal{T }_{k}} \frac{\beta_{s}{\mathbf{B}}^{\textsf{H}}_{\ell,g} \mathbf{R}_{s} {\mathbf{B}}_{\ell,g} }{1+\delta_{s}}+{1 \over {\rho b}} \mathbf{I}_{b}\right)^{-1}, \end{array} $$

(31)

$\mathcal {T }_{k}$ is the user set of the (ℓ,g)-th group and the associated adjacent groups excluding the kth user, $\delta _{s}={\lim }_{t \to \infty }\delta ^{(t)}_{s}$, for t=1,2,…

$$ {{\begin{aligned} {}\delta^{(t)}_{s} ={\beta_{s} \over {b}} \text{tr} \left({\mathbf{B}}^{\textsf{H}}_{\ell,g} \mathbf{R}_{s} {\mathbf{B}}_{\ell,g} \left({1\over {b}} \sum_{s' \in \mathcal{T}_{k}} \frac{\beta_{s'}{\mathbf{B}}^{\textsf{H}}_{\ell,g} \mathbf{R}_{s'} {\mathbf{B}}_{\ell,g} }{1+\delta^{(t-1)}_{s'}} +{1 \over {\rho b}} \mathbf{I}_{b}\right)^{-1} \right), \end{aligned}}} $$

(32)

with initial values $\delta ^{(0)}_{s}=\rho b$, for all users;

$$ {{\begin{aligned} {} \mathbf{T}'_{u,k} = \mathbf{T}_{u}{\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{k} {\mathbf{B}}_{\ell',g'} \mathbf{T}_{u} +\mathbf{T}_{u} {1 \over b} \sum_{s \in \mathcal{T}_{u}} \frac{\beta_{s} {\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{s} {\mathbf{B}}_{\ell',g'}\delta'_{s}}{(1+\delta_{s})^{2}}\mathbf{T}_{u}, \end{aligned}}} $$

(33)

where T _u and δ _s can be obtained similarly as (31) and (32), respectively, ${\delta}^{\prime }={\left[{\delta}_1^{\prime },\dots, {\delta}_{\left|{\mathcal{T}}_u\right|}^{\prime}\right]}^{\mathsf{T}}$, we have

$$ {\delta}^{\prime }={\left({\mathbf{I}}_{\left|{\mathcal{T}}_u\right|}-\mathbf{J}\right)}^{-1}\mathbf{v}, $$

(34)

where

$$\begin{array}{@{}rcl@{}} [\mathbf{J} ]_{n,i}=\frac{\text{tr} \left({\beta_{n}\beta_{i} {\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{n} {\mathbf{B}}_{\ell',g'} \mathbf{T}_{u} {\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{i} {\mathbf{B}}_{\ell',g'} \mathbf{T}_{u} }\right)}{b^{2} (1+\delta_{i})^{2}}, \end{array} $$

(35)

$$\begin{array}{@{}rcl@{}} [\mathbf{v}]_{n}= {1 \over b} \text{tr} \left({\beta_{n} {\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{n} {\mathbf{B}}_{\ell',g'} \mathbf{T}_{u} {\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{k} {\mathbf{B}}_{\ell',g'} \mathbf{T}_{u} }\right); \end{array} $$

(36)

$$ {{\begin{aligned} {}\mathbf{S}'_{u',k}= \mathbf{S}_{u'}{\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{u'} {\mathbf{B}}_{\ell',g'} \mathbf{S}_{u'} +\mathbf{S}_{u'} {1 \over b} \sum_{s \in \mathcal{T}_{u'}, s\neq k} \frac{\beta_{s} {\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{s} {\mathbf{B}}_{\ell',g'}\zeta'_{s}}{(1+\zeta_{s})^{2}}\mathbf{S}_{u'}, \end{aligned}}} $$

(37)

where

$$\begin{array}{@{}rcl@{}} \mathbf{S}_{u'} =\left({1\over {b}}\sum_{s \in \mathcal{T}_{u'}, s\neq k} \frac{\beta_{s}{\mathbf{B}}^{\textsf{H}}_{\ell',g'} \mathbf{R}_{s} {\mathbf{B}}_{\ell',g'} }{1+\zeta_{s}}+{1 \over {\rho b}} \mathbf{I}_{b}\right)^{-1}, \end{array} $$

(38)

$\zeta _{s}={\lim }_{t \to \infty }\zeta ^{(t)}_{s}$, for t=1,2,…

$$ {{\begin{aligned} {}\zeta^{(t)}_{s} \,=\,{\beta_{s} \over {b}} \text{tr} \left(\!{\mathbf{B}}^{\textsf{H}}_{\ell',g'} \mathbf{R}_{s} {\mathbf{B}}_{\ell',g'} \!\left(\! {1\over {b}} \!\sum_{s \in \mathcal{T}_{u'}, s\neq k} \frac{\beta_{s'}{\mathbf{B}}^{\textsf{H}}_{\ell',g'} \mathbf{R}_{s'} {\mathbf{B}}_{\ell',g'} }{1\,+\,\zeta^{(t-1)}_{s'}} \,+\,{1 \over {\rho b}} \mathbf{I}_{b}\!\right)^{-1} \!\right), \end{aligned}}} $$

(39)

with initial values $\zeta ^{(0)}_{s}=\rho b$, for all users, and ${\zeta}^{\prime }={\left[{\zeta}_1^{\prime },\dots, {\zeta}_{\left|{\mathcal{T}}_{u^{\prime }}\right|-1}^{\prime}\right]}^{\mathsf{T}}$,

$$ {\zeta}^{\prime }={\left({\mathbf{I}}_{\left|{\mathcal{T}}_{u^{\prime }}\right|-1}-{\mathbf{J}}^{\prime}\right)}^{-1}{\mathbf{v}}^{\prime }, $$

(40)

where

$$\begin{array}{@{}rcl@{}} [\mathbf{J}']_{n,i}=\frac{\text{tr} \left({\beta_{n}\beta_{i} {\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{n} {\mathbf{B}}_{\ell',g'} \mathbf{S}_{u'} {\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{i} {\mathbf{B}}_{\ell',g'} \mathbf{S}_{u'} }\right)}{b^{2} (1+\zeta_{i})^{2}}, \end{array} $$

(41)

$$\begin{array}{@{}rcl@{}} [\mathbf{v}']_{n}= {1 \over b} \text{tr} \left({\beta_{n} {\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{n} {\mathbf{B}}_{\ell',g'} \mathbf{S}_{u'} {\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{u'} {\mathbf{B}}_{\ell',g'} \mathbf{S}_{u'} }\right); \end{array} $$

(42)

$$\begin{array}{@{}rcl@{}} \mathbf{T}'_{\lambda_{k}} &=& \mathbf{T}_{k}^{2} +\mathbf{T}_{u} {1 \over b} \sum_{s \in \mathcal{T}_{k}} \frac{\beta_{s} {\mathbf{B}}^{\textsf{H}}_{\ell,g} {\mathbf{R}}_{s} {\mathbf{B}}_{\ell,g}\delta'_{\lambda,s}}{(1+\delta_{s})^{2}}\mathbf{T}_{k}, \end{array} $$

(43)

where T _k and δ _s can be derived based on (31) and (32), respectively, and ${\delta}_{\lambda_k}^{\prime }={\left[{\delta}_{\lambda_k,1}^{\prime },\dots, {\delta}_{\lambda_k,\left|{\mathcal{T}}_k\right|}^{\prime}\right]}^{\mathsf{T}}$,

$$ {\delta}_{\lambda_k}^{\prime }={\left({\mathbf{I}}_{\left|{\mathcal{T}}_k\right|}-{\mathbf{J}}_{\lambda_k}\right)}^{-1}{\mathbf{v}}_{\lambda_k}, $$

(44)

where

$$\begin{array}{@{}rcl@{}} [\mathbf{J}_{\lambda_{k}} ]_{n,i}=\frac{\text{tr} \left({\beta_{n}\beta_{i} {\mathbf{B}}^{\textsf{H}}_{\ell,g} {\mathbf{R}}_{n} {\mathbf{B}}_{\ell,g} \mathbf{T}_{k} {\mathbf{B}}^{\textsf{H}}_{\ell,g} {\mathbf{R}}_{i} {\mathbf{B}}_{\ell,g} \mathbf{T}_{k} }\right)}{b^{2} (1+\delta_{i})^{2}}, \end{array} $$

(45)

$$\begin{array}{@{}rcl@{}} [\mathbf{v}_{\lambda_{k}}]_{n}= {1 \over b} \text{tr} \left({\beta_{n} {\mathbf{B}}^{\textsf{H}}_{\ell,g} {\mathbf{R}}_{n} {\mathbf{B}}_{\ell,g} \mathbf{T}_{k}^{2} }\right). \end{array} $$

(46)

Proof

The signal power for the kth user is

$$ {{\begin{aligned} {}\rho|\mathbf{h}_{k}^{\textsf{H}} {\mathbf{B}}_{\ell,g} {\mathbf{p}}_{k} |^{2}&={\rho}|\check{\mathbf{h}}_{k}^{\textsf{H}} {\mathbf{p}}_{k} |^{2} \\ &={\rho\lambda_{k}} \left| \frac{\check{\mathbf{h}}_{k }^{\textsf{H}}}{ \sqrt{b} } \left({{1\over{\rho b }}{\mathbf{I}}_{b}}+ {\frac{\check{\mathbf{G}}_{k}}{ \sqrt{b} }}{\frac{\check{\mathbf{G}}_{k }^{\textsf{H}} }{ \sqrt{b} }} \right)^{-1} \frac{\check{\mathbf{h}}_{k}}{\sqrt{b} } \right|^{2} \\ &\asymp \rho\lambda_{k}\beta^{2}_{k} \left| \text{tr} \left({ \frac{1}{ b }{\mathbf{B}}^{\textsf{H}}_{\ell,g} \mathbf{R}_{k} {\mathbf{B}}_{\ell,g} \left({{1\over{\rho b }}{\mathbf{I}}_{b}}+ {\frac{\check{\mathbf{G}}_{k}}{ \sqrt{b} }}{\frac{\check{\mathbf{G}}_{k}^{\textsf{H}} }{ \sqrt{b} }} \right)^{-1} }\right) \right|^{2} \end{aligned}}} $$

(47)

$$ {{\begin{aligned} {\kern29pt}\asymp \frac{\rho\lambda_{k} \beta^{2}_{k}}{b^{2}} \left| \text{tr} \left({\mathbf{B}}^{\textsf{H}}_{\ell,g} \mathbf{R}_{k} {\mathbf{B}}_{\ell,g} \mathbf{T}_{k}\right) \right|^{2},\; k \in \mathcal{S}_{\ell,g}, \end{aligned}}} $$

(48)

where $\check {\mathbf {h}}_{k} \triangleq {\mathbf {B}}^{\textsf {H}}_{\ell,g} \mathbf {h}_{k}, \check {\mathbf {G}}_{k} \triangleq {\mathbf {B}}^{\textsf {H}}_{\ell,g}\tilde {\mathbf {H}}_{k}$, (47) comes from Lemma 4 in [3], and (48) is derived from Theorem 1 in [3], where T _k is given by (31).

The SLNR precoding for the kth user is related to other user channels. In order to use deterministic equivalents to calculate the interference power, we divide the interfering user channels into two groups: one is containing user channels which are not included by the precoding vector of the kth user, i.e., the users are not located in either the adjacent groups of the (ℓ,g)-th group or in the (ℓ,g)-th group itself, while the other is the opposite case. We denote these two groups by $\mathcal {I}_{\mathrm {n},k}$ and $\mathcal {I}_{\mathrm {y},k}$, respectively. So, the interference power for the kth user is the sum of these two parts, which is

$$\begin{array}{@{}rcl@{}} &&\rho \sum_{u \neq k} |\mathbf{h}_{k}^{\textsf{H}} {\mathbf{B}}_{\ell',g'}{\mathbf{p}}_{u} |^{2} \\ &=&\sum_{u \in\mathcal{I}_{\mathrm{n},k}} \rho|\mathbf{h}_{k}^{\textsf{H}} {\mathbf{B}}_{\ell',g'}{\mathbf{p}}_{u} |^{2} + \sum_{u' \in\mathcal{I}_{\mathrm{y},k} }\rho |\mathbf{h}_{k}^{\textsf{H}} {\mathbf{B}}_{\ell',g'}{\mathbf{p}}_{u'} |^{2} \\ &=& \sum_{u \in\mathcal{I}_{\mathrm{n},k}} \rho| \check{\mathbf{g}}_{k,\ell',g'}^{\textsf{H}} {\mathbf{p}}_{u} |^{2} + \sum_{u' \in\mathcal{I}_{\mathrm{y},k} }\rho| \check{\mathbf{g}}_{k,\ell',g'}^{\textsf{H}} {\mathbf{p}}_{u'} |^{2}, \\ &&u,u' \in \mathcal{S}_{\ell',g'},u,u' \neq k,\, \forall \ell',g', \end{array} $$

(49)

where $\check {\mathbf {g}}_{k,\ell ',g'} \triangleq {\mathbf {B}}^{\textsf {H}}_{\ell ',g'} \mathbf {h}_{k}$. When $u \in \mathcal {I}_{\mathrm {n},k} \bigcap \mathcal {S}_{\ell ',g'}$ and u≠k, the interference power is

$$ {{\begin{aligned} {}\rho| \check{\mathbf{g}}_{k,\ell',g'}^{\textsf{H}} {\mathbf{p}}_{u} |^{2} &= \rho {\lambda_{u}} \check{\mathbf{g}}_{k,\ell',g'}^{\textsf{H}} \mathbf{C}_{u }^{-1} \check{\mathbf{h}}_{u} \check{\mathbf{h}}_{u }^{\textsf{H}} \mathbf{C}_{u }^{-1} \check{\mathbf{g}}_{k,\ell',g'} \\ &\asymp \frac{\rho \lambda_{u}\beta_{k} }{b^{2}} \check{\mathbf{h}}_{u }^{\textsf{H}} (b\mathbf{C}_{u }^{-1}){\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{k} {\mathbf{B}}_{\ell',g'} (b \mathbf{C}_{u }^{-1}) \check{\mathbf{h}}_{u} \\ \end{aligned}}} $$

(50)

$$ {{\begin{aligned} &{\kern28pt}\asymp \frac{\rho \lambda_{u}\beta_{k}\beta_{u} }{b^{2}} \text{tr} \left(\! {\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{u} {\mathbf{B}}_{\ell',g'} (b \mathbf{C}_{u }^{-1}){\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{k} {\mathbf{B}}_{\ell',g'} \right.\\ &{\kern28pt}\left. \times (b \mathbf{C}_{u }^{-1}) {\vphantom{{\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{u} {\mathbf{B}}_{\ell',g'} (b \mathbf{C}_{u }^{-1}){\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{k} {\mathbf{B}}_{\ell',g'}}}\right) \\ \end{aligned}}} $$

(51)

$$ {{\begin{aligned} &{\kern28pt}\asymp \frac{\rho \lambda_{u}\beta_{k}\beta_{u} }{b^{2}} \text{tr} \left({\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{u} {\mathbf{B}}_{\ell',g'} \mathbf{T}'_{u,k} \right) \end{aligned}}} $$

(52)

where $\mathbf {C}_{u} \triangleq {1\over \rho }{\mathbf {I}}_{b}+\check {\mathbf {G}}_{u} \check {\mathbf {G}}_{u }^{\textsf {H}}$, (50)–(51) are obtained using Lemma 4 in [3], and (52) is based on Theorem 2 in [3] where T u,k′ is given by (33). When $u' \in \mathcal {I}_{\mathrm {y},k} \bigcap \mathcal {S}_{\ell ',g'}$ and u ^′≠k, the interference power is

$$ {{\begin{aligned} {}\rho| \check{\mathbf{g}}_{k,\ell',g'}^{\textsf{H}} {\mathbf{p}}_{u'} |^{2} &= \rho \lambda_{u'} \check{\mathbf{g}}_{k,\ell',g'}^{\textsf{H}} \mathbf{C}_{u'}^{-1} \check{\mathbf{h}}_{u'} \check{\mathbf{h}}_{u' }^{\textsf{H}} \mathbf{C}_{u'}^{-1} \check{\mathbf{g}}_{k,\ell',g'} \\ &\asymp \frac{\rho \lambda_{u'}\beta_{u'} }{b} \text{tr} \left({\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{u'} {\mathbf{B}}_{\ell',g'} b\mathbf{C}_{u'}^{-1} {\frac{\check{\mathbf{g}}_{k,\ell',g'}}{\sqrt{b}} \frac{\check{\mathbf{g}}_{k,\ell',g'}^{\textsf{H}}}{\sqrt{b}}} b\mathbf{C}_{u'}^{-1} \right) \end{aligned}}} $$

(53)

$$ {{\begin{aligned} {\kern31pt}= \frac{\rho \lambda_{u'}\beta_{u'} \text{tr} \left({\mathbf{B}}^{\textsf{H}}_{\ell',g'} {\mathbf{R}}_{u'} {\mathbf{B}}_{\ell',g'} \frac{\mathbf{D}_{u',k}^{-1}\check{\mathbf{g}}_{k,\ell',g'} }{ \sqrt{b} } \frac{\check{\mathbf{g}}_{k,\ell',g'}^{\textsf{H}} \mathbf{D}_{u',k}^{-1}}{ \sqrt{b} } \right)}{b \left(1+ \frac{\check{\mathbf{g}}^{\textsf{H}}_{k,\ell',g'}}{ \sqrt{b} } \mathbf{D}_{u',k}^{-1} \frac{\check{\mathbf{g}}_{k,\ell',g'} }{ \sqrt{b} } \right)^{2}} \end{aligned}}} $$

(54)

$$ {{\begin{aligned} {\kern31pt}\asymp \frac{ \rho \lambda_{u'}\beta_{u'}\beta_{k} \text{tr} \left(\!{\mathbf{B}}_{\ell',g'}^{\textsf{H}} {\mathbf{R}}_{k}{\mathbf{B}}_{\ell',g'} \mathbf{D}_{u',k}^{-1} {\mathbf{B}}_{\ell',g'}^{\textsf{H}} {\mathbf{R}}_{u'}{\mathbf{B}}_{\ell',g'} \mathbf{D}_{u',k}^{-1} \right) }{ b^{2}\left(1\,+\, \frac{\beta_{k}}{b} \text{tr}\left({\mathbf{B}}_{\ell',g'}^{\textsf{H}} {\mathbf{R}}_{k} {\mathbf{B}}_{\ell',g'} \mathbf{D}_{u',k}^{-1} \right)\right)^{2}} \end{aligned}}} $$

(55)

$$ {{\begin{aligned} {\kern31pt}\asymp \frac{\rho \lambda_{u'}\beta_{u'}\beta_{k} \text{tr} \left({\mathbf{B}}_{\ell',g'}^{\textsf{H}} {\mathbf{R}}_{k}{\mathbf{B}}_{\ell',g'} \mathbf{S}'_{u',k} \right)}{ b^{2}\left(1+ \frac{\beta_{k}}{b} \text{tr}\left({\mathbf{B}}_{\ell',g'}^{\textsf{H}} {\mathbf{R}}_{k} {\mathbf{B}}_{\ell',g'} \mathbf{S}'_{u',k} \right)\right)^{2}} \end{aligned}}} $$

(56)

where (53) and (55) come from Lemma 4 in [3], (54) is obtained using Lemma 1 in [3] where

$$\begin{array}{@{}rcl@{}} \mathbf{D}_{u',k} = {{1\over {\rho b}}{\mathbf{I}}}+ \sum_{s \in \mathcal{T }_{u'}, s\neq k} \frac{\check{\mathbf{g}}_{s,\ell',g'} \check{\mathbf{g}}_{s,\ell',g'}^{\textsf{H}}}{b}. \end{array} $$

(57)

(56) is based on Theorem 2 in [3], and S u ^′,k′ is given by (37).

Referring to Lemma 4 and Theorem 2 in [3], we approximate the regularization parameter in (19) by

$$\begin{array}{@{}rcl@{}} \lambda_{k}= {1 \over {\frac{\beta_{k}}{b^{2}} \text{tr} \left({\mathbf{B}}^{\textsf{H}}_{\ell,g}{\mathbf{R}}_{k}{\mathbf{B}}_{\ell,g} \mathbf{T}'_{\lambda_{k}} \right)}}, \end{array} $$

(58)

where $\mathbf {T}'_{\lambda _{k}}$ is given by (43).

Inserting (48), (52), (56) and (58) into (27), we obtain (30), which concludes the proof. □

5 Numerical results

In this section, we evaluate the performance of the proposed scheme by numerical simulations. The performance metric is sum rate in (26). We assume that the BS is elevated and seldom obstructed by local scatters, while users are located on the street level and surrounded by scatters. The BS employs an N×M URA with M antennas in each row and N rows in the elevation direction. The default configuration parameters of this section are the following: the BS antenna spacing divided by the carrier wavelength in (4) is D=0.5; the AoA and the associated AS in the azimuth direction are randomly distributed in [−180°,180°] and [0,20°], respectively; in the elevation direction, the AoA and AS intervals are [−90°,90°] and [0°,10°], respectively; the cell radius is 1600 m and the cell-hole radius is 100 m, i.e., the distance between the kth user and the BS is in the range of [100,1600] m when considering the path loss $\beta _{k}=\frac {1}{r_{k}^{\gamma }}$ in (2), and the path loss exponent is γ=3.8.

First, we show the tightness of the SINR approximations with simulation results in Fig. 4. The number of elevation BS antennas is N=10 and the number of azimuth BS antennas takes values in [10,30,50,70,90]. The number of users is K=10. The cell is split into G=4 subsectors and L=4 annular regions. The rank of azimuth DFT prebeamformer is $b_{\text {az}}=\left \lfloor {M \over G} \right \rfloor =[2,7,12,17,22]$ and the rank of elevation DFT prebeamformer is $b_{\text {el}}=\left \lfloor {N \over L} \right \rfloor =2$. The dimension of effective channel is b=b _az b _el=[4,14,24,34,44]. We can see that with the increasing number of BS antennas, the approximations are approaching to the numerical ones. It is validated that the deterministic equivalent approximations are tight when b and K are not infinitely large, which is suitable for practical scenarios. So, we will use the approximate SINRs for performance evaluation instead of the lengthy Monte Carlo simulations in the following parts.

Fig. 4

Sum rate (bps/Hz) versus number of BS antennas for the validation of SINR approximations

Then, we compare two user grouping algorithms in Fig. 5. One is the 2D-FQUG algorithm and the other is overall grouping method based on (12). The number of users is K=10. We evaluate two groups of performance. One is that the number of azimuth BS antennas is varying as M=[80,100,120] when the number of elevation BS antennas is fixed as N=10. The other is that N=[10,20,30] with M=80. Both groups of parameters are large enough to guarantee the tightness of asymptotic SINR. The cell is split into G=4 subsectors and L=4 annular regions. Obviously, these two algorithms are equivalent. Tables 1, 2, 3, 4, and 5 show that the complexities of 2D-FQUG and overall algorithms with varying M, N, G, L, and K, respectively, based on (13) and (14). We find that the proposed 2D-FQUG algorithm has lower complexity than the overall one under all circumstances. Based on the analysis in Section 3.2, the complexities of algorithms mainly come from the operations of matrix product and EVD, involving channel correlations, eigenvector matrices of channel correlations, and the DFT prebeamforming. According to Tables 1 and 2, with the increasing number of BS antennas and other fixed parameters, the complexities of both algorithms become higher, since the dimensions of matrices become larger. In Tables 3 and 4, the complexities are reduced as the number of groups is increasing, but the tendency is not very evident. This is because only the dimension of the DFT prebeamforming is reduced and other matrices are not affected since the number of BS antennas is fixed. In Table 5, we change the number of users while keeping other parameters constant, which causes an evident increase in complexities of both algorithms, due to the increasing number of summation of the complexity of each user.

Fig. 5

Sum rate (bps/Hz) versus transmit power (dBw) using 2D-FQUG and overall algorithms

Table 1 Complexities of 2D-FQUG and overall algorithms with N=10,G=L=4, and K=10

Table 2 Complexities of 2D-FQUG and overall algorithms with M=80,G=L=4, and K=10

Table 3 Complexities of 2D-FQUG and overall algorithms with M=80,N=10,L=4, and K=10

Table 4 Complexities of 2D-FQUG and overall algorithms with M=10,N=80,G=4, and K=10

Table 5 Complexities of 2D-FQUG and overall algorithms with M=80,N=10, and G=L=4

The performance of our proposed multi-user precoding scheme is illustrated in Fig. 6, and the associated complexities are summarized in Table 6. The transmit power is ρ=20 dBw. The number of users is K=20. The number of azimuth BS antennas is M=64 and the number of elevation BS antennas is N=12. The number of subsectors is G=[4,8,16,32,64], the number of annular regions is L=4, and the total number of groups is G L=[16,32,64,128,256]. For comparison, we evaluate three additional schemes, which are the SLNR precoding for suppressing all-group and intra-group interferences denoted by “all groups” and “one group” in the figure, respectively, and the generalized maximum ratio transmission (MRT) as a benchmark [38]. The generalized MRT for the kth user is p _k,mrt=u _k,az⊗u _k,el, where u _k,az and u _k,el are the eigenvectors associated with the largest eigenvalues of R _k,az and R _k,el, respectively. So, u _k,az⊗u _k,el is the eigenvector corresponding to the largest eigenvalue of R _k . Without user grouping, the received signal of the kth user in (9) is rewritten as $y_{k}= \sqrt \rho {\mathbf {h}}^{\textsf {H}}_{k} \sum _{u} {\mathbf {p}}_{u,\text {mrt}} d_{u} + n_{k}$.

Fig. 6

Sum rate (bps/Hz) versus number of groups with different multi-user precoding schemes

Table 6 Complexities (FLOPs) of four schemes in Fig. 6 with M=64,N=12,L=4, and K=20

The complexities of our proposal and the schemes for suppressing all-group and intra-group interferences in Table 6 are the sum of the complexities of 2D-FQUG user grouping algorithm and the SLNR precoding, which are given by C _f and C _g, in (13) and (23) respectively. The complexity of the generalized MRT originates only from the precoding approach. For each user, the generalized MRT involves EVD of R _k,az and R _k,el and the Kronecker product of u _k,az and u _k,el, which require M ³+N ³ and MN FLOPs, respectively. Thus, its total complexity of all users is (M ³+N ³+M N)K FLOPs. In Table 6, the generalized MRT has the lowest complexity among all schemes.

Considering the three schemes which are our proposal and the ones for suppressing all-group and intra-group interferences, the dimension of the effective channel vector is $b=b_{\text {az}}b_{\text {el}}=\left \lfloor {M \over G} \right \rfloor \left \lfloor {N \over L} \right \rfloor$. So, the determination of b is equivalent to that of the number of groups GL, when the number of BS antennas is fixed. In Sections 3.2 and 3.3, we have concluded theoretically that with larger GL and smaller b, the complexities of both 2D-FQUG user grouping algorithm and SLNR precoding are reduced. From Table 6, the complexities of our proposal and the schemes for suppressing all-group and intra-group interferences are decreasing with larger number of groups. Besides, the choice of b and GL also affects the sum-rate performance, as shown in Fig. 6. In Fig. 6, with the increasing number of groups, the performance is decreasing. The scheme considering all-group interferences has gradually worsening performance. This is because when the number of groups increases, the ranks of both the DFT prebeamforming and the effective channels become smaller. Some significant eigenmodes are not taken into account by the SLNR precoding, which leads to larger interference and worse sum-rate performance. For our proposal, another reason for degrading the performance is that when the cell is partitioned into more groups, the inter-group interferences come from not only adjacent groups but also distant groups, so the inter-group interferences become severe and the proposed algorithm does not work efficiently and achieves almost the same performance as the SLNR precoding for suppressing the intra-group interferences. And with the increasing number of groups, users are more likely to be partitioned into individual groups, so the performance of the SLNR precoding for suppressing the intra-group interferences becomes worse. When the number of subsectors is G=64 and the number of groups is G L=256, the rank of the DFT prebeamforming for each subsector is $b_{\text {az}}=\left \lfloor {M \over G} \right \rfloor =1$, which approximates the generalized MRT. So, the performances of the SLNR precoding for suppressing the intra-group interferences and our proposal approach to that of the generalized MRT when G L=256.

There is a trade-off between the sum-rate performance and complexity. The SLNR precoding for suppressing the all-group interferences is the best among these four schemes. But it has the highest complexity. The generalized MRT has the lowest complexity but the poorest performance. Our proposal works better than the one considering the intra-group interferences and the generalized MRT, with moderate complexity. So, the proposed scheme is a good balance between performance and complexity.

Tables 7, 8, 9, 10, and 11 show the complexity comparison of three SLNR precoding approaches with varying different parameters. The complexity of the proposal lies between the SLNR precoding considering all-group and intra-group interferences. The complexities are reduced with larger number of groups and become higher with more BS antennas and users.

Table 7 Complexities of three SLNR precoding approaches with M=64,N=12,L=4, and K=20

Table 8 Complexities of three SLNR precoding approaches with M=12,N=64,G=4, and K=20

Table 9 Complexities of three SLNR precoding approaches with N=64,L=G=4, and K=20

Table 10 Complexities of three SLNR precoding approaches with M=64,L=G=4, and K=20

Table 11 Complexities of three SLNR precoding approaches with M=64,N=12, and L=G=4

6 Conclusions

In this paper, we proposed a low-complexity 3D beamforming scheme for the massive MIMO system. We used two-stage beamforming where the second-stage prebeamforming is a Kronecker product of azimuth and elevation DFT prebeamformers. These DFT prebeamformers are used for cell splitting. We developed a low-complexity 2D-FQUG user grouping algorithm using only statistical CSI to partition users into groups. The first-stage beamforming is SLNR precoding based on the effective channels formed by DFT prebeamformers and instantaneous CSI. The designed SLNR precoding considers intra-group and adjacent-group interferences. We derived the approximate SINR for our proposal. We analyzed that the complexities of both 2D-FQUG user grouping algorithm and SLNR precoding are reduced with larger number of groups and become higher with more BS antennas and users. Numerical results show that the 2D-FQUG algorithm achieves the same sum-rate performance as the overall grouping method but has lower complexity. Our proposal works better than the one considering the intra-group interferences and the generalized MRT, and its complexity lies between the SLNR precoding for suppressing all-group interferences and the one considering the intra-group interferences. So, the proposed scheme is a good balance between performance and complexity.