A New Nonconvex Sparse Recovery Method for Compressive Sensing

As an extension of the widely used ℓ_r-minimization with 0 < r ≤ 1, a new non-convex weighted ℓ_r − ℓ₁ minimization method is proposed for compressive sensing. The theoretical recovery results based on restricted isometry property and q-ratio constrained minimal singular values are established. An algorithm that integrates the iteratively reweighted least squares algorithm and the difference of convex functions algorithm is given to approximately solve this non-convex problem. Numerical experiments are presented to illustrate our results.

1. Introduction

Compressive sensing (CS) has attracted a great deal of interests since its advent [1, 2], see the monographs [3, 4] and the references therein for a comprehensive view. Basically, the goal of CS is to recover an unknown (approximately) sparse signal x ∈ ℝ^N from the noisy underdetermined linear measurements

$\begin{array}{ll}y=Ax+e\in {ℝ}^m,\hfill & (1)\end{array}$

with m ≪ N, A ∈ ℝ^m×N being the pre-given measurement matrix and e ∈ ℝ^m being the noise vector. If the measurement matrix A satisfies some kinds of incoherence conditions (e.g., mutual coherence condition [5, 6], restricted isometry property (RIP) [7, 8], null space property (NSP) [9, 10], or constrained minimal singular values (CMSV) [11, 12]), then stable (w.r.t. sparsity defect) and robust (w.r.t. measurement error) recovery results can be guaranteed by using the constrained ℓ₁-minimization [13]:

$\begin{array}{cc}\underset{z\in {ℝ}^N}{\text{min}}\Vert z{\Vert }_1\text{subject to}\Vert Az-y{\Vert }_2\le \eta .& (2)\end{array}$

Here the ℓ₁-minimization problem works as a convex relaxation of ℓ₀-minimization problem, which is NP-hard to solve [14].

Meanwhile, non-convex recovery algorithms such as the ℓ_r-minimization (0 < r < 1) have been proposed to enhance sparsity [15–20]. ℓ_r-minimization enables one to reconstruct the sparse signal from fewer number of measurements compared to the convex ℓ₁-minimization, although it is more challenging to solve because of its non-convexity. Fortunately, an iteratively reweighted least squares (IRLS) algorithm can be applied to approximately solve this non-convex problem in practice [21, 22].

As an extension of the ℓ_r-minimization, we study in this paper the following weighted ℓ_r − ℓ₁ minimization problem for sparse signal recovery:

$\begin{array}{ll}\underset{z\in {ℝ}^N}{\text{min}}\Vert z{\Vert }_r^r-\alpha \Vert z{\Vert }_1^r\text{ subject to}\Vert Az-y{\Vert }_2\le \eta ,\hfill & (3)\end{array}$

where y = Ax + e with ∥e∥₂ ≤ η, 0 ≤ α ≤ 1, and 0 < r ≤ 1. Throughout the paper, we assume that α ≠ 1 when r = 1. Obviously, it reduces to the traditional ℓ_r-minimization problem when α = 0. This hybrid norm model is inspired by the non-convex Lipschitz continuous ℓ₁ − ℓ₂ model (minimizing the difference of ℓ₁ norm and ℓ₂ norm) proposed in Lou et al. [23] and Yin et al. [24], which improves the ℓ₁-minimization in a robust manner, especially for the highly coherent measurement matrices. Roughly speaking, the underlying logic of adopting these kinds of norm differences or the ratios of norms [25] comes from the fact that they can be viewed as sparsity measures, see the effective sparsity measure called q-ratio sparsity (involving the ratio of ℓ₁ norm and ℓ_q norm) defined later in Definition 2 of section 2.2. Other recent related literatures include [26–29], to name a few.

To illustrate these weighted ℓ_r − ℓ₁ norms ($\parallel ·{\parallel }_r^r-\alpha \parallel ·{\parallel }_1^r$), we present their corresponding contour plots in Figure 1¹. As is shown, different non-convex patterns arise while varying the difference weight α or the norm order r. And the level curves of weighted ℓ_r − ℓ₁ norms approach the x and y axes as the norm values get small, which reflects their ability to promote sparsity. In the present paper, we shall focus on both the theoretical aspects and the computational study for this non-convex sparse recovery method.

FIGURE 1

Figure 1. The contour plots for weighted ℓ_r − ℓ₁ norms with different α and r. The first row corresponds to the cases that r = 0.5 with different α = 0, 0.3, 1, while the second row shows the cases that r = 0.1, 0.4, 0.7 with fixed α = 1.

This paper is organized as follows. In section 2, we derive the theoretical performance bounds for the weighted ℓ_r − ℓ₁ minimization based on both r-RIP and q-ratio CMSV. In section 3, we give an algorithm to approximately solve the unconstrained version of the weighted ℓ_r − ℓ₁ minimization problem. Numerical experiments are provided in section 4. Section 5 concludes with a brief summary and an outlook on future extensions.

2. Recovery Analysis

In this section, we establish the theoretical performance bounds for the reconstruction error of the weighted ℓ_r − ℓ₁ minimization problem, based on both r-RIP and q-ratio CMSV. Hereafter, we say a signal x ∈ ℝ^N is s-sparse if $\parallel x{\parallel }_0=\sum _{i=1}^{N}1\left\{x_i\ne 0\right\}\le s$, and denote by x_S the vector that coincides with x on the indices in S ⊆ [N]: = {1, 2, ⋯ , N} and takes zero outside S.

2.1. r-RIP

We start with the definition of the s-th r-restricted isometry constant, which was introduced in Chartrand and Staneva [30].

Definition 1. ([30]) For integer s > 0 and 0 < r ≤ 1, the s-th r-restricted isometry constant (RIC) δ_s = δ_s(A) of a matrix A ∈ ℝ^m×N is defined as the smallest δ ≥ 0 such that

$\begin{array}{ll}(1-\delta )\Vert x{\Vert }_2^r\le \Vert Ax{\Vert }_r^r\le (1+\delta )\Vert x{\Vert }_2^r\hfill & (4)\end{array}$

for all s-sparse vectors x ∈ ℝ^N.

Then, the r-RIP means that the s-th r-RIC δ_s is small for reasonably large s. In Chartrand and Staneva [30], the authors established the recovery analysis result for ℓ_r-minimization problem based on this r-RIP. To extend this to the weighted ℓ_r − ℓ₁ minimization problem, the following lemma plays a crucial role.

Lemma 1. Suppose x ∈ ℝ^N, 0 ≤ α ≤ 1 and 0 < r ≤ 1, then we have

$\begin{array}{cc}(N-\alpha N^r){\left(\underset{i\in [N]}{\text{min}}|x_i|\right)}^r\le \Vert x{\Vert }_r^r-\alpha \Vert x{\Vert }_1^r\le (N^{1-r}-\alpha )\Vert x{\Vert }_1^r.& (5)\end{array}$

In particular, when S = supp(x) ⊆ [N] and |S| = s, then

$\begin{array}{ll}(s-\alpha s^r){\left(\underset{{}_{i\in S}}{\text{min}}|x_i|\right)}^r\le \Vert x{\Vert }_r^r-\alpha \Vert x{\Vert }_1^r\le (s^{1-r}-\alpha )\Vert x{\Vert }_1^r.\hfill & (6)\end{array}$

Proof. The right hand side of (5) follows immediately from the norm inequality $\parallel x{\parallel }_r\le N^{1/r-1}\parallel x{\parallel }_1$ for any x ∈ ℝ^N and 0 < r ≤ 1. As for the left hand side, it holds trivially if min_i∈[N] |x_i| = 0. When min_i∈[N] |x_i| ≠ 0, by dividing (min_i∈[N] |x_i|)^r on both sides, it is equivalent to show that

$\begin{array}{ll}{\displaystyle \sum _{j=1}^N{\left(\frac{|x_j|}{{\text{min}}_{i\in [N]}|x_i|}\right)}^r}-\alpha {\left({\displaystyle \sum _{j=1}^N\frac{|x_j|}{{\text{min}}_{i\in [N]}|x_i|}}\right)}^r\ge N-\alpha N^r.\hfill & (7)\end{array}$

By denoting $a_j=\frac{\left|x_j\right|}{{\min }_{i\in [N]}\left|x_i\right|}$, we have a_j ≥ 1 for any 1 ≤ j ≤ N, and to show (7) it suffices to show

${\displaystyle \sum _{j=1}^N{a}_j^r-\alpha {\left({\displaystyle \sum _{j=1}^N{a}_j}\right)}^r\ge N-\alpha N^r.}$

Assume the function $f(a_1,a_2,\cdots ,a_N)={\displaystyle \sum _{j=1}^N}{a}_j^r-\alpha {\left({\displaystyle \sum _{j=1}^N}{a}_j\right)}^r$. Then, as a result of

$\frac{\partial f}{\partial a_k}=r{a}_k^{r-1}-\alpha r{\left({\displaystyle \sum _{j=1}^N{a}_j}\right)}^{r-1}>0\text{ \hspace{0.05em}for any 1}\le k\le N,$

we have $f(a_1,a_2,\cdots ,a_N)\ge f(1,1,\cdots ,1)=N-\alpha N^r$. Thus, the left hand side of (5) holds and the proof is completed. (6) follows as we apply (5) to x_S.

Now, we are ready to present the r-RIP based bound for the ℓ₂ norm of the reconstruction error.

Theorem 1. Let the ℓ_r-error of best s-term approximation of x be ${\sigma }_s{(x)}_r=inf\left\{\parallel x-z{\parallel }_r,z\in {ℝ}^N\mathit{\text{is }}s\mathit{\text{-sparse}}\right\}$. We assume that a > 0 is properly chosen so that as is an integer. If

$\begin{array}{ll}b=\frac{{(as)}^{1-\frac{r}{2}}-\alpha {(as)}^{\frac{r}{2}}}{s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}}}>1\hfill & (8)\end{array}$

and suppose the measurement matrix A satisfies the condition

$\begin{array}{ll}{\delta }_{as}+b{\delta }_{(a+1)s}<b-1,\hfill & (9)\end{array}$

then any solution $\widehat{x}$ to the minimization problem (3) obeys

$\begin{array}{ll}\Vert \widehat{x}-x{\Vert }_2\le C_1{m}^{1/r-1/2}\eta +C_2{(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})}^{-1/r}{\sigma }_s{(x)}_r\hfill & (10)\end{array}$

with $C_1=\frac{2^{1/r}(1+b^{1/r})}{{(b-b{\delta }_{(a+1)s}-1-{\delta }_{as})}^{1/r}}$ and $C_2=\frac{2^{2/r-1}\left[{(1+{\delta }_{as})}^{1/r}+{(1-{\delta }_{(a+1)s})}^{1/r}\right]}{{(b-b{\delta }_{(a+1)s}-1-{\delta }_{as})}^{1/r}}$.

Proof. We assume that S is the index set that contains the largest s absolute entries of x so that ${\sigma }_s{(x)}_r=\parallel x_{S^c}{\parallel }_r$ and let $h=\widehat{x}-x$. Then we have

$\begin{array}{l}\Vert x_S{\Vert }_r^r+\Vert x_{S^c}{\Vert }_r^r-\alpha \Vert x{\Vert }_1^r\\ =\Vert x{\Vert }_r^r-\alpha \Vert x{\Vert }_1^r\\ \ge \Vert \widehat{x}{\Vert }_r^r-\alpha \Vert \widehat{x}{\Vert }_1^r\\ =\Vert x_S+x_{S^c}+h_S+h_{S^c}{\Vert }_r^r-\alpha \Vert x+h_S+h_{S^c}{\Vert }_1^r\\ \ge \Vert x_S+h_S{\Vert }_r^r+\Vert x_{S^c}+h_{S^c}{\Vert }_r^r-\alpha {\left(\Vert x+h_S{\Vert }_1+\Vert h_{S^c}{\Vert }_1\right)}^r\\ \ge \Vert x_S{\Vert }_r^r-\Vert h_S{\Vert }_r^r+\Vert h_{S^c}{\Vert }_r^r-\Vert x_{S^c}{\Vert }_r^r-\alpha \Vert x+h_S{\Vert }_1^r-\alpha \Vert h_{S^c}{\Vert }_1^r\\ \ge \Vert x_S{\Vert }_r^r-\Vert h_S{\Vert }_r^r+\Vert h_{S^c}{\Vert }_r^r-\Vert x_{S^c}{\Vert }_r^r\\ -\alpha \Vert x{\Vert }_1^r-\alpha \Vert h_S{\Vert }_1^r-\alpha \Vert h_{S^c}{\Vert }_1^r,\end{array}$

which implies

$\begin{array}{ll}\Vert h_{S^c}{\Vert }_r^r-\alpha \Vert h_{S^c}{\Vert }_1^r\le \Vert h_S{\Vert }_r^r+\alpha \Vert h_S{\Vert }_1^r+2\Vert x_{S^c}{\Vert }_r^r.\hfill & (11)\end{array}$

Using the Holder's inequality, we obtain

$\Vert Ah{\Vert }_r^r\le {\left({{\displaystyle \sum _{i=1}^m\left({|{\left(Ah\right)}_i|}^r\right)}}^{2/r}\right)}^{r/2}·{\left({\displaystyle \sum _{i=1}^{m}1}\right)}^{1-r/2}=m^{1-r/2}\Vert Ah{\Vert }_2^r.$

By ∥Ax − y∥₂ = ∥e∥₂ ≤ η and the triangular inequality,

$\begin{array}{ll}\Vert Ah{\Vert }_2\hfill & =\Vert (A\widehat{x}-y)-(Ax-y){\Vert }_2\hfill \\ \hfill & \le \Vert A\widehat{x}-y{\Vert }_2+\Vert Ax-y{\Vert }_2\le 2\eta .\hfill & (12)\end{array}$

Thus,

$\begin{array}{ll}\Vert Ah{\Vert }_r^r\le m^{1-r/2}\Vert Ah{\Vert }_2^r\le m^{1-r/2}{(2\eta )}^r.\hfill & (13)\end{array}$

Arrange $S^c=S_1\cup S_2\cup \cdots$, where S₁ is the index set of M = as largest absolute entries of h in S^c, S₂ is the index set of M largest absolute entries of h in ${(S\cup S_1)}^c$, etc. And we denote S₀ = S∪S₁. Then, by adopting Lemma 1, for each i ∈ S_k, k ≥ 2,

$\begin{array}{ll}|h_i|\le \underset{j\in S_{k-1}}{\min }|h_j|\Rightarrow |h_i{|}^r\le \text{}{\left(\underset{j\in S_{k-1}}{\min }\text{}|h_j|\text{}\right)}^r\le \frac{\Vert h_{S_{k-1}}{\Vert }_r^r-\alpha \Vert h_{S_{k-1}}{\Vert }_1^r}{M-\alpha M^r}.\hfill & (14)\end{array}$

Thus we have $\parallel h_{S_k}{\parallel }_2^r={\left({\displaystyle \sum _{i\in S_k}}|h_i{|}^2\right)}^{r/2}\le M^{r/2}\frac{\parallel h_{S_{k-1}}{\parallel }_r^r-\alpha \parallel h_{S_{k-1}}{\parallel }_1^r}{M-\alpha M^r}=\frac{\parallel h_{S_{k-1}}{\parallel }_r^r-\alpha \parallel h_{S_{k-1}}{\parallel }_1^r}{M^{1-\frac{r}{2}}-\alpha M^{\frac{r}{2}}}$. Hence it follows that

$\begin{array}{ll}{\displaystyle \sum _{k\ge 2}\Vert h_{S_k}{\Vert }_2^r}\le \frac{{\displaystyle \sum _{k\ge 1}\left({\Vert h_{S_k}\Vert }_r^r-\alpha {\Vert h_{S_k}\Vert }_1^r\right)}}{M^{1-\frac{r}{2}}-\alpha M^{\frac{r}{2}}}=\frac{{\displaystyle \sum _{k\ge 1}{\Vert h_{S_k}\Vert }_r^r}-\alpha {\displaystyle \sum _{k\ge 1}{\Vert h_{S_k}\Vert }_1^r}}{M^{1-\frac{r}{2}}-\alpha M^{\frac{r}{2}}}\text{}.\hfill & (15)\end{array}$

Note that

${\displaystyle \sum _{k\ge 1}{\Vert h_{S_k}\Vert }_r^r={\Vert h_{S^c}\Vert }_r^r}\text{ and }{\displaystyle \sum _{k\ge 1}{\Vert h_{S_k}\Vert }_1^r}\ge \Vert {\displaystyle \sum _{k\ge 1}{h}_{S_k}}{\Vert }_1^r={\Vert h_{S^c}\Vert }_1^r,$

therefore, with (11), it holds that

$\begin{array}{l}{\displaystyle \sum _{k\ge 2}\Vert h_{S_k}{\Vert }_2^r}\le \frac{\Vert h_{S^c}{\Vert }_r^r-\alpha \Vert h_{S^c}{\Vert }_1^r}{M^{1-\frac{r}{2}}-\alpha M^{\frac{r}{2}}}\\ \le \frac{\Vert h_S{\Vert }_r^r+\alpha \Vert h_S{\Vert }_1^r+2\Vert x_{S^c}{\Vert }_r^r}{M^{1-\frac{r}{2}}-\alpha M^{\frac{r}{2}}}\\ \le \frac{s^{1-\frac{r}{2}}\Vert h_S{\Vert }_2^r+\alpha s^{\frac{r}{2}}\Vert h_S{\Vert }_2^r+2\Vert x_{S^c}{\Vert }_r^r}{M^{1-\frac{r}{2}}-\alpha M^{\frac{r}{2}}}\\ \le \frac{(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})\Vert h_{S_0}{\Vert }_2^r+2\Vert x_{S^c}{\Vert }_r^r}{M^{1-\frac{r}{2}}-\alpha M^{\frac{r}{2}}}.& (16)\end{array}$

Meanwhile, according to the definition of r-RIC, we have

$\begin{array}{l}\Vert Ah{\Vert }_r^r=\Vert A{h}_{S_0}+{\displaystyle \sum _{k\ge 2}A}{h}_{S_k}{\Vert }_r^r\\ \ge \Vert A{h}_{S_0}{\Vert }_r^r-\Vert {\displaystyle \sum _{k\ge 2}A}{h}_{S_k}{\Vert }_r^r\\ \ge \Vert A{h}_{S_0}{\Vert }_r^r-{\displaystyle \sum _{k\ge 2}\Vert A{h}_{S_k}{\Vert }_r^r}\\ \ge (1-{\delta }_{M+s})\Vert h_{S_0}{\Vert }_2^r-(1+{\delta }_M){\displaystyle \sum _{k\ge 2}\Vert h_{S_k}{\Vert }_2^r}.\end{array}$

Thus by using (16), it follows that

$\begin{array}{l}\Vert Ah{\Vert }_r^r\ge (1-{\delta }_{M+s})\Vert h_{S_0}{\Vert }_2^r-(1+{\delta }_M)·\frac{(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})\Vert h_{S_0}{\Vert }_2^r+2\Vert x_{S^c}{\Vert }_r^r}{M^{1-\frac{r}{2}}-\alpha M^{\frac{r}{2}}}\\ =\left(1-{\delta }_{M+s}-\frac{1+{\delta }_M}{b}\right)\Vert h_{S_0}{\Vert }_2^r-\frac{2(1+{\delta }_M)}{M^{1-\frac{r}{2}}-\alpha M^{\frac{r}{2}}}\Vert x_{S^c}{\Vert }_r^r,\end{array}$

where $b=\frac{M^{1-\frac{r}{2}}-\alpha M^{\frac{r}{2}}}{s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}}}=\frac{{(as)}^{1-\frac{r}{2}}-\alpha {(as)}^{\frac{r}{2}}}{s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}}}$. Therefore, if δ_M + bδ_M+s < b − 1, then it yields that

$\begin{array}{ll}\Vert h_{S_0}{\Vert }_2^r\hfill & \le \frac{b}{b-b{\delta }_{M+s}-1-{\delta }_M}\Vert Ah{\Vert }_r^r+\frac{2(1+{\delta }_M){(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})}^{-1}}{b-b{\delta }_{M+s}-1-{\delta }_M}\Vert x_{S^c}{\Vert }_r^r\hfill \\ \hfill & \le \frac{b{m}^{1-r/2}{(2\eta )}^r}{b-b{\delta }_{M+s}-1-{\delta }_M}+\frac{2(1+{\delta }_M){(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})}^{-1}}{b-b{\delta }_{M+s}-1-{\delta }_M}\Vert x_{S^c}{\Vert }_r^r.\hfill & (17)\end{array}$

On the other hand,

$\begin{array}{ll}{\left({\displaystyle \sum _{k\ge 2}\Vert h_{S_k}{\Vert }_2}\right)}^r\hfill & \le {\displaystyle \sum _{k\ge 2}\Vert h_{S_k}{\Vert }_2^r}\hfill \\ \hfill & \le \frac{(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})\Vert h_{S_0}{\Vert }_2^r+2\Vert x_{S^c}{\Vert }_r^r}{M^{1-\frac{r}{2}}-\alpha M^{\frac{r}{2}}}\hfill \\ \hfill & =\frac{1}{b}\Vert h_{S_0}{\Vert }_2^r+\frac{2{(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})}^{-1}}{b}\Vert x_{S^c}{\Vert }_r^r\hfill \\ \hfill & \le \frac{1}{b}(\frac{b}{b-b{\delta }_{M+s}-1-{\delta }_M}\Vert Ah{\Vert }_r^r\hfill \\ \hfill & +\frac{2(1+{\delta }_M){(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})}^{-1}}{b-b{\delta }_{M+s}-1-{\delta }_M}\Vert x_{S^c}{\Vert }_r^r)\hfill \\ \hfill & +\frac{2{(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})}^{-1}}{b}\Vert x_{S^c}{\Vert }_r^r\hfill \\ \hfill & \le \frac{1}{b-b{\delta }_{M+s}-1-{\delta }_M}\Vert Ah{\Vert }_r^r\hfill \\ \hfill & +\frac{2(1-{\delta }_{M+s}){(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})}^{-1}}{b-b{\delta }_{M+s}-1-{\delta }_M}\Vert x_{S^c}{\Vert }_r^r\hfill \\ \hfill & \le \frac{m^{1-r/2}{(2\eta )}^r}{b-b{\delta }_{M+s}-1-{\delta }_M}\hfill \\ \hfill & +\frac{2(1-{\delta }_{M+s}){(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})}^{-1}}{b-b{\delta }_{M+s}-1-{\delta }_M}\Vert x_{S^c}{\Vert }_r^r.\hfill & (18)\end{array}$

Since ${(v_1^r+v_2^r)}^{1/r}\le 2^{1/r-1}(v_1+v_2)$ for any v₁, v₂ ≥ 0, combining (17) and (18) gives

$\begin{array}{ll}\Vert h{\Vert }_2\hfill & \le \Vert h_{S_0}{\Vert }_2+{\displaystyle \sum _{k\ge 2}\Vert h_{S_k}{\Vert }_2}\hfill \\ \hfill & \le 2^{1/r-1}(\frac{2b^{1/r}{m}^{1/r-1/2}\eta }{{(b-b{\delta }_{M+s}-1-{\delta }_M)}^{1/r}}\hfill \\ \hfill & +\frac{2^{1/r}{(1+{\delta }_M)}^{1/r}{(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})}^{-1/r}}{{(b-b{\delta }_{M+s}-1-{\delta }_M)}^{1/r}}\Vert x_{S^c}{\Vert }_r)\hfill \\ \hfill & +2^{1/r-1}(\frac{2m^{1/r-1/2}\eta }{{(b-b{\delta }_{M+s}-1-{\delta }_M)}^{1/r}}\hfill \\ \hfill & +\frac{2^{1/r}{(1-{\delta }_{M+s})}^{1/r}{(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})}^{-1/r}}{{(b-b{\delta }_{M+s}-1-{\delta }_M)}^{1/r}}\Vert x_{S^c}{\Vert }_r)\hfill \\ \hfill & =\frac{2^{1/r}{m}^{1/r-1/2}(1+b^{1/r})}{{(b-b{\delta }_{(a+1)s}-1-{\delta }_{as})}^{1/r}}\eta \hfill \\ \hfill & +\frac{2^{2/r-1}[{(1+{\delta }_{as})}^{1/r}+{(1-{\delta }_{(a+1)s})}^{1/r}]}{{(b-b{\delta }_{(a+1)s}-1-{\delta }_{as})}^{1/r}}\hfill \\ \hfill & \times {(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})}^{-1/r}\Vert x_{S^c}{\Vert }_r\hfill \\ \hfill & =C_1{m}^{1/r-1/2}\eta +C_2{(s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}})}^{-1/r}{\sigma }_s{(x)}_r.\hfill & (19)\end{array}$

The proof is completed.

Based on this theorem, we can obtain the following corollary by assuming that the original signal x is s-sparse (σ_s(x)_r = 0) and the measurement vector is noise free (e = 0 and η = 0), which acts as a natural generalization of Theorem 2.4 in Chartrand and Staneva [30] from the case α = 0 to any α ∈ [0, 1].

Corollary 1. For any s-sparse signal x, if the conditions in Theorem 1 hold, then the unique solution of (3) with η = 0 is exactly x.

Remarks. Observe that r-RIP based condition for exact sparse recovery given in Chartrand and Staneva [30] reads

${\delta }_{as}<a^{1-\frac{r}{2}}(1-{\delta }_{(a+1)s})-1,$

while ours goes to

${\delta }_{as}<b\left(1-{\delta }_{(a+1)s}\right)-1$

with $b=\frac{{(as)}^{1-\frac{r}{2}}-\alpha {(as)}^{\frac{r}{2}}}{s^{1-\frac{r}{2}}+\alpha s^{\frac{r}{2}}}<a^{1-\frac{r}{2}}$ when α ∈ (0, 1]. Thus, the sufficient condition established here is slightly stronger than that for the traditional ℓ_r-minimization in Chartrand and Staneva [30] if α ∈ (0, 1].

2.2. q-Ratio CMSV

Before the discussion of q-ratio CMSV, we start with presenting the definition of q-ratio sparsity as a kind of effective sparsity measure. We list the detailed statement here for the sake of completeness.

Definition 2. ([12, 31, 32]) For any non-zero z ∈ ℝ^N and non-negative q ∉ {0, 1, ∞}, the q-ratio sparsity level of z is defined as

$\begin{array}{ll}{s}_q(z)={\left(\frac{\Vert z{\Vert }_1}{\Vert z{\Vert }_q}\right)}^{\frac{q}{q-1}}.\hfill & (20)\end{array}$

The cases of q ∈ {0, 1, ∞}are evaluated as limits: $s_0(z)={\displaystyle \underset{q\to 0}{lim}}{s}_q(z)=\parallel z{\parallel }_0$, $s_{\infty }(z)={\displaystyle \underset{q\to \infty }{lim}}{s}_q(z)=\frac{\parallel z{\parallel }_1}{\parallel z{\parallel }_{\infty }}$, and $s_1(z)={\displaystyle \underset{q\to 1}{lim}}{s}_q(z)=exp(H_1(\pi (z)))$, where π(z) ∈ ℝ^N with entries π_i(z) = |z_i|/∥z∥₁ and H₁ is the ordinary Shannon entropy $H_1(\pi (z))=-\sum _{i=1}^N{\pi }_i(z)log{\pi }_i(z)$.

We are able to establish the performance bounds for both the ℓ_q norm and ℓ_r norm of the reconstruction error via a recently developed computable incoherence measure of the measurement matrix, called q-ratio CMSV. Its definition is given as follows.

Definition 3. ([12, 32]) For any real number s ∈ [1, N], q ∈ (1, ∞], and matrix A ∈ ℝ^m×N, the q-ratio constrained minimal singular value (CMSV) of A is defined as

$\begin{array}{ll}{\rho }_{q,s}(A)=\underset{z\ne 0,s_q(z)\le s}{\min }\frac{\Vert Az{\Vert }_2}{\Vert z{\Vert }_q}.\hfill & (21)\end{array}$

Then, when the signal is exactly sparse, we have the following q-ratio CMSV based sufficient condition for valid upper bounds of the reconstruction error, which are much more concise to obtain than the r-RIP based ones.

Theorem 2. For any 1 < q ≤ ∞, 0 ≤ α ≤ 1, and 0 < r ≤ 1, if the signal x is s-sparse and the measurement matrix A satisfies the condition

$\begin{array}{ll}{\rho }_{q,{\left(\frac{2}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)>0,\hfill & (22)\end{array}$

then any solution $\widehat{x}$ to the minimization problem (3) obeys

$\begin{array}{cc}\Vert \widehat{x}-x{\Vert }_q\le \frac{2\eta }{{\rho }_{q,{\left(\frac{2}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)},& (23)\end{array}$

$\begin{array}{cc}\Vert \widehat{x}-x{\Vert }_r\le {\left(\frac{2^{r+1}}{1-\alpha }\right)}^{1/r}·\frac{s^{1/r-1/q}\eta }{{\rho }_{q,{\left(\frac{2}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)}.& (24)\end{array}$

Proof. Suppose the support of x to be S with |S| ≤ s and $h=\widehat{x}-x$, then, based on (11), we have

$\begin{array}{cc}\Vert h_{S^c}{\Vert }_r^r-\alpha \Vert h_{S^c}{\Vert }_1^r\le \Vert h_S{\Vert }_r^r+\alpha \Vert h_S{\Vert }_1^r.& (25)\end{array}$

Hence, for any 1 < q ≤ ∞, it holds that

$\begin{array}{l}\Vert h{\Vert }_r^r-\alpha \Vert h{\Vert }_1^r\le \Vert h_S{\Vert }_r^r+\Vert h_{S^c}{\Vert }_r^r-\alpha \Vert h_S{\Vert }_1^r-\alpha \Vert h_{S^c}{\Vert }_1^r\\ \le \Vert h_S{\Vert }_r^r-\alpha \Vert h_S{\Vert }_1^r+\Vert h_S{\Vert }_r^r+\alpha \Vert h_S{\Vert }_1^r\\ \le 2\Vert h_S{\Vert }_r^r\le 2s^{1-r/q}\Vert h_S{\Vert }_q^r\le 2s^{1-r/q}\Vert h{\Vert }_q^r.\end{array}$

Then since $(1-\alpha )\parallel h{\parallel }_1^r\le \parallel h{\parallel }_r^r-\alpha \parallel h{\parallel }_1^r$, it implies that $(1-\alpha )\parallel h{\parallel }_1^r\le 2s^{1-r/q}\parallel h{\parallel }_q^r$. As a consequence,

$\begin{array}{l}{s}_q(h)={\left(\frac{\Vert h{\Vert }_1}{\Vert h{\Vert }_q}\right)}^{\frac{q}{q-1}}\le {\left(\frac{2s^{1-r/q}}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}\\ ={\left(\frac{2}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}.& (26)\end{array}$

Therefore, according to the definition of q-ratio CMSV the condition (22), and the fact that ∥Ah∥₂ ≤ 2η [see (12)], we can obtain that

$\begin{array}{l}{\rho }_{q,{\left(\frac{2}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)\le \frac{\Vert Ah{\Vert }_2}{\Vert h{\Vert }_q}\Rightarrow \Vert h{\Vert }_q\le \frac{\Vert Ah{\Vert }_2}{{\rho }_{q,{\left(\frac{2}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)}\\ \le \frac{2\eta }{{\rho }_{q,{\left(\frac{2}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)},& (27)\end{array}$

which completes the proof of (23). In addition, $(1-\alpha )\parallel h{\parallel }_r^r\le \parallel h{\parallel }_r^r-\alpha \parallel h{\parallel }_1^r\le 2s^{1-r/q}\parallel h{\parallel }_q^r$ yields

$\begin{array}{l}\Vert h{\Vert }_r\le {\left(\frac{2}{1-\alpha }\right)}^{1/r}{s}^{1/r-1/q}\Vert h{\Vert }_q\\ \le {\left(\frac{2^{r+1}}{1-\alpha }\right)}^{1/r}·\frac{s^{1/r-1/q}\eta }{{\rho }_{q,{\left(\frac{2}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)}.& (28)\end{array}$

Therefore, (24) holds and the proof is completed.

Remarks. Note that the results (11) and (12) in Theorem 1 of Zhou and Yu [12] correspond to the special case of α = 0 and r = 1 in this result. As a by-product of this theorem, we have that the perfect recovery can be guaranteed for any s-sparse signal x via (3) with η = 0, if there exists some q ∈ (1, ∞] such that the q-ratio CMSV of the measurement matrix A fulfils ${\rho }_{q,{(\frac{2}{1-\alpha })}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)>0$. As studied in Zhou and Yu [12, 32], this kind of q-ratio CMSV based sufficient conditions holds with high probability for subgaussian and a class of structured random matrices as long as the number of measurements is reasonably large.

Next, we extend the result to the case that x is compressible (i.e., not exactly sparse but can be well-approximated by an exactly sparse signal).

Theorem 3. For any 1 < q ≤ ∞, 0 ≤ α ≤ 1 and 0 < r ≤ 1, if the measurement matrix A satisfies the condition

$\begin{array}{cc}{\rho }_{q,{\left(\frac{4}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)>0,& (29)\end{array}$

then any solution $\widehat{x}$ to the minimization problem (3) fulfils

$\begin{array}{cc}\Vert \widehat{x}-x{\Vert }_q\le \frac{2\eta }{{\rho }_{q,{\left(\frac{4}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)}+s^{1/q-1/r}{\sigma }_s{(x)}_r,& (30)\end{array}$

$\begin{array}{l}\Vert \widehat{x}-x{\Vert }_r\le {\left(\text{}\frac{4}{1-\alpha }\text{}\right)}^{\text{}1/r}\text{}\frac{s^{1/r-1/q}\eta }{{\rho }_{q,{\left(\frac{4}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)}\\ +{\left(\frac{4}{1-\alpha }\right)}^{1/r}{\sigma }_s{\left(x\right)}_r.& (31)\end{array}$

Proof. We assume that S is the index set that contains the largest s absolute entries of x so that ${\sigma }_s{(x)}_r=\parallel x_{S^c}{\parallel }_r$ and let $h=\widehat{x}-x$. Then we still have (11), that is,

$\begin{array}{cc}\Vert h_{S^c}{\Vert }_r^r-\alpha \Vert h_{S^c}{\Vert }_1^r\le \Vert h_S{\Vert }_r^r+\alpha \Vert h_S{\Vert }_1^r+2\Vert x_{S^c}{\Vert }_r^r.& (32)\end{array}$

As a result,

$\begin{array}{l}(1-\alpha )\Vert h{\Vert }_r^r\le \Vert h{\Vert }_r^r-\alpha \Vert h{\Vert }_1^r\\ \le \Vert h_S{\Vert }_r^r+\Vert h_{S^c}{\Vert }_r^r-\alpha \Vert h_S{\Vert }_1^r-\alpha \Vert h_{S^c}{\Vert }_1^r\\ \le 2\Vert h_S{\Vert }_r^r+2\Vert x_{S^c}{\Vert }_r^r\\ \le 2s^{1-r/q}\Vert h{\Vert }_q^r+2\Vert x_{S^c}{\Vert }_r^r& (33)\end{array}$

holds for any 1 < q ≤ ∞, 0 ≤ α ≤ 1 and 0 < r ≤ 1.

To prove (30), we assume h ≠ 0 and $\parallel h{\parallel }_q>\frac{2\eta }{{\rho }_{q,{(\frac{4}{1-\alpha })}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)}$, otherwise it holds trivially. Then

$\begin{array}{l}\Vert h{\Vert }_q>\frac{\Vert Ah{\Vert }_2}{{\rho }_{q,{\left(\frac{4}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)}\Rightarrow s_q(h)>{\left(\frac{4}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}\\ \Rightarrow \Vert h{\Vert }_1^r>\frac{4}{1-\alpha }{s}^{1-r/q}\Vert h{\Vert }_q^r,\end{array}$

which implies that $(1-\alpha )\parallel h{\parallel }_r^r\ge (1-\alpha )\parallel h{\parallel }_1^r>4s^{1-r/q}\parallel h{\parallel }_q^r$. Then combining with (33), it yields that

$\begin{array}{cc}\Vert h{\Vert }_q\le {(s^{r/q-1})}^{1/r}\Vert x_{S^c}{\Vert }_r=s^{1/q-1/r}\Vert x_{S^c}{\Vert }_r=s^{1/q-1/r}{\sigma }_s{(x)}_r.& (34)\end{array}$

Therefore, we have

$\begin{array}{cc}\Vert h{\Vert }_q\le \frac{2\eta }{{\rho }_{q,{\left(\frac{4}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)}+s^{1/q-1/r}{\sigma }_s{(x)}_r,& (35)\end{array}$

which completes the proof of (30).

Moreover, by using (33) and the inequality ${(v_1^r+v_2^r)}^{1/r}\le 2^{1/r-1}(v_1+v_2)$ for any v₁, v₂ ≥ 0, we obtain that

$\begin{array}{l}\Vert h{\Vert }_r\le {\left(\frac{1}{1-\alpha }\right)}^{1/r}{\left(2s^{1-r/q}\Vert h{\Vert }_q^r+2\Vert x_{S^c}{\Vert }_r^r\right)}^{1/r}\\ \le {\left(\frac{1}{1-\alpha }\right)}^{1/r}{2}^{2/r-1}\left(s^{1/r-1/q}\Vert h{\Vert }_q+\Vert x_{S^c}{\Vert }_r\right)\\ \le {\left(\frac{1}{1-\alpha }\right)}^{1/r}{2}^{2/r-1}\text{}\left(\text{}{s}^{1/r-1/q}\frac{2\eta }{{\rho }_{q,{\left(\frac{4}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)}+2{\sigma }_s{(x)}_r\text{}\right)\\ \le {\left(\frac{4}{1-\alpha }\right)}^{1/r}\frac{s^{1/r-1/q}\eta }{{\rho }_{q,{\left(\frac{4}{1-\alpha }\right)}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}}(A)}+{\left(\frac{4}{1-\alpha }\right)}^{1/r}{\sigma }_s{\left(x\right)}_r.& (36)\end{array}$

Hence, (31) holds and the proof is completed.

Remarks. When we select α = 0 and r = 1, our results reduce to the corresponding results for the ℓ₁-minimization or Basis Pursuit in Theorem 2 of Zhou and Yu [12]. In general, the sufficient condition provided here and that in Theorem 2 are slightly stronger than those established for the ℓ₁-minimization in Zhou and Yu [12], noticing that ${(\frac{2}{1-\alpha })}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}\ge 2^{\frac{q}{q-1}}s$ and ${(\frac{4}{1-\alpha })}^{\frac{q}{r(q-1)}}{s}^{\frac{q-r}{r(q-1)}}\ge 4^{\frac{q}{q-1}}s$ for any 1 < q ≤ ∞, 0 ≤ α ≤ 1, and 0 < r ≤ 1. This is caused by the fact that the technical inequalities used like (25) and (32) are far from tight. And this is also the case in the r-RIP based analysis. In fact, both r-RIP and q-ratio CMSV based conditions are loose. The discussion on much tighter sufficient conditions such as the NSP based conditions investigated in Tran and Webster [33], is left for future work.

3. Algorithm

In this section, we discuss the computational approach for the unconstrained version of (3), i.e.,

$\begin{array}{cc}\underset{x\in {ℝ}^N}{\min }\frac{1}{2}\Vert Ax-y{\Vert }_2^2+\lambda (\Vert x{\Vert }_r^r-\alpha \Vert x{\Vert }_1^r),& (37)\end{array}$

with λ > 0 being the regularizer parameter.

We integrate the iteratively reweighted least squares (IRLS) algorithm [21, 22] and the difference of convex functions algorithm (DCA) [34, 35] to solve this problem. In the outer loop, we use the IRLS to approximate the term $\parallel x{\parallel }_r^r$, and use an iteratively reweighted ℓ₁ norm to approximate $\parallel x{\parallel }_1^r$. Specifically, we begin with $x^0=\underset{x\in {ℝ}^N}{\arg \min }{\Vert y-Ax\Vert }_2^2$ and ε₀ = 1, for n = 0, 1, ⋯ ,

$\begin{array}{cc}{x}^{n+1}=\arg \underset{x\in {ℝ}^N}{\min }\frac{1}{2}\Vert Ax-y{\Vert }_2^2+\lambda \Vert W^{n}x{\Vert }_2^2-\alpha \lambda v^n\Vert x{\Vert }_1,& (38)\end{array}$

where $W^n=\text{diag}\left\{{({(x_i^n)}^2+{\epsilon }_n)}^{r/4-1/2}\right\}$ and $v^n=\parallel x^n{\parallel }_1^{r-1}$. We let ε_n+1 = ε_n/10 if the error $\parallel x^{n+1}-x^n{\parallel }_2<\sqrt{{\epsilon }_n}/100$. The algorithm is stopped when ${\epsilon }_{n+1}<10^{-8}$ for some n.

As for the inner loop used to solve (38), we view it as a minimization problem of a difference of two convex functions, that is, the objective function $F(x)=(\frac{1}{2}\parallel Ax-y{\parallel }_2^2+\lambda \parallel W^{n}x{\parallel }_2^2)-\alpha \lambda v^n\parallel x{\parallel }_1=:G(x)-H(x)$. We start with x^n+1,0 = 0. For k = 0, 1, 2, ⋯ , in the k + 1 step, by linearizing H(x) with the approximation H(x^n+1,k) + 〈y^n+1,k, x − x^n+1,k〉 where y^n+1,k ∈ ∂H(x^n+1,k), i.e. y^n+1,k is a subgradient of H(x) at x^n+1,k. Then we have

$\begin{array}{l}{x}^{n+1,k+1}=\underset{x\in {ℝ}^N}{\text{arg}\text{ min}}\frac{1}{2}\Vert Ax-y{\Vert }_2^2+\lambda \Vert W^{n}x{\Vert }_2^2\\ -\left(\alpha \lambda v^n\Vert x^{n+1,k}{\Vert }_1+\langle \alpha \lambda v^n\text{sign}(x^{n+1,k}),x-x^{n+1,k}\rangle \right)\\ =\underset{x\in {ℝ}^N}{\text{arg}\text{ min}}\frac{1}{2}\Vert Ax-y{\Vert }_2^2+\lambda \Vert W^{n}x{\Vert }_2^2-\langle \alpha \lambda v^n\text{sign}(x^{n+1,k}),x\rangle \\ ={\left(A^{T}A+2\lambda {(W^n)}^T{W}^n\right)}^{-1}[A^{T}y+\alpha \lambda v^n\text{sign}(x^{n+1,k})],\end{array}$

where sign(·) is the sign function. The termination criterion for the inner loop is set to be

$\frac{\Vert x^{n+1,k+1}-x^{n+1,k}{\Vert }_2}{\text{max}\{\Vert x^{n+1,k}{\Vert }_2,1\}}<\delta$

for some given parameter tolerance parameter δ > 0. Basically, this algorithm can be regarded as a generalized version of IRLS algorithm. Obviously, when α = 0, it exactly reduces to the traditional IRLS algorithm used for solving the ℓ_r-minimization problem.

4. Numerical Experiments

In this section, some numerical experiments on the proposed algorithm in section 3 are conducted to illustrate the performance of the weighted ℓ_r − ℓ₁ minimization in simulated sparse signal recovery.

4.1. Successful Recovery

First, we focus on the weighted ℓ_r − ℓ₁ minimization itself. In this set of experiments, the s-sparse signal x is of length N = 256, which is generated by choosing s entries uniformly at random, and then choosing the non-zero values from the standard normal distribution for these s entries. The underdetermined linear measurements y = Ax + e ∈ ℝ^m, where A ∈ ℝ^m×N is a standard Gaussian random matrix and the entries of the noise vector $\left\{e_i,i=1,2,\cdots ,m\right\}\stackrel{i.i.d.}{~}N(0,{\sigma }^2)$. Here we fix the number of measurements m = 64 and select a sequence of s as 10, 12, ⋯ , 36. We run the experiments for both noiseless and noisy cases. In all the experiments, we let the tolerance parameter δ = 10⁻³. And all the results are averaged over 100 repetitions.

In the noiseless case, i.e., σ = 0, we set λ = 10⁻⁶. In Figure 2, we show the results of successful recovery rate for different α (i.e., α = 0, 0.2, 0.5, 0.8, 1) while fixing r but varying the sparsity level s. We view it as a successful recovery if $\parallel \widehat{x}-x{\parallel }_2/\parallel x{\parallel }_2<10^{-3}$. We do the experiments for r = 0.3 and r = 0.7, respectively. As we can see, when r is fixed, the influence of the weight α is negligible, especially in the case that r is relatively small. But the performance does improve in some scenarios when a proper weight α is used. However, the problem of adaptively selecting the optimal α seems to be challenging and is left for future work. In addition, we present the reconstruction performances for different r (i.e., r = 0.01, 0.2, 0.5, 0.8, 1) while the weight α is fixed to be 0.2 and 0.8 in Figure 3. Note that small r is favored when the weight α is fixed. And a non-convex recovery with 0 < r < 1 performs much better than the convex case (r = 1).

FIGURE 2

Figure 2. Successful recovery rate for different α with r = 0.3 and r = 0.7 in the noise free case, while varying the sparsity level s.

FIGURE 3

Figure 3. Successful recovery rate for different r with α = 0.2 and α = 0.8 in the noise free case, while varying the sparsity level s.

Next, we consider the noisy case, that is σ = 0.01. We set λ = 10⁻⁴. And we evaluate the recovery performance by the signal to noise ratio (SNR), which is given by

$\text{SNR}=20{\log }_{10}\left(\frac{\Vert x{\Vert }_2}{\Vert \widehat{x}-x{\Vert }_2}\right).$

As shown in Figures 4, 5, the findings aforementioned can still be seen here.

FIGURE 4

Figure 4. SNR for different α with r = 0.3 and r = 0.7 in the noisy case, while varying the sparsity level s.

FIGURE 5

Figure 5. SNR for different r with α = 0.2 and α = 0.8 in the noisy case, while varying the sparsity level s.

4.2. Algorithm Comparisons

Second, we compare the weighted ℓ_r − ℓ₁ minimization with some well-known algorithms. The following state-of-the-art recovery algorithms are operated:

• ADMM-Lasso, see Boyd et al. [36].

• CoSaMP, see Needell and Tropp [37].

• Iterative Hard Thresholding (IHT), see Blumensath and Davies [38].

• ℓ₁ − ℓ₂ minimization, see Yin et al. [24].

The tuning parameters used for these algorithms are the same as those adopted in section 5.2 of Yin et al. [24]. Specifically, for ADMM-Lasso, we choose λ = 10⁻⁶, β = 1, ρ = 10⁻⁵, ε^abs = 10⁻⁷, ε^rel = 10⁻⁵, and the maximum number of iterations maxiter = 5,000. For CoSaMP, maxiter=50 and the tolerance is set to be 10⁻⁸. The tolerance for IHT is 10⁻¹². For ℓ₁ − ℓ₂ minimization, we choose the parameters as ε^abs = 10⁻⁷, ε^rel = 10⁻⁵, ε = 10⁻², MAXoit = 10, and MAXit = 500. For our weighted ℓ_r − ℓ₁ minimization, we choose λ = 10⁻⁶, r = 0.5 but with two different weights α = 0 (denoted as ℓ_0.5) and α = 1 (denoted as ℓ_0.5 − ℓ₁).

We only consider the exactly sparse signal recovery in the noiseless case, and conduct the experiments under the same settings as in section 4.1. We present the successful recovery rates for different reconstruction algorithms while varying the sparsity level s in Figure 6. It can be observed that both ℓ_0.5 and ℓ_0.5 − ℓ₁ outperform over other algorithms, although their own performances are almost the same.

FIGURE 6

Figure 6. Sparse signal recovery performance comparison via different algorithms for Gaussian random matrix.

5. Conclusion

In this paper, we studied a new non-convex recovery method, developed as minimizing a weighted difference of ℓ_r (0 < r ≤ 1) norm and ℓ₁ norm. We established the performance bounds for this problem based on both r-RIP and q-ratio CMSV. An algorithm was proposed to approximately solve the non-convex problem. Numerical experiments show that the proposed algorithm provides superior performance compared to the existing algorithms such as ADMM-Lasso, CoSaMP, IHT and ℓ₁ − ℓ₂ minimization.

Besides, there are some open problems left for future work. One is the convergence study of the proposed algorithm in section 3. Another one is the generalization of this 1-D non-convex version to 2-D non-convex total variation minimization as done in Lou et al. [39] and the exploration of its application to medical imaging. Moreover, analogous to the non-convex block-sparse compressive sensing studied in Wang et al. [40], the study of the following non-convex block-sparse recovery minimization problem:

$\begin{array}{ll}\underset{z\in {ℝ}^N}{\min }\Vert z{\Vert }_{2,r}^r-\alpha \Vert z{\Vert }_{2,1}^r\text{subject to}\Vert Az-y{\Vert }_2\le \eta ,\hfill & (39)\end{array}$

where $\parallel z{\parallel }_{2,r}={(\sum _{i=1}^p\parallel z\left[i\right]{\parallel }_2^r)}^{1/r}$ with z[i] denoting the i-th block of z, 0 ≤ α ≤ 1, and 0 < r ≤ 1, is also an interesting topic for further investigation.

Author Contributions

ZZ contributed to the initial idea and wrote the first draft. JY provided critical feedback and helped to revise the manuscript.

Funding

This work is supported by the Swedish Research Council grant (Reg.No. 340-2013-5342).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnote

1. ^ All figures can be reproduced from the code available at https://github.com/zzy583661/Weighted-l_r-l_1-minimization

References

1. Candes EJ, Tao T. Decoding by linear programming. IEEE Trans Inf Theory. (2005) 51:4203–15. doi: 10.1109/TIT.2005.858979

CrossRef Full Text | Google Scholar

2. Donoho DL. Compressed sensing. IEEE Trans Inf Theory. (2006) 52:1289–306. doi: 10.1109/TIT.2006.871582

CrossRef Full Text | Google Scholar

3. Eldar YC, Kutyniok G. Compressed Sensing: Theory and Applications. Cambridge: Cambridge University Press (2012).

Google Scholar

4. Foucart S, Rauhut H. A Mathematical Introduction to Compressive Sensing. Vol. 1. New York, NY: Birkhäuser (2013).

Google Scholar

5. Gribonval R, Nielsen M. Sparse representations in unions of bases. IEEE Trans Inf Theory. (2003) 49:3320–5. doi: 10.1109/TIT.2003.820031

CrossRef Full Text | Google Scholar

6. Tropp JA. Greed is good: algorithmic results for sparse approximation. IEEE Trans Inf Theory. (2004) 50:2231–42. doi: 10.1109/TIT.2004.834793

CrossRef Full Text | Google Scholar

7. Candes EJ. The restricted isometry property and its implications for compressed sensing. Comptes Rendus Math. (2008) 346:589–92. doi: 10.1016/j.crma.2008.03.014

CrossRef Full Text | Google Scholar

8. Candes EJ, Romberg JK, Tao T. Stable signal recovery from incomplete and inaccurate measurements. Commun Pure Appl Math. (2006) 59:1207–23. doi: 10.1002/cpa.20124

CrossRef Full Text | Google Scholar

9. Cohen A, Dahmen W, DeVore R. Compressed sensing and best k-term approximation. J Am Math Soc. (2009) 22:211–31. doi: 10.1090/S0894-0347-08-00610-3

CrossRef Full Text | Google Scholar

10. Dirksen S, Lecué G, Rauhut H. On the gap between restricted isometry properties and sparse recovery conditions. IEEE Trans Inf Theory. (2018) 64:5478–87. doi: 10.1109/TIT.2016.2570244

CrossRef Full Text | Google Scholar

11. Tang G, Nehorai A. Performance analysis of sparse recovery based on constrained minimal singular values. IEEE Trans Signal Process. (2011) 59:5734–45. doi: 10.1109/TSP.2011.2164913

CrossRef Full Text | Google Scholar

12. Zhou Z, Yu J. Sparse recovery based on q-ratio constrained minimal singular values. Signal Process. (2019) 155:247–58. doi: 10.1016/j.sigpro.2018.10.002

CrossRef Full Text | Google Scholar

13. Chen SS, Donoho DL, Saunders MA. Atomic decomposition by basis pursuit. SIAM J Sci Comput. (1998) 20:33–61.

14. Natarajan BK. Sparse approximate solutions to linear systems. SIAM J Sci Comput. (1995) 24:227–34. doi: 10.1137/S0097539792240406

CrossRef Full Text | Google Scholar

15. Chartrand R. Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Process Lett. (2007) 14:707–10. doi: 10.1109/LSP.2007.898300

CrossRef Full Text | Google Scholar

16. Foucart S, Lai MJ. Sparsest solutions of underdetermined linear systems via ℓ_q-minimization for 0 < q ≤ 1. Appl Comput Harmon Anal. (2009) 26:395–407. doi: 10.1016/j.acha.2008.09.001

CrossRef Full Text | Google Scholar

17. Li S, Lin J. Compressed Sensing with coherent tight frames via l_q-minimization for 0 < q ≤ 1. Inverse Probl Imaging. (2014) 8:761–77. doi: 10.3934/ipi.2014.8.761

CrossRef Full Text | Google Scholar

18. Lin J, Li S. Restricted q-isometry properties adapted to frames for nonconvex l_q-analysis. IEEE Trans Inf Theory. (2016) 62:4733–47. doi: 10.1109/TIT.2016.2573312

CrossRef Full Text | Google Scholar

19. Shen Y, Li S. Restricted p–isometry property and its application for nonconvex compressive sensing. Adv Comput Math. (2012) 37:441–52. doi: 10.1007/s10444-011-9219-y

CrossRef Full Text | Google Scholar

20. Xu Z, Chang X, Xu F, Zhang H. L_1/2 regularization: a thresholding representation theory and a fast solver. IEEE Trans Neural Netw Learn Syst. (2012) 23:1013–27. doi: 10.1109/TNNLS.2012.2197412

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Chartrand R, Yin W. Iteratively reweighted algorithms for compressive sensing. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2008. (2008). p. 3869–72.

Google Scholar

22. Lai MJ, Xu Y, Yin W. Improved iteratively reweighted least squares for unconstrained smoothed ℓ_q minimization. SIAM J Numer Anal. (2013) 51:927–57. doi: 10.1137/110840364

CrossRef Full Text | Google Scholar

23. Lou Y, Yin P, He Q, Xin J. Computing sparse representation in a highly coherent dictionary based on difference of L1 and L2. J Sci Comput. (2015) 64:178–96. doi: 10.1007/s10915-014-9930-1

CrossRef Full Text | Google Scholar

24. Yin P, Lou Y, He Q, Xin J. Minimization of ℓ₁₋₂ for compressed sensing. SIAM J Sci Comput. (2015) 37:A536–63. doi: 10.1137/140952363

CrossRef Full Text | Google Scholar

25. Yin P, Esser E, Xin J. Ratio and difference of l₁ and l₂ norms and sparse representation with coherent dictionaries. Commun Inf Syst. (2014) 14:87–109. doi: 10.4310/CIS.2014.v14.n2.a2

CrossRef Full Text | Google Scholar

26. Lou Y, Yan M. Fast L1–L2 minimization via a proximal operator. J Sci Comput. (2018) 74:767–85. doi: 10.1007/s10915-017-0463-2

CrossRef Full Text | Google Scholar

27. Wang Y. New Improved Penalty Methods for Sparse Reconstruction Based on Difference of Two Norms. Optimization Online, the Mathematical Optimization Society (2015). Available online at: http://www.optimization-online.org/DB_HTML/2015/03/4849.html

28. Wang D, Zhang Z. Generalized sparse recovery model and its neural dynamical optimization method for compressed sensing. Circ Syst Signal Process. (2017) 36:4326–53. doi: 10.1007/s00034-017-0532-7

CrossRef Full Text | Google Scholar

29. Zhao Y, He X, Huang T, Huang J. Smoothing inertial projection neural network for minimization L_p−q in sparse signal reconstruction. Neural Netw. (2018) 99:31–41. doi: 10.1016/j.neunet.2017.12.008

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Chartrand R, Staneva V. Restricted isometry properties and nonconvex compressive sensing. Inverse Probl. (2008) 24:035020. doi: 10.1088/0266-5611/24/3/035020

CrossRef Full Text | Google Scholar

31. Lopes ME. Unknown sparsity in compressed sensing: denoising and inference. IEEE Trans Inf Theory. (2016) 62:5145–66. doi: 10.1109/TIT.2016.2587772

CrossRef Full Text | Google Scholar

32. Zhou Z, Yu J. On q-ratio CMSV for sparse recovery. arXiv [Preprint]. arXiv:180512022. (2018). Available online at: https://arxiv.org/abs/1805.12022

Google Scholar

33. Tran H, Webster C. Unified sufficient conditions for uniform recovery of sparse signals via nonconvex minimizations. arXiv preprint arXiv:171007348. (2017).

Google Scholar

34. Tao PD, An LTH. Convex analysis approach to dc programming: theory, algorithms and applications. Acta Math Vietnam. (1997) 22:289–355.

Google Scholar

35. Tao PD, An LTH. A DC optimization algorithm for solving the trust-region subproblem. SIAM J Optim. (1998) 8:476–505. doi: 10.1137/S1052623494274313

CrossRef Full Text | Google Scholar

36. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn. (2011) 3:1–122. doi: 10.1561/2200000016

CrossRef Full Text | Google Scholar

37. Needell D, Tropp JA. CoSaMP: iterative signal recovery from incomplete and inaccurate samples. Appl Comput Harmon Anal. (2009) 26:301–21. doi: 10.1016/j.acha.2008.07.002

CrossRef Full Text | Google Scholar

38. Blumensath T, Davies ME. Iterative hard thresholding for compressed sensing. Appl Comput Harmon Anal. (2009) 27:265–74. doi: 10.1016/j.acha.2009.04.002

CrossRef Full Text | Google Scholar

39. Lou Y, Zeng T, Osher S, Xin J. A weighted difference of anisotropic and isotropic total variation model for image processing. SIAM J Imaging Sci. (2015) 8:1798–23. doi: 10.1137/14098435X

CrossRef Full Text | Google Scholar

40. Wang Y, Wang J, Xu Z. Restricted p-isometry properties of nonconvex block-sparse compressed sensing. Signal Process. (2014) 104:188–96. doi: 10.1016/j.sigpro.2014.03.040

CrossRef Full Text | Google Scholar