Testing accuracy of qubit rotations on a public quantum computer

,


Introduction
Recent progress in the operation of quantum devices offered by IBM enables many researchers to perform quantum computations in a realistic setup [1][2][3][4][5][6].The fragility of the operating devices, user-defined actions, and readouts deserve constant diagnostic checks.The paradigm for operating these systems relies on the quantum description of a few-level Hilbert space and the unitary evolution controlled by a programmed sequence of gates.The devices and operations are not perfect in reality: the deviations come from the decoherence, environment noise, inaccuracy of the gate parameters, and presence of additional states.To diagnose the realistic implementation of the ideal model, one can perform various control tests, including gate set tomography [7,8], where the outcome statistics reveal the nature of such deviations, their possible sources, and hints for countermeasures [9].
In this paper, we propose to perform a simple experiment as a diagnostic test of the reliability of the rotation of single-qubit quantum gates.Quantum computers usually offer a limited set of basic operations (gate), but they can be controlled by a continuous parameter, the rotation angle.It means that the interlevel transition amplitudes are multiplied by e ±iθ , with θ being a control angle.In short, the test compares the outcome probability of the qubit in a specific state with the standard cos 2 θ, with θ being the phase shift of a given gate.Deviations from the fit to the combination of 1, cos, and sin beyond the statistical error mean corrupt rotation.Taking a list of angles, shuffled randomly, and repeating the test sufficiently many times, one can reveal potential deviations.There exist other testing approaches based on the state dimension [10].The specifics of our test are minimalistic circuit complexity, a single control parameter (angle θ), and robustness to many sources of noise, which makes it a convenient diagnostic tool to be run whenever the accuracy of the gate is critical.The test is also linear, being robust against drifts and calibration changes as long as the relaxation and decoherence are phase-independent.Our test goes beyond the standard randomized benchmarking [11], as we systematically monitor the dependence of the deviations on a control parameter.
The public quantum computer by IBM offers the possibility to perform such a test with sufficiently large statistics as the rotation is performed virtually by shifting the phase of the gate [12].We were able to run the experiment on several different devices, including a singlequbit one.The statistics we collected were sufficiently large to make confident conclusions.We found deviations at the relative level 10 -3 and far beyond 5 standard deviations.Our observations show that the deviations are not accidental, and corrections to the ideal model are necessary to explain them.We also tested the nonlinearities of the waveform generators [13] as the possible cause, and they only partially explain the data.The remaining discrepancies are still beyond 5 standard deviations, and their cause is still unknown.We encourage readers to investigate possible causes.There may be subtle technical reasons, but extraordinary models such as involving a larger Hilbert space [14] or more exotic concepts such as interacting many copies [15,16] should also be considered.We perform additional benchmarking tests to show that the deviations are independent of the inevitable decoherence caused by subsequent gates.
This paper is organized as follows.We start by describing the test of the π/2 rotation on a qubit, repeated n times, then explain implementation on IBM cloud computing, and next, discuss the obtained results and their significance, including the analysis of n = 1, 5 and n = 1, 5, 9 cases.We discuss the benchmark tests and conclude with the general summary.We present the calculation of model-based deviations in Supplementary Appendixes.

Test of a rotated gate on a qubit
We consider the two-level Hilbert space of a qubit, i.e., spun by states, |0〉, |1〉.We assume a θ-dependent quantum operation (gate) of a general form (Eq. 1): with the angle-dependent rotation (phase shift) around the z-axis on the Bloch sphere [17] (Eq.2), written in the basis |0〉, |1〉.It is motivated by the implementation of qubit gates, e.g., in the IBM quantum computer, as explained in the following.The operation S θ can be applied n times, resulting in the total operation S n θ S acting on the initial ground state |0〉.The sequence of operations is depicted schematically in Figure 1, with the details of pulse shapes, including the effect of the phase shift, shown in Figure 2.
We perform a dichotomic diagonal measurement M = α|0〉〈0| + β|1〉〈1| on the prepared state ρ, which gives the mutually exclusive outcomes 1, 0. A general form for the probability of 1, given some initial ρ and some given S, is (Eq.3): with some constants A n , B n , C n , which are independent of θ.This is the simplest possible test of the phase shift to run on a generic quantum computer.Due to linearity, even changes in the gate S (e.g., due to a different calibration) do not affect the test's validity.Only the A, B, and C constants will change.
Of course, this prediction will no longer be valid if (a) the actual Hilbert state is larger, with, e.g., an additional state |2〉, (b) M, S, or ρ depend on θ, and (c) M is not diagonal.
Only such effects can lead to deviations from (3).For (c), only a second harmonics occurs.We discuss these possibilities in detail in Section 3, focusing on potential perturbative corrections.

π/2 rotation gate
The operations we use conform to native gates [18] provided by IBM quantum cloud computing, realized by microwave pulses.The perfect operation corresponding to our particular choice is given by M = |1〉〈1|, ρ = S|0〉〈0|S † , and S ≡ RX π 2 ≡ X √ .We intend to use this native gate as the simplest choice, but the actual gate can be different, over/under-rotated, or even a completely different one.It Quantum circuit used to test the rotation of the gate S. The bare gate S acts on the initial state |0〉, giving the initial state ρ = S|0〉〈0|S † in Eq. 3, and the sequence of identical gates S θ , i.e., S rotated by θ, is applied before measurement.should be noted that S is just a π/2 rotation around the x-axis on the Bloch sphere [17] (Eq.4).
An ideal π/2 rotation has eigenvalues ± i and 1, and the probability reads as follows (Eq.5): We test (A) the fit from (3) against the measured outcome of the preparation sequence for a specific n and (B) if p 1 − p 5 = 0 and p 1 − 2p 5 + p 9 = 0 for an arbitrary θ.The rotation Z θ is virtual and is performed together with S [12], resulting in a single operation S θ .The actual effect of the Z gate is the rotation of in-phase and out-of-phase pulse components, not the gate pulse itself.Z gates (phase shifts) are a standard method to parameterize the native gates in the easiest possible way.The drive (interlevel) frequency is, in practice, much higher than the inverse time scale of the gate pulse.Therefore, the application of the Z gate reduces to a phase shift of the original gate pulse with the driving microwave field.It remains independent of the shape of the pulse as long as the shift is correct.It should be noted that an ideal gate S gives A = 0, but we do not have to assume it.On the contrary, keeping all three fit parameters, our test is robust against global phase shifts and decoherence.The great advantage these tests is the usage of only a single qubit, partial independence of unknown properties of the environment and quantum operations, and universality.It applies to any two-level system.In practical implementation, it is helpful to eliminate memory effects of its repetitions by picking a random θ from a range uniformly covering any interval of length 2π.The result of the test should not change if adding a definite number of operations, S, at the end.It should be noted that in the ideal case, an even number of Ss after Z θ gives the probability 1/2, while for an odd case, the probabilities of 0 and 1 get swapped every two Ss.

Implementation on the quantum computer
The IBM Quantum Platform cloud computing offers several devices, working as a collection of qubits, which can be manipulated by gates from a limited set-either single-qubit or two-qubit operations.The rotation Z is not a real but an instantaneous virtual gate VZ(θ) added to the next gate [12].In particular, the sequence of gates Z θ and S, shown in Figure 1, is realized by the native S θ .
The names of the devices we tested are Armonk, Lagos, Lima, Jakarta, Perth, Nairobi, and Bogota.The collected statistics depend on the number of jobs, circuits, and shots, see Supplementary Appendix SA1.The linearity of the fit (3) makes the test robust against calibrations.Probabilities from different jobs can be simply averages, and so do the fitting parameters.The fitting has been performed by least squares.We compared the outcome statistics, the measured rate of occurred value 1, with the fit to (3).We also ran the circuits consisting of two and three gates S θ instead of one to compare the possible deviations.For a benchmark, to estimate an error per gate, we used up to 70 S θ .We made our scripts and collected data publicly available [19].

General results
We present the results in Figures 3, 4. The standard deviation is calculated for the independent Bernoulli statistics, i.e., p(1 − p)/N , where N is the number of repetitions (jobs, times, shots, times, and circuits per angle).In all runs, the prediction (3) is verified down to level 10 -3 of relative error.However, the deviations of order 10 -3 are significant when compared to the predicted error (more than 5 standard deviations).The deviation is smallest on Armonk-the singlequbit device-but still significant.It should be noted that the execution time of S or S θ gates is 35.5 ns except on Armonk, where it is 71.1 ns.As similar results, exhibiting systematic θdependent deviations, have been obtained on different devices at different times (the data collection took 1 year, 2021-2022, while each run took from a few days to a few weeks), they deserve some physical explanation.The results from Bogota are consistent but indicate a noticeable phase shift; we discuss them separately in Supplementary Appendix SA2.The natural reason is that the actual performance of the gates can differ from the ideal expectations.For instance, a nonlinearity of the waveform generator can modify the pulse in a θ-dependent way [13].

Analysis of deviations
We analyze a possible explanation of the deviations by non-ideal execution of the gates under the following four assumptions: 1.The two-dimensional Hilbert space of the quantum system 2. The identical subsequent S θ gates 3. Decoherence independent of the θ-parameter 4. The small deviations, i.e., dominating the first-order correction The standard realization of the gate S, including generic deviations in the actual pulse, can be described as (Eq.6): for some complex ϵ(θ, ϕ) = ϵ r + iϵ i describing the parameterdependent imperfection of the gate, including over/underrotations and other coherent errors, where T denotes the chronological product in the Taylor expansion of the exponential of the integral with respect to ϕ corresponding to dimensionless gate operation time.The gate is ideal for ϵ = 0. We consider only offdiagonal corrections because the pulse only modulates the driving frequency of the transition between levels.We find the first-order correction to the probability (3) for n gates S θ in the form (see Supplementary Appendix SA3) At first sight, the model above could, in principle, explain the deviations because most of the deviations cross 0 at θ = 0, π in Figures 3, 4 (taking into account a general shift of the angle on Bogota) and the deviations for a single and three S θ sum approximately to zero if the symmetry ϵ(ϕ) = ϵ(π/2 − ϕ) is assumed.Nevertheless, this must be confirmed by tracing down to the actual pulse formation.The results for S 2 θ presented in Figure 4 are not fully compatible with this model at θ = +π/2, but it may be the result of the assumption of identical subsequent gates which may not be fully realized.Results of the tests on IBM quantum devices for n = 1 with 8,192 shots per job with 56 circuits per angle per job and 100 jobs, except Armonk with 4 circuits per angle and 1,556 jobs.The probability of registering 1 is fitted by least squares to (3) in the upper figure, while the lower figure presents the deviation from the fit.On the vertical axis, p denotes the probability of 1, while p fit is the fit of p to (3).

Additional tests
To fully test the model in Eq. 7, we re-run the tests to compare the results from 1 to 5 gates S θ , which should give identical deviations, according to the model, i.e., p 1 − p 5 = 0 in the first order.We performed the tests on Lagos (qubits 0 and 6), Lima (qubit 0), and Perth (qubit 0), running with shuffled angles to avoid memory effects.The results show extraordinary deviations for Lagos qubit 0 in the first run in February 2022 (however, we found such large deviations already in November 2021 in the benchmark test), but repeating the test in May 2022 gave much smaller deviations, see Figure 5.The deviation from the first run is large, ~10 −2 , and gets inverted between 1 and 5 gates.We additionally checked that the inversion took place in each of the four gates in the benchmark test.Here, the reason must have been completely different, e.g., a considerable technical or fundamental problem.Such a large deviation can be explained by an enlarged Hilbert space, including the multiplication of quantum states, in analogy to the many-copies idea [15,16], but this needs further analysis to confirm or rule out.Smaller deviations in the second run indicate that they may depend on calibration, which is applied to the qubit daily, although they stayed on the same level for 6 months, ruling out the effect of an incidental calibration.The qubit 6 from Lagos and qubits 0 from Lima and Perth also give much smaller deviations, Figures 6, 7, although still they do not match for 1 and 5 gates, small but still nonzero (beyond 5 standard deviations) differences.
Standard models of decoherence do not depend on gate parameters (θ in our work).Benchmarking the decoherence rate at ~10 −3 , comparable to the observed deviations, eliminates it as the main reason, see Supplementary Appendix SA4.The relaxation and decoherence times are of the order of at least ~10μs [20], which is far beyond the gate time of 35 ns.In Supplementary Appendix SA5, we present an analytical argument for such a model, including readout error, relaxation, depolarization, and phase damping [17], to show that p 5 ≃ p 1 still holds up to firstorder corrections.The magnitude of the statistical errors compares favorably with the real device error estimates, such as in Figure 6.Finally, using sequence or up to 9 gates S θ , Figure 1, we checked if p 1 − 2p 5 + p 9 = 0 on Nairobi (January-February 2023), which Comparison between 1 and 5 gates, p 1−5 = p 1 − p 5 with the fit to (3) subtracted in δp, for Lagos qubit 0, using 32,000 shots each and 9 repetitions for each case angle for 43 jobs in the first run in February and 80 jobs in the second run in May 2022; each data point is a result of 43 • 32,000 • 9 or 80 • 32,000 • 9 runs.The very large reversal deviation in the first run, reversed between 1 and 5, should be noted.Comparison between n = 1 and n = 5, p 1 − p 5 , for Lagos qubit 6 with 195 jobs with 32,000 shots each and 9 repetitions for each case angle, (top), Lima qubit 0 with 290 jobs, 20,000 shots, and 3 repetitions (middle), and Perth qubit 0 with 53 jobs, 100,000 shots each, and 9 repetitions (bottom).should hold up to second-order deviations, see Supplementary Appendix SA5.The result is beyond 4 standard deviations, see Figure 8.We conclude that the real device deviation from a test value, which we observe, goes beyond the standard models of noise and the known amount of leakage [21].

Frontiers in
A higher state [14] as an alternative explanation seems unlikely due to different transition frequencies and the fact that it is a second-order correction (see the analysis in Supplementary Appendix SA6).A simple in-phase/quadrature (I/Q) imbalance [12] cannot explain the dominating second harmonic in the deviations, as it would give only third harmonics (see Supplementary Appendix SA7).An even more complicated description, like a considerable extension of the Hilbert space [15,16], would be the last option.

Conclusion
We observed the deviation from the ideal π/2 rotation on several devices available at the IBM Quantum Platform.The deviations are significant, systematic, and with the amplitude exceeding 5 standard deviations.They exist in a similar form on different devices, different qubits within a device, tested over a long period.The deviations are smaller but persistent for a singlequbit Armonk.Our test is robust against non-idealities of the gates as long as the relaxation and decoherence are phase-independent as they do not affect the difference p 1 − p 5 in the first order.The benchmark test rules out the possibility of accumulation of decoherence error by many identical gates.Also, the angledependent contribution from higher states should remain negligible in the lowest order.The most likely solution, the imperfection of I/Q-mixers and waveform generators, close to sin 2θ (except Bogota), fails to reproduce equal deviations in the case of 1 and 5 gates S θ , so it is, at best, insufficient.The additional test of 1, 5, and 9 gates resulted in a deviation beyond 4 standard deviations, which deserves confirmation in a larger statistics collected within a relatively short time (in our case, below 2 months).Nevertheless, the systematic occurrence of the deviation should serve as a diagnostic test to enhance the calibration of the gates and find the correct description of the qubit.Due to our assumptions on the identical pulses and restricted Hilbert space, we cannot claim the deviations to be a signature of fundamental problems with the description of the tested qubits.However, we believe that the robustness of such tests will motivate further exploration of qubits diagnostics, an alternative to gate set tomography [7,8].
We used standard gates available at the IBM Quantum Platform, but one can continue the tests using OpenPulse API [22], which allows fine-tuning the gates or using more complicated tests to reveal the relevant dimensionality of transmon Hilbert space.Dimension tests based on equality can be used to determine the dimensionality [23].One can also similarly test two-qubit gates with two independent angles for each qubit.In any case, we believe that further improvement of quantum computers from IBM or other public providers will allow even more stringent tests of quantum predictions in the case of low-level systems.

FIGURE
Results for n = 1 and n = 5, as in Figure 5, after removing the fit to (3).Comparison between n = 1, n = 5, and n = 9, p 1 − 2p 5 + p 9 , for Nairobi qubit 0, with 100 jobs with 100,000 shots each and 6 repetitions.The actual probabilities almost overlap each other, but the combination gives a deviation over 4 standard deviations.

FIGURE 2
FIGURE 2Actual waveform of the pulse on the IBM quantum computer (Perth), first S and then n = 5 gates S θ for θ = 7π/8 ≈ 2.75.The discretization unit time is dt = 0.222 ns.Driving (level gap) frequency is denoted by D0.The light/dark shading corresponds to the inphase/out-of-phase amplitude component, respectively.The element VZ(2.75) is a zero-duration virtual gate, a part of native S θ .

FIGURE 4
FIGURE 4Results of tests as in Figure2but with n = 2, 3.It should be noted that the ideal angle dependence for n = 3 is reversed with respect to Figure2, as the two S θ swap the states |0〉 and |1〉.