A New Calibration Method of Stereo Line-Scan Cameras for 2D Coordinate Measurement

Camera calibration plays an important role in various applications including machine vision and optical measurement. In this paper, we propose a new calibration method of stereo line-scan cameras for 2D coordinate measurement using a simple calibration target rather than requiring complex calibration patterns or an auxiliary area-scan camera. Considering the viewing planes associated with the stereo line-scan cameras are coplanar, an imaging model combining perspective projection and lens distortion are established and can depict the relationship between the 2D world coordinate and its corresponding 1D pixel coordinates of the image pairs. A two-step calibration algorithm is proposed to obtain the optimal values of intrinsic, extrinsic and distortion parameters of stereo line-scan cameras. The performance of the proposed camera calibration method is evaluated on 2D coordinate measurement and the experimental results demonstrate that the proposed method is more effective and accurate than the existing method.


INTRODUCTION
Line-scan (also called linear or 1D) cameras are becoming widely used to various applications including machine vision and optical measurement, such as agricultural robot navigation [1], railway catenary inspection [2], surface defect detection [3], 2D coordinate measurement [4] and 3D shape measurement [5][6][7]. There are two types of line-scan cameras: single-line camera and multiline camera [8], which can be considered as a particular area-scan (also called matrix or 2D) camera consisting of a single row or multiple rows of pixel elements [9]. Compared with area-scan cameras, line-scan cameras can provide higher spatial resolution, higher imaging rates (up to 16384 pixels at 300 kHz) and lower manufacturing cost [9,10].
In the camera-based applications, camera calibration plays an important role and is a necessary step in order to extract metric information from 2D images [8,11]. The key of calibration is to find out the initial values of intrinsic and extrinsic camera parameters, which demonstrate the mapping relationship between the 2D coordinates in the image and the 3D coordinates in the object world [12] and will be refined by a non-linear optimization technique to obtain more accurate calibration parameters. Conventional calibration methods for area-scan cameras [11,13] are not applicable to the line-scan cameras because of their different imaging principles [8]. For line-scan cameras, the calibration process is more complicated and several calibration methods have been presented in the last decades. The authors have performed the investigation and reviewed on line-scan camera calibration methods [9]. Generally, these methods can be classified into two groups: static imaging calibration and dynamic imaging calibration [8,14].
Static imaging calibration employs the patterns on the calibration target comprising several feature lines. The 3D coordinates of intersection points that can be found by solving the viewing plane equation of the line-scan camera and the feature lines equations are applied for calibration. It is worth noting that each row data of the captured image is almost the same because there is no relative motion between the camera and the calibration target. Horaud et al. [15] proposed a multiline calibration method where cross-ratio invariance is used to compute the intersection points. However, it is necessary to move the calibration target along the Y w and/or Z w axises with known increments. In order to relax these constraints, Luna et al. [16] presented a novel calibration method using a 3D pattern consisting of two parallel planes with the lines like in [15]. Lilienblum et al. [17] designed a more complex 3D pattern with vertical and diagonal straight lines taken together in several coded targets on two levels for camera calibration. Li et al. [18] gave a alterative 3D pattern comprising two orthogonal planes where there are two groups of parallel feature lines; this pattern was introduced by Su et al. [1] to calibrate the hyperspectral linescan camera in agricultural robots. Song et al. [19] designed a new calibration target with a coded eight trigrams pattern which has a circular symmetry structure to get more effective coded information from different directions. Niu et al. [20] presented another 3D right-angle stereo target based on concentric rings. Liao et al. [14] designed a stereo target with hollow stripes to solve the eccentricity error problem. These methods [1,14,[16][17][18][19][20] do not require the precise target translation but need high-precision 3D calibration patterns. Instead of using the 3D patterns, Yao et al. [21] proposed an innovative calibration method using a stationary planar pattern which consists of repeated vertical and slanted lines and constructed a 2D calibration framework by projecting original 1D data onto its orthogonal direction. Li et al. [10] presented a flexible calibration technique where the line-scan camera is rigidly coupled to a calibrated area-scan camera to establish a pair of stereo cameras. The stereo cameras are required to observe a specially designed planar pattern shown at a few different orientations; thus the 3D feature points for line-scan camera calibration can be obtained by the extrinsic parameters of area-scan camera. Similarly, Sun et al. [22] also used a customized planar pattern and a precalibrated area-scan camera to get the 3D coordinates of the feature points. The area-scan cameras in [10,22] remove the necessity of precise pattern movement; however, it greatly increases the cost of the calibration set-up.
Dynamic imaging calibration estimates the line-scan camera parameters by a series of scanning operations. More generally, a linear relative motion with constant velocity between the camera and the calibration target is typically considered. Draréni et al. [23] utilized a controllable linear stage to translate the line-scan camera along the Y c axis, when the camera is watching a planar checkerboard pattern; hence a 2D scan image is captured. If the planar pattern is almost parallel to the image plane, however, this method can not work since it entails dividing by elements of rotation matrices. In order to avoid this problem, Donné et al. [24] improved the method [23] by dividing the homography H by its end element. Similar to [23], Hui et al. [25,26] also fixed the line-scan camera to a programmable linear stage. The camera scans a 3D calibration pattern along an arbitrary direction whose three motion parameters are added to the dynamic imaging model and does not require strict motion along the Y c axis like in [23]. Furthermore, Hui et al. [27] used an auxiliary area-scan camera and a 2D checkerboard pattern to calibrate the line-scan camera instead of using the linear stage and 3D pattern. Steger and Ulrich [28] proposed a versatile camera model for line-scan cameras with telecentric lenses using a planar calibration target. The model takes into account the telecentric lens distortions and supports arbitrary positions of the linear sensor with respect to the optical axis.
As for the stereo line-scan cameras, 2D coordinates or 3D shape can be calculated by the well-known triangulation method when they are properly calibrated. A few stereo line-scan camera systems for industrial applications have been reported in the literature. Ma et al. [4] presented a 2D coordinate measurement system based on stereo line-scan cameras which are calibrated by using several rods perpendicular to a flat plate at some special locations. The calibration is easy to implement due to its simple target, but its major defects are quite obvious: time consuming and low accuracy caused by occlusion imaging. Sun et al. [5] developed a high-speed 3D shape measurement system for moving objects. The stereo line-scan cameras can be calibrated by their static imaging calibration method [22] or [6]. Lilienblum et al. [29] extended the calibration method [17] for 3D measurement system consisting of stereo line-scan cameras. Zhan et al. [2] proposed an accurate and efficient measurement approach for railway catenary geometry parameters by using stereo line-scan cameras. Liao et al. [7] gave a dense 3D point cloud measurement system comprising dual line-scan cameras and a matching strategy to expand the depth measurement range. Most recently, Steger and Ulrich [30] introduced their line-scan camera model [28] to the multi-view case with an arbitrary number of cameras for 3D surface reconstruction.
In this paper, based on the work of Ma et al. [4] and our previous work [9], we propose a new calibration method of stereo line-scan cameras for 2D coordinate measurement. The rest of this paper is organized as follows. In section 2, we establish an imaging model of stereo line-scan cameras that serves as the basis for our method. In section 3, the proposed calibration method for intrinsic, extrinsic and distortion parameters is described in detail. Experimental results on 2D coordinate measurement are performed and discussed in section 4. Finally, we conclude our method in section 5.
Our main contributions are summarized as follows: 1) We propose a static imaging calibration method of stereo linescan cameras using a simple calibration target. Complex calibration patterns/targets are not required to calibrate cameras; auxiliary imaging devices, such as area-scan camera, do not need to aid the calibration. 2) We establish an imaging model for stereo line-scan cameras that are coplanar aligned. Lens distortion is taken into consideration to improve the model performance. 3) We design a calibration target with simple structure and low cost. The target comprises a group of magnetic rods perpendicular to a flat iron plate with a coordinate map which can give the world coordinates. 4) The time required for the calibration process is greatly reduced with the help of the simple target, translation and a two-step algorithm that determines the camera parameters using the images of a group of rods.

IMAGING MODEL OF STEREO LINE-SCAN CAMERAS Stereo Line-Scan Camera System
The stereo line-scan camera system has various arrangement modes [7]. For our system, the side-by-side mode is chosen. As depicted in Figure 1A, the system mainly includes two line-scan cameras (Dalsa, spL2048-140km) mounted lens (Nikon, AF Nikkor 14 mm f/2.8D), two line lasers, a industrial computer with two frame grabbers (Dalsa, OR-X4C0-XPF00), several mechanical and calibration devices. Like most stereo line-scan camera systems [2,[4][5][6][7]29], our system also requires the two linescan cameras to be coplanar aligned manually using a suitable mechanical device so that the two viewing planes determined by the projection center and the pixel elements are approximately coplanar. Consequently, the measurement field is the overlapping area of the two planes.

Imaging Model
It is necessary for an imaging model to define the coordinate systems in order to describe the intrinsic and extrinsic parameters. As shown in Figure 1B, the world coordinate system, the two camera coordinate systems and the two pixel coordinate systems are denoted by [R, T] are extrinsic parameters between the camera 1 and 2 coordinate systems, and [R 1 , T 1 ] and [R 2 , T 2 ] are extrinsic parameters between the world coordinate system and the camera coordinate system, where R, R 1 , R 2 and T, T 1 , T 2 are the rotation and translation matrixes, respectively. For the line-scan camera, perspective projection is only satisfied along the pixel elements direction, i.e., u 1 and u 2 . According to perspective projection [2,11], the relationship between a point P ck (k 1, 2) on the viewing plane and its image points p k (u k ) can be described as where α k and u k0 are the intrinsic parameters and denote the focal length in pixel units and the coordinate of the principle point of the camera k, respectively; (X ck , Z ck ) are the coordinates of P ck reference to the camera k coordinate systems, respectively. Because a world point P w only appears on the viewing planes can be imaged by the stereo line-scan cameras, for convenience, the O w − X w Y w is required to coincide with the viewing planes. Therefore, (X w , Y w ) denote the 2D coordinates in the measurement field. Furthermore, the world coordinate system and the camera k coordinate system can be related in the following ways where θ k is a rotation angle and [t kx t kz ] T is a translation vector. Taking Eq. 2 into Eq. 1, we can get −α 1 sin θ 1 + u 10 cos θ 1 α 1 cos θ 1 + u 10 sin θ 1 α 1 t 1x + u 10 t 1z By substituting the second equation into the first equation of (3) and substituting the fourth equation into the third equation of (3), Eq. 3 can be transformed into the following descriptions Obviously, the imaging model (3) or (4) can depict the relationship between the 2D world coordinate (X w , Y w ) and its corresponding 1D pixel coordinates (u 1 and u 2 ) of the image pairs.

Lens Distortion Model
It is easy to see that the imaging model (3) or (4) is linear. However, the actual imaging is a nonlinear process inevitably affected by different kinds of lens distortions [2]. To improve the model performance and measurement accuracy, lens distortion is taken into consideration.
Considering the specialty of line-scan camera [32],ỹ ≈ 0 and thus lens distortion model (5) can be simplified as If we ignore the third radial distortion and set k 0 s 1 + 3p 2 , Eq. 6 can be furthermore simplified tõ Considering the lens distortion, Eq. 1 can be rewritten as Substituting Eq. 8 into Eq. 4, we can derive the real distorted imaging model

CALIBRATION METHOD Calibration Target
As shown in Figure 2A, our designed calibration target comprises a group of magnetic rods (8 collinear feature points with 100 mm spacing) perpendicular to a flat iron plate. The plate is adjusted to be parallel to the viewing plane and pasted by a coordinate map which can give the 2D world coordinates (X w , Y w ) when the target translates. It can be seen from Figure 2A that the calibration target has a simple structure and auxiliary devices, such as linear stage and area-scan camera, do not need to assist the calibration. Therefore our target is low cost.

Computing the Initial Values
A two-step algorithm is proposed to determine the optimal values of intrinsic, extrinsic and distortion parameters. Firstly, we will compute the initial values of intrinsic and extrinsic camera parameters by solving the imaging model (4) with given feature points. Secondly, the initial values will be refined by a non-linear optimization technique to greatly improve the calibration accuracy. Given n(n ≥ 2) translations of the calibration target, the coefficients of H 1 [h′ 11 , h′ 12 , h′ 13 can be obtained by a sub-pixel location algorithm that can detect the centers of 8 rods in n captured images.
The matrix W k can be decomposed by using singular value decomposition (SVD Solving this equation, the intrinsic and extrinsic parameters can be obtained Obviously, according to the coordinate systems defined in Figure 1B, we can get t kz > 0.

Non-linear Optimization
In the initial calculation of the parameters, the distortion is neglected. In order to obtain more accurate calibration parameters, a non-linear optimization technique such as Levenberg-Marquardt algorithm is employed to refine the camera parameters. The goal is to find out the optimal intrinsic, extrinsic and distortion parameters which minimize the sum of squares of reprojection errors, i.e., min α k ,u k0 ,R k ,T k ,k k0 ,k k1 ,k k2 where u′ j k (X j w , Y j w ) is a function which projects the world points (X j w , Y j w ) into the image points according to the real distorted imaging model (9). For the initial values of distortion parameters, we may take k k0 k k1 k k2 0 since the lens distortion is usually small and close to zero.
Once the parameters of two line-scan cameras are respectively refined, we can furthermore minimize the sum of squares of reprojection errors for each camera, i.e., min α k ,u k0 ,R k ,T k ,R,T,k k0 ,k k1 ,k k2 to obtain the final accurate parameters of stereo line-scan cameras, where the initial values of R, T can be calculated by R

Calibration Experiment
To evaluate the effectiveness of the proposed method, we calibrate our stereo line-scan camera system. It can be observed from Figures 1A and 2A, when the calibration target is translated on the flat plate, the coordinate map can give the 2D world coordinates (X w , Y w ) for camera calibration. In our experiment, n 7, that is to say we translate target seven times, a total of 56(8 × 7) feature points. Figure 2B exhibits two of seven image pairs of 8 magnetic rods illuminated by the line lasers and the sub-pixel locations of rod centers. The first and third images are captured by camera 1 while the second and fourth images are captured by camera 2. We set the line-scan cameras to 300 rows per frame, so the image resolution is 2048 × 300. Table 1 shows the calibration results of stereo line-scan cameras. We implement our method in Matlab. The root mean square (RMS) of reprojection errors of two camera is less than 0.40 pixel. It is well mentioning that the time of our calibration process is no more than 5 min for both cameras while the time of the method [4] for each camera is about 30 min. The time required for our calibration process is greatly reduced mainly due to the help of the simple target and simple translation operation. Therefore it can be said that our proposed method is significantly effective.

2D Coordinate Measurement
Furthermore, our calibration parameters are verified by measuring the 2D coordinates in the measurement field with a range of 1000 mm × 800 mm.
A comparison between our and [4]'s measurement results is shown in Table 2. We test a total of 21 2D points similar to [4]. According to the results, it can be seen obviously that the maximum errors of our method for X w and Y w are 0.84 and 0.67 mm respectively, while the maximum errors of [4]'s method for X w and Y w are 1.41 and 1.07 mm, respectively. The RMS errors of X w and Y w of our method are 0.449 and 0.373 mm respectively, which are also less than [4]'s method. Generally, these errors can be more or less affected by the accuracy of coordinate map, the planeness of iron plate, the precision of calibration target, the parallelism of two viewing planes and iron plate, and the sub-pixel location algorithm.
In conclusion, the experimental results demonstrate that our proposed method can provide satisfactory 2D coordinate measurement and has much more superiority both in the calibration time and accuracy compared with the [4]'s method.

CONCLUSION
We have proposed a calibration method for the stereo linescan camera system. We firstly established the real distorted imaging model by combining a perspective projection and three types of lens distortions. On the basis of the model which can depict the relationship between the 2D world coordinate and its corresponding 1D pixel coordinates of the image pairs, we have proposed a two-step algorithm to find out the optimal parameters of stereo line-scan cameras. The initial values of intrinsic and extrinsic parameters were computed by solving the system of homogeneous equations derived by the image model and feature points. We employed the Levenberg-Marquardt optimization algorithm to refine the intrinsic, extrinsic and distortion parameters by minimizing the reprojection errors. Finally, the experimental results verified that our proposed method can provide satisfactory 2D coordinate measurement with less calibration time and a higher accuracy. Further study of line-scan camera calibration includes the flexible calibration target and noncoplanar alignment.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.