To solve the problem of the limited public NER dataset for CANER, a new kiwifruit-related annotated corpus, named KIWID, was collected and annotated under the guidance of plant protection experts from Northwest A&F University.

New word detection can identify OOV words and add them to the built-in dictionary of the word segmentation tool, thus improving the effect of common word segmentation tools (Du et al., 2016). Currently, new word detection is either rule-based (Huiming et al., 2003), statistics-based (Jin and Tanaka-Ishii, 2006), or based on both rules and statistics (Zheng and Wen-Hua, 2002). Methods that rely entirely or partly on rules rely on a manually built rule base. Although the rule base is helpful in improving the effectiveness of new word detection, the construction process is complex and time-consuming, and domain transferability is poor. As a result, this study adopts a statistics-based new word detection method. Corpus A was first segmented into strings using the N-gram method, and the garbage strings were then filtered in turn according to the three statistics of word frequency (WF), mutual information (MI), and contextual entropy (CE) of the strings. Subsequently, a new word set was obtained. This new word set was then added to the built-in dictionary of Jieba to improve its applicability to kiwifruit-related texts. Finally, the kiwifruit lexicon was constructed through the word segmentation of Corpus B by Jieba. This section first introduces the methods related to new word detection, and then introduces the lexicon construction process.

For a character-based CNER model, discrete text sequences are converted into low-dimensional densely distributed embedded representations, allowing the model to learn more semantic knowledge and improve its performance (Guo et al., 2020). As shown in Figure 1 , to obtain a high-quality embedded representation and make good use of the information in the corpus, Word2vec-CBOW (Mikolov et al., 2013) was used to train Corpus A in character form and transform the resulting agricultural lexicon into vectors. The input sequence of length n is s=(c₁, c₂, c₃,……,c_n)∈V_c , where V_c is the word set (including characters), and each word is represented by a trained dense vector x i c = e c ( c i ) , where e^c denotes the word embedding lookup table.

CCNet (Huang et al., 2019) is often used in semantic segmentation to aggregate contextual information from all pixels to obtain dense contextual information. This study considered the use of CCNet for text feature extraction. The overall structure of the CCNet is shown in Figure 3A .

Given a feature map M∈R^C×W×H , CCNet first generates two feature maps Q and K by applying two convolutional layers with a filter size of 1×1 on the feature map M. {Q, K}∈R^C’×W×H , where C’ is the number of channels of Q and K, which is less than C for dimension reduction. Another convolutional layer with filters of size 1×1 is applied on M to generate V∈R^C×W×H . Q_u∈R^C’ is the vector for each position u in the spatial dimension of the feature map Q. And vector set Ω_u∈R^(H+W-1)×C’ is obtained by extracting feature vectors from K which are in the same row with position u. Then, CCNet can obtain D∈R^{(H+W-1)×W×H} , which represents the degree of correlation between features Q_u and Ω_i,u (i=[1,…,|Ω_u|]) by the affinity operation, which is defined as follows:

where d_i,u∈D. Feature map A is then obtained by applying a softmax layer on D over the channel dimension. CCNet can also obtain vector V_u∈R^C and set θ_u∈R^(H+W-1)×C . The set θ_u is a collection of feature vectors in V that are in the same row as position u. Finally, the contextual information is collected by the aggregation operation:

where M_u’ is a feature vector in the output feature maps M’∈R^C×W×H at position u, and A_i,u is a scalar value at channel i and position u in A. Contextual information is added to local feature M to enhance the local features and augment the pixel-wise representation.

One of the tasks of CANER is to recognize the boundaries of agricultural entities, and word segmentation information provides good guidance for identifying entity boundaries. However, CANER is affected by the strong domain characteristics of agricultural texts and the uneven distribution of entity categories (Guo et al., 2020). Adding more pre-training information will help the model learn more agricultural characteristics, thus reducing the impact of the aforementioned problems. Therefore, this paper proposes an AttSoftlexicon based on Softlexicon (Peng et al., 2020) and CCNet (Huang et al., 2019), and integrates the word information in the lexicon into character representation, which helps the model to learn more kiwifruit text features.

Assume that the input sequence is s={c₁, c₂,…, c_n}, and w_i,j denotes its subsequence {c_i, c_i+1,…, c_j}. The first step is lexicon matching. Each character is matched from a lexicon to all words containing the character. According to the position of each character c_i in the different matched words (beginning, middle, end, or one-character word), the words matched by a character were divided into four-word sets B(c_i), M(c_i), E(c_i), and S(c_i). The set construction method is shown in formula (7)-(10).

As shown in formula (7)-(10), L denotes the lexicon, and w represents the words matched in the lexicon. If a word set of characters is empty, it is represented as {None}. Taking the input sequence “植物病害” (plant disease) as an example, the character “物” (matter) is matched with the pre-constructed lexicon, and the two words “植物病害” (plant disease) and “植物” (plant) are matched, and the four word sets corresponding to the character “物” (matter) are formed: B={“None”}, M={“植物病害”}, E={“植物”}, S={“None”}. The character “病” (disease) is matched with the pre-constructed lexicon, and the two words “病害” (disease and pest) and “病” (disease) are matched, and the four-word sets corresponding to the character “病” (disease) are formed: B={“病害”}, M={“None”}, E={“None”}, S={“病”}, as shown in Figure 4 . To integrate the word set information matched to each character into the corresponding character representation, the statistics-based static weighting method in Softlexicon (Peng et al., 2020) was used, where the frequency reflects the importance of the word.

The weighting method is given by formulae (11) and (12), where z(w) is the frequency with which a lexicon word w occurs in the statistical data and e^w is the word embedding lookup table. The weighted representation of word set S is obtained as follows:

Where:

In the last step, the original Softlexicon (Peng et al., 2020) combines the representations of four-word sets into the fix-dimensional feature and adds it to the representation of each character, as shown in formulae (13) and (14).

The original Softlexicon (Peng et al., 2020) designed four-word sets to take advantage of these four types of positional information. However, it only weighs the words in each word set according to the word frequency and does not distinguish the importance of different word sets. This does not allow the model to distinguish the four positions of the characters in the matched words.

CCNet (Huang et al., 2019) showed a strong contextual relationship extraction ability in the semantic segmentation task. Therefore, to make full use of these four types of position information, this study uses CCNet to learn the weights for different word sets, as shown in the formula (15). First, CCNet processes the representation of these four sets and automatically assigns weights to them based on the relationship between them. It is then transformed into a vector of 1×4 through q. Finally, the weight vector a_i (i∈[1,4]) with a value range of (0, 1) is obtained through the sigmoid function. a_i is a weight matrix of dimensions 1×4, where the four values represent the importance of the four word sets. As shown in formula (16), the four-word set representations are weighted and merged into the character representation.

The sequence features extracted by BiLSTM may have a few limits. First, with an increase in sentence length, the feature extraction ability of BiLSTM declines (Huang et al., 2019). In addition, LSTM has been shown to have weaker feature extraction ability than attention mechanism models, such as transformers, when dealing with longer sequence texts (Li et al., 2020b). Second, BiLSTM makes each character contribute equally to the task. In other words, BiLSTM is not good at assigning more weight to some important characters in the text sequence, which is very important for NER. In addition, the strong domain features of kiwifruit-related texts mentioned in Section 2.1.3 also pose challenges to the feature extraction ability of the BiLSTM. In short, the feature extraction ability of BiLSTM must be further improved when solving the problem of kiwifruit-named entity recognition. Therefore, a novel module, parallel connection criss-cross attention network (PCAT), is proposed to mitigate the impact of the above limits with the help of CCNet (Huang et al., 2019). The overall structure of the PCAT is shown in Figure 3B .

After the agricultural sentence is processed by BiLSTM, a feature map X∈R^C×W is obtained (C represents the dimension of BiLSTM and W represents the length of the sentence). In this work, agricultural sentences are regarded as pictures with a channel number of C and a size of W×1. Therefore, PACT first transforms X into a feature map M∈R^C×W×H (the value of H is 1) through an unsqueeze operation. Each pixel in the feature map M represents a character in the agriculture text.

To obtain richer semantic information. The PCAT uses two different convolutional layers with filter sizes of 1 × 1 and 1 × 3 on M to generate two feature maps, M₁ and M₂ . M₁ and M₂ are put into the CCNet for processing. To learn more complex features, PCAT applies two convolutional layers with filter sizes of 1 × 3 to M₁ and M₂ . Finally, M₁ and M₂ are added, and the output vector of the PCAT X’∈R^C×W is obtained through a squeeze operation.

Using CCNet to calculate the connection between each character, PCAT can assign different weights to different characters to give more attention to key characters. In addition, PCAT can solve the problem of long-distance dependency because it can calculate the degree of association between words in each position and other words that are not affected by distance. Through a parallel structure and convolutional layer, PCAT can obtain richer features from agricultural texts.

In this section, some typical NER models such as BiLSTM (Huang et al., 2015), TENER (Yan et al., 2019), LR-CNN (Gui et al., 2019a), LGN (Gui et al., 2019b) and Softlexicon-LSTM (Peng et al., 2020) are considered comparable models. In addition, this section also uses the previous CANER findings JMCA-ADP (Guo et al., 2020) and Att-BiLSTM-CRF (Zhao et al., 2021) as comparison models. Like KIWINER, LR-CNN, LGN and Softlexicon-LSTM are also lexicon-based models. The lexicon used in the experiments in this section are the Kiwifruit lexicon constructed in this study.

The experimental results for KIWID are shown in Table 4 . It could be observed that the model proposed in this study outperformed other models, and the F₁ of this model is at least 0.47 higher than other models, which illustrates the effectiveness of it recognizing kiwifruit-related entities. The performance of our model is significantly improved compared to the baseline model BiLSTM-CRF. This is due to the fact that KIWINER makes full use of kiwifruit lexical information with the help of AttSoftlexicon, and obtains deeper semantic features with the help of PCAT. Compared with CANER models Att-BiLSTM-CRF and JMCA-ADP, KIWNER has achieved obvious improvement, which further verifies the effectiveness of KIWINER. The lexicon-based models LR-CNN, LGN, Softlexicon-LSTM and KIWINER have clear advantages over the rest of the character-based models, illustrating the effectiveness of constructing a kiwifruit-related lexicon and incorporating lexical information into the model.

To verify the generalization of KIWINER, three public datasets were selected: Boson, ClueNER, and People’s Daily. The experimental results are listed in Table 5 .

The KIWINER model achieved the best F₁ of the three datasets, which were for Boston, ClueNER, and People’s Daily 85.13%, 80.52%, and 92.82%, respectively. The experimental results show that KIWINER not only has performance advantages on the KIWID corpus, but also has a certain generalization in other fields.

To verify the rationality of the PCAT module, several variants of it were designed, and the variant was used to replace the PCAT in KIWINER for experiments on Boson, ClueNER, KIWID, and People’s Daily. Variants A and B increased and decreased the depth of the PCAT, respectively. Variants C and D break the parallel connection structure of PCAT. The different variant structures of the PCAT are shown in Figure 6 . In addition, many researchers use the self-attention mechanism (Self-Att) to improve the feature extraction ability of the sequence encoding layer. In the field of CANER, Guo et al. (2020) introduced a self-attention module after the BiLSTM model to improve the feature extraction ability of sequence coding layer. Therefore, this section refers to the study by Guo et al. (2020) and uses Self-Att instead of PCAT for experiments. Attention unit and head number of Self-Att is 600 and 8. The experimental results are listed in Table 7 .

Compared with Variants A and B, the PCAT achieved better results, indicating that the depth design of the PCAT is reasonable. Compared with variables C and D, PCAT achieves better results, which shows that a parallel structure can effectively improve the feature extraction ability of the model and help the model obtain richer semantic information. PCAT achieves better results than Self-Att (Guo et al., 2020), which indicates that PCAT is more conducive to improving the model feature extraction capability than the commonly used module Self-Att. PCAT constructs a parallel structure with the help of two different convolutional layers, which allows the model to simultaneously process semantic information from two different perspectives. At the same time, with the help of CCNet, which has good long distance context semantic aggregation capability (Huang et al., 2019), the information can be processed again, and different weights can be given according to different information relationships. Therefore, PCAT can help the model make full use of the feature information input into the model.

This section discusses the recognition effects of KIWNER and the previous CANER studys Att-BiLSTM-CRF (Zhao et al., 2021) and JMCA-ADP (Guo et al., 2020) on each category of the kiwifruit dataset KIWID. BiLSTM-CRF (Huang et al., 2015), the baseline model of the above models, also participated in the experiments. The experimental results are shown in Table 8 , where F₁ is chosen as the evaluation metric, and the last column of the table is the running time of each model.

It can be clearly seen from the table that KIWNER has achieved the best results in each category, especially in disease, pest, pesticide, which contain strong domain features. Although Att-BiLSTM-CRF and JMCA-ADP have made efforts to integrate agricultural features into the model, KIWINER can obtain more agricultural features by using word information with the help of Attsoftlexicon and new word detection. In addition, PCAT can help the model to further make full use of these agricultural features. The category of location related entities usually contain boundary characters, such as “县” (county), “镇” (town), “村” (village), etc., and the category of part related entities have limited diversity and many repeated words, which leads to the recognition difficulty of the above two categories being relatively low. Therefore, KIWINER did not significantly improve the recognition effect of LOC and PART. From the last row of the table, we can see that KIWINER takes more time than other models, which is a disadvantage of KIWINER. KIWNER incorporates lexical information, so it will spend an extra part of time on processing lexical information compared with the character based model. Research on faster character and word matching methods and more efficient sequence encoding modules can be helpful to overcome this shortcoming.

In KIWNER, AttSoftlexicon module and PCAT module both adopt the CCNet model from semantic segmentation, and have achieved good results through experimental verification. With the help of new word detection and AttSoftlexicon, KIWINER incorporate the word information containing domain features into the model. And KIWINER has achieved significant improvement compared with previous which are character-based models. This shows that when solving problems with strong domain features such as CANER, it is a good solution to find a method to integrate more domain features into the model. In addition, the effectiveness of PCAT also shows the importance of making full use of these features.