European starlings unriddle the ambiguous-cue problem

The ambiguous-cue problem is deceptively simple. It involves two concurrently trained simultaneous discriminations (known as PA and NA trials), but only three stimuli. Stimulus A is common to both discriminations, but serves as non-reinforced stimulus (S-) on PA trials and as reinforced stimulus (S+) on NA trials. Typically, animals’ accuracy is lower on PA trials—the ambiguous-cue effect. We conducted two experiments with European starlings (Sturnus vulgaris) using Urcuioli and Michalek’s (2007, Psychon B Rev 14, 658–662) experimental manipulations as a springboard to test the predictions of two of the most important theoretical accounts of the effect: the interfering cue hypothesis and value transfer theory. Both experiments included two groups of birds, one trained with a regular ambiguous-cue problem (Group Continuous) and another trained with partial reinforcement on PA trials (Group PA-Partial). The experiments differed only in the number of sessions (18 vs. 36) and daily trials (360 vs. 60). As previously observed, we found faster acquisition on NA trials than on PA trials in both experiments, but by the end of training PA performance was surprisingly high, such that no ambiguous-cue effect was present in Group Continuous of either experiment. The effect was still present in both PA-Partial groups, but to a smaller degree than expected. These findings are inconsistent with the literature, in particular with the results of Urcuioli and Michalek (2007) with pigeons, and question the aforementioned theoretical accounts as complete explanations of the ambiguous-cue effect. In our view, to achieve such high levels of accuracy on PA trials, starlings must have attended to configural (i.e., contextual) cues, thus differentiating stimulus A when presented on PA trials from stimulus A when presented on NA trials. A post hoc simulation of a reinforcement-based configural model supported our assertion.


INTRODUCTION
Discrimination learning theory has been continuously challenged by the ambiguous-cue problem. This deceptively simple problem involves three stimuli arranged in two simultaneous discriminations. The critical feature is that the reinforced stimulus (S+) in one discrimination serves as the non-reinforced stimulus (S−) in the other, hence the ambiguous-cue problem (Thompson, 1954). The three stimuli are: P, the positive or always reinforced stimulus; N, the negative or always non-reinforced stimulus; and A, the ambiguous stimulus, which is negative or positive depending on whether it is presented together with P or N, respectively. The simultaneous discriminations are usually denoted as PA trials and NA trials.
Two main accounts have been proposed to explain the effect, both stressing the status of A as an excitatory stimulus in both discriminations. The dominant and initial approach, known as the interfering cue hypothesis (e.g., Zeaman and House, 1962;Fletcher and Garske, 1972) proposes that whereas PA trials involve an approach-approach conflict (P is always reinforced and A is partially reinforced across discriminations), no conflict is evident on NA trials (A is partially reinforced and N is never reinforced). In other words, PA trials involve a choice between stimuli with differential but positive value and NA trials involve a choice between a stimulus with positive value and a stimulus with either negative or zero value, thus making the NA discrimination much easier.
More recently, Urcuioli and Michalek (2007) proposed that value transfer may also contribute to the effect. Value transfer theory (von Fersen et al., 1991) proposes that associative value transfers from one stimulus to the other in a simultaneous discrimination and subsequent research has shown that value transfer (a) does indeed occur (Zentall and Sherburne, 1994), and (b) it occurs only from the S+ to the S− and not in the opposite direction (Clement et al., 1998; for a possible role of the hippocampus in value transfer, see Van Elzakker et al., 2003). One possible mechanism for this"transfer"is second-order conditioning (Davis, 1992). Assuming that in a simultaneous discrimination the animals observe the stimuli sequentially (Wright and Sands, 1981), then the S− is functionally followed by the S+ on approximately one half of the trials. In Pavlovian language, CS2 is followed by CS1 and then by the US. Such second-order conditioning should impart "value" to CS2 (Zentall et al., 1996;Zentall, 2004). According to this theory, the value of any stimulus in a simultaneous discrimination is the sum of its direct value (conveyed by reinforcement) plus the indirect value it receives through positive value transfer (for what appears to be an anticipation of value transfer theory, see Leary, 1956Leary, , 1958. To test whether or not the value transfer mechanism plays a role in an ambiguous-cue task, Urcuioli and Michalek (2007, Exper-iment 1) devised a test involving partially reinforcing P on PA trials. With this schedule, both P and A would be reinforced 50% of the time, but A would also acquire some value from P via value transfer. Accordingly, if value transfer operates, the overall value of A would exceed that of P, leading to the peculiar prediction of below chance performance on PA trials (i.e., animals should select the S− more frequently than the S+ on PA trials). In contrast, the interfering cue hypothesis which considers only direct values, predicts that if P is reinforced 50% of the times and A is also reinforced 50% of the times (although admittedly across trials), subjects should be indifferent between P and A. On NA trials, both accounts predict high levels of accuracy. Urcuioli and Michalek's (2007) pigeons attained accuracies of 9.2% on partially reinforced PA trials and 99.5% on NA trials, thus suggesting that value transfer contributes to ambiguous-cue performance and seriously questioning the predictions of the interfering cue hypothesis.
Given that the results of Urcuioli and Michalek (2007) stand alone in the ambiguous-cue literature in questioning the dominant account and simultaneously propose an alternative mechanism, this study re-examined the controversy using a new model species, the European starling (Sturnus vulgaris).

EXPERIMENT 1
This experiment was a systematic replication of Groups Continuous and PA-Partial ran by Urcuioli and Michalek (2007, Experiment 1). The main changes were the number of trials scheduled per daily session (60 in Urcuioli and Michalek, 2007, 360 here) and the model species (pigeons and starlings, respectively). A schematic of the design of this experiment is presented in Figure 1.

Subjects
Twelve experimentally naïve, wild-caught adult European starlings (S. vulgaris) participated in the experiment (English Nature license No. 20010082). During the experiment, starlings were housed in pairs in indoor cages which were visually, but not acoustically isolated. Each room contained two cages that served both as home and experimental cages. Indoor temperatures ranged from 15 to 18 • C and lights followed a 12:12 dark schedule with light from 07h00 to 19h00, and gradual transitions at dawn and dusk. The colors (red, green, and blue) associated with stimuli P, A, and N were counterbalanced across subjects. P, A, and N appeared equally often on the left and right keys; G denotes Group and p denotes probability.
After each daily experimental session, starlings had 4 h (13h00-17h00) of free access to turkey crumbs, Orlux© Remiline universal granules and 10 mealworms daily, and social interaction with the respective cage-partner. This regime maintains starlings at approximately 90% of their free-feeding weight (Bateson, 1993). All procedures were in accordance with the University of Oxford's animal care guidelines.
When not participating in an experiment, the starlings were housed together in two outdoor aviaries. While in the aviary, they received ad libitum food, a mixture of turkey crumbs, Orlux pellets and mealworms (Tenebrio sp.). Drinking and bathing water was always available and replaced daily. After the experiment, starlings were returned to the communal aviary. All subjects were released into the wild after participating in three similar experiments, and following at least 2 weeks of re-acclimatization to natural light in the outdoor aviary.

Apparatus
Each indoor cage [135 × 78.4 × 80 cm (l × w × h)] was composed of two units, vertically mounted (80 cm each). Each unit included two experimental areas separated by a common middle section. Each experimental area had a panel attached 10 cm above the floor. The panel was 40 cm tall with three sections: a middle sub panel, facing the cage (11.5 cm wide), and two side subpanels (same length) attached to the cage at a 120 • angle from the center subpanel. The middle subpanel had one response key in the center (11 cm from the bottom), and the food hopper (2.5 cm from the bottom), that was connected to the pellet dispenser (Campden Instruments ® ) containing 20 mg BioServ ® precision pellets. Each side subpanel had one response key in the center (11 cm from the bottom). Every key was composed by a 16 LED matrix that could display 16 different symbols in seven possible colors. A schematic of the panel is presented in Figure 2. A computer in an adjacent room controlled all experimental events.

Procedure
Prior to the start of the experiment, starlings were randomly divided into two groups: Group Continuous and Group PA-Partial.
Preliminary training. All starlings received autoshaping training sessions to ensure that they pecked all keys and hues used in this experiment. Autoshaping particulars were as described in Aw et al. (2011, Appendix 1). At the end of this phase, all starlings reliably pecked the white center key, and the red, green, and blue side-keys (all 4 × 4 LEDs) for reinforcement.

Discrimination training.
Next, all starlings were trained on an ambiguous-cue discrimination, in which red, green, and blue served as discriminative stimuli. All trials began with the white center key flashing (700 ms ON, 300 ms OFF). A single peck to this attention key switched its light off and immediately produced two discriminative stimuli on the side keys. A single peck to either key turned both stimuli off and produced either two food pellets or advanced the program to the next trial, depending on whether the response was correct or incorrect, respectively. Consecutive trials were always separated by a 40 s inter-trial interval (ITI) during which no operant panel lights were illuminated.
The P and A stimuli were presented together on half of the trials (PA trials) and the N and A stimuli (NA trials) were presented together on the remaining half. On PA trials, choosing P was always reinforced for starlings in Group Continuous but reinforced only half of the time for starlings in Group PA-Partial, whereas choice of A was always unreinforced in both groups. On NA trials, choosing A was always reinforced and choosing N was always unreinforced for both groups.
Starlings received one session per day for 18 days. Sessions ended after 360 trials or 5.5 h from the session start (7:30 am), whichever came first. Each completed session consisted of 180 PA trials and 180 NA trials, all with side key allocation counterbalanced across trials. PA and NA trials occurred in a pseudorandom order with the constraint that no more than three of each trial type could occur consecutively. The hues (red, green, and blue) used for the stimuli P, A, and N, were counterbalanced within each group.

Data analysis
Prior to analysis, all proportion data were normalized using an arcsine square root transformation, (Grafen and Hails, 2002). A Type-1 error rate of 0.05 was adopted for all statistical comparisons. Of the 12 starlings, one bird from Group Continuous was dropped from the study due to an injury during the initial sessions of discrimination training. Figure 3 shows the mean proportion of correct responses on PA and NA trials for both groups. As predicted, both groups revealed faster acquisition on NA than on PA trials. However, the pattern of results for PA trials in both groups is noticeably different from that reported by Urcuioli and Michalek (2007), with both groups reaching high levels of accuracy.

RESULTS AND DISCUSSION
A mixed analysis of variance (ANOVA) with group, trial type and session as factors revealed a significant main effect of group [F(1,9) = 6.72, p = 0.029, η 2 p = 0.43], trial type [F(1,9) = 54.87, To summarize, the results observed on NA trials are consistent with what has been repeatedly reported in the literature (e.g., Fletcher et al., 1968;Richards and Marcattilio, 1975;Hall, 1980;Urcuioli and Michalek, 2007). On the contrary, starlings' accuracy on PA trials reached levels that, to our knowledge, have never been reported in a standard ambiguous-cue discrimination task, at least in birds. Richards and Marcattilio (1975) and Urcuioli and Michalek (2007) did find high levels of accuracy on PA trials but only when NA trials were partially reinforced.
In fact, the terminal performance of both groups conflicts with the predictions of the interfering cue hypothesis and value transfer theory. On the one hand, the interfering cue hypothesis predicts a much larger PA-NA difference in both groups. In particular, NA performance should be high in both groups (as we indeed observed) and PA performance should be above chance (but below NA performance) in Group Continuous and at chance in Group PA-Partial. On the other hand, value transfer theory predicts high accuracy levels on NA trials in both groups, but chance and below chance performance on PA trials in Groups Continuous and PA-Partial, respectively. We observed none of these patterns. Nonetheless, the pattern of acquisition observed in the first few sessions (lower PA than NA performance) is predicted by both accounts, but not fully consistent with either of them (e.g., PA performance in Group Continuous).
A noticeable procedural difference between our study and the vast majority of studies reported in the literature is the number of trials per session. For instance, Urcuioli and Michalek (2007) included 60 trials per session compared to the 360 trials used here. At the end of session 4 (with a maximum of 720 trials per discrimination completed), the starling's PA performance was already significantly above chance in both groups. At roughly the same number of training trials, Urcuioli and Michalek's (2007) pigeons were either at chance (46.25% correct, Group Continuous) or below chance (9.17% correct, Group PA-Partial).
Naturally, we cannot guarantee that the massed daily presentation of 180 PA and NA trials is functionally the same as presenting 180 trials over six consecutive days. In fact, it is certainly not equivalent in terms of the dynamics of learning and memory (e.g., Roberts, 1974;Adams, 1982;Yin et al., 1994).
Experiment 2 attempts to eliminate these possible confounds in the comparison between studies by closely following the procedure used by Urcuioli and Michalek (2007).

EXPERIMENT 2
To have a clearer idea of how starlings' performance in the ambiguous-cue problem compare to other species and whether or not their performance questions the major explanatory accounts of the effect, we ran a second experiment comprising only 60 trials per daily session.

MATERIALS AND METHODS
Twelve experimentally naïve, wild-caught adult starlings participated in this experiment. Their licensing, housing and training conditions were as described in Experiment 1, except that 36 daily sessions were run, each composed of 60 trials.

RESULTS AND DISCUSSION
The overall pattern of results observed in this experiment was similar to that obtained in Experiment 1. Both groups reached relatively high levels of accuracy both in PA and NA trials, albeit at different stages of training. The mean proportion of correct responses on PA and NA trials for both groups over blocks of two sessions is shown in Figure 4. and F(17,170) = 2.55, p = 0.001, η 2 p = 0.20, respectively]. This pattern of results confirms once again a PA-NA difference in the rate of acquisition, but also what appears to be a strong effect of partial reinforcement on PA trials. A retardation of acquisition is clear in Group PA-Partial relative to Group Continuous on PA trials. The drop below chance in the first few sessions could be potentially caused by a value transfer mechanism, but asymptotic performance suggests that such mechanism, if it in fact operated early on, was subsequently overcome by other factors. A mixed ANOVA with Group and Block as factors restricted to PA performance confirmed a significant Group × Block interaction [F(17,170 Figure 4 shows (1.65%, on average, to be exact). One evident difference between experiments is that PA accuracy was generally lower in this experiment (cf. Figures 3 and 4), which is most probably due to the large difference in the amount of training.
Overall, this experiment confirmed the general findings of Experiment 1, although it also revealed a clear effect of partial reinforcement on PA trials that was probably masked in Experiment 1 by the larger number of daily trials. In fact, this was the only hint that a value transfer mechanism could potentially be operating early on. Nonetheless, starlings' asymptotic accuracy on PA trials defies both value transfer theory and the interfering cue hypothesis as complete explanations of the ambiguous-cue problem. While a significant PA-NA difference is still observed in Group PA-Partial, the accuracy levels attained by both groups cannot be explained by either theoretical account.

GENERAL DISCUSSION
The present findings are hard to reconcile with the major theoretical accounts of the ambiguous-cue effect, namely the interfering cue hypothesis and value transfer theory. The terminal PA-NA differences observed in both experiments are unanticipated by either account, thus suggesting that either starlings are better able to overcome the typical difficulties posed by the task or they are less susceptible to them. Although an interpretation based solely on the interfering cue hypothesis or value transfer might be insufficient to explain this result, starlings did show slower acquisition on PA trials and almost perfect performance on NA trials, which seems to indicate that some of the mechanisms proposed by such accounts may indeed operate early on.
Particularly noticeable is the difference between our results and those reported by Urcuioli and Michalek (2007, Experiment 1). Whereas their pigeons in Groups Continuous and PA-Partial showed chance and below chance performance on PA trials, respectively, our starlings learned the PA discrimination to high levels of accuracy in both groups. Of course, the use of different species (pigeons vs. starlings) makes comparisons between experiments complicated and we cannot exclude that that may indeed be one of the reasons for such discrepancies. Unfortunately, comparisons between species have been hampered by differences in research programs in comparative psychology and behavioral biology (Shettleworth, 1993), but the scenario is changing rapidly (Shettleworth, 2009(Shettleworth, , 2012. Regarding the comparison of the cognitive and learning abilities of pigeons and starlings very little is known, although both species have been exposed to similar tasks by different research groups, mainly in the domains of timing (e.g., Gibbon, 1977;Gibbon et al., 1988;Brunner et al., 1992;Kacelnik and Brunner, 2002), and decision making (e.g., Schuck-Paim and Shapiro et al., 2008;Lagorio and Hackenberg, 2010;Mazur, 2010;Vasconcelos et al., 2010;Kacelnik et al., 2011;Aw et al., 2012).
Another important difference between studies is related to the operant panels used. At first sight, our angled panels (cf. Figure 2) may have precluded value transfer from occurring. If, as proposed, the mechanism supporting value transfer is second-order conditioning (CS2 followed by CS1 and then by the US; cf. Zentall et al., 1996;Zentall, 2004), this is much more likely to occur with flat panels (as those used by Urcuioli and Michalek, 2007) which facilitate sequential observation of stimuli than with angled panels where the stimuli are more likely to be observed simultaneously provided, as we did, the birds are centrally located (for a similar discussion on the effect of apparatus variations on behavioral data, see Katz et al., 2008). Be that as it may, we did observe below chance performance on partially reinforced PA trials in the first few sessions of Experiment 2, which may render the argument moot.
Curiously, the most successful transitive inference models assume that each stimulus is, to some extent, bound to the context in which they occur (see, Rescorla, 1972;Whitlow and Wagner, 1972;Rescorla, 1973). In particular, Wynne's configural model (Wynne, 1995(Wynne, , 1998 and Siemann-Delius model  ; see also Siemann and Delius, 1998) propose that a stimulus is not functionally the same when presented in different discriminations: it has an elemental value but also a configural value dependent on other stimuli simultaneously present.
Returning to the ambiguous-cue task, this reasoning means that stimulus A is not functionally the same on PA and NA trials because of different configural cues (P and N, respectively). If this is indeed the case, then accuracy on PA trials should increase as animals learn the informational value of such cues. Given that this argument is consistent with our results, we fitted Wynne's configural model to the results obtained in Experiment 2. This model is well known in the literature and we will not detail it here (for mathematical details, see Wynne, 1995Wynne, , 1998Lazareva et al., 2004;Vasconcelos, 2008). The data of each starling were fitted individually, using the full sequence of trials presented during training. Because the model is not intended to account for acquisition data we searched for the combination of parameters that provides the best-fitting performance to the asymptotic performance (last four sessions) by minimizing the root-mean-square error (RMSE). Figures 5A,B depict the results of these simulations for Groups Continuous and PA-Partial, respectively. Overall, the predictions closely matched asymptotic performance, with average RMSEs of 0.03 and 0.04. This suggests that attention to configural cues may indeed be relevant to solve the ambiguous-cue problem. What remains to be answered is why Urcuioli and Michalek's (2007) pigeons seemed to ignore such cues, despite the ensuing decrease in food intake. One possibility rests again on the panels used. Perhaps configural cues are harder to learn with flat panels, where simultaneous observation of both stimuli is less likely. Still, without further tests, this belongs to the realm of speculation. An established fact, however, is that under some circumstances pigeons are able to attend to the gestalt of whole stimulus displays (i.e., configural learning; e.g., Wright, 1997;Katz et al., 2008) and to learn the more complex transitive inference task despite a rather slow acquisition (for a review, see Vasconcelos, 2008).
For the moment, it is clear that European starlings' terminal performance on the ambiguous-cue problem challenge the major theoretical accounts of the ambiguous-cue effect. Neither the interfering cue hypothesis nor value transfer theory can predict high PA accuracy, particularly in Group PA-Partial. Yet, the initial acquisition pattern observed in both experiments is at least partially consistent with such accounts. Perhaps the inconsistencies between this and previous studies are due to species differences (to our knowledge this was the first time starlings were tested on this task), differences in apparatus, or a combination of both.
Whilst the reasons for the discrepancies between starlings' and pigeons' performance await for further tests, this article will have served its function if the reader is at least partially convinced that current accounts of the ambiguous-cue problem are incomplete and that the dynamics of acquisition is not yet fully understood. Hopefully, this will lead to new theorizing about the problem, standardized procedures and informative new experiments.

AUTHOR CONTRIBUTIONS
Marco Vasconcelos and Tiago Monteiro jointly designed the experiments and collected the data, Marco Vasconcelos processed and analyzed the data, Marco Vasconcelos and Tiago Monteiro jointly wrote the manuscript.