Recent developments in areas such as Computer Supported Cooperation Work (CSCW) emphasize the concept of cooperation between humans and machines. Following several pioneering works in the area of teleoperation, the necessity of human supervision of automated machines has come into question, notably in the field of industrial robotics (Bejkzy, 1976; Sheridan, 1976; Vertut & Coiffet, 1984). Different types of architecture for such CSCW, for example in air traffic control, have also been proposed (Millot, 1988; Millot & Mandiau, 1995).

With cost control in mind, the robot was designed with limited means of perception, an odometer and an ultrasonic ring. Odometry gives the robot’s position and the orientation in relation to angular rotation of the wheels. The method is simple and low-cost but presents a systematic error, a lack of precision when measuring distance, and a non-systematic error, mainly due to wheel spin and sliding. As a result, the actual position of the robot may differ from its theoretical position as calculated by the wheels. An ultrasonic ring measures the distance between the robot and obstacles the robot’s environment. The accuracy of this ultrasonic technology is limited by proximity. At close range, a high rate of erroneous measures may occur, as the algorithms operate inaccurately under such difficult conditions.

To command the robot, the operator chooses a goal on the plan and the planning function is performed to calculate the path. Then, the navigation function plots the path followed by the robot.

The control station is composed of a screen which displays different types of information via enhanced reality techniques (fig. 4): the robot position on the 2D flat plan, the US measures and robot operating indicators.

We assume that trajectory errors may occur, the operator must detect errors and categorize them before manual intervention. We are then faced with the question of the operator’s detection capabilities and self-confidence in making such decisions.

Even if the trajectory on the screen is correct in relation to the goal, the actual robot trajectory could be erroneous. The only visual information available to indicate a trajectory error is dots plotted on the screen, representing US impacts. Trajectory error detection can, therefore, be analyzed as a signal detection situation. In this type of situation, the signal is the specific pattern of US dots on the screen compared to the US pattern that is supposed to correspond to the walls and object surfaces. We are also faced with the problem of some “lost” US dots that appear on the screen as noise, or dots not corresponding to the walls and objects in the environment. The signal is, therefore, the relative alignment of some US dots corresponding to the surface of an object, and the noise, some non-aligned US dots. Fortunately, at an early stage of the visual perception process, a colinearity function occurs to detect object edges (Perterhans & Von der Heydt, 1989; Boucard, Delord, & Giersch, 1994), which means that aligned dots will be interpreted as object surfaces in this situation. As a result, the detection of a trajectory error is based on the subject’s visual capacity to detect aligned dots and to recognize the line, corresponding to the wall’s position on the screen. If the perceived alignment of the US dots, representing the wall, does not correspond to the supposed position of the wall, it will mean that the robot’s actual distance from the wall is different to the supposed distance. In this case, the robot’s trajectory can be corrected.

In this study, self-confidence is defined as a human judgment on the acuity of a decision. When an operator detects an error, it is the result of perceptual and decisional processes. The decisional process itself is determined by rewards and penalties following each response, or social consequences in the form of the perceived effect the decision will have on the operator or others. Then, after the decision, when the sensory receptor sensibility remains constant, and/or there are the same decision criteria, an operator can be more or less confident in the “yes” or “no” responses. Self-confidence regarding a judgment is, therefore, a kind of meta-cognitive activity. For intellectual judgments Dawes (1980) showed that the subjects overestimate their capacities to judge correctly, but underestimate their capacities for perceptual judgments. For intellectual judgments, such as those involving memory, knowledge or deduction, overestimation in self-confidence is generally accepted. Recent works have shown controversial results regarding perceptual judgments (Winman & Juslin, 1993; Petrusic & Baranski, 1997). Human trust in automated processes and operator’s self-confidence in performing manual interventions has also been studied (Lee & Moray, 1992, 1994; Moray, Hiskes, Lee, & Muir, 1995). Results of these studies show that the operator’s intervention in an automatic process depends on the ratio of confidence calculated by the operators trust in automated systems and their self-confidence in manual control. The degree of automation the operator decides to employ varies according to the difference between trust in automatic control and self-confidence in manual control; this in turn depends on perceived failures in each mode. In this study, the human/machine reliability system is composed of the operator’s confidence in the robot’s displacements and the operator’s self-confidence in detecting the robot’s false trajectories. In other words, the system’s performance depends not only on the machine’s reliability and the operator’s ability to correctly detect the trajectory errors of the robot, and to correct it manually, but also on the operators level of self-confidence in their decisions.

An experiment was performed to see if a human operator is able to detect errors with only two kinds of information: the robot’s position given by the odometer and the US measures.

The material included a mobile robot, a control system and a screen, described earlier in the assistance system components.

34 subjects, all students and able-bodied, aged from 18 to 26 years old. 25 students in psychology out of the total of 34 subjects had No Background in Technology (NBT). The other 9 subjects were students in technology and therefore had a Background in Technology (BT). They were considered as more familiar with robotics compared to subjects with NBT.

The room was presented to the operator on a video screen (fig. 2). Feedback to the operators, available on the screen, was information relative to the robot’s past trajectory, robot’s present position, US past and present dots and a numerical matching between US dots and wall position.

According to the typical matrix of the SDT we calculated the number of detected trajectory errors (Te) corresponding to a signal detected (Hits), un-detected trajectory errors corresponding to an un-detected signal (Misses), detected correct trajectory (Tc) corresponding to an absence of correctly detected signal (Correct negatives) and an un-detected correct trajectory corresponding to a false detection of a trajectory error (False alarms) (table 1).

As the maximum of responses was not equal in each cell of the outcome matrix, we calculated the frequency for each level of self-confidence in the decisions. As there was no or few 0 level self-confidence ratings for TB and NTB subjects, at first approximation, we reduced the sensitivity of the confidence criteria to two levels, low and high confidence. Levels 0 and 1 and levels 2 and 3 were combined (fig. 5).

Good calibration means that the operators are confident in their correct decisions, hits and correct negatives and unconfident in their incorrect decisions, misses and false alarms. For example, if the percentage of trajectory errors detected (hits) is 93,5% and the average confidence rating under 93,5%, this means operators are under-confident in their decisions. If their average confidence rating is over this percentage they are over-confident in their decisions. Following Petrusic and Baranski (1997), we analysed the over/under confidence by computing a percent confidence rating C. We assume the four confidence levels as a scale with three intervals. C=nj(l)/3 with n self confidence frequency, j level on the self-confidence scale and l value on the three intervals scale (table 2).

As of late, robotics has been conceived as a way to exclude operator from the control loop. Consequently, there are very few evaluations in the area of human performance in robot control. Teleoperation situations lead us to the general idea of cooperation between robots and humans and emphasises the necessity to develop experimental studies. Nevertheless, knowledge in this type of application is poor. Theoretical frame should necessarily come from the general field of human/machine interaction, interaction between operators and more recently from human/machine cooperation (Hoc & Millot, 1999). For the last, empirical studies refer to very different work and technological contexts and system analysis is often a privileged methodology to propose general and theoretical frame of human/machine cooperation. One attempt of this work was to know if robotics cooperation was at least similar or not to human/machine interactions. This experiment sought to discover if human subjects could detect trajectory errors with only minimal information displayed on the screen (robot’s trajectory, US impacts and walls). We deliberately chose to use the antiquated signal detection model used in previous human/machine studies to analyse the data, even if the number of observations was small and the subjects were unfamiliar with the task. This is often the case when exploring new technology developments with little or no engineering or ergonomic background.

The result of this study leads to general considerations about the operator’s cognitive evolution regarding his type of cooperation with machine. Operators will frequently use automatic systems and appreciate it when begining. They will probably increasingly prefer to manually control the system when more skilled and experienced until the workload exceeds the operator’s capacity. In human/machine cooperation, it is often assumed operators switching on automatic mode control is related with mental workload. In addition, the decision to operate on manual or automatic mode will depend on meta-cognitive activities such as self-confidence, and confidence in automation as Moray et al. (1995) pointed out. We suggest it also varies with the operator’s percieved competence in controling machines which is in human/machine cooperation related to technological knowledge background. In the theory of motivation perceived competence is considered as a generic factor of motivation. Then the security of a system as well the operator’s satisfaction will also depend on this dynamic decision criteria.

The authors are gratefull to Thierry Goater for English corrections of the first version, Wendy Schubring and David Allen for general English revision of the second version of this paper. We thank the two anomymous reviewers and Jean-Michel Hoc for their comments and suggestions.