How radiographers visually perceive X-ray images with the task of accepting or rejecting them – a pilot study

Introduction: Radiographer ́s usual role in the medical imaging chain is to acquire relevant and qualitatively good images that help the radiologist or physician to diagnose most accurately. After the image acquisition, the radiographer does a quality evaluation based on established imaging criteria to decide if the image is satisfactory, or otherwise reject it and subsequently take a new one. Contrary to expectations that the number of image rejects should decrease substantially with the introduction of digital imaging, a number of studies have shown that it has not, although the reasons for rejects has changed from exposure errors to positioning and centring errors. Very little research has been on examining how radiographers visually perceive and evaluate the X-ray images in this acceptance/rejection process. Purpose: Investigate how radiographers and radiography students visually perceives X-ray images in the process of accepting or rejecting them on basis of radiographic imaging criteria, and see if there are differences in strategies across experience levels. Materials and methods: Three radiography students and five radiographers with varying years of experience were given the task of accepting or rejecting shoulder and knee projection images based on positioning criteria. Using eye tracking, we measured the participants’ number and duration of gaze fixations within 1) the field of view defined by the monitor display, 2) the part of the monitor displaying the X-ray image only, and 3) the region within the X-ray images considered to be most relevant given the imaging criteria task. The quantitative eye-tracking measurements were followedup by four qualitative questions. Results: Some differences in fixation patterns between the groups were found; the medium experienced radiographers spent statistically significant lesser number of fixations and lesser average single fixation durations than both the radiography students and the most experienced radiographers did, whereas the two latter groups scored almost equally. Conclusion: The study revealed that work experience might have some influence on how radiographers and radiography students assess X-ray images, but in subtler ways than expected. The study also revealed, however, quite large individual differences across experience. Dag Waaler, Sigrid Hammer, Camilla Langdalen, Linn Therese Håkonsen Haug How radiographers visually perceive X-ray images with the task of accepting or rejecting them Side 2 https://journals.hioa.no/index.php/radopen/index Introduction One of radiographer ́s roles in the imaging chain is to produce optimal images in order for the radiologist or doctor to make an accurate diagnosis. To this end an important task is to decide whether to upload a newly taken image to the PACS, or reject it and acquire a new one. Such image rejects /retakes impose challenges within radiographic imaging: they occupy unnecessary resources, expose patients to unnecessary ionizing radiation, and might indicate suboptimal quality management. A number of studies have shown that image reject /retake rates decreased substantially with the introduction of digital imaging, from 10-15 % to 3-5 % (1-4), mainly because exposure errors almost completely vanished, Later studies, however, have indicated that due to increasing occurrences of image positioning and centring errors, reject/retake rates once more have increased (5-7). One of these studies (7) reported rejects /retakes for knee and shoulder imaging to be 20,6 % and 9,4 %, respectively, in which positioning and centring errors contributed approx. 80 %. This led our interest to look into how radiographers actually visually perceive the images in the process of deciding an X-ray image to be accepted or rejected. Most of the research on the perceptual and cognitive processes that underlay the diagnostic imaging processes have been from the point of view of the diagnostic decision-maker, and less from the point of the radiographer or radiology technologist (8-10). Admittedly, there are several studies where radiographers themselves act as diagnostic reporters, in for instance CT imaging (11) and conventional (planar/ skeletal/ projected) X-ray imaging (12-15), but these are other types of imaging tasks than accepting/ rejecting. The pattern of the reader ́s focal movements when evaluating an image can be described as a series of focal “rest” at specific points (a “fixation”) in the image, followed by a rapid movement (“saccade”) to another fixation point (16). Several factors may influence eye movements in viewing a scene, including the imaging task and the prior knowledge of the viewer (17), as well as particular image properties (18). The present paper investigates how radiographers ́ focal attention “move” when examining an image during the reject/ retake decision process, quantified as the number and lengths of fixations, and if and how this is effected by their level of experience. Materials and Methods Images Two different skeletal X-ray imaging protocols were chosen for this study, Bontrager ́s “Lateral knee” (19) and Movin and Karlsson ́s “Elevated shoulder” (20) projection imaging, respectively, chosen because they are used by the hospital and because they represent imaging procedures known to have high image rejects rates (7). Furthermore, because positioning and centring errors are expected to account for most of the rejects, we focused on radiographic criteria relating to this: Bontrager «Lateral knee» (19) (p 247): Adjust rotation of body and leg until knee is in true lateral position (femoral epicondyles directly superimposed and plane of patella perpendicular to plane of image receptor. Flex knee 20° to 30° for lateral recumbent projection. Align and centre leg and knee to central ray, and to midline of table or image receptor. Movin and Karlsson «Elevated shoulder» (20) (p 139): Place scapula ́s dorsal surface towards the bucky (“film”). Upper arm is abducted 90° and externally rotated 90°. Central beam 3 cm caudal to upper edge of acromion. Dag Waaler, Sigrid Hammer, Camilla Langdalen, Linn Therese Håkonsen Haug How radiographers visually perceive X-ray images with the task of accepting or rejecting them Side 3 https://journals.hioa.no/index.php/radopen/index The shoulder blade is frontal projected, and the proximal part of the humerus side projected. At a tube angle of approx. 10° caudal, the acromioclavicular joint becomes relatively well depicted. (Translated from Swedish by us. ) For each of the protocols, two images were presented to the readers: one considered meeting the positioning and centration criteria (“Image 1”) and one that did not (“Image 2”). All participants evaluated all four images. The images were presented using Siemens ́ image processing and reporting software SIENET MagicView 300. Participants Eight people participated (convenience sample) in the survey. Group 1: three last year radiographer students, Group 2: two radiographers with no experience of film/screen radiography, and Group 3: three radiographers with past film/screen experience. As a consequence, all those in Group 3 had more than 15 years of work experience. Group 2 and 3 were recruited from the same hospital as where the images were taken and the experiments performed. All participating students were in the final year of their bachelor in radiography education at the same university college, and had previously participated in skeletal radiology work-practice in the same hospital. Prior to the experiment, the participants were informed that they would participate in an eye tracking study to study how they examined images. They were not informed that the results would be analysed in relation to an image criteria protocol. Contrary to a normal working situation, the participants were given no information in advance about the medical justification for the images, other that they were shoulder and knee projection images that should be evaluated for radiographic imaging criteria. Eye tracking equipment and data analysis The study was conducted using the eye tracking equipment SMI ETG 2.0 (from SensoMotoric Instruments GmbH, Teltow, Germany), with the associated analyzer software packages iView ETG and BeGaze (www.smivision.com). The equipment was calibrated for each participant. In the analysis, the participants ́ field of view as recorded by the eye-tracker was partitioned into 64 segments (8x8 grid). All subsequent analyses were then based on measurements within these segments, omitting fixations outside the actual x-ray image regions (as can be seen in Figure 2 below). Based on the number of fixations and the gaze time within the whole 8x8 grid we computed Group average gaze time pr. fixation Group average percentage of fixations, and average percentage gaze times, spent looking at anatomical structures most relevant according to the criteria, relative to fixations and gaze times spent looking within the region of the X-ray image as a whole. Two-way ANOVA and Fisher Exact tests were used to check for differences between groups. Follow-up questions Following their eye tracking session, each participant was asked four follow-up questions related to the examined images. The answers were analysed by content analysis identifying and describing common statements in each group, as well as statement differences within each group. 1 Skulderbladets dorsala yta mot filmen. Överarmen abducerad 90° och utåtroterad 90°. Centralstrålen 3 cm kaudalt om akromions övre kant. Skulderbladet frontalprojiceras och proximala delen av humerus sidoprojiceras. Vid en vinkling på c:a 10° kaudalt ifrån blir akromioklavikularleden relativt väl avbildad. Dag Waaler, Sigrid Hammer, Camilla Langdalen, Linn Therese Håkonsen Haug How radiographers visually perceive X-ray images with the task of accepting or rejecting them Side 4 https://journals.hioa.no/index.php/radopen/index Ethics The study was registered and conducted as a Quality Assurance 


Introduction
One of radiographer´s roles in the imaging chain is to produce optimal images in order for the radiologist or doctor to make an accurate diagnosis.To this end an important task is to decide whether to upload a newly taken image to the PACS, or reject it and acquire a new one.Such image rejects /retakes impose challenges within radiographic imaging: they occupy unnecessary resources, expose patients to unnecessary ionizing radiation, and might indicate suboptimal quality management.A number of studies have shown that image reject /retake rates decreased substantially with the introduction of digital imaging, from 10-15 % to 3-5 % (1-4), mainly because exposure errors almost completely vanished, Later studies, however, have indicated that due to increasing occurrences of image positioning and centring errors, reject/retake rates once more have increased (5)(6)(7).One of these studies (7) reported rejects /retakes for knee and shoulder imaging to be 20,6 % and 9,4 %, respectively, in which positioning and centring errors contributed approx.80 %.This led our interest to look into how radiographers actually visually perceive the images in the process of deciding an X-ray image to be accepted or rejected.Most of the research on the perceptual and cognitive processes that underlay the diagnostic imaging processes have been from the point of view of the diagnostic decision-maker, and less from the point of the radiographer or radiology technologist (8)(9)(10).Admittedly, there are several studies where radiographers themselves act as diagnostic reporters, in for instance CT imaging (11) and conventional (planar/ skeletal/ projected) X-ray imaging (12)(13)(14)(15), but these are other types of imaging tasks than accepting/ rejecting.
The pattern of the reader´s focal movements when evaluating an image can be described as a series of focal "rest" at specific points (a "fixation") in the image, followed by a rapid movement ("saccade") to another fixation point (16).Several factors may influence eye movements in viewing a scene, including the imaging task and the prior knowledge of the viewer (17), as well as particular image properties (18).
The present paper investigates how radiographers´ focal attention "move" when examining an image during the reject/ retake decision process, quantified as the number and lengths of fixations, and if and how this is effected by their level of experience.

Images
Two different skeletal X-ray imaging protocols were chosen for this study, Bontrager´s "Lateral knee" (19) and Movin and Karlsson´s "Elevated shoulder" (20) projection imaging, respectively, chosen because they are used by the hospital and because they represent imaging procedures known to have high image rejects rates (7).Furthermore, because positioning and centring errors are expected to account for most of the rejects, we focused on radiographic criteria relating to this: -Adjust rotation of body and leg until knee is in true lateral position (femoral epicondyles directly superimposed and plane of patella perpendicular to plane of image receptor.-Flex knee 20° to 30° for lateral recumbent projection.
-Align and centre leg and knee to central ray, and to midline of table or image receptor.

Movin and Karlsson «Elevated shoulder» (20) (p 139):
-Place scapula´s dorsal surface towards the bucky ("film").Upper arm is abducted 90° and externally rotated 90°.Central beam 3 cm caudal to upper edge of acromion.For each of the protocols, two images were presented to the readers: one considered meeting the positioning and centration criteria ("Image 1") and one that did not ("Image 2").All participants evaluated all four images.
The images were presented using Siemens´ image processing and reporting software SIENET MagicView 300.

Participants
Eight people participated (convenience sample) in the survey.Group 1: three last year radiographer students, Group 2: two radiographers with no experience of film/screen radiography, and Group 3: three radiographers with past film/screen experience.As a consequence, all those in Group 3 had more than 15 years of work experience.Group 2 and 3 were recruited from the same hospital as where the images were taken and the experiments performed.All participating students were in the final year of their bachelor in radiography education at the same university college, and had previously participated in skeletal radiology work-practice in the same hospital.
Prior to the experiment, the participants were informed that they would participate in an eye tracking study to study how they examined images.They were not informed that the results would be analysed in relation to an image criteria protocol.Contrary to a normal working situation, the participants were given no information in advance about the medical justification for the images, other that they were shoulder and knee projection images that should be evaluated for radiographic imaging criteria.

Eye tracking equipment and data analysis
The study was conducted using the eye tracking equipment SMI ETG 2.0 (from SensoMotoric Instruments GmbH, Teltow, Germany), with the associated analyzer software packages iView ETG and BeGaze (www.smivision.com).The equipment was calibrated for each participant.In the analysis, the participants´ field of view as recorded by the eye-tracker was partitioned into 64 segments (8x8 grid).All subsequent analyses were then based on measurements within these segments, omitting fixations outside the actual x-ray image regions (as can be seen in Figure 2 below).
Based on the number of fixations and the gaze time within the whole 8x8 grid we computed -Group average gaze time pr.fixation -Group average percentage of fixations, and average percentage gaze times, spent looking at anatomical structures most relevant according to the criteria, relative to fixations and gaze times spent looking within the region of the X-ray image as a whole.
Two-way ANOVA and Fisher Exact tests were used to check for differences between groups.

Follow-up questions
Following their eye tracking session, each participant was asked four follow-up questions related to the examined images.The answers were analysed by content analysis identifying and describing common statements in each group, as well as statement differences within each group.

Ethics
The study was registered and conducted as a Quality Assurance Project of the hospital, and is as such not subject to informed consent from patients according to the Norwegian Patient Rights Act.The images were provided by the hospital, and as patient data had been removed, and none of the images showed any pathology or other identifiable signs, any further consent from the depicted persons was considered unnecessary.
Employees at the Radiology Department were informed about the study in advance.Access to images and systems was supervised by the Radiology Department.All radiographers and students signed written consents to participate in the study.

Results
Three examples of scan path images are shown in Figure 1.As can be imagined looking at these images, there is hard to find any systematic saccade patterns, and thus seemingly no clear strategies in how the participants searched the images.
Although the seemingly quite "random" search patterns as seen in Figure 1, however, all participants seemed to focus mostly on the most important regions of the images with regards to the radiographic imaging criteria.This is also why we collected the results in terms of the number and duration of eye fixations within predefined regions of interest (ROIs), neglecting the actual sequence of the fixations.Figure 2 gives two examples of these results, the average number of eye fixations and the average gaze time spent within these predefined regions, respectively, for one of the participating groups (Group 2).Figure2 furthermore shows that the observers also paid some of their attention to regions outside the images themselves.As our interest, however, was mainly on their eye fixations within the image regions proper, Table 1 and

Discussion Fixations
As Table 1 shows, Group 3 (radiographers with past experience in film/screen radiography) had the highest average number of fixations per.image, followed by Group 1 (students) and Group 2 (radiographers without experience in film/screen radiography).The same tendency holds for the average total fixation time, as shown in Table 2.This is only partly consistent with what is reported in the literature; studying the effect of expertise on detection and localization of pulmonary nodules in thorax images and enlarged lymph nodes in CT images, respectively, generally showed that increasing experience entails fewer fixations per image (11,12).Radiology tasks, however, are quite different from radiography tasks, and a study where radiographer students and experienced radiographers were compared on imaging tasks that are closer to the radiographer´s usual tasks found that these groups were "strikingly similar" (13).
More and longer fixations may indicate less efficient scanning and search (21,22), thus that students use more fixations and have longer total fixation times than experienced radiographers is expected.In fact it is reported that students for some tasks can use up to twice as long time as experts (14).The question, however, why the most experienced radiographers (Group 3) in our study used at least as long time as the students did is open to discussion.It is of course a bit speculative to propose, but could this in some way reflect these radiographers "analogue history", i.e. their experience from a time when they, in addition to checking for proper positioning and centration, also had to check for proper exposure?Table 3 shows that Group 2 used shorter time per fixation than did Group 1 and 3; and albeit not by much, it is statistically significant (single factor ANOVA, p = 0,046).It is known that longer fixation time might indicate difficulty in extracting information or that the object is more engaging in some way (23).This might support an assumption that senior experienced radiographers are looking for more in the images than merely to check if positioning and beam centring is properly executed.
While the intra-participant variations in Group 3 were quite small, both with respect to the number and length of fixations, the relative variations within Group 1 and 2 were up to 2-3 times larger than for Group 3. Because of possible variations in individual prior knowledge, fatigue, expectations, preconceptions, habits, etc., such variability is to be expected (8).

Task foci
As Tables 4 and 5 shows, Group 2 was more focused on those regions in the images that the criteria emphasize important than did Group 1 and 3, both with regard to the number of fixations (single factor ANOVA, p = 0,001) and total fixation time (single factor ANOVA, p = 0,002).To our knowledge, similar results have not been reported in the literature.
The follow-up questions (Table 6) revealed that all participants across work experience answered questions 1 and 2 (almost) similarly.We interpret this to imply that all participants, regardless of their level of experience, focused on meeting the recommended image criteria, but also felt free to execute the assessment differently according to the medical justification and the imaging task.
For questions 3 and 4, however, we obtained slight differences between the groups, and partly also within the groups.The students were found to be the most homogeneous group regarding these questions in that they all exclusively commented on the radiographer´s tasks, i.e. if the positioning and centring were according to the criteria, and without (mentioning) any considerations of pathology.It is tempting to believe that an explanation for this is that they all are students of the same class, all having image criteria theory relatively fresh in memory.
One participant in Group 2 and two in Group 3, however, (erroneously) pointed out fracture indications in some of the images, and thus unsolicited revealed that they were also concerned with (potential) pathology.Admittedly, the differences between the three groups, computed as fractions pointing to pathology, were not statistically significant (two-tailed Fisher exact, p = 0,18).It supports, however, the findings discussed above, in that experienced radiographers also assess image quality by checking whether possible pathology emerges in an adequate manner, which however, is not their job.

Study limitations
Because of the relatively small amount and specific category of images used in this study, the results certainly is no final proof for how radiographers view and evaluate X-ray images in general.Yet we identified certain trends in how experience affects image quality considerations that may be general.Based on the results reported by others some of these trends were expected, while some of them are novel and unexpected ones.Furthermore, as the number of observers in each group were quite small, and recruited by convenience, the probabilistic uncertainties thus are quite large.Yet we found some statistically significant differences between the experience groups.

Conclusions
In terms of qualitative inspection of the scan path images, radiographers seem to lack common ways of how to visually examine X-ray images before deciding to accept or reject them.As this is not part of their curricula, however, it is not to be expected from them?Each seem to have their own way of doing this.
In terms of quantitative parameters, i.e. total number of fixations, accumulated fixation duration, average single fixation duration, and percentages of fixations and fixation durations spent on the most relevant anatomical structures, the larger differences were found between Groups 1 / 3, and Group 2. Group 2 had significantly less number of fixations and less average total fixation durations than the two other groups did, whereas Group 1 and 3 scored almost equally.
Even with few participants, this study has indicated that work experience might have some subtle correlation with in what way radiographers and radiography students assess X-ray image quality for acceptance /rejection.Although with quite large individual differences across experience, however, there are indications that more experienced radiographers are looking for more, and other, image quality features than solely those recommended by the radiographic imaging criteria.To know whether these are purely random or depend on some deeper explanation variables needs further research.

Fig. 1 .
Fig. 1.Scan path images of "Shoulder image 2" for Group 1, 2, and 3, respectively.Different colours represent different participants within the groups.The circle centres indicate the fixation points, the circle diameters represent fixation times, and the lines between them are the saccades.

Fig. 2 a
Fig. 2 a) Average number of eye fixations within predefined regions for Group 2 in their examination of "Knee image 1", b) Average gaze time (ms) spent within predefined regions by Group 2 when examining "Shoulder image 2".X-ray image regions are indicated by the red rectangles.Note that the participants also looked outside these regions.

Table 6 :
Responses to the follow-up questions for each group.
2 lists the average number of eye fixations and the average gaze Side 6 https://journals.hioa.no/index.php/radopen/index