Developing a New Medical Augmented Reality System

Developing a New Medical Augmented Reality System Frederick Morgan CMU-RI-TR-96- 19

Thc Robotics Instilute Carnegie Mellon University 5000 Forbes Avenue Pittsburgh. Pennsylvania 15213 15 May 1996 0 1996 Carnegie Mellon Lniversity

Acknowledgments

I would like to express m y fratitude to: Mike Blackwell, David Simvn. Bob O'Toole. Branko Jarnmaz. Lori Gregor, Drew Shefman. Takeo Kanade, LanFham Gleason and Debbie Morpn

i

Abstract Augmented reality i s a technique for combining supplcnlerltal imagery cuch t1131 it :ippt;ii-% ;LS part of the sccne and can he used for guidance, [raining; and Io~ationalaids. In tht meclic:il domain. augmented reality can be uscd to corrthine medical imagery to the phyrici:rn’.: v i m (11 a patient to h e l p the physician cstahlish a direct relation betwren the irn;igt.ry i ~ n dlllz p i i t k ~ i l . This pi-oject report will examine medical augmented reality systems for IISC iii ii .:ui-:ic:il wting. Four areas will be examined: ( 1 ) applications for augmented reality syrtzm In inciliciiic. ( 2 ) survey o f baqic technoloFies used for building augmented reality systems i n c l u r l i i y Ilk current state of the. art in medical augmentcd reality sytcrns. ( 3 ) the devclnpment of ;I ilrii augmented reality system and (4) tcsting and validating an augmented reality system for c l i n cnl use. The goal of this project report i s to dewlop ;I new design of a mediciil aiigmeiited t r ~ x ity sysiem; the design will draw upon a number o f different technologies in iin irttcinpt i o huilrl ;I mort: practical and capable system.

Key Words: Augmented Reality, Compirter Assisted Surgery, Rcgi5irntion. Triicki 11.; Devices; Image Overlay, Surgical Guidance, Tclcmcdicine.

..

11

llisplq

Table of Contents I Augmented Reality .................... .............................................................. What is Augmented Reality? ............................................................................................ I. I

I .2

Potenti;il Uses for A u p e n t e d Reality in Medicine.....................

I I

-7

.......................................................................... -3 2 Augmented Reality Systems 2.1 Basic Augmcntcd Rea . ino ogles ........................................................................... I 2.1.1 Registration ............................................................................................................... 4 2.1.2 Display ...................................................................................................................... 6 2.1.3 Tracking ............... ............7 2.2. Types of Augmented Reality Systems .............................................................................. S 2.2.1 Heads Up Displays/Hedd Mounted Displays ............................................................. 2.2.2. “Magic” winduw and planar displays .............. 2.2.3 Surgical Microscope Augmented Reality Syst II 2.3 Current Statc of the Art Systems ..................................................................................... 2.4 Litnitations o f Current Systems I ?.

3 The ?4RCAS Augmented Reality System .................................................................................. 3 . I MRCAS Augmented Reality Systcm - Ovcrvicw ........................................................... 3,2 Selecting Basic Tcchnologies for the Clinical MRCAS Systern ..................................... . . 3.2.1 Sh-coscnpic Displays ............................................................................................. 3.2.2 Displays Devices ..................................................................................................... 3.2.3 Registralion ............................................................................................................. 3.2.4 PalicntlSurgcon Tracking ........ ......................................................................... 3.3

The Clinical MRCAS Augmented Reality System

I4

II li l i

I7 71 I -? -

7;

4 Validating a n Augmented Reality System .................................................................................. 25 4.1 2.D Vs. 3D Performance Experiment .............................................................................. 25 4.2 3D Rcconstruction Vs. Slices in Volumetric Data .......................................................... 1s 4.3 Accuracy Test .................................................................................................................. 7‘) d.4 Ciidmer Study ........................................... .39 5 Furwe Woi-k and Conclusions..................................................................................................... 20 5 . I Future Work .................................................................................................................... 19 5.2 Conclusions ..................................................................................................................... 30

...

Ill

List of Figures Figurc 1-1 : Figure 1-2: Figure 2-1: Figure 2-2: Figure 2-3: Figure 2-3: Figure 2-5: Figure 2-6: Figure 2-7: Figurc 2-8: Figure 2-9: Figure 3- 1 : Figui-e 3-2: Figui-e 3-3: Figui-e 3-4: Figure 3-5: Figure 4- 1: Figure 4-2: Figui-e 4-3: Figure 4-4: Figure 4-5:

View through an augmented reality system ..................................................... s Remote assistance with an augmented reality system. The Registration process ........... Template based registration. .......................................................................... I I A polarization based planar stereoscopic display system .. Effects of tracking ..................... The tracking process ...................................................................................... I4 Heads Up Display/Head Mounted Display ..................... The magic window display ....................................... Background motion with window displays ......... Schematic of an augmented reality surgical microscopc. The MRCAS augmented reality prototype syste 22 Activc shutter glasses ....................................................... 71 The effect of the screen geometry on [he virtual image produccd. ._ Screen curvature distorting an object i n the 2 direction. ............................... 25 Curvature and refraction distortion. ............................................................... 25 A 2D augmented reality system ..................................................................... ;Z ......... .> 1 The MRCAS augmcntcd reality system ... Results bctwccn a 2D and 3D augmented reality system. ............................. :.3 . . The stereoscopic projections ......................................................................... 34 34 The effect or using the wrong IPD in the stereo projections. . 1 1

List of Tables Tiiblc 3-1: Table 3-2: ‘Table 3-3: Tahle 3-4:

7:. .. , , ..Different ways of displaying stereoscopic imagery ......................................... 36 A cumparison of diffcrent display devices Registration rncthods ....................................................................................... 38 30 A ccirnparison of tracking systems ......................................

1 Augmented Reality Augmented reality is a technique for combining supple.mental imagery that ;tpprnrs 10 hc part of [IICscciw used for guidance. training. and locational aids to the user. The supplemental imagzry ciin bc i t s simplc it'i a r r o w offering directional guidance to the user; or as complicated as a three dimensinnal i i i d ~ d l [Iic scene. This projecl report will examine a nuiiiber of d rent techniques for buildin:. leslin: ilnd vidid;i~iii; a medical augmented reality system for surgical USC. The medical domain can benefit greatly from augmented reality. Adviinced imaging Izchniquc\. w c h i i \ Magnetic Resonance Imaging (MRI), or Computerized Axial Tomography (CT or CAT). o l l r ti ~ ~ v - i n v ~ i sive look inside a patient. These imaging techniques can be combinrd w i t h augmented reality IC! givc i t p l y sician the ability look beneath the surface of their patient's skin to the anatomical structuirr helwv. I n addition, the imager). can be used to offer supple.mentn1localizalion, guidance and Iraininy itidcs dririily w r gcr).. Augmented I-ealityhas the potential of greatly reducing surgical timcs. while irnpro\~iiigo\m111 patictit otitconies. While there are many used for augmentcd reality in medicine, this project r q m T will focus on augmented reality system for use in a surgical setting.

1.1 What is Augmented Reality? In the realm of medicine. augmented reality is a display technique that offers guidance :uid physician in locating anatomical structures on the patient. Augmented reality combines sulipleiiiciit;iI i n agery lo the physicians view of the patient (Figure 1 - 1 ) . Traditional methods used by phq"ici;ins t n vic\v medical imagery include: films on light tables, imagery on a workstation monitor. etc. Howew-. it iiiLijor drawback with traditional approaches is the lack of a direct relationship between the patient and Ihc iinaycry. When tlie imagery is used to locate anatomical structures during surgeiy the sui-geon view:. tlic s t n w r i m in rhc imagery and then tries to locate it on the patient. To aid in the localization of [he m~itmiiciil s[riichtres, the surgeon is often required to mentally transform the imagery to that nf the smie (>riciit:itiim and poqirion a s the patient. Augmented reality provides to the physician a dire.ct spatial relationship between imagery atid piilieiil. Through the use of augmented reality, a physician can view the imagery while at the same time viewing 1 1 1 ~ patient. The imagery is overlaid on the patient to appear in the eract orie.nration 2nd position ds the umcrpnnding anatomical structures. The ability to view the imagery in the correct position and oriciitiitioii while vicwing the patient allows the physician to visualize internal structures in proper pnsition over thc piiticiit, The dit-ect relationship between the imagery and the patient can be beneficial to 21 surgeon. During ii mrgiciil procedure it is orten ncccssar). to locate structures on the patient that h a w been vicwcd in h c iiiia:cvy. Without tlie use of augmented reality, a physician may have a difficult time of locating various suhciitiiiicous structures. For example, in neurosurgery, when a surgeon locates a brain lesion (tuiiini-)i i i ii CT nt-hlR1 scan. they might have a difficult time visualizing the exact location of the tumor in the pitient bcihrc s w gery. A substantial amount of time is spent localizing the tumor on the patient usin; a tr~ditionalimcthotl 0 1 viewing tlic imagery, such aresented by an augmented reality system. Without the remote assistance of the expert sur:con. the palie111 would have to he flown to the expert's hospital for the case to be performed. Hou.cvcr, if ;in expert c u i assist on Ihc case remotely by using an augmented reality system. the cost of trmspcrrtinf l l ~ epatient ca11h e >avc the student during surgep with an augmented reality system. AI the same time. the student‘\ surficiil i i i s l i - i i inents can be tracked to determine if they are locared within the safe region. If thc iiisIr~~mcn~s arc milvide ofthe safe region. the system can notify the student of the potenrial problem. The ability lo c:isil! \:ic\v Ilic safe and dangerous regions during surgery can potentially aid the student in learning thew hriund:iric> r ~ w r than by traditional methods. ii

I n all of the above augmented reality applications, it is critical that the ovcrlaid irnafe.; he ; ~ c c i i ~ ; iIpmI~~l~ tioned i n the scene, If the imagery being presented to the surgeon i s not correctly positioned. iniicciiriilc i l l formation will he conveyed. In the following chapters. the problem of pro\.iding accuwtc ovcrliiys t o ~ l i c surgeon will be addressed by examining a number of different technologies iricludinf: p;iticnt~iina:rr). rL~yistration, display techniques and patient-surgeon tracking.

2 Augmented Reality Systems All augmented reality syste.ms have to solve some basic technological hurdles in ordcr l o hc i i s c ~ ~i ni li i ri'iil wwld applicalion. Thcsc technological hurdles include: image-patient registration. type of dispkiy d e v i c e \ and patient surgeon tracking. The success of the augmented reality systems ielies on srlrhtiii:~t h e I n s i c teclinologies that solve these hurdles in a way that make the system safe and easy tn use. 4 iiiiiiilic~oI(li1fei-ent display techniques can he used when building augmented reality systeinh. A f w ot'tlie m t c of tlic xi medical ougmcnrcd reality systems. each utilizing a different display technique. will be evalti;itcd lor L+

3

fectiveness orconveying supplemenral imagery to the user and their ease of use in ;I mrdic;il s e t l i t i ? .

2.1 Basic Augmented Reality Technologies 2.1. I Registration The direct relaiion hetwccn imagery and the world' provided by :in au~mentedreality system i s I ~ Y iu I tlic overlaid imagery is not correctly aligned to the world. The process of tiligning the overlaid itiiiigcry i\ iitllcil regirtration and is composed o f several parts. First, the anatomical structure o f intcrcst i\ sciitnicd. i ~ n d relation between thc imagery and a 3-dimensional modal i s created. A registration sensor i s then uszd to collcct data frnm the anatomical structure o f interest to be used for the registration process (F'igui-e 2-1 t . T l ~ c registration process relates the data from the registration sensor tn that nf the model tn find J t t - ~ i n s f o r ~ i ~ ~ ~ t i o n between the anatomy and the overlaid imagev. When the imagery i s adjusted by this tr~nsfnrmntion.it hecomes correctly iiligned to the anatomy. Two of the most frequently used techniques to pcrform rc of the object being tracked. If the markers move relative to one another and the motion is not accountcd ~ ( I I - . trncking errors will be introduced

World Object

Correctly registered wurld object and overlay image

Obscrvcr'F u;int;isc l x i i i i t translated t o the triglit

Overlay Image

Obscrver and world object tracked to maintain correct registration

Ob,ject in world iiinvcil .Away from obsci-vcr

lo Figure 2-4: (a) The world object and the overlay image. chj The overlay objcct correctly i-cyisk~t-cd world ob-ject and tracked regardless of head or object motion (c) The cl'l'ccts o l n o l t i - x k i i i c thl, object or vantage point movements. Both types of motions will causc the uverliiid iiniapery I O iio longer be registered to the world.

Optical trackers work by sensing tlie position of rigidly fixed markers usin:_.cilhcr infrared 0 1 v i s i l i c lislit sensors. The relation hctwcen these markers is known. thus allowing thc calculntinn of tlie pipsirion and oricn~atiotiof tlie markers (a minimrim o f 3 markers is required to obtain the 6 degrees of fi-rrdom (L)OF) d f the system). Magnetic systems work by artificially creating thiee orthogonal magiietic fields. A I-cccivcr i \ placcd in this magnetic field with three orthogonal pickup coils. By sensing 'hc vo1laF.c iici-o+ itic I l i r c c coils. it is possible to determine the position and orientation ol' the dcvice. I!ltriiwiic sctimr\ arc vcIy s i n ilar to optical oneq. except that they use sound instead of light. Three microphones inexsrirc l l i c time i l tahL~s for sound to travel from the transmitter (spark p p j to each of the rnicrophunes. The position o f thz 11:1n\mitter is derived from the time required for thc sound to reach cach micrnphnnc via triaiigula~ii~~i. A iiuiiitiu of differrnl range sensors arc currently being developed. Range sensors retiirii the distnnce t o ciicti ~ p i > i i i iti i the scene. a depth map, as opposed to the intensity iis a regular ciimera does. The depth map is used \villi :I surface based registration approach to find the position and nrientatinn of the iihject. Radio ft-cquerrcy triicksrs are very similar to optical systems. The system locates the nhject i n space by triangtilating the tr,iiisinitter's location relativc to a number of receivers.

I"

xJ

Figure 2-5: The tracking process. The registration process (grayed out area) from Section 2. I . I toiind tlic initial position of the anatomy. The coordinate system the iregistration seiisor used is ciiiihratc t o the display device to obtain the relation between the sensor and the display. Trackin? i h t l i c n i i d to measure the position of the surgeon's vantage point and the anatomy. Any cliangc i n positiiiii from the initial starting point is then used to update the registratinn betwceii thc o ~ c r l ; i y siiiid iliz anatomy.

2.2 S p e s of Augmented Reality Systems Augmented reality systems can be built using a number of different display techniques. Exli techniques l i a h its advanrages over the other systems. Three of the most commonly discussed s y s t e m for use iii ~ncdi~iilic include heads tip displays/head mounted displays. "Magic" windows, and iiiicroscnpe ovrrlay syrtein.;.

2.2.1 Heads Up DisplaydHead Mounted Displays Hcad Mountcd Display (HMD) and Head's Up Display' (HLD) consist of an displiiy dt.\zice mi)uiitcd w i t l i I .HUD in this cilsc rcfcrs

t~ a

hcad warn system and not a system that presents the iiu;igsry ~ l i l i r i d 13~ 1 :I ~

headset such as system foiiiid in some aircraft cockpit,

8

i n n headset that presents imagery directly to thcuser’s eyes. HMDs only allow the tisei-Io view tlic dilificd I)! no in: t o track the surgeon's head; instead, the micrnscope itself is tracked. The stirgcon's v:ii microscope's view of the world because the microscope constrains what the wi-gcrin 5 hxed systems also allow for a second wrzeon to view the augmented environnie~~l hy [hc :id(Iitioii ~11:i \cc'oiid ocuI;ir to the scope. Addins a secnnd user to the system i s somelimes iinportirnt to :iIlo\v wr:con\ ;mI lhcir assistants to have a common view of the supplementnl imagery.

Bcnm Splirtcr

%&F i p r t : 2-9: Schcmatic of an augmented reality surgical microscope Surgical microscope augmented I-eality systems do have a few drawbacks. The IIW 01' I l i i c r ( l \ c o I x :iIIqIiicIIic d rcality i s limited to cases where ;I microscope is avail;rblc. Surgical microscopcs a s o prc'retlt i i \ ~ n i l l ll'iclcl of vie\\; (apprnximately l O c m X 10cm); for some surgical procedures, i t m;iy he desii-iildc to h a w i i imuc.11 larger f i e l d of view. 11) addition, a sur:eon's peripheral view d the su~soundingopcm[iiig romri i\ liiiiil with a views o f the patient's head to assist in localiring tlie site o f a craniotomy iopcnirig ( l i e skull i t 1 cnposc the hrain for tumor removal). The system they have.developed coiisists ofthe lollowing p i i i l s : ii i i i i i i CKI looking down the sight lines o f the surgeon (fur colleclion of the !xi>rld image'). I a v x I-iinyc hndcr (iixcil for I-egistralion arid tracking), and a monitor (Tor displaying the world and nverlaid i m q c r y ) . The sroi~p II;I\ dcvorcd a significant amount of time to thc dc\.elopinent o f automatic registration and patient trachiiis t z i l i i1iquc.s. The 3D range finder is used to collect a three-dimensional depth map 0 1 tlir p i i i t v i l . Tlic dL.pih iniip i s matched to n model of the patient created from the imafery by means of surfiicc hassd registr:tti(>ii, The currcnt system has been used on iit least 8 different neurolofical c3ses at Brighiurn and W o m c n ' x f 1 t i s p i ~ ; i l i n Boston, MA. While using the system. they have repoited a great reduction in wrgical t i i i i c iiiicl ;III iiiiproved accuracy in the placement of thc incision [9].

Davey et 31. have proposed a similar system to that of Grimson's gruup [SI.The iinjor cliTcrcncc IX,I\\ their work and that of Grimson's i s in their application. Davey's gruup i h usin; lhc sysiciii hi d~:lciininei t there are any changes in the anatomy between the time the MRI, CT or DSA \viis tahcii iind the lprtwiit. TIic registi-ation process between the image and the patient i s curently done by hand They propox I l i c 1 ~ 01 ' ;I stereoscopic cameia based system and surface based registration to ohtain i'astei-. mor-e xuiriiic iin:i;epatient registration. but have nut yet implerncntctl i t . The Davcy system i s also integ~te?il \ v i l l i iiii IS(; \.ic\vin$ wand (ISG Technologies. Mississaoga, Canada) to assist i n intra-surgical Ioca1iz;itioii a i d y u i d ; i ~ ~ w 'l'lic . 1SG viewing wand i s a commerciiil system that ilIlo\vs a surgeon to point to 311 miitoiiiiciil Ic.atu~-coii the piitient and then view this specific feature in the imagery on a workstation. By coiiihininf the ISC; v i c \ v i n s \vartd with their system, additional guidance and surgical cues can be obtained snd ;in i i i t t i i t i w i i i k r l x c 1~5r the surgeon i s provided 10 a s s i s t in the detection of anatomical changes between wlicn tlic ini:ipi-y \VV;IX i i i quit-ed and the present.

The surgical microscope i s currently the most conunonly enplored technique fur niedic:il a u ~ n i e n r c d~ c ~ ; i I i i y .;?.>terns.A number o f groups are working on developing complete microscope systems [ 2 0 . 71.I n ;idilitiiiii. :I commercial system has been developed [ 2 7 ] .Roberts' group from Dartinoutli W;IF one ofihc h i 10 5t:iri developing and clinically using an augmented reality system hascd on a surgical microscope. Tli,. cllii-clli system consists of a stereoscopic microscope. a sonic di:itizcr. a mininiiirc CRT arid I ~ i m y p l imi-toi. iniiic'~ fusion The gi-oup hiis been developing a low cost. accnratc sonic digitirer to a l h v fix c;i\y trackitic 01111c rnicroscope and fiducial localization. The complete system hns been u c d ill ;ipprvniniiiicly 3 1 C;IW I\ i l l 1 I-eliitivcly good results 1181.

Edwards et ill. have been developing a system that i s very similar to tlie Roberrs surgici niicr+cc~l~e171. Both systems currently only present simple contour outlines :ind giiid:ince cues :IS owri:iici iiiiayc,t-y. IIIVC ~ thczc i\siies including: stcrcoscopic display techniques. display deviccr; registration (fidiici;il h a w s . sIi;ipc

hased; plus a new novel template based system) and finally patientlsurgeon trackin;.

3.2.1 Stereoscopic Displays Thc MRCAS systcm currently uses active shutter glasses to produce the stereoscopic ovcrl;iys. TIK ;iilivc shurter glasses (CrystalEycs, Stcreo Graphics Inc.; Sari Rafael CA) run thc display ;it t\vicl. i l s irLyilar Ci-uiic 1-ate.The increased Crame rate is uscd to scquentially display the correct ski-eo image to eiicli i.xi.\ w l i i l c attempting tn eliminate image flicker. The user wears a pair of active sliuttei- ;lnsscs ~ y n c l i r i m i z ~\villi d 1111. inoiiitot- to occlude one o f the eyes while the other eye v i e w i t s pait o f t l i r stereoscopic i i i x y c I I:i:ilrc 7 - 2 ).

For active shutter glasses to work without producing ii flickering display. the displiiy d w i c c uscil i i i i i z i hc capable of running a[ the increased refresh rate. Most high definition monitors and \:ideo projcctoi- it v i i - t u t i i image that is tlic same shape B S the display surface. Therefore. ;I flat display produces a flat \:irtual iin;i:_.cmid a uii-vt,il dis-

17

pl;iy w i l l pi-oduce a curved virtual image (Figure 3-3). The

im;lfc will be Inc;itctl ii di%tmcL, l~\~luii tlie beam splitter equal to the distance the display is above the beam splitter'. Thc nwi- \ ic\\:x Ilk. \IIIII:II im;ife. 5 0 the shape and posilion of the virtual image cuntrolr where a user will sce the nvct-l;iy.'rIlc pro1~1c111 W i t h a curved vinitiil image is that it rvill cause the overlay to be distorted :tiid hc iiiciii-rccil! ~pI;ic.cd (Fifiire 7-4). Both the distortion and the misplacement c;m he crirrccletl fni- i n t h e Liti'i-m prnsl>cctivc l?roicctioii. but this grcatly complicates the stereo projections because a frometi-ic iiintlel n l t l i c cIiq>l;iy \iirfiicc iiiusi bc intrnduced. To reduce the rendcring time for the overlaid inxigel-y. the htei-Lx) ]mi\]icctivc Ilni,icctinlib should he as simple as possible. \zirtiial

Virtual Imqr

Fistire 3-3: The effect of the scrren geometry on the virtual image prnduced

Thc 21'' CK'T monitor used in the prototype systems had a center point 12.0 i i i i i i hifhci- tlicti tlic cil;cs (Figtti-e 3-5a). To keep the stereo projections simple for the prototype system. the ovei-Iiiys arc r c r i i l c I c d oil ;I flat im;ige planc located half way between the center and edge points, causing the :icltiiil wetday to 112 i i i i ? ~ rcpistcr hy as much as 6.0 mm. In addition, the curved screen shifts the locntion? of 11ic iiiicisc w c i i i i \ ii function of the view point due to purallnx (Figure 3-Saj. Finally. Iargc ciirvcd CRT d i y k i y s \\:ill iiilroiliicc Thc rcfraclioti i n the g l ; iiiid ~ [ l i e di.,pl:iy reftaction distortion from the thick front face plate (Fignre 3%). ctirvature tend to distort the image in upposite dirsctions but do not cancel out. Tlic problmi witli IIic\c cli,,tnitinns i y that they are \:iewpoitit dependent and non-linear making them more diF('iciilt10 coi-ir~~zit 1 1 1 i i i 1111. ytereo pro,jections [ h i . 411 of these effects drive for the use of a flat panel dispkiy for the n e l t Scnzriitioli system Finally, the purtahility of the display should be considered when selectin: a display devicc. I t is Ipiissililc 10 Find a display that fulfills all nf the above requirements. but is tno big and :iwkw:iId to easily ititc:iuk i i i l i > the ciirrrnl system. Thc dcsired working volume of the new system is approxim;itcly 1 1 . 3 i i i ' I I i i i S Il..iiii X (1.5m). If thc system is much larger than this, it will be difficult t o use it in an opcrntinf r!ii?ii> wtliii: d i i c IO tlic cotifincd nature of the nperatinf room. 4 iiumher of displays were e\:aluated against the preceding criteria (Table 3-2). The display clcviccs iiiclude: Liquid Crystal Displays (LCD) and projecturs. plasma displays. Electi-olumiiiesceiit ( E L ) displiiyh. high resolution CRT monitors and projectors, Super High Definition Display Television tSHITl'V:i. ;iiiL1i i t l i er novel displays including the Texas Instruments micro rnirriir display [2 I I :ind :I dil'fractir,ii grahii; d i % pl;1yll 1.

I.Becausc tlie virtual image cxactly mirrors Ihe display device and it locatinn is ii ltiiictioii t > t ' n c i c the ienl image originates. volumetric display can also be used to present 3-dcnictiii;il i i v c d c i > imagery.

Virtual Image

& $ $ &

Figure 3-4: a) A flat screen will produce a flat un-distorted virtual itnnEe. bj A ctii-ved s c r w i i w i l l IprciJucc ;I curved virtual irnngc that has been distorted by the screen curvature miihitiy i c no Ionyet- rcsistcito object in the Z direcrion.

it Curvature Error

Center point IZtiini different from edge point

Figure 3-3: a j Curuature distortion. Point P is the pixel illuminated and point P' is the percri\wl pisel. T l i i h distortion u:ill cause the image to appear reduced. b j Refraction distortion. Point P is rlic IpixcI illuminated and point P' is the perceived pixel illuminated. This distortion u~illc;iiise tlic iitiiigc 10 appear magnified. These two distortions have opposite effect but do not cancel out. Tlic iIistniiioii< we ii funckion ofthe cyc position [6].

I r a polarized singlc sourcc stereoscopic system can he found that does not sigtiificiltitly darkcn tlic c n v i i - o i l tilent (Section 3.?.I),then an LCD projection system \vi11 be used for the next feneration 01 t l i MRCAS ~ system. LCD pro-lectors are capable of producing very bright imagery ;it high resolution Crom :i I'Iiit sit-ceii and are available i n a smaller physical volume than CRT bnsrd pmjectiun systems. If xi x1eqii;itc l i i i l i i r i m tinti hasrd stereoscopic method can not be found or developed and the active shuttet ;I:isws i i i u s t h e i i , d a high resolution CRT projection system will be used for the display device. The CRT projection syskcni aic capablc uf handling the. increased frame rakc rcquired with a c h e shurkr : l a s s o . Like the LC'U lprojec,tifi-iilion rcquircs high resolutior scans around the fiducial marks to allo\v for iicciiriitc miii Iocilli/iitiivl i n 1111' imagery, Shape b x e d registration. however, requires a high resolutinn surface m o d c l ivIicrcvc~i-tlir ii\L1i inislit collect points on the anatomiciil structurc. The ternplate based apprnach can l>c i i ~ i i s t r i i i i i ~>o i l tliiil the template will only makc contact with the patient where tlie scan data i s present. Such ;III ;iplmxich h L ~ 5 not place rccliiiremenls on the scan resolution and spacing suck as is t-equired with the sh:ipc ;iiid t t i l i i i , i i i l based iipproach.

Nit turns ut11that the required accuracy i s not obtainable with template based re&r:ition, tlii'ti sh:ipc l>as1'd rcgistr;ilion will examined further. Shape based i s more desirable than fiducinl bawd bcc3usc of t h e ficccloni iii point selection and not hauing to attach markers to the patient before the sciiii i s pet-formed. T h c iidJitiiiii of marks introduces additional trauma from the marks, and an extra scan iiz w e l l :is cos tu tlic ciisc. H o w ever. the reliability of the shape based approach is currcntly uncertain for usc ill incdiciil iloiiliiiii. II_it ~ I I ~ out that shape b o d registration docs not have the required accuracy in the mediciil d o m i l i n IIICII tlic I'icliiiiiil appt-uach will hiive to be used bccause of thc higher accuracy obtainablc.

I.Thc inter slicc spxcing is ths dihlance b c l u c c n the discs that crmipiise tlie vnliiinetr-ic h t ; i WI. 1 . ~ 1 ri i i i i r t CT systcnis this can hc as small iis lmin between slices. Typical CT protocols t i x vihLi;iliutiim c w r c n i l ? LIW 5 - I [)inin inter slice spacing.

21

I I ~

I