0022-1554/79/2701-0128$02.00/0 THE

JOURNAL

Copyright

OF HISTOCHEMISTRY

AND

Iterative

Vol.

CYTOCHEMISTRY

© 1979 by The Hiatochemical

Society,

Image

1, pp.

128-135,

Printed

Transformations

for an Automatic Smears FERNAND

Centre

27, No.

Inc.

de Morphologie

Mathematique,

Ecole

Received

for

Screening

1979

in U.S.A.

of Cervical

MEYER des

Mines

de

publication

June

Paris, 19,

77305

Fontainebleau,

France

1978

The new generation of image analysis systems permits the use of iterative image transformations. It is now possible to construct algorithms where the elementary steps are not arithmetic operations but image transformations. This will be illustrated by two examples. In the first, the absorption image of Feulgen Stained nuclei is processed by contrast algorithms in order to detect suspect cells. In the second, free lying cells are separated from overlapping cells and other artefacts by the use of skeletonization procedures.

Through the presentation ofa program ofcervical screening we want to introduce the reader to a new methodology pattern recognition. The whole process is done automatically by

a real

time

T.V.

image

analysis

system

described

defmed; the eroded set is the unburnt part of at time t = X, (in gray in Fig. 1A). Dilating a the same thing as eroding its complementary If we want to restore the eroded set A’ to

of

elsewhere

we can

(13). The contrast of the nuclei in the absorption image of a Feulgen stained slide is used for the detection of suspect cells and the elimination of many types of artefacts. At a higher resolution the fluorescence image gives a mask for the nuclei; shape analyzing of them eliminates the artefacts and automatically

separates

method,

no

the

clustering

measurements

cells.

are

made

With

in the

regard

to

whole

present But

the

in a field at the main interest

same time lies in the

basic transformations. Since they hardware, high speed and processing And only under that very condition be of any practical interest.

and not simplicity

one

the

agnosed,

comes

details

about

(5)).

material,

The

stained staining

from

the

all

Dr.

apparatus

analysis

system

scribed

in (14).

we with

after of the

other.

We

will

introduce

morphology

an Acriflavine double staining group

in

is bit

can

the

classical

planes,

Feulgenand di-

Leiden

be

2, 6-10,

12,

13).

Imagine

that

to analyze

is composed

background Suppose

of the figure a fire is started

the

boundary.

the

middle

the

fire

The

of dry

fire

of the

at times

important

texture

extensively

figure. t

=

interior

grass

is composed simultaneously will

propagate Figure

0, 2, 4. The

tools but

the

size

used

rigorous

These

points points

then of

skeleton

eliminating

of the

and

that

the

at uniform

erosion

figure

of unburnable at all

1A shows of size

the

(see we

wet points

have

speed

toward lines

X is then

will

or grass. along

front

of

of

128

size

followed

comparison

size

X works

of like

a

called

skeleton

constant.

intersect fronts

points

skeleton

advancing

the

fire

center

of

an

velocity can

the

thus

skeleton

is less than the branches

application

than and

of

the

a

the

from each

fire;

figure.

lines

will

circle.

the If the

describe

radius

the

in Figure the

of the

of

decreasing

We

from

figure

the fire front will extinguish

quench the

the

at

original

until

For

the

Figure

1

1D. of the

define all

fire

is higher

a new

branches

than

1

transformation along

which

the

a given value k > 1. In Figure of the skeleton along which 4. The each

opening

skeleton of a connected of its components marks binary In this to

contrast

images but in last section we analysis

in

gray tone images. A gray tone image may be represented a chain of mountains, where the highest points stand for

as the

darkest

the

gray tains

easily

see

erosion

The

(Fig. 1C). from the

a convex part of the original figure. Until now we have dealt only with reality we meet only gray tone images.

also

exterior

will two

is represented the

by

An

opening. opening

continuously

extinguished

is not

1C).

aptly called a skeleton. The fire middle of the figure with uniform points the advancing line of fire from

defines

circles

but

an

remain extracts

are

is a circle, is

(Fig.

the

boundary and the

propagation velocity 1E we retained only

de-

in mathe-

way

is

that

the velocity was higher figure is now disconnected,

in an intuitive

A”.

set to

X. The small particles passing through the sieve The big particles are cut into two parts: small through the sieve vanish and only the coarsest

quench

Along

in (1)

Leitz

more

fire the

(more

found

same shows

of the region

concentric

technique

several

of

figure

METhODS

matical

of A”

one region some other set

Ploem’s

use

eight

A and

stick-figure representation propagates toward the speed; however, at some

can easily be realized in power can be combined. can automated cytology

in SITS

preparation

a dilation

sets

we get

parts not passing through The next transformation

MATERIAL Cytological Stilbenisothiocyanate

by

it and

sieve of mesh are eliminated. parts passing

program.

The processing is made solely by iterative image transformations. The absence of measurement permits us to process objects another.

dilate

the original set amounts set. its original

points

tone and

image. peaks

and

the

The may

represent

contrast

discrimination be done with

peaks

between a topper

zones

smooth (hat).

in

mounIf it is

AUTOMATED

CYTOLOGY

AND

MATHEMATICAL

possible

A

top

to force

of

the

2. If we have

hat

detected

on the

mountain

in such

h2

a, by the

of contrast

when

there

h1 + h such as: threshold something

=

upper

details

Low

may

be found

Resolution

exist

that

two

is detected

h is burst); b, the lower threshold of size R (The peak enters into

More

a way

the

is

a peak

h1 and

height opening

hat

burst, then the mountain was a peak, it was not. This may be clearly understood in Figure come back to the gray tone image, it means that we

otherwise

olds

129

MORPHOLOGY

thresh-

(the

hat

of

disappears by an the hat of radius R).

in (10).

Detection

of Suspect

B

Cells

in Cervical

Smears

An

intensive

use

of the

top

hat

transformation

is outlined

here. Description Figures

of

3A

and

they two

appear successive

on

field

of 250

criteria see

light

3

Now

we

slide: normal

2

The

staining

With

some

have

seen

by

D

4A

a leukocyte,

an

and

shows The

may

thus

be

and

erosion.

B, Grassfire

D, skeleton.

and

dilation.

E, conditional

C,

they

are

represented no suspect

the the

or

dust

of on a

or

carcinoma

a

staining

with

reduced cells in a 4C will be

in situ

corresponding detection

cells

a narrow by

B detects leukocytes

eliminated.

Finally

carcinoma the results of

and

alarms

in Figures cells were

3A and present,

between

detected

On

hat

the

C,

only

The and

6A

and the

next

can dys-

section

preliminary

show fields.

that The

comparison cells

and

contrary,

in situ these

themselves

top

remain. in reality.

6B) permit intermediate

positive.

the carcinoma leukocytes. Since

peak

results: Figures 5A and been found in two different

obviously

procedure.

theoretically

cells

cannot detects

highly contrasted and particles coy-

upside-down and

in situ obtained

the

only

white an

peak also

stage

by an elimination

hat all

lution views (Figs. SB and that are found with normal

dilation:opening.

present

profiles of all these top hats of Figure

there remain of degenerative

detected

Description statistical alarm has

top-hat

types

in situ cell, cell covered

leukocytes

be followed

present be

events

of cells

of the

and narrower If we eliminate

by leukocytes, and the groups

they

the

various

a carcinoma a squamous

the three

because A. Since

it must

plastic and will present

bisectrix

in

of all other cells and artefacts. The hat A cells and the granulocytes, but ignores the cells are low-contrasted. It also ignores

artefact the hat

leukocytes

E

by

as

between a square

j

considered

that

different

(clump

recognition

The higher leukocytes.

“tools”

ofdegenerative

sufficient

granulocytes,

A, Grassfire

one

us to detect

of the

artefact

a group

for the

section

to the right, a granulocyte,

4B

the large enter into

followed

In fields

is Acriflavine-Feulgen imagination

enables

contrast. Figure relief representation.

ered cells

the

is a drawing

the setting aside detects the suspect normal intermediate

&

in

from the left intermediate,

artefact),

lA-I).

illustration:

microscope

at a resolution of 1 TV screen represents

(5).

precisely

peaks. Figure

FIG.

an

different

the darkness of the points as their altitude. Thus, the cells would appear as mountains of various heights and shapes. A normal intermediate cell would be a small hill, a granulocyte a group of three sharp peaks, and overlapping cells would appear as a chain of mountains. transformation

erosion

and

two

the TV screen TV lines. The

x 250

transmitted

C

the

B we

in

a positive high resoof the show the

B, where many leukocytes nothing has been detected.

cells that fields but

130

MEYER

FIG.

FIGS.

3A

and

2.

Microscope

B.

Top

hat

transformation.

Fields

at the

TV.

screen

(250iL

x 250i).

1’ #{149}1#{149}

Afl C

A

n FIG.

4.

Three

top hats

for the detection

of cells.

AUTOMATED

FIGS.

5A and

B.

A, a cell

CYTOLOGY

has

been

found

:

AND

(in white).

:

#{149}.

at a higher

resolution.

The

cell

is positive.

B, confirmation

at a higher

resolution.

The

cell

is positive.

L,

.

and

B, confirmation

,.

..

6A

131

MORPHOLOGY

.1

J

FIGS.

MATHEMATICAL

B.

A, a cell

.

has

been

found

(in

white).

4r.. 44’’

FIG.

7.

Detected

small

cell

carcinoma

cell

(magnified

view).

FIG.

8.

Detected

false

positive

two

overlapping

normal

nuclei.

132

MEYER

Most

detection

DNA

content,

and

cell carcinoma. of all cases. high

work

completely

on

the

size miss

of cells

or

case

of small

the

And yet, this form of disease represents Dr. Ploem and his group has shown that

DNA

missing

methods therefore

cells

them

exist is

even

not

in

these

negligible.

cases,

but

Fortunately

risk

of

method

of

detecting unenlarged but highly contrasted cells perfectly detects these cells of small cell carcinoma. This is illustrated on Figure 7 where one of these cells has been found near three normal intermediate cells. Let

us

consist as

now

speak

merely

such.

two

in

One

of

of

some

failures

artefacts

these

overlapping

the

of

which

our cannot

is represented

cases

normal

nuclei

method. be

They

recognized

in Figure

were

recognized

Conclusion: many

30% a few

the

our

on the

validity

will

be safer

would

slides of

be

the

if we an

first

must

results

be

method

can

add

seem

screened be

miss

matic

cells with high DNA content so typical would provide us with a more comfortable detected cells with regard to the false negative probably cause a slight increase in the number tives.

This

is

not

them. The next or how to separate Shape

suspect

serious

since

we

Recognition

cells. The

evaluation we

of the

method

treated

can

1000

only

fields

in

be done

each

of

statistically. 18 slides.

We

stopped before 1000 fields were viewed when, after a few hundred fields, a sufficient number of suspect cells were found. The slides chosen at random, with the only condition being to have a suitable staining. The results are shown in

ofArtefacts

Table

I.

The

machine

fields in available,

has

been

each slide. refocusing

programmed

to

go

through

1000

stops and asks if it has found a true I shows the results. The slides have

been

beforehand

screened

manually

According

normal;

“2”

is for

stage;

and

estimation We point

inflammation; “4”

means

slides, was

suspect

reasonably

“

1” means

“3”

for

dysplasia

carcinoma

cells

Slide

Diagnosis

18452 349

in

were

has

has is zero

found.

The

number

o f 18 Slides Investigated Number Cells

for

or Fal

se Positi

of

Nrlr

of s

Ic

True positive

Alarms 18

60,000

1,000

33

3b

10,000

1,000

17

10

1,000

54

41

0

86 0

3912

2

15,000

1,000

0

9

17484

3b

70,000

1,000

2

7

R 3512 R 2582

4

2,000

1,000

14

2

3a-b

150,000

1,000

6

22

R3l32

2

100,000

500

0

57

R2182

1

1,000

0

0

1,000

6

55

6,000

3a

100,000

2

30,000

0

8

R2592

5

3,000

200

39

2

R2662

4

6,000

41

2

R 2072

3a-b

600 500

31

51

R

3a

50,000

1,000

1

8

2

100,000

1,000

0

27

2082

within

is necessary

of a chromosome Since the chromo-

a mitosis,

before

is

a reorientation

visual

recognition.

This is not the case in cytology: the cytologist is able to recognize a cell from an artefact whatever their orientation or position within the field of view may be. He may even work at different magnifications. We would like our recognition

False Positive

Cells

29

R 3162

recognition is correct.

ye Cells

1,000

3772

of

I True

1,000

R

Separation

Cells

of false

3,000

17545

artefacts

made. in all

4

H

eliminate

A rough been since

2

197

random

to

or precansitu.

18312

R

at

how

been

3a-b

3b-4

located

chromosome

of dysplasia. number of rate. It would of false posi-

completely

60,000 100,000

18382

are

It

hypchro-

low.

TABLE of analysis

Results

R

a diagnosis

code,

of the number of cells measured out that the false negative rate

positive alarms

and

to a classical

cerous

of each

method.

large

to eliminate

and

results

Since an automatic focus is not yet was sometimes necessary. At each posi-

tive alarm, the machine or a false positive. Table given.

Choice of a method: The only possible if its orientation somes

know

chapter explains how clustering cells.

Clustering

Therefore

very

on

the

to our

the

but

assessment

Probably

criterion

to not

promising

any

niade.

a detection

advantage

very

before

This

8 where as

The

more

60,000

1,000

FIGS.

940,000

isolated

9 and

cells,

10. several

The conditional in artefacts.

hisectrix

marks

one

kernel

in

AUTOMATED

CYTOLOGY

AND

A

MATHEMATICAL

133

MORPHOLOGY

B

hA

FIGS.

and

B.

with

the

kernels

removed,

isolated

A

cells

have

one

hole;

artefacts

have

several.

B ‘S

I I

I

I I ‘S

...-..-

‘5,.

12A

FIGS.

and

B.

#{248},,

The

skeleton:

FIGS.

procedure

to have

the

the

clustering

This

constraint

is

on size

features

we

in the

Methods

constant

along

the In

objects Figures

are are

point

is higher

out

only

for

isolated

cells.

Triple

points

A and

B in artefacts.

B

very

than

strong

an and

In particular, transformations

section

that

skeleton

from

the and

the

FIGS.

whatever

in

Therefore for

image.

the

should

from

prohibited.

and not 9 and

on these

or

tools. all

the

Initial

flexibility:

program

looking

represented in gray. called ro-conditional fire

B.

objects

cells

the range of possible on the magnification, based

same

of the used,

from

and

13A

the

orientation

magnification cell

loop

B

A

or

one

the

field,

position

whatever

recognize

the

the

pictures

that

considerably

since

Dilation.

A

we

restrains

the size depends or measurements had

skeleton.

speed

of the only

minimal

B.

an isolated

to derive We

grassfire on

the

have

the

seen

is not shape

of

on their size position or orientation. 10 various shapes with various sizes

six times

and

artefact.

depends

White represents bisectrix, along

14A

speed

of the

cells

are

fire.

iSA

and

B.

Skeleton

of

the

dilated

part;

triple

points

for

are

the part of the skeleton which the speed of the

all isolated

FIGs.

artefacts.

We

marked

by

one

by

several

How cessing:

white

kernel kernels.

inside, That

skeletonization We

now

while

will

all

be our

permits have

two

other

shapes

are

marked

criterion. further

categories

parallel of

objects:

pro1)

cell

134

MEYER

characterized We

by

could

after particle,

continue

another, and

one with

counting

kernel;

2) artefacts

our

procedure

the

number

eliminating

all

by

several

by taking of

particles

kernels.

one

particle

parallel.

one

kernels

inside

each

The

containing

more

than

kerneLs

FIGs.

17A

and

But

then

set difference

our

processing

between

produces

one

hole

the in

16A,

B.

B and

A, cells

C.

Two

skeleton

in white;

artefacts

lines

for

in grey.

18A

and

B.

A, a cluster

of cancerous

cells.

isolated

C

each

object.

B, two

Triple

skeleton

points

lines

for

generated

artefacts.

by the

A

FIGS.

flO

original

the

B

A

FIGS.

kernel.

B, the

cells

are

automatically

separated.

program.

longer

image cells

would

and and

the

he

white several

AUTOMATED

holes

in

clustering

Figures

1 la

cells

and

changed; the holes. If we

b.

cells now

preserves

the

The

observation

recognition

can

easily

be achieved

cells

skeleton

of

the

or

more

having

no

points

of the

image, reduction

points,

have

remain.

vanished,

border. The too irregular. 13b

original zation

an

we

take

the

border For

(Figs.

iSa

leads

all

difference

and

b).

Each

to a small

of triple

points

appendices important

artefacts,

characterizing by

between

the

the

shorter irregularities

than

of

previous only by an

dilated

are

to

be

Note

can

the

procedure

points

be

and done

in

by a Golay

(in white)

could

step

in

and

the

if we

set

detection

being

and

the can

by the

(Fig.

of the

sets

process, 16c).

And

the

consecutive how

the

Finally, sures

Figure in

their

Each

the

17b

shows each

skeletons

object,

the

two

particle. are

which

different convex parts. the resulting skeleton. and works effectively

skeleton

The

triple

eliminaisolated

overlapping

isolated

is not

lines

particles

cells

cells

cells,

“round”

generated

without the

others

enough

and

by the

triple are

artefacts.

is segmented

18a and b. Figure l8a In Figure l8b these

or in

distribution.

and

we could

artefacts

and

we required

to the

position

to the magnification only a few tools may

insensitive

to the

experimental

reliability

base

of

used. be used.

fluctuations procedure

of

the

en-

results.

ACKNOWLEDGMENTS

ings

of

mated 2.

the

report

represents cells have

of the Engineering

VIth

Cytology,

Digabel

in

CITED Leiden Automation group, Foundation Conference

J Histochem

H, Lantuejoul

Ch:

Cytochem, Iterative Vol.

Pattern

Proceedon auto-

in press

algorithms,

8, Riederer

Special Verlag

Classification

Wiley, New York, 1973 4. Golay MJE: Hexagonal Pattern Vol. C 18 N#{176} 8, August 1969

Issues

Gmbh,

and

Scene

Analysis,

lEE

Trans

on Comp,

Transforms,

of

Stutt-

5.

Cornelisse CJ, Ploem JS: A new type of two-color fluorescence staining for cytology specimens. J Histochem Cytochem 24: 72, 1976 6. Klein JC, Serra J: The texture analyzer, J Micros 95:349, 1972 7. Matheron G: Elements pour une th#{233}oriedes milieux poreux, Masson et Cie, Paris, 1967 8. Matheron G: Random Sets and Integral Geometry, Wiley, New York, 1975

9. Meyer

F: Automated

10.

Meyer

Paris,

F: Contrast

Metallography,

points

Therefore the triple points appear in But the same procedure can be used for the separation of clustering cells.

This is illustrated in Figures a cluster of cancerous cells.

value

fluctuations

or in the

reproducibility

Symposium

for

and

of tools,

object

of different

illumination

gray

is insensitive

the field constraints

choice

in the

insensitive

be

These results could not have been obtained without the friendly collaboration of Mrs A. van Driel. She adjusted the parameters of the contrast program for an optimal detection of suspect cells and gave her diagnosis on all the alarms described in Table I.

artefacts. program

the

inherent

such

algorithms. of cells which

within strong

a method

in the the

about

When one inherent

where only nuclei were detected, Since the contrast of an object

under

Practical Metallography, gart, 1977 3. Duda Ro, Hart E:

leads two

these

a parallel

particle.

shows

stable

procedure

the objects Under these

all

analysis 16b). These

detection

from

almost

remarks

may

variation

program on contrast For the shape recognition

1. Al I: Progress

eliminate

Since

cell

crossing

we detect only In this way we

particle.

each

17a

be recognized

original

lines,

smoothness B in Fig.

parallel

Figure

remains

a few

to chose

changes

at a gray value is now apparent.

the

we wish to characterize. and illustration: To sumto one or more holes inside

each

for

transform

artefacts.

the

of each

procedure. the instabilities

fluctuations

a slight

completely

For example the cytoplasm

and

These

example

staining

to make

LITERATURE

that

skeletonization

one

ends

of the

for

For

like

recognition to analyze

analyzes

and

set

and

a given length of the borders.

skeletonized

disconnected,

this

of

chromatin

border

dilation

on these

each particle (set A in Fig. 16b), the to a ring around each particle (set

the

one

fluctuations.

types.

the

these

a small

irregularity

appendix

again.

objects

the

our

analysis

point”. of skeletonization

appears

to

we would

of a pattern study one has

a recognition

For

choose the degree of smoothness Combination ofboth criteria marize, the angular analysis leads

tion

the

particles

border

lateral

main line at a “triple Thus the same procedure

are

inside

a few binary

of a cell is always smooth or at example, Figure 13a represents a

artefact.

dilate

set

B).

one. We thus get Figures 14a and b. The skeletoniof these new sets leads to a closed line for each particle

now

sets

only

a complex

Most artefacts are detected by those that are not are characterized

Figure

of the

A and

and

with

the robustness starts a new in the

loop,

lines

points

criterion-smoothness

irregularities,

because one

three

the

Starting

ending with a few points of information is important.

irregular least not

the

12B,

completely

artefacts

the borders: method but

we

where

Figure

the

Conversely,

than

The

CONCLUSION

quickly detected by a simple Golay the treatment is completed, the cells,

A complementary

cell,

points,

(see

These points can be transform. Thus, after

more

segmented. separately.

As a conclusion

now

line.

been automatically can then be analyzed

b.

in parallel

having

triple

crossing

in

135

MORPHOLOGY

several which

that

closed

MATHEMATICAL

particles

12a and

shows

is a simple

artefacts,

our

have algorithm

Figures

pictures

the

are

we obtain

AND

represented of

artefacts

these

of the two

as

of

skeleton

presents

artefacts, connectivity

one hole while a skeletonization

connectivity, of cells

in

the

have apply

the

skeleton

or

Thus

CYTOLOGY

Cytology,

Proceedings,

20-24 June, 1977 features extraction,

Vol. 8. Riederer

Buffon

Special

Verlag

Gmbh,

bi-Centenary

Issues

Stuttgart,

of Practical

1977

1 1. Preston K: Feature extraction by Golay hexagonal pattern transforms, IEEE Trans on Comp, Vol C20, N#{176} 2, Sept, 1971 12. Serra J: One, two, three . . . infinity: Special Issues of Practical Metallography, Vol. 8, Riederer Verlag Gmbh, Stuttgart, 1977

13. Serra Press, 14.

Vrolijk cytology,

ence

J: Image London, J:

Analysis in press

A real Presented

on Automated

time

by Mathematical

TV at the

image analysis VIth Engineering

Cytology.

J Histochem

Morphology, system for Foundation

Cytochem,

Academic automated Confer-

in press