Visibilty: Finding the Staircase Kernel in Orthogonal Polygons

We consider the problem of finding the staircase kernel in orthogonal polygons, with or without holes, in the plane. Orthogonal polygon is a simple polygon in the plane whose sides are either horizontal or vertical. We generalize the notion of visibility in the following way: We say that two points a and b in an orthogonal polygon P are visible to each other via staircase paths if and only if there exist an orthogonal chain connecting a and b and lying entirely in the interior of P. Furthermore, the orthogonal chain should have the property that the angles between the consecutive segments in the chain are either or , and these should alternate along the chain. There are two principal types of staircases, NW-SE and NE-SW. The notion of staircase visibility has been studied in the literature for the last three decades. Based on this notion we can generalize the notion of star-shapedness. A polygon P is called star-shaped under staircase visibility, or simply s-star if and only if there is nonempty set of points S in the interior of P, such that any point of S sees any point of P via staircase path. The largest such set of points is called the staircase kernel of P and denoted ker P. Our work is motivated by the work of Breen [1]. She proves that the staircase kernel of an orthogonal polygon without holes is the intersection of all maximal orthogonally convex polygons contained in it. We extend Breen's results for the case when the orthogonal polygon has holes. We prove the necessary geometric properties, and use them to derive a quadratic time, O() algorithm for computing the staircase kernel of an orthogonal polygon with holes, having n vertices in total, including the holes' vertices. The algorithm is based on the plane sweep technique, widely used in Computational Geometry[4]. Our result is optimal in the case of orthogonal polygon with holes, since the kernel (as proven) can consist of quadratic number of disjoint regions. In the case of polygon without holes, there is a linear time algorithm by Gewali[3], that is specific to the case of a polygon without holes. We present examples of our algorithm's results.