On a Subclass of Analytic Functions Related to a Hyperbola

The object of the present investigation is to solve Fekete-Szeg？ problem and determine the sharp upper bound to the second Hankel determinant for a new class of analytic functions in the unit disk. We also obtain a sufficient condition for an analytic function to be in this class. 1. Introduction and Preliminaries Let be the class of functions of the form: which are analytic in the open unit disk . A function is said to be starlike function of order and convex function of order , respectively, if and only if and , for and for all . By usual notations, we denote these classes of functions by and , respectively. We write and , the familiar subclasses of starlike functions and convex functions in . Furthermore, a function is said to in the class , if it satisfies the inequality: Note that is a subclass of close-to-convex functions of order in . Let denote the class of analytic functions of the form: satisfying the condition in . Let the functions and be analytic in . We say that is subordinate to , written as or , if there exists a Schwarz function , which (by definition) is analytic in with and . Furthermore, if the function is univalent in , then we have the following equivalence relation (cf., e.g., [1]): For the functions analytic in and given by the power series their Hadamard product (or convolution), denoted by is defined as Note that is analytic in . The Gauss hypergeometric function is defined by the infinite series where denotes the Pochhammer symbol (or shifted factorial) given, in terms of the Gamma function , by We note that the series, given by (7), converges absolutely for and hence the function represents an analytic function in the unit disc [2]. We further observe that the Gauss hypergeometric function plays an important role in the study of various properties and characteristics of subclasses of univalent/multivalent functions in geometric function theory (cf., e.g. [3–5]). In our present investigation, we consider the incomplete beta function , defined by By making use of the Hadamard product and the function , Carlson and Shaffer [6] defined the linear operator by If is given by (1), then it follows from (10) that The operator extends several operators introduced and studied by earlier researchers in geometric function theory.？？For example, , the well-known Ruscheweyh derivative operator [7] of and , the familiar Owa-Srivastava fractional differential operator [8] of . With the aid of the linear operator , we introduce a subclass of as follows. Definition 1. A function is said to be in the class , if it satisfies the following