Let $H$ denote the class of functions of the form

$ f(z)=z+\sum\limits_{n=2}^{\infty}a_{n}z^{n}, $

(1.1)

which are analytic in the open unit disk $U=\{z:|z|<1\}$ . Let

$ T=\{q\in H:q(z)=z-\sum\limits_{n=2}^{\infty}|d_{n}|z^{n}\}. $

It is obvious that $T\subset H$ . Let $\Omega$ denote the class of functions $w(z)$ regular in $U$ and satisfying the conditions $w(0)=0$ , $|w(z)|<1$ for $z\in U$ .

Let $f, g$ be analytic in $U$ . Then $ g$ is said to be subordinate to $f$ , written $g\prec f$ , if there exists a Schwarz function $\omega(z)\in \Omega$ , such that $g(z)=f(\omega(z))~(z\in U)$ . In particular, if the function $f(z)$ is univalent in $U$ , then

$ g(z)\prec f(z)~~(z\in U)\Longleftrightarrow g(0)=f(0)~{\text{and}}~g(U)\subset f(U). $

Let $P(A, B)$ $(-1\leq B<A\leq 1)$ denote the class of functions of the form $p(z)=1+\sum\limits_{n=1}^{\infty}p_{n}z^{n}$ , which are analytic in $U$ and satisfying the condition $p(z)\prec \frac{1+Az}{1+Bz}$ . It is clear that $P(1, -1)=P$ , the well-known class of positive real functions (see [1]). The classes of all starlike functions, convex functions and close-to-convex functions are respectively denoted by $S^*$ , $K$ and $C$ .

Li and Tang [2] defined the following two subclasses of the function class $H$ ,

$ \begin{eqnarray*}&&S^*_{\beta}(A, B) =\left\{g\in H:\frac{zg'(z)}{g(z)}-\beta\left|\frac{zg'(z)}{g(z)}-1\right|\prec \frac{1+Az}{1+Bz}, \beta\geq0, -1\leq B<A\leq 1\right\}, \\ &&K_{\beta}(A, B)=\left\{g\in H:\frac{(zg'(z))'}{g'(z)}-\beta\left|\frac{(zg'(z))'}{g'(z)}-1\right|\prec \frac{1+Az}{1+Bz}, \beta\geq0, -1\leq B<A\leq 1\right\}.\end{eqnarray*} $

It is obvious that

$ \begin{equation*} g(z)\in K_{\beta}(A, B) \Longleftrightarrow zg'(z)\in S_\beta^*(A, B). \end{equation*} $

(1.2)

$K_{1}(1, -1)=UCV$ is the class of uniformly convex functions (see [3-5]), $S^*_{1}(1, -1)=S_p$ is the class of parabolic starlike functions (see [4]).

In this paper, we generalize the class of $S_\beta^*(A, B)$ and $K_{\beta}(A, B)$ obtained by Li and Tang [2], and give the following two subclasses of the function class $H$ .

Definition 1.1 Let the function $f(z)\in H$ , if there exists a function $g(z)\in S^*_{\beta}(A, B)$ such that

$ \frac{zf'(z)}{g(z)}-\alpha\left|\frac{zf'(z)}{g(z)}-1\right|\prec \frac{1+Cz}{1+Dz}~~~(\alpha\geq0, -1\leq D<C\leq 1, z\in U), $

then $f(z)\in C_{\alpha, \beta}(A, B;C, D)$ .

Definition 1.2 Let the function $f(z)\in H$ , if there exists a function $g(z)\in K_{\beta}(A, B)$ such that

$ \frac{(zf'(z))'}{g'(z)}-\alpha\left|\frac{(zf'(z))'}{g'(z)}-1\right|\prec \frac{1+Cz}{1+Dz} ~~~(\alpha\geq0, -1\leq D<C\leq 1, z\in U), $

then $f(z)\in Q_{\alpha, \beta}(A, B;C, D)$ .

Obviously, we have

$ \begin{equation*}\label{xinzendengjia2} f(z)\in Q_{\alpha, \beta}(A, B;C, D) \Longleftrightarrow zf'(z)\in C_{\alpha, \beta}(A, B;C, D). \end{equation*} $

(1.3)

Remark 1.3 For suitable choices of parameters $A, B, C, D$ involved in Definitions 1.1 and 1.2, we also obtain the following subclasses which were studied in many earlier works.

(ⅰ) $C_{\beta, \beta}(A, B;A, B)=S^*_{\beta}(A, B)~(f(z)=g(z))$ and $ Q_{\beta, \beta}(A, B;A, B)=K_{\beta}(A, B)~(f(z)=g(z))$ (see Li et al. [2]);

(ⅱ) $C_{0, 0}(1, -1;1, -1)=C$ (see Reade [6]);

(ⅲ) $C_{0, 0}(A, B;C, D)=K(A, B;C, D)$ (see Harjinder et al. [7]);

(ⅳ) $C_{0, 0}(1, -1;C, D)=K(C, D)$ (see Harjinder et al. [7] and Mehrok [8]).

Let

$ \begin{eqnarray*}&&\overline{S}_\beta^{*}(A, B)=S_\beta^{*}(A, B)\cap T, ~~\overline{K}_\beta(A, B)=K_{\beta}(A, B)\cap T, ~~\overline{S}_{p}=S_{p}\cap T, \\&&\overline{C}_{\alpha, \beta}(A, B;C, D)=C_{\alpha, \beta}(A, B;C, D)\cap T, ~~\overline{Q}_{\alpha, \beta}(A, B;C, D)=Q_{\alpha, \beta}(A, B;C, D)\cap T.\end{eqnarray*} $

In this paper, we obtain unsolved coefficients inequalities of $S^*_{\beta}(A, B)$ and $K_{\beta}(A, B)$ defined in [2], and discuss coefficients inequalities of $C_{\alpha, \beta}(A, B;C, D)$ and $Q_{\alpha, \beta}(A, B;C, D)$ . Some of our results generalize previously known results obtained by [6-11].

In order to obtain unsolved coefficients inequalities of $S^*_{\beta}(A, B)$ and $K_{\beta}(A, B)$ defined in [2], we need the following lemmas.

Lemma 2.1 (see [12]) Let $\alpha\geq 0$ , $a, b\in R$ , $a\neq b$ and $|b|\leq 1$ . If $p(z)$ is an analytic function with $p(0)=1$ , then

$ p(z)-\alpha|p(z)-1|\prec \frac{1+az}{1+bz}\Longleftrightarrow p(z)(1-\alpha e^{-i\varphi})+\alpha e^{-i\varphi}\prec \frac{1+az}{1+bz}~~~(\varphi\in R). $

Lemma 2.2 (see [13]) Let $P(z)=\frac{1+Cw(z)}{1+Dw(z)}=1+\sum\limits_{n=1}^{\infty}p_{n}z^{n}$ , then

$ |p_{n}|\leq C-D. $

Lemma 2.3 (see [9]) Let $g(z)=\sum\limits_{k=q}^{\infty}b_{k}z^{k}$ and $G(z)=\sum\limits_{k=q}^{\infty}D_{k}z^{k}$ , $q\geq 0$ . If $g(z)=w(z)G(z)$ , where $w(z)\in \Omega$ , then $b_{q}=0$ and

$ \sum\limits_{k=q+1}^{n}|b_{k}|^{2}\leq\sum\limits_{k=q}^{n-1}|D_{k}|^{2}~~~(n=q+1, q+2, \cdots). $

We now prove coefficient inequalities of $\overline{S}_1^*(A, B)$ and $\overline{K}_1(A, B)$ .

Theorem 2.4 Let $q(z)=z-\sum\limits_{n=2}^{\infty}|d_{n}|z^{n}\in T$ , then $q(z)\in \overline{S_1}^*(A, B)$ if and only if

$ \begin{equation*}\label{2.7} \sum\limits_{n=2}^{\infty}[(2-B)(n-1)-(nB-A)]|d_{n}|\leq A-B , \end{equation*} $

(2.1)

where $-1\leq B<0$ .

Proof Suppose that inequality (2.1) is true. We define the function $p(z)$ by

$ p(z)=\frac{zq'(z)}{q(z)}-\left|\frac{zq'(z)}{q(z)}-1\right|. $

To prove $q(z)\in \overline{S}_1^*(A, B)$ , it suffices to show that

$ \left|\frac{p(z)-1}{A-Bp(z)}\right|<1~~~(z\in U). $

Since $B<0, -1\leq B<A\leq 1$ , we have $ nB-A=(n-1)B+B-A<0 $ , therefore

$ \begin{array}{l}\begin{split} \left|\frac{p(z)-1}{A-Bp(z)}\right| &=\left|\frac{\frac{zq'(z)}{q(z)}-\left|\frac{zq'(z)}{q(z)}-1\right|-1}{A-B\left(\frac{zq'(z)}{q(z)}-\left|\frac{zq'(z)}{q(z)}-1\right|\right)}\right|\notag\\ &=\left|\frac{zq'(z)-q(z)-e^{i\theta}|zq'(z)-q(z)|}{Aq(z)-Bzq'(z)+Be^{i\theta}|zq'(z)-q(z)|}\right| ~~\left(\frac{q(z)}{|q(z)|}=e^{i\theta}\right)\notag\\ &=\left|\frac{-\sum\limits_{n=2}^{\infty}(n-1)|d_{n}|z^{n}-e^{i\theta}\left|-\sum\limits_{n=2}^{\infty}(n-1)|d_{n}|z^{n}\right|}{(A-B)z+\sum\limits_{n=2}^{\infty}(nB-A)|d_{n}|z^{n}+Be^{i\theta}\left|-\sum\limits_{n=2}^{\infty}(n-1)|d_{n}|z^{n}\right|}\right|\notag\\ &\leq\frac{2\sum\limits_{n=2}^{\infty}(n-1)|d_{n}||z|^{n}}{(A-B)|z|-\sum\limits_{n=2}^{\infty}|nB-A||d_{n}||z|^{n}-|B|\sum\limits_{n=2}^{\infty}(n-1)|d_{n}||z|^{n}}\notag\\ \quad\quad\quad\quad&=\frac{2\sum\limits_{n=2}^{\infty}(n-1)|d_{n}||z|^{n-1}}{(A-B)+\sum\limits_{n=2}^{\infty}(nB-A)|d_{n}||z|^{n-1}+B\sum\limits_{n=2}^{\infty}(n-1)|d_{n}||z|^{n-1}}\\ &\leq\frac{2\sum\limits_{n=2}^{\infty}(n-1)|d_{n}|}{(A-B)+\sum\limits_{n=2}^{\infty}(nB-A)|d_{n}|+B\sum\limits_{n=2}^{\infty}(n-1)|d_{n}|}. \end{split} \end{array} $

From (2.1), we obtain

$ \left|\frac{p(z)-1}{A-Bp(z)}\right|\leq1~(z\in U)~~~(|z|=1). $

By using Maximum modulus theorem, we have $q(z)\in \overline{S}_1^*(A, B)$ .

Conversely, suppose that $q(z)\in \overline{S}_1^*(A, B)$ , then

$ \left|\frac{p(z)-1}{A-Bp(z)}\right|<1~~~(z\in U) $

or

$ \begin{array}{l}\begin{split} \left|\frac{p(z)-1}{A-Bp(z)}\right| =\left|\frac{\sum\limits_{n=2}^{\infty}(n-1)|d_{n}|z^{n}+e^{i\theta}\left|\sum\limits_{n=2}^{\infty}(n-1)|d_{n}|z^{n}\right|}{(A-B)z+\sum\limits_{n=2}^{\infty}(nB-A)|d_{n}|z^{n}+Be^{i\theta}\left|\sum\limits_{n=2}^{\infty}(n-1)|d_{n}|z^{n}\right|}\right| <1. \end{split} \end{array} $

Using the fact that $|{\text{Re}}z|\leq|z|$ for all $z$ , it follows that

$ \begin{equation*}\label{xinjiashizi} {\text{Re}}\left\{\frac{\sum\limits_{n=2}^{\infty}(n-1)|d_{n}|z^{n}+e^{i\theta}\left|\sum\limits_{n=2}^{\infty}(n-1)|d_{n}|z^{n}\right|}{(A-B)z +\sum\limits_{n=2}^{\infty}(nB-A)|d_{n}|z^{n}+Be^{i\theta}\left|\sum\limits_{n=2}^{\infty}(n-1)|d_{n}|z^{n}\right|}\right\}<1. \end{equation*} $

(2.2)

Now choose $z=r\in[0, 1)\subset U$ on the real axis, it is easy to show that $\theta=0$ or $\pi$ . Taking $\theta=0$ in (2.2), we obtain

$ \frac{2\sum\limits_{n=2}^{\infty}(n-1)|d_{n}|r^{n-1}}{(A-B) +\sum\limits_{n=2}^{\infty}(nB-A)|d_{n}|r^{n-1}+B\sum\limits_{n=2}^{\infty}(n-1)|d_{n}|r^{n-1}}<1. $

Let $r\rightarrow 1^{-}$ , we obtain (2.1).

Remark 2.5 Since $\overline S_1^*(1, -1)=\overline{S}_p$ , taking $A=1$ , $B=-1$ in Theorem 2.4, we obtain the following corollary.

Corollary 2.6 (see [10]) Let $f(z)=z-\sum\limits_{n=2}^{\infty}|a_{n}|z^{n}$ , then $f\in \overline{S}_{p}$ if and only if

$ \sum\limits_{n=2}^{\infty}(2n-1)|a_{n}|\leq 1. $

Corollary 2.7 Let $q(z)=z-\sum\limits_{n=2}^{\infty}|d_{n}|z^{n}\in \overline{S}_1^*(A, B)$ , then for $-1\leq B<0$ ,

$ |d_{n}|\leq\frac{A-B}{(2-B)(n-1)-(nB-A)}. $

Proof From Theorem 2.4, we obtain

$ \sum\limits_{n=2}^{\infty}[(2-B)(n-1)-(nB-A)]|d_{n}|\leq A-B. $

Since $B<0$ , $-1\leq B<A\leq 1$ , we have

$ (2-B)(n-1)-(nB-A)=(2-B)(n-1)-[(n-1)B+(B-A)]>0. $

Hence

$ |d_{n}|\leq\frac{A-B}{(2-B)(n-1)-(nB-A)}. $

From (1.2) and Theorem 2.4, we get the following Theorem 2.8

Theorem 2.8 Let $q(z)=z-\sum\limits_{n=2}^{\infty}|d_{n}|z^{n}\in T$ , then $q(z)\in \overline{K}_1(A, B)$ if and only if

$ \sum\limits_{n=2}^{\infty}n[(2-B)(n-1)-(nB-A)]|d_{n}|\leq A-B, $

where $-1\leq B<0$ .

Remark 2.9 Since $\overline{K_1}(1, -1)=UCV$ , taking $A=1, B=-1$ in Theorem 2.8, we obtain the following corollary.

Corollary 2.10 (see [10]) Let $f(z)=z-\sum\limits_{n=2}^{\infty}|a_{n}|z^{n}$ , then $f\in UCV$ if and only if

$ \sum\limits_{n=2}^{\infty}n(2n-1)|a_{n}|\leq 1. $

From Theorem 2.8, we get the following Corollary 2.11.

Corollary 2.11 Let $q(z)=z-\sum\limits_{n=2}^{\infty}|d_{n}|z^{n}\in \overline{K}_1(A, B)$ , then for $-1\leq B<0$ ,

$ |d_{n}|\leq\frac{A-B}{n[(2-B)(n-1)-(nB-A)]}. $

Theorem 2.12 Let $g(z)=z+\sum\limits^{\infty}_{n=2}b_nz^n\in S^*_{\beta}(A, B)$ , then

$ \begin{equation*}\label{2.1} |b_{n}|\leq \frac{1}{(n-1)!}\prod\limits_{k=2}^{n}\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-(k-1)B\right|, \end{equation*} $

(2.3)

where

$ \begin{equation*}\label{2.2} \left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-(n-1)B\right|\geq(n-2)~~~(n\geq 3) \end{equation*} $

(2.4)

and

$ e^{-i\varphi}=\left|\frac{zg'(z)}{g(z)}-1\right|\cdot\left(\frac{zg'(z)}{g(z)}-1\right)^{-1}. $

Proof Suppose that $g\in S^*_{\beta}(A, B)$ . Then, from Lemma 2.1, we obtain

$ \frac{zg'(z)}{g(z)}(1-\beta e^{-i\varphi})+\beta e^{-i\varphi}\prec\frac{1+Az}{1+Bz}. $

It follows from the definition of subordination that

$ \frac{zg'(z)}{g(z)}(1-\beta e^{-i\varphi})+\beta e^{-i\varphi}=\frac{1+Aw(z)}{1+Bw(z)}, ~~w(z)\in \Omega, $

which is equivalent to

$ \frac{zg'(z)}{g(z)}=\frac{1+\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi}}w(z)}{1+Bw(z)}. $

After some computation, we obtain

$ \sum\limits_{k=1}^{\infty}(k-1)b_{k}z^{k}=\sum\limits_{k=2}^{\infty}(k-1)b_{k}z^{k}=w(z)\sum\limits_{k=1}^{\infty}\left(\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-Bk\right)b_{k}z^{k}~~~(b_{1}=1). $

From Lemma 2.3, we obtain

$ \sum\limits_{k=2}^{n}(k-1)^{2}|b_{k}|^{2}\leq \sum\limits_{k=1}^{n-1}\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-Bk\right|^{2}|b_{k}|^{2}, ~~n=2, 3, \cdots, $

which evidently yields

$ \begin{align} |b_{n}|^{2}\leq& \frac{1}{(n-1)^{2}}\notag\times\left[\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-B\right|^{2}\right.\\ & \left.+\sum\limits_{k=2}^{n-1}\left(\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-Bk\right|^{2}-(k-1)^{2}\right)|b_{k}|^{2}\right], ~n=2, 3, \cdots. \end{align} $

(2.5)

By using (2.4), all the terms under the summation sign in (2.5) are positive. It is obvious from (2.5) that $|b_{2}|\leq |\frac{A-\beta B e^{-i\varphi}}{1-\beta e^{-i\varphi}}-B|$ satisfies (2.3). Assume that (2.1) is true for all $k=3, 4, \cdots, n-1$ .

We now prove

$ |b_{n}|\leq \frac{1}{(n-1)!}\prod\limits_{k=2}^{n}\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-(k-1)B\right|. $

In order to complete the proof, it is sufficient to show that

$ \begin{align} &\frac{1}{(m-1)^{2}}\left[\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-B\right|^{2}+ \sum\limits_{k=2}^{m-1}\left(\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-Bk\right|^{2}-(k-1)^{2}\right)|b_{k}|^{2}\right]\notag\\ \leq&\frac{1}{((m-1)!)^{2}}\prod\limits_{ j=2}^{m}\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-(j-1)B\right|^{2}, ~~m=3, 4, \cdots, \end{align} $

(2.6)

where $\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-(m-1)B\right|\geq(m-2).$

We now use mathematical induction to prove (2.6).

It readily follows from $|b_{2}|\leq\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-B\right|$ and $\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-2B\right|\geq 1$ that (2.6) is true for $m=3$ . Assume that (2.6) is true for all $m$ , $3<m\leq n-1$ . Then from (2.5), we have

$ \begin{array}{l}\begin{split} |b_{n}|^{2}\leq& \frac{1}{(n-1)^{2}}\left[\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-B\right|^{2}+ \sum\limits_{k=2}^{n-1}\left(\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-Bk\right|^{2}-(k-1)^{2}\right)|b_{k}|^{2}\right]\\ =&\frac{(n-2)^{2}}{(n-1)^{2}}\frac{1}{(n-2)^{2}}\left[\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-B\right|^{2}+ \sum\limits_{k=2}^{n-2}\left(\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-Bk\right|^{2}-(k-1)^{2}\right)|b_{k}|^{2}\right]\\ &+\frac{1}{(n-1)^{2}}\left[\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-(n-1)B\right|^{2}- (n-2)^{2}\right]|b_{n-1}|^{2}\\ \leq&\frac{(n-2)^{2}}{(n-1)^{2}}\frac{1}{((n-2)!)^{2}}\prod\limits_{ j=2}^{n-1}\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-(j-1)B\right|^{2}\\ &+\frac{1}{((n-1)!)^{2}}\left[\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-(n-1)B\right|^{2}- (n-2)^{2}\right]\prod\limits_{ j=2}^{n-1}\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-(j-1)B\right|^{2}\\ =&\frac{1}{((n-1)!)^{2}}\prod\limits_{ j=2}^{n}\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-(j-1)B\right|^{2}. \end{split} \end{array} $

Hence

$ |b_{n}|\leq \frac{1}{(n-1)!}\prod\limits_{j=2}^{n}\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-(j-1)B\right|. $

Remark 2.13 (ⅰ) If $f(z)\in S_0^*(1, -1)=S^*$ , then $|a_n|\leq n$ , we get Theorem 4.10 [11]. (ⅱ) Taking $\beta=0$ in Theorem 2.3, we get Theorem 1 [9].

Corollary 2.14 Let $g\in S^*_{\beta}(A, B)$ , then

$ \begin{equation*}\label{2.5} |b_{n}|\leq \frac{1}{(n-1)!}\prod\limits_{j=2}^{n}\left[\frac{|A|+\beta |B| }{|1-\beta| }+(j-1)|B|\right], ~~\beta\neq 1, \end{equation*} $

(2.7)

where

$ \begin{equation*}\label{2.6} \frac{||A|-\beta |B||}{1+\beta }-(n-1)|B|\geq n-2~~(n\geq 3). \end{equation*} $

(2.8)

Proof From (2.8), we obtain

$ \begin{eqnarray*}\left|\frac{A-\beta B e^{-i\varphi}}{1-\beta e^{-i\varphi} }-(n-1)B\right| &\geq&\left|\left|\frac{A-\beta B e^{-i\varphi}}{1-\beta e^{-i\varphi} }\right|-(n-1)|B|\right|\\ &\geq&\frac{||A|-\beta |B|| }{1+\beta }-(n-1)|B|\geq n-2 ~~(n\geq 3).\end{eqnarray*} $

By using Theorem 2.4, we obtain

$ |b_{n}|\leq \frac{1}{(n-1)!}\prod\limits_{j=2}^{n}\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-(j-1)B\right|. $

Therefore

$ |b_{n}|\leq \frac{1}{(n-1)!}\prod\limits_{j=2}^{n}\left[\frac{|A|+\beta |B| }{|1-\beta| }+(j-1)|B|\right], ~~\beta\neq 1. $

From (1.2) and Theorem 2.3, we get the following Theorem 2.15.

Theorem 2.15 Let $g(z)=z+\sum\limits^{\infty}_{n=2}b_nz^n\in K_{\beta}(A, B)$ , then

$ |b_{n}|\leq \frac{1}{n!}\prod\limits_{k=2}^{n}\left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-(k-1)B\right|, $

where

$ \left|\frac{A-\beta Be^{-i\varphi}}{1-\beta e^{-i\varphi} }-(n-1)B\right|\geq n-2~~~(n\geq 3) $

and

$ e^{-i\varphi}=\left|\frac{zg'(z)}{g(z)}-1\right|\cdot\left(\frac{zg'(z)}{g(z)}-1\right)^{-1}. $

Remark 2.16 If $f(z)\in K_0(1, -1)=K$ , then $|a_n|\leq 1$ , we get Theorem 4.13 [11].

By using Theorem 2.15, it is easy to obtain the following Corollary 2.17.

Corollary 2.17 Let $g\in K_{\beta}(A, B)$ , then

$ |b_{n}|\leq \frac{1}{n!}\prod\limits_{j=2}^{n}\left[\frac{|A|+\beta |B| }{|1-\beta| }+(j-1)|B|\right], ~~\beta\neq 1, $

where

$ \frac{||A|-\beta |B||}{1+\beta }-(n-1)|B|\geq n-2~~~(n\geq 3). $

In this section, by using Corollary 2.7, Corollary 2.11, Corollary 2.14 and Corollary 2.16, we obtain coefficient inequalities of $C_{\alpha, \beta}(A, B;C, D)$ and $Q_{\alpha, \beta}(A, B;C, D)$ .

Theorem 3.1 Let $f(z)\in C_{\alpha, \beta}(A, B;C, D)$ , then for $\frac{||A|-\beta |B||}{1+\beta }-(n-1)|B|\geq n-2~~(n\geq 3)$ ,

$ \begin{align} |a_{n}|\leq&\frac{1}{n!}\prod\limits_{j=2}^{n}\left[\frac{|A|+\beta|B|}{|1-\beta|}+(j-1)|B|\right]\notag\\ &+\frac{C-D}{|1-\alpha|n} \left(1+\sum\limits_{k=2}^{n-1}\frac{1}{(k-1)!}\prod\limits_{j=2}^{k}\left[\frac{|A|+\beta|B|}{|1-\beta|}+(j-1)|B|\right]\right), \end{align} $

(3.1)

where $0\leq\alpha, \beta\neq1$ .

Proof Suppose that $f(z)\in C_{\alpha, \beta}(A, B;C, D)$ . Then, there exits $g\in S^*_{\beta}(A, B)$ such that

$ \frac{zf'(z)}{g(z)}-\alpha\left|\frac{zf'(z)}{g(z)}-1\right|\prec \frac{1+Cz}{1+Dz}. $

From Lemma 2.1, we obtain

$ \frac{zf'(z)}{g(z)}(1-\alpha e^{-i\varphi})+\alpha e^{-i\varphi}\prec\frac{1+Cz}{1+Dz}~~~(\varphi\in R). $

It follows from the definition of subordination that

$ \frac{zf'(z)}{g(z)}(1-\alpha e^{-i\varphi})+\alpha e^{-i\varphi}=\frac{1+Cw(z)}{1+Dw(z)}=p(z), ~~p(z)\in P, $

or, equivalently,

$ zf'(z)(1-\alpha e^{-i\varphi})=(p(z)-\alpha e^{-i\varphi})g(z). $

After some computation, we have

$ \begin{align} &(z+2a_{2}z^{2}+3a_{3}z^{3}+\cdots)(1-\alpha e^{-i\varphi})\notag\\ =&(1-\alpha e^{-i\varphi}+p_{1}z+\cdots+p_{n}z^{n}+\cdots) (z+b_{2}z^{2}+\cdots+b_{n}z^{n}+\cdots). \end{align} $

(3.2)

Equating the coefficients of $z^{n}$ in (3.2), we get

$ na_{n}=b_{n}+\frac{1}{1-\alpha e^{-i\varphi}}(p_{1}b_{n-1}+p_{2}b_{n-2}+\cdots+p_{n-2}b_{2}+p_{n-1}). $

From Lemma 2.2, we obtain

$n|a_{n}|\leq|b_{n}|+\frac{C-D}{|1-\alpha|}(1+\sum\limits_{k=2}^{n-1}|b_{k}|). $

By using Corollary 2.14, we get

$ \begin{align} n|a_{n}|\leq&\frac{1}{(n-1)!}\prod\limits_{j=2}^{n}\left[\frac{|A|+\beta|B|}{|1-\beta|}+(j-1)|B|\right] \notag\\ &+\frac{C-D}{|1-\alpha|} \left(1+\sum\limits_{k=2}^{n-1}\frac{1}{(k-1)!}\prod\limits_{j=2}^{k}\left[\frac{|A|+\beta|B|}{|1-\beta|}+(j-1)|B|\right]\right), \notag \end{align} $

which proves (3.1).

Similarly, by using Corollary 2.16, we can prove the following result.

Theorem 3.2 Let $f(z)\in Q_{\alpha, \beta}(A, B;C, D)$ , then for $\frac{||A|-\beta |B||}{1+\beta }-(n-1)|B|\geq n-2~~(n\geq 3)$ ,

$ \begin{align} |a_{n}|\leq&\frac{1}{n\cdot n!}\prod\limits_{j=2}^{n}\left[\frac{|A|+\beta|B|}{|1-\beta|}+(j-1)|B|\right]\notag\\ &+\frac{C-D}{|1-\alpha|n^{2}} \left(1+\sum\limits_{k=2}^{n-1}\frac{1}{(k-1)!}\prod\limits_{j=2}^{k}\left[\frac{|A|+\beta|B|}{|1-\beta|}+(j-1)|B|\right]\right), \end{align} $

(3.3)

where $0\leq\alpha, \beta\neq1$ .

By using Corollary 2.7, we get

Theorem 3.3 Let $f(z)\in \overline{C}_{\alpha, 1}(A, B;C, D)$ , then

$ |a_{n}|\leq\frac{1}{n}\frac{A-B}{(2-B)(n-1)-(nB-A)} +\frac{C-D}{|1-\alpha|n} \left(1+\sum\limits_{k=2}^{n-1}\frac{A-B}{(2-B)(k-1)-(kB-A)}\right), $

where $0\leq\alpha\neq1$ and $-1\leq B<0$ .

By using Corollary 2.11, we have

Theorem 3.4 Let $f(z)\in \overline{Q}_{\alpha, 1}(A, B;C, D)$ , then

$ |a_{n}|\leq\frac{1}{n^{2}}\frac{A-B}{(2-B)(n-1)-(nB-A)} +\frac{C-D}{|1-\alpha|n^{2}} \left(1+\sum\limits_{k=2}^{n-1}\frac{A-B}{(2-B)(k-1)-(kB-A)}\right), $

where $0\leq\alpha\neq1$ and $-1\leq B<0$ .

Remark 3.5 (ⅰ) Taking $A=1, B=-1, C=1, D=-1, \alpha=0, \beta=0$ in Theorem 3.1, we obtain $|a_{n}|\leq n $ which is the result due to Reade [6].

(ⅱ) Putting $\alpha=0, \beta=0, -1\leq B<0<A\leq 1$ in Theorem 3.1, we get the results obtained by Harjinder and Mehrok [7, Theorem 3.1].

(ⅲ) If $f(z)\in C_{0, 0}(1, -1;C, D)=K(C, D)$ , then $|a_{n}|\leq 1+\frac{(n-1)(C-D)}{2}$ , we have the results obtained by Mehrok [8].