Let $ D $ be a domain in the complex plane $ \mathbb{C} $, and $ \mathcal{F} $ be a family of meromorphic functions in $ D $. The family $ \mathcal{F} $ is said to be normal in $ D $, in the sense of Montel, if every sequence $ \{f_n\}\subset \mathcal{F} $ contains a subsequence $ \{f_{n_j}\} $ such that $ \{f_{n_j}\} $ converges spherically locally uniformly on $ D $ to a meromorphic function or $ \infty $ [1, 2, 3].

In 2009, Xu, Wu and Liao[4](cf. [5]) proved the following result.

Theorem A  Let $ a(\neq 0) $ and $ b $ be finite complex numbers, $ n, k $ be two positive integers with $ n\geq k+1 $, and let $ \mathcal{F} $ be a family of meromorphic functions in a domain $ D $. If, for each $ f\in \mathcal{F} $, $ f $ has only zeros of multiplicity at least $ k+1 $, and $ f+a(f^{(k)})^n\neq b $ in $ D $, then $ \mathcal{F} $ is normal in $ D $.

In 2013, Lei, Fang and Zeng[6] showed that Theorem A is still valid for $ n\geq 2 $. Under the condition that $ f\neq 0 $ for each $ f\in \mathcal{F} $, Li[7] obtained the following result.

Theorem B  Let $ a(\neq 0) $ and $ b $ be finite complex numbers, $ n(\geq 2), k $ be two positive integers, and let $ \mathcal{F} $ be a family of zero-free meromorphic functions in a domain $ D $. If, for each $ f\in \mathcal{F} $, $ f+a(f^{(k)})^n-b $ has at most $ nk $ distinct zeros in $ D $, ignoring multiplicities, then $ \mathcal{F} $ is normal in $ D $.

Chen[8] extended $ f+a(f^{(k)})^n-b $ in Theorem B to $ f^m+a(f^{(k)})^n-b $, as follows.

Theorem C  Let $ \mathcal{F} $ be a family of zero-free meromorphic functions in a domain $ D $, and let $ n, k $ and $ m $ be positive integers with $ n\geq m+1 $, and $ a(\neq 0) $ and $ b $ be finite complex numbers. If, for each $ f\in \mathcal{F} $, $ f^m+a(f^{(k)})^n-b $ has at most $ nk $ distinct zeros in $ D $, ignoring multiplicities, then $ \mathcal{F} $ is normal in $ D $.

Let $ a_1(z), a_2(z), \cdots, a_k(z) $ be holomorphic functions. Set

$ \begin{eqnarray} L[f](z)=f^{(k)}(z)+a_1(z)f^{(k-1)}(z)+\cdots+a_k(z)f(z), \end{eqnarray} $

(1.1)

we call $ L[f] $ a linearly differential polynomial of $ f $.

In this paper, we replace '$ f^{(k)} $' by '$ L[f] $' in Theorem C, and prove the following result.

Theorem 1  Let $ \mathcal{F} $ be a family of zero-free meromorphic functions in a domain $ D $, let $ a(z)(\neq 0) $ be a holomorphic function in $ D $, and let $ n, k $ and $ m $ be positive integers with $ n\geq m+1 $, and $ b $ a finite complex number. If, for each $ f\in \mathcal{F} $, $ f^m(z)+a(z)(L[f](z))^n-b $ has at most $ nk $ distinct zeros (ignoring multiplicities) in $ D $, then $ \mathcal{F} $ is normal in $ D $, where $ L[f](z) $ is defined as in (1.1).

A natural problem arises: does Theorem C still hold if the constant $ a(\neq 0) $ is replaced by the function $ a(z)(\not \equiv 0) $? In this paper, we shall prove the following result.

Theorem 2  Let $ \mathcal{F} $ be a family of zero-free meromorphic functions in a domain $ D $, let $ a(z)(\not \equiv 0) $ be a holomorphic function in $ D $, whose zeros have multiplicities at most $ n-1 $, and let $ n, k $ and $ m $ be positive integers with $ n\geq m+1 $, and $ b $ a finite complex number. If, for each $ f\in \mathcal{F} $, $ f^m(z)+a(z)(L[f](z))^n-b $ has at most $ nk $ zeros (counting multiplicities) in $ D $, then $ \mathcal{F} $ is normal in $ D $, where $ L[f](z) $ is defined as in (1.1).

Remark 1  Clearly, Theorems 1 and 2 generalize Theorems B and C.

Remark 2  The condition that all zeros of $ a(z) $ have multiplicities at most $ n-1 $ in Theorem 2 is necessary, as is shown by the following example.

Example 1  Let $ m=k=1, n=2 $, $ D=\{z: |z|<1\} $, $ a(z)=z^2, b=-1 $, and $ \mathcal{F}=\{f_j\} $, where $ f_j(z)=\{1/jz\}(j=1, 2, \cdots) $.

We know that

$ f_j(z)+a(z)(f'_j(z))^2-b=\frac{jz^2+jz+1}{j^2z^2} $

has at most $ 2 $ zeros. But $ \mathcal{F} $ is not normal at $ 0 $.

Remark 3  Since normality is a local property, the condition that for each $ f\in \mathcal{F} $, $ f^m(z)+a(z)(L[f](z))^n-b $ has at most $ nk $ zeros (ignoring multiplicities in Theorem 1, counting multiplicities in Theorem 2) in $ D $ can be replaced by that for each $ z_0\in D $, there exists some $ r>0 $ such that $ f^m(z)+a(z)(L[f](z))^n-b $ has at most $ nk $ zeros (ignoring multiplicities in Theorem 1, counting multiplicities in Theorem 2) in $ \Delta_r(z_0)=\{z: |z-z_0|<r\} $.

Throughout this paper, we denote by $ \mathbb{C} $ the complex plane, by $ \mathbb{C}^* $ the punctured complex plane $ \mathbb{C}\setminus\{0\} $. For $ r>0 $, we shall write $ \Delta_r=\{z: |z|<r\} $ and $ \Delta'_r=\{z: 0<|z|<r\} $. When $ r=1 $, we drop the subscript.

To prove our results, we need some lemmas.

Lemma 1  [9, 10] Let $ \mathcal{F} $ be a family of zero-free functions meromorphic in a domain $ D $. If $ \mathcal{F} $ is not normal at $ z_0 \in D $, then for each $ -1<\alpha <\infty $, there exist a sequence of complex numbers $ z_n \in D $ with $ z_n \rightarrow z_0 $, a sequence of positive numbers $ \rho_n \rightarrow 0 $ and a sequence of functions $ f_n \in \mathcal{F} $ such that

$ \begin{equation*} g_n(\zeta)=\cfrac{f_n(z_n+\rho_n \zeta)}{\rho_n^{\alpha}} \to g(\zeta) \end{equation*} $

converges spherically and locally uniformly on $ \mathbb{C} $, where $ g $ is a nonconstant zero-free meromorphic function of order at most $ 2 $. In particular, $ g $ is of order at most $ 1 $ if $ g $ is entire on $ \mathbb{C} $.

Lemma 2  [11] Let $ f $ be a nonconstant zero-free rational function, and $ k $ a positive integer. Then $ f^{(k)}-1 $ has at least $ k+1 $ distinct zeros.

Lemma 3  [8] Let $ n\geq 2, m, k $ be positive integers, and $ a\in \mathbb{C}^* $. If $ f $ is a zero-free transcendental meromorphic function in $ \mathbb{C} $, then $ f^m+a(f^{(k)})^n $ has infinitely many zeros.

Lemma 4  [8] Let $ n, m, k $ be positive integers with $ n\geq m+1 $, let $ a(\neq 0), b $ be two finite complex number. If $ f $ is a nonconstant zero-free rational function, then $ f^m+a(f^{(k)})^n-b $ has at least $ nk+1 $ distinct zeros in $ \mathbb{C} $.

Proof of Theorem 1  Suppose, on the contrary, that $ \mathcal{F} $ is not normal at $ z_0\in D $. We consider two cases.

Case 1. $ b=0 $.

by Lemma 1, there exist points $ z_j\to z_0 $, positive numbers $ \rho_j\to 0 $ and functions $ f_j $ such that

$ \begin{eqnarray} g_j(\zeta)=\rho_j^{-\frac{nk}{n-m}}f_j(z_j+\rho_j\zeta)\to g(\zeta) \end{eqnarray} $

(3.1)

spherically uniformly on compact subsets of $ \mathbb{C} $, where $ g $ is a nonconstant zero-free meromorphic function of order at most $ 2 $. In particular, $ g $ is of order at most $ 1 $ if $ g $ is entire on $ \mathbb{C} $.

By Lemmas 3 and 4, $ (g(\zeta))^m+a(z_0)(g^{(k)}(\zeta))^n $ has at least $ nk+1 $ distinct zeros in $ \mathbb{C} $. Let $ \zeta_1, \zeta_2, \cdots, \zeta_{nk+1} $ are distinct zeros of $ g^m(\zeta)+a(z_0)(g^{(k)}(\zeta))^n $.

We claim that $ g^m(\zeta)+a(z_0)(g^{(k)}(\zeta))^n\not\equiv 0 $. Otherwise, $ g^m(\zeta)+a(z_0)(g^{(k)}(\zeta))^n\equiv 0 $, then $ g $ has no poles since $ n\geq m+1 $, that is, $ g $ is entire. Hence $ g $ is of order at most $ 1 $. Noting that $ g $ is nonconstant zero-free, $ g $ has the form $ g(\zeta)=e^{c\zeta+d} $, with $ c\in \mathbb{C}^* $ and $ d\in \mathbb{C} $. This gives

$ g^m(\zeta)+a(z_0)(g^{(k)}(\zeta))^n=e^{m(c\zeta+d)}[1+a(z_0)c^{nk}e^{(n-m)(c\zeta+d)}]\not\equiv 0 $

since $ n\geq m+1 $, a contradiction. The claim is proved.

By (3.1), we have

$ g_j^{(t)}(\zeta)=\rho_j^{t-\frac{nk}{n-m}}f_j^{(t)}(z_j+\rho_j\zeta)\to g^{(t)}(\zeta), (t=1, 2, \cdots.) $

locally uniformly on $ \mathbb{C}\setminus g^{-1}(\infty) $. Since $ a_1(z), a_2(z), \cdots, a_k(z) $ are holomorphic functions, we obtain that

$ \begin{eqnarray} &&\rho_j^{-\frac{mnk}{n-m}}\left[f^m_j(z_j+\rho_j\zeta)+a(z_j+\rho_j\zeta)\left(L[f_j](z_j+\rho_j\zeta)\right)^n\right]\\ &=&g^m_j(\zeta)+a(z_j+\rho_j\zeta)\left(g_j^{(k)}(\zeta)+\rho_ja_1(z_j+\rho_j\zeta)g_j^{(k-1)}(\zeta)+\cdots+\rho_j^k a_k(z_j+\rho_j\zeta)g_j(\zeta)\right)^n\\ &\to & g^m(\zeta)+a(z_0)(g^{(k)}(\zeta))^n \end{eqnarray} $

locally uniformly on $ \mathbb{C}\setminus g^{-1}(\infty) $.

This and Hurwitz's theorem imply that, for $ i=1, 2, \cdots, nk+1 $, there exist $ \zeta_{j, i} $ such that $ \zeta_{j, i}\to \zeta_i $ ($ j\to \infty $), and for sufficiently large $ j $,

$ f^m_j(z_j+\rho_j\zeta_{j, i})+a(z_j+\rho_j\zeta_{j, i})\left(L[f_j](z_j+\rho_j\zeta_{j, i})\right)^n=0. $

On the other hand, we know that $ f^m_j+a(L[f_j])^n $ has at most $ nk $ distinct zeros. It follows that there exists $ 1\leq i\neq i'\leq nk+1 $ such that

$ z_j+\rho_j\zeta_{j, i}=z_j+\rho_j\zeta_{j, i'}, $

so that $ \zeta_{j, i}=\zeta_{j, i'} $. Letting $ j\to \infty $, $ \zeta_i=\zeta_{i'} $, a contradiction.

Case 2. $ b\neq 0 $.

By Lemma 1, there exist points $ z_j\to z_0 $, positive numbers $ \rho_j\to 0 $ and functions $ f_j $ such that

$ \begin{eqnarray} g_j(\zeta)=\rho_j^{-k}f_j(z_j+\rho_j\zeta)\to g(\zeta) \end{eqnarray} $

(3.2)

spherically uniformly on compact subsets of $ \mathbb{C} $, where $ g $ is a nonconstant zero-free meromorphic function of order at most $ 2 $. By (3.2), we have

$ \begin{eqnarray} &&f^m_j(z_j+\rho_j\zeta)+a(z_j+\rho_j\zeta)\left(L[f_j](z_j+\rho_j\zeta)\right)^n-b \\ &=&\rho_j^{mk}g^m_j(\zeta)+a(z_j+\rho_j\zeta)\left(g_j^{(k)}(\zeta)+\rho_j a_1(z_j+\rho_j\zeta)g_j^{(k-1)}(\zeta)+\cdots+\rho_j^k a_k(z_j+\rho_j\zeta)g_j(\zeta)\right)^n-b \\ &\to& a(z_0)(g^{(k)}(\zeta))^n-b \end{eqnarray} $

locally uniformly on $ \mathbb{C}\setminus g^{-1}(\infty) $.

Obviously, $ a(z_0)(g^{(k)}(\zeta))^n-b\not\equiv 0 $. Otherwise, $ a(z_0)(g^{(k)}(\zeta))^n-b\equiv 0 $, then $ (g^{(k)}(\zeta))^n=b/a(z_0) $, and thus $ g $ is a polynomial of degree $ k $, which is impossible since $ g\neq 0 $.

Then, arguing as in Case 1, $ a(z_0)(g^{(k)}(\zeta))^n-b $ has at most $ nk $ distinct zeros in $ \mathbb{C} $ since $ f^m_j+a(L[f_j])^n-b $ has at most $ nk $ distinct zeros in $ D $.

We write

$ \begin{eqnarray} a(z_0)(g^{(k)}(\zeta))^n-b=a(z_0)\prod\limits_{i=1}^n(g^{(k)}(\zeta)-c_i), \end{eqnarray} $

(3.3)

where $ c_1, c_2, \cdots, c_n $ are distinct zeros of $ z^n-b/a(z_0) $. If $ g $ is a rational function, it follows from Lemma 2 that $ g^{(k)}(\zeta)-c_i $ has at least $ k+1 $ distinct zeros for $ 1\leq i\leq n $. By (3.3), $ a(z_0)(g^{(k)}(\zeta))^n-b $ has at least $ n(k+1)>nk+1 $ distinct zeros, a contradiction. Then $ g $ is transcendental. By (3.3), and Nevanlinna's first and second fundamental theorems, we have

$ \begin{eqnarray} T(r, g^{(k)})&\leq& \bar{N}(r, g^{(k)})+\sum\limits_{i=1}^n\bar{N}\left(r, \cfrac{1}{g^{(k)}-c_i}\right)+S(r, g^{(k)}) \\ &\leq& \cfrac{1}{k+1}N(r, g^{(k)})+\bar{N}\left(r, \cfrac{1}{a(z_0)(g^{(k)}(\zeta))^n-b}\right)+S(r, g^{(k)})\\ &\leq& \cfrac{1}{k+1}T(r, g^{(k)})+S(r, g^{(k)}), \end{eqnarray} $

a contradiction. Theorem 1 is thus proved.

Proof of Theorem 2  By Theorem 1, it is enough to prove that $ \mathcal{F} $ is normal at the zeros of $ a(z) $ in $ D $. Without loss of generality, we assume $ D=\Delta $, $ a(z)=z^l\varphi(z) $, where $ l(\leq n-1) $ is a positive integer, $ \varphi(z) $ is holomorphic and zero-free in $ \Delta $, and $ \varphi(0)=1 $. Now $ \mathcal{F} $ is normal in $ \Delta' $ by Theorem 1, and we need to prove that $ \mathcal{F} $ is normal at $ z=0 $.

Suppose, on the contrary, that $ \mathcal{F} $ is not normal at $ z=0 $. Since for each $ f\in \mathcal{F} $, $ f(z)\neq 0 $ in $ \Delta $, we know that the family $ \mathcal{F}_1={\{}1/f: f\in {\mathcal{F}}{\}} $ is holomorphic in $ \Delta $ and normal on $ \Delta' $, but not normal at $ z=0 $. Then there exists a sequence $ \{ 1/f_j \}\subset \mathcal{F}_1 $ which converges locally uniformly in $ \Delta' $, but not on $ \Delta $. The maximum modulus principle implies that $ 1/f_j\to \infty $ in $ \Delta' $. Thus $ f_j\to 0 $ converges locally uniformly in $ \Delta' $, and hence $ f^{(i)}_j\to 0 $ converges locally uniformly in $ \Delta' $ for $ 1\leq i\leq k $ by Weierstrass theorem.

Clearly, there exists $ 0<\delta<1 $ such that

$ f_j^m(z)+a(z)(L[f_j(z)])^n-b\neq 0 $

on $ |z|=\delta $. If $ \{f_j\} $ is holomorphic in $ \Delta_{\delta} $, again by the maximum modulus, $ f_j\to \infty $ converges locally uniformly in $ \Delta'_{\delta} $, which contradicts $ f_j\to 0 $ in $ \Delta' $. So $ \{f_j\} $ is not holomorphic in $ \Delta_{\delta} $.

By the argument principle, we have

$ \begin{align*} n\left(\delta, \frac{1}{f_j^m(z)+a(z)(L[f_j(z)])^n-b}\right)&- n\left(\delta, f_j^m(z)+a(z)(L[f_j(z)])^n-b\right)\\ &=\frac{1}{2\pi i}\int_{|z|=\delta}\frac{\left(f_j^m(z)+a(z)(L[f_j(z)])^n-b\right)'}{f_j^m(z)+a(z)(L[f_j(z)])^n-b}dz\\ &\to 0, \end{align*} $

where $ n(r, 1/f) $ is the number of zeros of $ f $ in $ \Delta_r $ and $ n(r, f) $ is the number of poles of $ f $ in $ \Delta_r $, counting multiplicity. Then for sufficiently large $ j $,

$ \begin{equation} n\left(\delta, \frac{1}{f_j^m(z)+a(z)(L[f_j(z)])^n-b}\right)= n\left(\delta, f_j^m(z)+a(z)(L[f_j(z)])^n-b\right). \end{equation} $

(3.4)

In view of $ n\geq m+1 $ and $ l\leq n-1 $, we see that

$ \begin{equation*} n\left(\delta, f_j^m(z)+a(z)(L[f_j(z)])^n-b\right)\geq n(k+1)-l>nk. \end{equation*} $

On the other hand, by the assumption of Theorem,

$ \begin{equation*} n\left(\delta, \frac{1}{f_j^m(z)+a(z)(L[f_j(z)])^n-b}\right)\leq nk. \end{equation*} $

These, together with (3.4), yield that $ nk>nk $, a contradiction. This completes the proof of Theorem 2.