Over the past decades, the optimal dividend problem has been a hot issue and a large number of papers in this field have came out. The study of dividend problem has a realistic sense: for a joint-stock company, it has responsibility to pay dividends to its shareholder, therefore choosing a divided strategy is important for this company. The research of dividend problem stems from the work of De Finetti [1]. He is the first to suggest that a company should maximize the expected discounted dividend payout. In earlier research of this field, scholars focused on two kinds of dividend strategies. The first one is the constant barrier strategy. In this model, we have a barrier $ b $, splitting region into two parts. Under such policy, the surplus cannot cross the barrier $ b $ at any time $ t>0 $, and when it hits the barrier, it will either stay at the barrier $ b $ or decrease below the barrier (see for example Gerber and Shiu [2], Albrecher et al.[3], Albrecher and Kainhofer [4], Li and Garrido [5], Loeffen et al. [6]). The second one is threshold strategy, specifically, dividends can be paid out at certain rate if the surplus exceeds a threshold (see for example Gerber and Shiu [2], Lin and Pavlova [7]). Stochastic control theory has been introduced to solve the optimal dividend problem. HJB equation, QVI, and singular control as tools were used to deal with dividend problem from different aspects. Asmussen and Taksar [3] considered optimal continuous dividend for diffusion model. In the paper Azcue and Muler [8], the author not only studied the optimal continuous dividend strategy but also considered the reinsurance policy in the compound Poisson model. Impulsive dividend and reinsurance strategies for diffusion model can be found in paper Cadenillas et al. [9].

Compared with continuous time risk models, obviously discrete time risk models with the dividend payment strategies have not attracted much attention, although De Finetti [1] gave the problem first in a discrete time model. However, discrete time models also have their merits, for example, they are closer to reality and they can be seen as the approximations of continuous time models. For the optimal dividend payment problem in discrete models, see [1] and [10] for the classic results. In addition, several recent contributions can be found. Gerber et al. [11] modeled the surplus of an insurance company (before dividends) as a time-homogeneous Markov chain with possible changes of size $ + 1 , 0 , -1, -2 , ... $ in one period, and showed that the optimal dividend-payment strategy is band strategy irrespective of the penalty at ruin. Tan et al. [12] considered the discrete Sparre Andersen risk model and its derivative models by anew setting up the initial times, and showed that the optimal value function is the unique bounded solution of a set of discrete Hamilton-Jacobi-Bellman equations.

However, in these papers, it is assumed that there are not any transaction costs when dividends are paid out. Practically, transaction costs cannot be neglected. Here, to make the dividend analysis even more realistic, we take into account fixed transaction cost $ K>0 $ which is incurred each time the dividend is paid out. In addition to transaction costs, we assume that the shareholders are required to pay taxes with a tax rate $ 1-k $ $ (0<k<1) $. Due to the additional difficulty, the optimal dividend problems subject to transaction costs are still rarely considered in the insurance literature. As far as we know, the optimal dividend problem with transaction costs and taxes was considered only under the diffusion and the classical model. Jeanblanc- Picque and Shiryaev [13] applied impulse control theory to obtain the optimal dividend strategy in which only the dividend control was allowed and there was a fixed transaction cost. Cadenillas et al. [9] considered the optimal dividend problem with proportional reinsurance control when payments were subject to taxes as well as transaction costs. For a general diffusion model and the case of no reinsurance, the optimal dividend problem with transaction costs and taxes was solved in Paulsen [14]. Bai and Guo [15] studied the optimal dividend problem in the classical risk model with transaction costs and taxes which were required when dividends occured.

Therefore, it is necessary to look for an optimal dividend strategy when dividend payments are subject to both transaction costs and taxes in discrete risk model.

In this paper, we study the optimal dividend problem in the discrete model. Transaction costs and taxes are required when dividends occur. The outline of this paper is as follows. In Section 2, we give a rigorous mathematical formulation of the problem. In Section 3, we heuristically derive the set of discrete HJB equations for the optimal value functions and introduce the transformation method. Section 4 firstly gives properties of the optimal image functions and then by solving the HJB equation, the optimal image functions are approximately obtained.

Consider the discrete Andersen surplus process

$ U(t) = u+ct-\sum\limits^{N(t)}_{i = 1}X_{i}, \ \ \ \ t = 0, 1, 2, \cdots , $

where $ U(0) = u $ is the initial surplus, which is a non-negative integer. $ c $ is the premium received in each time period, which is a positive integer. The counting process $ N(t), t = 0, 1, 2, \cdots $ denotes the number of claims up to time $ t $ and is defined as $ N(t) = \max\{n:T_{1}+T_{2}+\cdots+T_{n}\leq t\}, $ where claim inter-occurrence times $ T_{i} $ are assumed i.i.d. positive and integer-valued random variables with common probability function $ p(k) = P_{r}(T_{i} = k), k = 1, 2, \cdots $ and $ ET_{i} < \infty $. Therefore, $ N = \{N(t)\} $ is an ordinary renewal process. The random variable $ X_{i} $ represents the $ i $th claim, independent of $ \{N(t)\} $. The random variables $ X_{i}, i = 1, 2, \dots $ are i.i.d random variables with common probability function $ f(k) = P_{r}(X_{i} = k), k = 1, 2, \dots $. Denote the filtration by $ \{F_{t}\} $ $ (t = 0, 1, 2, \cdots ) $, where $ F_{t} $ comprises all the information before time $ t $.

We do not need positive safety loading condition in this paper. We say the company ruins if the surplus becomes negative. Put $ \sigma_{k} = T_{1} + T_{2} + \dots + T_{k} $ $ (k = 1 , 2 , \dots) $ and $ \sigma_{0} = 0 $. Let

$ \bar{P} ( 0 ) = 1, \bar{P} ( n ) = 1- \sum\limits_{k = 1}^{n}p( k ), \forall n \geq 1 , $

and assume that $ \bar{P} ( n ) > 0 $ for any positive integer $ n $. Let $ s = s ( t ) $ denote the distance between $ N ( t ) $ and $ t $ $ ( t = 0 , 1 , 2 , \dots) $, namely, $ s(t) = t -N ( t ) $. Obviously, the set of all possible values of $ s $ is $ \{ 0 , 1 , 2 , \dots\} $.

We now enrich the model. Let the company pay dividends to its shareholders according to some dividend strategies. To make the dividend analysis even more realistic, we take into account fixed both transaction costs $ K $ and transaction taxes with tax rate $ 1-k $ which are incurred each time that dividends are paid out (no matter how much).

A strategy is described by

$ \pi = (\tau_{1}, \tau_{2}, \dots \tau_{n}, \dots; \xi_{1}, \xi_{2}, \dots, \xi_{n}, \dots), $

where $ \tau_{n} $ and $ \xi_{n} $ denote the times and amounts of dividends. Thus, when applying strategy $ \pi $, the resulting reserve process $ U^{\pi}(t) $ is given by

$ U^{\pi}(t) = u+ct-\sum\limits^{N(t)}_{i = 1}X_{i}-\sum\limits^{\infty}_{n = 1}I_{(\tau_{n}<t)}\xi_{n}, \ \ \ \ t = 0, 1, 2, \cdots . $

A strategy is said to be admissible if

(i) Dividend payment at every time $ t $ will not lead to ruin;

(ii) Dividend paid at every time $ t $ will not exceed a positive integer upper bound denoted by $ c_0 $;

(iii) Dividend paid at time $ t $ is integer-valued and $ F_{t} $ -measurable;

(iv) $ \frac{K}{k}<\xi_{n}\leq U^{\pi}(\sigma_{n}), n = 0, 1, 2, \cdots . $

We denote the set of all admissible strategies by $ \Pi $.

The condition (4) means that when the dividends are paid out, the net amount of money that the shareholders receive must be positive (i.e. $ \frac{K}{k}<\xi_{n} $), and the total amount of dividends cannot exceed the reserve available at that time (i.e. $ \xi_{n}\leq U^{\pi}(\sigma_{n}) $).

It is easy to see that the optimal admissible payment at the initial time 0 is a function with respect to $ U{( 0 )} $. With each admissible strategy $ \pi $, $ \phi^{\pi}_{0}(u) $ denotes the optimal payment given that the initial surplus is $ u $, where the subscript 0 refers to the so-called "zero-delayed case" and $ s ( 0 ) = 0 $. At time $ t = 1 $, we give attention to two scenarios. The first is that if a claim occurs, the surplus process really gets renewed when the claim does not lead to ruins. Hence, the optimal payment at this time is $ \phi^{\pi}_{0}(x) $ where the subscript $ 0 $ is exactly equal to the value of $ s ( 1 ) $ and $ x = u-\phi^{\pi}_{0}(u) + c - X_{1} $. The second scenario is that if no claim occurs, the rest of the surplus process is a new surplus process with the occurrence of the claims described by a delayed renewal process. Similarly, the optimal payment at the initial time in the new surplus process is only dependent on its initial surplus. We denote by $ \phi^{\pi}_{1}(x) $ the optimal payment when the initial surplus is $ x $, where the subscript $ 1 $ is now the value of $ s(1) $ and refers to the ''delayed case'' with the probability function of the epoch of the first claim being $ g_{1}(k) = \frac{p(k + 1)}{\bar{P}(1)}( k = 1 , 2 , \dots) $. Then, analogizing in turn for scenarios at time $ 2 , 3 , \dots, $ we can find that the all possible values of the optimal payment at arbitrary time $ t $ can be denoted by $ \phi^{\pi}_{s}(x), s = 0 , 1 , 2 , \dots; x = 0 , 1 , 2 , \dots, $ where $ x $ is the surplus of time $ t $, and $ s = t -N ( t ) $ refers to the "delayed case" (or "zero-delayed case") with the probability function of the epoch of the first claim being $ g_{s}(m) = \frac{p(m+s)}{\bar{P}(s)}(m = 1, 2, \dots). $ In other words, the optimal dividend payment at time $ t $ is only dependent on the distance $ s $ and the surplus $ x $ of the time. Because of the fact, we only discuss a type of admissible strategies, namely the optimal payments that are functions of two variables: the distance $ s $ and the surplus $ x $. For any $ \pi\in\Pi $, the corresponding value function is defined as

$ \begin{align} V^{\pi}(u) = E_{u}\left[\sum\limits^{\tau}_{t = 0}r^{t}(k\phi_{s(t)}(U^{\pi}(t))-K)\right], \end{align} $

(2.1)

where $ r\in(0, 1) $ is the unit time discount factor, $ \tau $ is the time of ruin and $ E_{u} $ represents the conditional expectation on the initial surplus $ u $. The optimal value function is defined as

$ \begin{align} V^{*}(u) = \sup\limits_{\pi\in\Pi}V^{\pi}(u). \end{align} $

(2.2)

and the corresponding optimal strategy is denoted by $ \pi^{*} $ such that $ V^{*}(u) = V^{\pi^{*}}(u) $. If we regard an arbitrary time $ t $ $ (t = 1, 2, \dots) $ as initial time and the surplus of the time as initial surplus, then we get a new surplus process with the probability function of the epoch of the first claim being $ g_{s(t)}(k) $. Let $ V^{\pi}_{s(t)}(u) $ denote the value function of the strategy $ \pi $ and $ V^{*}_{s(t)}(u) $ denote the optimal value function in the derivative surplus process. Our object is to find the optimal strategy which maximizes the corresponding value functions.

Theorem 3.1   For an initial surplus $ u \in {\Bbb N} $, the optimal value functions $ V^{*}_{m}(u) (m = 0, 1, 2, ...) $ satisfy the following discrete HJB equations

$ \begin{align} V^{*}_{m}(u) = &\max\limits_{d = 0, 1, 2, \dots, c_{0}\bigwedge u}\{(kd-K)+\frac{\bar{P}(m+1)}{\bar{P}(m)}rV^{\pi}_{m+1}(u-d+c)\\ &+\frac{P(m+1)}{\bar{P}(m)}r\sum\limits^{u-d+c}_{j = 1}V^{\pi}_{0}(u-d+c-j)f(j)\}, m = 0, 1, 2, \dots. \end{align} $

(3.1)

Proof   Assume that $ \pi\in\Pi $ and the corresponding value functions in different surplus processes are $ V^{\pi}_{m}(u)(m = 0, 1, 2, \dots). $ Then,

$ \begin{align*} V^{\pi}_{0}(u) = & (k\phi^{\pi}_{0}(u)-K)+( 1-p(1))rV^{\pi}_{1}(u-\phi^{\pi}_{0}(u)+c)\\ &+p(1)r\sum\limits^{u-\phi^{\pi}_{0}(u)+c}_{j = 1}V^{\pi}_{0}(u-\phi^{\pi}_{0}(u)+c-j)f(j), \\ V^{\pi}_{1}(u) = & (k\phi^{\pi}_{1}(u)-K)+\frac{\bar{P}(2)}{\bar{P}(1)}rV^{\pi}_{2}(u-\phi^{\pi}_{1}(u)+c)\\ &+\frac{P(2)}{\bar{P}(1)}r\sum\limits^{u-\phi^{\pi}_{1}(u)+c}_{j = 1}V^{\pi}_{0}(u-\phi^{\pi}_{1}(u)+c-j)f(j), \\ &\vdots\\ V^{\pi}_{n}(u) = & (k\phi^{\pi}_{n}(u)-K)+\frac{\bar{P}(n+1)}{\bar{P}(n)}rV^{\pi}_{n+1}(u-\phi^{\pi}_{n}(u)+c)\\ &+\frac{P(n+1)}{\bar{P}(n)}r\sum\limits^{u-\phi^{\pi}_{n}(u)+c}_{j = 1}V^{\pi}_{0}(u-\phi^{\pi}_{n}(u)+c-j)f(j), \\ &\vdots \end{align*} $

It is obvious that the following holds,

$ \begin{align} V^{*}_{m}(u) = &\max\limits_{d = 0, 1, 2, \dots, c_{0}\bigwedge u}\{(kd-K)+\frac{\bar{P}(m+1)}{\bar{P}(m)}rV^{\pi}_{m+1}(u-d+c)\\ &+\frac{p(m+1)}{\bar{P}(m)}r\sum\limits^{u-d+c}_{j = 1}V^{\pi}_{0}(u-d+c-j)f(j)\}, m = 0, 1, 2, \dots. \end{align} $

(3.2)

Therefore, the proof of Theorem 3.1 is complete.

Since the dividend paid at any time does not exceed $ c_{0} $, for any non-negative integer $ m $,

$ \begin{equation} V^{\pi}_{m}(u)\leq \frac{kc_{0}-K}{1-r}. \end{equation} $

(3.3)

For the arbitrary admissible strategy $ \pi\in\Pi $, we transform its value functions $ V^{\pi}_{m}(u) $ by

$ \begin{equation} V^{\pi}_{m}(u) = (k\phi^{\pi}_{m}(u)-K)+{W}^{\pi}_{m}(u-\phi^{\pi}_{m}(u)). \end{equation} $

(3.4)

Then, by (3.2) the image functions $ W^{\pi}_{m}(u) $ satisfy the following equations

$ \begin{align} W^{\pi}_{m}(u) = &\frac{\bar{P}(m+1)}{\bar{P}(m)}r[W^{\pi}_{m+1}(u+c-\phi^{\pi}_{m+1}(u+c))+k\phi^{\pi}_{m+1}(u+c)-K]\\ &+\frac{P(m+1)}{\bar{P}(m)}r\sum\limits^{u+c}_{j = 1}[W^{\pi}_{0}(u+c-\phi^{\pi}_{0}(u+c-j)-j)+k\phi^{\pi}_{0}(u+c-j)-K]f(j), \\ &m = 0, 1, 2, \dots. \end{align} $

(3.5)

According to (3.5), $ W^{\pi}_{m}(u) $ can apparently be interpreted as all expected discounted dividends except for the possible dividend paid at initial time in the surplus process $ U^{\pi}(t+n); $ $ n = 0, 1, \cdots $ with $ s(t) = m. $ Then, for any non-negative integer $ m $, we have

$ \begin{equation} W^{\pi}_{m}(u)\leq\frac{r}{1-r}(kc_{0}-K). \end{equation} $

(3.6)

The following Theorem 3.2 will tell us that the payment strategy corresponding to maximal image functions is optimal, $ i.e. $ it maximizes both $ W^{\pi}_{m}(u) $ and $ V^{\pi}_{m}(u) $ for arbitrary non-negative integers $ u $ and $ m $. Therefore, we only need to obtain the optimal image functions to reach our objective.

Lemma3.1   (Contraction mapping principle) Suppose $ ({\bf{X}}, \rho) $ is a complete metric space. Let $ {\bf{T}}:{\bf{X}}\rightarrow {\bf{X}} $ be a contraction mapping on $ {\bf{X}} $, i.e. there is a nonnegative real number $ q<1 $ such that $ ({\bf{T}}x, {\bf{T}}y)\leq q(x, y) $ for all $ x, y\in {\bf{X}} $. Then the map $ {\bf{T}} $ admits one and only one fixed point $ x^{'} $, i.e. $ {\bf{T}}(x^{'}) = x^{'}. $

Theorem 3.2   Assume that $ \pi^{*}\in\Pi, W^{\pi^{*}}_{m}(u) $ and $ \phi^{\pi^{*}}_{m}(u) (m = 0, 1, \dots) $ are the images of its value functions and the corresponding dividend payment. Then, $ W^{\pi^{*}}_{m}(u) $ are all maximal if and only if

$ \begin{equation} \phi^{\pi^{*}}_{m}(u) = \arg \max\limits_{d = 0, 1, 2, \dots, c_{0}\bigwedge u}\{W^{\pi^{*}}_{m}(u-d)+kd-K\} \end{equation} $

(3.7)

holds for any non-negative integer m and any non-negative integer u. In addition, $ \phi^{\pi^{*}}_{m}(u) $ obtained by (3.7) also maximizes all of the value functions $ V^{\pi^{*}}_{m}(u) (m = 0, 1, \dots). $

Proof   ("only if" part) We assume that $ W^{\pi^{*}}_{m}(u) = \sup_{\pi\in\Pi}W^{\pi}_{m}(u)\ \ (m = 0, 1, \dots) $ are all the maximal image functions. By (3.5), it follows that $ W^{\pi^{*}}_{m}(u) \ \ (m = 0, 1, \dots) $ satisfy the equations

$ \begin{align} W^{\pi^{*}}_{m}(u) = &\sup\limits_{\pi\in\Pi}W^{\pi}_{m}(u)\\ = &\frac{\bar{P}(m+1)}{\bar{P}(m)}r[W^{\pi}_{m+1}(u+c-\phi^{\pi}_{m+1}(u+c))+k\phi^{\pi}_{m+1}(u+c)-K]\\ &+\frac{P(m+1)}{\bar{P}(m)}r\sum\limits^{u+c}_{j = 1}[W^{\pi}_{0}(u+c-\phi^{\pi}_{0}(u+c-j)-j)+k\phi^{\pi}_{0}(u+c-j)-K]f(j), \\ &m = 0, 1, 2, \dots. \end{align} $

(3.8)

Obviously, (3.8) is equivalent to

$ \begin{align} W^{\pi^{*}}_{m}(u) = &\frac{\bar{P}(m+1)}{\bar{P}(m)}r \max\limits_{d = 0, 1, 2, \dots, c_{0}\bigwedge u}{W^{\pi^{*}}_{m+1}(u+c-d)+kd-K}\\ &+\frac{P(m+1)}{\bar{P}(m)}r\sum\limits^{u+c}_{j = 1}\max\limits_{d = 0, 1, 2, \dots, c_{0}\bigwedge u}[W^{\pi^{*}}_{0}(u+c-j-d)+kd-K]f(j), \\ &m = 0, 1, 2, \dots. \end{align} $

(3.9)

A comparison of (3.5) and (3.9) with $ W^{\pi^{*}}_{m}(u) $ and $ \phi^{\pi^{*}}_{m}(u) $ interpreted as the optimal image functions and the corresponding dividend payment respectively, for any non-negative integer $ m $ and any non-negative integer $ u $, we have

$ \begin{equation} (k\phi^{\pi^{*}}_{m}(u)-K)+W^{\pi^{*}}_{m}(u-\phi^{\pi^{*}}_{m}(u)) = \max\limits_{d = 0, 1, 2, \dots, c_{0}\bigwedge u}{W^{\pi^{*}}_{m}(u-d)+kd-K} \end{equation} $

(3.10)

from which (3.7) follows.

("if" part) We consider the complete metric space $ l^{\infty} $, the set of all bounded real sequences. Obviously, $ \forall \pi\in\Pi, W^{\pi}_{m}(u)\in l^{\infty} $ for any $ m $, the patchwork made up of them, denoted by $ W = (W^{\pi}_{0}(u), W^{\pi}_{2}(u), \dots), $ also belongs to $ l^{\infty} $. Assume that $ F_{m}(u), G_{m}(u)\in l^{\infty} (m = 0, 1, \dots), F = (F_{0}(u), F_{1}(u), F_{2}(u), \dots) $, and $ G = (G_{0}(u), G_{1}(u), G_{2}(u), \dots). $ Define operator $ {\bf{T}}: l^{\infty}\rightarrow l^{\infty} $ by

$ \begin{align} {\bf{T}}F = &(\bar{P}(1)r{\bf{T_{1}}}F_{1}+P(1)r{\bf{T_{0}}}F_{0}, \frac{\bar{P}(2)}{\bar{P}(1)}r{\bf{T_{1}}}F_{2}+\frac{P(2)}{\bar{P}(1)}r{\bf{T_{0}}}F_{0}, \dots, \\ &\frac{\bar{P}(n)}{\bar{P}(n-1)}r{\bf{T_{1}}}F_{n}+\frac{P(n)}{\bar{P}(n-1)}r{\bf{T_{0}}}F_{0}, \frac{\bar{P}(n+1)}{\bar{P}(n)}r{\bf{T_{1}}}F_{n+1}+\frac{P(n+1)}{\bar{P}(n)}r{\bf{T_{0}}}F_{0}, \dots), \end{align} $

(3.11)

where the operator $ {\bf{T_{1}}} $ defined by

$ \begin{equation} {\bf{T_{1}}}F_{m}(u) = F_{m}(u+c-\phi_{F_{m}}(u+c))+k\phi_{F_{m}}(u+c)-K, \end{equation} $

(3.12)

the operator $ {\bf{T_{0}}} $ defined by

$ \begin{equation} {\bf{T_{0}}}F_{0}(u) = \sum\limits^{u+c}_{j = 1}[F_{0}(u+c-\phi_{F_{0}}(u+c-j)-j)+k\phi_{F_{0}}(u+c-j)-K]f(j), \end{equation} $

(3.13)

and

$ \phi_{F_{m}}(u) = \arg \max\limits_{d = 0, 1, 2, \dots, c_{0}\bigwedge u}\{F_{m}(u-d)+kd-K\}. $

$ \rho({\bf{T}}F, {\bf{T}}G) = \sup_{n\geq 0}\rho({\bf{T}}F_{n}, {\bf{T}}G_{n}) $ denotes the distance between $ {\bf{T}}F $ and $ {\bf{T}}G $, $ \rho(F, G) = \sup_{n\geq 0}\rho(F_{n}, G_{n}) $ denotes the distance between $ F $ and $ G $. Then,

$ \begin{align} \rho({\bf{T}}F, {\bf{T}}G)& = \sup\limits_{n\geq 0}\rho({\bf{T}}F_{n}, {\bf{T}}F_{n})\\ &\leq \sup\limits_{n\geq 1}{\frac{\bar{P}(n)}{\bar{P}(n-1)}rd({\bf{T_{1}}}F_{n}, {\bf{T_{1}}}G_{n}) +\frac{p(n)}{\bar{P}(n-1)}r\rho({\bf{T_{0}}}F_{0}, {\bf{T_{0}}}G_{0})}. \end{align} $

(3.14)

For any fixed $ u $ and any fixed $ m $, assume that, without loss of generality,

$ F_{m}(u+c-\phi_{F_{m}}(u+c))+k\phi_{F_{m}}(u+c)-K \geq G_{m}(u+c-\phi_{G_{m}}(u+c))+k\phi_{G_{m}}(u+c)-K. $

Then,

$ \begin{align} \rho({\bf{T_{1}}}F_{m}, {\bf{T_{1}}}G_{m}) = &\sup\limits_{u}|(F_{m}(u+c-\phi_{F_{m}}(u+c))+k\phi_{F_{m}}(u+c)-K)\\ &-(G_{m}(u+c-\phi_{G_{m}}(u+c))+k\phi_{G_{m}}(u+c)-K)|\\ \leq& \sup\limits_{u}|F_{m}(u+c-\phi_{F_{m}}(u+c))+k\phi_{F_{m}}(u+c)-K)\\ &-(G_{m}(u+c-\phi_{F_{m}}(u+c))+k\phi_{F_{m}}(u+c)-K)|\\ = &\sup\limits_{u}|(F_{m}(u+c-\phi_{F_{m}}(u+c))-G_{m}(u+c-\phi_{F_{m}}(u+c))|\\ = &\rho(F_{m}, G_{m})\\ \leq& \rho(F, G). \end{align} $

(3.15)

Similarly, we can obtain

$ \begin{equation} \rho({\bf{T_{0}}}F_{0}, {\bf{T_{0}}}G_{0}) \leq \rho(F_{0}, G_{0}) \leq \rho(F, G). \end{equation} $

(3.16)

By (3.14), (3.15) and (3.16), we have

$ \begin{align} \rho({\bf{T}}F, {\bf{T}}G) & \leq \sup\limits_{n\geq 1}{\frac{\bar{P}(n)}{\bar{P}(n-1)}r\rho({\bf{T_{1}}}F_{n}, {\bf{T_{1}}}G_{n}) +\frac{p(n)}{\bar{P}(n-1)}rd({\bf{T_{0}}}F_{0}, {\bf{T_{0}}}G_{0})}\\ & \leq \sup\limits_{n\geq 1}{\frac{\bar{P}(n)}{\bar{P}(n-1)}r\rho(F, G)+\frac{p(n)}{\bar{P}(n-1)}r\rho(F, G)}\\ & = r\rho(F, G), \end{align} $

(3.17)

from which we find that $ {\bf{T}} $ is a contraction mapping because of the fact that $ r<1 $. By lemma 3.1, we conclude that for each $ m $, Eq.(3.9) has a bounded and unique solution. By (3.4), for arbitrary non-negative integers $ u $ and $ m $, $ V^{\pi^{*}}_{m}(u) $ is also maximal when $ \phi^{\pi^{*}}_{m}(u) $ satisfies (3.7).

Theorem4.1   Assume that $ \pi\in\Pi $, $ V^{\pi}_{m}(u) $ and $ W^{\pi}_{m}(u)(k = 0, 1, \dots) $ are the value functions and their corresponding image functions. Then

$ \begin{align} &\sup\limits_{u}V^{\pi}_{m}(u)\leq \sup\limits_{u}V^{\pi}_{0}(u)+\frac{\sum^{\infty}_{n = 1}\bar{P}(m+n-1)}{\bar{P}(m)}r^{n-1}(kc_{0}-K), m = 0, 1, 2, \dots; \end{align} $

(4.1)

$ \begin{align} &\sup\limits_{u}W^{\pi}_{m}(u)\leq \sup\limits_{u}W^{\pi}_{0}(u)+\frac{\sum^{\infty}_{n = 1}\bar{P}(m+n-1)}{\bar{P}(m)}r^{n}(kc_{0}-K), m = 0, 1, 2, \dots, \end{align} $

(4.2)

Proof   For any positive integer $ m $ and any non-negative integer $ u_{0} $, from (3.2) we have

$ \begin{aligned} V^{\pi}_{m}(u_{0}) = & (k\phi_{m}(u_{0})-K)+\frac{\bar{P}(m+1)}{\bar{P}(m)}rV^{\pi}_{m+1}(u_{0}-\phi_{m}(u_{0})+c)\\ &+\frac{P(m+1)}{\bar{P}(m)}r\sum\limits^{u_{0}-\phi_{m}(u_{0})+c}_{j = 1}V^{\pi}_{0}(u_{0}-\phi_{m}(u_{0})+c-j)f(j).\end{aligned} $

Taking $ \sup_{u} $ on both sides, due to that $ \phi_{n}(u)\leq c_{0}, n = 0, 1, \dots $, we get

$ \begin{aligned} \sup\limits_{u}V^{\pi}_{m}(u) \leq& (kc_{0}-K)+\frac{\bar{P}(m+1)}{\bar{P}(m)}r\sup\limits_{u}V^{\pi}_{m+1}(u) +\frac{p(m+1)}{\bar{P}(m)}r\sup\limits_{u}V^{\pi}_{0}(u).\\ \leq& (kc_{0}-K)+\frac{\bar{P}(m+1)}{\bar{P}(m)}r[(kc_{0}-K)+\frac{\bar{P}(m+2)}{\bar{P}(m+1)}r\sup\limits_{u}V^{\pi}_{m+2}(u)\\ &+\frac{p(m+2)}{\bar{P}(m+1)}r\sup\limits_{u}V^{\pi}_{0}(u)]+\frac{p(m+1)}{\bar{P}(m)}r\sup\limits_{u}V^{\pi}_{0}(u)\\ = &(1+\frac{\bar{P}(m+1)}{\bar{P}(m)}r)(kc_{0}-K)+(\frac{p(m+1)}{\bar{P}(m)}r+\frac{\bar{P}(m+2)}{\bar{P}(m)}r^{2})\sup\limits_{u}V^{\pi}_{0}(u)\\ &+\frac{\bar{P}(m+2)}{\bar{P}(m)}r^{2}\sup\limits_{u}V^{\pi}_{m+2}(u). \end{aligned} $

As is deduced above, we can recursively deduce

$ \begin{aligned} &\sup\limits_{u}V^{\pi}_{m}(u) \leq (1+\frac{\bar{P}(m+1)}{\bar{P}(m)}r+\frac{\bar{P}(m+2)}{\bar{P}(m)}r^{2}+\dots+\frac{\bar{P}(m+n-1)}{\bar{P}(m)}r^{n-1})(kc_{0}-K)\\ &+(\frac{p(m+1)}{\bar{P}(m)}r+\frac{\bar{P}(m+2)}{\bar{P}(m)}r^{2}+\dots+\frac{\bar{P}(m+n)}{\bar{P}(m)}r^{n})\sup\limits_{u}V^{\pi}_{0}(u) +\frac{\bar{P}(m+n)}{\bar{P}(m)}r^{n}\sup\limits_{u}V^{\pi}_{m+n}(u). \end{aligned} $

Letting $ n \rightarrow \infty $ yields

$ \begin{equation} \sup\limits_{u}V^{\pi}_{m}(u)\leq \sum\limits^{\infty}_{n = 1}\frac{\bar{P}(m+n-1)}{\bar{P}(m)}r^{n-1}(kc_{0}-K) +\sum\limits^{\infty}_{n = 1}\frac{\bar{P}(m+n)}{\bar{P}(m)}r^{n}\sup\limits_{u}V^{\pi}_{0}(u), \end{equation} $

(4.3)

from which (4.1) follows. Similarly, from (3.5), we can obtain

$ \begin{equation} \sup\limits_{u}W^{\pi}_{m}(u)\leq \sum\limits^{\infty}_{n = 1}\frac{\bar{P}(m+n-1)}{\bar{P}(m)}r^{n}(kc_{0}-K) +\sum\limits^{\infty}_{n = 1}\frac{\bar{P}(m+n)}{\bar{P}(m)}r^{n}\sup\limits_{u}W^{\pi}_{0}(u), \end{equation} $

(4.4)

from which we see that (4.2) holds.

For any $ F \in l^{\infty} $, we define

$ \begin{equation} \phi_{F}(u) = \arg \max\limits_{d\in \{0, 1, 2, \dots, c_{0}\bigwedge u\}}\{F(u-d)+kd-K\}, u = 0, 1, 2, \dots. \end{equation} $

(4.5)

Theorem 4.2   Given two positive integers $ n $ and $ l $, assume that $ W_{m} (m = 0, 1, \dots) $ are the maximal image functions, $ \tilde{W}_{n+l} $ is an approximation of $ W_{n+l} $ such that $ 0 <\tilde{W}_{n+l}(u)\leq \frac{r}{1-r}(kc_{0}-K) $ for any non-negative integer $ u $. Define $ n+l-1 $ functions $ \tilde{W}_{m}(m = n+l-1, n+l-2, \dots, 1) $ in turn by

$ \begin{eqnarray} \tilde{W}_{m}(u)& = &\frac{\bar{P}(m+1)}{\bar{P}(m)}r[\tilde{W}_{m+1}(u+c-\phi_{\tilde{W}_{m+1}}(u+c))+k\phi_{\tilde{W}_{m+1}}(u+c)-K]\\ &&+\frac{P(m+1)}{\bar{P}(m)}r\sum\limits^{u+c}_{j = 1}[W_{0}(u+c-\phi_{W_{0}}(u+c-j)-j)+k\phi_{W_{0}}(u+c-j)-K]f(j). \end{eqnarray} $

(4.6)

Then, as $ l\rightarrow \infty $,

$ \begin{equation} d(\tilde{W}_{i}, W_{i})\rightarrow 0, \ \ \forall i\leq n. \end{equation} $

(4.7)

Proof   For any positive integer $ m $ such that $ 1\leq m \leq n + l -1 $, we get

$ \begin{equation} d(\tilde{W}_{m}, W_{m})\leq \frac{\bar{P}(m+1)}{\bar{P}(m)}rd(\tilde{W}_{m+1}, W_{m+1}), \end{equation} $

(4.8)

from which we have

$ \begin{equation} d(\tilde{W}_{i}, W_{i})\leq \frac{\bar{P}(n+l)}{\bar{P}(i)}r^{n+l-i}d(\tilde{W}_{n+l}, W_{n+l}) \leq \frac{\bar{P}(n+l)}{\bar{P}(n)}r^{l}d(\tilde{W}_{n+l}, W_{n+l}) \end{equation} $

(4.9)

hold for any positive integer $ i $ such that $ 1\leq i\leq n $. Because of $ d(\tilde{W}_{n+l}, W_{n+l})\leq \frac{r}{1-r}(kc_{0}-K), $ letting $ l\rightarrow \infty $ (4.7) follow from (4.9).

Theorem 4.2 above plays a key role in solving the optimization problem. To obtain the optimal strategy and its value functions, according to (4.7), we choose $ l $ which is great enough, letting $ \tilde{W}_{n+l}(u) = W_{0}(u) $, then we will obtain $ n+l-1 $ functions $ \tilde{W}_{m}(m = n+l-1, n+l-2, \dots, 1) $ by recursion formula (4.6), starting with

$ \begin{equation} \begin{aligned}\tilde{W}_{n+l-1}(u) = &\frac{\bar{P}(m+l)}{\bar{P}(n+l-1)}r[W_{0}(u+c-\phi_{W_{0}}(u+c))+k\phi_{W_{0}}(u+c)-K]\\ &+\frac{P(n+l)}{\bar{P}(n+l-1)}r\sum\limits^{u+c}_{j = 1}[W_{0}(u+c-\phi_{W_{0}}(u+c-j)-j)+k\phi_{W_{0}}(u+c-j)-K]f(j). \end{aligned} \end{equation} $

(4.10)