% From mirica@ecdif.math.unibuc.ro Mon Feb 22 07:40:27 1999
% Date: Mon, 22 Feb 1999 14:44:23 +0200 (EET)
% From: Stefan Mirica <mirica@ecdif.math.unibuc.ro>
% To: sussmann@hamilton.rutgers.edu
% Subject: Paper
%
%Dear Professor Sussman,
%
% Enclosed is the LATEX file (written in "Scientific Workplace" - Windows)
% of my (joint) paper:
%
% "Verification theorems for discontinuous value functions in optimal
% control"
%
% which seems to be related to some of your work.
% 
% Since our libraries can afford to buy only very few Journals and books, I
% would be very greatful if you could send me your recent papers on related
% topics.
%
% Sincerely yours,
%
% Stefan Mirica,
% Faculty of Mathematics,
% University of Bucharest,
% Academiei 14,
% 70109 Bucharest, Romania
%
%% This document created by Scientific Word (R)
%% Version 2.0


\documentclass[12pt,thmsa]{article}
\usepackage{amssymb}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\usepackage{sw20mitp}

%TCIDATA{TCIstyle=article/art1.lat,mitp,mathart1}

%TCIDATA{Created=Thu Sep 10 15:15:00 1998}
%TCIDATA{LastRevised=Tue Jan 19 11:52:04 1999}
%TCIDATA{Language=American English}

\input tcilatex
\begin{document}


\noindent \textbf{Verification Theorems for Discontinuous Value Functions in
Optimal Control\bigskip }

\noindent V. Lupulescu$^{1}$ and \c{S}t. Miric\u {a}$^{2}$\smallskip

\noindent $^{1}${\small Department of Applied Mechanics, ''C.Br\^{a}ncusi''
University of Tg-Jiu, Republicii 1, 1400 Tg-Jiu, Romania\smallskip }

\noindent $^{2}${\small Faculty of Mathematics, University of Bucharest,
Academiei 14, 70109 Bucharest, Romania (mirica@math.math.unibuc.ro)}$%
\bigskip $

\textbf{Abstract}{\small . We prove several new sufficient optimality
conditions of Dynamic Programming type which, together with the existing
ones give a ''list of basic verification theorems'' for value functions
which are least semicontinuous; next, we use certain end-point monotonicity
results to obtain several verification theorems for optimal control problems
whose value functions are neither lower nor upper semicontinuous. The
relevance of the new results is illustrated on a classical example in
Calculus of Variations.\medskip }

\textbf{Key words and phrases}{\small .} {\small Optimal control problems,
differential inclusions, Dynamic Programming, value functions, verification
theorem, contingent derivatives.\medskip }

\textbf{Mathematics Subject Classification}. {\small 49L20; 49J52; 34A60;
26A4}\bigskip

\noindent \textbf{1. Introduction\medskip }

The aim of this paper is to prove several new types of verification theorems
for optimal control problems whose value functions are ``strongly
discontinuous'' in the sense that they are neither lower nor upper
semicontinuous; our work is motivated by a very simple example in classical
Calculus of Variations to which none of the existing verification theorems
seems to be applicable.

In contrast with other approaches of Dynamic Programming, the general
problem of sufficient optimality conditions we are considering in Section 3,
may be formulated as follows: \textit{assuming that a certain selection of
admissible trajectories is given, one looks for the weakest hypotheses on
the associated value function implying the optimality of all the
trajectories of the selection.}

This approach may be interpreted as a return to the theory of ''fields of
extremals'' in classical Calculus of Variations (e.g. Cesari[5]) in which
its main tool, ''the invariance of the Hilbert integral'', is replaced by
differential properties of the value function, which seem to be far more
efficient.

We recall that most authors in the field (e.g. Bardi and
Capuzzo-Dolcetta[4], Cesari[5], Clarke[6], Fleming and Rishel[7],
Frankowska[8], Zhou[17], etc.), take different types of generalized
solutions of the associated Hamilton - Jacobi - Bellman (HJB) equation as
''verification functions'' to\textit{\ check the optimality }of individual
trajectories; on the other hand, not only the natural framework of Dynamic
Programming but also the ''practice'' of solving concrete problems (e.g.
Cesari[5], etc.) are considering the optimization problem for every
''initial point'' in a given subset of the phase space.

We are not considering here the problem of \textit{choosing}
(''describing'', etc.) a ``feasible'' selection of (possibly optimal)
admissible trajectories which is of a totally different nature; however, we
point out that for this first step of Dynamic Programming Method in Optimal
Control, one may use either necessary optimality conditions (when available)
or different types of extensions of Cauchy's Method of Characteristics to
construct ''generalized Hamiltonian flows'' and fields of extremals
([3],[12],[15], etc.) that generate ''feasible'' selections of possibly
optimal trajectories; the experience shows that the later approach, when
applicable, has the advantage of providing certain differential inequalities
for the value function that are essential in the statements of the
verification theorems.

In this setting, the monotonicity property (along the admissible
trajectories) of the value function in optimal control (e.g. [5]-[8],etc.)
reduces the problem of sufficient conditions of Dynamic Programming type to
the apparently more general one of \textit{monotonic functions along
trajectories of differential inclusions} (in particular, control systems).

The main difficulties in the study of this problem are generated by the
strongly nonsmooth character of the value functions of many significant
examples of optimal control problems.

The paper is organized as follows: in Section 2 we present the example that
motivates our work and in Section 3, the Dynamic Programming approach based
on selections of possibly optimal trajectories is presented.

In Section 4 we recall first some existing results and prove several new
ones to obtain a ''list of basic verification theorems'' for value functions
that have certain regularity properties (differentiability, Lipschitzianity,
semicontinuity) on their effective domains and note that in certain cases
none of these results may be applied due, mainly, to the possible irregular
behavior of the value functions at the points of the ``terminal sets''.

The main results of the paper are contained in Section 5, where we use the
''end-point monotonicity results'' in [9] to obtain sufficient optimality
conditions for ''strongly'' discontinuous value functions that may be
neither lower nor upper semicontinuous; we show in Section 6 that, in
particular, these results may be applied to the examples in Section
2.\medskip

\noindent \textbf{2. Motivation\medskip }

As a marginal function of infimum-type (e.g. [1], [2],etc.) one may expect
the value function of an optimal problem to be at least lower semicontinuous
(e.g. [8], Appendix) but the following simple example shows that this may
not be the case.

\textbf{Example 2.1. }In order to use the Dynamic Programming approach, we
reformulate the problem in Calculus of Variations that is studied in Cesari
[5], \S 3.9, as follows: for every ''initial point'' $(s,y)\in E_{0}:=[0,\pi
)\times R,$ minimize the ''cost functional'' $C(.)$ defined by: 
\begin{equation}
C(x(.)):=\int_{s}^{\pi }[-(x(t))^{2}+(x^{\prime }(t))^{2}]dt  \tag{2.1}
\end{equation}
in the class $\Omega _{1}(s,y)$ of ''admissible trajectories'' which are the
absolutely continuous functions $x(.)\in AC([s,\pi ];R)$ satisfying the
conditions: 
\begin{equation}
x(s)=y,\text{ }x(\pi )=0,\;(s,y)\in E_{0}:=[0,\pi )\times R  \tag{2.2}
\end{equation}
and for which the (Lebesgue) integral in (2.1) exists.

Using either the classical Euler-Lagrange equations in Calculus of
Variations (e.g. Cesari[5]), or the Method of Characteristics (e.g. [12],
[15], etc.) for the associated Hamilton-Jacobi-Bellman (HJB) equation 
\begin{equation}
\frac{\partial W}{\partial t}(t,x)-\frac{1}{4}(\frac{\partial W}{\partial x}%
(t,x))^{2}-x^{2}=0,\;(t,x)\in E_{0},\;W(\pi ,0)=0  \tag{2.3}
\end{equation}
one finds the following ``multi-selection'' of extremal (``possibly
optimal'') trajectories: 
\begin{equation}
\begin{array}{l}
\mathcal{A}(s,y)=\left\{ 
\begin{array}{l}
\{[s,\pi ]\ni t\mapsto x_{s,y}(t):=\frac{y}{\sin s}\sin t\}\;if\text{ }%
(s,y)\in E_{0}^{0} \\ 
\{[s,\pi ]\ni t\mapsto x_{q}(t):=q\sin t,\;q\in R\}\text{ }if\text{ }%
(s,y)\in \widetilde{E}_{0}^{0}
\end{array}
\right. \\ 
E_{0}^{0}:=(0,\pi )\times R,\;\widetilde{E}_{0}^{0}=\{(0,0)\},\;\widetilde{E}%
_{0}:=\widetilde{E}_{0}^{0}\cup E_{0}^{0}
\end{array}
\tag{2.4}
\end{equation}
to which one may associate a corresponding ``value function'' defined by: 
\begin{equation}
W_{\mathcal{A}}(s,y):=\left\{ 
\begin{array}{l}
0\text{ \quad \quad }if\text{ }(s,y)\in E_{1}:=\left\{ (\pi ,0)\right\} \\ 
C(x(.))\text{ }if\text{ }(s,y)\in \widetilde{E}_{0}:=\widetilde{E}%
_{0}^{0}\cup E_{0}^{0},\;x(.)\in \mathcal{A}(s,y)
\end{array}
\right.  \tag{2.5}
\end{equation}
which is well defined since for any $q\in R$ one has $C(x_{q}(.))=0,$ and is
given by: 
\begin{equation}
W_{\mathcal{A}}(s,y)=\left\{ 
\begin{array}{l}
0\text{ \quad \quad \quad }if\text{ }(s,y)\in \widetilde{E}_{0}^{0}\cup E_{1}
\\ 
-y^{2}ctg(s)\text{ }if\text{ }(s,y)\in E_{0}^{0}
\end{array}
\right.  \tag{2.6}
\end{equation}

We recall that the classical results in Calculus of Variations allow the
proof of the optimality of the trajectories $x_{s,y}(.),\;(s,y)\in E_{0}^{0}$
in (2.4) in the class $\Omega _{\infty }$ of Lipschitzian trajectories while
for the optimality of the trajectories $x_{q}(.)\in \mathcal{A}(0,0),q\in R,$
one needs several rather involved ''ad hoc'' arguments ([5], \S 3.9); on the
other hand, one may prove directly, that for the initial points $(s,y)\in
E_{0}\setminus \widetilde{E}_{0}=\{0\}\times R^{*}$ ($R^{*}:=R\setminus
\{0\})$ the problem in (2.1)-(2.2) does not have optimal trajectories (e.g.
[5]).

As it is apparent from the results in Section 6 below, the proof of the
optimality of the trajectories in (2.4) becomes much simpler in the
framework of Dynamic Programming, \textit{provided one has suitable
verification theorems to apply; }unfortunately, this is not the case here
since, on one hand, the existing verifications theorems (e.g. [4]-[8],
[11]-[13], [17], etc.) assume the value function to be at least (either
lower or upper) semicontinuous while the function $W_{\mathcal{A}}(.,.)$ in
(2.6), which coincides with the restriction to the subset $\widetilde{E}:=%
\widetilde{E}_{0}\cup E_{1}$ of value function $W(.,.)$, of the problem, it
is neither lower nor upper semicontinuous: 
\begin{equation}
\begin{array}{l}
W(\pi ,0)=0=\stackunder{\widetilde{E}_{0}\backepsilon (s,y)\rightarrow (\pi
,0)}{\lim \inf }W(s,y)<\stackunder{\widetilde{E}_{0}\backepsilon
(s,y)\rightarrow (\pi ,0)}{\lim \sup }W(s,y)=+\infty \\ 
W(0,0)=0=\stackunder{\widetilde{E}_{0}\backepsilon (s,y)\rightarrow (0,0)}{%
\lim \sup }W(s,y)>\stackunder{\widetilde{E}_{0}\backepsilon (s,y)\rightarrow
(0,0)}{\lim \inf }W(s,y)=-\infty
\end{array}
\tag{2.7}
\end{equation}

However, as in many other cases, the restriction $W_{0}(.,.):=W(.,.)|%
\widetilde{E}_{0}$ is upper semicontinuous while at the points in the
''terminal set'', $E_{1},$ the value function has a different type of
''irregular'' behavior.

\textbf{Example 2.2. }The situation is even more complicated if the problem
(2.1)-(2.2) is considered for all initial points in the larger subset: 
\begin{equation}
(s,y)\in E_{0}:=(-\infty ,\pi )\times R  \tag{2.8}
\end{equation}

The same tools mentioned above (i.e. either Euler-Lagrange equations or
Cauchy's Method of Characteristics) produce a similar multi - selection of
''extremal'' trajectories: 
\begin{equation}
\begin{array}{l}
\mathcal{A}(s,y)=\left\{ 
\begin{array}{l}
\left\{ x_{s,y}(.)\right\} \text{ }if\text{ }(s,y)\in \bigcup\limits_{k\in
-N}E_{0}^{k},\;-N=\{0,-1,-2,...\;\} \\ 
\left\{ x_{q}(.);\;q\in R\right\} \text{ }if\text{ }(s,y)\in
\bigcup\limits_{k\in -N}\widetilde{E}_{0}^{k}
\end{array}
\right. \\ 
E_{0}^{k}:=(k\pi ,(k+1)\pi )\times R,\;\widetilde{E}_{0}^{k}=\{(k\pi ,0)\},\;%
\widetilde{E}_{0}:=\bigcup\limits_{k\in -N}[\widetilde{E}_{0}^{k}\cup
E_{0}^{k}]
\end{array}
\tag{2.9}
\end{equation}
where $x_{s,y}(.),\;x_{q}(.)$ are defined as in (2.4), whose value function,
defined as in (2.5), is given by: 
\begin{equation}
W_{\mathcal{A}}(s,y)=\left\{ 
\begin{array}{l}
0\text{ \quad \quad \quad }if\text{ }(s,y)\in \bigcup\limits_{k\in -N}%
\widetilde{E}_{0}^{k}\cup E_{1} \\ 
-y^{2}ctg(s)\text{ }if\text{ }(s,y)\in \bigcup\limits_{k\in -N}E_{0}^{k}
\end{array}
\right.  \tag{2.10}
\end{equation}
and is even more irregular than the one in Example 2.1 since the restriction 
$W_{0}(.,.):=W(.,.)|\widetilde{E}_{0}$ is neither lower nor upper
semicontinuous.

We note that while direct arguments show that problem (2.1)-(2.2) does not
have optimal trajectories for the initial points $(s,y)\in E_{0},\;s<0,$ the
arguments of Dynamic Programming type in Section 6 below show that the
trajectories in (2.9) \textit{are optimal for the problem }(2.1)-(2.2) 
\textit{restricted to the subset }$\widetilde{E}_{0}$ in the sense that the
admissible trajectories are assumed to satisfy the additional
''phase-space'' constraints: 
\begin{equation}
(t,x(t))\in \widetilde{E}_{0}\;\forall \;t\in [s,\pi )  \tag{2.11}
\end{equation}

Therefore, example (2.1)-(2.2) motivates the necessity of looking for
refinements of the existing verification theorems that should be applicable
to strongly discontinuous value functions of the form in (2.6), (2.10).

Other examples of problems with discontinuous values functions are given in
[11] and [16].

We note that problem (2.1)-(2.2) has been considered also in [3], but in an
entirely different context.\medskip

\noindent \textbf{3. A general setting of Dynamic Programming Method in
Optimal Control\medskip }

In the context of Dynamic $\Pr $ogram$\min $g, a general (non-autonomous)
Bolza optimal control problem for differential inclusions (in particular,
for control systems) consists in \textit{minimizing}, for every $(s,y)\in
E_{0}\subset R\times R^{n},$ a cost-functional of the form: 
\begin{equation}
C(x(.)):=g(t_{1},x(t_{1}))+\int_{s}^{t_{1}}f_{0}(t,x(t),x^{\prime }(t))dt 
\tag{3.1}
\end{equation}
over a prescribed set, $\Omega _{a}(s,y)\subseteq AC([s,t_{1}];R^{n})$, of
admissible trajectories which are assumed to be $\Omega _{a}$-type solutions
of a differential inclusion of the form: 
\begin{equation}
x^{\prime }(t)\in F(t,x(t))\text{ }a.e.([s,t_{1}]),\text{ \quad }x(s)=y 
\tag{3.2}
\end{equation}
that satisfy phase-space and, respectively, end-point constraints of the
form: 
\begin{equation}
(t,x(t))\in E_{0}\text{ }\forall \;t\in [s,t_{1})  \tag{3.3}
\end{equation}
\begin{equation}
(t_{1},x(t_{1}))\in E_{1}\subseteq Cl(E_{0})  \tag{3.4}
\end{equation}
and for which the functional $C(.)$ in (3.1) is well defined, i.e. 
\begin{equation}
f_{0}(.,x(.),x^{\prime }(.))\in L^{1}([s,t_{1}];R).  \tag{3.5}
\end{equation}

We note first that the time-interval $[s,t_{1}]$ in (3.1)-(3.4) may not be
fixed and $t_{1}>s$, defined by the ``terminating rule'' in (3.3)-(3.4) is
usually taken as the first moment at which (3.4) is verified, and depends on
the trajectory $x(.)$; on the other hand, the prescribed class, $\Omega _{a}$
of admissible trajectories may be one of the classes, $\Omega _{pc},\Omega
_{r},\Omega _{p},$ $p\in [1,\infty ]$ of absolutely continuous mappings $%
x(.) $, for which the derivatives, $x^{\prime }(.)$ and the composite
functions $f_{0}(.,x(.),x^{\prime }(.))$ are \textit{piecewise continuous if 
}$x(.)\in \Omega _{pc}$, \textit{regulated if }$x(.)\in \Omega _{r}$ and $p$-%
\textit{integrable if} $x(.)\in \Omega _{p}$, $p\in [1,\infty ].$

We note that the explicit distinction between different classes of
admissible trajectories is consistent not only with the so called
''Lavrentiev phenomenon'' (e.g., Cesari[5], \S 18.5) but also with the
different types of hypotheses required in the verification theorems in
Sections 4,5 below.

As it is well known, the Dynamic Programming method consists in using the
basic properties of the \textbf{value function }defined by: 
\begin{equation}
W(s,y):=\left\{ 
\begin{array}{l}
g(s,y)\text{ \quad \quad \quad \quad \quad \quad }if\text{ }(s,y)\in E_{1}
\\ 
\stackunder{x(.)\in \Omega _{a}(s,y)}{\inf }C(x(.))\text{ \quad }if\text{ }%
(s,y)\in E_{0}
\end{array}
\right.  \tag{3.6}
\end{equation}
to obtain necessary and/or sufficient optimality conditions.

In this context, the most efficient way of using Dynamic Programming to
solve concrete problems seems that of using the basic properties of the
value function to check that a \textbf{given multi-selection of admissible
trajectories:} 
\begin{equation}
E_{0}\backepsilon (s,y)\mapsto \mathcal{A}(s,y)\subset \Omega _{a}(s,y) 
\tag{3.7}
\end{equation}
\textbf{is optimal}, in the sense that, for any $(s,y)\in E_{0}$, any
trajectory $\widetilde{x}(.)\in \mathcal{A}(s,y)$ is optimal i.e. 
\begin{equation}
C(\widetilde{x}(.))\leq C(x(.))\text{\ }\forall \;x(.)\in \Omega _{a}(s,y). 
\tag{3.8}
\end{equation}

Therefore, a ``feasible multi-selection'' of possibly optimal trajectories
should have the property: 
\begin{equation}
C(\widetilde{x}(.))=C(\widetilde{y}(.)),\text{ }\forall \;\widetilde{x}(.),%
\widetilde{y}(.)\in \mathcal{A}(s,y),\text{ }(s,y)\in E_{0}  \tag{3.9}
\end{equation}

Obviously, a multi-selection $\mathcal{A}(.,.)$ of admissible trajectories
with the property in (3.9) is optimal iff the \textit{associated value
function,} $W_{\mathcal{A}}(.,.)$, defined by: 
\begin{equation}
W_{\mathcal{A}}(s,.y)=\left\{ 
\begin{array}{l}
g(s,y)\text{ \quad }if\text{ }(s,y)\in E_{1} \\ 
C(\widetilde{x}(.))\text{ \quad }if\text{ }\widetilde{x}(.)\in \mathcal{A}%
(s,y)\text{, }(s,y)\in E_{0}
\end{array}
\right.  \tag{3.10}
\end{equation}
coincides with the value function in (3.6) of the problem.

Using a term, introduced apparently by Fleming and Rishel [7], one usually
calls ``verification theorem'' any result containing hypotheses on $W_{%
\mathcal{A}}(.,.)$ that imply the optimality of the selection $\mathcal{A}%
(.,.)$ of admissible trajectories.

The starting point in obtaining verification theorems is the following
simple result relying on the basic properties of the value function, $W(.,.)$%
, in (3.6) of the problem (3.1)-(3.5):

\textbf{Theorem} \textbf{3.1. }(Abstract verification theorem: Cesari [5],
Fleming and Rishel [7],etc.). \textit{A multi-selection }$\mathcal{A}(.,.)$%
\textit{\ of admissible trajectories with the property in }(3.9)\textit{\ is
optimal if and only if the associated value function, }$W_{\mathcal{A}}(.,.)$%
\textit{\ in }(3.10) \textit{has the following monotonicity property along
the admissible trajectories:}

(M) \textit{For any }$(s,y)\in E_{0}$\textit{\ and any admissible trajectory 
}$x\left( .\right) \in \Omega _{a}(s,y),$\textit{\ the real function, }$%
\omega _{x}(.)$\textit{\ defined by: } 
\begin{equation}
\omega _{x}(t):=W_{\mathcal{A}}(t,x(t))+\int_{s}^{t}f_{0}(\sigma ,x(\sigma
),x^{\prime }(\sigma ))d\sigma ,\text{ }t\in [s,t_{1}]  \tag{3.11}
\end{equation}
\textit{is increasing ( i.e}. $\omega _{x}(s_{1})\leq \omega _{x}(s_{2}),$ $%
\forall \;s\leq s_{1}<s_{2}\leq t_{1}$)%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

Thus, the problem of sufficient optimality conditions of Dynamic Programming
type is reduced to the apparently more general one of ``monotonic functions
along trajectories of differential inclusions'' ( e.g. [1], [8], [11]-[13],
etc.).

In fact, if one denotes: 
\begin{equation}
\begin{array}{l}
\widehat{F}(t,\widehat{x}):=\left\{ (v,f_{0}(t,x,v));\;v\in F(t,x)\right\} ,%
\text{ }(t,x)\in domF,\widehat{x}=(x,x_{0}) \\ 
\widehat{W}(t,\widehat{x}):=W_{\mathcal{A}}(t,x)+x_{0},\text{ }(t,\widehat{x}%
)=(t,(x,x_{0}))\in \widehat{E}:=E\times R
\end{array}
\tag{3.12}
\end{equation}
then the monotonicity property (M) in Theorem 3.1 may equivalently be
formulated as follows: \textit{the function} $\widehat{W}(.,.)$ \textit{in}
(3.12) \textit{is increasing along the trajectories} $\widehat{x}%
(.):=(x(.),x_{0}(.))\in \Omega _{a}$ \textit{of the ``extended differential
inclusion'':} 
\begin{equation}
\widehat{x}^{\prime }\in \widehat{F}(t,\widehat{x})  \tag{3.13}
\end{equation}
that have the properties in (3.3)-(3.5).

\textbf{Remark 3.2. }Certain results on invariance and monotonicity along
solutions of differential inclusions require rather restrictive assumptions
on the data $E,\;F(.,.),\;f_{0}(.,.,.)$ of the problem while the value
function, $W_{\mathcal{A}}(.,.),$ is allowed to be only semicontinuous (e.g.
[1], [8], [11]-[13], etc.).

However, the results in Sections 4,5 below show that the ''list'' of these
type of verification theorems may significantly be enlarged by a careful use
of more sophisticated monotonicity theorems (e.g. [14] and its References),
directly to the real functions $\omega _{x}(.)$ in (3.11), as these results
go far beyond the well known Lebesgue Theorem on monotonicity of absolutely
continuous functions.

\textbf{Remark}\ \textbf{3.3.} It may happen, as it is the case of Examples
2.1, 2.2, that a multi-selection $\mathcal{A}(.,.)$ of admissible
trajectories may be found only at the points in a proper subset $\widetilde{E%
}_{0}\subset E_{0};$ in this case the original problem is replaced by 
\textit{its restriction to }$\widetilde{E}_{0}$ replacing (3.3) by the
following (more restrictive) phase-space constraints: 
\begin{equation}
(t,x(t))\in \widetilde{E}_{0}\;\forall \;t\in [s,t_{1})  \tag{3.14}
\end{equation}

We note that this restriction may be either an \textit{apparent }one, (as it
is the case of Example 2.1) or a \textit{proper restriction, }as in the case
of Example 2.2.

\textbf{Remark 3.4. }If the subset $E_{0}=\widetilde{E}_{0}\subset R\times
R^{n}$ in (3.3) does not coincide with the domain, $domF(.,.)$, of the
orientor field in (3.2) then the initial value problems in (3.2) may have
solutions $x(.)$ that do not satisfy the phase-space constraints in (3.3)
while the monotonicity property (M) in Theorem 3.1 involves only the
admissible trajectories which, by definition, should satisfy not only (3.3)
but also the terminal conditions in (3.4).

On the other hand, under certain conditions, the monotonicity property (M)
in Theorem 3.1 may be reduced to corresponding invariance properties either
of the epigraph or the subgraph of the function $\widehat{W}(.,.)$ in (3.12)
with respect to an associated differential inclusion (e.g. [8], [12], [13],
etc.).\medskip

\noindent \textbf{4. Basic verification theorems for semicontinuous value
functions\medskip }

Since the value function $W(.,.)$ in (3.6) ($W_{\mathcal{A}}(.,.)$ in (3.10)
in our case) is, usually, not differentiable, its domain, $E:=E_{0}\cup
E_{1} $ is neither an open subset nor a differentiable manifold and since
the admissible trajectories, $x(.)$, are at most piecewise differentiable,
realistic conditions implying the monotonicity property (M) in Theorem (3.1)
may be expressed only in the framework of the so called Nonsmooth Analysis
using suitable concepts of tangent cones and corresponding generalized
derivatives ( e.g. [1], [2], [6], [8]-[17], etc.).

In the results to follow we shall use the \textit{contingent} and also the 
\textit{quasitangent cones }to a subset $X\subseteq R^{n}$, defined
respectively, by: 
\begin{equation}
\begin{array}{l}
K_{x}^{\pm }X:=\{v\in R^{n};\exists (\theta _{m},v_{m})\rightarrow (0_{\pm
},v):x+\theta _{m}v_{m}\in X\;\forall m\in N\} \\ 
Q_{x}^{\pm }X:=\left\{ v\in R^{n};\forall \theta _{m}\rightarrow 0_{\pm
}\;\exists v_{m}\rightarrow v:x+\theta _{m}v_{m}\in X\;\forall m\in N\right\}
\end{array}
\tag{4.1}
\end{equation}

For real-valued functions, $f(.):X\subseteq R^{n}\rightarrow R,$ we shall
use the \textit{extreme contingent derivatives }defined by: 
\begin{equation}
\begin{array}{l}
\overline{D}_{K}^{\pm }f(x;v):=\stackunder{(\theta ,u)\rightarrow (0_{\pm
},v)}{\lim \sup }\frac{f(x+\theta u)-f(x)}{\theta },\text{ }v\in K_{x}^{\pm
}X \\ 
\underline{D}_{K}^{\pm }f(x;v):=\stackunder{(\theta ,u)\rightarrow (0_{\pm
},v)}{\lim \inf }\frac{f(x+\theta u)-f(x)}{\theta },\text{ }v\in K_{x}^{\pm
}X
\end{array}
\tag{4.2}
\end{equation}
which coincide with the Dini derivatives if $X\subseteq R$ is an interval
and $v=1\in R.$

Moreover, though the admissible trajectories $x(.)\in \Omega _{a}$ are
differentiable outside a \textit{null subset }$J_{x}\subset [s,t_{1}]$ which
is \textit{countable} \textit{if} $\Omega _{a}=\Omega _{r}$ and \textit{%
finite if} $\Omega _{a}=\Omega _{pc},$ in certain monotonicity results one
needs its (generalized) derivatives at every point; we shall use the
set-valued contingent derivatives defined by: 
\begin{equation}
\begin{array}{l}
K^{\pm }x(t;1):=\{v\in R^{n};\exists \theta _{m}\rightarrow 0_{\pm }:\frac{%
x(t+\theta _{m})-x\left( t\right) }{\theta _{m}}\rightarrow v\} \\ 
K^{\pm }x(t;0_{+}):=\{v\in R^{n};\exists (\theta _{m},r_{m})\rightarrow
(0_{\pm },0_{+}):\frac{x(t+\theta _{m}r_{m})-x\left( t\right) }{\theta _{m}}%
\rightarrow v\}
\end{array}
\tag{4.3}
\end{equation}

For other equivalent definitions, basic properties and the relations of
these concepts with other concepts in Nonsmooth Analysis we refer to [1],
[2], [6], [8]-[17] and the references therein.

We mention here the fact that the contingent derivatives in (4.2), (4.3) are
natural generalizations of the classical (Fr\'{e}chet) derivatives, this
statement being best illustrated by intermediate case of \textbf{contingent}
(''homogeneously'')\textbf{\ differentiable} \textbf{mappings} for which: 
\begin{equation}
\exists \;f_{K}^{\pm }(x;v):=\stackunder{x+\theta u\in X}{\stackunder{%
(\theta ,u)\rightarrow (0_{\pm },v)}{\lim }}\frac{f(x+\theta u)-f(x)}{\theta 
},\text{\ }v\in K_{x}^{\pm }X.  \tag{4.4}
\end{equation}

Another intermediate case is that of ''stratified sets and mappings''
defined as follows:

\textbf{Definition 4.1 }A subset $X\subseteq R^{n}$ is said to be \textit{%
weakly countable }$C^{1}$\textit{-stratified }if it has a countable
partition $\mathcal{S}_{X}=\left\{ S_{j};\;j\in J\right\} $ into $C^{1}$%
-differentiable submanifolds of $R^{n}$; in this case the\textit{\ tangent
space to} $x\in X$ (with respect to the stratification $\mathcal{S}_{X})$ is
defined by: 
\begin{equation}
T_{x}X:=T_{x}S_{j}\text{ }if\text{ }x\in S_{j}\in \mathcal{S}_{X}  \tag{4.5}
\end{equation}
where $T_{x}S_{j}\subset R^{n}$ is the tangent space in the sense of
Differential Geometry.

The mapping $f(.):X\subset R^{n}\rightarrow R^{m}$ is said to be (weakly) 
\textbf{differentiably-stratified} if there exists a $C^{1}$-stratification $%
\mathcal{S}_{X}$ of $X$ such that for any $S_{j}\in \mathcal{S}_{X}$, the
restriction $f_{j}(.):=f(.)\mid S_{j}$ is differentiable ( in the sense of
Differentiable Geometry); in this case, \textit{the derivative }of $f(.)$
with respect to the stratification $\mathcal{S}_{X}$ is defined by: 
\begin{equation}
Df(x):=Df_{j}(x)\in L(T_{x}S_{j},R^{n})\text{ \quad }if\text{ }x\in S_{j}\in 
\mathcal{S}_{X}  \tag{4.6}
\end{equation}

A first set of verification theorems may be obtained directly from different
types of monotonicity theorems (e.g. [14]) applied to the real functions $%
\omega _{x}(.)$ in (3.11), provided the value function $W_{\mathcal{A}}(.,.)$
in (3.10) has enough regularity properties.

This type of results rely on certain ``chain rules'' for composite functions
of the form: 
\begin{equation}
\omega _{x}(t):=\widehat{W}(t,\widehat{x}(t)),\text{ }t\in [s,t_{1}] 
\tag{4.5}
\end{equation}
where $\widehat{W}(.,.)$ is the function in (3.12) and $\widehat{x}%
(.)=(x(.),x_{0}(.))$ is a solution of the extended differential inclusion in
(3.13).

The chain rules we are using in the proof of the next results are expressed
in terms of the extreme contingent derivatives in (4.2) of the value
function $W_{\mathcal{A}}(.,.)$ in (3.10) and of the \textit{relative
contingent directional derivatives }of the admissible trajectories, $x(.),\;%
\widehat{x}(.),$ defined by: 
\begin{equation}
\begin{array}{l}
K_{E}^{\pm }x(t;r):=\{v\in K^{\pm }x(t;r);\;(r,v)\in K_{(t,x(t))}^{\pm
}E\},\;r\in \{1,0_{+}\} \\ 
K_{E}^{\pm }\widehat{x}(t;r):=\{\widehat{v}=(v,v_{0})\in K^{\pm }\widehat{x}%
(t;r);\;(r,v)\in K_{E}^{\pm }x(t;r)\}
\end{array}
\tag{4.7}
\end{equation}

As it is easy to see, the most general chain rules for the functions $\omega
_{x}(.)$ in (3.11) are the following: 
\begin{equation}
\overline{D}_{K}^{\pm }\omega _{x}(t;r)\geq \sup\limits_{\widehat{v}%
=(v,v_{0})\in K_{E}^{\pm }\widehat{x}(t;r)}[\underline{D}_{K}^{\pm }W_{%
\mathcal{A}}((t,x(t));(r,v))+v_{0}]  \tag{4.8}
\end{equation}
while in certain particular cases (e.g. if $W_{\mathcal{A}}(.,.)$ is locally
Lipschitz, $\widehat{x}(.)$ is differentiable at $t,$ etc.) one may obtain
some obvious refinements (e.g.[12], [13], [16], etc.).

In the cases in which the subsets $E_{0}$ and $E:=E_{0}\cup E_{1}$ are not
assumed to be invariant with respect to the trajectories of the differential
inclusion in (3.2), we may use the ``smaller'' orientor fields $F_{Q}^{\pm
}(.,.),F_{Q}(.,.)$ defined by: 
\begin{equation}
F_{Q}^{\pm }(t,x):=\{v\in F(t,x);\;(1,v)\in Q_{(t,x)}^{\pm
}E\},\;F_{Q}(t,x)=F_{Q}^{+}(t,x)\cap F_{Q}^{-}(t,x)  \tag{4.9}
\end{equation}
where $Q_{(t,x)}^{\pm }E$ are the ''unilateral quasitangent cones'' defined
in (4.1).

Everywhere in what follows $\mathcal{A}(.,.)$ is a multi-selection of
admissible trajectories that has the property in (3.9) and $W_{\mathcal{A}%
}(.,.)$ is the associated value function in (3.10); moreover, the function $%
f_{0}(.,.,.)$ in (3.1) is assumed to be continuous.

The next 5 theorems may be considered as variants of certain, more or less
known, results ([5]-[8], [11]-[14], [17], etc.) so we shall omit their
proofs which, in fact, are very similar to the proofs of Theorems 4.7-4.11
below.

A slight refinement of what may be called ``the elementary verification
theorem'' ([5],[7], etc.), in which the value function is usually assumed to
be of class $C^{1},$ is the following:

\textbf{Theorem 4.2} (Elementary verification theorem [12],[13]). \textit{If}
$W_{\mathcal{A}}(.,.)$ \textit{is contingent differentiable ( in the sense
of }(4.4)\textit{) and satisfies the differential inequality:} 
\begin{equation}
\begin{array}{l}
\max \{(W_{\mathcal{A}})_{K}^{+}((t,x);(1,v)),(W_{\mathcal{A}%
})_{K}^{-}((t,x);(1,v))\}+f_{0}(t,x,v)\geq 0 \\ 
\text{ \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad
\quad \quad \quad \quad }\forall \;v\;\in F_{Q}(t,x),\;(t,x)\in E
\end{array}
\tag{4.10}
\end{equation}
\textit{and if} $\Omega _{a}\subseteq \Omega _{r}$ \textit{(i.e. the
admissible trajectories are of least regular) then} $\mathcal{A}(.,.)$%
\textit{\ is optimal}.

\textit{Moreover, if in addition, }$W_{\mathcal{A}}(.,.)$\textit{\ is
locally-Lipschitz }(in particular, of class $C^{1})$ \textit{then the same
conclusion holds for any class} $\Omega _{a}\subseteq AC=\Omega _{1}$, 
\textit{of admissible trajectories}.%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

A significant extension of Theorem 4.1 concerns locally-Lipschitz value
functions that are not necessarily contingent differentiable:

\textbf{Theorem 4.3} (Locally-Lipschitz value functions [12], [13]).\textit{%
\ If} $W_{\mathcal{A}}(.,.)$ \textit{is locally-Lipschitz and verifies the
differential inequality: } 
\begin{equation}
\begin{array}{l}
\max \{\overline{D}_{K}^{+}W_{\mathcal{A}}((t,x);(1,v)),\overline{D}%
_{K}^{-}W_{\mathcal{A}}((t,x);(1,v))\}+f_{0}(t,x,v)\geq 0, \\ 
\text{ \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad \quad
\quad \quad \quad \quad }\forall \;v\in F_{Q}(t,x),\;(t,x)\in E
\end{array}
\tag{4.11}
\end{equation}
\textit{then} $\mathcal{A}(.,.)$ \textit{is optimal}.%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

We note that in the case of Lipschitzian value functions, the inequality in
(4.11) is implied by the fact that $W_{\mathcal{A}}(.,.)$ is a ''generalized
solution in the sense of Clarke'' of the associated HJB equation: 
\[
\min\limits_{(p_{0},p)\in \partial
_{C}W(t,x)}[p_{0}+H(t,x,p)]=0,\;H(t,x,p)=\inf_{v\in
F(t,x)}[<p,v>+f_{0}(t,x,v)]; 
\]
however, for certain particular classes of problems, the value function in
(3.6) is proved to be such a solution ([6], [17], etc.) hence actually, in
such cases these two conditions are equivalent though the one in (4.11) is
apparently weaker and therefore, possibly easier to verify in concrete
examples.

Similarly, for some classes of problems, for which the HJB equation has a
unique ''viscosity solution'', condition (4.11) becomes equivalent with the
fact that $W_{\mathcal{A}}(.,.)$ is such a solution ([4], [17]); however, as
simple examples show, in the general case there may exist viscosity
solutions that do not have the monotonicity property (M) in Theorem 3.1.

An important intermediate case between Theorems 4.2 and 4.3 is that of
stratified value functions in Def.4.1:

\textbf{Theorem 4.4} (Stratified value functions [12], [13]).\textit{\ Let} $%
W_{\mathcal{A}}(.,.)$ \textit{be differentiably - stratified and such that
verifies the following differential inequality: } 
\begin{equation}
\begin{array}{l}
DW_{\mathcal{A}}(t,x)\cdot (1,v)+f_{0}(t,x,v)\geq 0\;\forall \;v\in
F_{T}(t,x),(t,x)\in E \\ 
F_{T}(t,x):=\{v\in F(t,x);\;(1,v)\in T_{(t,x)}E\}
\end{array}
\tag{4.12}
\end{equation}

\textit{If, in addition}, $W_{\mathcal{A}}(.,.)$\textit{\ is continuous and} 
$\Omega _{a}\subseteq \Omega _{r}$\textit{\ then} $\mathcal{A}(.,.)$ \textit{%
is optimal and if} $W_{\mathcal{A}}(.,.)$\textit{\ is} \textit{%
locally-Lipschitz then the same conclusion holds for any class}, $\Omega
_{a}\subseteq \Omega _{1}$ \textit{of admissible} \textit{trajectories}.%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

Other verification theorems using the \textit{upper contingent derivatives}
of the value functions with less regularity properties may be obtained only
at the ''price'' of rather restrictive hypotheses on the data $%
E,\;F(.,.),\;f_{0}(.,.,.),\;$of the problem and using more involved
arguments concerning invariance and monotonicity along solutions of
differential inclusions ([1], [6], [8], [12], [13], etc.):

\textbf{Theorem 4.5} (Lower semicontinuous value functions [12], [13]). 
\textit{Let }$E:=E_{0}\cup E_{1}$\textit{\ be a locally closed subset of }$%
R^{n}$\textit{\ and let }$F(.,.),\;f_{0}(.,.,.)$\textit{\ be such that the
multifunction }$\widehat{F}(.,.)$\textit{\ in }(3.12) \textit{is
locally-Lipschitz with respect to the Hausdorff distance and such that:} 
\begin{equation}
(1,F(t,x))\subset K_{(t,x)}^{-}E\;\forall \;(t,x)\in E,  \tag{4.13}
\end{equation}
\textit{let }$\mathcal{A}(.,.)$\textit{\ be a multi-selection of admissible
trajectories with the property in }(3.9)\textit{\ and such that }$W_{%
\mathcal{A}}(.,.)$\textit{\ in }(3.10)\textit{\ is lower semicontinuous }%
(l.s.c.) \textit{and satisfies the inequality:} 
\begin{equation}
\overline{D}_{K}^{-}W_{\mathcal{A}}((t,x);(1,v))+f_{0}(t,x,v)\geq 0\;\forall
\;v\in F(t,x),\;(t,x)\in E.  \tag{4.14}
\end{equation}
\textit{Then }$\mathcal{A}(.,.)$\textit{\ is an optimal selection of
admissible trajectories}.%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

\textbf{Theorem 4.6} (Upper semicontinuous value functions [8], [12], [13]). 
\textit{Let }$E$\textit{\ be a locally closed subset of }$R^{n}$\textit{\
and let }$F(.,.),\;f_{0}(.,.,.)$\textit{\ be such that the multifunction }$%
\widehat{F}(.,.)$\textit{\ in }(3.12)\textit{\ is locally-Lipschitz\ and
such that: } 
\begin{equation}
(1,F(t,x))\subset K_{(t,x)}^{+}E\;\forall \;(t,x)\in E.  \tag{4.15}
\end{equation}
\textit{If }$W_{\mathcal{A}}(.,.)$\textit{\ is upper semicontinuous (}%
u.s.c.) \textit{and satisfies the inequality:} 
\begin{equation}
\overline{D}_{K}^{+}W_{\mathcal{A}}((t,x);(1,v))+f_{0}(t,x,v)\geq 0\;\forall
\;v\in F(t,x),(t,x)\in E  \tag{4.16}
\end{equation}
\textit{then }$\mathcal{A}(.,.)$\textit{\ is optimal}.%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

We note that the statements in Theorems 4.5, 4.6 are mere interpretations in
the present context of the invariance and monotonicity results in Theorems
4.6, 4.8, in [13].

On the other hand, using the general chain rule in (4.8) and the
monotonicity results for real functions in [14] one may obtain other
variants of the verification theorems above, without the restrictive
hypotheses on the data but expressed in terms of the \textit{lower
contingent derivatives:}

\textbf{Theorem 4.7} (Continuous value functions and regular trajectories). 
\textit{If} $\Omega _{a}\subseteq \Omega _{r}$ \textit{(i.e. the admissible
trajectories are at least regular) and }$W_{\mathcal{A}}(.,.)$\textit{\ is
continuous and verifies \textbf{one} of the following differential
inequalities: } 
\begin{equation}
\begin{array}{l}
\underline{D}_{K}^{+}W_{\mathcal{A}}((t,x);(1,v))+f_{0}(t,x,v)\geq
0\;\forall \;v\in F_{Q}(t,x),(t,x)\in E \\ 
\underline{D}_{K}^{-}W_{\mathcal{A}}((t,x);(1,v))+f_{0}(t,x,v)\geq
0\;\forall \;v\in F_{Q}(t,x),(t,x)\in E
\end{array}
\tag{4.17}
\end{equation}
\textit{then} $\mathcal{A}(.,.)$\textit{\ is an optimal selection}.

\textit{Proof.} In view of Theorem 3.1 it is enough to prove that for any $%
(s,y)\in E_{0}$ and any admissible trajectory $x(.)\in \Omega _{a}(s,y),$
the real function $\omega _{x}(.)$ in (3.11) is increasing. Since $\Omega
_{a}\subseteq \Omega _{r}$ and $f_{0}(.,.,.)$ is assumed to be continuous
there exists a subset $J_{x}\subset (s,t_{1})$ which is at most countable,
such that: 
\begin{equation}
\begin{array}{l}
\exists \;\widehat{x}^{\prime }(t)=(x^{\prime }(t),x_{0}^{\prime }(t))\in 
\widehat{F}(t,x(t))\;\forall \;t\in (s,t_{1})\setminus J_{x} \\ 
x_{0}(t):=\int_{0}^{t}f_{0}(\sigma ,x(\sigma ),x^{\prime }(\sigma ))d\sigma .
\end{array}
\tag{4.18}
\end{equation}

Moreover, since $x(.)$ satisfies (3.3), from the definition in (4.1) of the
quasitangent cone it follows that 
\begin{equation}
(1,x^{\prime }(t))\in Q_{(t,x(t))}E_{0}:=Q_{(t,x(t))}^{+}E_{0}\cap
Q_{(t,x(t))}^{-}E_{0}\;\forall \;t\in (s,t_{1})\setminus J_{x}  \tag{4.19}
\end{equation}
hence from (4.18) it follows: 
\begin{equation}
x^{\prime }(t)\in F_{Q}(t,x(t))\;\forall \;t\in (s,t_{1})\setminus J_{x}. 
\tag{4.20}
\end{equation}

On the other hand, from the chain rule in (4.8) and the fact that if $%
\widehat{x}(.)=(x(.),x_{0}(.))$ then $K_{E}^{\pm }\widehat{x}%
(t;1)=\{(x^{\prime }(t),x_{0}^{\prime }(t))\}\;\forall \;t\in
(s,t_{1})\setminus J_{x},$ it follows: 
\[
\overline{D}_{K}^{\pm }\omega _{x}(t;1)\geq \underline{D}_{K}^{\pm }W_{%
\mathcal{A}}((t,x(t));(1,x^{\prime }(t)))+f_{0}(t,x(t),x^{\prime
}(t))\;\forall t\in (s,t_{1})\setminus J_{x} 
\]
and therefore, if one of the inequalities in (4.17) is verified then,
respectively: 
\[
\overline{D}_{K}^{+}\omega _{x}(t;1)\geq 0,\;\overline{D}_{K}^{-}\omega
_{x}(t;1)\geq 0\;\forall \;t\in (s,t_{1})\setminus J_{x}. 
\]

Finally, since $W_{\mathcal{A}}(.,.)$ is assumed to be continuous, the
function $\omega _{x}(.)$ has the same property and since the ''excepted''
subset $J_{x}\subset (s,t_{1})$ is at most countable, from the corollary of
Zygmund's Lemma (e.g. [14]) it follows that $\omega _{x}(.)$ is increasing
and the theorem is proved.%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

Similar results may be obtained for ( either lower or upper) semicontinuous
value functions using corresponding monotonicity theorems (corollaries 3.7
and 3.8 in [14]); however, in this case one needs an additional assumption
on the orientor field $F(.,.)$ in (3.2).

\textbf{Theorem 4.8} (L.s.c.value functions and regular trajectories). 
\textit{If the multifunction }$F(.,.)$\textit{\ has closed graph, }$\Omega
_{a}\subseteq \Omega _{r}$\textit{\ and }$W_{\mathcal{A}}(.,.)$\textit{\ is
lower semicontinuous and satisfies the inequality}: 
\begin{equation}
\underline{D}_{K}^{-}W_{\mathcal{A}}((t,x);(1,v))+f_{0}(t,x,v)\geq
0\;\forall \;v\in F_{Q}^{-}(t,x),(t,x)\in E  \tag{4.21}
\end{equation}
\textit{then }$\mathcal{A}(.,.)$\textit{\ is optimal.}

\textit{Proof.} Following the same ideas as in the proof of Theorem 4.7,
this time the function $\omega _{x}(.)$ in (3.11) is only lower
semicontinuous and instead of the Corollary of Zygmund's Lemma we use the
monotonicity result in [14], Corollary 3.8 and Remark 3.9 according to
which, if $\omega _{x}(.)$ is lower semicontinuous then it is increasing
iff: 
\begin{equation}
\overline{D}_{K}^{-}\omega _{x}(t;1)\geq 0\;\forall \;t\in (s,t_{1}]. 
\tag{4.22}
\end{equation}

In order to prove that (4.21) implies (4.22) we note that since $\Omega
_{a}\subseteq \Omega _{r}$, the derivatives $x^{\prime }(.),x_{0}^{\prime
}(.)$ are regulated hence they have one-sides limits at each point: 
\begin{equation}
\exists \;\widehat{x}^{\prime }(t\pm )=\lim\limits_{\tau \rightarrow t\pm }%
\widehat{x}^{\prime }(\tau )=\widehat{x}_{\pm }^{\prime
}(t):=\lim\limits_{\theta \rightarrow 0\pm }\frac{\widehat{x}(t+\theta )-%
\widehat{x}(t)}{\theta }\;\forall t\in (s,t_{1}).  \tag{4.23}
\end{equation}

Moreover, since $F(.,.)$ is assumed to have closed graph, from (3.3)-(3.4)
it follows: $x^{\prime }(t\pm )=x_{\pm }^{\prime }(t)\in F_{Q}^{\pm
}(t,x(t))\;\forall \;t\in (s,t_{1})$ hence using the chain rule in (4.8) we
infer that (4.21) implies (4.22) and the theorem is proved.%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

The same type of proof, using the symmetric monotonicity result in [14],
Corollary 3.7 for upper semicontinuous functions leads to the following
analogous result:

\textbf{Theorem 4.9} (U.s.c.value functions and regular trajectories)\textit{%
. If the multifunction }$F(.,.)$\textit{\ has closed graph, }$\Omega
_{a}\subseteq \Omega _{r}$\textit{\ and }$W_{\mathcal{A}}(.,.)$\textit{\ is
upper semicontinuous and satisfies the inequality: } 
\begin{equation}
\underline{D}_{K}^{+}W_{\mathcal{A}}((t,x);(1,v))+f_{0}(t,x,v)\geq
0\;\forall \;v\in F_{Q}^{+}(t,x),(t,x)\in E  \tag{4.23}
\end{equation}
\textit{then }$\mathcal{A}(.,.)$\textit{\ is a selection of optimal
trajectories}.%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

The results in Theorems 4.8, 4.9 may be extended to the larger class, $%
\Omega _{\infty }$, of Lipschitzian trajectories using ``the sets of
contingent directions to the trajectories'' of the differential inclusion in
(3.2) ([10]) defined by: 
\begin{equation}
\begin{array}{l}
K_{F,E}^{\pm }(s,y):=\left\{ v\in R^{n};\;\exists x(.)\in S_{F,E}(s,y):v\in
K^{\pm }x(s;1)\right\} \\ 
K_{F,E}^{0,\pm }(s,y):=\left\{ v\in R^{n};\;\exists x(.)\in
S_{F,E}(s,y):v\in K^{\pm }x(s;0_{+})\right\}
\end{array}
\tag{4.24}
\end{equation}
when $S_{F,E}(s,y)$ denotes the set of absolutely continuous solutions of
the initial value problem in (3.2) whose graphs remain in the subset $%
E\subseteq R\times R^{n}.$

We recall that according to Theorem 3.3 in [10], the sets of contingent
directions in (4.24) have the following ''upper estimates'' 
\begin{equation}
\begin{array}{l}
K_{F,E}^{-}(s,y)\subseteq \underrightarrow{F}(s,y)=\bigcap\limits_{J\in
L_{0},\theta ,r>0}\overline{co}F(((t-\theta ,t]\setminus J)\times
B_{r}(y)\cap E) \\ 
K_{F,E}^{0,-}(s,y)\subseteq \underrightarrow{F}^{\infty
}(s,y)=\bigcap\limits_{J\in L_{0},\theta ,r>0}[0,q)\overline{co}F(((t-\theta
,t]\setminus J)\times B_{r}(y)\cap E)
\end{array}
\tag{4.25}
\end{equation}
where $L_{0}$ denotes the family of subsets $J\subset R$ of zero Lebesgue
measure and $\overline{co}A$ denotes the closed convex hull; similar
estimates are obtained for the sets $K_{F,E}^{+}(s,y),\;K_{F,E}^{0,+}(s,y),$
in terms of the multifunctions $\underleftarrow{F}(s,y),\;\underleftarrow{F}%
^{\infty }(s,y)$ which are defined in the same way using the intervals $%
[t,t+\theta ),\;\theta >0.$

We note also that some ''lower estimates'' and even ''exact
characterizations'' of the sets of contingent directions in (4.24) are given
in [10], Theorems 4.1, 4.3 and Remark 3.4, under certain regularity
assumptions on the orientor field $F(.,.).$

Practically with the same type of proofs as those of Theorems 4.8, 4.9 we
obtain the following results:

\textbf{Theorem 4.10 }(L.s.c.value functions and Lipschitzian trajectories). 
\textit{If }$\Omega _{a}\subseteq \Omega _{\infty }$\textit{\ and }$W_{%
\mathcal{A}}(.,.)$\textit{\ is lower semicontinuous and satisfies the
inequality}: 
\begin{equation}
\underline{D}_{K}^{-}W_{\mathcal{A}}((t,x);(1,v))+v_{0}\geq 0\;\forall \;%
\widehat{v}=(v,v_{0})\in \underrightarrow{\widehat{F}}(t,x),\;(t,x)\in E 
\tag{4.26}
\end{equation}
\textit{where} $\underrightarrow{\widehat{F}}(.,.)$ \textit{is the
multifunction in }(4.25) \textit{associated to the extended orientor field }$%
\widehat{F}(.,.)$ \textit{in }(3.12), \textit{then }$\mathcal{A}(.,.)$%
\textit{\ is an optimal selection.}

\textit{Proof. }Since $x(.)\in \Omega _{\infty }(s,y)$ and $f_{0}(.,.,.)$ is
assumed to be continuous, the extended trajectory, $\widehat{x}%
(.)=(x(.),x_{0}(.))$ in (4.18) is also Lipschitzian hence one has: 
\begin{equation}
\emptyset \neq K^{\pm }\widehat{x}(t;1)\subset K_{\widehat{F},E}^{\pm
}(t,x(t))\subset \underleftrightarrow{\widehat{F}}(t,x)\;\forall \;t\in
(s,t_{1});  \tag{4.27}
\end{equation}
therefore from (4.26) and from the chain rule in (4.8) we obtain the
inequality in (4.21) for the l.s.c. function $\omega _{x}(.)$ in (3.11)
which, according to the monotonicity result in [14] (Corollary 3.8) implies
the fact that $\omega _{x}(.)$ is increasing..%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

A similar proof, using the symmetric monotonicity result, Corollary 3.7 in
[14] leads to the following:

\textbf{Theorem 4.11 }(U.s.c.value functions and Lipschitzian trajectories). 
\textit{If }$\Omega _{a}\subseteq \Omega _{\infty }$\textit{\ and }$W_{%
\mathcal{A}}(.,.)$\textit{\ is upper semicontinuous and satisfies the
inequality}: 
\begin{equation}
\underline{D}_{K}^{+}W_{\mathcal{A}}((t,x);(1,v))+v_{0}\geq 0\;\forall \;%
\widehat{v}=(v,v_{0})\in \underleftarrow{\widehat{F}}(t,x),\;(t,x)\in E 
\tag{4.28}
\end{equation}
\textit{then }$\mathcal{A}(.,.)$\textit{\ is an optimal selection.}%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

\textbf{Remark 4.12} The results in Theorems 4.7-4.11 may be considered less
satisfactory than those in Theorems 4.2-4.6 not only since the classes of
admissible trajectories are severely restricted to $\Omega _{r},\;\Omega
_{\infty }$ but also since the basic differential inequalities in (4.17)-
(4.28) are expressed in terms of the \textit{lower contingent derivatives }
of the value function, which may be difficult to estimate in practical
applications; for instance, in the case the multi-selection $\mathcal{A}%
(.,.) $ is defined by certain generalized Hamiltonian flows ''via'' a
minimizing process (e.g. [12], [15]), one may obtain natural
characterizations only for the upper left contingent derivatives $\overline{D%
}_{K}^{-}W_{\mathcal{A}}((t,x);(.,.)).$ However, the applications to
Examples 2.1 and 2.2 in Section 6 below show that Theorems 4.7-4.11 may be
useful in the study of some classes of problems

\textbf{Remark 4.13 }At least for the\textbf{\ }''non-autonomous'' problems
of the form in (2.1)-(2.5), the list of ''basic verification theorems'' 4.2
- 4.11 seems far from ''complete'' since the Lipschitz property in Theorems
4.5, 4.6 may be relaxed to a certain ''unilateral dissipativity'' property
as in [12], [13]; moreover, it seems highly desirable to extend this type of
results to orientor fields that are only of Carath\'{e}odory - type, i.e.
measurable with respect to the time-variable.

\textbf{Remark 4.14 }The apparently ''cumbersome'' discussion in Theorems
4.2-4.11 of a multitude of regularity properties, generalized derivatives
and classes of admissible trajectories may turn out to be very useful in
applications and is consistent with the so called ''Lavrentiev phenomenon''
(e.g., Cesari [5]) and also with the different types of monotonicity results
for non-differentiable functions (e.g. [13], [14]).$\medskip $

\noindent \textbf{5. End-point monotonicity results and other verification
theorems\smallskip }

The problems in Section 2 as well as many other significant examples (e.g.
[5], [11], [16], etc.) show that the ''basic'' verification theorems
4.2-4.11 in Section 4 are far from sufficient to cover all classes of
optimal control problems.

The main reason seems to be the possible irregular behavior of the value
functions at the points of the terminal sets $E_{1}$ in (3.4), in contrast
with its restriction, $W_{0}(.,.):=W(.,.)\mid E_{0},$ to the set of initial
points, which may have more regularity properties.

Moreover, the same problem in (2.1), (2.8), considered on the subset $%
E_{0}:=(-\infty ,\pi )\times R$ as Example 2.2, shows that even the
restriction, $W_{0}(.,.)$ may be neither lower nor upper semicontinuous.

In order to cover such cases, in [11], [12] has been used the following
additional hypothesis on the value function $W_{\mathcal{A}}(.,.)$ in
(3.10): 
\begin{equation}
\limsup\limits_{E_{0}\ni (s,y)\rightarrow (\tau ,\xi )}W_{\mathcal{A}%
}(s,y)\leq W_{\mathcal{A}}(\tau ,\xi )=g(\tau ,\xi )\;\forall \;(\tau ,\xi
)\in E_{1}  \tag{5.1}
\end{equation}
and the following, rather obvious, ''end-point monotonicity'' result:

\textbf{Proposition 5.1 }\textit{If the restriction }$\omega (.)|[s,t_{1})$ 
\textit{is increasing then }$\omega (.):[s,t_{1}]\rightarrow R$ \textit{is
increasing (on the whole interval, }$[s,t_{1}])$ \textit{iff:} 
\begin{equation}
\limsup\limits_{t\nearrow t_{1}}\omega (t)=\lim\limits_{t\nearrow
t_{1}}\omega (t)\leq \omega (t_{1}).  \tag{5.2}
\end{equation}
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

If $W_{\mathcal{A}}(.,.)$ has the ''relative upper semicontinuity'' property
in (5.1) then from (3.3) it follows that for any admissible trajectory $%
x(.)\in \Omega _{a}(s,y),\;(s,y)\in E_{0},$ the function $\omega _{x}(.)$ in
(3.11) satisfies: 
\[
\limsup\limits_{t\nearrow t_{1}}\omega _{x}(t)\leq \limsup\limits_{E_{0}\ni
(s,y)\rightarrow (t_{1},x(t_{1}))}W_{\mathcal{A}}(s,y)+%
\int_{s}^{t_{1}}f_{0}(\sigma ,x\left( \sigma \right) ,x^{\prime }(\sigma
))d\sigma \leq \omega _{x}(t_{1}) 
\]
and therefore the list of verification theorems in Section 4 may be
completed as follows:

\textbf{Theorem 5.2 (}[11],[12]) \textit{Let }$\mathcal{A}(.,.)$ \textit{be
a multiselection of admissible trajectories satisfying} (3.9), \textit{of
the problem }(3.1)-(3.5), \textit{such that the corresponding value function 
}$W_{\mathcal{A}}(.,.)$ \textit{in }(3.10) \textit{has the following
properties:}

(i) \textit{The restriction }$W_{0}(.,.):=W_{\mathcal{A}}(.,.)|E_{0}$ 
\textit{and the class }$\Omega _{a}$ \textit{of admissible trajectories
satisfy the hypotheses of }\textbf{one }\textit{of the verification theorems 
}4.2 - 4.11;

(ii) $W_{\mathcal{A}}(.,.)$ \textit{has the ''relative upper
semicontinuity'' property in }(5.1).

\textit{Then }$\mathcal{A}(.,.)$ \textit{is an optimal selection.}%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

However, Examples 2.1, 2.2 and possibly many others (e.g. [11], [16], etc.)
show that the hypothesis in (5.1) may not be verified.

In order to cover such cases we shall use in what follows the significant
refinements in [9] of the end-point monotonicity result in Prop.5.1:

\textbf{Theorem 5.3} ([9]) \textit{If }$\omega (.):[s,t_{1}]\subset
R\rightarrow R$\textit{\ is a real function such that the restriction }$%
\omega (.)\mid (s,t_{1})$\textit{\ is increasing then the following
statements hold:}

(i)\textit{\ the restriction }$\omega (.)|(s,t_{1}]$\textit{\ is increasing
iff }\textbf{one }\textit{of the following }(equivalent) \textit{conditions
holds:}

\begin{equation}
\overline{D}_{K}^{-}\omega (t_{1};1)>-\infty ;  \tag{5.3}
\end{equation}

\begin{equation}
\overline{D}_{K}^{-}\omega (t_{1};0_{+})>-\infty .  \tag{5.4}
\end{equation}

(ii) \textit{The restriction }$\omega (.)|[s,t_{1})$ \textit{is increasing
iff }\textbf{one }\textit{of the following }(equivalent) \textit{conditions
holds:} 
\begin{equation}
\overline{D}_{K}^{+}\omega (s;1)>-\infty ;  \tag{5.5}
\end{equation}
\begin{equation}
\overline{D}_{K}^{+}\omega (s;0_{+})>-\infty .  \tag{5.6}
\end{equation}
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

\textbf{Remark 5.4}. The results in [9] show that if $\omega (.)|(s,t_{1})$
is increasing then each of the properties in (5.3), (5.4) is equivalent also
with (5.2) and with each of the following properties: 
\begin{equation}
\underline{D}_{K}^{-}\omega (t_{1};1)>-\infty ,\;\underline{D}_{K}^{-}\omega
(t_{1};0_{+})\geq 0,\;\underline{D}_{K}^{-}\omega (t_{1};0_{+})>-\infty 
\tag{5.7}
\end{equation}
but, as sufficient conditions, they are weaker than the ones in (5.3),
(5.4); moreover, the general chain rule in (4.8) allow estimations only of
the ''upper'' contingent derivatives in (5.3), 5.4).

As a possible better alternative to the result in Theorem 5.2 we shall prove
now:

\textbf{Theorem 5.5 }\textit{Let }$\mathcal{A}(.,.)$ \textit{be a
multiselection of admissible trajectories satisfying} (3.9), \textit{such
that the corresponding value function }$W_{\mathcal{A}}(.,.)$ \textit{in }%
(3.10) \textit{has the following properties:}

(i) \textit{The restriction }$W_{0}(.,.):=W_{\mathcal{A}}(.,.)|E_{0}$ 
\textit{and }$\Omega _{a}$ \textit{satisfy the hypotheses of }\textbf{one }%
\textit{of the verification theorems }4.2 - 4.11;

(ii) \textit{Either }$\Omega _{a}\subseteq \Omega _{\infty }$ (i.e. the
admissible trajectories are Lipschitzian) \textit{and }$W_{\mathcal{A}}(.,.)$
\textit{satisfies the inequality:} 
\begin{equation}
\underline{D}_{K}^{-}W_{\mathcal{A}}((\tau ,\xi );(1,v))>-\infty \;\forall \;%
\widehat{v}=(v,v_{0})\in \underrightarrow{\widehat{F}}(\tau ,\xi ),(\tau
,\xi )\in E_{1}  \tag{5.8}
\end{equation}
\textit{or }$W_{\mathcal{A}}(.,.)$ \textit{satisfies }(5.8) \textit{and also
the following one at each point }$(\tau ,\xi )\in E_{1}$\textit{:} 
\begin{equation}
\underline{D}_{K}^{-}W_{\mathcal{A}}((\tau ,\xi );(0_{+},v))>-\infty
\;\forall \widehat{v}=(v,v_{0})\in \underrightarrow{\widehat{F}}^{\infty
}(\tau ,\xi )\setminus \{(0,0)\}.  \tag{5.9}
\end{equation}
\textit{where }$\underrightarrow{\widehat{F}}(\tau ,\xi ),\underrightarrow{%
\widehat{F}}^{\infty }(\tau ,\xi )$ \textit{are the multifunctions defined in%
} (4.25) \textit{for the extended orientor field in }(3.12).

\textit{Then }$\mathcal{A}(.,.)$ \textit{is an optimal selection.}

\textit{Proof. }If Hypothesis (i) is satisfied then from any of the Theorems
4.2-4.11 and from Theorem 3.1 it follows that the restriction $%
W_{0}(.,.):=W_{\mathcal{A}}(.,.)|E_{0}$ has the monotonicity property (M) in
Theorem 3.1.

Therefore, if $(s,y)\in E_{0}$ and $x(.)\in \Omega _{a}(s,y)$ is an
admissible trajectory then from (3.3) it follows that the function $\omega
_{x}(.)$ in (3.11) is increasing on the interval $[s,t_{1}).$

In view of statement (i) of Theorem 5.3, in order to prove that $\omega
_{x}(.)$ is increasing on the whole interval, $[s,t_{1}],$ it is enough to
prove that it satisfies one of the inequalities (5.3), (5.4).

In the case of Lipschitzian admissible trajectories (i.e. $\Omega
_{a}\subseteq \Omega _{\infty }),$ as it is well known, the contingent
directional derivative in (4.3), (4.24), in direction $1\in R,$ of the
''extended'' trajectory, $\widehat{x}(.)=(x(.),x_{0}(.))$ (where $x_{0}(.)$
is defined in (4.18)) is nonempty and satisfies (4.27) hence from the chain
rule in (4.8) and condition (5.8) it follows that there exists $\widehat{v}%
=(v,v_{0})\in K_{E}^{-}\widehat{x}(t_{1};1)$ such that: 
\[
\overline{D}_{K}^{-}\omega _{x}(t_{1};1)\geq \underline{D}_{K}^{-}W_{%
\mathcal{A}}((t_{1},x(t_{1}));(1,v))+v_{0}>-\infty 
\]
and the theorem is proved in this case.

In the complementary case, in which $\Omega _{a}=\Omega _{p}$ for some $p\in
[1,\infty ),$ one may happen that the first condition in (4.27) is not
satisfied, i.e., $K_{E}^{-}\widehat{x}(t_{1};1)=\emptyset ;$ in this case,
as it is well known (e.g. [9], Prop.3.1 ), since $\widehat{x}(.,.)$ is
(absolutely) continuous, one has: 
\begin{equation}
\emptyset \neq K_{E}^{-}\widehat{x}(t_{1};0_{+})\setminus \{(0,0)\}\subseteq 
\underrightarrow{\widehat{F}}^{\infty }(t_{1},x(t_{1}))  \tag{5.10}
\end{equation}
hence from the chain rule in (4.8) and from (5.9) it follows that there
exists $\widehat{v}=(v,v_{0})\in K_{E}^{-}\widehat{x}(t_{1};0+)\setminus
\{(0,0)\}$ such that: 
\[
\overline{D}_{K}^{-}\omega _{x}(t_{1};0_{+})\geq \underline{D}_{K}^{-}W_{%
\mathcal{A}}((t_{1},x(t_{1}));(0_{+},v))+v_{0}>-\infty 
\]
and the theorem is completely proved.%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

Practically the same proof, using also statement (ii) of Theorem 5.3, leads
to the following refinement of Theorem 5.5:

\textbf{Theorem 5.6 }\textit{Let }$\mathcal{A}(.,.)$ \textit{be a
multiselection of admissible trajectories satisfying} (3.9) \textit{such
that the corresponding value function }$W_{\mathcal{A}}(.,.)$ \textit{in }%
(3.10) \textit{has the following properties:}

(i) \textit{The set }$E_{0}$ \textit{of ''initial points'' admits a
partition }$E_{0}=\widetilde{E}_{0}^{0}\cup E_{0}^{0}$ (i.e. $\widetilde{E}%
_{0}^{0}\cap E_{0}^{0}=\emptyset )$ \textit{such that:} 
\begin{equation}
(t,x(t))\in E_{0}^{0}\;\forall t\in (s,t_{1}),(s,y)\in E_{0}=\widetilde{E}%
_{0}^{0}\cup E_{0}^{0},x(.)\in \Omega _{a}(s,y)  \tag{5.11}
\end{equation}

(ii) \textit{The restriction }$W_{0}^{0}(.,.):=W_{\mathcal{A}%
}(.,.)|E_{0}^{0} $ \textit{and the class, }$\Omega _{a}$ \textit{of
admissible trajectories satisfy the hypotheses of }\textbf{one }\textit{of
the verification theorems }4.2 - 4.11;

(iii) \textit{Either }$\Omega _{a}\subseteq \Omega _{\infty }$ \textit{and }$%
W_{\mathcal{A}}(.,.)$ \textit{satisfies the inequality in }(5.8) \textit{and
also the following one:} 
\begin{equation}
\underline{D}_{K}^{+}W_{\mathcal{A}}((s,y);(1,v))>-\infty \;\forall \;%
\widehat{v}=(v,v_{0})\in \underleftarrow{\widehat{F}}(s,y),\;(s,y)\in 
\widetilde{E}_{0}^{0}  \tag{5.12}
\end{equation}
\textit{or }$W_{\mathcal{A}}(.,.)$ \textit{satisfies the inequalities in }%
(5.8), (5.9), (5.12) \textit{and also the following one, at each point }$%
(s,y)\in \widetilde{E}_{0}^{0}$\textit{:} 
\begin{equation}
\underline{D}_{K}^{+}W_{\mathcal{A}}((s,y);(0_{+},v))>-\infty \;\forall \;%
\widehat{v}=(v,v_{0})\in \underleftarrow{\widehat{F}}^{\infty
}(s,y)\setminus \{(0,0)\}.  \tag{5.13}
\end{equation}

\textit{Then }$\mathcal{A}(.,.)$ \textit{is an optimal selection.}

\textbf{Remark 5.7 }The result in Theorem 5.6 may obviously be extended to
more complex cases like the one in Example 2.2; however, the ''description''
of such cases may become cumbersome so it seems more reasonable to use
Theorem 5.6 and some ''ad hoc'' arguments as we shall do in the proof of
Prop.6.2 below.\medskip

\noindent \textbf{6. Applications\medskip }

We apply first Theorem 5.6 to Example 2.1 in Section 2:

\textbf{Proposition 6.1 }\textit{The multiselection }$\mathcal{A}(.,.)$ 
\textit{in }(2.4) \textit{is optimal for the problem }(2.1)-(2.2) \textit{%
restricted to the subset }$\widetilde{E}_{0}$ \textit{in }(2.4) \textit{in
the class }$\Omega _{\infty }$ \textit{of Lipschitzian trajectories.}

\textit{Proof. }We note first that the partition $\widetilde{E}_{0}=%
\widetilde{E}_{0}^{0}\cup E_{0}^{0}$ in (2.4) has the property in (5.11).

Next, the restriction, $W_{0}^{0}(.,.):=W_{\mathcal{A}}(.,.)|E_{0}^{0}$ of
the value function in (2.6) is obviously of class $C^{1}$ and verifies the
HJB equation in (2.3) (this verification is no more necessary if we use
Cauchy's Method of Characteristics) hence it satisfies the hypotheses of the
Elementary verification Theorem 4.2.

Further on, elementary computations show that the lower contingent
derivatives of the function $W_{\mathcal{A}}(.,.)$ in (2.6) at the
''end-points'' ($\pi ,0),$ and $(0,0)$ are given by: 
\[
\underline{D}_{K}^{-}W_{\mathcal{A}}((\pi ,0);(1,v))=\underline{D}_{K}^{+}W_{%
\mathcal{A}}((0,0);(1,v))=-v^{2}\;\forall \;v\in R 
\]
hence the conditions in (5.8), (5.12) are automatically verified i.e. there
is no need to compute the multifunctions $\underleftrightarrow{\widehat{F}}%
(.,.)$ in (4.25)).%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

We apply now Theorem 5.6 (in fact, Prop.6.1 and Theorem 3.1) to Example 2.2
to obtain the following result:

\textbf{Proposition 6.2 }\textit{The multiselection }$\mathcal{A}(.,.)$ 
\textit{in }(2.9) \textit{is optimal for the problem }(2.1), (2.8), \textit{%
restricted to the subset }$\widetilde{E}_{0}$ \textit{in }(2.9) (\textit{in
the sense of }(2.11)) \textit{over the class }$\Omega _{\infty }$ \textit{of
Lipschitzian trajectories.}

\textit{Proof. }As in the proof of Prop.6.1, we verify first that the
restrictions $W^{k}(.,.):=W_{\mathcal{A}}(.,.)|E^{k},\;k\in -N,$ of the
value function in (2.9) to the subsets $E^{k}:=\widetilde{E}_{0}^{k}\cup
E_{0}^{k}\cup \widetilde{E}_{0}^{k+1},\;k\in -N$ satisfy the hypotheses of
Theorem 5.6. Therefore, from Prop.6.1 and Theorem 3.1 it follows that if $%
(s,y)\in E_{0}^{m}\subset \widetilde{E}_{0},\;m<0$ and $x(.)\in \Omega
_{\infty }(s,y)$ then the restrictions $\omega _{x}(.)|[s,(m+1)\pi
],\;\omega _{x}(.)|[k\pi ,(k+1)\pi ],\;k\in \{m+1,...\;0\},$ are increasing
hence $\omega _{x}(.)$ is increasing on the whole interval $[s,\pi ]$ and
the conclusion follows from Theorem 3.1.%
%TCIMACRO{\TeXButton{End Proof}{\endproof}}
%BeginExpansion
\endproof%
%EndExpansion

\textbf{Remark 6.3 }In the case of Example 2.1, since the restriction $%
W_{0}(.,.):=W_{\mathcal{A}}(.,.)|\widetilde{E}_{0}$ is upper semicontinuous
we may use also Theorems 5.5 and 4.9 to prove Prop.6.1.

On the other hand, it is interesting to note that Theorems 5.5, 5.6 cannot
be used to prove the optimality of the selection $\mathcal{A}$(.,.) in (2.6)
with respect to a larger class, $\Omega _{p},$ $p\in [2,\infty ),$ of
admissible trajectories since one has: 
\[
\underline{D}_{K}^{-}W_{\mathcal{A}}((\pi ,0);(0_{+},v))=\underline{D}%
_{K}^{+}W_{\mathcal{A}}((0,0);(0_{+},v))=-\infty \;\forall v\in R 
\]
and the sets of ''unbounded directions'' in (4.25) of the extended orientor
field 
\[
\widehat{F}(t,\widehat{x}):=\{(v,-x^{2}+v^{2});\;v\in R\},\;(t,\widehat{x}%
)=(t,(x,x_{0}))\in R^{3} 
\]
are given by: $\underleftarrow{\widehat{F}}^{\infty }(0,0)=\underrightarrow{%
\widehat{F}}^{\infty }(\pi ,0)=[0,\infty )\times \{0\}$ hence none of the
conditions (5.9), (5.13) is verified.

Thus, the result in Prop.6.1 matches the one in Cesari [5],\S 3.9 but it
remains still open the problem whether the selection in (2.6) is optimal in
a larger class of admissible trajectories.\medskip

\noindent \textbf{References\medskip }

1. \textbf{J.P. Aubin and A. Cellina (}1984\textbf{)}, \textit{Differential
Inclusions}, Springer-Verlag, New York, Berlin.

2.\textbf{\ J.P. Aubin} \textbf{and} \textbf{H. Frankowska (}1990\textbf{), }%
\textit{Set Valued Analysis, }Birkhauser, Boston\textit{.}

3\textbf{. J. Baranowicz and A. Nowakowski (}1990\textbf{),} \textit{%
Generalized Canonical Formalism with Applications}, Control and Cybernetics,
vol. \textbf{19} (1990), No. 1-2, p. 93-115.

4. \textbf{M. Bardi and I. Capuzzo-Dolcetta }(1997), \textit{Optimal Control
and Viscosity Solutions for Hamilton - Jacobi - Bellman Equations, }%
Birkh\"{a}user, Boston.

5.\textbf{\ L. Cesari (}1983), \textit{Optimization-Theory and Applications}%
, Springer-Verlag, New York.

6. \textbf{F. H. Clarke (}1983\textbf{)}, \textit{Optimization and Nonsmooth
Analysis}, John Wiley, Sons., New York.

7.\textbf{\ W. H. Fleming and R. W. Rishel (}1975\textbf{)}, \textit{%
Deterministic and Stochastic Optimal Control}, Springer-Verlag, New York,
Berlin.

8. \textbf{H. Frankowska (}1989\textbf{)}, \textit{Optimal Trajectories
Associated to a Solutions Contingent Hamilton-Jacobi Equations}, Appl. Math.
Optim., \textbf{19} (1989), p. 291-311.

9\textbf{. V. Lupulescu and \c{S}t. Miric\u {a} }(1998)\textbf{, }\textit{%
End-point monotonicity of real functions and applications, }Anal.
''Ovidius'' Univ. Constan\c{t}a, Mat., (to appear).

10\textbf{. \c{S}t. Miric\u {a} (}1992a\textbf{)}, \textit{Tangent and
Contingent Directions to Trajectories of Differential Inclusions}, Revue
Roum. Math. Appl., \textbf{37}, p. 421-444.

11.\textbf{\ \c{S}t. Miric\u {a} (}1992b\textbf{)}, \textit{Verification
Theorems of Dynamic Programming Type in Optimal Control}, in vol.
``Optimization, Optimal Control and Partial Differential Equations'', V.
Barbu and all. Eds., Birkhauser, Berlin, p. 181-192.

12\textbf{. \c{S}t. Miric\u {a} (}1993\textbf{)}, \textit{Nonsmooth Fields
of Extremals and Constructive Dynamic Programming in Optimal Control}, Part.
I: Autonomous Problems, Preprint N. 18, Dipartimento di Mat. Pura ed. Appl.,
Univ. Padova, Italia.

13\textbf{. \c{S}t. Miric\u {a} (}1995a\textbf{)}, \textit{Invariance and
Monotonicity for Autonomous Differential Inclusions}, Stud Cerc. Mat., Tom
47, nr. \textbf{2}, p. 179-204.

14.\textbf{\ \c{S}t. Miric\u {a} (}1995b\textbf{)}, \textit{Monotonicity of
real functions without regularity assumptions}, Anal. Univ.Bucure\c{s}ti,
Mat., \textbf{44}, p. 49-60.

15\textbf{. \c{S}t. Miric\u {a} }(1996), \textit{Dynamic Programming
solution to Bernoulli's Brachistochrone problem, }Bull. Math. Soc. Sci.
Math. Roumanie, \textbf{39}(87), 221-242.

16.\textbf{\ \c{S}t. Miric\u {a} }(1997), \textit{Optimal feedback control
in closed form via Dynamic Programming, }Revue Roum. Math. Pures Appl. 
\textbf{42, }621-648.

17\textbf{. X.Y. Zhou }(1993), \textit{Verification theorems within the
framework of viscosity solutions, }J. Math. Anal. Appl.., \textbf{177}, p.
208-225.

\end{document}
 
