Ans:
Ellipse $\dfrac{{{x^2}}}{{{a^2}}} + \dfrac{{{y^2}}}{{{b^2}}} = 1$
Let $A B C$, be the triangle inscribed in the ellipse where vertex $C$ is at $(a,0)$. Since the ellipse is symmetrical with $x$-axis and $y - axis$ ${y_1} = \pm \dfrac{b}{a}\sqrt {{a^2} - x_1^2} .$
Coordinates of $A$ are $\left( { - {x_{1,}}\dfrac{b}{a}\sqrt {{a^2} - {x^2}1} } \right)$ and coordinates of $B$ are $\left( {{x_1}, - \dfrac{b}{a}\sqrt {{a^2} - x_1^2} .} \right)$ As the point $\left( { - {x_1},{y_1}} \right)$ lies on the ellipse, the area of triangle $A B C$ is $A = \dfrac{1}{2}\mid a\left( {\dfrac{{2b}}{a}\sqrt {{a^2} - x_1^2} } \right) + \left( { - {x_1}} \right)\left( { - \dfrac{b}{a}\sqrt {{a^2} - x_1^2} } \right) + \left( { - {x_1}} \right)\left( { - \dfrac{b}{a}\sqrt {{a^2} - {x^2}1} } \right)$
$\Rightarrow A = ba\sqrt {{a^2} - {x^2}1} + {x_1}\dfrac{b}{a}\sqrt {{a^2} - x_1^2} $
$\therefore \dfrac{{dA}}{{d{x_1}}} = \dfrac{{ - 2xb}}{{2\sqrt {{a^2} - x_1^2} }} + \dfrac{b}{a}\sqrt {{a^2} - x_1^2} - \dfrac{{2bx_1^2}}{{{a^2}\sqrt {{a^2} - x_1^2} }}$
$= \dfrac{b}{{2\sqrt {{a^2} - x_1^2} }}\left[ { - {x_1}a + \left( {{a^2} - x_1^2} \right) - x_1^2} \right]$
$= \dfrac{{b\left( { - 2x_1^2 - x_1^2 + {a^2}} \right)}}{{a\sqrt {{a^2} - x_1^2} }}$
$\dfrac{{dA}}{{d{x_1}}} = 0$
$\Rightarrow - 2x_1^2 - {x_1}a + {a^2} = 0$
$\Rightarrow {x_1} = \dfrac{{a \pm \sqrt {{a^2} - 4( - 2)\left( {{a^2}} \right)} }}{{2( - 2)}}$
$= \dfrac{{a \pm \sqrt {9{a^2}} }}{{ - 4}}$
$= \dfrac{{a \pm 3a}}{{ - 4}}$
$\Rightarrow {x_1} = - a,\dfrac{a}{2}$
${x_1}$ cannot be equal to $a$. $\therefore {x_1} = \dfrac{a}{2} \Rightarrow {y_1} = \dfrac{b}{a}\sqrt {{a^2} - \dfrac{{{a^2}}}{4}} = \dfrac{{ba}}{{2a}}\sqrt 3 = \dfrac{{\sqrt {3b} }}{2}$
Now,$\dfrac{{{d^2}A}}{{d{x^2}1}} = \dfrac{b}{a}\left\{ {\dfrac{{\sqrt {{a^2} - x_1^2} \left( { - 4{x_1} - a} \right) - \left( { - 2x_1^2 - {x_1}a + {a^2}} \right)\dfrac{{\left( { - 2{x_1}} \right)}}{{2\sqrt {{a^2} - x_1^2} }}}}{{{a^2} - x_1^2}}} \right\}$
$= \dfrac{b}{a}\left\{ {\dfrac{{\left( {{a^2} - x_1^2} \right)\left( { - 4{x_1} - a} \right) + {x_1}\left( { - 2x_1^2 - {x_1}a + {a^2}} \right)}}{{{{\left( {{a^2} - x_1^2} \right)}^{\dfrac{2}{3}}}}}} \right\}$
$= \dfrac{b}{a}\left\{ {\dfrac{{2{x^3} - 3{a^2}x - {a^3}}}{{{{\left( {{a^2} - x_1^2} \right)}^{\dfrac{3}{2}}}}}} \right\}$
when ${x_1} = \dfrac{a}{2}$,
$\dfrac{{{d^2}A}}{{dx_1^2}} = \dfrac{b}{a}\left\{ {\dfrac{{2\dfrac{{{a^3}}}{8} - 3\dfrac{{{a^3}}}{2} - {a^3}}}{{{{\left( {\dfrac{{3{a^2}}}{4}} \right)}^{\dfrac{3}{2}}}}}} \right\} = \dfrac{b}{a}\left\{ {\dfrac{{\dfrac{{{a^3}}}{4} - \dfrac{3}{2}{a^3} - {a^3}}}{{{{\left( {\dfrac{{3{a^2}}}{4}} \right)}^{\dfrac{3}{2}}}}}} \right\}$
$= \dfrac{b}{a}\left\{ {\dfrac{{\dfrac{9}{4}{a^3}}}{{{{\left( {\dfrac{{3{a^2}}}{4}} \right)}^{\dfrac{3}{2}}}}}} \right\} < 0$
Area is the maximum when ${x_1} = \dfrac{a}{2}$. Maximum area of the triangle is $A = b\sqrt {{a^2} - \dfrac{{{a^2}}}{4}} + \left( {\dfrac{a}{2}} \right)\dfrac{b}{a}\sqrt {{a^2} - \dfrac{{{a^2}}}{4}} $
$= ab\dfrac{{\sqrt 3 }}{2} + \left( {\dfrac{a}{2}} \right)\dfrac{b}{a} \times \dfrac{{a\sqrt 3 }}{2}$
$= \dfrac{{ab\sqrt 3 }}{2} + \dfrac{{ab\sqrt 3 }}{4} = \dfrac{{3\sqrt 3 }}{4}ab$