##A = (25sqrt(3))/2##

First, let’s look at a picture.

Some initial observations:

• The area ##A## of the rectangle is ##A=bh##.

• By symmetry, the base of the triangle is of length ##b+2t##, and thus, as it is of length ##10##, we have ##b+2t = 10 => t = 5-b/2##

• If we decide ##b## that also determines ##h##, and thus we can write ##h## as a function of ##b##.

To write ##h## as a function of ##b##, we can look at the right triangle with legs ##t## and ##h##. As it shares an angle with an equilateral triangle, we know it is a ##30^@##-##60^@##-##90^@## triangle, and thus ##t/h = 1/sqrt(3)##. Solving for ##h## gives us ##h = sqrt(3)t = sqrt(3)(5-b/2)## by our initial observation.

Then, we can rewrite our formula for the area as

##A = b*sqrt3(5-b/2) = sqrt(3)(-1/2b^2 + 5b)##

If we look at the graph for ##A## we will see it is a downward facing parabola, and thus will have a maximum at its vertex. Then, we can complete the square to find

##A = sqrt(3)(-1/2b^2+5b)## ## = -sqrt(3)/2(b^2-10b)## ## = -sqrt(3)/2((b-5)^2-25)##

And so the vertex, and thus the maximum area, is at ##b = 5##.

Finally, we calculate the area from this to get

##A = -sqrt(3)/2((5-5)^2-25) = (25sqrt(3))/2##