JEE Advanced 2024, Paper 1. Previous Year Question.
Problem
Let $x\in\left(\dfrac{\pi}{2},\pi\right)$ be such that $\cot x = -\dfrac{5}{\sqrt{11}}$. Then
$$\left(\sin\frac{11x}{2}\right)\left(\sin6x-\cos6x\right) + \left(\cos\frac{11x}{2}\right)\left(\sin6x+\cos6x\right)$$
is equal to:
Key insight. Expanding the given expression term by term and regrouping reveals two hidden angle-sum/difference identities — the whole unwieldy expression collapses to just $\cos(x/2)+\sin(x/2)$. Squaring that sum turns it into $1+\sin x$, which is easy to find once $\cot x$ pins down a right triangle in the correct quadrant.
Watch this explained step by step → · More on the Shiwam’s Classes channel
Approach
Expand the given expression and regroup terms to reveal the angle-sum and angle-difference formulas for cosine and sine, collapsing everything to $\cos(x/2)+\sin(x/2)$. Square this sum to relate it to $1+\sin x$, find $\sin x$ from the given $\cot x$ (using the correct quadrant), and simplify the resulting nested square root by recognising it as a perfect square.
Solution
Step 1 — Expand the given expression
$$\sin\frac{11x}{2}\sin6x – \sin\frac{11x}{2}\cos6x + \cos\frac{11x}{2}\sin6x + \cos\frac{11x}{2}\cos6x$$
Step 2 — Group into two recognisable identities
Grouping the cos-cos and sin-sin terms:
$$\cos\frac{11x}{2}\cos6x+\sin\frac{11x}{2}\sin6x = \cos\left(6x-\frac{11x}{2}\right) = \cos\left(\frac{x}{2}\right)$$
Grouping the remaining two terms:
$$\cos\frac{11x}{2}\sin6x – \sin\frac{11x}{2}\cos6x = \sin\left(6x-\frac{11x}{2}\right) = \sin\left(\frac{x}{2}\right)$$
Step 3 — Combine
$$\text{Expression} = \cos\frac{x}{2}+\sin\frac{x}{2}$$
Step 4 — Square this sum
$$\left(\cos\frac{x}{2}+\sin\frac{x}{2}\right)^2 = 1+2\sin\frac{x}{2}\cos\frac{x}{2} = 1+\sin x$$
Step 5 — Find sin x from cot x
Since $\cot x=-\dfrac{5}{\sqrt{11}}$ and $x$ is in the second quadrant ($\sin x>0$, $\cos x<0$): using a right triangle with opposite $=\sqrt{11}$, adjacent $=5$ (magnitude), hypotenuse $=\sqrt{11+25}=\sqrt{36}=6$:
$$\sin x = \frac{\sqrt{11}}{6}$$
Step 6 — Substitute into 1 + sin x
$$1+\sin x = 1+\frac{\sqrt{11}}{6} = \frac{6+\sqrt{11}}{6}$$
Step 7 — Simplify the square root
$$\left(\cos\frac{x}{2}+\sin\frac{x}{2}\right)^2 = \frac{6+\sqrt{11}}{6}$$
Multiplying numerator and denominator by $2$:
$$\frac{12+2\sqrt{11}}{12} = \frac{11+2\sqrt{11}+1}{12} = \frac{(\sqrt{11}+1)^2}{12}$$
Since $\dfrac{x}{2}\in\left(\dfrac{\pi}{4},\dfrac{\pi}{2}\right)$, both $\sin\dfrac{x}{2}$ and $\cos\dfrac{x}{2}$ are positive, so the square root is positive:
$$\cos\frac{x}{2}+\sin\frac{x}{2} = \sqrt{\frac{(\sqrt{11}+1)^2}{12}} = \frac{\sqrt{11}+1}{\sqrt{12}} = \frac{\sqrt{11}+1}{2\sqrt3}$$
Answer
$$\frac{\sqrt{11}+1}{2\sqrt3}$$
Common mistakes
- Missing that the nested square root is secretly a perfect square. Trying to evaluate $\sqrt{\frac{6+\sqrt{11}}{6}}$ directly, without first scaling by $2$ to reveal the $(\sqrt{11}+1)^2$ pattern, leaves the answer in an unrecognisable form that doesn’t match any option.
- Using the wrong sign for $\cos x$ or $\sin x$. Since $x$ is in the second quadrant, $\cos x$ is negative but $\sin x$ is positive — getting this backwards would flip the sign inside the square root and produce a completely different result.
Practise next
- Let $y\in\left(0,\dfrac{\pi}{2}\right)$ be such that $\tan y=\dfrac{3}{\sqrt5}$. Evaluate $\left(\cos\dfrac{9y}{2}\right)\left(\cos4y+\sin4y\right)-\left(\sin\dfrac{9y}{2}\right)\left(\cos4y-\sin4y\right)$, using the same expand-and-regroup approach.
Show answer
$\sqrt{1-\dfrac{3}{\sqrt{14}}}\approx0.4452$. Expanding and regrouping is the whole exercise:
$$\left[\cos\tfrac{9y}{2}\cos4y+\sin\tfrac{9y}{2}\sin4y\right]+\left[\cos\tfrac{9y}{2}\sin4y-\sin\tfrac{9y}{2}\cos4y\right]=\cos\tfrac{y}{2}-\sin\tfrac{y}{2}.$$
Squaring gives $1-\sin y$, and $\tan y=\tfrac{3}{\sqrt5}$ means $\sin y=\tfrac{3}{\sqrt{14}}$. Since $y\in\left(0,\tfrac{\pi}{2}\right)$ we have $\tfrac{y}{2}<\tfrac{\pi}{4}$, so the cosine exceeds the sine and the root is positive.
Hence the value is $\sqrt{1-\tfrac{3}{\sqrt{14}}}$. The nine-halves and four-fold angles all disappear in the regrouping — they never have to be evaluated.

Doubts are answered by Shiwam, usually within a day. Ask about this question specifically — a general question about the chapter is better asked in class.