Unit Vector Perpendicular to b, c at Angles cos⁻¹(-1/3) and π/3

JEE Main 2025 — 29 January, Evening Shift. Previous Year Question.

Problem

Let $\hat{a}$ be a unit vector perpendicular to the vectors $\vec{b} = \hat{i}-2\hat{j}+3\hat{k}$ and $\vec{c} = 2\hat{i}+3\hat{j}-\hat{k}$, and let $\hat{a}$ make an angle of $\cos^{-1}\left(-\tfrac{1}{3}\right)$ with the vector $\hat{i}+\hat{j}+\hat{k}$. If $\hat{a}$ makes an angle of $\tfrac{\pi}{3}$ with the vector $\hat{i}+\alpha\hat{j}+\hat{k}$, then the value of $\alpha$ is:

Key insight. A unit vector perpendicular to two given vectors has to be built from their cross product — but a cross product gives a direction, not a sign, so the first angle condition exists purely to tell you which of the two possible directions ($+$ or $-$) is the right one. Only after that is settled does the second angle condition become solvable for $\alpha$.

Watch this explained step by step →  ·  More on the Shiwam’s Classes channel

Approach

Any vector perpendicular to both $\vec{b}$ and $\vec{c}$ must be a scalar multiple of $\vec{b} \times \vec{c}$, so the unit vector $\hat{a}$ is $\pm \dfrac{\vec{b}\times\vec{c}}{|\vec{b}\times\vec{c}|}$ — the sign is genuinely unknown until fixed by a condition. That’s exactly what the first angle condition, $\cos^{-1}(-\tfrac13)$ with $\hat{i}+\hat{j}+\hat{k}$, is for: substituting both sign choices into the dot-product angle formula reveals which one actually gives $-\tfrac13$. With the sign fixed, $\hat{a}$ is a fully known vector, and the second angle condition becomes a single equation in $\alpha$.

Solution

Step 1 — Compute $\vec{b} \times \vec{c}$

$$ \vec{b} \times \vec{c} = \begin{vmatrix} \hat{i} & \hat{j} & \hat{k} \\ 1 & -2 & 3 \\ 2 & 3 & -1 \end{vmatrix} = \hat{i}(2-9) – \hat{j}(-1-6) + \hat{k}(3+4) = -7\hat{i} + 7\hat{j} + 7\hat{k} $$

Step 2 — Find the unit vector, with an undetermined sign

The magnitude is $|\vec{b}\times\vec{c}| = \sqrt{7^2+7^2+7^2} = 7\sqrt{3}$, so the unit vector perpendicular to both $\vec{b}$ and $\vec{c}$ is:

$$\hat{a} = \pm\left(\frac{-\hat{i}+\hat{j}+\hat{k}}{\sqrt{3}}\right)$$

Since flipping the sign only reverses the direction and leaves the magnitude unchanged, both signs are genuinely possible so far — the next condition decides between them.

Step 3 — Use the first angle condition to fix the sign

Taking the dot product of $\hat{a}$ with $\hat{i}+\hat{j}+\hat{k}$ (which has magnitude $\sqrt{3}$) and setting it equal to $|\hat{a}| \cdot \sqrt{3} \cdot \cos\theta$:

$$\pm\left(\frac{-1}{\sqrt{3}} + \frac{1}{\sqrt{3}} + \frac{1}{\sqrt{3}}\right) = \sqrt{3}\cos\theta$$

The left side simplifies to $\pm\dfrac{1}{\sqrt{3}}$, and since $|\hat{i}+\hat{j}+\hat{k}| = \sqrt{3}$, the right side is just $\sqrt{3}\cos\theta$, so after dividing through, the equation reads $\pm\dfrac{1}{3} = \cos\theta$. Since we’re told $\cos\theta = -\dfrac{1}{3}$, only the minus sign is consistent. So:

$$\hat{a} = \frac{\hat{i}}{\sqrt{3}} – \frac{\hat{j}}{\sqrt{3}} – \frac{\hat{k}}{\sqrt{3}}$$

Step 4 — Apply the second angle condition to solve for $\alpha$

Now dot $\hat{a}$ with $\hat{i}+\alpha\hat{j}+\hat{k}$, and set the result equal to $|\hat{a}|\cdot|\hat{i}+\alpha\hat{j}+\hat{k}|\cdot\cos\tfrac{\pi}{3}$:

$$\frac{1}{\sqrt{3}} – \frac{\alpha}{\sqrt{3}} – \frac{1}{\sqrt{3}} = \sqrt{1+\alpha^2+1}\times\frac{1}{2}$$

The left side simplifies to $-\dfrac{\alpha}{\sqrt{3}}$, giving:

$$-\frac{\alpha}{\sqrt{3}} = \frac{\sqrt{2+\alpha^2}}{2}$$

Squaring both sides to clear the square root:

$$\frac{\alpha^2}{3} = \frac{2+\alpha^2}{4} \implies 4\alpha^2 = 3(2+\alpha^2) \implies \alpha^2 = 6 \implies \alpha = \pm\sqrt{6}$$

Step 5 — Choose the sign that satisfies the original (unsquared) equation

Squaring can introduce extraneous roots, so both signs need checking against $-\dfrac{\alpha}{\sqrt{3}} = \dfrac{\sqrt{2+\alpha^2}}{2}$ directly. The right-hand side is always positive (it’s a square root), so the left-hand side must be positive too — which means $\alpha$ itself must be negative. Only $\alpha = -\sqrt{6}$ satisfies this.

Answer

Common mistakes

• Forgetting the cross product’s sign ambiguity. A vector perpendicular to two others always has two possible directions; skipping the first angle condition and guessing a sign risks the wrong final answer even with otherwise correct algebra.
• Not checking which root survives squaring. Squaring $-\dfrac{\alpha}{\sqrt3}=\dfrac{\sqrt{2+\alpha^2}}{2}$ produces both $\pm\sqrt6$, but only one of them keeps the left side’s sign consistent with the right side, which is always non-negative.

Practise next

• Let $\hat{a}$ be a unit vector perpendicular to $\vec{u}=\hat{i}+\hat{j}$ and $\vec{v}=\hat{j}+\hat{k}$, making an obtuse angle with $\hat{i}+\hat{k}$. Find $\hat{a}$, and then the angle it makes with $\hat{i}+\hat{j}+\hat{k}$. The sign-fixing step is the same one this problem needs.