NCERT Class 12 Mathematics — Matrices, Exercise 3.4. The single question solved.
Exercise 3.4 is the shortest in the book: one multiple-choice question. It used to contain a long run of elementary row and column operation problems, but those were removed when the syllabus was rationalised, and only this item remains.
What it tests is the definition of an inverse. For square matrices A and B of the same order, B is the inverse of A when
$$\mathrm{AB} = \mathrm{BA} = \mathrm{I}$$
Key insight. Both products are required, and both must give the identity. That is the whole question. Because matrix multiplication does not commute, it is not automatic that $\mathrm{AB} = \mathrm{BA}$, so the definition has to demand it rather than assume it.
Question 1
Matrices A and B will be inverse of each other only if
Solution. Take the options one at a time.
(A) $\mathrm{AB} = \mathrm{BA}$ says only that A and B commute. That is far weaker than being inverses — for instance $\mathrm{A} = \mathrm{B} = \mathrm{I}$ commute, but so do $\mathrm{A} = \mathrm{I}$ and $\mathrm{B} = 2\mathrm{I}$, which are not inverses of each other.
(B) $\mathrm{AB} = \mathrm{BA} = \mathrm{O}$ is the opposite of what is wanted. If a matrix had an inverse and their product were the zero matrix, then multiplying $\mathrm{AB} = \mathrm{O}$ by $\mathrm{A}^{-1}$ would force $\mathrm{B} = \mathrm{O}$, and the zero matrix has no inverse.
(C) $\mathrm{AB} = \mathrm{O}$, $\mathrm{BA} = \mathrm{I}$ cannot happen at all for square matrices. If $\mathrm{BA} = \mathrm{I}$ then both are invertible, and multiplying $\mathrm{AB} = \mathrm{O}$ on the left by $\mathrm{B}^{-1}$ and on the right by $\mathrm{A}^{-1}$ gives $\mathrm{I} = \mathrm{O}$, which is false.
(D) $\mathrm{AB} = \mathrm{BA} = \mathrm{I}$ is exactly the definition given in the chapter, and is therefore correct.
As a concrete check, take $\mathrm{A} = \begin{bmatrix} 2 & 3 \\ 1 & 2 \end{bmatrix}$ and $\mathrm{B} = \begin{bmatrix} 2 & -3 \\ -1 & 2 \end{bmatrix}$:
$$\mathrm{AB} = \begin{bmatrix} 4-3 & -6+6 \\ 2-2 & -3+4 \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} = \mathrm{I}$$
and $\mathrm{BA} = \mathrm{I}$ as well, so these two are genuinely inverses.
$$\text{(D)}\quad \mathrm{AB} = \mathrm{BA} = \mathrm{I}$$
Common mistakes
- Settling for $\mathrm{AB} = \mathrm{I}$ alone. The definition asks for both products. For square matrices one does in fact imply the other, but that is a theorem, not the definition, and option (A) is designed to catch anyone reasoning loosely about it.
- Confusing “commute” with “inverse”. Option (A) states commuting only. Every pair of inverses commutes, but not every commuting pair is a pair of inverses.
- Reading O for I. Options (B) and (C) differ from (D) only in swapping the zero matrix for the identity — read the symbols carefully under exam pressure.
Practise next
- Miscellaneous Exercise on Chapter 3 — proofs about symmetric matrices and matrix equations.
- Chapter 4, Exercise 4.4 — how an inverse is actually computed, from the determinant and the adjoint.

Doubts are answered by Shiwam, usually within a day. Ask about this question specifically — a general question about the chapter is better asked in class.