If \vec{a} = \frac{1}{\sqrt{10}}(3\hat{i}+\hat{k}) and \vec{b} = \frac{1}{7}(2\hat{i}+3\hat{j}-6\hat{k}), then the value of (2\vec{a}-\vec{b}).[ (\vec{a} \times \vec{b}) \times (\vec{a}+2\vec{b})] is?

We have (2\vec{a}-\vec{b}).[ (\vec{a} \times \vec{b}) \times (\vec{a}+2\vec{b})]
= (2\vec{a}-\vec{b}).[ (\vec{a} \times \vec{b}) \times \vec{a} + 2(\vec{a} \times \vec{b}) \times \vec{b} ]
= (2\vec{a}-\vec{b}).[ (\vec{a}.\vec{a})\vec{b} - (\vec{a}.\vec{b})\vec{a} + 2(\vec{a}.\vec{b})\vec{b} - 2(\vec{b}.\vec{b})\vec{a} ]
= (2\vec{a}-\vec{b}).[ (\vec{b} - 0 + 0 - 2\vec{a}) ]
= (2\vec{a}-\vec{b}).(\vec{b}-2\vec{a})
= 2\vec{a}.\vec{b} - 4(\vec{a}.\vec{a}) - (\vec{b}.\vec{b}) + 2(\vec{a}.\vec{b})
= 4(\vec{a}.\vec{b}) - 4(\vec{a}.\vec{a}) - (\vec{b}.\vec{b})
Putting values we get = -5
Hence, option '-5' is correct.