The IELTS test is a test of English accuracy. Although many people may be very conversant in daily life, the accuracy of their language often needs attention. In the IELTS test, a sentence with basic grammatical mistakes could be considered as example of inability by an examiner.
This is especially true in the speaking test where candidates often forfeit accuracy for quantity in a bid to impress the examiner. Although you should not worry unnecessarily about making sure your grammar is 100% perfect in every sentence you speak, awareness of basic mistakes people make will help you consider your choice of words.