Overrides are specific permissions designed to override a role in a specific context, allowing you to "tweak" your permissions as required.
For example, if users with the role Student in your course can usually start new discussions in forums, but there is one particular forum for which you want to restrict that capability, then you can set an override that PREVENTS the capability for Students to "Start new discussions".
Overrides can also be used to "open up" areas of your site and courses to give users extra permissions where it makes sense. For example, you may want to experiment giving Students the ability to grade some assignments.
The interface is similar to the one for defining roles, except sometimes only relevant capabilities are shown, and you will also see some capabilities highlighted to show you what the permission for that role would be WITHOUT any override active (ie when your override is set to INHERIT).