I'm not sure you can do that. You can set the alert logic so that the warning is reset when either the problem is resolved OR the condition moves into the critical range (and ditto for critical).
But what you are asking for is essentially to put logic on the alert trigger itself (send email UNLESS disk utilization > 96%), which is not something I think can be done.
but I can't wait to see if any of the other Thwack-izens will prove me wrong!
I can't seeing it being done either within Alert Manager. I can't see how you can have an alert that relies on the status of another alert.
I cannot admit to using this feature yet in production but I remember during beta testing the web-based alert manager's complex conditions for this exact sort of situation.
this is a screenshot of the 2nd condition after the first but I don't think you can control the actions from this - you would still need to reset the warning alert before it stops it's actions and critical takes over.