Roles and Permissions

I am looking for a little assistance actually an example would be nice. I have created a HR Category that I want to hide from normal requester and task roles. I created a new task user role for HR that part I had no issues with however I must be missing something because I can not get that category to not be visible for non HR members. I would rather hide a category then to restrict by department. Maybe I have just been staring at the screen to long, any help is appreciated.