When Jim is talking to Logan Barrett in the prison yard he is shown with his sidearm. In an open prison yard of that sort he would not be permitted to take any sort of weapon with him as he could easily be overrun by inmates who would now have a firearm.