Bogart is recruited for the bank robbery because he is supposed to be some sort of mastermind in planning jobs. Trouble is the plan they come up with to rob the armored car is crap. It's a crappy plan! There's nothing that makes it very clever or likely to succeed. And I think they would need a spotter to be able to tell just when the armored car is coming down the street to know when to go. From their starting position they couldn't possibly see the armored car coming.
Then Bogart makes a plan to break out of prison. Again the plan is crap! Its a terrible plan and again with nothing clever that would make it likely to succeed.
And as far as their hiding place goes, its pretty but wouldn't the first thing Duke would have asked (if he's such a mastermind) is "does anybody else know about this place?"
There's also the embarrassment of fellow convict Dancer dressing in blackface and dancing with a black doll that makes this film probably unairable in most markets today.