If I go to jail and manage to get my stuff and break out without the guards seeing me, will the guards outside of the jail attempt to attack/arrest me again, or will they just leave alone for a while and say that famous, "Wait, I know you!" (xd)
12 Answers
If you escape from jail, your bounty will remain, plus you risk an additional 100 gold bounty if you're caught while trying to get out.
When you perform a jailbreak a bounty of 100 gold is added to your current bounty - this happens regardless of getting caught. After completing the jailbreak guards will not interrogate or attack you, only saying "Wait, I know you!".
There was conflicting information about this online so I tested it without Mods on a fresh character. I stole a knife in Riverwood (9 gold bounty), ran to Whiterun and talked to a guard, got sent to prison, escaped, grabbed items on way out, ran around town jumping in front of guards with 109 gold bounty.
To remove the bounty you'll need to pay it. Or,
If you are part of the thieves guild and have enough reputation in the specific hold there will be a bribe a guard option to pay the bounty at a discount. If you use this bounty payment option you can immediately steal back the bounty from the guard (so I've read online).
There's also wikia guide on how to escape each of the jails.
