I'd try cutting out the psyllium slowly and see if that helps. Remember that all triggers (inc. the harder SFSs) don;t have to take effect straight away to be causing mid-long term issues.
Plus you do need to eat more often...try and get into the habit of making yourself have a teeny bit of SF straight after you wake up. I know going w/out food is actually a big bloat trigger for me.