Jesse, to me it sounds like a vaccum leak and another possibility is intake manifold gasket leaking.
Have a look in the workshop manual for the vaccum layout, will show you all the hoses - make sure they are all good.
To search for leaks in intercooler piping and manifolds, grab a can of Aerostart or similar and with the engine running, spray around all the joints in the cooler piping and intake manifold - if the revs pick up you've found your leak.