Alignment when using android phone

I am not techy. I am not an astronomer. I simply stumbled across this app by accident but think it is EXCELLENT especially for my fairly rudimentary needs. However, on occasion when using the free android plain and simple basic app, and holding my phone up to the sky (for example looking at a known item such as Jupiter) the phone image is shifted over to one side. So, if (as a rough example) Jupiter were due south, my arm outstretched holding the phone also due south (with Jupiter behind it) then the phones image of Jupiter would not show. I would have to push my arm to the left (or right) a number of degrees. As if the calibration/alignment is off. Is it merely a case of walking around a little to get the compass or GPS to realign?