This can be caused by two things. One is caused by high differential currents travelling between the battery and the ESCs - those large red and black wires on your power spider cable. If these are too close to the magnetometer then the sensor will read these differential currents like a magnet affects a compass.

  We typically don't think DC current creates an electromagnetic field but during changes in throttle or motor demand the amount of amperage changes thus causing Alternating Current, even though the voltage stays constant. Only during the change in current draw does an electromagnetic field develop. Once amperage stabilizes the system returns to DC current laws and thus no EM field.

You should be able to solve this by adding more space (standoffs) between the center plate and board. You may also be able to use EM shielding (try tin foil) but this will shield the sensor from the earth field too depending on where the shielding is and the copter's orientation. Basically, the shield may create a shadow in the earth's EM field and depending on the quad orientation the sensor could end up in this shadow. Perhaps shield the spider cable itself, like cable TV coax cable shields the inner core. The shield should be tied to GND as well so the EM currents have somewhere to go.

The other reason could be an imbalance of the copters' props, motor output, or thrust vectors because of any of the following:

1. damaged or unbalanced props
2. bent motor shafts - possibly caused by crashes or improper torquing of the motor shaft during prop installation.
3. bad/worn bearings or fatigued motors
4. frame out of alignment or loose - the props thrust vectors are not all pointed directly up

Diagnose and correct each of these issues. A prop balancer can be bought cheap on many RC sites. Bent motor shafts and worn bearings can often be diagnosed by hearing a rattling sound when you spin the prop/motor by hand. The motor rotation will sound different than a motor and prop in good condition.

To check frame alignment, place the copter up-side-down onto a flat table. Turn each prop slowly to check that the prop is always parallel to the table surface as it completes a rotation. Usually it suffices to correct mis-alignment by torquing the frame back into alignment and retighten screws - then recheck. A properly aligned frame does make a big difference in overall flight performance! 

A loose frame can be secured using a neat RC modelers trick. Fill the loose frame connections with baking soda, then add a drop of crazy glue to the baking soda. When baking soda and crazy glue mixes it forms a plastic styrene-like material that fills in the cracks solid. Crazy glue along can work, but since it doesnt actually fill the cracks it isn't as strong overall.