Got it.
Its got to do with the CPU's Enhanced Intel Speedstep Technology
Meaning that activities that isn't CPU intensive cause the processor to use a lower multiplier causing a lower power consumption.

I did a bench test and it went up to X11 and stayed there until i terminated the program.

Going to test my laptop now.