My TV does something similar. Best/worst example is if I display a black console with a blinking white cursor: the backlight flashes on and off from min to max brightness in time with the cursor. It is an abomination, a gimmick whose only purpose is to cheat at contrast ratings. If I ever meet the engineer who came up with this BS, I will stab them with a strobe light.
My old Nexus 7 (2012) had a similar feature called PRISM/SmartDimmer which was equally annoying, though it could be disabled via sysfs. Many people did not notice it though, so maybe it was a net win...
There is no way that you can't turn it off, I find it highly doubtful that they do this in hardware with some ASIC that reads the display buffer in the LCD and averages the color of the screen from there.
It's more than likely uses the screen brightness API that was introduced with Windows 8 which talks to things like colorimeters and light sensors.