Even then he has a fiduciary duty - provided he acts in bad faith (and it is egregious enough); I think it would still be possible to oust him due to that.