Yes, you should report it. No supervisor should be asking his employees out. It is sexual harassment.
It is illegal, in my state, for teachers to date their own students, even if the student is an adult, because the teacher has authority over the student. The teacher could say, "Date me, or fail the class." Which is sexual harassment and abuse of power.
The same should be done with the employer / employee relationship. It should be illegal for an employer to date his own employee for the same reason. The employer could say, "Date me, or you're fired." Which is sexual harassment and abuse of power. If that happens, you can sue.
No teacher, no employer, no one else should be forcing people into a relationship. Even the appearance of force is wrong. Your boss asking you out is the appearance of such force.
Reject his advances and report him to his boss. If you do, and he is not fired, demoted, or re-assigned, report him and the company to the authorities.