Yep, plates belong to the owner (every state that I have checked). If you drive with the old plates you can/will be charged with fictitious plates/registration. Usually the back of the registration card is filled out and taped in the window as the temporary registration.
For liability reasons...