It is virtually impossible to take a certified aircraft to the experimental category, unless you're willing to take it apart and re-manufacture a lot of parts. Here are the options in the experimental category:
The following is a list of the different types of Experimental certificates (quoted from 14-CFR-21.191) which are available and how they apply to the installation of a non-certified powerplant onto a certified airframe.
(a) Research and Development: Testing new aircraft design concepts, new aircraft equipment, new aircraft equipment, new aircraft installations.
NOTE: An R&D certificate must be renewed annually. Your FSDO contact will usually issue the first one fairly easily. You will have to prove some substantial reasons to justify the issuance for a second year. In my region, there has never been one issued for the third year.
ALSO, after the expiration of your R&D certificates, you will find it extremely difficult to convert the aircraft back into its original form (ie, restore it to being a standard Cessna-185 or whatever) because now the burden is on you, the modifier, to prove that the aircraft complies with the original type certificate .
(b) Showing Compliance with Regulations: Conducting flight tests and other operations to show compliance with the airworthiness regulations including flights to show compliance for issuance of type certificates and supplemental type certificates, flights to substantiate major design changes, and flights to show compliance with the function and reliability requirements of the regulations.
This classification is what you would use for the flight test verification to obtain an STC for the installation of a different certified powerplant onto a certified airframe. The number of times this certificate can be renewed is also limited.
(c) Crew Training: Training of the applicant's flight crews.
This requires substantiation, periodic inspections, and ongoing renewals. The allowable operations under this classification are limited.
(d) Exhibition: Exhibiting the aircraft's flight capabilities, performance, or unusual characteristics at air shows, motion picture, television and similar productions, and the maintenance of exhibition flight proficiency, including (for persons exhibiting aircraft) flying to and from such air shows and productions.
The allowable aircraft operations under this classification are very limited.
(e) Air Racing: Participating in air races, including (for such participants) practicing for such air races and flying to and from such events.
The allowable aircraft operations under this classification are very limited.
(f) Market Surveys: Use of aircraft for purposes of conducting market surveys, sales demonstrations, and customer crew training only as provided in paragraph 21.195.
FAR 21.195 is very specific about the restrictions that apply here.
(g) Operating Amateur-built Aircraft: Operating an aircraft, the major portion of which has been fabricated and assembled by persons who undertook the construction project solely for their own education or recreation.
(h) Operating Kit-built Aircraft: Operating a primary category aircraft that meets the criteria of paragraph 21.24(a)(1) that was assembled by a person from a kit manufactured by the holder of a production certificate for that kit, without the supervision and quality control of the production certificate holder, under paragraph 21.184(a).