Inclusion Criteria:
* Histologic proof of a solid tumor that is now unresectable, not amenable to any other standard therapies, or patient refuses standard therapy
* Metastatic adenocarcinoma of the pancreas amenable to biopsies (cohort II MTD only)
* Absolute neutrophil count (ANC) \>= 1,500/μL
* Platelets \>= 100,000/μL
* Total bilirubin =\< upper limit of normal (ULN)
* Aspartate aminotransferase (AST) =\< 3 times upper limit of normal (ULN)
* Creatinine =\< 1.5 times ULN
* Hemoglobin \>= 9.0 g/dL
* International Normalized Ratio (INR) within normal limits (for patients treated at the MTD)
* Ability to provide informed consent
* Willingness to return to Mayo Clinic for follow up
* Life expectancy \>= 12 weeks
* Willingness to provide the biologic specimens as required by the protocol
* Negative serum pregnancy test done =\< 7 days prior to registration
* Eastern Cooperative Oncology Group (ECOG) performance status (PS) 0-2
* Able to swallow or have medication administered through a G-tube and absorb the medication
* Participant agrees to use an acceptable form of contraception; acceptable forms of contraception:
* Latex condom (always used with spermicide)
* Diaphragm (always used with spermicide)
* Cervical cap (always used with spermicide)
Acceptable forms of secondary contraceptions, when used along with a barrier method:
* Hormonal contraception methods, including pills, patches, rings, or injections except progestin-only containing pills (i.e. "Mini-pill")
* Tubal ligation
* Partner's vasectomy
* Intrauterine device (IUD) (non-progesterone T)
* Vaginal sponge (containing spermicide)
* 100% commitment to abstinence
Unacceptable forms of contraception for women of childbearing potential:
* Oral contraception containing progestins only
* IUD progesterone T
* Female condom
* Natural family planning (rhythm method) or breastfeeding
* Fertility awareness
* Withdrawal
* Cervical shield
* Willing to abstain from smoking
* Willing to complete a daily pill diary
Exclusion Criteria:
* Known standard therapy for the patient's disease that is potentially curative or definitely capable of extending life expectancy
* Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements
* Any of the following prior therapies:
* Chemotherapy =\< 4 weeks prior to registration
* Mitomycin C/nitrosoureas =\< 6 weeks prior to registration
* Immunotherapy =\< 4 weeks prior to registration
* Biologic therapy =\< 4 weeks prior to registration
* Radiation therapy =\< 4 weeks prior to registration
* Radiation to \> 25% of bone marrow
* Failure to fully recover from acute, reversible effects of prior therapy regardless of interval since last treatment
* New York Heart Association classification III or IV
* Seizure disorder
* Central nervous system (CNS) metastases if not stable for at least 2-3 months based on imaging, clinical assessment, and use of steroids, or seizure disorder
* Pregnant women
* Nursing women
* Men or women of childbearing potential who are unwilling to employ adequate contraception until 12 months after last study drug dose
* Other concurrent chemotherapy, immunotherapy, radiotherapy, or any ancillary therapy considered investigational (utilized for a non-Food and Drug Administration \[FDA\]-approved indication and in the context of a research investigation)
* Current therapy with a CYP3A4 inhibitor or inducer
* Immunocompromised patients (other than that related to the use of corticosteroids) including patients receiving highly active antiretroviral therapy (HAART) treatment
* Receiving any other investigational agent which would be considered as a treatment for the primary neoplasm
* Active other malignancy, excepting non-melanotic skin cancer or carcinoma-in-situ of the cervix; if there is a history of prior malignancy, they must not be receiving other specific treatment (other than hormonal therapy) for their cancer
* History of myocardial infarction =\< 6 months, or congestive heart failure requiring use of ongoing maintenance therapy for life-threatening ventricular arrhythmias
* Abnormalities of the cornea based on history (e.g., dry eye syndrome, Sjogren's syndrome), congenital abnormality (e.g., Fuch's dystrophy), abnormal slit-lamp examination using a vital dye (e.g., fluorescein, Bengal Rose), and/or an abnormal corneal sensitivity test (Schirmer test or similar tear production test)
* More than 2 prior chemotherapy regimens for the current metastatic malignancy; full dose chemotherapy used in conjunction with concurrent radiation therapy will be included as prior therapy; Note: prior hormonal therapy (e.g. leuprolide, aromatase inhibitors, tamoxifen) will be allowed and not included as a prior chemotherapy
* Previous therapy with a hedgehog inhibitor
