Inclusion Criteria:
* Ability to comprehend the investigational nature of the study and provide written informed consent
* Age 18 to ≤ 65 years (yrs), at the time of consent
* All gender identities, races, or ethnicities are eligible
* Newly documented, previously untreated diagnosis of AML or myelodysplastic syndrome (MDS) with marrow blasts ≥ 10%, in agreement with 2022 European LeukemiaNet criteria (ELN22)
* Leukapheresis and treatment with cytarabine or hydroxyurea prior to study initiation is permitted for cytoreduction in patients with proliferative disease. NOTE: Treatment with cytarabine is limited to up to 2 grams total at least 14 days prior to starting on protocol defined therapy
* Eastern Cooperative Oncology Group (ECOG) performance status of 0-1
* Willingness to undergo hematopoietic stem cell transplant (HSCT)
* Ability to take medications by mouth or feeding tube
* Adequate hematologic and organ function
* Institutional standards, New York Heart Association (NYHA) criteria for cardiac function
* Calculated creatinine clearance (according to the Cockcroft-Gault equation) \> 40 mL/min
* Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\]) ≤ 3 x upper limit of normal (ULN), unless considered due to leukemic involvement
* Alanine aminotransferase (ALT) (serum glutamic pyruvic transaminase \[SGPT\]) ≤ 3 x ULN, unless considered due to leukemic involvement
* Total bilirubin ≤ 1.5 x ULN, unless due to Gilbert's disease or leukemic involvement
* Willing and able to
* Adhere to study schedule of activities and lifestyle restrictions while on treatment;
* Provide bone marrow (BM) aspirate and core biopsy samples; AND
* Accept supportive and prophylactic care for hematologic toxicities, infection, and immediate sequelae, including transfusions
* Negative pregnancy test within 3 days of start of treatment for persons of childbearing potential (PCBP)
* Based on animal studies and the known pharmacology of the study drugs, PCBP and sperm-producing participants who are sexually active with a PCBP must comply with study requirements for contraception
* PCBP (participants and PCBP partners of participants) must agree to use an approved contraception and to refrain from donating / cryopreserving ova from cycle (C) 1 day (D) 1 until 6 months following the last dose of study treatment
* Participants who produce viable sperm and who have intercourse with PCBP must agree to use an approved contraception method and to refrain from donating sperm from C1D1 until 3 months following the last dose of study treatment
Exclusion Criteria:
* Documented t(15;17) (acute promyelocytic leukemia \[APL\]), and/or mutation(s) to FLT3 ITD or core binding factor (CBF). Point mutations within the tyrosine kinase domain (FLT3 TKD) are allowed
* Another active malignancy within the previous 5 years, except treated early stage carcinomas of the skin, or at the investigator's discretion
* Known, active central nervous system (CNS) involvement with AML
* Recent and significant medical interventions, such as major surgery within 28 days of start of treatment
* GVHD or autologous stem cell transplant within 100 days of start of treatment
* Currently receiving investigational therapy or chemotherapy within 28 days, or 5 half-lives, whichever is longer, with the exception of hydroxyurea or cytarabine for cytoreduction purposes
* Prior treatment with a BCL 2 inhibitor within 12 months prior to the start of treatment
* Use of strong or moderate CYP3A4 inducers or inhibitors or P-gp inhibitors within 2 days or 3 half-lives, whichever is longer, prior to start of treatment with venetoclax or at the discretion of the investigator if dose reductions, based on the interaction, have been specified
* History of allergic response to any of the interventional agents or any excipients in the formulations
* Inadequate organ function, including the following (or at the discretion of the investigator):
* History of New York Heart Association (NYHA) class III or IV congestive heart failure or left ventricular ejection fraction (LVEF) \< 40% by echocardiogram (ECHO) or multigated acquisition (MUGA) scan
* Unstable/uncontrolled angina pectoris, history of severe and/or uncontrolled ventricular arrhythmias, or history of myocardial infarction within the last 6 months
* A white blood cell count (WBC) \> 25 x 10\^⁹/L
* Known dysphagia in the absence of a feeding tube, short-gut syndrome, or other conditions or causes that would affect the ingestion and/or gastrointestinal absorption of drugs administered orally
* Active hepatic disorder or documented positive hepatitis B or C virus (HBV/HCV, respectively) status, except in cases of undetectable HBV/HCV viral load for at least 3 months prior to the start of treatment. (Hepatitis B or C testing is not required for eligibility assessment.)
* Individuals with positive serology for human immunodeficiency virus (HIV) who are undergoing treatment with highly active antiretroviral therapy (HAART) (or another therapy that may interfere with metabolism of study agents) are not eligible. If the HIV infection is controlled with another medication type or if an acceptable alternative HIV treatment can be substituted for HAART, enrollment may proceed
* Uncontrolled infection. Participants with controlled infection must be afebrile and hemodynamically stable for at least 72 hours prior to start of treatment and must be amenable to alternate treatment if current treatment will interact with investigational regimen
* Psychiatric illness/social situations that would limit compliance with study requirements
* Unwillingness to stop breastfeeding. Because there is a potential risk for adverse events in nursing infants secondary to treatment of the mother, breastfeeding is not allowed throughout the study for 6 weeks after the last dose of study drug
