Oguzhan Alagoz Dissertation Examples

We provide a tutorial on the construction and evaluation of Markov decision processes (MDPs), which are powerful analytical tools used for sequential decision making under uncertainty that have been widely used in many industrial and manufacturing applications but are underutilized in medical decision making (MDM). We demonstrate the use of an MDP to solve a sequential clinical treatment problem under uncertainty. Markov decision processes generalize standard Markov models in that a decision process is embedded in the model and multiple decisions are made over time. Furthermore, they have significant advantages over standard decision analysis. We compare MDPs to standard Markov-based simulation models by solving the problem of the optimal timing of living-donor liver transplantation using both methods. Both models result in the same optimal transplantation policy and the same total life expectancies for the same patient and living donor. The computation time for solving the MDP model is significantly smaller than that for solving the Markov model. We briefly describe the growing literature of MDPs applied to medical decisions.

1. Roberts MS. Markov process-based Monte Carlo simulation: a tool for modeling complex disease and its application to the timing of liver transplantation. Proceedings of the 24th Conference on Winter Simulation. 1992:1034–40.

2. Beck JR, Pauker SG. The Markov process in medical prognosis. Med Decis Making. 1983;3(4):419–58.[PubMed]

3. Detsky AS, Naglie G, Krahn MD, Naimark D, Redelmeier DA. Primer on medical decision analysis. Part 1: getting started. Med Decis Making. 1997;17(2):123.[PubMed]

4. Sandikci B, Maillart LM, Schaefer AJ, Alagoz O, Roberts MS. Estimating the patient's price of privacy in liver transplantation. Oper Res. 2008;56(6):1393–410.

5. Puterman ML. Markov Decision Processes. John Wiley and Sons; New York: 1994.

6. Schaefer AJ, Bailey MD, Shechter SM, Roberts MS. Handbook of Operations Research/Management Science Applications in Health Care. Kluwer Academic Publishers; Boston, MA: 2004. Modeling medical treatment using Markov decision processes. pp. 593–612.

7. Bertsekas DP. Dynamic Programming and Stochastic Control. 1 and 2. Athena Scientific; Belmont, MA: 2001.

8. Bellman RE. Dynamic Programming. Princeton University Press; Princeton, NJ: 1957.

9. Denardo EV. Dynamic Programming: Models and Applications. Dover Publications; Mineola, NY: 2003.

10. Lefevre C. Optimal control of a birth and death epidemic process. Oper Res. 1981;29(5):971–82.[PubMed]

11. Hu C, Lovejoy WS, Shafer SL. Comparison of some suboptimal control policies in medical drug therapy. Oper Res. 1996;44(5):696–709.

12. Hauskrecht M, Fraser H. Planning treatment of ischemic heart disease with partially observable Markov decision processes. Artif Intell Med. 2000;18(3):221–44.[PubMed]

13. Ahn JH, Hornberger JC. Involving patients in the cadaveric kidney transplant allocation process: a decision-theoretic perspective. Manage Sci. 1996;42(5):629–41.

14. Alagoz O, Maillart LM, Schaefer AJ, Roberts MS. The optimal timing of living-donor liver transplantation. Manage Sci. 2004;50(10):1420–30.

15. Alagoz O, Maillart LM, Schaefer AJ, Roberts MS. Determining the acceptance of cadaveric livers using an implicit model of the waiting list. Oper Res. 2007;55(1):24–36.

16. Alagoz O, Maillart LM, Schaefer AJ, Roberts MS. Choosing among living-donor and cadaveric livers. Manage Sci. 2007;53(11):1702.

17. Shechter SM, Bailey MD, Schaefer AJ, Roberts MS. The optimal time to initiate HIV therapy under ordered health states. Oper Res. 2008;56(1):20–33.

18. Alterovitz R, Branicky M, Goldberg K. Algorithmic Foundation of Robotics VII. Springer Publications; Berlin/Heidelberg, Germany: 2008. Constant-curvature motion planning under uncertainty with applications in image--guided medical needle steering. pp. 319–334.

19. Maillart L, Ivy J, Ransom S, Diehl KM. Assessing dynamic breast cancer screening policies. Oper Res. 2008;56(6):1411–1427.

20. Faissol D, Griffin P, Kirkizlar E, Swann J. Timing of Testing and Treatment of Hepatitis C and Other Diseases. Georgia Institute of Technology; Atlanta: 2006. http://www2.isye.gatech.edu/~pgriffin/orhepc.pdf.

21. Chhatwal J, Burnside ES, Alagoz O. When to biopsy in breast cancer diagnosis? A quantitative model using Markov decision processes.. 29th Annual Meeting of the Society for Medical Decision Making; Pittsburgh, PA. October 20-24, 2007.

22. Denton BT, Kurt M, Shah ND, Bryant SC, Smith SA. Optimizing the start time of statin therapy for patients with diabetes. Med Decis Making. 2009;29(3):351–67.[PubMed]

23. Kreke JE, Bailey MD, Schaefer AJ, Roberts MS, Angus DC. Modeling hospital discharge policies for patients with pneumonia-related sepsis. IIE Transactions. 2008;40:853–60.

24. Kreke JE. Modeling Disease Management Decisions for Patients with Pneumonia-Related Sepsis [PhD dissertation] University of Pittsburgh; Pittsburgh: 2007.

25. Kurt M, Denton B, Schaefer AJ, Shah N, Smith S. At what lipid ratios should a patient with type 2 diabetes initiate statins? Available from: http://www.ie.pitt.edu/~schaefer/Papers/StatinInitiation.pdf.

26. Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-Effectiveness in Health and Medicine. Oxford University Press; New York: 1996.

27. Winston WL. Operations Research: Applications and Algorithms. Duxburry Press; Belmont, CA: 1997.

28. Alagoz O, Bryce CL, Shechter S, et al. Incorporating biological natural history in simulation models: empirical estimates of the progression of end-stage liver disease. Med Decis Making. 2005;25(6):620–32.[PubMed]

29. Roberts MS, Angus DC, Bryce CL, Valenta Z, Weissfeld L. Survival after liver transplantation in the United States: a disease--specific analysis of the UNOS database. Liver Transpl. 2004;10(7):886–97.[PubMed]

30. Alagoz O. Optimal Policies for the Acceptance of Living- and Cadaveric-Donor Livers [PhD dissertation] University of Pittsburgh; Pittsburgh: 2004.

31. Version release 1. TreeAge Software; Williamstown, MA: 2006. Pro T Suite [computer program].

32. Powell WB. Approximate Dynamic Programming: Solving the Curses of Dimensionality. John Wiley; Hoboken, NJ: 2007.

0 thoughts on “Oguzhan Alagoz Dissertation Examples

Leave a Reply

Your email address will not be published. Required fields are marked *