Personal Hamming Problem #1: Upgrade risk-benefit tradeoff metrics in AI for drug development.

The therapeutic index is an incredibly important metric in drug development. It operationalizes a new drug’s net therapeutic benefit to patients. Even in earlier stages of drug development, we might boil it down to efficacy-toxicity tradeoffs evaluated in in-vitro and in-vivo readouts of various kinds.

From a rigorous decision making perspective —

• the worst thing you can do is to create a ratio metric such as some version of efficacy to toxicity ratio or on-target to off-target ratio to measure this.

• a good thing you can do is to convert it into a generalized probability index that trades off two preferences (Fig. 2 is a great conceptual input to what would go into this)

• the best thing for high stakes decision making might be to incorporate the science of multi-learning multi-calibration, which hasn’t yet made its way into multi-property drug design (even the ML side).

On the basic research side, ML has already produced multi-learning multi-calibration as a solution to problems in fairness in algorithmic behavior. It’s time to shift the way multi-property optimization is conceptualized in drug development because it can upgrade a crucial metric in ways that some drug developers already recognize!