How CZ Biohub's AI Agents Might Assist Drug Developers Beyond R&D

A conversation with James Zou, Ph.D., Chan Zuckerberg Biohub San Francisco investigator

Researchers at Chan Zuckerberg Biohub San Francisco and Stanford University have blended the natural-language hypothesis generation of large language models with the prediction algorithms powering target identification platforms like Alphabet’s AlphaFold.

Their trial run explored nanobodies to treat novel SARS-CoV-2 variants; however, researchers say the potential applications span the scientific universe, including process development and regulatory submission planning for biologic drugs.

The open-source project, called Virtual Lab, is a collection of AI personas that interact with each other to solve open-ended scientific problems and conduct virtual experiments.

Deploying the virtual team starts with oversight from a human user. In the trial run, the team included a principal investigator, an immunologist, a machine learning specialist, and a computational biologist. On the periphery, a critical generalist pokes holes and reveals blind spots as an orthogonal check.

We had questions about the practical uses beyond preclinical research. James Zou, Ph.D., a CZ Biohub San Francisco Investigator and an award-winning AI researcher at Stanford whose group developed Virtual Lab in collaboration with the Biohub’s Protein Science team, agreed to help us start conceptualizing how else it might be used. His answers are edited for clarity.

Are the personas available for anyone to use? How do researchers find them and start working with them?

Zou: Yes. We made the Virtual Lab open source, so it's all publicly accessible and people can use it. It's available on GitHub and the link to that is available as a part of the Nature paper that we published.

You demonstrated results in protein design, but the world of biologics is vast. It includes antibodies, enzymes, viral vectors, and cell therapies. Is the architecture flexible enough to adapt across modalities?

Zou: Yes. We designed the Virtual Lab to be a very versatile framework, so it could be applied to many problems in the life sciences and beyond.

Virtual Lab and its agents were able to collaborate and design protein candidates, but how do you see translating that to process development?

Zou: The Virtual Lab could also be useful in the manufacturing setting. So, for example, as a part of Virtual Lab, we could have agents that have expertise in the manufacturing process, and those agents could provide feedback on how to optimize the process.

So, they're not restricted to just conceptualizing protein structures and therapeutic molecules.

Zou: No. They’re not limited to just the research part.

Things get complicated when you start to apply that to scaling up and thinking about your regulatory package. Can Virtual Lab agents simulate scale-up parameters and create process robustness that's appropriate for GMP manufacturing?

Zou: We haven't experimented in that direction yet, so we need to do more experiments before we can answer that question.

How are you thinking about things like model drift and those factors for which you must maintain agents throughout the development life cycle?

Zou: One way to try to address that is to use a specific checkpoint of the model. So, in that case, we could make sure that the model does not drift if we're using exactly the same version of the model. We also have a critic agent whose job is to help to monitor and to provide feedback and reduce errors made by our other agents. I think that also helps.

How does that work exactly? How is that critic different from any of the other characters in Virtual Lab ecosystem?

Zou: The critic agent is also a large language model, and it looks at the transcripts of all the discussions from the other agents, and then it provides some more skeptical or sometimes conservative feedback to the other agents.

For example, if one of the other scientist agents says, “Oh, let's try this,” the critic might try to find some weaknesses or vulnerabilities in that idea and then point that out. So, it's often quite useful to have a critic agent to provide a counterbalance.

You and your colleagues have touted Virtual Labs as overcoming the 'black box' issue, but we should clarify what you mean by that. Are you talking about the underlying infrastructure of artificial intelligence or some other previously murky component of AI/human interaction?

Zou: They communicate through human language — through English — similar to how I can talk to my colleague, and my colleague has a very different area of expertise. Maybe their expertise is in chemistry and my expertise is in computer science.

We have different training and different backgrounds, but because we speak the same language, we can still explain concepts to each other. My colleague can teach me things, and I can teach them. That's similar to how the agents can interact with us. They have different training, they have different backgrounds, but they can still explain things to me in English, then help me to understand their reasoning.

So, if you were to prod them, you would be able to peel back the layers and look at the reasoning behind their output. I imagine that's an important factor when it comes to explainability and presenting this to regulators when, for example, moving toward an IND.

Zou: That's right. Yes.

Can you expound on that a little? What do you think regulators will expect when they're looking at a package from somebody who is getting ready for first-in-human trials or getting ready to submit for an IND?

Zou: In these regulatory packages, they require pretty detailed documentation and request specific types of information. Those are the types of things that these language models could be helpful with, because we could train these models on previous IND submissions, and the model can look at all those previous FDA submissions.

They actually learn about the types of formatting, the kinds of information required, and the writing that makes a successful application.

And there’s still a human involved at every step of the process, correct?

Zou: Yes, that's right. Humans will still provide the oversight.

Were there any observations or moments in this development that gave you optimism or surprised you?

Zou: The main takeaway for us is that the agents in the Virtual Lab were able to quite quickly come up with a computational workflow to tackle this very challenging problem designing binders to the recent COVID variants.

Within a couple of days, they were able to come up with a new computational algorithm using different tools like AlphaFold and Rosetta to help them design the nanobodies in silico.

I think that's quite impressive. That certainly saved a lot of time because it could have taken us much longer, maybe weeks or months, to actually do that ourselves, and there's no guarantee that we would have found the same good solution.

Of course, wet-lab experimental validation is still critical. We're also quite surprised and impressed that our wet lab experiments showed promising binding for the antibodies designed by the Virtual Lab agents.

About The Expert:

James Zou, Ph.D., is an associate professor of biomedical data science at Stanford University and an investigator of the Chan Zuckerburg Biohub San Francisco. He is the director of the Stanford Data4Health hub and a member of the Stanford AI lab. Previously, he worked as a machine learning researcher for Microsoft. He received his Ph.D. from Harvard University.