Collaborative AI: Practical tips for AI development

 Pitfall 1: Treating domain experts as users instead of co-designers

🙅 Unproductive Question:
How can we get experts to trust and use our models?

🙌 Productive Question:
How can we involve experts and decision makers in deciding what our model should focus on from the beginning

Pitfall 2: Aiming to produce “unbiased” predictions that experts will follow

🙅 Unproductive Question:
How can we produce predictions that are better and less biased than those currently offered by experts?

🙌 Productive Question:
What new insights, not currently available to experts but helpful for their decisions, can we realistically offer?

Pitfall 3: Underestimating the collective effort required to organize and label data

🙅 Unproductive Question:
What goals do we want to optimize for and what datasets are available for building new models?

🙌 Productive Question:
What specific goal can we agree on together with experts and decision makers, so that we can focus our data sourcing and labeling efforts on the most critical parameters?

Tip 1: Co-design with experts

💡Involve experts early to ensure the model addresses real needs.

Tip 2: Give experts a new perspective

💡 Deliver new insights, not just accuracy.

Tip 3: Plan for data challenges

💡 Ensure aims are narrow enough to be operationalized.