AI Task Delegability

Which tasks should be delegated to AI: A human-centered approach.

Recent developments in machine learning have catapulted the promise of artificial intelligence into the public consciousness and driven a resurgence in research. But while machine capabilities have reached new heights, the questions of which tasks to automate and the extent to do so have lagged behind.

We wish to explore these shortcomings.

AI is the new electricity. Just as electricity transformed almost everything 100 years ago, today I actually have a hard time thinking of an industry that I don’t think AI will transform in the next several years.

For some tasks, such as dangerous or routine work, complete automation may be the right answer. For others, we may still want AI to play an assistive role while maintaining overall human control (i.e., machine-in-the-loop designs). Clearly machine ability figures into this decision. But human preferences should also play a role. Even as machines grow more capable, people may want to keep control over tasks for a variety of reasons, including privacy, ethics, or motivation. A better understanding of the factors behind a delegation preference, and the relation of the delegability of different tasks to each-other could help researchers and industry alike.

Approach

Towards this end, we take three steps.

1. Develop a framework of human perception of task delegability to AI, considering four high-level factors which may contribute to a delegation decision
2. Build a dataset of 100 tasks for evaluation under our framework. These 100 tasks are meant as a starting point towards the eventual goal of spanning the whole delegable "task space".
3. Develop and administer a survey to collect peoples' perception of the tasks under our framework, including the extent to which they prefer AI involvement.

Results

We find little preference for full AI control and a strong preference for machine-in-the-loop designs, in which humans play the leading role. Our framework can effectively predict human preferences in the degree of AI assistance. Among the four factors, trust is the most predictive of human preferences of optimal human-machine delegation.

Exploring the data

The responses are available on this site in interactive forms for exploration, either as a table with the raw numbers, or visualized as a multidimensional plot.

We also have the surveys available. You can view the survey we administered to collect the data, or if you like, submit responses of your own!

Or, just grab the raw data from our github here!