May 11, 2021

This qualifies as premature optimization. Until you identify a problem, there is no problem to solve. Increasing the amount of work on a single thread reduces the availability of threads for other work. Load balancing is a performance consideration. Individual calls would certainly be more performant if executed synchronously, as you are essentially suggesting. However this simply blocks other calls, with a subsequent impact on latency of all calls. A trade off is made with consideration of the thread switching cost against the size of the work. Larger jobs are broken up into smaller jobs, and small jobs are aggregated into larger jobs. The optimal size of any job can only be determined by performance testing. Until you have measured this difference an informed decision on which trade off to make is not possible. Therefore, premature optimization.