We used a two stage procedure to predict which stroke patients would have chronic difficulties gesturing how to use an object when object recognition and hand movements were intact. First, we searched our PLORAS database by behavior and identified 5 patients who had chronic difficulty gesturing object use but no difficulty recognising objects, comprehending words or moving their hands. High definition lesion analyses showed that all 5 patients had damage to the white matter underlying the left ventral supramarginal gyrus, (A) close to the cortex, (B) deep towards the midline and (C) extending into the temporal lobe. In addition, 2 patients had damage to (D) the left posterior middle temporal cortex, and 3 patients had damage to (E) the left dorsal supramarginal gyrus and (F) the left premotor cortex. Second, we searched our database by lesion location for patients who had damage to any part of regions ABCDEF. The incidence of gesturing difficulties was higher in patients with damage to ABCD (7/9), ABCE (7/10) or ABCDE (10/13) than ABCF (7/13), ABC (8/16) or partial damage to ABCF (6/32). Thus behaviour was best predicted by the combination of regions that were damaged rather than on the basis of each region alone or overall lesion size. Our results identify which parts of the temporal and parietal lobes impair the ability to gesture object use and which parts need to be intact to support it after damage. Our methods provide a framework for future studies aiming to predict the consequences of brain damage.