Background: Studying tumor cell–T cell interactions in the tumor microenvironment (TME) can elucidate tumor immune escape mechanisms and help predict responses to cancer immunotherapy. Methods: We selected 14 pairs of highly tumor-reactive tumor-infiltrating lymphocytes (TILs) and autologous short-term cultured cell lines, covering four distinct tumor types, and co-cultured TILs and tumors at sub-lethal ratios in vitro to mimic the interactions occurring in the TME. We extracted gene signatures associated with a tumor-directed T cell attack based on transcriptomic data of tumor cells. Results: An autologous T cell attack induced pronounced transcriptomic changes in the attacked tumor cells, partially independent of IFN-γ signaling. Transcriptomic changes were mostly independent of the tumor histological type and allowed identifying common gene expression changes, including a shared gene set of 55 transcripts influenced by T cell recognition (Tumors undergoing T cell attack, or TuTack, focused gene set). TuTack scores, calculated from tumor biopsies, predicted the clinical outcome after anti-PD-1/anti-PD-L1 therapy in multiple tumor histologies. Notably, the TuTack scores did not correlate to the tumor mutational burden, indicating that these two biomarkers measure distinct biological phenomena. Conclusions: The TuTack scores measure the effects on tumor cells of an anti-tumor immune response and represent a comprehensive method to identify immunologically responsive tumors. Our findings suggest that TuTack may allow patient selection in immunotherapy clinical trials and warrant its application in multimodal biomarker strategies.