Purpose: To develop a framework for automated prediction of hepatobiliary (HB) toxicity after liver stereotactic body radiation therapy (SBRT). Materials and methods: A newly recognized toxicity type, named central or HB liver toxicity, had been reported, manifestation of which strongly correlates with the dose delivered to portal vein (PV) during SBRT. We propose a novel framework for automated HB toxicity prediction by combining deep learning-based auto-segmentation, PV anatomy analysis and the previously reported HB toxicity model. For validation of the framework, an IBR approved representative database of 72 patients treated with SBRT from primary (37) and metastatic (35) liver cancer was assembled. Each case included a pre-treatment CT, manual segmentations of tumor and PV, approved treatment plan, and the record of acute and late post-treatment toxicities. Performance of the developed framework was evaluated by quantitative comparison against manual predictions of HB toxicity, as well as post-treatment toxicity follow-ups. Results: The manual and automated predictions of HB toxicity were in agreement for 94% cases using either V_BED1030 ≥ 45 cc or V_BED1040 ≥ 37 cc dosimetric predictors. When compared to post-treatment follow-ups for primary liver cancer, the proposed automated framework made 86% and 83% correct predictions in comparison to 83% and 80% correct manual predictions using V_BED1030 ≥ 45 cc or V_BED1040 ≥ 37 cc, respectively. Conclusion: The proposed framework automates the HB toxicity prediction with the accuracy similar to manual analysis-based HB toxicity prediction. The strategy is quite general and extendable to the automated prediction of toxicities of other organs.