Transformer-Derived Contextual Embeddings for Random Forest-Based Prediction of Monthly Rice Harvested Area

Climate variability increasingly disrupts rice harvested area across seasons, while conventional statistical and standalone machine learning methods struggle to capture nonlinear, temporally dependent climate–yield relationships. This study proposes a Transformer-Derived Contextual Embedding-Based Random Forest Regression model to predict monthly rice harvested area in East Java, Indonesia. A Transformer encoder extracts contextual embeddings from climate sequences with window lengths of 3, 6, 9, and 12 months, which are then used as input features for Random Forest regression, with performance compared against a conventional Random Forest baseline. On the testing dataset, the hybrid model achieved RMSE of 34,442, MAE of 27,451, and R² of 0.8426, outperforming the standalone Random Forest (R² = 0.720). The predictions were further used to generate a preliminary planting-window recommendation supporting climate-resilient rice cultivation planning.