Ground truth generation for crop classification using Street View

Land-use understanding at agricultural field level is critical for decision making and addressing food and water security challenges, especially in smallholder regions. Using remote sensing and modern machine learning practices for the same is an active area of research. Satellite crop classification models are built using ground truth from manual agricultural surveys namely location, timestamp and crop label. However manual surveys are prone to labeling errors and unlikely to be scalable due to cost prohibitions. Since machine learning models don’t generalize in a sparse data regime, this affects the accuracy and applicability of these models. Google Streetview has captured a significant portion of field imagery next to roads across the globe. We propose large labeled ground truth data creation for satellite crop classification at planetary scale from a model which labels street view imagery. Since street view has wide coverage with data points spread across multiple years, and has reliable accuracy in terms of location and timestamp, our streetview labels dataset would be at least an order of magnitude larger than the next best dataset, allowing a single large satellite model to generalize for addressing crop classification. Recent studies have explored this method for pilots in the USA and South Korea, and we aim to generalize the work at planetary scale. We use pre-trained vision models to generate the training labels for a satellite image based crop classification model.  We aim to open source the labeled field polygons by the end of 2024 to enable the research community to use the dataset in other models.