LoCEBERT: Transformer-Based Line-of-Code Embeddings for Smart Contract Code Understanding and Vulnerability Detection

Smart contracts play a vital role in blockchain technology, but are vulnerable to security risks due to inadequate semantic representations of their source code. Current transformer-based approaches such as BERT, CodeBERT and CodeT5 excel in lexical and contextual understanding. However, they operate mainly at the token level. This limitation prevents the capture of relationships across multiple source code lines or functions. For instance, BERT and CodeBERT are restricted to 512 tokens, thereby causing truncation of longer sequences, loss of long-range dependencies and reduced vulnerability detection accuracy in complex contracts. To address these, we propose a Line-of-Code (LoC)-based embedding approach, treating each line as a separate entity to preserve program structure. Each line is encoded using CodeBERT, and embeddings are aggregated using summation pooling to retain global context. Experiments on the SmartBugs Solidity dataset validate the effectiveness of the approach. LoC embeddings improve clustering performance, increasing Silhouette Score to 0.0888 from 0.0252 and reducing Davies-Bouldin Index to 2.9521 from 5.2315. Vulnerability detection experiments achieved classification accuracy of 96.2%, outperforming token-level approaches at 87.3%. The proposed approach improves interpretability, scalability and vulnerability detection. Future research should explore larger datasets and additional blockchain codebases. These directions will strengthen robustness, performance and generalization capabilities.