Abstract: Most pen-based user interfaces are incapable of recognizing both graphical symbols and text with a single recognizer. Thus, it is essential to distinguish between graphical strokes and textual ones before feeding them into the appropriate recognizer. An approach for classifying sketched strokes is presented using Support Vector Random Field (SVRF). Inputting strokes as well as the interactions among them are jointly modeled by the random field. Not only the unary features of strokes themselves are utilized for discriminative classification, but also their temporal and spatial context are exploited through neighborhood system and features of binary stroke pairs. After applying Loopy Belief Propagation (LBP) inferring, the joint labeling solution according to maximum posterior marginal criterion is estimated. Experimental results show that the classification accuracy of the approach outperforms the Support Vector Machine (SVM) classifier as well as the Markov Random Field (MRF)-based joint classification approach which utilizes spatial context. The speed of classification meets basically the requirement of real-time interaction. Thus the feasibility and effectiveness of the proposed approach are verified.