Segmentation and Recognition of Electronic Components in Hand-Drawn Circuit Diagrams

This paper presents an e ﬀ ective technique for segmentation and recognition of electronic components from hand-drawn circuit diagrams. Segmentation is carried out by using a series of morphological operations on the binarized images of circuits and discriminating between three categories of components (closed shape, components with connected lines, disconnected components). Each segmented component is characterized by computing the Histogram of Oriented Gradients (HOG) descriptor while classiﬁcation is carried out using Support Vector Machine (SVM). The system is evaluated on 100 hand-drawn circuit diagrams with a total of 350 components. A segmentation accuracy of 87.7% while a classiﬁcation rate of 92% is realized demonstrating the e ﬀ ectiveness of the proposed technique


Introduction
Sketches are widely used in engineering and architecture fields especially for the early design phases.This is mainly due to the fact that a sketch is a convenient tool to catch rough ideas without going into the intricate details.Although humans are able to recognize sketches and their components effortlessly, the problem is very challenging when it comes to machine recognition of these sketches.While computerized recognition of printed and handwritten text has been one of the most researched pattern classification problems, relatively less research attention has been devoted to recognition and understanding of hand-drawn sketches and shapes.Document image analysis has been an active and challenging area of research over the last three decades.Document images usually comprise of both text and graphics.Machine recognition of handwritten text involves classification of characters, Mathematical expressions, symbols and digits.Likewise, computerized processing of hand-drawn graphical entities involves analysis of circuit diagrams, flow charts, tables, maps and architectural drawings etc.The focus of the present study lies on hand-drawn circuit diagrams.We present a system for segmentation of electronic components from digitized images of circuit diagrams and recognize these components.The segmentation part relies on morphological processing while the individual electronic components are represented using the Histogram of Gradients (HOG) descriptor.HOG features extracted from the circuit components are employed to train a one-againstall Support Vector Machine (SVM) that learns to discriminate between different component classes.The recognition system offers a number of useful applications including computerized digitization of hand-drawn circuit diagrams, development of tools for solving the circuits and development of augmented reality based applications to help students learn about different components.
The paper is organized as follows.Section 2 enlists the significant related works on this problem.Section 3 details the architecture and working of the proposed system.Section 4 presents the experimental setup and a discussion on the realized results while Section 5 concludes the paper.

Related Works
Automatic analysis of electronic circuits and components has been researched in a number of studies.Among one of the earliest studies on this problem [3], authors preprocess the scanned images of hand-drawn circuits and binarize them usig global thresholding.The image is then represented using chain codes and circuit primitives like lines, circles, arrows etc. are identified.A rule-based approach is then employed to group these primitives and match them with the stored templates of different components.In a later work [5], authors introduce a novel line vectorization technique for recognition of circuit diagram images.A syntax based classifier is then employed to identify the open symbols.Loop based symbols are then recognized using a supervised approach.The technique was evaluated on ten hand made circuit diagrams where all the components were successfully recognized except a few errors in classifying battery as capacitor.
In another study [7], a system for recognition of hand-drawn circuit diagrams is presented.Scanned images of circuit diagrams are first preprocessed and binarized.A set of morphological operations is then applied to obtain a clean and thinned version of the image.Authors distinguish between connections, nodes and components in a circuit which are segmented using appropriate thresholds on the pixel density.A set of 39 features including Hu moment invariants, pixel histograms, geometric measurements and polygonal representations were used to recognize a set of nine components.A total of 449 samples of hand drawn components were used out of which 349 were employed to train the classifier and 100 were used for testing purposes.All samples were drawn by the same individual, nevertheless many variations were introduced.A component accuracy rate of approximately 86% is reported in this study.
Among well-known recent contributions, authors in [15], employ Line Primitive Extraction for segmentation of hand-drawn diagrams which are subsequently recognized using Support Vector Machine (SVM) classifier.A total of six shape classes are considered in the study with 120 images per class in training and 20 images per class in the test set.The authors report 89% correct classification rate.In another recent study [14], authors employ image moments as features and train a feed forward artificial neural network to recognize a total of 31 circuit components, including digits, selected alphabets, units In [13], authors employ Fourier descriptors as features for recognition of digital logic circuit diagrams.Three components (AND, OR and NOT gates) were considered in the study with a total of 60 images.Classification with SVM realized a classification rate of 83%.Recognition of online hand-drawn circuit diagrams is studied in [8] where the authors employ two-dimensional dynamic programming (2D-DP) technique to segment and recognize the symbols.Likewise, recognition of architectural drawings [1], flow charts [10], Mathematical expressions [2], UML diagrams [9] and free hand sketches [11] has also been investigated in a number of studies.

Methodology
The proposed recognition methodology can be divided into four important steps i.e.Image acquisition and preprocessing, Segmentation, Feature Extraction and Classification.The overall working of our system is illustrated in Figure 1 and each of these steps is discussed in the following sections.

Image Acquisition and Preprocessing
Input images can be categorized as offline or online based on the method of acquisition.Scanned input images are considered offline while the ones acquired on a digital tablet are online.Different processing techniques are applied on the two kinds of images.Our system works on offline images of hand drawn electronic circuit diagrams which are digitized by scanning.A series of pre-processing operations is then performed on these scanned offline images to enhance image features for further processing.Images are first converted to grayscale followed by application of smoothing filter to remove the noisy components.
Images are then binarized using global thresholding and processed further as discussed in the following.

Segmentation
Automated handwritten document image analysis is a challenging problem where variations in drawings make it difficult to detect and segment the intended object.The objective of segmentation module in our proposed system is to separate the electronic components from the circuit image.The isolated components are then fed to the classifier as test input where features are extracted and classification is performed.In most cases a complete circuit is a whole connected component, hence segmentation of any component is not a simple task.Shape can be a vital feature for detection of component symbols.Electronic component symbols can be broadly categorized into three groups based on their geometric structure i.e. symbols with closed shapes, symbols with connected line segments and symbols with disconnected line segments.Each group requires a different approach for segmentation depending upon its particular common attributes.Components with closed shapes include AC/DC voltages, diodes and lamps, all these shapes have a common attribute of enclosure.This primary attribute can therefore be used in order to segment these shapes from the rest.By applying region filling on the binarized image of the circuit, all shapes with enclosed areas are filled as shown in Figure 3-c.Morphological opening with a suitable structuring element is applied to remove connecting wires and other components.The remaining connected components in the image are then localized (Figure 3d) by mapping the component bounding boxes on the original image.
Once the first group of components is removed from the circuit, morphological closing is applied on the rest of the circuit which results in dilating components with connected lines like resistors and inductors as shown in Figure 4-b.Morphological opening removes the rest of the circuit leaving only the dilated components which are then localized.Finally the last group of components with disconnected line segments is segmented.Most of these components are disconnected in nature and hand drawn image can result in variation of distance between their components hence it is difficult to determine a suitable structuring element for morphological operations.Morphological thickening followed by thinning is therefore applied on the entire circuit which results in connecting any disconnected    have been applied to handwriting and hand-drawn shapes [4,6,12] and have reported significant performance enhancement.In our work, we have chosen to employ the Histogram of Oriented Gradients (HOG) as descriptor to characterize the electronic components.HOG descriptor captures the localized information about intensity gradients as shown in Figure 6.Varying the block size in computation of HOG corresponds to different scales of observation at which the features are extracted.The sensitivity of block size to recognition performance is discussed later in the paper.The dimensionality of the HOG feature vector is a function of the image size.We, therefore, resize

Classification
Classification is carried out using one-against-all Support Vector Machine.HOG features extracted from the training images are employed to train the SVM that learns to discriminate between the ten component classes used in our study.Features of the query component are then fed to the trained SVM to recognize the component.

Experimental Setup and Results
Evaluations are carried out on a total of 100 handdrawn circuit diagrams containing 350 electronic components, 35 for each of the 10 component classes considered in our study (Figure 2).Segmentation is applied on the complete set of 100 images while for classification 200 components (20 per class) are employed in the training set and 150 (15 per class) in the test set.
For segmentation, among the 350 circuit components, 307 are correctly segmented realizing a segmentation accuracy of 87.7%.Component wise segmentation results are summarized in Table 1.It can be seen that most of the components are correctly segmented reporting good segmentation accuracy.Major proportions of errors stem from the third group of components, battery, switch and capacitor.This is very much natural as these components comprise of disconnected lines which are very similar to the connections/wires and are therefore more challenging to segment.For recognition, an overall classification rate of 92% is realized on the 150 test components.The corresponding confusion matrix is illustrated in Table 2 while component-wise performance measures including sensitivity, specificity and precision are summarized in Table 3.It can be seen from the realized results that most of the components are correctly recognized by the system.Relatively low classification rates are reported on diode, capacitor and switches while inductors, resistors, grounds and lamps report 100% recognition rates.It should be noted that from the view point of application, the output of the segmentation is fed to the classification module for recognition of components.However, incorrectly segmented components cannot be recognized and, in such cases, the classification rates do not truly reflect the effectiveness of the recognition module.Consequently, we present the classification rates on individual components by using 200 components in the training set and 150 in the test set.
We also study the performance variation of the system with respect to the block size in computation of the HoG descriptor.Block size refers to the scale of observation in feature computation.A small block size captures local information on gradients while increasing the block size increases the scale of observation.The classification rates as a function of block size are summarized in Figure 7.It can be seen that very small block sizes capture the information on lines and curves which are very similar across multiple components hence realizing low classification rates.Increasing the block size captures the component-specific gradient discriminative information consequently realizing high classification rates.A maximum classification rate of 92% is observed on a window size of 32 × 32.

Conclusion and Future Works
An effective technique for segmentation and recognition of electronic components in hand-drawn circuit diagrams is presented.The segmentation part relies on a series of morphological operations while recognition is carried out using HOG descriptor with SVM classifier.
hand-drawn circuits with a total of 350 components while a classification rate of 92% is realized on 150 query components.The present work presents the initial findings of an ongoing study which is intended to be extended to include complex circuit diagrams and a larger set of electronic and digital components.We also plan to recognize the symbols, units and the textual content on these images.In addition to HOG, other object recognition descriptors like SIFT, SURF, BRISK and FAST are also planned to be investigated.Moreover, with a complex system having a wide range of components, we also plan to implement multiple classifiers and combine their outputs to enhance the overall system performance.The system can also be extended to recognize online circuit diagrams in addition to the offline images.
A segmentation accuracy of 87,75 % is reported on 100

Figure 1 .
Figure 1.An overview of the proposed system

Figure 2 ,
presents our proposed categories with examples of each kind of symbols which are being used in this study.

Figure 2 .
Figure 2. Component categories considered in our study

Feature
are measurable attributes of an object which can be used for classification purposes.In document image analysis problems, classification of characters, shapes and symbols offers a variety of challenges.The most important of these are intra-class variation and inter-class similarity.Typically, low level image features are employed to characterize these classes.Recently, high level computer vision descriptors 3 EAI Endorsed Transactions on Scalable Information Systems 12 2017 -04 2018 | Volume 4 | Issue 16 | e12

Figure 3 .
Figure 3. Segmentation of components with closed shape: (a) Original image (b) Inverted binarized image (c) Image after region filling (d) Detection of components after morphological opening (e) Localization of components on original image (f) Segmented components

Figure 4 .Figure 5 .
Figure 4. Segmentation of components with connected lines: (a) Image after morphological closing (b) Removal of other components and detection of respective component (c) Localization of component in original image

Figure 6 .
Figure 6.Visualization of HOG features of DC voltage: (a) Feature vector of length 34596 at 8x8 (b) Feature vector of length 8100 at 16x16 (c) Feature vector of length 1764 at 32x32 (d) Feature vector of length 324 at 64x64

Figure 7 .
Figure 7. Classification rates as a function of block size in HOG computation

Table 1 .
Segmentation Accuracy of Components

Table 2 .
Confusion Matrix Component Diode Battery Resistor Capacitor Inductor Ground Switch AC Vol.DC Vol.Lamp