cqu_5742+ATTACHMENT02+ATTACHMENT02.4.pdf (276.32 kB)

An automatic intelligent language classifier

Download (276.32 kB)
conference contribution
posted on 06.12.2017, 00:00 by Brijesh VermaBrijesh Verma, Hong Suk LeeHong Suk Lee, J Zakos
The paper presents a novel sentence-based language classifier that accepts a sentence as input and produces a confidence value for each target language. The proposed classifier incorporates Unicode based features and a neural network. The three features Unicode, exclusive Unicode and word matching score are extracted and fed to a neural network for obtaining a final confidence value. The word matching score is calculated by matching words in an input sentence against a common word list for each target language. In a common word list, the most frequently used words for each language are statistically collected and a database is created. The preliminary experiments were performed using test samples from web documents for languages such as English, German, Polish, French, Spanish, Chinese, Japanese and Korean. The classification accuracy of 98.88% has been achieved on a small database.

History

Parent Title

Advances in neuro-information processing : 15th International Conference, ICONIP 2008, Auckland, New Zealand, November 2008, revised selected papers, Part II

Start Page

639

End Page

646

Number of Pages

8

Start Date

01/01/2009

ISBN-13

9783642030390

Location

Bangkok, Thailand

Publisher

Springer-Verlag

Place of Publication

Berlin Heidelberg

Peer Reviewed

Yes

Open Access

No

External Author Affiliations

Faculty of Arts, Business, Informatics and Education; Institute for Resource Industries and Sustainability (IRIS); MyCyberTwin;

Era Eligible

Yes

Name of Conference

ICONIP (Conference)

Usage metrics

Exports