You can see a live<\/em> demo<\/em> of Instaparse online with this grammar here.<\/a><\/p>\n\n\n\n Update: 2020-1-8: A fully recursive parser can be found here http:\/\/instaparse-live.matt.is\/#\/-Ly5y7xvJoDkHhIs0UKD\/v1<\/a><\/p>\n\n\n\n It’s like a regular expression for sentences. “Extended Backus-Naur Form.” If you’re familiar, skip ahead. If not… Can we make any real headway using computer syntax<\/em> to decipher human language? English seems impossible for native language speakers to parse at times. Is it really possible to parse Japanese with EBNF and label parts of speech accurately?<\/p>\n\n\n\n Yes! However, for real<\/strong> Japanese from the wild there would need to be many many rules, and any slight deviations from the rules would not result in a full parse. So, rather than try and build our own google translate today, why don’t we just try to cover the maximum amount of Japanese with the minimum amount of rules.<\/p>\n\n\n\n Japanese has nouns, verbs, particles, and sentence-final-fitters. That’s it. * With 5 concepts we can cover the whole Japanese grammar.<\/strong> Amazing. Japanese is regular enough that its composition can be described in a handful of rules.<\/p>\n\n\n\n With just 5 main rules we can parse all of Japanese. That’s exciting! Granted, this is not negation or past tense, just present affirmative, but with small extensions those advancements are accommodatable. <\/p>\n\n\n\n Pictured Above: Any number of Complete Bunsetsu Jars (means a Jar with contents and a Lid = a particle) plus a Sentence-Final-Fitter make a Japanese sentence.<\/p>\n\n\n\n \u3052\u3093\u304d\u306a\u3072\u3068\u304c\u3046\u3064\u304f\u3057\u3044\u30e2\u30f3\u30bf\u30ca\u306b\u3044\u308b\u3002 Notice Japanese has no spaces, all parsing relies on accurate detection of symbol in sequence. ImagineifEnglishhadnospaceswoulditbehardtoread?<\/p>\n\n\n\n By extending Rule 2 (JAR = … ) to include JAR*, we achieve full recursion on Japanese terms using \u306e [no]<\/p>\n\n\n\n http:\/\/instaparse-live.matt.is\/#\/-Ly63ijYzrhVjVNf7b7a\/v1<\/a><\/p>\n\n\n\n Japanese has nouns that take \u306a [na] and nouns that take \u306e [no] when they are used as modifiers. <\/p>\n\n\n\n Japanese also has verbs that end with an \u3046 [u = oo as in “food”] sound and verbs that end with an \u3044 [i = ee as in “week”] sound.<\/p>\n\n\n\n Bringing “adjectives” into the mix just gets things really dirty and offers no benefit to the learner. Stick with 2 kinds of verb and 2 kinds of noun and Japanese will be easy to master.<\/p>\n\n\n\n Important in language learning are the exercises that parse written language correctly. Learning how the written language is parsed can help one’s mental model, and this will lend to stronger audible comprehension thanks to a deeper understanding of the underlying structure.<\/p>\n\n\n\n Thanks for reading! This breakdown brought to you by Japanese Complete<\/a>, Rapid Fluency Acquisition Platform. <\/p>\n\n\n\n Related Initiatives *When we say “that’s it,” we are putting words like \u304a\u3044\u3057\u3044 into the verb-i column, and words like \u3052\u3093\u304d\u306a into the noun-na column. Simplifying into two categories, nouns and verbs, actually proves immensely useful and practical in Japanese. It’s a simplification that should be in all the textbooks and literature. Congrats 2020.<\/p>\n\n\n\n We create a 5-rule parser to efficiently and accurately parse affirmative present tense sentences in Japanese. With just 5 rules you can understand Japanese grammar deeply.<\/p>\n","protected":false},"author":2,"featured_media":777,"comment_status":"open","ping_status":"open","sticky":true,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[59,60,138,142,149,158,232,259],"yoast_head":"\nWhat’s an EBNF grammar?<\/h2>\n\n\n\n
Here’s a useful slideshow.<\/a> The wikipedia article is not for the uninitiated, but this article<\/a> is a bit clearer if you are new to this way of representing grammars. EBNF is from the realm of “Context-Free Grammars” and is mainly used in computing. Given that it is definitely about grammar and syntax, EBNF proves very useful in language analysis and generation. You make rules for breaking down sequences of characters (sentences) into labeled output chunks (parse tree).<\/p>\n\n\n\nIs it possible to parse Japanese with EBNF?<\/h2>\n\n\n\n
\u306f\u3044\uff01 Yes!<\/h3>\n\n\n\n
![\"5](\"https:\/\/japanesecomplete.com\/articles\/wp-content\/uploads\/2020\/01\/screen-shot-2020-01-04-at-18.52.52.png\")
Rules for an EBNF Japanese Grammar<\/h2>\n\n\n\n
S = JAR* SFF<\/strong><\/li>
JAR = M* noun-no P | JAR* noun-no no<\/strong>
[UPDATE 2020-1-8: fully recursive with extension of rule 2]<\/em>
<\/li>
P = \u304c | \u3092 | \u3068 | \u3067 | \u306b …<\/strong><\/li>
SFF = noun-no \u3060\u3002 | noun-na \u3060\u3002 | verb-u\u3002 | verb-i\u3002
<\/strong>(sentence-final period included in this rule)<\/li>
M = noun-no \u306e | noun-na \u306a | verb-u | verb-i<\/strong><\/li><\/ol>\n\n\n\nVisuals to Help Expound<\/h2>\n\n\n\n
![\"\"](\"http:\/\/articles.japanesecomplete.com\/wp-content\/uploads\/2020\/01\/parse_1.png\")
![\"\"](\"http:\/\/articles.japanesecomplete.com\/wp-content\/uploads\/2020\/01\/parse_2.png\")
<\/figure>\n\n\n\n![\"\"](\"http:\/\/articles.japanesecomplete.com\/wp-content\/uploads\/2020\/01\/parse_3.png\")
<\/figure>\n\n\n\n![\"\"](\"http:\/\/articles.japanesecomplete.com\/wp-content\/uploads\/2020\/01\/ebnf_6.png?w=900\")
<\/figure>\n\n\n\n![\"\"](\"http:\/\/articles.japanesecomplete.com\/wp-content\/uploads\/2020\/01\/ebnf_5.png?w=900\")
<\/figure>\n\n\n\nExperimental Data<\/h2>\n\n\n\n
\u3044\u308b\u306e\u304c\u3046\u3064\u304f\u3057\u3044\u30e2\u30f3\u30bf\u30ca\u3060\u3002<\/p>\n\n\n\nResults from InstaParsing<\/h2>\n\n\n\n
![\"\"](\"http:\/\/articles.japanesecomplete.com\/wp-content\/uploads\/2020\/01\/screen-shot-2020-01-04-at-18.54.04.png\")
<\/figure>\n\n\n\nRecursion Update 2020-1-8<\/strong><\/h2>\n\n\n\n
![\"\"](\"http:\/\/articles.japanesecomplete.com\/wp-content\/uploads\/2020\/01\/screen-shot-2020-01-08-at-14.31.11.png?w=576\")
<\/figure>\n\n\n\nDeath to Adjectives<\/h2>\n\n\n\n
![\"\"](\"http:\/\/articles.japanesecomplete.com\/wp-content\/uploads\/2020\/01\/etym-altar-5.png\")
<\/figure><\/div>\n\n\n\nParsing Improves Comprehension<\/h2>\n\n\n\n
![\"\"](\"http:\/\/articles.japanesecomplete.com\/wp-content\/uploads\/2020\/01\/buddha-stone.jpg\")
<\/figure>\n\n\n\n![\"\"](\"http:\/\/articles.japanesecomplete.com\/wp-content\/uploads\/2019\/12\/babyfox2.jpg?w=580\")
<\/figure>\n\n\n\n
<\/strong>Standard Morphological Analyzer MeCab <\/a>
Google IME input method super fast lookups with a lossless trie<\/a>
Tokenizer “Sentencepiece”<\/a>
Parsing Convo from 2010 on WaniKani Community<\/a>
Awesome demo of our simplistic parser on Instaparse Live<\/a><\/p>\n\n\n\n![\"\"](\"http:\/\/articles.japanesecomplete.com\/wp-content\/uploads\/2019\/12\/zen.jpg\")
<\/figure>\n","protected":false},"excerpt":{"rendered":"