text command parsers?

[mrflibble]

For instance, whenever you implement a new device driver, you can also add a shell command to talk to it. The shell command is closely associated with the device it's talking to and usually lives in the same source file as the driver. Add that driver to a new project, and you automatically get the shell command with it. Very cool.

The first thing I do in a new project is blink an LED. The next thing is to bring up a command-line shell. It's a great way to get stuff running quickly.

I agree with the turk   Getting an interactive monitor/shell running is a best practice first step.

*grin* Same thing here. First up is the mandatory led blinkie. Cannot have a good project if it didn't start life with a led blinkie.

And yes, a shell is quite helpful. Also, I find it makes the development process more fun. In the sense that coding up some new functionality, and then getting immediate output in response to test commands can be very rewarding. Yes you can stare at variables in the debugger but it's not quite the same as typing a test command and then getting "Thingy::Stuff() here: your answer of the day is 42." on the serial port.

I sometimes have done this with a forth shell. The basic primitives are there in the default interpreter, lots of libraries are reusable, and I can create device drivers interactively in forth to test out stuff immediately.  It's really great for embedded work and a seriously lost art.

The "create device drivers interactively" part sounds intriguing. In broad terms, how would that work?

[SL4P]

(OP: is this bill from the old cisco days?)

I'll see if I can find it... a long time ago I wrote a symmetric 'Cisco-IOS-like' command parser - that would allow you to dump the config via the terminal in ASCII, and reload it directly into the command line (using a terminal app).  Very useful, along with a human readable copy of the config settings.

[andyturk]

The "create device drivers interactively" part sounds intriguing. In broad terms, how would that work?

Don't know about using Forth, but I use a generic "i2c" shell command that talks to anything on the bus:

Code: [Select]

> i2c ?
i2c -- read/write to the I2C bus
syntax: i2c r <addr> <reg> <length>
        i2c w <addr> <reg> <bytes>
        i2c scan

<addr> and <reg> are 2-digit hex numbers
<bytes> is an even-length hex byte string
<length> a decimal number greater than 0
> i2c scan
  found device at 0x60
  found device at 0x68
> i2c r 0x68 0x00 16
read addr=0x68, register=0x00, len=16
c4 cd e0 e9 a0 00 16 2f ee 1a fb fd 00 5f 59 7c
>
When developing a driver for an I2C device, you're mostly just sending and receiving short byte sequences. Being able to figure those sequences out without having a compile-link-flash in between each attempt is pretty handy.

[legacy]

about lex & yacc, who is good at them ?

i am learning them, i downloaded C99 grammar, i arranged lex and yacc sources and … i am having issues with "typedef".

i mean

typedef { blablabla } my_t;

that is fine

but

my_t foo() {}

is not fine cause i am not able to say "hey, my_t is a new type"

so my problem is that i have pre-defined types. I am stopped at this point

let me know if you need the full sources

[westfw]

my_t foo() {}

is not fine

presumably you have to define the my_t portion of your function definition as a generic identifier, and check later whether it's a valid type name, or you have to have the list of typenames that you accept in a function be dynamic somehow.
IIRC, C does not have a "pleasant" grammer when it comes to parsing with regular tools. (this is one of the reasons that C compiler error messages suck so badly.) (Everyone liked pascal, though.)

[miguelvp]

I know this is overkill for what you want, but just throwing it out there.

There is a hardware xml parser out there and I've seen fpga based parsers that can process input very fast. But yeah, that's for something more than just reading a command line, but if you need speed for reading those scripted inputs or if they are networked commands or many other applications, then maybe put in some hardware to do it.

[codeboy2k]

The "create device drivers interactively" part sounds intriguing. In broad terms, how would that work?

Don't know about using Forth, but I use a generic "i2c" shell command that talks to anything on the bus:
[ ... ]
When developing a driver for an I2C device, you're mostly just sending and receiving short byte sequences. Being able to figure those sequences out without having a compile-link-flash in between each attempt is pretty handy.

similar with Forth, except the basic forth shell is a small interpreter written in C that has the basic primitives (called words) for reading and writing memory or reading and writing I/O space.

So if the shell is loaded I can write a new word called 'i2Creadbyte' which reads a single byte from a device on the i2c bus
then I can build up on that word, to write a new forth word that can read multiple bytes, i2creaddata
Similarly I can create new forth words for i2cwrite and i2cwritedata.

Then I can build up on that word further to write a new word that be a device driver , using the previous i2creaddata and i2cwritedata.

If the i2c device is a 2x16 line display, for example, I can then write a new word "printmsg" that uses the new i2cwritedata words.

basically, the forth shell is interpreted, interactive and can create new primitives (words) that can also be used inside more new forth words.
The forth shell is a toolset of primitive words and more complex words built up from those primitives.

From Wikipedia:

A Forth environment combines the compiler with an interactive shell. The user interactively defines and runs subroutines, or "words," in a virtual machine similar to the runtime environment. Words can be tested, redefined, and debugged as the source is entered without recompiling or restarting the whole program. All syntactic elements, including variables and basic operators, appear as such procedures. Even if a particular word is optimized so as not to require a subroutine call, it is also still available as a subroutine. On the other hand, the shell may compile interactively typed commands into machine code before running them. (This behavior is common, but not required.) Forth environments vary in how the resulting program is stored, but ideally running the program has the same effect as manually re-entering the source. This contrasts with the combination of C with Unix shells, wherein compiled functions are a special class of program objects and interactive commands are strictly interpreted. Most of Forth's unique properties result from this principle. By including interaction, scripting, and compilation, Forth was popular on computers with limited resources, such as the BBC Micro and Apple II series, and remains so in applications such as firmware and small microcontrollers.

[andyturk]

i am learning them, i downloaded C99 grammar, i arranged lex and yacc sources and … i am having issues ...

First off, this is probably not a good place to get help/advice on parser generators. Second point, why aren't you using flex and bison? Third, there's more to parsing than just copying someone else's grammar. You're also going to need a symbol table to keep track of things like 'my_t'.

You'll learn a lot more about parsers if you start writing your own simple grammar. Fixing reduce/reduce errors on a big grammar is no fun. In fact, if you're just starting out, try implementing a recursive descent parser because it's easier. You can use lex/flex for the tokenizer, but recursive descent can be done just with carefully written C functions--no need to generate code.

[legacy]

First off, this is probably not a good place to get help/advice on parser generators.

should i open a thread

Second point, why aren't you using flex and bison?

i am using flex, lex, bison, yacc, all the stuff you can get from linux (i can emerge whatever you suggest me, i am on a gentoo machine), it is my first time with these tools

Third, there's more to parsing than just copying someone else's grammar. You're also going to need a symbol table to keep track of things like 'my_t'.

yes, but i have read a few lex & yacc tutorials, which are very easy about grammar, in fact the example is a "calculator", which i am able to write, and i wrote, by my hands.

You'll learn a lot more about parsers if you start writing your own simple grammar. Fixing reduce/reduce errors on a big grammar is no fun.

I have also written a Pascal-similar interpreter, you can find it in the link i have put in the first page of this thread. It is written easily, without any tools.

here it is the link of my thread, it is written in C++

In fact, if you're just starting out, try implementing a recursive descent parser because it's easier. You can use lex/flex for the tokenizer, but recursive descent can be done just with carefully written C functions--no need to generate code.

just asking for a tips&tricks, i repeat, it is my first time with yacc and C, also i need a C grammar implemented in order to realize a sort of sub-MISRA code validator. Too much work about designing it by hands, i mean, for me.

[legacy]

Code: [Select]

# obj/my --debug

[ .k.a.n.o.j.o. ]
[pico.slu.sh.ell]
v4.2m, 2013

compile info:
  branch 4th_issue_r0.2/2013

# help
kanojo: in Japanese to refer to the female 3rd person

help................ - hope it can help
cmdlist............. - list of all available cmds
cmds................ - list of all available cmds
?................... - list of all available cmds
ver................. - show the version
env................. - show all the environment entries
env_show............ - show all the environment entries
env_save............ - save all the environment
env_set............. - edit/create an environment entry
env_cp.............. - cp env into new env
exit................ - force an exit
rpn................. - rpn a linea (experimental, it points to "is_expr_valid", no action perfomed)

# rpn "var1+var2+(var3*var4)"
parsing ("rpn" var1+var2+(var3*var4)""), argv[]={[rpn] [var1+var2+(var3*var4)] }
invoking lexer (from lib_lexer)
sentence=[var1+var2+(var3*var4)]
1: ok
2: 3: tokens_show v[]={  0  1  2  3  4  5  6  7  8 } ntoken=9
 - token[0]: token=[var1] TokenAlphaNum
 - token[1]: token=[+] TokenPlus
 - token[2]: token=[var2] TokenAlphaNum
 - token[3]: token=[+] TokenPlus
 - token[4]: token=[(] TokenOpenBracket
 - token[5]: token=[var3] TokenAlphaNum
 - token[6]: token=[*] TokenAsterisk
 - token[7]: token=[var4] TokenAlphaNum
 - token[8]: token=[)] TokenCloseBracket
invoking rpn_shunting_yard with [var1+var2+(var3*var4)]
input: var1+var2+(var3*var4)
-------- to_lexer begin --------
-------- rpn_core_for_lex begin ---------
-------- rpn_core_for_lex end ---------
output v[]={ 0 2 1 5 7 6 3 }
{
token=[var1] TokenAlphaNum
token=[var2] TokenAlphaNum
token=[+] TokenPlus
token=[var3] TokenAlphaNum
token=[var4] TokenAlphaNum
token=[*] TokenAsterisk
token=[+] TokenPlus
} stack_balance=1
-------- to_lexer end --------
#########################
#    good expression    #
#########################
ans=(1)



I have also written this toy-tini-shell with an RPN inside
it has its lexer library
all manually designed and written by my hands
but … yacc seems more difficult for me ='(


the idea about this project aims to the same target of this thread!