I don't have the code to show up as it's in a corporate environment, but I'll explain where is my difficulty.
... [see prior post for code]
So, in the first version I built, the synchronous one, the
'inside_block' value is passed back and forth so that 'processLine'
know where does the context of the line.
But how can I do this when using channels for an asynchronous version?
It would be something like:
when 'chan event $chan readable ...' detects a new line arriving,
it should read the result of 'parseLine $chan $inside_block', so that for the next line the new value of inside_block could be passed as an argument to 'parseLine'.
I'd appreciate some help on this.
Thank you,
You have several options.I want to have several instances running at the same time from several channels, so no global variable.
1) make 'inside_block' a global:
But, then, you can only have one instance of this parse going on at one
time.
2) move "processLine" into a namespace, and use a namespace variable:So, would it be a namespace created dynamically?
namespace eval SomeName {
variable inside_block 0
proc processLine {line} {
if {[string match "SOMETEXT *" $line] && $inside_block == 0} {
# block name or header set inside_block 1 set block_name
.....
dict set parsed $block_name (simplified)
}
if {$inside_block && blank line..} {
# it's a block separator set inside_block 0
}
if {$inside_block} {
# lines after the block header and before the next blank
line # do some parsing and fill in parsed dict
}
}
}
And then setup the callback to call SomeName::processLine
With this method you can have more than one parse at a time running,
provided you create differently named namespaces
3) Make processLine an object and use an object variable (code omitted)Never used OO in Tcl, not a fan of OO, but I'm trying to build something around it.
With this method, you can create "processLine" objects as needed to
attach to the callbacks, and they each get their own unique variable namespace automatically. If you do plan on running plural parses at the
same time then this is your better option, as it frees you from needing
to work out how to handle making custom named namespaces to have plural parses going on.
If you only ever plan to parse one thing at a time, either a global or a namespace work. Which is 'better' is somewhat dependent upon what kind
of 'code review' you might have to endure.
See https://www.magicsplat.com/articles/oo.html for a good description
of objects.
4) You can redefine the callback as you go, so you 'could' pass theWhat should I change in this line?
value in via the callback by redefining the callback to contain the
value that should be present for the next call to processLine.
Hi Rich,
Thank you for your answer.
Please, see below.
On Mon, 15 Apr 2024 22:11:35 -0000 (UTC), Rich wrote:
You have several options.I want to have several instances running at the same time from several channels, so no global variable.
1) make 'inside_block' a global:
But, then, you can only have one instance of this parse going on at one
time.
2) move "processLine" into a namespace, and use a namespace variable:So, would it be a namespace created dynamically?
namespace eval SomeName {
variable inside_block 0
proc processLine {line} {
if {[string match "SOMETEXT *" $line] && $inside_block == 0} {
# block name or header set inside_block 1 set block_name
.....
dict set parsed $block_name (simplified)
}
if {$inside_block && blank line..} {
# it's a block separator set inside_block 0
}
if {$inside_block} {
# lines after the block header and before the next blank
line # do some parsing and fill in parsed dict
}
}
}
And then setup the callback to call SomeName::processLine
With this method you can have more than one parse at a time running,
provided you create differently named namespaces
Could you please share how a small example?
3) Make processLine an object and use an object variable (code omitted)
See https://www.magicsplat.com/articles/oo.html for a good description
of objects.
Never used OO in Tcl, not a fan of OO, but I'm trying to build
something around it.
The two main variable inside_block and chan could be set for the
instance to be visible all over the class and then I'd create
instances of the object like so?
oo::class create Parse {
variable inside_block chan
method parseLine {} {...}
method processLine {} {...}
}
For the main part of the script:
foreach id $exec_ids {
set inst${id} [Parse new]
}
4) You can redefine the callback as you go, so you 'could' pass theWhat should I change in this line?
value in via the callback by redefining the callback to contain the
value that should be present for the next call to processLine.
chan event $chan readable [list parseLine $chan $inside_block]
3) Make processLine an object and use an object variable (code
omitted)
See https://www.magicsplat.com/articles/oo.html for a good description
of objects.
Never used OO in Tcl, not a fan of OO, but I'm trying to build
something around it.
Once you start to use it some, you'll be surprised as all the places
where you created a "partial object layer" to avoid global variables,
but didn't realize you were building part of an 'object' system at the
time.
The two main variable inside_block and chan could be set for the
instance to be visible all over the class and then I'd create instances
of the object like so?
oo::class create Parse {
variable inside_block chan method parseLine {} {...} method
processLine {} {...}
}
Yes, that's the general template of what you'd want. Note you can also
have a 'constructor' method that can do extra work to 'create' the
object when you call 'new' or 'create' and you can have a 'destructor'
method that can cleanup (such as by being sure that 'chan' is closed and
the like).
For the main part of the script:
foreach id $exec_ids {
set inst${id} [Parse new]
}
Yes, and each 'new' call gives you an object with its own private "inside_block" and 'chan' variabes, so you can have plural parses going simultaneously with no conflict (at least no conflict at this level of
your project). And if you setup a 'destructor' then you can simply call 'destroy' on the object, and have everything cleaned up ('chan' channel closed, any memory allocated that does not get cleaned up by Tcl's
reference counting freed, etc.).
On Tue, 16 Apr 2024 18:27:34 -0000 (UTC), Rich wrote:
3) Make processLine an object and use an object variable (code
omitted)
See https://www.magicsplat.com/articles/oo.html for a good description >>>> of objects.
Never used OO in Tcl, not a fan of OO, but I'm trying to build
something around it.
oo::class create Parse {
variable inside_block chan method parseLine {} {...} method
processLine {} {...}
}
Yes, that's the general template of what you'd want. Note you can also
have a 'constructor' method that can do extra work to 'create' the
object when you call 'new' or 'create' and you can have a 'destructor'
method that can cleanup (such as by being sure that 'chan' is closed and
the like).
For the main part of the script:
foreach id $exec_ids {
set inst${id} [Parse new]
}
Yes, and each 'new' call gives you an object with its own private
"inside_block" and 'chan' variabes, so you can have plural parses going
simultaneously with no conflict (at least no conflict at this level of
your project). And if you setup a 'destructor' then you can simply call
'destroy' on the object, and have everything cleaned up ('chan' channel
closed, any memory allocated that does not get cleaned up by Tcl's
reference counting freed, etc.).
I've created a minimum example that is not working.
=== object oriented ===
#!/usr/local/bin/tclsh
oo::class create Parse {
variable chan_r
constructor {comm} {
set chan_r [open |[list $comm] r]
chan configure $chan_r -blocking 0 -buffering line
chan event $chan_r readable [my parseLine]
}
It seems that placing `chan event $chan_r readable [my parseLine]` in the constructor method makes that the context for the chan event is lost somehow.
How can this work?
Another question.
What I'm building is supposed to read a log from an Ansible process,
so lines are being sent to stdout for some minutes, one shortly after another. I'd like the script to parse the log of several Ansible
processes running in parallel, thus the approach to 'set
run_${exec_id} [Parse new ...]'. Does the option '-blocking' make a difference in this context?
Hi,
I don't have the code to show up as it's in a corporate environment, but
I'll explain where is my difficulty.
I've already built a script to parse text output from a command.
processLine does what I need, but I'd like to have it assyncrhonouly:
proc processLine {line inside_block} {
if {[string match "SOMETEXT *" $line] && $inside_block == 0} {
# block name or header
set inside_block 1
set block_name .....
dict set parsed $block_name (simplified)
return $inside_block
}
if {$inside_block && blank line..} {
# it's a block separator
set inside_block 0
}
if {$inside_block} {
# lines after the block header and before the next blank line
# do some parsing and fill in parsed dict
}
}
proc parseLine {chan inside_block} {
set status [catch {chan gets $chan line} nchars]
if {$status == 0 && $nchars >= 0} {
set inside_block [processLine $line $inside_block]
return
}
if error or EOF
return
return $inside_block
}
And the main:
set inside_block 0
set chan [open |[list {*}shell_command] r]
chan event $chan readable [list parseLine $chan $inside_block]
vwait...
show dict parsed
So, in the first version I built, the synchronous one, the 'inside_block' value is passed back and forth so that 'processLine' know where does the context of the line.
But how can I do this when using channels for an asynchronous version?
It would be something like:
when 'chan event $chan readable ...' detects a new line arriving,
it should read the result of 'parseLine $chan $inside_block', so that for
the next line the new value of inside_block could be passed as an argument
to 'parseLine'.
I'd appreciate some help on this.
Thank you,
Luís Mendes
=== object oriented ===
#!/usr/local/bin/tclsh
oo::class create Parse {
variable chan_r constructor {comm} {
set chan_r [open |[list $comm] r]
chan configure $chan_r -blocking 0 -buffering line chan event
$chan_r readable [my parseLine]
}
If you do a quick test, you'll see what has gone wrong:
$ rlwrap tclsh % oo::class create Parse {
constructor {} {
puts stderr "in constructor '[my parseLine]'"
}
method parseLine {} {
}
}
::Parse % Parse new in constructor ''
::oo::Obj12 %
The [my] call, in the constructor, returned nothing. Additionally, [my]
is defined only for use in methods, for calling other methods of the
same object. Event callbacks in Tcl register bits of script code to
execute later in time (i.e., not at the time of definition) and almost allways execute the attached code in the global context (i.e., outside
of your object).
You need to instead setup the event script as if you were calling the
object from the outside (which you in fact are doing) by calling the
desired method on the object's name):
$ rlwrap tclsh % oo::class create Parse {
constructor {} {
puts stderr "in constructor '[self object]'"
}
}
::Parse % Parse new in constructor '::oo::Obj12'
::oo::Obj12 %
So your chan event needs to read:
chan event $chan_r readable [list [self object] parseLine]
And things should begin working (or at least the reason for not
receiving a callback will go away).
On 4/15/2024 12:54 PM, Luis Mendes wrote:
Hi,
I don't have the code to show up as it's in a corporate environment,
but I'll explain where is my difficulty.
I've already built a script to parse text output from a command.
processLine does what I need, but I'd like to have it assyncrhonouly:
proc processLine {line inside_block} {
if {[string match "SOMETEXT *" $line] && $inside_block == 0} {
# block name or header set inside_block 1
set block_name .....
dict set parsed $block_name (simplified)
return $inside_block
}
if {$inside_block && blank line..} {
# it's a block separator
set inside_block 0
}
if {$inside_block} {
# lines after the block header and before the next blank line # do
some parsing and fill in parsed dict
}
}
proc parseLine {chan inside_block} {
set status [catch {chan gets $chan line} nchars]
if {$status == 0 && $nchars >= 0} {
set inside_block [processLine $line $inside_block]
return
}
if error or EOF
return
return $inside_block
}
And the main:
set inside_block 0 set chan [open |[list {*}shell_command] r]
chan event $chan readable [list parseLine $chan $inside_block]
vwait...
show dict parsed
So, in the first version I built, the synchronous one, the
'inside_block' value is passed back and forth so that 'processLine'
know where does the context of the line.
But how can I do this when using channels for an asynchronous version?
It would be something like:
when 'chan event $chan readable ...' detects a new line arriving,
it should read the result of 'parseLine $chan $inside_block', so that
for the next line the new value of inside_block could be passed as an
argument to 'parseLine'.
I'd appreciate some help on this.
Thank you,
Luís Mendes
I've only quickly read your post, so I didn't see what you want to do
with the parsing output.
Monitoring of N other processes by creating OO objects each involving
event driven code is more like trying to simulate separate processes or threads all in a single threaded program.
Assuming you need to correlate the output of each "parser" thread in a
single place (otherwise just exec-ing processes would be enough), then
tcl threads is what you really want. And if you're not a fan of OO, then
you might as well do a crash course on threads instead of tclOO :)
The advantage of threads is that each thread can do simple sequential
coding:
open channels etc. store in global variables loop
read-with-block, parse, report
endloop
(I'm assuming you have a modern tcl, say 8.6.6 or later)
Each new thread would also have it's own private set of global variables (sort of like variable in a class).
To collect the parse data from all the threads, (the report step) you
would thread::send from the parser threads back to the main thread a
call to some collection proc, with the collected data as the arguments.
The main thread would just do a vwait for-exit. The collection proc
would be called fifo from each parser thread, and provided you don't do
any updates or vwaits, could also run "sequentially".
How you intend to know when to exit or fire off some additional
monitoring threads is a bookkeeping job, left as an exercise for the
reader.
On Wed, 17 Apr 2024 21:00:28 -0700, et99 wrote:
On 4/15/2024 12:54 PM, Luis Mendes wrote:
Hi,
I don't have the code to show up as it's in a corporate environment,
but I'll explain where is my difficulty.
I've already built a script to parse text output from a command.
processLine does what I need, but I'd like to have it assyncrhonouly:
proc processLine {line inside_block} {
if {[string match "SOMETEXT *" $line] && $inside_block == 0} {
# block name or header set inside_block 1
set block_name .....
dict set parsed $block_name (simplified)
return $inside_block
}
if {$inside_block && blank line..} {
# it's a block separator
set inside_block 0
}
if {$inside_block} {
# lines after the block header and before the next blank line # do
some parsing and fill in parsed dict
}
}
proc parseLine {chan inside_block} {
set status [catch {chan gets $chan line} nchars]
if {$status == 0 && $nchars >= 0} {
set inside_block [processLine $line $inside_block]
return
}
if error or EOF
return
return $inside_block
}
And the main:
set inside_block 0 set chan [open |[list {*}shell_command] r]
chan event $chan readable [list parseLine $chan $inside_block]
vwait...
show dict parsed
So, in the first version I built, the synchronous one, the
'inside_block' value is passed back and forth so that 'processLine'
know where does the context of the line.
But how can I do this when using channels for an asynchronous version?
It would be something like:
when 'chan event $chan readable ...' detects a new line arriving,
it should read the result of 'parseLine $chan $inside_block', so that
for the next line the new value of inside_block could be passed as an
argument to 'parseLine'.
I'd appreciate some help on this.
Thank you,
Luís Mendes
I've only quickly read your post, so I didn't see what you want to do
with the parsing output.
Monitoring of N other processes by creating OO objects each involving
event driven code is more like trying to simulate separate processes or
threads all in a single threaded program.
Assuming you need to correlate the output of each "parser" thread in a
single place (otherwise just exec-ing processes would be enough), then
tcl threads is what you really want. And if you're not a fan of OO, then
you might as well do a crash course on threads instead of tclOO :)
The advantage of threads is that each thread can do simple sequential
coding:
open channels etc. store in global variables loop
read-with-block, parse, report
endloop
(I'm assuming you have a modern tcl, say 8.6.6 or later)
Each new thread would also have it's own private set of global variables
(sort of like variable in a class).
To collect the parse data from all the threads, (the report step) you
would thread::send from the parser threads back to the main thread a
call to some collection proc, with the collected data as the arguments.
The main thread would just do a vwait for-exit. The collection proc
would be called fifo from each parser thread, and provided you don't do
any updates or vwaits, could also run "sequentially".
How you intend to know when to exit or fire off some additional
monitoring threads is a bookkeeping job, left as an exercise for the
reader.
Hi et99,
Yes, that's a good idea and threads I'd like to play with threads.
Some time ago, I tried a very simple example and it was segfaulting
although the code run sequentially was not. And didn't considered threads for this example but could fit very well for solving the problem.
Thank you,
Luís
Hi Luis:
A while back I really didn't know much about the tcl threads package.
But then I bought Ashok's book. I can't recommend it more highly.
Whatever technique you decide on, threads, event driven, channels,
tclOO, it's all in the book with great examples you can copy/paste right
into your program (from the pdf version). As Rich pointed out, you can
read the tclOO chapter for free on line.
-et
Sysop: | DaiTengu |
---|---|
Location: | Appleton, WI |
Users: | 916 |
Nodes: | 10 (1 / 9) |
Uptime: | 51:55:30 |
Calls: | 12,172 |
Calls today: | 2 |
Files: | 186,522 |
Messages: | 2,234,770 |