Hello,
I am interested to know if someone has performed benchmarks and/or
some test on reporter component? I heard that the OCL reporter is not
"hard" real-time, and that it can influence the real-time data flow.
So, how much of this is true / not true? What about logging "large"
amounts of data in real-time, cca 500 doubles @ 50-100 ms, or possibly
more?
On the other hand, is there around any reporter / logging component
that does not influence the real-time data flow? Or maybe a logger
that really dumps bytes in a binary file (not ASCII).
Thanx :) BTW, I have not experienced any problems during my work with
OROCOS and reporter component so far, but my current application would
possibly require dumping "larger" amounts of data -- so I asked all
the questions above :)
With kind regards,
Milan
Reporter limitations? Benchmarks? Alternatives?
On Tue, Jun 5, 2012 at 10:58 PM, Milan Vukov
<milan [dot] vukov [..] ...>wrote:
> Hello,
>
> I am interested to know if someone has performed benchmarks and/or
> some test on reporter component? I heard that the OCL reporter is not
> "hard" real-time, and that it can influence the real-time data flow.
> So, how much of this is true / not true? What about logging "large"
> amounts of data in real-time, cca 500 doubles @ 50-100 ms, or possibly
> more?
>
> On the other hand, is there around any reporter / logging component
> that does not influence the real-time data flow? Or maybe a logger
> that really dumps bytes in a binary file (not ASCII).
>
> Thanx :) BTW, I have not experienced any problems during my work with
> OROCOS and reporter component so far, but my current application would
> possibly require dumping "larger" amounts of data -- so I asked all
> the questions above :)
>
The Reporter does not influence the real-time data flow. The thing which is
not real-time about the reporter is that it may drop samples if it can't
keep up. Another issue with the Reporter is that it's a little bit
under-maintained which means that there are issues in there which are
fixable, but no one cared to fix yet.
We've been testing the netcdf reporter lately in combination with kst2 plot
and discovered some minor issues. There's also an issue with the period
which make it sometimes slow to stop if it can't keep up with the data.
This is fixable as well and somewhere lost in the pipe line...
If you want to change the buffer size, you can work around this by first
reporting the component's port, then blowing up the connection
(disconnect() ) and then create a new connection with the required buffer
size. It will then use the new buffer.
The major problem with the reporter today is that it's not a circular
buffer, meaning that it can fill up with old data (like hours old) and then
read this first before reading the new data. Circular buffers must be
introduced in RTT asap such that this can be fixed. There were various
discussions about this on this list in the past. We all agree, such a patch
would get accepted pretty quickly.
>
> With kind regards,
> Milan
>
>
Peter
Reporter limitations? Benchmarks? Alternatives?
On Jun 16, 2012, at 17:00 , Peter Soetens wrote:
> The Reporter does not influence the real-time data flow.
Are you sure about this? While it's been a couple of years, I remember tracing the Reporter implementation and finding some dynamic memory allocation somewhere in the type or type info library. This was v1 though.
S
Reporter limitations? Benchmarks? Alternatives?
On Mon, Jun 18, 2012 at 1:49 PM, S Roderick <kiwi [dot] net [..] ...> wrote:
> On Jun 16, 2012, at 17:00 , Peter Soetens wrote:
>
> The Reporter does not influence the real-time data flow.
>
>
> Are you sure about this? While it's been a couple of years, I remember
> tracing the Reporter implementation and finding some dynamic memory
> allocation somewhere in the type or type info library. This was v1 though.
> S
>
The only allocations occuring are in the updateHook() of the reporter
itself, which is the not realtime-ness I'm referring to.
Peter
Reporter limitations? Benchmarks? Alternatives?
On Jun 18, 2012, at 12:31 , Peter Soetens wrote:
> On Mon, Jun 18, 2012 at 1:49 PM, S Roderick <kiwi [dot] net [..] ...> wrote:
> On Jun 16, 2012, at 17:00 , Peter Soetens wrote:
>
>> The Reporter does not influence the real-time data flow.
>
> Are you sure about this? While it's been a couple of years, I remember tracing the Reporter implementation and finding some dynamic memory allocation somewhere in the type or type info library. This was v1 though.
> S
>
> The only allocations occuring are in the updateHook() of the reporter itself, which is the not realtime-ness I'm referring to.
>
> Peter
Ok, so what did you mean by "realtime" then? The updateHook() appears to both sample and report, and is definitely using non-realtime constructs (e.g. new). So how can we possibly say that this component could be realtime, and how could it possibly keep up with every sample of a system if we wanted it to?
Confused ...
S
Reporter limitations? Benchmarks? Alternatives?
On Sat, Jun 23, 2012 at 1:48 PM, Stephen Roderick <kiwi [dot] net [..] ...> wrote:
> On Jun 18, 2012, at 12:31 , Peter Soetens wrote:
>
> On Mon, Jun 18, 2012 at 1:49 PM, S Roderick <kiwi [dot] net [..] ...> wrote:
>
>> On Jun 16, 2012, at 17:00 , Peter Soetens wrote:
>>
>> The Reporter does not influence the real-time data flow.
>>
>>
>> Are you sure about this? While it's been a couple of years, I remember
>> tracing the Reporter implementation and finding some dynamic memory
>> allocation somewhere in the type or type info library. This was v1 though.
>> S
>>
>
> The only allocations occuring are in the updateHook() of the reporter
> itself, which is the not realtime-ness I'm referring to.
>
> Peter
>
>
> Ok, so what did you mean by "realtime" then? The updateHook() appears to
> both sample and report, and is definitely using non-realtime constructs
> (e.g. new). So how can we possibly say that this component could be
> realtime, and how could it possibly keep up with every sample of a system
> if we wanted it to?
>
The realtime part is about the rest of the application 'still running
real-time, even if instrumented by the reporter'. It's the part of Orocos
where we say 'non realtime threads don't disturb realtime threads due to
lock-free data exchange'. But the non-realtime threads (and hence the
reporter) *do* loose samples...
>
> Confused ...
>
Sorry !
Peter
Reporter limitations? Benchmarks? Alternatives?
On Jun 24, 2012, at 15:58 , Peter Soetens wrote:
> On Sat, Jun 23, 2012 at 1:48 PM, Stephen Roderick <kiwi [dot] net [..] ...> wrote:
> On Jun 18, 2012, at 12:31 , Peter Soetens wrote:
>
>> On Mon, Jun 18, 2012 at 1:49 PM, S Roderick <kiwi [dot] net [..] ...> wrote:
>> On Jun 16, 2012, at 17:00 , Peter Soetens wrote:
>>
>>> The Reporter does not influence the real-time data flow.
>>
>> Are you sure about this? While it's been a couple of years, I remember tracing the Reporter implementation and finding some dynamic memory allocation somewhere in the type or type info library. This was v1 though.
>> S
>>
>> The only allocations occuring are in the updateHook() of the reporter itself, which is the not realtime-ness I'm referring to.
>>
>> Peter
>
> Ok, so what did you mean by "realtime" then? The updateHook() appears to both sample and report, and is definitely using non-realtime constructs (e.g. new). So how can we possibly say that this component could be realtime, and how could it possibly keep up with every sample of a system if we wanted it to?
>
> The realtime part is about the rest of the application 'still running real-time, even if instrumented by the reporter'. It's the part of Orocos where we say 'non realtime threads don't disturb realtime threads due to lock-free data exchange'. But the non-realtime threads (and hence the reporter) *do* loose samples...
Ok, then perhaps we should document this within the reporter? That it won't disturb realtime-ness of anything it instruments BUT that it _will_ loose samples (as the reporter itself isn't real-time).
This is the original reason we ditched the OCL::Reporter and rolled our own - the fact that it couples sampling and storage (which is inherently non-realtime most often).
No longer confused ... :-)
S
Reporter limitations? Benchmarks? Alternatives?
Hello,
On Sat, Jun 16, 2012 at 11:00 PM, Peter Soetens
<peter [..] ...> wrote:
> On Tue, Jun 5, 2012 at 10:58 PM, Milan Vukov <milan [dot] vukov [..] ...>
> wrote:
>>
>> Hello,
>>
>> I am interested to know if someone has performed benchmarks and/or
>> some test on reporter component? I heard that the OCL reporter is not
>> "hard" real-time, and that it can influence the real-time data flow.
>> So, how much of this is true / not true? What about logging "large"
>> amounts of data in real-time, cca 500 doubles @ 50-100 ms, or possibly
>> more?
>>
>> On the other hand, is there around any reporter / logging component
>> that does not influence the real-time data flow? Or maybe a logger
>> that really dumps bytes in a binary file (not ASCII).
>>
>> Thanx :) BTW, I have not experienced any problems during my work with
>> OROCOS and reporter component so far, but my current application would
>> possibly require dumping "larger" amounts of data -- so I asked all
>> the questions above :)
>
>
> The Reporter does not influence the real-time data flow. The thing which is
> not real-time about the reporter is that it may drop samples if it can't
> keep up. Another issue with the Reporter is that it's a little bit
> under-maintained which means that there are issues in there which are
> fixable, but no one cared to fix yet.
So, this is a very valuable piece of information, that reporter is not
influencing RT data flow. I think I can live with some missing samples
:)
>
> We've been testing the netcdf reporter lately in combination with kst2 plot
> and discovered some minor issues. There's also an issue with the period
> which make it sometimes slow to stop if it can't keep up with the data. This
> is fixable as well and somewhere lost in the pipe line...
So, is there any example/guide how to dump data to kst2 plotter? I
would be very interested to use this.
>
> If you want to change the buffer size, you can work around this by first
> reporting the component's port, then blowing up the connection (disconnect()
> ) and then create a new connection with the required buffer size. It will
> then use the new buffer.
Can you give me a snippet which illustrates how to do this, please? I
do not know how to specify a connection policy between reporter port
and a component port. I am using RTT 2.4 and there are only two
arguments for reportPort method.
>
> The major problem with the reporter today is that it's not a circular
> buffer, meaning that it can fill up with old data (like hours old) and then
> read this first before reading the new data. Circular buffers must be
> introduced in RTT asap such that this can be fixed. There were various
> discussions about this on this list in the past. We all agree, such a patch
> would get accepted pretty quickly.
>
Understand. I will take a look at the existing code, possibly. I would
be very glad to contribute.
>>
>>
>> With kind regards,
>> Milan
>>
>
> Peter
>
Best,
Milan
>
>
> --
> Orocos-Users mailing list
> Orocos-Users [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-users
>
Reporter limitations? Benchmarks? Alternatives?
On 06/18/2012 02:01 PM, Milan Vukov wrote:
> Hello,
>
> On Sat, Jun 16, 2012 at 11:00 PM, Peter Soetens
> <peter [..] ...> wrote:
>> On Tue, Jun 5, 2012 at 10:58 PM, Milan Vukov<milan [dot] vukov [..] ...>
>> wrote:
>>>
>>> Hello,
>>>
>>> I am interested to know if someone has performed benchmarks and/or
>>> some test on reporter component? I heard that the OCL reporter is not
>>> "hard" real-time, and that it can influence the real-time data flow.
>>> So, how much of this is true / not true? What about logging "large"
>>> amounts of data in real-time, cca 500 doubles @ 50-100 ms, or possibly
>>> more?
>>>
>>> On the other hand, is there around any reporter / logging component
>>> that does not influence the real-time data flow? Or maybe a logger
>>> that really dumps bytes in a binary file (not ASCII).
>>>
>>> Thanx :) BTW, I have not experienced any problems during my work with
>>> OROCOS and reporter component so far, but my current application would
>>> possibly require dumping "larger" amounts of data -- so I asked all
>>> the questions above :)
>>
>>
>> The Reporter does not influence the real-time data flow. The thing which is
>> not real-time about the reporter is that it may drop samples if it can't
>> keep up. Another issue with the Reporter is that it's a little bit
>> under-maintained which means that there are issues in there which are
>> fixable, but no one cared to fix yet.
>
> So, this is a very valuable piece of information, that reporter is not
> influencing RT data flow. I think I can live with some missing samples
> :)
>
>>
>> We've been testing the netcdf reporter lately in combination with kst2 plot
>> and discovered some minor issues. There's also an issue with the period
>> which make it sometimes slow to stop if it can't keep up with the data. This
>> is fixable as well and somewhere lost in the pipe line...
>
> So, is there any example/guide how to dump data to kst2 plotter? I
> would be very interested to use this.
It should work out of the box...
addPeer("Reporter","Kuka")
...
Reporter.reportPort("Kuka","Pose_tool_base")
...
Reporter.configure
Reporter.start
and then you import your report.dat with the wizard (as kst)
>
>>
>> If you want to change the buffer size, you can work around this by first
>> reporting the component's port, then blowing up the connection (disconnect()
>> ) and then create a new connection with the required buffer size. It will
>> then use the new buffer.
>
> Can you give me a snippet which illustrates how to do this, please? I
> do not know how to specify a connection policy between reporter port
> and a component port. I am using RTT 2.4 and there are only two
> arguments for reportPort method.
>
>>
>> The major problem with the reporter today is that it's not a circular
>> buffer, meaning that it can fill up with old data (like hours old) and then
>> read this first before reading the new data. Circular buffers must be
>> introduced in RTT asap such that this can be fixed. There were various
>> discussions about this on this list in the past. We all agree, such a patch
>> would get accepted pretty quickly.
>>
>
> Understand. I will take a look at the existing code, possibly. I would
> be very glad to contribute.
>
>>>
>>>
>>> With kind regards,
>>> Milan
>>>
>>
>> Peter
>>
>
> Best,
> Milan
>
>>
>>
>> --
>> Orocos-Users mailing list
>> Orocos-Users [..] ...
>> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-users
>>
>
>
>
Reporter limitations? Benchmarks? Alternatives?
On Mon, Jun 18, 2012 at 1:29 PM, Gianni Borghesan <
gianni [dot] borghesan [..] ...> wrote:
>
>
> On 06/18/2012 02:01 PM, Milan Vukov wrote:
>
>> Hello,
>>
>> On Sat, Jun 16, 2012 at 11:00 PM, Peter Soetens
>> <peter [..] ...> wrote:
>>
>>> On Tue, Jun 5, 2012 at 10:58 PM, Milan Vukov<milan.vukov@esat.**
>>> kuleuven.be <milan [dot] vukov [..] ...>>
>>> wrote:
>>>
>>>>
>>>> Hello,
>>>>
>>>> I am interested to know if someone has performed benchmarks and/or
>>>> some test on reporter component? I heard that the OCL reporter is not
>>>> "hard" real-time, and that it can influence the real-time data flow.
>>>> So, how much of this is true / not true? What about logging "large"
>>>> amounts of data in real-time, cca 500 doubles @ 50-100 ms, or possibly
>>>> more?
>>>>
>>>> On the other hand, is there around any reporter / logging component
>>>> that does not influence the real-time data flow? Or maybe a logger
>>>> that really dumps bytes in a binary file (not ASCII).
>>>>
>>>> Thanx :) BTW, I have not experienced any problems during my work with
>>>> OROCOS and reporter component so far, but my current application would
>>>> possibly require dumping "larger" amounts of data -- so I asked all
>>>> the questions above :)
>>>>
>>>
>>>
>>> The Reporter does not influence the real-time data flow. The thing which
>>> is
>>> not real-time about the reporter is that it may drop samples if it can't
>>> keep up. Another issue with the Reporter is that it's a little bit
>>> under-maintained which means that there are issues in there which are
>>> fixable, but no one cared to fix yet.
>>>
>>
>> So, this is a very valuable piece of information, that reporter is not
>> influencing RT data flow. I think I can live with some missing samples
>> :)
>>
>>
>>> We've been testing the netcdf reporter lately in combination with kst2
>>> plot
>>> and discovered some minor issues. There's also an issue with the period
>>> which make it sometimes slow to stop if it can't keep up with the data.
>>> This
>>> is fixable as well and somewhere lost in the pipe line...
>>>
>>
>> So, is there any example/guide how to dump data to kst2 plotter? I
>> would be very interested to use this.
>>
>
> It should work out of the box...
>
> addPeer("Reporter","Kuka")
> ...
> Reporter.reportPort("Kuka","**Pose_tool_base")
>
This creates a fixed size buffer. To increase this buffer size, do now:
Reporter.Kuka_Pose_tool_base.disconnect()
var ConnPolicy cp
cp.type = 1
cp.size = 100
connect(Reporter.Kuka_Pose_tool_base, Kuka.Pose_tool_base, cp)
and then proceed to configure and start:
> ...
>
> Reporter.configure
> Reporter.start
>
> and then you import your report.dat with the wizard (as kst)
The wizard allows you to specify to 'tail' the end of the file, ie to
update if new data arrives.
Peter
Reporter limitations? Benchmarks? Alternatives?
Hello,
On Mon, Jun 18, 2012 at 6:36 PM, Peter Soetens <peter [..] ...> wrote:
> On Mon, Jun 18, 2012 at 1:29 PM, Gianni Borghesan
> <gianni [dot] borghesan [..] ...> wrote:
>>
>>
>>
>> On 06/18/2012 02:01 PM, Milan Vukov wrote:
>>>
>>> Hello,
>>>
>>> On Sat, Jun 16, 2012 at 11:00 PM, Peter Soetens
>>> <peter [..] ...> wrote:
>>>>
>>>> On Tue, Jun 5, 2012 at 10:58 PM, Milan
>>>> Vukov<milan [dot] vukov [..] ...>
>>>> wrote:
>>>>>
>>>>>
>>>>> Hello,
>>>>>
>>>>> I am interested to know if someone has performed benchmarks and/or
>>>>> some test on reporter component? I heard that the OCL reporter is not
>>>>> "hard" real-time, and that it can influence the real-time data flow.
>>>>> So, how much of this is true / not true? What about logging "large"
>>>>> amounts of data in real-time, cca 500 doubles @ 50-100 ms, or possibly
>>>>> more?
>>>>>
>>>>> On the other hand, is there around any reporter / logging component
>>>>> that does not influence the real-time data flow? Or maybe a logger
>>>>> that really dumps bytes in a binary file (not ASCII).
>>>>>
>>>>> Thanx :) BTW, I have not experienced any problems during my work with
>>>>> OROCOS and reporter component so far, but my current application would
>>>>> possibly require dumping "larger" amounts of data -- so I asked all
>>>>> the questions above :)
>>>>
>>>>
>>>>
>>>> The Reporter does not influence the real-time data flow. The thing which
>>>> is
>>>> not real-time about the reporter is that it may drop samples if it can't
>>>> keep up. Another issue with the Reporter is that it's a little bit
>>>> under-maintained which means that there are issues in there which are
>>>> fixable, but no one cared to fix yet.
>>>
>>>
>>> So, this is a very valuable piece of information, that reporter is not
>>> influencing RT data flow. I think I can live with some missing samples
>>> :)
>>>
>>>>
>>>> We've been testing the netcdf reporter lately in combination with kst2
>>>> plot
>>>> and discovered some minor issues. There's also an issue with the period
>>>> which make it sometimes slow to stop if it can't keep up with the data.
>>>> This
>>>> is fixable as well and somewhere lost in the pipe line...
>>>
>>>
>>> So, is there any example/guide how to dump data to kst2 plotter? I
>>> would be very interested to use this.
>>
>>
>> It should work out of the box...
>>
>> addPeer("Reporter","Kuka")
>> ...
>> Reporter.reportPort("Kuka","Pose_tool_base")
>
>
> This creates a fixed size buffer. To increase this buffer size, do now:
>
> Reporter.Kuka_Pose_tool_base.disconnect()
> var ConnPolicy cp
> cp.type = 1
> cp.size = 100
> connect(Reporter.Kuka_Pose_tool_base, Kuka.Pose_tool_base, cp)
>
> and then proceed to configure and start:
>
>>
>> ...
>>
>> Reporter.configure
>> Reporter.start
>>
>> and then you import your report.dat with the wizard (as kst)
>
>
> The wizard allows you to specify to 'tail' the end of the file, ie to update
> if new data arrives.
Thank you very much for help :)
BTW, KST looks really really nice :)
Best,
Milan
>
> Peter
>
>
> --
> Orocos-Users mailing list
> Orocos-Users [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-users
>
Reporter limitations? Benchmarks? Alternatives?
Hello,
On Mon, Jun 18, 2012 at 2:29 PM, Gianni Borghesan
<gianni [dot] borghesan [..] ...> wrote:
>
>
> On 06/18/2012 02:01 PM, Milan Vukov wrote:
>> Hello,
>>
>> On Sat, Jun 16, 2012 at 11:00 PM, Peter Soetens
>> <peter [..] ...> wrote:
>>> On Tue, Jun 5, 2012 at 10:58 PM, Milan Vukov<milan [dot] vukov [..] ...>
>>> wrote:
>>>>
>>>> Hello,
>>>>
>>>> I am interested to know if someone has performed benchmarks and/or
>>>> some test on reporter component? I heard that the OCL reporter is not
>>>> "hard" real-time, and that it can influence the real-time data flow.
>>>> So, how much of this is true / not true? What about logging "large"
>>>> amounts of data in real-time, cca 500 doubles @ 50-100 ms, or possibly
>>>> more?
>>>>
>>>> On the other hand, is there around any reporter / logging component
>>>> that does not influence the real-time data flow? Or maybe a logger
>>>> that really dumps bytes in a binary file (not ASCII).
>>>>
>>>> Thanx :) BTW, I have not experienced any problems during my work with
>>>> OROCOS and reporter component so far, but my current application would
>>>> possibly require dumping "larger" amounts of data -- so I asked all
>>>> the questions above :)
>>>
>>>
>>> The Reporter does not influence the real-time data flow. The thing which is
>>> not real-time about the reporter is that it may drop samples if it can't
>>> keep up. Another issue with the Reporter is that it's a little bit
>>> under-maintained which means that there are issues in there which are
>>> fixable, but no one cared to fix yet.
>>
>> So, this is a very valuable piece of information, that reporter is not
>> influencing RT data flow. I think I can live with some missing samples
>> :)
>>
>>>
>>> We've been testing the netcdf reporter lately in combination with kst2 plot
>>> and discovered some minor issues. There's also an issue with the period
>>> which make it sometimes slow to stop if it can't keep up with the data. This
>>> is fixable as well and somewhere lost in the pipe line...
>>
>> So, is there any example/guide how to dump data to kst2 plotter? I
>> would be very interested to use this.
>
> It should work out of the box...
>
> addPeer("Reporter","Kuka")
> ...
> Reporter.reportPort("Kuka","Pose_tool_base")
> ...
>
> Reporter.configure
> Reporter.start
>
> and then you import your report.dat with the wizard (as kst)
I was not referring to that kind of logging/plotting. I know to do
this, of course. I am interested in online (RT) data streaming from
OROCOS to KST.
Best,
Milan
>
>>
>>>
>>> If you want to change the buffer size, you can work around this by first
>>> reporting the component's port, then blowing up the connection (disconnect()
>>> ) and then create a new connection with the required buffer size. It will
>>> then use the new buffer.
>>
>> Can you give me a snippet which illustrates how to do this, please? I
>> do not know how to specify a connection policy between reporter port
>> and a component port. I am using RTT 2.4 and there are only two
>> arguments for reportPort method.
>>
>>>
>>> The major problem with the reporter today is that it's not a circular
>>> buffer, meaning that it can fill up with old data (like hours old) and then
>>> read this first before reading the new data. Circular buffers must be
>>> introduced in RTT asap such that this can be fixed. There were various
>>> discussions about this on this list in the past. We all agree, such a patch
>>> would get accepted pretty quickly.
>>>
>>
>> Understand. I will take a look at the existing code, possibly. I would
>> be very glad to contribute.
>>
>>>>
>>>>
>>>> With kind regards,
>>>> Milan
>>>>
>>>
>>> Peter
>>>
>>
>> Best,
>> Milan
>>
>>>
>>>
>>> --
>>> Orocos-Users mailing list
>>> Orocos-Users [..] ...
>>> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-users
>>>
>>
>>
>>
>
> --
> ------------------------------------------------
> Gianni Borghesan, PhD
> Robot Assisted Surgery group
> Robotics Research group
> KU Leuven,
> Department of Mechanical Engineering,
> Division of PMA
> Celestijnenlaan 300B, B-3001 Heverlee, Belgium.
> Tel: +32 16 322515.
> Fax: +32 16 322987
> ------------------------------------------------
> --
> Orocos-Users mailing list
> Orocos-Users [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-users
Reporter limitations? Benchmarks? Alternatives?
On Mon, 18 Jun 2012, Milan Vukov wrote:
> Hello,
On Mon, Jun 18, 2012 at 2:29 PM, Gianni Borghesan
<gianni [dot] borghesan [..] ...> wrote:
>
>
> On 06/18/2012 02:01 PM, Milan Vukov wrote:
>> Hello,
>>
>> On Sat, Jun 16, 2012 at 11:00 PM, Peter Soetens
>> <peter [..] ...> wrote:
>>> On Tue, Jun 5, 2012 at 10:58 PM, Milan Vukov<milan [dot] vukov [..] ...>
>>> wrote:
>>>>
>>>> Hello,
>>>>
>>>> I am interested to know if someone has performed benchmarks and/or
>>>> some test on reporter component? I heard that the OCL reporter is not
>>>> "hard" real-time, and that it can influence the real-time data flow.
>>>> So, how much of this is true / not true? What about logging "large"
>>>> amounts of data in real-time, cca 500 doubles @ 50-100 ms, or possibly
>>>> more?
>>>>
>>>> On the other hand, is there around any reporter / logging component
>>>> that does not influence the real-time data flow? Or maybe a logger
>>>> that really dumps bytes in a binary file (not ASCII).
>>>>
>>>> Thanx :) BTW, I have not experienced any problems during my work with
>>>> OROCOS and reporter component so far, but my current application would
>>>> possibly require dumping "larger" amounts of data -- so I asked all
>>>> the questions above :)
>>>
>>>
>>> The Reporter does not influence the real-time data flow. The thing which is
>>> not real-time about the reporter is that it may drop samples if it can't
>>> keep up. Another issue with the Reporter is that it's a little bit
>>> under-maintained which means that there are issues in there which are
>>> fixable, but no one cared to fix yet.
>>
>> So, this is a very valuable piece of information, that reporter is not
>> influencing RT data flow. I think I can live with some missing samples
>> :)
>>
>>>
>>> We've been testing the netcdf reporter lately in combination with kst2 plot
>>> and discovered some minor issues. There's also an issue with the period
>>> which make it sometimes slow to stop if it can't keep up with the data. This
>>> is fixable as well and somewhere lost in the pipe line...
>>
>> So, is there any example/guide how to dump data to kst2 plotter? I
>> would be very interested to use this.
>
> It should work out of the box...
>
> addPeer("Reporter","Kuka")
> ...
> Reporter.reportPort("Kuka","Pose_tool_base")
> ...
>
> Reporter.configure
> Reporter.start
>
> and then you import your report.dat with the wizard (as kst)
> I was not referring to that kind of logging/plotting. I know to do
> this, of course. I am interested in online (RT) data streaming from
> OROCOS to KST.
Kst can work with unix file pointers also to get the new data added to a
file at runtime. Don't forget that "everything is a file" in Unix :-)
<http://en.wikipedia.org/wiki/Everything_is_a_file>
> Best,
> Milan
Herman
Reporter limitations? Benchmarks? Alternatives?
On Jun 5, 2012, at 17:58 , Milan Vukov wrote:
> Hello,
>
> I am interested to know if someone has performed benchmarks and/or
> some test on reporter component? I heard that the OCL reporter is not
> "hard" real-time, and that it can influence the real-time data flow.
> So, how much of this is true / not true? What about logging "large"
> amounts of data in real-time, cca 500 doubles @ 50-100 ms, or possibly
> more?
I'm fairly certain that the existing reporter is _not_ hard realtime.
We use the OCL real-time logging, with custom reporters that produce text strings for logging. It works very well, though it takes a little more boiler-plate code to setup than the existing OCL reporting. Our implementation is hard real-time, and we use it to log ~3kb per cycle at 500 Hz (~1.5 MB/s). With careful tuning of buffer sizes this works very well.
We're also doing this on top of logging 400kb images at 20 Hz ... and this is all with off the shelf computers with HDDs (not SSDs).
> On the other hand, is there around any reporter / logging component
> that does not influence the real-time data flow? Or maybe a logger
> that really dumps bytes in a binary file (not ASCII).
Binary vs ASCII isn't the primary issue. That mostly just affects disk bandwidth. The issue is sampling in real-time and getting those samples out to a non-real-time item that then logs the data.
> Thanx :) BTW, I have not experienced any problems during my work with
> OROCOS and reporter component so far, but my current application would
> possibly require dumping "larger" amounts of data -- so I asked all
> the questions above :)
Have you analysed any diagnostics to do with real-time to prove you aren't having problems? Do you show any major faults during operations?
HTH
S
Reporter limitations? Benchmarks? Alternatives?
Hello,
On Wed, Jun 6, 2012 at 12:15 AM, S Roderick <kiwi [dot] net [..] ...> wrote:
> On Jun 5, 2012, at 17:58 , Milan Vukov wrote:
>
>> Hello,
>>
>> I am interested to know if someone has performed benchmarks and/or
>> some test on reporter component? I heard that the OCL reporter is not
>> "hard" real-time, and that it can influence the real-time data flow.
>> So, how much of this is true / not true? What about logging "large"
>> amounts of data in real-time, cca 500 doubles @ 50-100 ms, or possibly
>> more?
>
> I'm fairly certain that the existing reporter is _not_ hard realtime.
>
> We use the OCL real-time logging, with custom reporters that produce text strings for logging. It works very well, though it takes a little more boiler-plate code to setup than the existing OCL reporting. Our implementation is hard real-time, and we use it to log ~3kb per cycle at 500 Hz (~1.5 MB/s). With careful tuning of buffer sizes this works very well.
>
> We're also doing this on top of logging 400kb images at 20 Hz ... and this is all with off the shelf computers with HDDs (not SSDs).
Before all, thank you very much for the quick reply! :) Can you give
some indications (hints?) how can I setup real-time logging using OCL
real-time logging?
>
>> On the other hand, is there around any reporter / logging component
>> that does not influence the real-time data flow? Or maybe a logger
>> that really dumps bytes in a binary file (not ASCII).
>
> Binary vs ASCII isn't the primary issue. That mostly just affects disk bandwidth. The issue is sampling in real-time and getting those samples out to a non-real-time item that then logs the data.
I agree that the issue is how to extract samples from real-time flow.
After that, "one" can put extracted data into one e.g. circular FIFO,
and later some component (running in a separate, non-real-time thread)
should get data from FIFO and save it to disk -- I assume we have
enough of RAM memory for caching. If one is able to allocate a large
buffer in RAM (>= 100 MB?), I think the disk bandwidth should not be
an issue. Moreover, I really do not care how long it can take for data
to be saved -- 0.1 s or 2s. I just need it saved at some point of
time.
>From my point of view, saving the data in ASCII format is...
introducing unnecessary overhead. I think of it as an eye-candy. :)
Reading this link:
http://www.orocos.org/forum/orocos/orocos-users/realtime-logging
I found about the Netcdf reporter. Are there any instructions how to
use it in OROCOS?
Related to "non-real-time" reporter in OROCOS -- is there a way to
change the number of samples in the buffer? If I remember correctly,
by default it is something like 10(?) samples and I have not seen an
option to change this...
>
>> Thanx :) BTW, I have not experienced any problems during my work with
>> OROCOS and reporter component so far, but my current application would
>> possibly require dumping "larger" amounts of data -- so I asked all
>> the questions above :)
>
> Have you analysed any diagnostics to do with real-time to prove you aren't having problems? Do you show any major faults during operations?
Hm... not exactly. To be honest, I do not know how can I run some
"true" diagnostics. We put some "time-stamping" in our components, so
that we can analyze the results off-line. Do you have any other
suggestion?
>
> HTH
> S
Best regards,
Milan
>
> --
> Orocos-Users mailing list
> Orocos-Users [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-users
Reporter limitations? Benchmarks? Alternatives?
On Jun 6, 2012, at 07:21 , Milan Vukov wrote:
> Hello,
>
> On Wed, Jun 6, 2012 at 12:15 AM, S Roderick <kiwi [dot] net [..] ...> wrote:
>> On Jun 5, 2012, at 17:58 , Milan Vukov wrote:
>>
>>> Hello,
>>>
>>> I am interested to know if someone has performed benchmarks and/or
>>> some test on reporter component? I heard that the OCL reporter is not
>>> "hard" real-time, and that it can influence the real-time data flow.
>>> So, how much of this is true / not true? What about logging "large"
>>> amounts of data in real-time, cca 500 doubles @ 50-100 ms, or possibly
>>> more?
>>
>> I'm fairly certain that the existing reporter is _not_ hard realtime.
>>
>> We use the OCL real-time logging, with custom reporters that produce text strings for logging. It works very well, though it takes a little more boiler-plate code to setup than the existing OCL reporting. Our implementation is hard real-time, and we use it to log ~3kb per cycle at 500 Hz (~1.5 MB/s). With careful tuning of buffer sizes this works very well.
>>
>> We're also doing this on top of logging 400kb images at 20 Hz ... and this is all with off the shelf computers with HDDs (not SSDs).
>
> Before all, thank you very much for the quick reply! :) Can you give
> some indications (hints?) how can I setup real-time logging using OCL
> real-time logging?
http://www.orocos.org/wiki/rtt/examples-and-tutorials/using-real-time-lo...
>>>
>>> On the other hand, is there around any reporter / logging component
>>> that does not influence the real-time data flow? Or maybe a logger
>>> that really dumps bytes in a binary file (not ASCII).
>>
>> Binary vs ASCII isn't the primary issue. That mostly just affects disk bandwidth. The issue is sampling in real-time and getting those samples out to a non-real-time item that then logs the data.
>
> I agree that the issue is how to extract samples from real-time flow.
> After that, "one" can put extracted data into one e.g. circular FIFO,
> and later some component (running in a separate, non-real-time thread)
> should get data from FIFO and save it to disk -- I assume we have
> enough of RAM memory for caching. If one is able to allocate a large
> buffer in RAM (>= 100 MB?), I think the disk bandwidth should not be
> an issue. Moreover, I really do not care how long it can take for data
> to be saved -- 0.1 s or 2s. I just need it saved at some point of
> time.
Ah yes, you do care how long it takes to save the data. This becomes a factor in how big your buffers/FIFOs need to be. The longer it takes to flush to disk, the bigger your buffers need to be.
> From my point of view, saving the data in ASCII format is...
> introducing unnecessary overhead. I think of it as an eye-candy. :)
Eye-candy, maybe. But it works well and is completely hard real-time.
> Reading this link:
>
> http://www.orocos.org/forum/orocos/orocos-users/realtime-logging
>
> I found about the Netcdf reporter. Are there any instructions how to
> use it in OROCOS?
Not that I know of.
> Related to "non-real-time" reporter in OROCOS -- is there a way to
> change the number of samples in the buffer? If I remember correctly,
> by default it is something like 10(?) samples and I have not seen an
> option to change this...
The problem with Orocos buffers is that their size is set upon construction. There were bugs in v1 (at least) that prevented changing the size of an already connected buffer (or something similar). We end up compiling in the required size. :-(
>>> Thanx :) BTW, I have not experienced any problems during my work with
>>> OROCOS and reporter component so far, but my current application would
>>> possibly require dumping "larger" amounts of data -- so I asked all
>>> the questions above :)
>>
>> Have you analysed any diagnostics to do with real-time to prove you aren't having problems? Do you show any major faults during operations?
>
> Hm... not exactly. To be honest, I do not know how can I run some
> "true" diagnostics. We put some "time-stamping" in our components, so
> that we can analyze the results off-line. Do you have any other
> suggestion?
Then with all respect, you don't _know_ that you're actually running in real time. Put at least some min/average/max across both the work time of your components, _and_ across the cycle time. We do that for all components, and we always log this data at the end of each session. We also note the number of major and minor faults, in the process and within each thread of interest, and log those. Major faults are bad. Minor faults you can (sometimes) live with.
HTH
S